JP2006301664A - Visual field matching type information presentation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To present various information such as road guide information to a user in a specified area. <P>SOLUTION: A mobile information terminal 11 has an image pickup means 11A and can acquire image information and display the image information by a display means 11B. A server 12 computes position and orientation relations of the mobile information terminal 11 itself during photography by using a known marker 14 included in an image photographed by the image pickup means 11A (image acquisition section 1) of the mobile information terminal 11, and processes specified information previously associated with the photographed marker 14 on the basis of the result, and the mobile information terminal 11 superimposes the specified information (for example, an arrow 15) having been processed on actually photographed video photographed by the image pickup means 11A and displayed on the screen of the display means 11B (display section 6) and displays the information at the same time. Further, a recommended schedule is generated and displayed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention presents high-density information to an image observer by simultaneously displaying information related to an image captured by an imaging unit in a limited specific area while being superimposed on the captured image. The present invention relates to a visual field coincidence type information presentation system capable of performing the same.

  Conventionally, in order to obtain the position and orientation information of the image pickup means, a method of calculating using a plurality of markers has been proposed.

  For example, Patent Document 1 confirms the positions of markers and / or feature points arranged around the reference marker by reading information included in the reference marker in order to obtain the approximate position and orientation of the image acquisition unit. A possible method has been proposed. According to the technique disclosed in Patent Document 1, it is possible to estimate the position and orientation of the image capturing unit using this information by confirming the positions of four or more markers and / or feature points.

を最小にするようにｘ’を求めることで姿勢が求まる。 Further, Non-Patent Document 1 discloses a method for obtaining the position and orientation of the image pickup means from the reference marker position. In FIG. 9, when O _c is the viewpoint, p ₀ is the projected image diagonal intersection, and P ₀ is the rectangular diagonal intersection,

The posture is obtained by obtaining x ′ so as to minimize.

である。 Where the variable vector x ′ is

It is.

As for the position, d is estimated by least squares so as to minimize the distance from O _cpi to the vertex P _i .

  Then, the position can be determined by the average of the depth values obtained from each vertex.

In Non-Patent Document 2, the initial position of the image pickup means is obtained by using a known reference point called Fiducial Mark. At that time, a plurality of feature points are extracted to obtain a projection position in the input image. The feature point is tracked according to the movement of the input device, and the optimum solution for the position of the image pickup means is obtained.
JP 2001-126051 A "Precise real-time estimation of rectangular marker position and orientation with a single eye for VR interface" (3D Image Conference '96 Proceedings pp.67-172 Akira Takahashi, Ikuo Ishii, Hideo Makino, Makoto Nakashige 1996) “Mixed Reality” (ISBN 4-274-90283-8 Ohmsha, Ltd.) Chapter 6: Augmented Reality Tracking in Natural Feature Environments ”U. Nehmann, S. You, Y. Cho, J. Lee and J. Park 1999 p. 101-130

  On the other hand, in a specific area, such as an amusement park, theme park, museum, etc., many attractions and exhibits are usually scattered, and in order to go directly to the target attraction or exhibit, the specific area must be It is a common method for visitors to search for destinations by relying on a map of the booklet distributed by the operating company and a guide map in the main passage.

  In addition, for shopping at a souvenir shop in a specific area, it is usual to go to the store and purchase goods at the store. Recently, information on what stores are handled in advance in a guidebook or the like can be obtained in advance, but the actual purchase is usually in store only.

  By applying the above-described technology to a device carried by the user, the present inventors can find out where the portable device is located and which direction the user carries, that is, which direction the user is facing. Knowledge that it is possible to guide the user to the desired attractions, exhibitions, product display locations, etc. within the specific area Got.

  Moreover, the knowledge that the various information which a user can utilize can be further shown by limiting the apparatus which such a user carries to the use in the above specific areas.

  However, here, how to configure the device carried by the user, what functions are necessary to perform the actual route guidance, and information to be presented to the user in a specific area As to what kind of information is desirable, sufficient consideration must still be given.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a field-of-view information presentation system capable of presenting various types of information such as route guidance information to a user within a specific area.

  In order to achieve the above object, a field-of-view information presentation system according to the invention described in claim 1 is for capturing a plurality of known markers arranged in a specific area targeted by the system and an image. The image pickup means, the marker detection means for detecting the known marker present in the image taken by the image pickup means, the known marker detected by the marker detection means, and the specified known Position / orientation relationship calculating means for calculating the position / orientation relationship between the marker and the image capturing means, and obtaining related information for acquiring predetermined information associated with the known marker specified by the position / orientation relationship calculating means Information processing means for processing the predetermined information obtained by the means and the related information obtaining means on the basis of the position and orientation relationship calculated by the position and orientation relationship calculating means. And display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means, and at least the image pickup means and the display means are provided on the portable information terminal. And other means are provided in the server capable of wireless communication with the portable information terminal, and there are a plurality of attractions and / or events in the specific area. The portable information terminal further includes the plurality of attractions. Designation means for designating a plurality of desired attractions and / or events in the attraction and / or event, and the server or the portable information terminal further includes a plurality of attractions and / or events by the designation means. It has a schedule generation means to generate a recommended schedule that can be used efficiently according to the specification. Portable information terminal has a function of displaying the recommended schedule generated by the schedule generating means on the display means, characterized in that.

  According to a second aspect of the present invention, there is provided a field-of-view information presentation system including a plurality of known markers arranged in a specific area targeted by the system, an image capturing unit for capturing an image, Marker detection means for detecting the known marker present in the image captured by the image imaging means, a known marker detected by the marker detection means, the identified known marker and the image imaging means Position-and-orientation relationship calculating means for calculating a position-and-orientation relationship between the position and orientation relationship, a related-information acquiring means for acquiring predetermined information associated with the known marker specified by the position-and-orientation relationship calculating means, and the related information acquisition Information processing means for processing the predetermined information obtained by the means based on the position and orientation relationship calculated by the position and orientation relationship calculating means, and the image capturing means Display means for superimposing and displaying information processed by the information processing means on the photographed image, at least the image pickup means and display means are provided in the portable information terminal, and other means And a server capable of wireless communication with the portable information terminal, and there are a plurality of attractions and / or events in the specific area, and the server further participates in each attraction and / or event at the present time. Waiting time storage means for storing information about the waiting time is provided, and the portable information terminal communicates with the server to display the information about the waiting time stored in the waiting time storage means of the server. It is comprised so that browsing is possible by the means.

ADVANTAGE OF THE INVENTION According to this invention, the visual field matching type | formula information presentation system which can show a user various information, such as road guidance information, in a specific area can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
(A) of FIG. 1 is a figure which shows the structure of the visual field coincidence type information presentation system which concerns on the 1st Embodiment of this invention.

  This system includes a large number of portable information terminals 11 (only one is shown in the figure as a representative) used in a specific area, and a server 12 that can communicate with these portable information terminals 11.

  Here, the portable information terminal 11 is a small information terminal that can be carried by a user such as a PDA with a camera or a notebook personal computer, and includes at least an image capturing unit 11A for capturing an image, and the image capturing unit 11A. Display means 11 </ b> B for displaying images and various types of information captured in FIG.

  By the way, the above-mentioned “image” includes a moving image and a photographic image of a still image, and is similarly treated in the following notation.

  Further, the server 12 may be disposed within the specific area or may be disposed outside the specific area. In the latter case, although not particularly illustrated, a relay device that relays communication between the portable information terminal 11 and the server 12 is included in the specific area.

  Further, in the specific area, there are scattered facilities 13 as photographing objects by the image capturing means 11A of the portable information terminal 11, and markers 14 are installed in the vicinity thereof. Here, a unique pattern or symbol is written on the information display surface of the marker 14 so that the marker 14 can be distinguished from other markers. The installation position of each marker 14 and the direction in which the marker 14 faces are stored in a database (not shown) of the server 12. The database of the server 12 stores information associated with each marker 14.

  In the case of using the marker 14 having information indicating the position and orientation displayed on the information display surface as a pattern representing it as disclosed in JP-A-2001-118187 or the like, It is not necessary to store these installation positions and directions in the database of the server 12. Further, the facility 13 can be photographed in advance from each direction and registered in the database of the server 12, and it can be determined from which direction the image capturing means 11A of the portable information terminal 11 has photographed by pattern matching or the like. If configured, the marker 14 can be omitted.

  In addition, although not shown, each facility 13 is provided with a device that performs wireless or wired communication with the portable information terminal 11 and / or the server 12. From the server 12, reservation / reservation of the facility, purchase reservation of products sold at the facility, or delivery request to a predetermined place in the specific facility can be performed.

  FIG. 1B is a functional block diagram of the field-of-view matching information presentation system according to the present embodiment. This system includes an image acquisition unit 1, a marker detection (feature point detection) unit 2, a marker recognition (feature) It comprises a point recognition unit 3, a related information acquisition unit 4, a model data creation (rendering) unit 5, and a display unit 6.

  Here, the image acquisition unit 1 corresponds to the image capturing unit 11 </ b> A of the portable information terminal 11.

  The marker detection unit 2 detects the known marker 14 existing in the image acquired by the image acquisition unit 1 (that is, detects a feature point).

  The marker recognizing unit 3 recognizes the known marker 14 detected by the marker detecting unit 2 and identifies which position the marker 14 is directed to which direction. Using such a method, the position and orientation of the image acquisition unit 1 (image capturing means 11A), that is, where the user of the portable information terminal 11 is directed and which direction is calculated. .

  The related information acquisition unit 4 is for acquiring predetermined information associated with the marker 14 specified by the marker recognition unit 3. The predetermined information is, for example, image data of an image such as an arrow for instructing the moving direction.

  The model data creation unit 5 processes the predetermined information acquired by the related information acquisition unit 4 based on the position and orientation calculated by the marker recognition unit 3. For example, an image for instructing the moving direction based on the route from the recognized arrangement position of the marker 14 to a preset destination and the position and orientation, that is, the direction in which the image capturing unit 11A faces. By processing the image generated by the data, a guide image indicating the moving direction to the destination is generated.

  The display unit 6 corresponds to the display unit 11B of the portable information terminal 11, and displays the information generated by the model data creation unit 5 superimposed on the image acquired by the image acquisition unit 1. To do.

  Here, as described above, the image acquisition unit 1 and the display unit 6 need to be provided in the portable information terminal 11, but the marker detection unit 2 as in the following (1) to (8). The marker recognition unit 3, the related information acquisition unit 4, and the model data creation unit 5 may be provided in any of the portable information terminal 11 and the server 12.

  That is, (1) the image acquisition unit 1, the marker detection unit 2, the marker recognition unit 3, the model data creation unit 5, and the display unit 6 are provided in the portable information terminal 11, and the related information acquisition unit 4 is provided in the server 12. There is no problem if the processing capability of the portable information terminal 11 is sufficient. In this case, since only the information indicating what the marker 14 is needs to be transmitted to the server 12, less transfer data is required. Further, a large storage capacity necessary for storing related information is not required. However, since the process of the model data creation unit 5 requires a large processing capacity, there is a possibility that the processing time of other work performed on the portable information terminal 11 may be affected.

  (2) The portable information terminal 11 includes the image acquisition unit 1, the marker detection unit 2, the marker recognition unit 3, and the display unit 6, and the server 12 includes the related information acquisition unit 4 and the model data creation unit 5. In this case, since only the information indicating what the marker 14 is needs to be transmitted to the server 12, less transfer data is required. Further, a large storage capacity necessary for storing related information is not required. Furthermore, since the server 12 performs the processing of the model data creation unit 5 that requires a large processing capacity, there is no problem of the above (1).

  (3) The image acquisition unit 1, the marker detection unit 2, the model data creation unit 5, and the display unit 6 are provided in the portable information terminal 11, and the marker recognition unit 3 and the related information acquisition unit 4 are provided in the server 12. In this method, the marker detection result is transferred to the server and the subsequent processing is covered by the processing capability of the server. In this case as well, a large storage capacity necessary for storing related information is not required. However, as in the case of (1) above, the processing of the model data creation unit 5 requires a large processing capacity, and thus may affect the processing time of other work performed on the portable information terminal 11.

  (4) The image acquisition unit 1, the marker detection unit 2, and the display unit 6 are provided in the portable information terminal 11, and the marker recognition unit 3, the related information acquisition unit 4, and the model data creation unit 5 are provided in the server 12. As in the case of (2) above, the server 12 does not require a large storage capacity necessary for storing related information and requires a large processing capacity. There is no problem of (1).

  (5) The image acquisition unit 1, the model data creation unit 5, and the display unit 6 are provided in the portable information terminal 11, and the marker detection unit 2, the marker recognition unit 3, and the related information acquisition unit 4 are provided in the server 12. The large storage capacity required for storing related information is not required. However, since the input image itself is transferred to the server, an increase in traffic related to the transfer of the image itself becomes a problem. Further, as in the case of (1) above, the processing of the model data creation unit 5 requires a large processing capacity, and thus may affect the processing time of other work performed on the portable information terminal 11.

  (6) The image acquisition unit 1 and the display unit 6 are provided in the portable information terminal 11, and the marker detection unit 2, the marker recognition unit 3, the related information acquisition unit 4, and the model data creation unit 5 are provided in the server 12. In this case, as in the case of (2) above, the server 12 does not require the large storage capacity necessary for storing the related information, and the processing of the model data creation unit 5 that requires a large processing capacity is performed by the server 12. Therefore, there is no problem of the above (1). Therefore, an inexpensive portable information terminal with low processing capability can be used. However, as in the case of (5) above, an increase in traffic related to transfer of the image itself becomes a problem.

  (7) The portable information terminal 11 includes all of the image acquisition unit 1, marker detection unit 2, marker recognition unit 3, related information acquisition unit 4, model data creation unit 5, and display unit 6. This is a case where the portable information terminal 11 is in a stand-alone state. It is necessary to store related information in the portable information terminal 11, and it is considered that there is a limit to the retained data.

  (8) The image acquisition unit 1, the marker detection unit 2, the marker recognition unit 3, the related information acquisition unit 4, and the display unit 6 are provided in the portable information terminal 11, and the model data creation unit 5 is provided in the server 12. Since the server 12 performs the process of the model data creation unit 5 that requires a large processing capacity, there is no problem (1).

  In the case of (7) above, the server 12 is not necessary, but it is possible to provide various types of information that does not depend on the marker recognition result, so it is preferable to leave the server 12.

  Hereinafter, an actual application example of the visual field coincidence type information presentation system according to the present embodiment will be described.

[Theme park]
First, the case where a specific area is a theme park will be described. In addition, although the case of said (6) is demonstrated to an example here, of course, the structure in any case of other (1)-(5), (7), (8) is also possible.

  That is, when a user of a theme park purchases an admission ticket, the ticket sales counter also declares the rental of the portable information terminal 11 prepared by the theme park operator, and enters the entrance fee and rental fee. pay. You can pay with a credit card when making this payment. Then, the purchased admission ticket is received, and the admission ticket enters the theme park as the specific area from the admission gate. The admission ticket is printed with a unique number such as a serial number (preferably further magnetically recorded), and the person in charge at the ticket sales counter operates the counter terminal when issuing the admission ticket. Thus, the ticket number, the rental application information of the portable information terminal, and the credit card number are registered in the server 12.

  After entering the theme park, the user who applied for rental of the portable information terminal 11 proceeds to the rental information window of the portable information terminal provided near the entrance gate, and performs the rental procedure there. That is, the admission ticket is handed over to the person in charge. The person in charge who received the admission ticket transmits the printed (magnetically recorded) ticket unique number to the server 12 by operating the lending window terminal, so that the user can use the portable information terminal. You can check whether it is registered in the loan list. If the confirmation is made, an ID dedicated to the portable information terminal is issued, and this ID is additionally registered in the record of the server 12. This ID is also printed on the admission ticket. Then, the printed admission ticket is handed over to the user, that is, the user of the portable information terminal, together with the portable information terminal 11 that is lent out.

  In addition, when handing over to this user, it is preferable that a simple lecture is given about how to use the portable information terminal 11. Of course, if the user is not sure how to use, the user can immediately know how to use by calling the HELP screen of the portable information terminal 11 and can also stop and listen to the staff in the theme park.

  The ID newly printed on the admission ticket returned here does not need to be sequentially referred to in the theme park, but the user logs in to a web page operated by the theme park after returning home. This is necessary when referring to snapshots and action route history. This ID has a time limit and becomes unavailable after a certain period of time. If the user wants to continue using the ID, an extension procedure is performed from the Web page. In addition, the same ID can be used continuously for a certain period as well as the period that can be extended, and in that case, the ID that can be used continuously is described when purchasing an admission ticket. If you show the admission ticket and purchase the admission ticket for the day, or declare the continuation ID, you can check the credit card user data used to purchase the admission ticket, and continue using it. It can be judged. However, if the credit card to be used is changed from the one used last time or continuously, the ID must be declared. In that case, re-registration of the relationship between the user and the ID of the mobile terminal use and the credit card number is necessary, but the admission ticket sales work is not significantly complicated.

  Now, the user has the rented portable information terminal 11 and proceeds to the theme park. Here, the user can call up a schedule creation function from the menu of the portable information terminal and create a schedule of how to go around the events and attractions of the day.

  That is, as shown in FIG. 2A, first, whether or not the user is a repeater is specified, and if the user is not a repeater, that is, when the theme park is used for the first time (step S11), the user The composition of the visitors including is input (step S12). Then, the portable information terminal 11 displays the recommended schedule prepared by the theme park operator, which is stored in advance or obtained through communication with the server 12, on the display means 1 (step S13). As this recommended schedule, there are schedules for each group such as for families, for lovers, and for children, that is, recommended attractions and event participation plans according to the audience, and the composition of the visitors entered above A combination is selected and provided. Then, the user modifies the schedule according to his / her preference (step S14), the schedule is completed, and is registered in the server 12 corresponding to the ID (step S15). In the above correction, by displaying the attraction and event waiting time at the current time point, it is possible to change, in particular, those that have a long waiting time to join later from those that are available.

  If the user is a repeater (step S11), further, an ID input is accepted, and if there is an ID input with a time limit (step S16), the server 12 uses the ID for the previous schedule. And an action history is acquired and displayed (step S17). Based on this, the user additionally creates a schedule (step S18), and registers the completed schedule in the server 12 in association with the ID (step S15). In this way, it is also possible to support continuous actions such as going around this time from attractions and events that missed the previous visit.

  On the other hand, if it is a repeater (step S11), but no ID is input (step S16), a schedule is arbitrarily created (step S19), and the server 12 is made to correspond to the ID of the portable information terminal. Will be registered. Note that it is preferable that such a repeater not only allows the user to set a schedule independently, but also presents additional information such as limited attractions and events that are held only at that time.

  In addition, it is more preferable that, for example, a restaurant reservation can be made when creating such a schedule. In other words, it is possible to create an all-day schedule for the day, including when to enter a specific restaurant during breaks between attractions and events. Of course, the restaurant reservation status and congestion status can be centrally managed by the server 12 so that it can be presented to the mobile information terminal 11 at any time, and reservations can be made while avoiding vacant seats and busy time zones. Keep it like that. Of course, cancellation is possible at any time, and if it is not likely to be in time for the scheduled time while participating in an attraction or event, the reservation can be canceled on this portable information terminal.

  Users who have completed a series of schedules in this way participate in attractions and events as scheduled. Depending on the schedule, the next venue may be separated. In the case of a repeater, it may be possible to know all the venues where each attraction or event is held, but for visitors who came for the first time, look at the map distributed at the entrance sequentially It will be moved after confirming. However, recent theme parks have a sufficiently large space, so it is necessary to walk a considerable distance when moving from end to end.

  The marker 14 is installed in many places in the theme park as the specific area. As described above, by capturing the marker 14 with the image capturing unit 11A of the portable information terminal 11, the server 12 can grasp where the user of the portable information terminal is currently located. Therefore, the server 12 determines the optimum route from the current position to the next target position based on the created schedule or when the user inputs the next destination. Judgment is performed, and guidance display is performed on the display means 11B of the portable information terminal 11.

  That is, this guidance display is three-dimensionally displayed on the actually captured image as described above. For example, as shown in FIG. 1C, the guidance display 11B of the portable information terminal. In the screen, an instruction such as an arrow 15 appears along the road. A normal map is a two-dimensional display, and it is common to see the entire theme park from the sky and connect the destination as the current location with a line, but the display in this system is used As the person moves, the direction of this instruction also changes. For example, when the head is shaken sideways, the road that is being viewed simultaneously moves within the field of view as much as the head is shaken sideways. Similarly, the display of the present system changes the direction of the instruction that is shaken and always indicates the direction in which the user should move. This system has a so-called navigation function that detects the line of sight of the portable information terminal 11, accurately tracks changes in the attitude of the portable information terminal 11, and displays appropriate information. Of course, as shown in FIG. 1D, it is possible to display in two dimensions as in the case of a conventional map. In FIG. 1D, reference numeral 16 is a walking locus display, 17 is a photographed marker, 18 is a current position, and 19 is a destination.

  Similarly, the marker 14 can be photographed to obtain guide information in the facility. That is, when the marker 14 is photographed and the building guide menu is selected, the three-dimensional superimposed model of the building where the marker 14 is photographed can be displayed from the current location. The model can also be a so-called plain display, and it is possible to graphically obtain information on the inside of the building, for example, information such as this position for reception and here for toilet. Of course, the navigation function mentioned above is also effective in this building. Even after entering the building, if the destination is a specific room, it will continue to give instructions accurately while moving to that location. Will guide you.

  This marker recognition is left as a movement history of the user in the background. That is, this data is stored as history information for that day until it leaves the theme park, is stored on the server 12 in association with the ID for a certain period of time, and even after the user returns home, It can be viewed on the page as a trajectory in the theme park of the day of the user of this ID.

  Further, as described above, since the portable information terminal 11 includes the image capturing unit 11A that is a camera, it is possible to take a snapshot. The photographed snapshot is stored on the server 12 with the latest positional information of the marker 14 and the photographing time attached. Therefore, it is not necessary for the photographer to memorize the detailed shooting location. If you look at the Web page after returning home, this snap data is pasted on the image diagram of the entire facility, where and what was taken, and when was it The information of can be understood at a glance.

  In addition, there are shops (shops) that sell high value-added products that can only be purchased in the theme park. Such merchandise can also be purchased remotely using information presented on the portable information terminal 11 without actually going to a store.

  In this case, as shown in FIG. 2B, on the portable information terminal, a method of searching from a product, a method of searching from a store, and a method of searching from a store limited to each child theme land in a theme park. Can be selected, and purchase methods are prepared according to the level of the user.

  That is, when searching from the product (step S21), a list of all products handled at the store in the theme park (including at least images and price information) is transmitted from the server 12 to the portable information terminal 11 and displayed. (Step S22). Then, the user of the portable information terminal 11 selects a desired product from the list (step S23) and requests the server 12 for a purchase procedure (step S24).

  When searching from a store (step S25), a list of stores in the theme park is transmitted from the server 12 to the portable information terminal 11 and displayed (step S26). Then, the user of the portable information terminal 11 selects a desired store from the list (step S27). Thus, when a desired store is selected, a list of products handled at the selected store (including at least images and price information) is transmitted from the server 12 to the portable information terminal 11 and displayed (step S22). Upon receiving the product selection by the user (step S23), the portable information terminal 11 requests the server 12 to purchase the product (step S24).

  When searching from a store in the child theme land (step S25), a list of child theme lands is transmitted from the server 12 to the portable information terminal 11 and displayed (step S28). Then, the user of the portable information terminal 11 selects a desired child theme land from the list (step S29). In response to the selection of the child theme land, a list of stores in the child theme land is displayed (step S26), and if a desired store is selected (step S27), the products handled in the selected store are further displayed. List (including at least images and price information) is displayed (step S22). Then, upon receiving the product selection by the user (step S23), the portable information terminal 11 requests the server 12 to purchase the product (step S24).

  The purchase procedure may be a purchase reservation for the product, or a delivery request to a predetermined place in the theme land, for example, a reception window near the exit gate. The server 12 instructs the staff in charge at the store handling the product or the terminal at the reception window. In this case, it is more preferable that the payment of the purchased product is a payment with a credit card issued as ID authentication when the portable information terminal 11 is rented. By making a credit card payment in this way, there is no need to line up at the cash register when receiving the product. Also, if you receive your goods in bulk at the reception counter near the exit gate when you return home, you will be freed from the trouble of moving through the theme park with heavy luggage.

[Museum]
Next, a case where the specific area is a museum will be described.

  In the case of a highly public facility such as a museum, the business of renting the portable information terminal 11 is not assumed, but it is necessary to guarantee the identity of the renting user in some form. Here, as in the case of the theme park, it is assumed that the portable information terminal is rented using a credit card. However, it is also possible to write on the loan form, present the identification card, and perform the loan business. Will. Moreover, the form of using only in a certain room or the form of an installation type terminal may be considered instead of being carried around freely. In the case of use in an individual room, it is used in the form of borrowing at the entrance when entering the exhibition room and returning it when leaving.

  Now, the user borrows the portable information terminal 11 at the entrance of the exhibition room and proceeds to the facility. Proceeding to the front of the exhibit in the hall, a marker 14 is attached to the side of the display panel of the exhibit or near the exhibit. When the marker 14 is taken by the portable information terminal 11 or the image pickup means 11A of the terminal installed in front of the exhibit, information associated with the marker 14 appears on the screen of the display means 11B.

  For example, in the case of a dinosaur skeleton sample, a three-dimensional image of a dinosaur with skin attached thereto is displayed there. In this display, the drawing position is accurately changed (processed) in synchronization with the line of sight of the image pickup means 11A of the portable information terminal 11, so that even if the observation is performed around the specimen, Accurately synchronized and drawn. That is, when viewing from the head side of the skeletal specimen while moving toward the tail, the drawing data of the image data changes from the head toward the tail accordingly. As a result, you can get a feeling that you are looking at a specimen of a simple skeleton, and that the skin is stretched around the skeleton, as if a stuffed dinosaur is displayed on an actual scale. .

  Next, an example of an exhibit that exhibits the life of ancient human beings is as follows: In the case of a ruins of a dwelling, the soil that only has holes in the pillars of the dwelling is excavated, In some cases, the excavation remains are displayed as residence remains (see FIG. 3A). Or, there is a case where the restored house is constructed at the place, but by using this system, as shown in FIG. It is possible to see the exhibit as if a residence has been constructed. That is, a three-dimensional model in which the trace of the hole is the basis of the pillar is created as a restored image, and is drawn and displayed on the screen of the display unit 11B of the portable information terminal 11. When the marker 14 is placed on the dwelling site and the marker 14 is observed, the three-dimensional model is drawn. Like the skeleton sample above, it is synchronized with the viewpoint, so the drawing image changes depending on the direction the user is looking, and it feels as if the real thing is in front of you. Here, if the user looks at the actual landscape, the real space that there is only a trace of the residence spreads, but this contrast accurately obtains a sense of scale of the actual residence and village The merit that it becomes possible will be considered.

  By using this system, it is also possible to increase the added value by making a normal explanation of an exhibit, for example, what is described in text or what is explained in video, more sophisticated.

  For example, it is assumed that there is an explanatory panel for the habitat of the stuffed animal before the stuffed animal. Usually there will only be a description of the length that is easy to read along with something like a distribution chart. At the same time, the marker 14 is described on the explanation panel, and the marker 14 is captured in the field of view by the image pickup means 11A of the portable information terminal 11, so that more detailed contents are displayed on the screen of the display means 11B of the portable information terminal 11. Information is described. For example, it is hierarchized, living area, staple food, speed of growth, personality, breeding method, what species it belongs to and what kind of evolution it has passed, or when you actually live The video or the like may be drawn and displayed on the portable information terminal 11.

[Shopping mall]
Next, a case where the specific area is a shopping mall will be described.

  In this case, in the case of lending the portable information terminal 11 like the above theme park. The method of renting the user by presenting a credit card is considered the most appropriate. However, for users, the main purpose is to purchase at a shopping mall, so it is not efficient to incur costs for renting a terminal, and the shopping mall operator bears this cost. It is important to eliminate the financial burden on the user side. A user presenting a credit card is not a main purpose to reduce the burden of charging, but is a use in the sense of identification.

  The information provided to the user at the portable information terminal 11 is, first of all, the map information of the shopping mall, that is, the location information of the store, and the navigation information to the target store, as in the case of the theme park. Furthermore, individual information of each store, for example, information on special sales may be considered.

  In particular. The user stands at the entrance of the shopping mall, takes a picture of the marker 14 installed on the spot with the image pickup means 11A of the portable information terminal 11, and based on the selected branch displayed on the screen of the display means 11B. For example, when map information or guidance to a destination store is selected, the contents are displayed. According to the navigation, when the user moves inside the mall and enters the store, there is a marker 14 at the entrance. When the marker 14 is photographed by the image capturing means 11A, the store's unique guidance is displayed. The guidance is information on the sale information of the store or the display position of the product being handled. Relying on the screen display, the user can grasp the display position of the target product without asking the store clerk, and it is not necessary to obtain information in advance through newspaper insert advertisements, etc. Can be purchased. Furthermore, as shown in FIG. 4, when an individual marker 14 is arranged on the display product 20 and individual information is registered as supplementary information for the marker 14, the marker 14 is stored in the portable information terminal. If the detailed image is obtained by photographing with the 11 image pickup means 11A, it is not necessary to obtain a catalog or the like, and the detailed information 21 of the product can be obtained and displayed. By doing so, it is possible for the user to know the desired product and the contents of the desired product without any prior knowledge in advance and without the explanation of the sales staff, which is very convenient. Rise. Further, not only for users but also for business operators, there is a great merit in terms of management that it is not necessary to hire sales personnel more than necessary.

[Second Embodiment]
Next, with reference to FIG. 5, a visual field coincidence type information presentation system according to a second embodiment of the present invention will be described.

  The field-of-view information presentation system according to the present embodiment includes a specific marker 14, a portable information terminal 11, and an image input unit 1A. The portable information terminal 11 includes a marker extraction unit 2A and an attitude calculation unit 3A. The content possessing means 4A, the information presenting means 5A, and the display means 6A are provided.

  Here, the image input means 1A is a camera or the like that can be attached to and detached from the portable information terminal 11, and can be attached to the portable information terminal 11 to transmit real-time image data to the portable information terminal 11. . Of course, as indicated by a broken line in FIG. 5, the image input means 1A is integrally provided in the portable information terminal 11 like a PDA with camera or a mobile phone with camera, as in the first embodiment. May be.

The user captures a specific marker 14 having a unique pattern or symbol written on the information display surface so that the marker 14 can be distinguished from other markers by the image input means 1A. As a result, this image data is transferred to the marker extraction means 2A provided in the portable information terminal 11, and a specific marker 11 portion in the image data is extracted. The geometric attribute of this specific marker 14 is registered in advance in the posture calculation means 3A, and the posture calculation means 3A determines how the marker 14 obtained as the image data looks and the geometric attribute. By comparing these attributes with each other, the attitude in which the image input means 1A is spatially obtained to obtain the main image data is calculated. As a result, for example, if it is calculated that the image input unit 1A rotates (θ _x , θ _y , θ _z ) with respect to the axis of the marker 14, it is stored in advance in the content possessing unit 4A. If the predetermined information associated with the marker 14 is an arrow, the information presenting means 5A offsets (θ _x , θ _y , θ _z ) into the posture at the time of registration of the arrow. Is displayed as presentation data on the screen of the display means 6A.

The operational image of this embodiment is shown below.
Signs with markers 14 are installed at various places in the sightseeing spot. The user takes an image of the marker 14 with the image input means 1A attached to or incorporated in the portable information terminal 11 that is held by the user. As a result, the name and direction of the next recommended tour site, which is predetermined information associated with the marker 14, is displayed on the screen of the display means 6A of the portable information terminal 11, and the user continues sightseeing according to it. . When photographing the marker 14, the user does not necessarily have to catch the marker 14 from the front. If an arrow is registered in accordance with the spatial coordinates where the marker 14 is arranged, the marker 14 is always The spatial positional relationship of the arrows is displayed on the portable information terminal 11 without being destroyed.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.

  FIG. 6 is a diagram showing the configuration of the field-of-view information presentation system according to the present embodiment. In this embodiment, the portable information terminal 11 includes the marker extraction means 2A and the posture in the second embodiment. Instead of the calculation means 3A, a marker design discrimination means 2B and a position calculation means 3B are provided.

In the present embodiment, image data is transferred from the image input means 1A to the marker design determination means 2B, and the design of the specific marker 14 portion in the image data is determined. That is, the design attribute of the specific marker 14 is registered in advance in the marker design discriminating means 2B, and the marker design discriminating means 2B is now in view of the appearance of the marker 14 obtained as image data. A search is made as to which of the obtained markers corresponds. Then, the position calculation means 3B knows how the acquired marker design has changed with respect to the registered marker design, so that the position of the image input means 1A in terms of space is determined. To calculate whether this data was obtained. As a result, for example, if it is calculated that the image input unit 1A translates (δ _x , δ _y , δ _z ) with respect to the axis of the marker 14, it is registered in advance in the content possessing unit 4A. If the predetermined information associated with the marker 14 is an arrow, the information presenting means 5A has an offset of (δ _x , δ _y , δ _z ) in the posture at the time of registration of the arrow. Is displayed as presentation data on the screen of the display means 6A.

The operation image of the third embodiment is shown below.
A signboard with markers 14 is installed on the observation deck of the sightseeing spot. The user takes an image of the marker 14 with the image input means 1A attached to or incorporated in the portable information terminal 11 that is held by the user. As a result, on the screen of the display means 6A of the portable information terminal 11, the name of the mountain in the distant landscape and the arrow indicating the peak position of the mountain, which are predetermined information associated with the marker 14, Is displayed and you can enjoy the scenery you can see from the observatory. Further, when the marker 14 is photographed, the user does not necessarily have to catch the marker 14 from the front. If an arrow is registered in accordance with the spatial coordinates where the marker 14 is arranged, the marker 14 must be captured. The spatial positional relationship between 14 and the arrow is displayed on the portable information terminal 11 without breaking.

[Fourth Embodiment]
Next, a visual field coincidence type information presentation system according to a fourth embodiment of the present invention will be described.

  FIG. 7 is a diagram showing a configuration of the field-of-view information presentation system according to the present embodiment. In the present embodiment, the portable information terminal 11 includes the marker extracting unit 2A in the second embodiment, Position / orientation calculation means 3C having both functions of the marker design discrimination means 2B in the third embodiment, the attitude calculation means 3A in the second embodiment and the position calculation means 3B in the third embodiment. And.

In other words, in the present embodiment, the image data from the image input unit 1A is transferred to the marker extraction unit 2A provided in the portable information terminal 11, and a specific marker portion in the image data is extracted. . Then, the extracted marker part data is transferred to the marker design discriminating means 2B, and the design of a specific marker part in the marker part data is discriminated. The geometric attribute and the design attribute of the specific marker 14 are registered in advance in the position / orientation calculation unit 3C, and the appearance of the marker obtained as image data by the position / orientation calculation unit 3C; By comparing these attribute data, it is calculated at what position and orientation the image input means 1A has acquired the image data. As a result, for example, it is calculated that the image input unit 1A rotates (θ _x , θ _y , θ _z ) and translates (δ _x , δ _y , δ _z ) with respect to the axis of the marker 14. Assuming that the predetermined information associated with the marker 14 registered in advance in the content possessing means 4A is an arrow, the information presenting means 5A takes the above-mentioned (( θ _x , θ _y , θ _z ) rotation and (δ _x , δ _y , δ _z ) translation offset are provided and displayed as presentation data on the screen of the display means 6A.

The operational image of this embodiment is shown below.
A signboard with markers 14 is placed on the excavation site of the Jomon ruins (park). The user takes an image of the marker 14 with the image input means 1A attached to or incorporated in the portable information terminal 11 that is held by the user. As a result, on the screen of the display means 6A of the portable information terminal 11, a restored CG image of the restored building as predetermined information associated with the marker 14 is placed on the foundation of the ancient building that has been revealed by excavation. Is displayed. Further, when the marker 14 is photographed, the user does not necessarily have to catch the marker 14 from the front. It is possible to view the restored building from various directions. Also in this case, the spatial positional relationship between the marker 14 and the restored building is displayed on the portable information terminal 11 without being destroyed.

[Fifth Embodiment]
Next, a visual field matching information presentation system according to a fifth embodiment of the present invention will be described.

  FIG. 8 is a diagram showing the configuration of the field-of-view information presentation system according to the present embodiment. In the present embodiment, the marker extraction means 2A and the marker in the fourth embodiment are included in the portable information terminal 11. The design discriminating means 2B, the information presenting means 5A, and the display means 6A are arranged, and the position / orientation calculating means 3C and the content possessing means 4A in the fourth embodiment are configured to be communicable with the portable information terminal 11. Are arranged.

  That is, in the fourth embodiment, each functional part of the (image input means 1A), marker extraction means 2A, marker design discrimination means 2B, position / orientation calculation means 3C, information presentation means 5A, and content possession means 4A The mobile information terminal 11 is mounted on the mobile information terminal 11 and operates as a single unit of the mobile information terminal 11. As described in the first embodiment, the mobile information terminal is included in this functional group. Other than the image input means 1A that is mounted on or mounted on the mobile terminal 11, the mobile information terminal 11 is not mounted on the mobile information terminal 11. The mobile information terminal 11 is mounted on the external server 12 to which the communication connection is possible. It is also possible to perform the processing in the server 12, send only the result to the portable information terminal 11, and present the information on the display means 6A.

  In this case, each functional part of the marker extraction means 2A, the marker design discrimination means 2B, the position / orientation calculation means 3C, and the information presentation means 5A operates independently, and which of these functions is the portable information terminal 11 Whether it is within or outside the server 12 does not affect the final result. Therefore, the configuration can be determined in view of the performance of the mobile information terminal 11 to be used, the size of information to be distributed, and the actual operation status. That is, the sort as shown in FIG. 7 is merely an example, and the present invention is not limited to this. Further, the number of servers is not limited to one, and the above means may be distributed to a plurality of servers.

  The operation example of the present embodiment is shown below by expanding the operation example of the fourth embodiment.

  A signboard with markers 14 is placed on the excavation site of the Jomon ruins (park). The user takes an image of the marker 14 with the image input means 1A attached to or incorporated in the portable information terminal 11 that is held by the user. The marker portion is extracted from the image data at this time by the marker extracting means 2A located in the portable information terminal 11, and this result is then passed to the marker design discriminating means 2B also located in the portable information terminal 11. Then, the marker detection data obtained as a result of the calculation is transferred to the external server 12.

  The external server 12 passes the transferred marker detection data to the position / orientation calculation unit 3C, and calculates the coordinate value of the image input unit 1A in the spatial coordinate system in which the marker 14 is arranged. At the same time, the presentation data which is predetermined information associated with the marker 14 separately registered in the content possessing means 4A separately for the marker design is searched, and final presentation information is created. Then, the presentation information is distributed to the information presentation means 5 </ b> A of the portable information terminal 11.

  As a result, on the screen of the display means 6 </ b> A of the portable information terminal 11, a restored CG image of the restored building, which is predetermined information associated with the marker 14, on the foundation of the ancient building that has been revealed by excavation. Is displayed. Further, when the marker 14 is photographed, the user does not necessarily have to catch the marker 14 from the front, and the marker 14 looks as if it is walking around the base, that is, as if looking around the building. It is possible to view the restored building from various directions. At this time, the spatial positional relationship between the marker 14 and the restored building is displayed on the portable information terminal 11 without being destroyed.

  The effect shown in the fifth embodiment is that, in general, a portable information terminal having a low processing capacity does not perform all the processing, but distributes the processing to efficiently operate the system, The real-time property of the system operation can be secured.

  The communication and distribution of the data are not particularly limited as long as the environment can perform data transfer such as a wired LAN, a wireless LAN, a PHS line, a mobile phone line, infrared communication, Bluetooth (registered trademark), and the like.

  The present invention has been described above based on the embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the gist of the present invention. It is.

  For example, the present invention can be applied to mobile phones and car navigation / ubiquitous computing. Furthermore, it goes without saying that the present invention is not limited to the tourism purpose described in the embodiment, and can be applied to the construction and manufacturing industries.

  In recent years, mobile phones with cameras that are widely used recognize the markers (such as specific signs and logos on buildings) set in the city from the camera, and superimpose information related to the camera screen. In the case of the system to be used, as an application example of the present invention, marker recognition processing and position / orientation calculation processing may be performed by setting different processing distribution (terminal side and server side) for each mobile phone model, It can be said that the function processing is appropriately distributed.

(Appendix)
The invention having the following configuration can be extracted from the specific embodiment described above.

(1) a plurality of known markers arranged in a specific area targeted by the system;
Image capturing means for capturing an image;
Marker detection means for detecting the known marker present in the image captured by the image imaging means;
A position and orientation relationship calculating means for specifying a known marker detected by the marker detecting means and calculating a position and orientation relationship between the identified known marker and the image capturing means;
Related information obtaining means for obtaining predetermined information associated with a known marker identified by the position and orientation relation computing means;
Information processing means for processing the predetermined information obtained by the related information obtaining means based on the position and orientation relationship calculated by the position and orientation relationship calculating means;
Display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means;
Comprising
At least the image capturing means and the display means are provided in a portable information terminal,
Other means are provided in the server capable of wireless communication with the portable information terminal.
Field-of-view information presentation system characterized by this.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system described in (1).
(Function and effect)
According to the field-of-view information presentation system described in (1), the portable information terminal has the image capturing unit, can acquire the image information, and can display the image information on the display unit. Is possible. The server uses the known marker in the image obtained (captured) by the image capturing means to calculate the position and orientation relationship of the portable information terminal itself at the time of shooting, and is shot based on the result. The mobile information terminal processes the predetermined information previously associated with the marker, and the mobile information terminal captures the processed image in the live-action image captured by the image capturing unit and displayed on the screen of the display unit. Are superimposed and displayed simultaneously. Accordingly, it is possible to display related information simultaneously on the live-action screen. Furthermore, since operations requiring a large storage capacity and high-performance processing capability are performed by the server, the portable information terminal only needs to have a processing capability sufficient to capture and superimpose images.

(2) The portable information terminal further includes the marker detection unit,
The server includes the position / orientation relationship calculating means, the related information obtaining means, and the information processing means.
The visual field coincidence type information presentation system according to (1), characterized in that:

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system described in (2).
(Function and effect)
According to the field-of-view information presentation system described in (2), work requiring a large storage capacity and high-performance processing is performed by the server. It suffices if the processing capability is sufficient to detect a known marker and perform superposition display. Furthermore, since only the marker detection result needs to be transmitted from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(3) The portable information terminal further includes the information processing means.
The server includes the marker detection unit, the position and orientation relation calculation unit, and the related information acquisition unit.
The visual field coincidence type information presentation system according to (1), characterized in that:

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system described in (3).
(Function and effect)
According to the field-of-view information presentation system described in (3), work requiring a large storage capacity and high-performance processing is performed by a server. It is sufficient if the processing capability is sufficient to perform information processing and superimposed display.

(4) The portable information terminal further includes the marker detection unit and the position and orientation relation calculation unit.
The server includes the related information acquisition unit and the information processing unit.
The visual field coincidence type information presentation system according to (1), characterized in that:

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system described in (4).
(Function and effect)
According to the field-of-view information presentation system described in (4), work requiring a large storage capacity and high-performance processing is performed by the server. It suffices if the processing capability is sufficient to detect a known marker, calculate the position and orientation, and perform superimposed display. Furthermore, since only the marker detection result and the position / orientation calculation result need be transmitted from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(5) The portable information terminal further includes the marker detection unit and the information processing unit.
The server includes the position / orientation relationship calculation means and the related information acquisition means.
The visual field coincidence type information presentation system according to (1), characterized in that:

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system described in (5).
(Function and effect)
According to the field-of-view information presentation system described in (5), work requiring a large storage capacity and high-performance processing is performed by the server. It suffices if the processing capability is sufficient to detect a known marker, process information, and perform superimposition display. Furthermore, since only the marker detection result needs to be transmitted from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(6) The portable information terminal further includes the marker detection unit, the position / orientation relationship calculation unit, and the related information acquisition unit.
The server includes the information processing means.
The visual field coincidence type information presentation system according to (1), characterized in that:

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the visual field matching information presentation system described in (6).
(Function and effect)
According to the field-of-view information presentation system described in (6), work requiring high-performance processing capability is performed by the server. Therefore, the portable information terminal can capture images and use known markers. It is only necessary to have a processing capability sufficient to perform detection and position / posture calculation, acquisition of related information, and superimposition display. Furthermore, since only the marker detection result, the position / orientation calculation result, and the related information need to be transmitted from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(7) The portable information terminal further includes the marker detection unit, the position / orientation relationship calculation unit, and the information processing unit.
The server includes the related information acquisition unit.
The visual field coincidence type information presentation system according to (1), characterized in that:

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the visual field matching information presentation system described in (7).
(Function and effect)
According to the field-of-view type information presentation system described in (7), since work that requires a large storage capacity and high-performance processing is performed by the server, the portable information terminal is capable of capturing images. It suffices if the processing capability is sufficient to detect known markers, calculate the position and orientation, process the information, and perform superimposed display. Furthermore, since only the information indicating what the marker is is sent from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(8) image capturing means for capturing an image;
Marker detection means for detecting the known marker present in the image captured by the image imaging means;
The known marker detected by the marker detection means is identified from among a plurality of known markers arranged in a specific area, and between the identified known marker and the image capturing means. A position and orientation relationship calculating means for calculating the position and orientation relationship of
Related information obtaining means for obtaining predetermined information associated with a known marker identified by the position and orientation relation computing means;
Information processing means for processing the predetermined information obtained by the related information obtaining means based on the position and orientation relationship calculated by the position and orientation relationship calculating means;
Display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the portable information terminal for the field-of-view information presentation system described in (8).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (8), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, using a known marker reflected in the image obtained (captured) by the image capturing means, the position / orientation relationship of the portable information terminal itself at the time of capturing is calculated, and capturing is performed based on the result. The predetermined information previously associated with the marked marker is processed, and the processed predetermined information is superimposed on the live-action image captured by the image capturing unit and displayed on the screen of the display unit. Display at the same time. Accordingly, it is possible to display related information simultaneously on the live-action screen.

(9) Image capturing means for capturing an image, and by wirelessly transmitting the captured image to the server, a plurality of known markers existing in the transmitted image are transmitted to the server in a specific area. It identifies which of the arranged known markers, calculates the position and orientation relationship between the identified known marker and the image capturing means, and is associated with the identified known marker Image capturing means that obtains predetermined information and functions to process it based on the calculated position and orientation relationship;
Display means for wirelessly receiving the processed predetermined information from the server and displaying the information superimposed on an image captured by the image capturing means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the portable information terminal for the field-of-view information presentation system described in (9).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (9), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, by transmitting the image obtained (captured) by the image capturing means to the server, the server uses the known marker reflected in the image to determine the position and orientation relationship of the portable information terminal itself at the time of capturing. The predetermined information associated with the marker photographed based on the result of the calculation is processed, and the portable information terminal receives the processed predetermined information from the server, and receives it from the image. The image pickup unit superimposes and displays the image on the real image displayed on the screen of the display unit. Accordingly, it is possible to display related information simultaneously on the live-action screen. Furthermore, since operations requiring a large storage capacity and high-performance processing capability are performed by the server, the portable information terminal only needs to have a processing capability sufficient to capture and superimpose images.

(10) image capturing means for capturing an image;
By detecting the presence of any of a plurality of known markers arranged in a specific area from the image captured by the image capturing means, and wirelessly transmitting an image of a portion corresponding to the known marker to the server, the above The server identifies a known marker in the transmitted image, calculates the position and orientation relationship between the identified known marker and the image capturing means, and associates the known marker with the identified known marker. Marker detection means that obtains predetermined information and functions to process it based on the calculated position and orientation relationship;
Display means for wirelessly receiving the processed predetermined information from the server and displaying the information superimposed on an image captured by the image capturing means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the portable information terminal for the field-of-view information presentation system described in (10).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (10), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, the presence of any of a plurality of known markers arranged in a specific area is detected from the image obtained (captured) by the image capturing means, and an image of a portion corresponding to the known marker is transmitted to the server. In the server, the known marker in the image of the part is specified, the position and orientation relationship of the portable information terminal itself at the time of shooting is calculated using the specified known marker, and the result is obtained. The predetermined information previously associated with the marker photographed based on the processed information is processed, and the portable information terminal receives the processed predetermined information from the server, and the image imaging means captures and displays it. It is superimposed on the live-action image displayed on the screen of the means and displayed simultaneously. Accordingly, it is possible to display related information simultaneously on the live-action screen. In addition, since work that requires large storage capacity and high-performance processing capability is performed by the server, the portable information terminal has sufficient processing capability to capture images, detect known markers, and perform overlay display. If there is. Furthermore, since it is only necessary to transmit not the entire captured image but only the portion corresponding to the detected marker from the portable information terminal to the server, communication traffic can be reduced.

(11) Image capturing means for capturing an image, and by wirelessly transmitting the captured image to a server, a plurality of known markers existing in the transmitted image are transmitted to the server in a specific area. It identifies which of the arranged known markers, calculates the position and orientation relationship between the identified known marker and the image capturing means, and is associated with the identified known marker Image capturing means for functioning to obtain predetermined information;
Information processing means for wirelessly receiving the position and orientation relationship and the predetermined information from the server, and processing the predetermined information based on the position and orientation relationship;
Display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the portable information terminal for the field-of-view information presentation system described in (11).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (11), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, by transmitting the image obtained (captured) by the image capturing means to the server, the server uses the known marker reflected in the image to determine the position and orientation relationship of the portable information terminal itself at the time of capturing. The portable information terminal receives the predetermined information and the position / orientation relationship from the server, and obtains predetermined information associated in advance with the marker photographed based on the result. Based on the relationship, the predetermined information is processed, and the image capturing means captures and superimposes it on the live-action image displayed on the screen of the display means and displays it simultaneously. Accordingly, it is possible to display related information simultaneously on the live-action screen. In addition, work that requires a large storage capacity and high-performance processing capability is performed by the server, so the portable information terminal has enough processing capability to capture images, process information, and perform overlay display. If there is.

(12) an image capturing means for capturing an image;
Marker detection means for detecting the presence of any of a plurality of known markers arranged in a specific area from the image captured by the image capturing means;
The known marker detected by the marker detection means is specified, the position and orientation relationship between the specified known marker and the image capturing means is calculated, and the specified known marker and the calculated position are calculated. By transmitting the posture relationship to the server wirelessly, the server obtains predetermined information associated with the transmitted known marker and processes it based on the transmitted position and posture relationship. A position / orientation relationship calculation means for functioning to process;
Display means for wirelessly receiving the processed predetermined information from the server and displaying the information superimposed on an image captured by the image capturing means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the portable information terminal for the field-of-view information presentation system described in (12).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (12), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, the presence of any of the plurality of known markers arranged in a specific area is detected and specified from within the image obtained (captured) by the image pickup means, and the specified known marker and the above image pickup are detected. By calculating the position / orientation relationship with the means and transmitting the specified known marker and the calculated position / orientation relationship to the server, in the server, the predetermined information previously associated with the known marker Is processed based on the position and orientation relationship, and the portable information terminal receives the processed predetermined information from the server, captures it by the image capturing means, and displays it on the screen of the display means. It is superimposed on the live-action video and displayed simultaneously. Accordingly, it is possible to display related information simultaneously on the live-action screen. In addition, since work that requires large storage capacity and high-performance processing capability is performed by the server, the portable information terminal performs image capture, detection of a known marker, calculation of a position and orientation relationship, and superimposition. It is sufficient that the processing capacity is sufficient for display. Furthermore, since only the marker detection result and the position / orientation calculation result need be transmitted from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(13) Image capturing means for capturing an image;
By detecting the presence of any of a plurality of known markers arranged in a specific area from the image captured by the image capturing means, and wirelessly transmitting an image of a portion corresponding to the known marker to the server, the above The server identifies a known marker in the transmitted image, calculates the position and orientation relationship between the identified known marker and the image capturing means, and associates the known marker with the identified known marker. Marker detecting means for functioning to obtain predetermined information, and
Information processing means for wirelessly receiving the position and orientation relationship and the predetermined information from the server, and processing the predetermined information based on the position and orientation relationship;
Display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the portable information terminal for the field-of-view information presentation system described in (13).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (13), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, the presence of any of a plurality of known markers arranged in a specific area is detected from the image obtained (captured) by the image capturing means, and an image of a portion corresponding to the known marker is transmitted to the server. Thus, in the server, the known marker is specified, the position and orientation relationship between the specified known marker and the image capturing means is calculated, and the predetermined marker associated with the specified known marker is calculated. The portable information terminal receives the positional and orientation relationship and the predetermined information from the server, processes the predetermined information based on the positional and orientation relationship, and the image capturing means The image is superimposed and displayed on the live-action image displayed on the screen of the display means. Accordingly, it is possible to display related information simultaneously on the live-action screen. In addition, since work that requires large storage capacity and high-performance processing is performed by the server, the portable information terminal performs image capturing, detection of a known marker, information processing, and superposition. It is sufficient that the processing capacity is sufficient for display. Furthermore, since only the marker detection result needs to be transmitted from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(14) image capturing means for capturing an image;
Marker detection means for detecting the presence of any of a plurality of known markers arranged in a specific area from the image captured by the image capturing means;
A position and orientation relationship calculating means for specifying a known marker detected by the marker detecting means and calculating a position and orientation relationship between the identified known marker and the image capturing means;
Predetermined information associated with a known marker specified by the position / orientation relationship calculating means is acquired, and the acquired predetermined information is wirelessly transmitted to the server together with the position / orientation relationship calculated by the position / orientation relationship calculating means. A related information obtaining means for causing the server to function to process the predetermined information that has been transmitted based on the position and orientation relationship that has been transmitted;
Display means for wirelessly receiving the processed predetermined information from the server and displaying the information superimposed on an image captured by the image capturing means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the portable information terminal for the field-of-view information presentation system described in (14).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (14), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, the presence of any of the plurality of known markers arranged in a specific area is detected and specified from within the image obtained (captured) by the image pickup means, and the specified known marker and the above image pickup are detected. Calculating the position and orientation relationship with the means, obtaining predetermined information associated with the identified known marker, and transmitting the obtained predetermined information together with the position and orientation relationship to the server. In the server, the predetermined information is processed based on the position and orientation relationship, and the portable information terminal receives the processed predetermined information from the server, and the image capturing means captures it. The images are superimposed and displayed on the live-action image displayed on the screen of the display means. Accordingly, it is possible to display related information simultaneously on the live-action screen. In addition, since work that requires high-performance processing capability is performed by the server, the portable information terminal performs image capturing, detection of known markers and calculation of position and orientation relations, and acquisition of related information. It suffices if the processing capability is sufficient to perform superimposed display. Furthermore, since only the marker detection result, the position / orientation calculation result, and the related information need to be transmitted from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(15) Image capturing means for capturing an image;
Marker detection means for detecting the presence of any of a plurality of known markers arranged in a specific area from the image captured by the image capturing means;
The known marker detected by the marker detection unit is specified, the position and orientation relationship between the specified known marker and the image capturing unit is calculated, and the identification result of the known marker is wirelessly transmitted to the server. A position / orientation relationship calculation means for causing the server to function to obtain predetermined information associated with the identified known marker that has been transmitted;
Information processing processing means for wirelessly receiving the predetermined information from the server and processing it based on the position and orientation relationship calculated by the position and orientation relationship calculating means;
Display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means;
A portable information terminal for a field-of-view information presentation system.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the portable information terminal for the field-of-view information presentation system described in (15).
(Function and effect)
According to the portable information terminal for the field-of-view information presentation system described in (15), it has image capturing means, can acquire image information, and displays the image information on the display means. Is possible. Further, the presence of any of the plurality of known markers arranged in a specific area is detected and specified from within the image obtained (captured) by the image pickup means, and the specified known marker and the above image pickup are detected. While calculating the position and orientation relationship with the means, by transmitting the specified result of the known marker to the server, the server obtains predetermined information associated with the identified known marker, The portable information terminal receives the predetermined information from the server, processes it based on the position / orientation relationship, captures the actual image captured by the image capturing unit and displayed on the screen of the display unit. Simultaneously display in the video. Accordingly, it is possible to display related information simultaneously on the live-action screen. In addition, since work that requires large storage capacity and high-performance processing is performed by the server, the portable information terminal captures an image, detects a known marker, calculates a position and orientation relationship, It only needs to have a processing capability sufficient to perform information processing and superimposed display. Furthermore, since only the information indicating what the marker is is sent from the portable information terminal to the server, not the captured image itself, communication traffic can be reduced.

(16) The known marker represents three-dimensional information of each installation location in the specific area,
The identification of the known marker and the calculation of the position and orientation relationship are performed based on the three-dimensional information.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (8) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-matching information presentation system or portable information terminal described in (16).
(Function and effect)
According to the visual field coincidence type information presentation system described in (16), the marker can be easily identified by recording the marker installation location and the direction in which the marker is facing, It is possible to detect the position and the direction of the user of the image capturing means, that is, the portable information terminal.

  (17) The server or the portable information terminal further stores an order of a plurality of known markers specified within a predetermined period, and the stored order of the markers and an arrangement position of the markers in the specific area. (8) The visual field coincidence type information presentation system according to any one of (1) to (7), further comprising behavior route calculation means for calculating and storing a user's behavior route from the relationship Thru | or the portable information terminal in any one of (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-matching information presentation system or portable information terminal described in (17).
(Function and effect)
According to the visual field coincidence type information presentation system or the portable information terminal described in (17), the shooting order of known markers is stored, and the user's action route is determined from the arrangement positional relationship of the markers in a specific area. Is calculated and stored, so it is possible to calculate what route the user has acted on. In addition, if the incidental information such as the photographing time is stored at the same time when the marker is stored, the temporal transition of the user's behavior can be grasped.

  (18) The field-of-view information presentation system according to any one of (1) to (7) or (9) to (9), wherein the server is configured to be compatible with a plurality of the portable information terminals. The portable information terminal according to any one of (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system or the portable information terminal described in (18).
(Function and effect)
According to the visual field matching information presentation system or the portable information terminal described in (18), it is possible to deal with a plurality of portable information terminals, that is, a plurality of users.

  (19) The server further stores the order of the plurality of known markers identified for each portable information terminal, and the order of the stored markers and the positional relationship of the markers in the specific area. And calculating and storing a behavior route of the user of the portable terminal, and further comprising a density confirmation means for confirming a density of each section in the specific area at each time of all users of the plurality of portable terminals. The visual field coincidence type information presentation system or portable information terminal according to (18), wherein

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-of-view information presentation system or portable information terminal described in (19).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (19), the order in which the known markers are photographed is stored for each portable information terminal, and the arrangement positional relationship of this marker in a specific area From this, the behavior route of each user is calculated and stored, so that it is possible to calculate what route each user acted on. Further, if the incidental information such as the photographing time is stored at the same time when the marker is stored, the temporal transition of each user's action can be grasped. Furthermore, it is possible to grasp the overcrowding or depopulation state of a specific facility or section in the specific area at a certain time or at the current time.

(20) The server further promptly guides the user to a specific facility or section in the specific area based on the confirmed density at the current time point or as desired by the administrator of the server. Is transmitted to a user's or any portable information terminal located in a dense section,
The field-of-view information presentation system or the portable information terminal according to (19), wherein when the portable information terminal receives the information prompting the guidance, the portable information terminal displays the information on the display means.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-matching information presentation system or portable information terminal described in (20).
(Function and effect)
According to the field-of-view information presentation system or portable information terminal described in (20), when the server manager is heavily congested based on the grasped or depopulated state of the specific facility or section at the current time point The user can be guided to a relatively vacant specific facility or section, or can be guided to hold a specific event to arbitrarily control the flow of users in the specific area.

(21) The portable information terminal further includes destination setting means for a user of the portable information terminal to set a desired destination,
The predetermined information associated with the known marker is image data for instructing the moving direction,
The processing for processing the predetermined information includes a path from the arrangement position of the marker to the destination set by the destination setting unit, and a direction of the image capturing unit determined by the calculated position and orientation relationship. A guide image indicating the moving direction to the destination is generated by processing the image generated by the image data for instructing the moving direction based on the moving direction.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (8) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-matching information presentation system or portable information terminal described in (21).
(Function and effect)
According to the visual field coincidence type information presentation system or the portable information terminal described in (21), the position information of the photographed marker is the path from the currently photographed marker, that is, the position where the current user is located to the next destination to be visited. And the route is displayed based on the direction in which the user points the image capturing means. Therefore, it is no longer necessary to expand the map as in the case of using a printed map in the past, such as “I ’m here now and my destination is here, so go here…” Since the optimum route is displayed simply by pointing the portable information terminal appropriately, the action efficiency within the specific area is dramatically improved.

  (22) The server or the portable information terminal has at least a shooting date and time of an image taken by the image pickup unit and a shooting location specified by an arrangement position of the specified known marker in the specific area. The visual field matching type information presentation system according to any one of (1) to (7), or the visual information matching system according to any one of (8) to (15), further comprising an image storage unit that stores the image information together with the captured image information. Mobile information terminals.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system or the portable information terminal described in (22).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (22), an image snap-photographed using the image capturing unit recognizes the marker before and after the image, and the position information of the marker and other information. By storing the information registered in the marker together with the date and time of shooting and the original data of the shooting, for example, where the snapshot was taken at what time, and a unique event was held. It is not necessary for the user of the portable information terminal to take a note of whether or not it is during the period during which it is in progress.

(23) The server further includes:
An image storage unit that stores an image captured by the image capturing unit together with shooting information including at least the shooting date and time and the shooting location specified by the arrangement position of the specified known marker in the specific area;
Means for causing a photographer to download the image and shooting information stored in the image storage means via the Internet;
The field-of-view information presentation system according to any one of (1) to (7), comprising:

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the visual field matching information presentation system described in (23).
(Function and effect)
According to the field-of-view information presentation system described in (23), the snap image taken by the image capturing means of the portable information terminal is stored on the server, and the user of the portable information terminal returns from the specific facility. Later, since this snap image can be obtained via the Internet, the user does not need to prepare a personal camera in the specific area.

(24) The portable information terminal further includes schedule setting means for a user of the portable information terminal to set a schedule for the day,
The server or the portable information terminal further includes schedule storage means for storing the schedule set by the schedule setting means,
The display means has a function of displaying a schedule stored in the schedule storage means in response to an operation of a user of the portable information terminal.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (8) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-of-view information presentation system or portable information terminal described in (24).
(Function and effect)
According to the field-of-view information presentation system or portable information terminal described in (24), the user inputs the schedule of the day, stores it on the portable information terminal or server, and displays it as necessary. It is possible to enjoy events and attractions in the specific area efficiently. In addition, by managing a plurality of portable information terminals, that is, users' schedules on the server, it becomes possible to predict congestion of events and attractions in the specific area, and based on the prediction, the server administrator side Arbitrary flow of users in the specific area, such as by sending a guide to events and attractions that are relatively free at that time to users who have incorporated events and attractions of that time into the schedule Control.

  (25) The server or the portable information terminal may be further identified by the schedule stored in the schedule storage unit, the current time, and the position of the identified known marker in the specific area. (24) characterized by comprising schedule management means for collating with a place where the user of the information terminal is present and displaying on the display means information based on an error from the schedule obtained by the collation. The field-of-view information presentation system or portable information terminal described.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-matching information presentation system or portable information terminal described in (25).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (25), by checking the schedule against time and place, it is possible to determine whether or not it is traveling within the specific area according to the created schedule. Can also be confirmed.

(26) The server or the portable information terminal is configured to be capable of wireless or wired communication with a facility existing in the specific area,
The server or the portable information terminal further includes installation location information of the marker in the specific area, usage status and reservation status obtained by communication with the facility, and the user of the portable information terminal to the facility. Comprising management means for storing and managing specific area information including at least one of the reservation status and the user's action history in the specific area;
The portable information terminal is configured so that the specific area information stored and managed by the management means can be appropriately viewed on the display means.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (8) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment relating to the field-of-view information presentation system or the portable information terminal described in (26).
(Function and effect)
According to the visual field coincidence type information presentation system or the portable information terminal described in (26), the specific area information can be browsed on the display unit as necessary. In addition, if the server is provided with a management means for storing and managing the specific area information without performing centralized management of the mobile information terminal alone, it is possible to effectively utilize the limited storage of the mobile information terminal. In addition, it becomes possible to present information based on comprehensive relations including information obtained from other than portable information terminals.

(27) There are a plurality of attractions and / or events in the specific area,
The server further comprises a waiting time storing means for storing information about the current waiting time for each attraction and / or event.
The portable information terminal is configured to communicate with the server so that information about the waiting time stored in the waiting time storage means of the server can be viewed on the display means.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (9) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment related to the field-of-view information presentation system or the portable information terminal described in (27).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (27), the user of the portable information terminal can be provided with real-time attraction or event participation waiting time information. , You can participate in attractions and events efficiently according to the real-time waiting time information. Moreover, since the situation can be grasped in a remote state without actually going to the attraction or event, the mental pain can be reduced.

(28) There are a plurality of attractions and / or events in the specific area,
The portable information terminal further comprises designation means for designating a plurality of desired attractions and / or events among the plurality of attractions and / or events,
The server or the portable information terminal further includes a schedule generation unit that generates a recommended schedule that can be efficiently turned around in accordance with the designation of a plurality of attractions and / or events by the designation unit,
The portable information terminal has a function of displaying a recommended schedule generated by the schedule generation unit on the display unit.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (8) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-of-view information presentation system or portable information terminal described in (28).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (28), it is possible to display on the portable information terminal a recommended order in which the owner of the portable information terminal can efficiently travel the desired event / attraction. Therefore, the user can not only effectively use time but also easily consider an action plan within the specific facility.

(29) In the specific area, there is a reservable facility having a wireless or wired communication function with the server,
The server further stores information indicating the usage status of the facility acquired by communication with the facility, and transmits the information indicating the usage status to the mobile information terminal in response to a request from the mobile information terminal. And facility management means for making reservations and canceling reservations for the above facilities,
The portable information terminal further requests the facility management means of the server to transmit information indicating the usage status of the facility according to a user's operation, and receives the usage status information received in response to the request. And a reservation means for transmitting a request for reservation or cancellation of the facility to the facility management means of the server according to a user operation.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (9) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-matching information presentation system or portable information terminal described in (29).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (29), the user can use the portable information terminal via the server without actually going to the facility in the specific area. The real-time usage status of the facility can be confirmed, and further, reservation of the facility and cancellation of the reservation can be performed.

(30) There are facilities that can be reserved in the specific area,
The portable information terminal further includes:
A communication means for performing wireless or wired communication with the reservable facility;
In accordance with the operation of the user of the portable information terminal, information indicating the usage status of the facility is acquired by the communication means, the acquired information indicating the usage status is displayed on the display means, According to the reservation means for transmitting a request for reservation or cancellation of the facility to the facility by the communication means;
(8) The portable information terminal according to (8).

(Corresponding embodiment)
The embodiment relating to the portable information terminal described in (30) corresponds to the first embodiment.
(Function and effect)
According to the portable information terminal described in (30), the user can check the real-time usage status by communicating with the facility using the portable information terminal without actually going to the facility in the specific area. In addition, the facility can be reserved and the reservation can be canceled.

(31) In the specific area, there is a sales facility that sells products that can be purchased by the user of the portable information terminal,
The server further stores the list information of the product, transmits the list information of the product to the portable information terminal, and the product desired by the user in response to a request from the portable information terminal. Product management means to carry out purchase procedures for
The portable information terminal further requests the merchandise management means of the server to transmit the merchandise list information according to a user operation, and the display means displays the merchandise list information received in response to the request. And a purchase means for transmitting a purchase procedure request for the selected product to the product management means of the server according to the product selection by the user's operation,
The server and the sales facility in the specific area are configured to be capable of wireless or wired communication,
The purchase procedure of the product by the product management means transmits purchase reservation information of the selected product or an instruction to transfer the selected product to a predetermined receiving position in the specific area to the sales facility of the selected product. It is to be,
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (9) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-of-view information presentation system or portable information terminal described in (31).
(Function and effect)
According to the field-of-view information presentation system or portable information terminal described in (31), the user of the portable information terminal obtains a list of products that can be purchased at a sales facility in a specific area from the server, and It is possible to carry out a purchase procedure without visiting a sales facility for a desired product. In addition, if the received goods are received at a predetermined receiving position in the specific area, for example, the exit if the facility is an amusement park, theme park, museum, etc., the purchased goods are held. Since there is no need to travel within the specific area, it is free from the complexity of participating in attractions and events. In addition, since it is possible to make a purchase reservation in a remote state, it is also convenient in that it does not need to travel significantly within a specific area. Furthermore, the waiting time can be used effectively when the purchase reservation is arranged in a queue for participation such as an attraction or until food is served at a restaurant or the like.

(32) In the specific area, there is a sales facility that sells products that can be purchased by the user of the portable information terminal,
The portable information terminal further includes:
A communication means for performing wireless or wired communication with the sales facility;
Product information storage means for storing the product list information;
According to the operation of the user of the portable information terminal, the list information of the products stored in the product information storage means is displayed on the display means, and the communication means according to the product selection by the user operation A purchasing means for transmitting a purchase request for the selected product to a sales facility or a request for transferring the selected product to a predetermined receiving position in the specific area;
(8) The portable information terminal according to (8).

(Corresponding embodiment)
The embodiment relating to the portable information terminal described in (32) corresponds to the first embodiment.
(Function and effect)
According to the portable information terminal described in (32), the user of the portable information terminal displays a list of products that can be purchased at a sales facility in a specific area on the portable information terminal, and sells the desired product. You can proceed with the purchase without going to the facility. In addition, if the received goods are received at a predetermined receiving position in the specific area, for example, the exit if the facility is an amusement park, theme park, museum, etc., the purchased goods are held. Since there is no need to travel within the specific area, it is free from the complexity of participating in attractions and events. In addition, since it is possible to make a purchase reservation in a remote state, it is also convenient in that it does not need to travel significantly within a specific area. Furthermore, the waiting time can be used effectively when the purchase reservation is arranged in a queue for participation such as an attraction or until food is served at a restaurant or the like.

  (33) The server or the portable information terminal further uses the portable information terminal so that the purchase price of the product can be paid in a lump sum through a credit company corresponding to the stored credit card number. The field-of-view information presentation system according to (31) or the portable information terminal according to (31) or (32), further comprising card number storage means for storing a credit card number of the person.

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-of-view information presentation system or portable information terminal described in (33).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (33), since the purchase price of the product can be settled in a lump, the user must line up in the long line of the cash register every time the product is purchased. This frees you from the complexity of payment.

(34) The predetermined information associated with the known marker is image data for instructing a moving direction,
Within the specific area, there is a sales facility that sells products that users of the mobile information terminal can purchase.
The server or the portable information terminal further stores, for each product, detailed information including an image, a price, a sales facility location, a display position in the sales facility, and a product operated by a user operation of the portable information terminal. In response to a list browsing request, it comprises product management means for displaying list information of all products including at least images and prices in the detailed information on the display means,
In the processing of the predetermined information, in response to a product selection instruction from the user, the location of the sales facility and the display position of the selected product are retrieved from the product management means, and the marker placement position is Based on the image data for instructing the moving direction based on the route to the display position of the selected product that has been searched and the direction in which the image capturing means is directed determined by the calculated position and orientation relationship. Processing the generated image to generate a guide image indicating a moving direction to the display position.
The field-of-view information presentation system according to any one of (1) to (7) or the portable information terminal according to any one of (8) to (15).

(Corresponding embodiment)
The first embodiment corresponds to the embodiment of the field-of-view information presentation system or portable information terminal described in (34).
(Function and effect)
According to the field-of-view information presentation system or the portable information terminal described in (34), the sales facility that recognizes the current user's position from the markers photographed before and after and handles the product desired by the user from the current position Since the route to the display location and the route to the display position of the product in the sales facility can be displayed on the portable information terminal, the target product location can be obtained accurately.

(35) Image input means for acquiring an observation image;
Marker extraction means for extracting a specific marker captured in the observation image acquired by the image input means;
Attitude calculating means for calculating attitude information of the image input means using the specific marker extracted by the marker extracting means;
Information presenting means for presenting predetermined information based on the posture information calculated by the posture calculating means;
Content possessing means for storing information to be presented as the predetermined information,
An information presentation system comprising:

(Corresponding embodiment)
The embodiment relating to the information presentation system described in (35) corresponds to the second embodiment.
(Function and effect)
According to the information presentation system described in (35), the posture of the image input unit at the time of photographing can be calculated by capturing a specific marker in the observation screen by the image input unit. And it becomes possible to display the predetermined information according to the posture information in the observation screen.

  (36) A portable information terminal comprising the image input means and a display means for displaying predetermined information presented by the information presentation means, the marker extraction means, the posture calculation means, the information presentation means, the content (35) The information presentation system according to (35), further comprising at least one of possessing means, wherein the other means is retained in a server connectable to the portable information terminal by communication.

(Corresponding embodiment)
The fifth embodiment corresponds to the embodiment related to the information presentation system described in (36).
(Function and effect)
According to the information presentation system described in (36), only the information is transmitted to the server, the calculation is performed by the server, and only the result is used by the portable information terminal. It becomes possible to compensate for the decrease in speed.

(37) Image input means for acquiring an observation image;
Marker design identification means for identifying the design of a specific marker captured in the observation image, acquired by the image input means,
Position calculating means for calculating position information of the image input means from the design of the specific marker identified by the marker design identifying means;
Information presenting means for presenting predetermined information based on the position information calculated by the position calculating means;
Content possessing means for storing information to be presented as the predetermined information,
An information presentation system comprising:

(Corresponding embodiment)
The embodiment related to the information presentation system described in (37) corresponds to the third embodiment.
(Function and effect)
According to the information presentation system described in (37), the position of the image input unit at the time of photographing can be calculated by capturing a specific marker in the observation screen by the image input unit. And it becomes possible to display the predetermined information according to the position information in the observation screen.

  (38) A mobile information terminal comprising the image input means and a display means for displaying predetermined information presented by the information presentation means, the marker design identification means, the position calculation means, the information presentation means, (37) The information presentation system according to (37), further comprising at least one of content-owning means, wherein the other means is stored in a server connectable to the portable information terminal by communication.

(Corresponding embodiment)
The fifth embodiment corresponds to the embodiment related to the information presentation system described in (38).
(Function and effect)
According to the information presentation system described in (38), only the information is transmitted to the server, the calculation is performed by the server, and only the result is used by the portable information terminal. It becomes possible to compensate for the decrease in speed.

(39) Image input means for acquiring an observation image;
Marker extraction means for extracting a specific marker captured in the observation image acquired by the image input means;
Marker design identification means for identifying a design included in the specific marker extracted by the marker extraction means;
Position and orientation calculation means for calculating position and orientation information of the image input means from the specific marker extracted by the marker extraction means and the design of the specific marker identified by the marker design identification means;
Information presenting means for presenting predetermined information based on the position and orientation information calculated by the position and orientation calculating means;
Content possessing means for storing information to be presented as the predetermined information,
An information presentation system comprising:

(Corresponding embodiment)
The embodiment relating to the information presentation system described in (39) corresponds to the fourth embodiment.
(Function and effect)
According to the information presentation system described in (39), the position and orientation of the image input unit at the time of photographing can be calculated by capturing a specific marker in the observation screen by the image input unit. And it becomes possible to display the predetermined information according to the position and orientation information in the observation screen.

  (40) In a portable information terminal comprising the image input means and a display means for displaying predetermined information presented by the information presentation means, the marker extraction means, the marker design identification means, the position and orientation calculation means, The information presentation according to (39), wherein at least one of the information presentation means and the content possession means is provided, and the other means is stored in a server connectable to the portable information terminal by communication. system.

(Corresponding embodiment)
The fifth embodiment corresponds to the embodiment related to the information presentation system described in (40).
(Function and effect)
According to the type information presentation system described in (40), only the information is transmitted to the server, the calculation is performed by the server, and only the result is used by the portable information terminal. It becomes possible to compensate for the decrease in processing speed.

(41) display means;
Marker extraction means for receiving an observation image attached to the portable information terminal or acquired by an image input means included in the portable information terminal and extracting a specific marker photographed in the observation image;
Posture calculating means for calculating posture information of the image input means using the marker extracted by the marker extracting means;
Information presenting means for presenting the predetermined information by causing the display means to display predetermined information based on the posture information calculated by the posture calculating means;
Content possessing means for storing information to be presented as the predetermined information,
A portable information terminal comprising:

(Corresponding embodiment)
The embodiment relating to the portable information terminal described in (41) corresponds to the second embodiment.
(Function and effect)
According to the portable information terminal described in (41), the position of the image input unit at the time of photographing can be calculated by capturing a specific marker in the observation screen by the image input unit. And it becomes possible to display the predetermined information according to the position information in the observation screen.

(42) display means;
Marker design identifying means for receiving an observation image attached to the portable information terminal or acquired by an image input means included in the portable information terminal and identifying a design of a specific marker photographed in the observation image; ,
Position calculating means for calculating position information of the image input means from the design of the specific marker identified by the marker design identifying means;
Information presenting means for presenting the predetermined information by causing the display means to display predetermined information based on the position information calculated by the position calculating means;
Content possessing means for storing information to be presented as the predetermined information,
A portable information terminal comprising:

(Corresponding embodiment)
The embodiment relating to the portable information terminal described in (42) corresponds to the third embodiment.
(Function and effect)
According to the portable information terminal described in (42), the position of the image input unit at the time of photographing can be calculated by capturing a specific marker in the observation screen by the image input unit. And it becomes possible to display the predetermined information according to the position information in the observation screen.

(43) display means;
Marker extraction means for receiving an observation image attached to the portable information terminal or acquired by an image input means included in the portable information terminal and extracting a specific marker photographed in the observation image;
Marker design identification means for identifying a design included in the specific marker extracted by the marker extraction means;
Position and orientation calculation means for calculating position and orientation information of the image input means from the specific marker extracted by the marker extraction means and the design of the specific marker identified by the marker design identification means;
Information presenting means for presenting the predetermined information by causing the display means to display predetermined information based on the position and orientation information calculated by the position and orientation calculating means;
Content possessing means for storing information to be presented as the predetermined information,
A portable information terminal comprising:

(Corresponding embodiment)
The embodiment relating to the portable information terminal described in (43) corresponds to the fourth embodiment.
(Function and effect)
According to the portable information terminal described in (43), the position and orientation of the image input unit at the time of photographing can be calculated by capturing a specific marker in the observation screen by the image input unit. And it becomes possible to display the predetermined information according to the position and orientation information in the observation screen.

(44) A server connectable by communication to a portable information terminal comprising a display means,
Marker extracting means for extracting a specific marker that is photographed in an observation image attached to the portable information terminal or acquired by an image input means provided in the portable information terminal;
Posture calculating means for calculating posture information of the image input means using the marker extracted by the marker extracting means;
Information presenting means for presenting the predetermined information by causing the display means of the portable information terminal to display predetermined information based on the posture information calculated by the posture calculating means;
Content possessing means for storing information to be presented as the predetermined information,
Comprising at least one of
A server which transmits and receives necessary information to and from each means of a portable information terminal or other server provided with each of the means except for the means provided in the server.

(Corresponding embodiment)
The embodiment related to the server described in (44) corresponds to the fifth embodiment.
(Function and effect)
According to the server described in (44), by capturing a specific marker in the observation screen by the image input unit, the posture of the image input unit at the time of shooting can be calculated. And it becomes possible to display the predetermined information according to the posture information in the observation screen. At the same time, only the information is transmitted to the server, the calculation is performed by the server, and only the result is used by the portable information terminal, thereby making it possible to compensate for the decrease in processing speed caused by the portable information terminal having a low calculation capability.

(45) A server connectable by communication to a portable information terminal comprising a display means,
Marker design identifying means for identifying the design of a specific marker that is attached to the above or that is photographed in an observation image acquired by an image input means provided in the portable information terminal;
Position calculating means for calculating position information of the image input means from the design of the specific marker identified by the marker design identifying means;
Information presenting means for presenting the predetermined information by causing the display means of the portable information terminal to display predetermined information based on the position information calculated by the position calculating means;
Content possessing means for storing information to be presented as the predetermined information,
Comprising at least one of
A server which transmits and receives necessary information to and from each means of a portable information terminal or other server provided with each of the means except for the means provided in the server.

(Corresponding embodiment)
The embodiment related to the server described in (45) corresponds to the fifth embodiment.
(Function and effect)
According to the server described in (45), by capturing a specific marker in the observation screen by the image input means, the posture of the image input means at the time of photographing can be calculated. And it becomes possible to display the predetermined information according to the posture information in the observation screen. At the same time, only the information is transmitted to the server, the calculation is performed by the server, and only the result is used by the portable information terminal, thereby making it possible to compensate for the decrease in processing speed caused by the portable information terminal having a low calculation capability.

(46) A server connectable by communication to a portable information terminal comprising a display means,
Marker extracting means for extracting a specific marker that is photographed in an observation image attached to the portable information terminal or acquired by an image input means provided in the portable information terminal;
Marker design identification means for identifying a design included in the specific marker extracted by the marker extraction means;
Position and orientation calculation means for calculating position and orientation information of the image input means from the specific marker extracted by the marker extraction means and the design of the specific marker identified by the marker design identification means;
Information presenting means for presenting the predetermined information by causing the display means of the portable information terminal to display predetermined information based on the position and orientation information calculated by the position and orientation calculating means;
Content possessing means for storing information to be presented as the predetermined information,
Comprising at least one of
A server which transmits and receives necessary information to and from each means of a portable information terminal or other server provided with each of the means except for the means provided in the server.

(Corresponding embodiment)
The embodiment relating to the server described in (46) corresponds to the fifth embodiment.
(Function and effect)
According to the server described in (46), by capturing a specific marker in the observation screen by the image input means, it is possible to calculate the position and orientation at the time of shooting of the image input means. And it becomes possible to display the predetermined information according to the position and orientation information in the observation screen. At the same time, only the information is transmitted to the server, the calculation is performed by the server, and only the result is used by the portable information terminal, thereby making it possible to compensate for the decrease in processing speed caused by the portable information terminal having a low calculation capability.

FIG. 1A is a diagram showing a configuration of a visual field matching information presentation system according to the first embodiment of the present invention, and FIG. 1B is a diagram of a visual field matching information presentation system according to the first embodiment. FIG. 1C is a functional block diagram, and FIG. 1C is a diagram showing an image of superimposition information displayed on the display means, and FIG. 1D is a diagram showing a two-dimensional navigation display example. FIG. 2A is a diagram illustrating a flowchart for explaining a schedule creation operation, and FIG. 2B is a diagram illustrating a flowchart for explaining a product purchase operation. FIG. 3A is a diagram illustrating a live-action display state of an ancient ruin, and FIG. 3B is a diagram illustrating a state in which a restoration model of the ancient ruin is superimposed and displayed. FIG. 4 is a diagram showing a state in which the detailed information of the product is displayed. FIG. 5 is a diagram showing a configuration of a visual field coincidence type information presentation system according to the second embodiment of the present invention. FIG. 6 is a diagram showing a configuration of a visual field coincidence type information presentation system according to the third embodiment of the present invention. FIG. 7 is a diagram showing a configuration of a visual field coincidence type information presentation system according to the fourth embodiment of the present invention. FIG. 8 is a diagram showing a configuration of a visual field coincidence type information presentation system according to the fifth embodiment of the present invention. FIG. 9 is a diagram for explaining a conventional method for obtaining the position and orientation of the image pickup means from the reference marker position.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 1 ... Image acquisition part, 1A ... Image input means, 2 ... Marker detection (feature point detection) part, 2A ... Marker extraction means, 2B ... Marker design discrimination means, 3 ... Marker recognition (feature point recognition) part, 3A ... Posture Calculation means, 3B ... Position calculation means, 3C ... Position / attitude calculation means, 4 ... Related information acquisition section, 4A ... Content possession means, 5 ... Model data creation (rendering) section, 5A ... Information presentation means, 6 ... Display section, 6A ... display means, 11 ... portable information terminal, 11A ... image capturing means, 11B ... display means, 12 ... server, 13 ... facility, 14 ... marker, 15 ... arrow, 16 ... display of trajectory, 17 ... photographed marker 18 ... Current position, 19 ... Destination, 20 ... Display product, 21 ... Detailed information.

Claims (2)

A plurality of known markers placed in a specific area targeted by the system;
Image capturing means for capturing an image;
Marker detection means for detecting the known marker present in the image captured by the image imaging means;
A position and orientation relationship calculating means for specifying a known marker detected by the marker detecting means and calculating a position and orientation relationship between the identified known marker and the image capturing means;
Related information obtaining means for obtaining predetermined information associated with a known marker identified by the position and orientation relation computing means;
Information processing means for processing the predetermined information obtained by the related information obtaining means based on the position and orientation relationship calculated by the position and orientation relationship calculating means;
Display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means;
Comprising
At least the image capturing means and the display means are provided in a portable information terminal,
Other means are provided in the server capable of wireless communication with the portable information terminal,
Within the specific area, there are multiple attractions and / or events,
The portable information terminal further comprises designation means for designating a plurality of desired attractions and / or events among the plurality of attractions and / or events,
The server or the portable information terminal further includes a schedule generation unit that generates a recommended schedule that can be efficiently turned around in accordance with the designation of a plurality of attractions and / or events by the designation unit,
The portable information terminal has a function of displaying a recommended schedule generated by the schedule generation unit on the display unit.
Field-of-view information presentation system characterized by this. A plurality of known markers placed in a specific area targeted by the system;
Image capturing means for capturing an image;
Marker detection means for detecting the known marker present in the image captured by the image imaging means;
A position and orientation relationship calculating means for specifying a known marker detected by the marker detecting means and calculating a position and orientation relationship between the identified known marker and the image capturing means;
Related information obtaining means for obtaining predetermined information associated with a known marker identified by the position and orientation relation computing means;
Information processing means for processing the predetermined information obtained by the related information obtaining means based on the position and orientation relationship calculated by the position and orientation relationship calculating means;
Display means for superimposing and displaying the information processed by the information processing means on the image taken by the image pickup means;
Comprising
At least the image capturing means and the display means are provided in a portable information terminal,
Other means are provided in the server capable of wireless communication with the portable information terminal,
Within the specific area, there are multiple attractions and / or events,
The server further comprises a waiting time storing means for storing information about the current waiting time for each attraction and / or event.
The portable information terminal is configured to communicate with the server so that information about the waiting time stored in the waiting time storage means of the server can be viewed on the display means.
Field-of-view information presentation system characterized by this.

Images