JP5685436B2 - Augmented reality providing device, augmented reality providing system, augmented reality providing method and program - Google Patents

Description

  The present invention relates to a technique capable of combining and displaying a real space image and computer graphics image data.

  In recent years, Augmented Reality (Augmented Reality) technology has been known in which a virtual object is superimposed and displayed as if it exists in real space. As a method of such augmented reality technology, when a real space is imaged and a marker (identification information) in the captured image is recognized, each marker is set according to the type, size, and position of the marker. In addition, there is a technique in which computer-managed data (hereinafter referred to as computer graphics image data) such as characters, figures, still images, and moving images is combined and displayed at a predetermined position in a captured image. In this manner, by setting various patterns such as various patterns on the marker, various computer graphics image data corresponding to each marker can be displayed in a superimposed manner.

  For example, Patent Literature 1 discloses a method of presenting a virtual real world in which the appearance such as the size and color of a real object and the surrounding environment are harmonized without arranging the real object in the real space to the user. . More specifically, an object number indicating index having a plurality of markers each having identification information and having a known positional relationship with each other is photographed with a camera, and the object number indicating index is determined based on the captured image. In addition to specifying, the 3D position / orientation placed in the real space is acquired, and the 3D model corresponding to the detected identification information is placed in the 3D position / orientation obtained from the object number indication index in the virtual space. Thus, a method of creating a mixed reality space is disclosed.

JP 2003-256876 A

  However, in order to display various computer graphics image data, markers with various patterns and the like set corresponding to these are necessary, and setting various markers has been a troublesome work. . If the number of types of markers is increased to display various computer graphics image data, the possibility of erroneous recognition of the markers increases, and computer graphics image data different from the intended computer graphics image data is synthesized and superimposed. The problem of being displayed occurs. In other words, it becomes difficult to distinguish between markers by setting various markers, and there is a problem that the resolution of the camera etc. must be increased or a large marker must be created in order to accurately identify the marker To do.

  Further, as various markers are set, it takes time to extract computer graphics image data corresponding to the markers, and there is a problem that processing until the computer graphics image data is displayed becomes slower. Therefore, a mechanism that can superimpose and display a variety of information with a small number of marker types is desired.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a technique that enables various information to be superimposed and displayed with a small number of markers.

Therefore, the augmented reality providing device of the present invention is an augmented reality providing device capable of combining and displaying an image of real space and computer graphic image data, and an imaging means for photographing an image including identification information. A recognition means for recognizing identification information included in image data of an image photographed by the photographing means; a rule storage means for storing a correspondence rule between the identification information and the computer graphic image data; and the recognition means. Determining means for determining computer graphics image data corresponding to the recognized identification information based on the correspondence rule; and combining the computer graphics image data determined by the determining means with an image in real space for display and display means for the display means, the same by the recognizing means If another information is more recognized, superimposed and synthesized with the image of the real space by reading the benzalkonium computer graphics image data against the same plurality of identification information based on the correspondence rule, the same by the recognizing means When a plurality of the identification information items are not recognized, the computer is associated with each identification information by referring to a correspondence table in which predetermined identification information and computer graphics image data are associated one-to-one. It is characterized in that graphics image data is read out and superimposed with a real space image .

The augmented reality providing system of the present invention is an augmented reality in which an information processing terminal and an information processing apparatus are communicably connected to each other, and can superimpose and display an image of real space and computer graphics image data. In the providing system, the information processing terminal combines a photographing unit that captures an image including identification information and computer graphics image data determined by the determining unit of the information processing device with a real space image. Display means for causing the information processing apparatus to recognize the identification information included in the image data of the image photographed by the photographing means, and the correspondence relationship between the identification information and the computer graphics image data Rule storage means for storing rules, and a computer group corresponding to the identification information recognized by the recognition means And a determination means for determining based on the fixed image data on the correspondence relation rules, the display means, the identification information identical with the recognition means when a plurality of recognition, a plurality of identical, based on the correspondence relationship rules the superimposed displayed by synthesizing the image in real space by reading the benzalkonium computer graphics image data against the identification information when the identification information identical with the recognition means is not more recognized, identification information predetermined The computer graphics image data associated with each identification information is read out with reference to a correspondence table in which computer graphics image data is associated with computer graphics image data on a one-to-one basis, and is superimposed and displayed in combination with an image in the real space. It is characterized by that.

The augmented reality providing method of the present invention is an augmented reality providing method executed by an augmented reality providing apparatus capable of combining and displaying an image of a real space and computer graphics image data, and includes identifying information. A recognition step based on a correspondence rule between the identification information and the computer graphic image data, a photographing step for photographing an image including the recognition step, a recognition step for recognizing identification information included in the image data of the photographed image. a determination step of determining a recognized computer graphics image data corresponding to the identification information was in step, the determined computer graphics image data by combining the image of the current real space and a display step that presents, in the display step, the same said identification information in said recognition step When it is more recognized, the relationship rules superimposed and synthesized with the image of the real space by reading the benzalkonium computer graphics image data against the same plurality of identification information based on the identification identical in said recognition step When a plurality of pieces of information are not recognized, a computer graphics image associated with each piece of identification information is referred to by referring to a correspondence table in which predetermined identification information and computer graphics image data are associated one-to-one. It is characterized in that data is read out and superimposed with an image in a real space .

The program of the present invention is a program for causing a computer to execute an augmented reality providing method executed by an augmented reality providing apparatus capable of combining and displaying a real space image and computer graphics image data. there are a shadow it has been recognition step of recognizing identification information included in the image data of the image shooting based on the correspondence between the rules and the identification information and the computer graphic image data, are recognized in the recognition step wherein a determination step of determining a computer graphics image data corresponding to the identification information, determined the computer graphics image data by combining the image of the current real space to execute a display step that presents the computer, the display step In the previous recognition step, When a plurality of pieces of identification information are recognized, computer graphics image data corresponding to the same pieces of identification information is read based on the correspondence rule, combined with a real space image, and superimposed and displayed in the recognition step. When a plurality of pieces of information are not recognized, a computer graphics image associated with each piece of identification information is referred to by referring to a correspondence table in which predetermined identification information and computer graphics image data are associated one-to-one. It is a program for reading out data and combining it with an image in the real space to display it superimposed.

  According to the present invention, various kinds of information can be superimposed and displayed with a small number of markers.

It is a figure which shows an example of a structure of AR provision system in embodiment of this invention. It is a figure which shows an example of the hardware constitutions of AR provision apparatus in embodiment of this invention. It is a figure which shows an example of the software structure of the AR provision apparatus in embodiment of this invention. It is a figure which shows an example of the hardware constitutions of the server apparatus in embodiment of this invention. It is a figure which shows an example of the software structure of the server apparatus in embodiment of this invention. It is a flowchart which shows the flow of a process of the augmented reality provision system in embodiment of this invention. It is a figure which shows an example of the priority table in embodiment of this invention. It is a figure which shows an example of a structure of the AR provision system in the 2nd Embodiment of this invention. It is a figure which shows an example of the priority table in the 2nd Embodiment of this invention. It is a figure which shows an example of the priority table in the 3rd Embodiment of this invention. It is a figure which shows an example of the superimposed display in the 3rd Embodiment of this invention. It is a figure which shows an example of the priority table in the 4th Embodiment of this invention. It is a figure which shows an example of the superimposed display in the 4th Embodiment of this invention. It is a figure which shows an example of the superimposed display in the 5th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, embodiment described below is for description and does not limit the scope of the present invention. Accordingly, those skilled in the art can employ embodiments in which each or all of these elements are replaced by equivalents thereof, and these embodiments are also included in the scope of the present invention.

<First Embodiment>
First, a first embodiment of the present invention will be described. FIG. 1 is a diagram illustrating an example of a configuration of an augmented reality providing system according to the present embodiment. As shown in FIG. 1, the augmented reality providing system according to the first embodiment includes an information processing terminal (AR providing device) 100 and an information processing device (server device) 200. The AR providing apparatus 100 and the server apparatus 200 are communicably connected to each other via a network such as a wireless communication line.

  The AR providing device 100 is a portable information terminal, a mobile phone, an HMD (Head Mounted Display), a digital camera, or the like, and at least a photographing device for photographing an image, an image photographed by the photographing device, and computer graphics image data. A display device for displaying is provided. The computer graphics image data corresponding to the marker 300 recognized by the server device 200 is superimposed and displayed at a position that matches the real space that can be perceived via the AR providing device 100, thereby expanding an augmented reality (AR). )I will provide a. In the present invention, it is allowed to install a plurality of identical markers, and different computer graphics image data can be superimposed and displayed even though they are exactly the same markers.

  FIG. 2 is a diagram illustrating an example of a hardware configuration of the AR providing apparatus 100 according to the embodiment of the present invention. As shown in FIG. 2, the AR providing apparatus 100 includes a control device 101, a storage device 102, a communication device 103, a display device 104, and a photographing device 105 as hardware configurations, and is connected via a bus. It has a configuration.

  The control device 101 is a CPU (Central Processing Unit), for example, and reads a program from the storage device 102 and executes the program as necessary. By executing the program, a function to be described later in the AR providing apparatus 100 and a process according to a flowchart to be described later are realized.

  The storage device 102 is a ROM (Read Only Memory), a RAM (Random Access Memory), an HD (Hard Disk), or the like, and stores data or data used when the control device 101 executes processing based on the program. Or remember. More specifically, the storage device (ROM) 102 stores a program that is read first when the AR providing device 100 is powered on. The storage device (RAM) 102 functions as a main memory of the AR providing device 100. The storage device (HD) 102 functions as an external memory.

  The AR providing apparatus 100 may acquire various types of information stored in the storage device 102 from a recording medium such as a CD-ROM, or may download the information via a network.

  The communication device 103 communicates with the server device 200 via a network, and transmits image data to the server device 200, receives various information from the server device 200, and the like. Various types of information received by the communication device 103 are stored in the storage device 102. The display device 104 is a liquid crystal display or the like and displays various images. The imaging device 105 converts a subject image incident through an optical lens and formed on an image sensor into an electrical signal, and outputs moving image data or still image data.

  FIG. 3 is a diagram illustrating an example of a software configuration of the AR providing apparatus 100 according to the embodiment of the present invention. As illustrated in FIG. 3, the AR providing apparatus 100 includes a capturing unit 1001, a display control unit 1002, and an image acquisition unit 1003 as a software configuration (functional configuration). The capturing unit 1001, the display control unit 1002, and the image acquisition unit 1003 are configured by, for example, a program stored in the HD and loaded into the RAM as necessary, and a CPU that executes the program.

  The take-in unit 100 1 takes in information necessary for superimposed display from a storage device 202 (described later) of the server device 200. The display control unit 1002 generates an image to be superimposed and displayed on the display device 104 based on the information acquired by the image acquisition unit 1003 and the acquisition unit 1001. At this time, the display control unit 1002 performs coordinate matching so that the computer graphics image data is aligned with the display target portion in the real space and displayed on the display device 104.

  FIG. 4 is a diagram illustrating an example of a hardware configuration of the server device 200 according to the embodiment of the present invention. The server device 200 manages various computer graphics image data to be displayed superimposed on the AR providing device 100. As illustrated in FIG. 4, the server device 200 includes a control device 201, a storage device 202, a communication device 203, a display device 204, and an input device 205, and has a configuration connected via a bus.

  The control device 201 is a CPU (Central Processing Unit), for example, and reads the program from the storage device 202 and executes the program as necessary. By executing the program, a function described later in the server device 200 and a process related to a flowchart described later are realized.

  The storage device 202 is a ROM (Read Only Memory), a RAM (Random Access Memory), an HD (Hard Disk), or the like, and stores data or data used when the control device 201 executes processing based on the program. Or remember. More specifically, the storage device (ROM) 202 stores a program that is read first when the server device 200 is powered on. The storage device (RAM) 202 functions as a main memory of the server device 200. The storage device (HD) 202 functions as an external memory. The server device 200 may acquire various types of information stored in the storage device 200 from a recording medium such as a CD-ROM, or may download the information via a network or the like.

  The communication device 203 communicates with the AR providing device 100 via a network, receives image data from the AR providing device 100, transmits computer graphics image data to the AR providing device 100, and the like. The display device 204 is a liquid crystal display or the like and displays various information. The input device 205 is a keyboard and mouse operated by the user, and inputs various information to the server device 200.

  The server device 200 manages computer graphics image data displayed on the AR providing device 100 corresponding to each marker. When the server apparatus 200 receives the image data from the AR providing apparatus 100, the server apparatus 200 recognizes the marker 300 included in the image data, reads out the computer graphics image data corresponding to the marker 300 from the storage apparatus 202, and sends the computer to the AR providing apparatus 100. Send graphics image data. When the AR providing apparatus 100 receives the computer graphics image data from the server apparatus 200, the AR providing apparatus 100 synthesizes and superimposes the computer graphics image on the real space image displayed on the display device 104.

  FIG. 5 is a diagram illustrating an example of a software configuration of the server apparatus 200 according to the embodiment of the present invention. As illustrated in FIG. 5, the server apparatus 200 includes a marker recognition unit 2001 and a determination unit 2002 as a software configuration (functional configuration). The marker recognizing unit 2001 and the determining unit 2002 have a configuration realized by, for example, a program stored in the HD and loaded into the RAM as necessary and a CPU that executes the program. The marker recognizing unit 2001 performs processing for recognizing the marker 300 from the image data of the captured image that is captured and transmitted by the imaging device 105 of the AR providing device 100. The marker recognition unit 2001 identifies the type, size, position, and the like of the marker 300 by recognizing a pattern such as the pattern of the marker 300.

  The server device 200 stores computer graphics image data corresponding to the type of the marker 300. The determination unit 2002 selects computer graphics image data corresponding to the type of the marker 300 recognized by the recognition unit 2001 from the computer graphics image data stored in the storage device 202 of the server device 200, and displays the superimposed image. It is determined as data for performing.

  The display device 104 of the AR providing device 100 combines and superimposes the image data captured by the imaging device 105 and the computer graphics image data determined by the determination unit 2002.

  FIG. 6 is a flowchart showing a process flow of the augmented reality providing system in the embodiment of the present invention. Hereinafter, the process of the augmented reality providing system in the present embodiment will be described with reference to FIGS. 1 and 6.

  In step S101, the user captures an image including the marker 300 with the imaging device 105 of the AR providing device 100, and the imaging device 105 generates image data of the captured image. The AR providing apparatus 100 transmits the image data generated in step S101 to the server apparatus 200 via the network by the communication apparatus 103 (step S102).

  The server device 200 receives the image data of the captured image transmitted from the AR providing device 100 (step S201). Processing for recognizing the marker 300 from the image data received in step 201 is performed (step S202). When the marker 300 is recognized, the type of the marker 300 is specified by the pattern such as the pattern of the marker 300, and the process of associating the coordinate information (X, Y) with the position of the marker 300 in the image data is performed (step S203).

  Subsequently, it is determined whether or not a plurality of markers 300 of the same type are included in the image data (step S204). At this time, when a plurality of markers 300 of the same type are recognized in the image data, different computer graphics image data for the same type of marker 300 is determined in accordance with a predetermined rule (step S205) stored in the storage device 202. Is determined (step S207), and the computer graphics image data is transmitted to the AR providing apparatus 100 (step S208).

  On the other hand, when the same type of marker 300 is not recognized in the image data, a correspondence table (step S206) in which a predetermined type of marker 300 and computer graphics image data are associated one-to-one. Then, the computer graphics image data to be displayed in a superimposed manner is determined (step S207), and the computer graphics image data is transmitted to the AR providing apparatus 100 (step S208). Note that the computer graphics image data corresponding to the first marker 300 with the priority in a priority table to be described later may be selected.

  The predetermined rule stored in the storage device 102 of the server device 200 is, for example, a priority table 400 shown in FIG. 7, and different computer graphics image data is transmitted to the AR providing device 100 according to the priority table 400. To do.

  In this embodiment, when three markers M1 of the same type are captured at the same time, “ABC” is determined as computer graphics image data for the marker M1 having the first priority, and is transmitted to the AR providing apparatus 100. . Similarly, “DEF” is determined as computer graphics image data for the marker M1 with the second priority and “GHI” is determined with respect to the marker M1 with the third priority, and is transmitted to the AR providing apparatus 100. To do.

  At this time, for each of the three markers M1, for example, M1 of coordinates (X1, Y1) is the first priority, coordinates (X2, Y2) based on the coordinate information associated in advance and the priorities of the respective markers M1. ) Is designated second and M1 of coordinates (X3, Y3) is designated third, so that different computer graphics image data can be superimposed and displayed even for the same type of marker M1.

  Note that the priority order of the markers is set so that, for example, the marker 300 with a shorter distance has a higher priority according to the distance between the AR providing apparatus 100 and the marker 300. By using the same size marker 300, the distance between the AR providing apparatus 100 and the marker 300 can be determined by comparing the size of the marker 300. That is, in the marker 300 having the same size and the same pattern, it is determined that the larger the marker 300 is, the smaller it is, and the smaller the marker 300 is.

  Further, the priority order of the marker 300 may be set according to the purpose, such as increasing the priority order of the marker 300 closer to the center of the display screen regardless of the size of the marker 300. For example, the distance between the center coordinates and each marker 300 can be compared based on the center coordinates (X0, Y0) on the display screen and the coordinates (Xa, Ya) of each marker 300.

  Alternatively, the higher-priority marker 300 may be displayed with larger computer graphics image data to be superimposed and the lower priority marker 300 may be displayed with smaller computer graphics image data to be superimposed.

  In step S103, the AR providing apparatus 100 receives the computer graphics image data determined and transmitted by the server apparatus 200. The AR providing apparatus 100 synthesizes the computer graphics image data received in step S103 and the real space image (step S104), and superimposes the computer graphics image data on the display screen (step S105).

  According to the present embodiment, when the same type of marker 300 is installed in the real space and a plurality of the same marker 300 is captured at the same time, different computer graphics image data is displayed for each marker 300. Various types of information can be displayed with a few types of markers 300.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. In the second embodiment, differences from the above-described embodiment will be described. In the present embodiment, the marker 300 is attached to a vehicle such as a bus or a train, an airplane, or the like, and computer graphics image data such as advertisement information and area information is superimposed and displayed on the vehicle or the aircraft.

  As illustrated in FIG. 8, when an image of a vehicle or the like is captured by the AR providing apparatus 100 possessed by the user, image data of the captured image is transmitted to the server apparatus 200. When the server apparatus 200 receives the image data transmitted from the AR providing apparatus 100, the server apparatus 200 performs processing for recognizing the marker 301 from the received image data. In addition, it is preferable to use the feature point of the original color conventionally drawn on the vehicle etc. as the marker 301. FIG. By setting it as such a structure, the effort which attaches a marker newly is reduced.

  When the server device 200 recognizes the marker 301 that is the original color of the vehicle, the server device 200 identifies the type, size, and position of the marker 301 and determines whether or not a plurality of the same type of marker 301 is included in the image data. I do.

  At this time, when a plurality of markers 301 of the same type are recognized, different computer graphics image data for the same type of marker 301 according to the priority table 401 (FIG. 9) stored in the storage device 202 of the server device 200. Is transmitted to the AR providing apparatus 100.

  On the other hand, when the same type of marker 301 is not recognized in the image data, a correspondence table (not shown) in which a predetermined type of marker 301 and computer graphics image data are associated one-to-one. Thus, the computer graphics image data to be displayed in a superimposed manner is determined, and the computer graphics image data is transmitted to the AR providing apparatus 100.

  When a plurality of the same type of markers 301 are recognized, the priority table 401 performs control such that information with higher priority is displayed in the foreground in the image displayed on the AR providing apparatus 100.

  In this embodiment, when three markers M1 of the same type (marker 301, which is the original color of the vehicle in FIG. 8 in this embodiment) is captured at the same time, priority is given to the first marker M1. Determines “Department store A” as computer graphics image data and transmits it to the AR providing apparatus 100. Similarly, “super B” is determined as the computer graphics image data for the second marker M1 and “shop C” for the third marker M1, and is transmitted to the AR providing apparatus 100. .

  When two or more types of markers M1 and M2 are recognized in the image data, in addition to the above, the priority of M2 is “restaurant D” with respect to the first marker M2, and the second marker. “Restaurant E” for M2 and “ramen F” for the third marker M2 are determined as computer graphics image data and transmitted to the AR providing apparatus 100.

  According to this embodiment, it is easier to change the content of the advertisement as compared with a wrapping advertisement that is conventionally performed by painting or sticking a sticker on a vehicle or the like. When captured, since different computer graphics image data is displayed for each marker, it is possible to provide a technique for effectively presenting various advertisements, regional information, and the like to the user.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. In the third embodiment, differences from the above-described embodiment will be described. In the present embodiment, computer graphics image data such as advertisement information and area information according to the area (position) is superimposed and displayed by the AR providing apparatus 100 with a GPS function possessed by the user.

  The AR providing apparatus 100 with a GPS function possessed by the user includes a position information calculation unit as a software configuration. When an image including a vehicle or the like is captured by the AR providing apparatus 100, image data of the captured image is generated, and position information (here, latitude and longitude) of the AR providing apparatus 100 based on data received from a GPS satellite. Is calculated. The AR providing apparatus 100 transmits the calculated position information to the server apparatus 200 together with the generated image data.

  When the server apparatus 200 receives the image data and the position information transmitted from the AR providing apparatus 100, the server apparatus 200 recognizes the marker 300 from the received image data and specifies the area based on the position information. In this case, not only the normal marker 300 but also the original color of the vehicle may be used as the marker 301 as in the second embodiment.

  Here, the area may be specified based on the position information by using an area specifying table in which areas corresponding to latitude and longitude are set in advance. For example, the area corresponding to (latitude x1, longitude y1) is specified as “Shinkawa area”, and the area corresponding to (latitude x2, longitude y2) is specified as “Marunouchi area”.

  When the marker 300 is recognized, the type of the marker 300 is specified, and the computer graphics image data is sent to the AR providing apparatus 100 based on the priority order tables 402, 403, and 404 in the area specified in the area specifying table (FIG. 10). Send. FIG. 11 is a diagram showing an example of superimposed display in the third embodiment of the present invention. As shown in FIG. 11, the AR providing apparatus 100 displays “Department Store A” in the Shinkawa area on the marker M1, based on the priority order tables 402, 403, and 404 of the specified area, and in the Marunouchi area. “Super B” is superimposed and “Shop C” is superimposed in Yaesu area.

  In FIG. 10, “Restaurant D” is superimposed and displayed in the Shinkawa district and “Ramen F” is superimposed and displayed in the Marunouchi and Yaesu districts with respect to the first priority of the marker M2.

  Further, the process of transmitting the latest position information from the AR providing apparatus 100 to the server apparatus 200 may be performed every predetermined time interval or whenever the AR providing apparatus 100 calculates the latest position information. May be.

  According to this embodiment, it is possible to set various computer graphics image data to be displayed depending on the region where the user is staying even if the marker 300 is the same type, and it is necessary to create the marker 300 for each region. Therefore, it is possible to provide a technique for effectively presenting various advertisements, regional information, and the like to the user according to the region.

<Fourth Embodiment>
Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, differences from the above-described embodiment will be described. In the present embodiment, computer graphics image data such as advertisement information and area information according to time is superimposed and displayed by the AR providing apparatus 100 with a time measurement function possessed by the user.

The AR providing apparatus 100 with a time measurement function possessed by the user includes a time measurement unit that measures time as a software configuration. When an image including a vehicle or the like is captured by the AR providing apparatus 100, image data of the captured image is generated, and the time is measured by the time measurement unit. The AR providing apparatus 100 transmits the measured time information to the server apparatus 200 together with the generated image data.
When the server apparatus 200 receives the image data and time information transmitted from the AR providing apparatus 100, the server apparatus 200 recognizes the marker 300 from the received image data, specifies the type, size, and position of the marker 300, and also includes the time information. Based on the priority order table 405, 406, or 407 shown in FIG. The computer graphics image data is transmitted to the AR providing apparatus 100 based on the priority order of the priority order table selected based on the time.

  FIG. 13 is a diagram showing an example of superimposed display in the fourth embodiment of the present invention. As shown in FIG. 13, when the current time is “9:00” for the first priority marker M1, it is included in the priority table 405 for the time zone “6: 00-10: 00”. The advertisement of “Morning Set” is superimposed and displayed according to the priority table 405 of the time zone “6: 00-10: 00”. When the current time is “18:00”, first, it is determined whether or not it is included in the priority table 405 of the time zone “6: 00-10: 00”. If it is determined that it is not included, it is next determined whether or not it is included in the priority table 406 of the time zone “17: 00-21: 00”. Here, if it is determined that it is included in the priority table 406 of the time zone “17: 00-21: 00”, the “dinner” is set according to the priority table 406 of the time zone “17: 00-21: 00”. "Set" ads are superimposed. If the current time is “15:00”, it is determined that it is not included in either “6: 00-10: 00” or “17: 00-21: 00”, and the priority table of “Others” In accordance with 407, the advertisement of “recommended set” is displayed in a superimposed manner.

  In addition to setting a priority table corresponding to a specific time as described above, for example, a priority table specifying an application time is set, and three priority tables A, B, and C are set every five hours. Switching from A → B → C → A →... May be performed.

  Further, the server device side 200 may be provided with a time measurement function, and the computer graphics image data may be superimposed and displayed based on the time when the image data is received from the AR providing device 100.

  According to the present embodiment, it is possible to set various computer graphics image data to be displayed depending on the time when the user stays even for the same type of marker 300, and it is necessary to create the marker 300 for each time. Therefore, it is possible to provide a technique for effectively presenting various advertisements, regional information, and the like to the user according to time.

<Fifth Embodiment>
Next, a fifth embodiment of the present invention will be described. In the fifth embodiment, differences from the above-described embodiment will be described. In the present embodiment, when a plurality of markers 300 of the same type are recognized, large computer graphics image data is superimposed and displayed based on a predetermined rule.

  When the AR providing apparatus 100 possessed by the user captures an image of a vehicle or the like, the image data of the captured image is transmitted to the server apparatus 200. When the server apparatus 200 receives the image data transmitted from the AR providing apparatus 100, the server apparatus 200 performs processing for recognizing the marker 300 from the received image data.

  When the marker 300 is recognized, the type, size, and position of the marker 300 are specified, and when a plurality of the same type of markers 300 are recognized, the plurality of markers 300 are assigned according to the table stored in the storage device 202 of the server device 200. On the other hand, one piece of computer graphics image data is transmitted to the AR providing apparatus 100.

  In the present embodiment, when only one marker M1 is captured, computer graphics image data corresponding to the marker M1 is transmitted to the AR providing apparatus 100 as in the normal AR technology. For example, as shown in FIG. When two markers M1 attached to the vehicle in the real space are captured, one large advertisement is superimposed on the display screen of the AR providing apparatus 100.

  According to the present embodiment, when a plurality of markers of the same type are captured, one large computer graphics image data can be displayed, so that various advertisements, regional information, and the like can be effectively presented to the user. Can be provided.

  In the above description, a case where one advertisement is superimposed and displayed on two markers M1 is exemplified, but one advertisement may be superimposed and displayed on three markers M1, for example. , One advertisement may be superimposed and displayed on four or more. In addition, a plurality of markers M1 are captured, and one large advertisement is superimposed and displayed only when they are captured in a predetermined arrangement, for example, linearly as shown in FIG. The same processing as in the other embodiments may be performed.

<Other embodiments>
In the embodiment described above, the AR providing apparatus 100 and the server apparatus 200 share functions, but these may be performed by one apparatus. For example, all of the functions described as the server apparatus 200 are performed by the AR providing apparatus. 100 may be provided.

  Further, the server device 200 may combine the image data photographed by the AR providing device 100 and the computer graphics image data. In addition, when the number of markers 300 is detected exceeding the number set in the priority order table, it is repeatedly displayed again from the computer graphics image data with the higher priority order to the computer graphics image data with the lower priority order. Good.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the technical means disclosed in different embodiments can be appropriately combined. Such embodiments are also included in the technical scope of the present invention.

100: AR providing device 200: Server device 300: Marker

Claims (8)

An augmented reality providing apparatus capable of combining and displaying a real space image and computer graphic image data,
Photographing means for photographing an image including identification information;
Recognizing means for recognizing identification information included in image data of an image photographed by the photographing means;
Rule storage means for storing a correspondence rule between the identification information and the computer graphic image data;
Determining means for determining computer graphics image data corresponding to the identification information recognized by the recognition means based on the correspondence rule;
Display means for displaying the computer graphics image data determined by the determination means in combination with an image in real space, and
The display means, if the same said identification information is more recognized by the recognizing means, and an image of the physical space reads benzalkonium computer graphics image data against the same plurality of identification information based on the correspondence rule When a plurality of the same identification information is not recognized by the recognizing means, a combination table in which predetermined identification information and computer graphics image data are associated one-to-one is referred to Then, the augmented reality providing apparatus is characterized in that computer graphics image data associated with each identification information is read out and superimposed with an image in a real space .   When a plurality of the same identification information is recognized by the recognition means, different computer graphics image data is read out from the same plurality of identification information, combined with a real space image, and superimposed and displayed. The augmented reality providing device according to claim 1.   When a plurality of the same identification information is recognized by the recognition means, one computer graphics image data is read out for the same plurality of the identification information, combined with a real space image, and superimposed and displayed. The augmented reality providing device according to claim 1.   And further comprising position information calculation means for calculating position information, wherein computer graphics image data is read out based on the correspondence rule corresponding to the position information, and is superimposed and displayed in combination with an image in real space. The augmented reality providing device according to any one of claims 1 to 3.   2. The method according to claim 1, further comprising measuring means for measuring time, wherein the computer graphics image data is read based on the correspondence rule corresponding to the measured time, and is superimposed on the real space image. 5. The augmented reality providing device according to any one of 4 above. An augmented reality providing system in which an information processing terminal and an information processing apparatus are communicably connected to each other and can superimpose and display an image in real space and computer graphics image data,
The information processing terminal
Photographing means for photographing an image including identification information;
Display means for displaying the computer graphics image data determined by the determination means of the information processing apparatus in combination with an image in real space, and
The information processing apparatus includes:
Recognizing means for recognizing identification information included in image data of an image photographed by the photographing means;
Rule storage means for storing a correspondence rule between the identification information and the computer graphics image data;
Determining means for determining computer graphics image data corresponding to the identification information recognized by the recognition means based on the correspondence rules;
The display means, the image of the case where the same said identification information is more recognized by the recognition unit, the correspondence between physical space reads benzalkonium computer graphics image data against the same plurality of the identification information based on rules When a plurality of the same identification information is not recognized by the recognition means, a correspondence table in which predetermined identification information and computer graphics image data are associated with each other in a one-to-one correspondence is displayed. A system for providing augmented reality, characterized in that computer graphics image data associated with each identification information is read out, synthesized with an image in real space, and superimposed and displayed . An augmented reality providing method executed by an augmented reality providing apparatus capable of combining and displaying a real space image and computer graphics image data,
A photographing step for photographing an image including identification information;
A recognition step for recognizing identification information included in image data of the captured image;
A determination step of determining computer graphics image data corresponding to the identification information recognized in the recognition step based on a correspondence rule between the identification information and the computer graphic image data;
A display step that presents the determined computer graphics image data by combining the image of the current real space
Including
In the display step, when the identification information same in said recognition step is more recognized, the image in real space by reading the benzalkonium computer graphics image data against the same plurality of identification information based on the correspondence rule When a plurality of the same identification information is not recognized in the recognition step, a correspondence table in which predetermined identification information and computer graphics image data are associated one-to-one is referred to A method for providing augmented reality, comprising: reading out computer graphics image data associated with each piece of identification information and combining the data with an image in a real space to display the augmented reality. A program for causing a computer to execute an augmented reality providing method executed by an augmented reality providing device capable of combining and displaying an image of real space and computer graphics image data ,
A recognition step of recognizing identification information included in the image data of the shadow by said image shooting,
A determination step of determining computer graphics image data corresponding to the identification information recognized in the recognition step based on a correspondence rule between the identification information and the computer graphic image data;
A display step that presents the determined computer graphics image data by combining the image of the current real space
To the computer,
In the display step, when a plurality of the same identification information is recognized in the recognition step, computer graphics image data for the same plurality of identification information is read based on the correspondence rule and synthesized with an image in the real space. When a plurality of the same identification information is not recognized in the recognition step, with reference to a correspondence table in which predetermined identification information and computer graphics image data are associated one-to-one, A program for reading out computer graphics image data associated with identification information, combining it with an image in real space, and displaying it superimposed .

Images