JP2019125342A - Program, information processing device and information processing method - Google Patents

Abstract

To provide a program which enables each user to freely set a content.SOLUTION: An information processing program causes a computer to execute processes to: receive selection of display image to be superimposed on a photographed image of a sphere having a marker on a surface thereof; acquire positional information corresponding to the display image; and transmit the received display image and acquired positional information in association with a user ID. Thus, the information processing program enables each user to freely set a content.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a program, an information processing apparatus, and an information processing method.

In recent years, products have been developed using an augmented reality (AR) system. AR
In the system, an application program for recognizing an AR marker is installed in an information device having a camera and a display unit such as a smartphone, a tablet terminal, a portable game machine, and the like. When the subject with the AR marker is photographed using such an information device, the information device recognizes the AR marker, and the content corresponding to the recognized AR marker is superimposed on the photographed image and displayed. Thus, the user can view an image in which the content is superimposed on the subject to be photographed on the display unit. Patent Document 1 discloses a system for photographing a three-dimensional object and superimposing additional information such as an image or a character on the three-dimensional object in the photographed image.

JP, 2013-92407, A

  However, in the prior art, there is a problem that the content displayed superimposed on the photographed image of the subject is only provided unilaterally from the vendor.

  The present invention has been made under such circumstances, and an object of the present invention is to provide a program and the like that allow each user to freely set contents.

  In the information processing program according to the present invention, a plurality of types of spheres having markers on the surface exist in a computer, a sphere ID identifying the spheres is specified, and selection of content for overlapping and displaying on a photographed image of the spheres , And acquires position information corresponding to the content, and executes processing for transmitting the received content and the acquired position information in association with the specified sphere ID and user ID.

  According to the present invention, it is possible to provide a program or the like which allows each user to set contents freely.

It is a schematic diagram which shows the structural example of AR system. FIG. 2 is a block diagram showing an example of the configuration of a server according to the first embodiment. 5 is an explanatory diagram showing an example of a record layout of the user DB of the first embodiment. FIG. It is explanatory drawing which shows an example of the record layout of display image DB. It is explanatory drawing which shows an example of the record layout of explanatory information DB. It is explanatory drawing which shows an example of the record layout of open information DB. It is a block diagram which shows the structural example of a terminal. FIG. 6 is an explanatory diagram for explaining the operation of the AR system of the first embodiment. It is an image figure showing an example of acquisition of position information. It is an image figure showing an example of acquisition of user ID by a terminal. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 1 transmitting a display image. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 1 receiving a display image. FIG. 6 is an explanatory view showing the explanatory information of the terminal of the embodiment 1; It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 1 transmitting explanatory information. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 1 receiving explanatory information. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 1 transmitting publication | presentation information. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 1 receiving a display image based on open information. 5 is a flowchart showing a processing procedure of a subroutine of open range determination according to the first embodiment. FIG. 14 is an explanatory view for explaining an operation of order display of the terminal of the embodiment 2; It is a flowchart which shows the sort process procedure in order of the date at the time of the terminal of Embodiment 2 receiving several display images. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 2 performing switching display. It is a flowchart which shows the sort process procedure of the terminal of Embodiment 2. FIG. FIG. 14 is an explanatory diagram for explaining an operation including trajectory information of the terminal of the second embodiment. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 2 performing the switching display containing locus | trajectory information. FIG. 18 is a block diagram showing an example of the configuration of a server according to a third embodiment. FIG. 18 is an explanatory drawing showing an example of the record layout of the user DB of the third embodiment. It is explanatory drawing which shows an example of the record layout of music DB. FIG. 18 is an explanatory diagram for explaining the operation of the AR system of the third embodiment. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 3 transmitting a display image and a music. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 3 receiving a display image and music. It is a flowchart which shows the process sequence of the game of Embodiment 3. FIG. FIG. 18 is a block diagram illustrating an exemplary configuration of a server according to a fourth embodiment. FIG. 18 is an explanatory view showing an example of a record layout of the user DB of the fourth embodiment. It is an explanatory view showing an example of a record layout of sphere DB. FIG. 18 is an explanatory diagram for explaining the operation of the AR system of the fourth embodiment. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 4 transmitting a display image based on bulb | ball ID. It is a flowchart which shows the process sequence at the time of the terminal of Embodiment 4 receiving a display image based on bulb | ball ID.

  Hereinafter, the present invention will be described in detail based on the drawings showing the embodiments.

(Embodiment 1)
FIG. 1 is a schematic view showing a configuration example of an AR system. The AR system of the present embodiment includes an information processing apparatus 1, an information processing terminal 2, and an earth ball 3 which is a sphere having an AR marker (marker) on its surface. The information processing apparatus 1 and the information processing terminal 2 transmit and receive information via the network N such as the Internet.

  The information processing apparatus 1 is an information processing apparatus that performs various information processing, information storage, and information transmission and reception, and is, for example, a server apparatus, a personal computer, or the like. In the present embodiment, the information processing apparatus 1 is assumed to be a server apparatus, and hereinafter the server 1 will be read for simplicity.

  The information processing terminal 2 is a terminal device of each user, and is, for example, an information processing device such as a smartphone, a mobile phone, a tablet, a personal computer terminal, or a game machine. Hereinafter, the information processing terminal 2 is replaced with the terminal 2 for the sake of simplicity.

  The earth ball 3 is taken as a subject, and the earth ball 3 is photographed by the terminal 2 and used. The earth ball 3 is, for example, a ball made of PVC (polyvinyl chloride) that becomes a sphere by being filled with air, but is not limited thereto. A map on the earth is printed on the surface of the earth ball 3 and AR markers are attached to appropriate positions in the map. For example, in the example shown in FIG. 1, each of the sea and the continent in the map is given a name, and each country and the city are given a country name and a city name. These characters constitute an AR marker. Moreover, although omitted in FIG. 1, the border lines of each country are included in the map, and the border lines also constitute an AR marker. Further, rivers, plains, mountain ranges, etc. in the map are color-coded with predetermined colors for each topography, and each color also constitutes an AR marker. Furthermore, the seas in the map are marked with predetermined marks M1 to M4 (for example, marks M1 and M2 of ships, and marks M3 and M4 of animals such as turtles or gulls) to identify the respective seas, sea areas, etc There is. In addition, predetermined marks M5 to M7 are attached to the continents in the map in order to identify the respective continents, regions, and the like. These marks M1 to M7 also constitute an AR marker.

  FIG. 2 is a block diagram showing a configuration example of the server 1 of the first embodiment. The server 1 includes a control unit 11, a storage unit 12, a communication unit 13, a large capacity storage unit 14, and a reading unit 15. Each configuration is connected by a bus B.

  The control unit 11 includes an arithmetic processing unit such as a central processing unit (CPU) or a micro-processing unit (MPU), and reads and executes the control program 1P stored in the storage unit 12 to execute various types of processing related to the server 1. Perform information processing, control processing, etc. Although the control unit 11 is described as a single processor in FIG. 2, it may be a multiprocessor. The storage unit 12 includes memory elements such as a random access memory (RAM) and a read only memory (ROM), and stores a control program 1 P or data required for the control unit 11 to execute a process. In addition, the storage unit 12 temporarily stores data and the like necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 includes a processing circuit and the like for performing processing related to communication, and transmits and receives information with the terminal 2 and the like through the network N.

  The large capacity storage unit 14 is a large capacity storage device including, for example, a hard disk. The large capacity storage unit 14 stores a user DB 141, a display image DB 142, an explanatory information DB 143, and a public information DB 144. The user DB 141 stores user information on each user, display image ID and position information. The display image DB 142 stores various display images to be superimposed and displayed on the photographed image of the earth ball 3. The explanatory information DB 143 stores explanatory information in the form of voice or text for the display image. The public information DB 144 stores public information on display images.

  In the present embodiment, the storage unit 12 and the large capacity storage unit 14 may be configured as an integrated storage device. Further, the large capacity storage unit 14 may be configured by a plurality of storage devices. Further, the mass storage unit 14 may be an external storage device connected to the server 1.

  The reading unit 15 reads a portable storage medium 1 a including a CD (Compact Disc) -ROM or a DVD (Digital Versatile Disc) -ROM. The control unit 11 may read the control program 1 P from the portable storage medium 1 a via the reading unit 15 and store the control program 1 P in the large-capacity storage unit 14. The control unit 11 may download the control program 1P from another computer via the network N or the like and store the control program 1P in the large-capacity storage unit 14. Furthermore, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

  In the present embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input unit that receives an operation input, and a display unit that displays information related to the server 1. In addition, although the server 1 is demonstrated as a single-piece thing by this embodiment, it does not restrict to this. For example, they may be physically configured by a plurality of servers, or may be configured by a plurality of virtual machines.

  FIG. 3 is an explanatory view showing an example of the record layout of the user DB 141 of the first embodiment. The user DB 141 includes a user ID column, a user rank column, an image ID column, and a position (latitude, longitude) column. The user ID column stores the ID of the uniquely identified user in order to identify each user. The user rank column stores rank information of user accounts. For example, the rank of the user is provided with three ranks of “regular”, “silver”, and “gold”. A point system for the user's rank is introduced, points are added according to the number of purchases, purchase price, etc., and when the points reach a certain level, ranks up in the order of "regular", "silver", "gold". The image ID column stores an ID of a display image to be displayed superimposed on the photographed image of the earth ball 3. The position (latitude, longitude) column stores position information associated with the display image. Position information includes latitude and longitude. The position information may be a zip code, an address or a facility name.

  FIG. 4 is an explanatory view showing an example of the record layout of the display image DB 142. As shown in FIG. The display image DB 142 includes an image ID column, a type column, an image column, a description information ID column, and a public information ID column. The image ID column stores the ID of the display image uniquely identified in order to identify the display image. The type column stores the type of display image. Types of display images include still images, stereoscopic images, and moving images. The image sequence stores data of a display image corresponding to the image ID. The explanation information ID column stores the ID of the explanation information associated with the display image. The public information ID column stores the ID of public information associated with the display image. The disclosure information is not limited to setting the disclosure information for each display image, and the disclosure information may be set for the entire content. For example, all display images of the user may be disclosed to the friend of the user.

  FIG. 5 is an explanatory view showing an example of the record layout of the explanation information DB 143. As shown in FIG. The explanatory information DB 143 includes an explanatory information ID column, a type column, and an explanatory information column. The explanation information ID column stores the ID of the uniquely specified explanation information in order to identify the explanation information associated with the display image. The type column stores the type of explanatory information. The type of explanatory information includes voice information or text information. The text information may be a simple character or may be a character including a URL (Uniform Resource Locator) or the like. The explanation information string stores voice data or text data corresponding to the explanation information ID.

  FIG. 6 is an explanatory view showing an example of the record layout of the public information DB 144. As shown in FIG. The public information DB 144 includes a public information ID column, a type column, and a public range column. The public information ID column stores the ID of the uniquely identified public information in order to identify the public information. The type column is the type of publication. In the present embodiment, the types of publication include publication, non-publication and partial publication. The disclosure range column stores information on the disclosure range corresponding to the disclosure information ID. In the present embodiment, the disclosure types are classified into "disclosure", "disclosure", and "partial disclosure". If "Public" is set as the public type, "All" is set in the Public range column. When "disclosure" is set as the disclosure type, "-" is set in the disclosure range column. When "partial release" is set for the release type, the rank of the user is set in the release range column. The rank of the user is, for example, the same rank as the user rank of the user DB 141, and there are three ranks of "regular", "silver", and "gold". By setting the disclosure range in accordance with the rank of the user, it becomes possible to disclose the pre-sales information, the sales promotion item information, the advantageous bonus information, and the like only to a specific user.

  Note that the disclosure range is not limited to the above configuration, and the disclosure range may be set based on, for example, the region of the world. Specifically, it may be divided into six according to geographical division: Africa, Asia, Europe, North America, Central and South America, Oceania (Pacific State). Further, for example, with respect to Africa, East Africa, Central Africa, North Africa, South Africa, West Africa may be divided. By setting the release range according to the area, it is possible to provide various services such as a presales area, a mail order prohibited area or a sale end area. Furthermore, the user may freely set the disclosure range. For example, the information may be disclosed only to friends specified by the user.

  FIG. 7 is a block diagram showing a configuration example of the terminal 2. The terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, an album 24, an input unit 25, a display unit 26, a photographing unit 27, a clock unit 28, a microphone 29, and a speaker 20. Each configuration is connected by a bus B.

  The control unit 21 includes an arithmetic processing unit such as a CPU and an MPU, and reads and executes the control program 2P stored in the storage unit 22 to perform various information processing, control processing, and the like related to the terminal 2. Although the control unit 21 is described as a single processor in FIG. 7, it may be a multiprocessor. The storage unit 22 includes memory elements such as a RAM and a ROM, and stores a control program 2P or data required for the control unit 21 to execute a process. The storage unit 22 also temporarily stores data and the like necessary for the control unit 21 to execute arithmetic processing. The communication unit 23 includes a processing circuit and the like for performing processing related to communication, and transmits and receives information to and from the server 1 via the network N. The album 24 includes a display image and music. The input unit 25 is a keyboard or a mouse. The display unit 26 is a liquid crystal display device. Further, the input unit 25 may be a touch panel integrated with the display unit 26.

  The photographing unit 27 is, for example, a photographing device such as a charge coupled device (CCD) camera or a complementary metal oxide semiconductor (CMOS) camera. The control unit 21 can recognize marker information or a QR code (registered trademark) or the like on the surface of the sphere in the photographed image of the ground ball 3 by the photographing unit 27. The photographing unit 27 may not be built in the terminal 2 and may be directly connected to the terminal 2 outside so as to be capable of photographing.

  The clock unit 28 is a circuit that counts time or elapsed time, etc., and provides the control unit 21 with a time measurement result in accordance with a request from the control unit 21. The clock unit 28 also provides a timer function. The timer function is a function of notifying the control unit 21 of that when a preset time has passed since the start instruction was given. Alternatively, the timer function is a function of responding to an inquiry from the control unit 21 whether or not a preset time has passed since the start instruction was issued.

  The microphone 29 is a device that converts sound into an electrical signal. The speaker 20 is a device that converts an electrical signal into a sound. The microphone 29 and the speaker 20 may be a headset connected to the terminal 2 by a short distance wireless communication method such as Bluetooth (registered trademark).

  FIG. 8 is an explanatory diagram for explaining the operation of the AR system of the first embodiment. In this embodiment, a coffee plantation sales system is described as an example of the embodiment of the AR system. FIG. 8A is an explanatory view showing the entire operation. The control unit 21 of the terminal 2 of the coffee plantation receives the selection of the display image to be superimposed and displayed on the photographed image of the earth ball 3. The display image includes a still image, a stereoscopic image, and a moving image. The display image may be acquired from the album 24 of the terminal 2 of the coffee plantation, or may be photographed using the photographing unit 27 of the terminal 2 of the coffee plantation. Furthermore, you may receive from the external device which is not shown in figure via the communication part 23 of the terminal 2 of a coffee plantation. For example, the received display image is a photograph of a coffee flower blooming in a farm, a crop photograph of a coffee seed, or a shipping photograph of coffee beans.

  The control unit 21 of the terminal 2 of the coffee plantation acquires position information on the received display image. The acquisition process of the position information may be acquired on the server 1 side. In this case, latitude and longitude information attached to the image may be acquired. Regarding acquisition of position information, when an image is photographed using a digital camera with a GPS function or a smartphone, the position information may be extracted from an image photographed by adding latitude and longitude information by GPS. Further, position information may be manually input by the input unit 25 of the terminal 2 of the coffee plantation.

  Also, location information may be acquired by a Google (registered trademark) map (Google (registered trademark) Map) service. For example, using the Google Maps API (Application Programming Interface), the map is embedded and displayed in the application of the terminal 2 based on the data of Google Map. The terminal 2 receives the location information of the screen tapped on the map or the keyword information of the input address, and transmits the received information to the Google Map server via the Google Maps API. The Google Map server acquires position information such as latitude and longitude corresponding to the information transmitted from the terminal 2, and transmits the position information to the terminal 2. The communication unit 23 of the terminal 2 receives the position information transmitted from the Google Map server.

  Furthermore, position information may be extracted based on marker information in the photographed image of the earth ball 3. FIG. 9 is an image diagram showing an example of acquisition of position information. It is an example of acquisition of position information based on AR marker information in a photography picture of earth ball 3.

  FIG. 9A is an image diagram showing an example of acquisition of position information by a drag and drop operation. The control unit 21 of the terminal 2 moves the selected display image to the set destination in the photographed image of the earth ball 3 by the drag and drop operation. The control unit 21 of the terminal 2 recognizes AR marker information in accordance with the set destination in the captured image of the earth ball 3 and extracts position information based on the recognized AR marker information.

  FIG. 9B is an image diagram showing an example of acquisition of position information by menu operation. The control unit 21 of the terminal 2 recognizes the AR marker information according to the setting destination in the photographed image by the tap operation of the setting destination in the photographed image of the earth ball 3. The control unit 21 of the terminal 2 extracts position information based on the recognized AR marker information, and generates a menu for position selection based on the extracted position information. Via the display unit 26 of the terminal 2, the control unit 21 of the terminal 2 displays the generated menu for position selection superimposed on the photographed image of the ground ball 3. For example, the control unit 21 of the terminal 2 receives a tap operation in the “South Africa” area in the captured image, and recognizes an AR marker located at or near the tapped point. The control unit 21 of the terminal 2 acquires classification information of the “South Africa” area based on the recognized AR marker information, and generates a menu based on the acquired classification information of the “South Africa” area. For example, the items of the generated menu are "Swaziland", "Namibia", "Botswana", "South Africa" and "Lesotho". The control unit 21 of the terminal 2 superimposes the generated menu on the photographed image of the earth ball 3 and displays the menu via the display unit 26 of the terminal 2. The control unit 21 of the terminal 2 may receive a tap operation of an item of the menu, and may acquire position information corresponding to the item of the received menu. The display format is not limited to the menu, and may be a dialog box or a combo box.

  The control unit 21 of the terminal 2 of the coffee plantation associates the selected display image and the position information associated with the display image with the user ID of the coffee plantation. The communication unit 23 of the terminal 2 of the coffee plantation transmits the user ID of the coffee plantation, the display image, and the position information associated with the display image to the server 1. In the present embodiment, the control unit 21 of the terminal 2 first receives the selection of the display image and then receives the acquisition of the position information, but the present invention is not limited to this. For example, control part 21 of terminal 2 may receive acquisition of position information first, and may receive selection of a display picture corresponding to the received position information.

  The communication unit 13 of the server 1 receives the user ID of the coffee plantation transmitted from the terminal 2 of the coffee plantation, the display image to be displayed superimposed on the photographed image of the earth ball 3 and the position information associated with the display image . The control unit 11 of the server 1 stores the received display image and the position information associated with the display image in the large-capacity storage unit 14 in association with the received user ID of the coffee plantation. Specifically, the control unit 11 of the server 1 assigns a unique display image ID to the display image, and stores the coffee farm user ID, the display image ID, and the position information in the user DB 141 as one record. The control unit 11 of the server 1 stores the display image ID and the data of the display image in the display image DB 142 as one record.

  Subsequently, a process of displaying a transmitted display image by the user of the coffee plantation will be described. In addition to the coffee plantation, the type of content displayed superimposed on the photographed image of the earth ball 3 may include cruise operation, a treasure hunt game, airplane operation, world music and world heritage. In the present embodiment, an example using contents of a coffee plantation will be described. The communication unit 23 of the customer terminal 2 transmits the user ID of the coffee plantation to the server 1 in order to display the display image corresponding to the user ID of the coffee plantation superimposed on the photographed image of the earth ball 3. FIG. 10 is an image diagram showing an example of acquisition of a user ID by the terminal 2. Specifically, the control unit 21 of the customer terminal 2 acquires a photographed image of the ground ball 3 photographed by the photographing unit 27 of the customer terminal 2. The control unit 21 of the terminal 2 of the customer receives the tap operation of “coffee plantation” by the input unit 25 of the terminal 2 of the customer, and acquires the user ID of the coffee plantation. The communication unit 23 of the terminal 2 of the customer transmits the acquired user ID of the coffee plantation to the server 1. The user ID may be acquired not only by the above method but also by manual input, for example. Although the example which transmits the user ID of a coffee plantation was shown in this embodiment, it does not restrict to this. It is also possible to transmit information such as a URL or a content ID that can specify the type of content to be displayed superimposed on the photographed image of the earth ball 3.

  The communication unit 13 of the server 1 receives the user ID of the coffee plantation transmitted from the terminal 2 of the customer. The control unit 11 of the server 1 searches the user DB 141 of the large capacity storage unit 14 based on the received user ID of the coffee plantation, and acquires the display image ID and the position information of the display image corresponding to the user ID of the coffee plantation. . The control unit 11 acquires a display image corresponding to the acquired display image ID from the display image DB 142. The communication unit 13 of the server 1 transmits the acquired display image and the position information associated with the display image to the terminal 2 of the customer.

  The communication unit 23 of the terminal 2 of the customer receives the display image corresponding to the coffee farm user ID transmitted from the server 1 and the position information associated with the display image. As shown in FIG. 8B, the control unit 21 of the terminal 2 of the customer specifies the ball position in the photographed image of the ground ball 3 corresponding to the received position information. Through the display unit 26 of the terminal 2 of the customer, the control unit 21 of the terminal 2 of the customer displays the thumbnail of the received display image superimposed on the specified spherical position. Further, when the input unit 25 of the customer terminal 2 accepts a tap operation on the display image, the control unit 21 of the customer terminal 2 performs enlargement processing of the display image. The control unit 21 of the customer terminal 2 can display the enlarged display image so as to be superimposed on the photographed image of the ground ball 3 through the display unit 26 of the customer terminal 2. In addition, you may cover and display not only the above-mentioned display format but the display image received on the whole screen of the display part 26 of the terminal 2 of a customer. If the received display image is a moving image, playback of the moving image may be started by a tap operation.

  In the present embodiment, although an example in which the display image of the user of the coffee plantation is received and displayed on the terminal 2 of the customer is described, this is not limited. The user may receive and display the transmitted display image. For example, by transmitting the user ID of the coffee plantation to the server 1, the terminal 2 of the coffee plantation receives from the server 1 a display image corresponding to the user ID of the user and position information associated with the display image. The control unit 21 of the terminal 2 of the coffee plantation specifies the position of the sphere in the photographed image of the earth ball 3 corresponding to the received position information. Through the display unit 26 of the terminal 2 of the coffee plantation, the control unit 21 of the terminal 2 of the coffee plantation may superimpose and display the received display image on the specified sphere position.

  FIG. 11 is a flowchart showing a processing procedure when the terminal 2 of the first embodiment transmits a display image. The control unit 21 of the terminal 2 receives an input of authentication information such as a user ID through the input unit 25 of the terminal 2. The communication unit 23 of the terminal 2 transmits the received authentication information to the server 1 (step S201). The control unit 11 of the server 1 performs an authentication process on authentication information such as a user ID received by the communication unit 13 of the server 1 (step S101). The control unit 11 of the server 1 determines whether user authentication has succeeded (step S102). If it is determined that the user authentication has succeeded (YES in step S102), the communication unit 13 of the server 1 transmits, to the terminal 2, an indication of success. If it is determined that the user authentication has failed (NO in step S102), the control unit 11 of the server 1 ends the process. When the communication unit 23 of the terminal 2 receives that the user authentication is successful, the control unit 21 of the terminal 2 receives a selection of a display image to be displayed superimposed on the photographed image of the ground ball 3 (step S202) . The control unit 21 of the terminal 2 acquires position information corresponding to the selected display image (step S203), and associates the selected display image and the acquired position information with the user ID. The communication unit 23 of the terminal 2 transmits the user ID, the display image, and the position information to the server 1 (step S204). The communication unit 13 of the server 1 receives the user ID, the display image, and the position information associated with the display image transmitted from the terminal 2 (step S103). The control unit 11 of the server 1 associates the received display image and the position information associated with the display image with the user ID, and stores the user ID, the display image and the position information in the large capacity storage unit 14 (step S104). In addition, although the example which transmits user ID with a display image was shown in step S204, it does not restrict to this. Association with the display image may be performed based on the user ID received in step S101.

  FIG. 12 is a flowchart showing a processing procedure when the terminal 2 of the first embodiment receives a display image. About the content which overlaps with FIG. 11, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 21 of the terminal 2 acquires a photographed image of the ground ball 3 photographed by the photographing unit 27 of the terminal 2 (step S205). The communication unit 23 of the terminal 2 transmits the user ID corresponding to the display image to be displayed superimposed on the photographed image of the earth ball 3 to the server 1 (step S206). The communication unit 13 of the server 1 receives the user ID transmitted from the terminal 2 (step S105). The control unit 11 of the server 1 compares the received user ID with the user ID stored in the user DB 141 of the large capacity storage unit 14, and acquires the display image ID and the position information corresponding to the same user ID (step S106). ). The control unit 11 of the server 1 acquires a display image corresponding to the acquired display image ID from the display image DB 142 of the large capacity storage unit 14 (step S107). The communication unit 13 of the server 1 transmits the acquired display image and position information associated with the display image to the terminal 2 (step S108). The communication unit 23 of the terminal 2 receives the display image and the position information associated with the display image transmitted from the server 1 (step S207). If there are a plurality of display images, they may be received at one time or may be divided into a plurality of times and received. The control unit 21 of the terminal 2 specifies the ball position in the photographed image of the earth ball 3 corresponding to the received position information (step S208). Via the display unit 26 of the terminal 2, the control unit 21 of the terminal 2 displays the received display image so as to overlap the position of the sphere in the specified captured image (step S209).

  Subsequently, a process of setting explanation information for a display image to be displayed superimposed on the photographed image of the earth ball 3 will be described. The explanatory information includes voice information or text information. The control unit 21 of the terminal 2 may receive voice information via the microphone 29 of the terminal 2. The communication unit 23 of the terminal 2 may receive voice information from an external device. The control unit 21 of the terminal 2 receives text information via the input unit 25 of the terminal 2. The explanatory information may be set simultaneously with the display image, or may be set separately from the display image. When setting separately from the display image, the display image ID of the display image corresponding to the user ID is acquired, and the description information is set for the acquired display image ID.

  In the present embodiment, a process when the terminal 2 of the coffee plantation sets the explanation information for the transmitted display image will be described. The communication unit 23 of the terminal 2 of the coffee plantation transmits the user ID of the coffee plantation to the server 1. The control unit 11 of the server 1 acquires the display image ID of the display image corresponding to the user ID of the coffee plantation received by the communication unit 13 of the server 1. The communication unit 13 of the server 1 transmits the acquired display image ID to the terminal 2 of the coffee plantation.

  The communication unit 23 of the terminal 2 of the coffee plantation receives the display image ID transmitted from the server 1. The control unit 21 of the terminal 2 of the coffee plantation receives an input of the explanation information, and associates the received explanation information with the user ID of the coffee plantation and the display image ID. The communication unit 23 of the terminal 2 of the coffee plantation transmits the user ID of the coffee plantation, the display image ID and the explanation information to the server 1. The communication unit 13 of the server 1 receives the user ID of the coffee plantation, the display image ID, and the explanation information transmitted from the terminal 2 of the coffee plantation. The control unit 11 of the server 1 stores the received explanation information in the large-capacity storage unit 14 in association with the user ID of the coffee plantation and the display image ID. Specifically, the control unit 11 of the server 1 assigns a unique explanation information ID to the explanation information, and the explanation information ID is stored in the user DB 141 corresponding to the received user ID of the coffee plantation and the display image ID. Remember. The control unit 11 of the server 1 stores the data of the explanation information ID and the explanation information as one record in the explanation information DB 143.

  As a result, it is possible to superimpose the explanatory information associated with the display image on the photographed image of the earth ball 3 on the display unit 26 of the terminal 2 of the coffee plantation or the terminal 2 of the customer. For example, the communication unit 23 of the terminal 2 of the customer transmits the user ID of the coffee plantation to the server 1. The control unit 11 of the server 1 acquires a display image corresponding to the user ID of the coffee plantation received by the communication unit 13 of the server 1 and position information and explanation information associated with the display image. The communication unit 13 of the server 1 transmits the acquired display image and the position information and the explanation information associated with the display image to the terminal 2 of the customer.

  FIG. 13 is an explanatory view showing the explanatory information of the terminal 2 of the first embodiment. The communication unit 23 of the terminal 2 of the customer receives the display image transmitted from the server 1 and the position information and the explanation information associated with the display image.

  FIG. 13A is a display image diagram of explanatory information in text format. The control unit 21 of the terminal 2 of the customer specifies the ball position in the photographed image of the earth ball 3 corresponding to the received position information. Via the display unit 26 of the customer terminal 2, the control unit 21 of the customer terminal 2 superimposes and displays the display image on the specified spherical position. Further, when the input unit 25 of the customer terminal 2 accepts a tap operation on the display image, the control unit 21 of the customer terminal 2 performs enlargement processing of the display image. The control unit 21 of the customer terminal 2 can display the enlarged display image and the explanation information associated with the display image superimposed on the photographed image of the earth ball 3 through the display unit 26 of the customer terminal 2 It becomes. Note that the display image may be displayed on the entire screen of the display unit 26 of the customer terminal 2 without being limited to the display format described above, and may be displayed covering the display image and the explanation information associated with the display image.

  FIG. 13B is an image diagram of the explanatory information on the audio format. The control unit 21 of the terminal 2 of the customer specifies the ball position in the photographed image of the earth ball 3 corresponding to the received position information. Via the display unit 26 of the customer terminal 2, the control unit 21 of the customer terminal 2 superimposes and displays the display image on the specified spherical position. In addition, when a tap operation of a display image is received by the input unit 25 of the customer terminal 2, the control unit 21 of the customer terminal 2 performs enlargement processing of the display image, and generates an explanatory information reproduction button BTN1. The control unit 21 of the customer terminal 2 superimposes and displays the enlarged display image and the generated explanation information reproduction button BTN 1 on the photographed image of the earth ball 3 via the display unit 26 of the customer terminal 2. When the control unit 21 of the customer terminal 2 receives the tap operation of the explanation information reproduction button BTN1 by the input unit 25 of the customer terminal 2, the control unit 21 reproduces the voice information received through the speaker 20 of the customer terminal 2 Is possible. Note that the reproduction of audio information is not limited to the above-described format. For example, when the control unit 21 of the terminal 2 of the customer displays the display image superimposed on the photographed image of the earth ball 3 without generating the explanation information reproduction button BTN 1, the control unit 21 of the terminal 2 via the speaker 20 of the customer 2 Audio information may be reproduced automatically.

  FIG. 14 is a flowchart showing a processing procedure when the terminal 2 of the first embodiment transmits the explanation information. About the content which overlaps with FIG. 11, FIG. 12, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 11 of the server 1 acquires a display image ID corresponding to the user ID transmitted from the terminal 2 from the user DB 141 of the large capacity storage unit 14 (step S109). The communication unit 13 of the server 1 transmits the acquired display image ID to the terminal 2 (step S110). The communication unit 23 of the terminal 2 receives the display image ID transmitted from the server 1 (step S210). The control unit 21 of the terminal 2 sets the explanatory information including the audio information or the text information in association with the user ID and the received display image ID (step S211). The communication unit 23 of the terminal 2 transmits the user ID, the display image ID and the explanation information to the server 1 (step S212). The communication unit 13 of the server 1 receives the user ID, the display image ID, and the explanation information transmitted from the terminal 2 (step S111). The control unit 11 of the server 1 stores the received explanation information in the explanation information DB 143 of the large capacity storage unit 14 in association with the received user ID and display image ID (step S112).

  FIG. 15 is a flowchart showing a processing procedure when the terminal 2 of the first embodiment receives the explanatory information. About the content which overlaps with FIG. 12, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 11 of the server 1 acquires the display image ID and the position information corresponding to the received user ID from the user DB 141 of the large capacity storage unit 14 (step S113). The control unit 11 acquires the display image and the explanation information ID corresponding to the acquired display image ID from the display image DB 142 of the large capacity storage unit 14 (step S114). The control unit 11 acquires the explanation information corresponding to the acquired explanation information ID from the explanation information DB 143 of the large capacity storage unit 14 (step S115). The communication unit 23 of the server 1 transmits the acquired display image and the position information and the explanation information associated with the display image to the terminal 2 (step S116). The communication unit 23 of the terminal 2 receives the display image transmitted from the server 1 and the position information and the explanation information associated with the display image (step S213). The control unit 21 of the terminal 2 specifies the ball position in the photographed image of the earth ball 3 corresponding to the received position information. Via the display unit 26 of the terminal 2, the control unit 21 of the terminal 2 displays the received display image and the explanation information associated with the display image superimposed on the specified spherical position (step S214).

  Subsequently, setting processing of public information with respect to a display image to be displayed superimposed on the photographed image of the earth ball 3 will be described. Public information includes public, private and partial public. In the case of disclosure, the displayed image is disclosed to everyone without restriction. In the case of non-disclosure, the display image is disclosed only to the user who owns the display image, and is not disclosed to other users. In the case of partial release, make it available to users who satisfy the conditions within the release range. In the present embodiment, the disclosure range that can be set to be partially published is the same rank as the user rank of the user DB 141, and there are three of “regular”, “silver”, and “gold”. In summary, it is made available to users who have been set to a specified rank or higher. Specifically, when “regular” is set to partial disclosure, the display image is disclosed to users belonging to so-called “regular”, “silver” and “gold” ranks. When "Silver" is set for partial disclosure, the display image is disclosed to users belonging to so-called "Silver" and "Gold" ranks. When "gold" is set for partial release, the display image is made available only to users belonging to the "gold" rank. In addition, regarding the public information, the public information may be set when the display image is selected, and may be transmitted to the server 1 simultaneously with the display image, or the public information is added to the display image already transmitted. You may send it. The control unit 11 of the server 1 stores the public information transmitted from the terminal 2 in the public information DB 144 of the large capacity storage unit 14.

  FIG. 16 is a flowchart showing a processing procedure when the terminal 2 of the first embodiment transmits the public information. About the content which overlaps with FIG. 14, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 21 of the terminal 2 sets the public information in association with the user ID and the received display image ID (step S215). The communication unit 23 of the terminal 2 transmits the user ID, the display image ID, and the disclosure information to the server 1 (step S216). The communication unit 13 of the server 1 receives the user ID, the display image ID, and the disclosure information transmitted from the terminal 2 (step S117). The control unit 11 of the server 1 stores the received public information in the public information DB 144 of the mass storage unit 14 in association with the received user ID and display image ID (step S118).

  FIG. 17 is a flowchart showing a processing procedure when the terminal 2 of the first embodiment receives a display image based on public information. About the content which overlaps with FIG. 12, FIG. 14, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 11 of the server 1 acquires the public information ID corresponding to the acquired display image ID from the display image DB 142 of the large capacity storage unit 14 (step S119). The control unit 11 determines the disclosure range of the display image based on the acquired disclosure information ID (step S120). The subroutine of open range determination will be described later. The control unit 11 of the server 1 determines whether the release range determination result is release (step S121). When it is determined that the open range result is open (YES in step S121), the control unit 11 of the server 1 acquires the display image and the position information associated with the display image. When it is determined that the disclosure range result is non-disclosure (NO in step S121), the control unit 11 of the server 1 ends the process.

  FIG. 18 is a flowchart showing the processing procedure of the open range determination subroutine of the first embodiment. The control unit 11 of the server 1 determines whether the received user ID and the logged-in user ID are the same (step S1801). If it is determined that the received user ID and the logged-in user ID are the same (YES in step S1801), the user has no display restriction on the transmitted display image, and the control unit 11 discloses the determination result. The setting is made (step S1809). If it is determined that the received user ID and the logged-in user ID are not the same (NO in step S1801), the control unit 11 acquires public information corresponding to the public information ID from the public information DB 144 of the large capacity storage unit 14 ( Step S1802). The control unit 11 determines whether the public information is private (step S1803). If it is determined that the public information is non-disclosure (YES in step S1803), the control unit 11 sets non-disclosure as the determination result (step S1810). If it is determined that the public information is other than private (NO in step S1803), the control unit 11 determines whether the public information is public (step S1804). If it is determined that the public information is public (YES in step S1804), the control unit 11 sets public as the determination result (step S1809). If it is determined that the public information is other than public (NO in step S1804), the control unit 11 determines whether part of the public information is public (step S1805). If it is determined that the public information is partially public (YES in step S1805), control unit 11 acquires the rank of the user corresponding to the logged-in user ID by user DB 141 of large-capacity storage unit 14 of server 1 (Step S1806). The control unit 11 compares the acquired rank of the user with the public information (step S1807). The control unit 11 determines whether the rank of the user is equal to or greater than the public information (step S1808). If it is determined that the rank of the user is equal to or greater than the public information (YES in step S1808), the control unit 11 sets public as the determination result (step S1809). If it is determined that the rank of the user is low according to the public information (NO in step S1808), the control unit 11 sets the determination result as non-disclosure (step S1810). If it is determined that the disclosure information is other than partial disclosure (NO in step S1805), the control unit 11 sets the determination result to non-disclosure (step S1810). After the processing of step S1809 and step S1810, the control unit 11 returns the disclosure range determination result (step S1811).

  According to the present embodiment, it is possible to extract the user's activity in response to the user setting the display image freely.

  According to the present embodiment, it is possible to display the explanatory information associated with the display image superimposed on the photographed image of the sphere.

  According to the present embodiment, it is possible to share the display image with the user targeted for publication. In addition, the security level can be enhanced by setting public, private and partial public. Furthermore, it becomes possible to set the open range appropriately and to realize a versatile AR system.

Second Embodiment
In the present embodiment, a cruise operation system will be described as an example of an embodiment of the AR system. The control unit 21 of the terminal 2 of the travel agency receives a selection of display images of a plurality of tourist spots to be superimposed and displayed on the photographed image of the earth ball 3 based on the cruise operation route. The display image of the tourist spot may be photographed by the photographing unit 27 of the terminal 2 of the travel company, or may be selected from the album 24 of the terminal 2 of the travel company, and furthermore, the communication unit of the terminal 2 of the travel company It may be received from an external device (not shown) via 23. The control unit 21 of the travel company terminal 2 acquires position information of each of the received plurality of display images, and associates the received plurality of display images and the acquired position information with the user ID. The acquisition process of the position information is the same as the acquisition process of the position information of the first embodiment, and thus the description thereof is omitted. The communication unit 23 of the travel company terminal 2 transmits the user ID, the plurality of received display images, and the position information associated with the display images to the server 1. The server 1 stores the user ID, the plurality of display images, and the position information associated with the display images transmitted from the terminal 2 of the travel company in the large-capacity storage unit 14.

  As a result, it is possible to superimpose a display image on cruise operation information provided by a travel company on the photographed image of the earth ball 3 through the display unit 26 of the terminal 2 of the traveler. The control unit 21 of the traveler terminal 2 acquires a photographed image of the earth ball 3 photographed by the photographing unit 27 of the traveler terminal 2. The communication unit 23 of the traveler terminal 2 transmits the user ID of the travel company to the server 1.

  The communication unit 13 of the server 1 receives the user ID of the travel company transmitted from the terminal 2 of the traveler. The control unit 11 of the server 1 acquires a plurality of display images corresponding to the user ID of the travel company and position information associated with the display images by the user DB 141 of the large-capacity storage unit 14. The communication unit 13 of the server 1 transmits the plurality of acquired display images and position information associated with the display images to the terminal 2 of the traveler.

  The communication unit 23 of the traveler's terminal 2 receives a plurality of display images regarding the cruise operation information of the travel company transmitted from the server 1 and position information associated with the display images. It becomes possible to switch and display the photographed image of the earth ball 3 in order of date and time or in the order of display according to the plurality of received display images. The order of date and time is the old or new date order. The order of the display order may be set by manual input or may be set by another method. For example, the control unit 21 may set the display order at random for a plurality of display images.

  FIG. 19 is an explanatory diagram for explaining an operation of order display of the terminal 2 of the second embodiment. Below, the example displayed on the order of date is mentioned and demonstrated. For example, P 1, P 2, P 3 and P 4 shown in the figure represent tourist spots on the cruise route. The communication unit 23 of the traveler terminal 2 receives the display images of P1, P2, P3 and P4. The received display images of P1, P2, P3 and P4 are images created in the order of the old date and time based on the traveling direction of the cruise operation route. For example, when P1 is the departure point, the date and time of the display image of P1 is the oldest date, and when P4 is the final destination, the date and time of the display image of P4 is the latest date and time. The control unit 21 of the traveler terminal 2 extracts date and time information from the received display images of P1, P2, P3 and P4, respectively. The control unit 21 of the traveler terminal 2 sorts the display images of P1, P2, P3 and P4 in the order of the old date and time indicated by the extracted date and time information. For example, the sorting result of the display image is in the order of P1, P2, P3 and P4. The control unit 21 of the traveler terminal 2 sets the display time interval to 5 seconds for the plurality of sorted display images. The display time interval may be set at an arbitrary time interval.

  The control unit 21 of the traveler's terminal 2 obtains the first display image from among the plurality of sorted display images. The control unit 21 of the traveler's terminal 2 specifies the position of the sphere in the captured image corresponding to the position information associated with the first display image. Via the display unit 26 of the traveler terminal 2, the control unit 21 of the traveler terminal 2 superimposes and displays the first display image on the specified spherical position. The timer by the clock unit 28 of the traveler's terminal 2 is activated to calculate the elapsed time. After 5 seconds, the control unit 21 of the traveler's terminal 2 clears the first display image being displayed from among the photographed images of the earth ball 3 and acquires the second display image from among the plurality of display images. The control unit 21 of the traveler's terminal 2 specifies the position of the sphere in the captured image corresponding to the position information associated with the second display image. Via the display unit 26 of the traveler terminal 2, the control unit 21 of the traveler terminal 2 superimposes and displays the second display image on the sphere position via the specified display unit 26. In addition, before acquiring the next display image out of a plurality of display images, it is not necessary to clear the display image being displayed at the ball position in the photographed image. The third or fourth display process is the same as the second, and therefore the description thereof is omitted. After the fourth display image is superimposed and displayed on the position of the sphere in the captured image corresponding to the position information associated with the fourth display image, the timer is ended. Note that the above-described processing may be continued and executed without ending the timer. In this case, for example, the first display image is acquired again from among the plurality of display images, and the plurality of displays are displayed by overlapping and displaying on the spherical position in the photographed image corresponding to the position information associated with the first display image. Images can be repeatedly switched and displayed.

  FIG. 20 is a flowchart showing a sorting process procedure in order of date and time when the terminal 2 of the second embodiment receives a plurality of display images. About the content which overlaps with FIG. 12, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 11 of the server 1 acquires a plurality of display image IDs and position information corresponding to the received user ID from the user DB 141 of the large capacity storage unit 14 (step S131). The control unit 11 of the server 1 acquires each display image corresponding to each acquired display image ID from the display image DB 142 of the large capacity storage unit 14 (step S132). The communication unit 13 of the server 1 transmits the plurality of acquired display images and position information associated with the display images to the terminal 2 (step S133). The communication unit 23 of the terminal 2 receives the plurality of display images transmitted from the server 1 and the position information associated with the display images (step S231). The control unit 21 of the terminal 2 specifies a display range based on the acquired AR marker information in the captured image of the ground ball 3 and extracts a display image within the specified display range from among the plurality of received display images (FIG. Step S232). For the process of specifying the display range, for example, the control unit 21 of the terminal 2 extracts the display center point and the display radius in the photographed image of the earth ball 3, and the latitude and longitude position information corresponding to the extracted display center point recognize. The control unit 21 of the terminal 2 may specify the display range according to the position information within the radius extracted centering on the recognized latitude and longitude points. The control unit 21 of the terminal 2 sequentially extracts date and time information from the plurality of extracted display images (step S233), and determines whether the display image is the last (step S234). If it is determined that the display image is not the last (NO in step S234), the control unit 21 of the terminal 2 returns to the process of extracting date and time information of the display image and extracts date and time information of the next display image (step S233). If it is determined that the display image is the last (YES in step S234), the control unit 21 of the terminal 2 sorts the plurality of display images in the chronological order or the chronological order according to the plurality of extracted date and time information (step S235). ). The control unit 21 of the terminal 2 sets a time interval for switching and displaying the plurality of sorted display images (step S236).

  FIG. 21 is a flowchart showing a processing procedure when the terminal 2 of the second embodiment performs switching display. In order to monitor the time interval, a timer is started and executed by the clock unit 28 of the terminal 2 (step S237). The control unit 21 of the terminal 2 sequentially acquires the display image and the position information associated with the display image from among the plurality of sorted display images (step S238), and the ball position in the photographed image corresponding to the acquired position information Are identified (step S239). Via the display unit 26 of the terminal 2, the control unit 21 of the terminal 2 superimposes and displays a display image on the position of the sphere in the specified captured image (step S240). The control unit 21 of the terminal 2 determines whether the acquired display image is the last display image (step S241). If it is determined that the displayed image is the last one (YES in step S241), the control unit 21 of the terminal 2 ends the timer (step S244). If it is determined that the display image is not the last one (NO in step S241), the control unit 21 of the terminal 2 determines whether the set time interval has elapsed (step S242). If it is determined that the set time interval has not elapsed (NO in step S 242), the control unit 21 of the terminal 2 continuously monitors the time by the timer. If it is determined that the set time interval has elapsed (YES in step S242), the control unit 21 of the terminal 2 clears the display image being displayed on the captured image (step S243). The control unit 21 of the terminal 2 returns to the display image and position information acquisition process, and acquires position information associated with the next display image and the display image from among the plurality of display images (step S238).

  FIG. 22 is a flowchart of the sorting process of the terminal 2 of the second embodiment. About the content which overlaps with FIG. 20, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 21 of the terminal 2 receives an input of display order for the plurality of received display images (step S245). The control unit 21 of the terminal 2 sorts the plurality of display images in the order of the received display order (step S246). The next process is the same as that of FIG.

  FIG. 23 is an explanatory diagram for explaining an operation including trajectory information of the terminal 2 of the second embodiment. On the basis of an example in which a plurality of display images described above are displayed by being superimposed in order of date and time, locus information is added, and an example of being superimposed and displayed on a photographed image of the earth ball 3 will be described. The control unit 21 of the traveler's terminal 2 sorts the display images P1, P2, P3 and P4 of the plurality of tourist spots in order of date and time. For example, the sorting result of the display images of the tourist spots is in the order of P1, P2, P3 and P4. The control unit 21 of the traveler terminal 2 sets a time interval for switching and displaying a plurality of display images. For example, the time interval is set to 5 seconds.

  As shown in FIG. 23A, the control unit 21 of the traveler terminal 2 acquires position information associated with the first display image P1 and the first display image P1 among the plurality of display images. The control unit 21 of the traveler terminal 2 generates locus information T1 based on the position information associated with the acquired first display image P1. The control unit 21 of the traveler terminal 2 specifies the position of the sphere in the photographed image of the ground ball 3 corresponding to the positional information associated with the acquired first display image P1. Via the display unit 26 of the traveler terminal 2, the control unit 21 of the traveler terminal 2 superimposes and displays the acquired first display image P1 and the generated trajectory information T1 on the specified spherical position.

  Subsequently, as shown in FIG. 23B, after the set time interval of 5 seconds has elapsed along the cruise operation route, the control unit 21 of the traveler's terminal 2 sequentially displays the next display image P2 among the plurality of display images. And the positional information linked | related with the following display image P2 is acquired. The control unit 21 of the traveler terminal 2 updates the existing trajectory information T1 based on the position information associated with the first display image P1 and the position information associated with the next display image P2. The control unit 21 of the traveler terminal 2 clears the first display image P1 being displayed from among the photographed images of the earth ball 3. The control unit 21 of the traveler terminal 2 specifies the position of the sphere in the photographed image of the ground ball 3 corresponding to the positional information associated with the acquired next display image P2. Via the display unit 26 of the traveler terminal 2, the control unit 21 of the traveler terminal 2 displays the acquired next display image P2 and the updated trajectory information T1 superimposed on the identified spherical position.

  Subsequently, as shown in FIG. 23C, after the set time interval of 5 seconds has elapsed along the cruise operation route, the control section 21 of the traveler's terminal 2 sequentially displays the next display image P3 among the plurality of display images. And the positional information linked | related with the following display image P3 is acquired. The control unit 21 of the traveler terminal 2 updates the existing trajectory information T1 based on the display image P1, the display image P2, and the position information associated with the display image P3. The control unit 21 of the traveler terminal 2 clears the display image P2 being displayed from among the photographed images of the earth ball 3. The control unit 21 of the traveler terminal 2 specifies the position of the sphere in the photographed image of the earth ball 3 corresponding to the positional information associated with the acquired next display image P3. Via the display unit 26 of the traveler terminal 2, the control unit 21 of the traveler terminal 2 displays the acquired next display image P3 and the updated trajectory information T1 superimposed on the identified spherical position.

  Subsequently, as shown in FIG. 23D, after the set time interval of 5 seconds has elapsed along the cruise operation route, the control section 21 of the traveler's terminal 2 sequentially displays the next display image P4 among the plurality of display images. And the positional information linked | related with the following display image P4 is acquired. The control unit 21 of the traveler terminal 2 updates the existing trajectory information T1 based on the display image P1, the display image P2, the display image P3, and the position information associated with the display image P4. The control unit 21 of the traveler terminal 2 clears the display image P3 being displayed from among the photographed images of the earth ball 3. The control unit 21 of the traveler terminal 2 specifies the position of the sphere in the photographed image of the ground ball 3 corresponding to the positional information associated with the acquired next display image P4. The control unit 21 of the traveler terminal 2 superimposes the acquired next display image P4 and the updated trajectory information T1 on the identified spherical position and displays the timer via the display unit 26 of the traveler terminal 2. finish. Note that the above-described processing may be continued and executed without ending the timer. In this case, for example, it is possible to repeatedly switch and display the plurality of display images together with the trajectory information by acquiring the first display image again from among the plurality of display images and clearing and regenerating the existing trajectory information. It becomes.

  The display format of the trajectory information is not limited to updating the trajectory information at every set time interval along the cruise operation route. For example, the control unit 21 of the traveler's terminal 2 acquires all of the position information associated with the plurality of display images, and generates trajectory information of the entire operation route. Via the display unit 26 of the traveler's terminal 2, the control unit 21 of the traveler's terminal 2 superimposes and displays the generated trajectory information of the entire operating route at the ball position in the photographed image corresponding to the position information Also good. Alternatively, only the generated trajectory information may be overlapped and displayed without displaying the display image on the display unit 26 of the traveler terminal 2.

  FIG. 24 is a flowchart showing a processing procedure when the terminal 2 of the second embodiment performs switching display including trajectory information. The reception process of the plurality of display images and the sort process in the order of date and time are the same as those in FIG. Moreover, in FIG. 24, the same reference numerals are given to contents overlapping with those in FIG. 21, and the description will be omitted. The control unit 21 of the terminal 2 determines whether or not trajectory information exists (step S247). If it is determined that no trajectory information exists (NO in step S247), the control unit 21 of the terminal 2 generates trajectory information based on the position information associated with the acquired display image (step S248). If it is determined that the trajectory information exists (YES in step S247), the control unit 21 of the terminal 2 updates the existing trajectory information based on the position information associated with the acquired display image (step S249). The control unit 21 of the terminal 2 specifies the position of the sphere in the captured image corresponding to the position information associated with the display image. Through the display unit 26 of the terminal 2, the control unit 21 of the terminal 2 displays the display image and the trajectory information superimposed on the specified spherical position (step S250).

  According to the present embodiment, it is possible to superimpose and display on the sphere position in the captured image corresponding to the position information associated with the display image in the order of the date and time indicated by the date and time information.

  According to the present embodiment, it is possible to superimpose and display on the sphere position in the captured image corresponding to the position information associated with the display image in the order of display order.

  According to the present embodiment, trajectory information is generated based on position information associated with a plurality of display images, and the generated trajectory information is superimposed on the spherical position in the captured image corresponding to the position information associated with the display image. It becomes possible to display.

  According to this embodiment, it is possible to provide various information such as an image of a tourist spot on cruise operation, a stay schedule and the like. Depending on the visualization of the operation route, it is possible to provide a service such as an operation route or a traveling direction.

  According to this embodiment, it is possible to use a mountain climbing route or an airline flight route. For example, in the case of using a mountain climbing route, the height of a mountain, a river, a lake or the sea is color-coded and drawn using a drawing method such as a 3D object to generate a three-dimensional image. It becomes possible to overlap and display on a photography picture.

(Embodiment 3)
In the present embodiment, a treasure hunt game system will be described by taking an embodiment in which an AR system is implemented as an example.

  FIG. 25 is a block diagram showing a configuration example of the server 1 of the third embodiment. About the content which overlaps with FIG. 2, the same code | symbol is attached | subjected and description is abbreviate | omitted. The large capacity storage unit 14 includes a music DB 145. The music DB 145 stores data such as music or sound effects.

  FIG. 26 is an explanatory diagram of an example of the record layout of the user DB 141 of the third embodiment. Descriptions of contents overlapping with FIG. 3 will be omitted. The user DB 141 includes a music ID sequence. The music ID string stores the ID of the music that can uniquely identify the music transmitted from the terminal 2 in association with the user ID.

  FIG. 27 is an explanatory view showing an example of the record layout of the music DB 145. As shown in FIG. The music DB 145 includes a music ID string and a music string. The music ID column stores an ID of music that is uniquely identified in order to identify music. The music string stores music data corresponding to the music ID.

  The game content of the treasure hunt game includes display images and music. The terminal 2 of the game company providing the treasure hunt game associates the display image of the treasure hunt game, the position information associated with the display image, and the music with the user ID of the game company, and transmits it to the server 1. In the present embodiment, for example, music is a sound effect relating to a game scenario. The control unit 11 of the server 1 stores the display image, the position information, and the sound effect transmitted from the terminal 2 of the game company in the large-capacity storage unit 14 in association with the user ID of the game company. Specifically, the control unit 11 of the server 1 shakes a unique display image ID on a display image, and shakes a unique music ID on sound effects. The control unit 11 of the server 1 stores the user ID of the game company, the display image ID, the music ID, and the position information in the user DB 141 as one record. The control unit 11 of the server 1 stores the display image ID and the data of the display image in the display image DB 142 as one record. The control unit 11 of the server 1 stores the music ID and the music data in the music DB 145 as one record.

  As a result, the game content provided by the game company can be displayed superimposed on the photographed image of the earth ball 3 through the display unit 26 of the player's terminal 2. The communication unit 23 of the player's terminal 2 receives the display image corresponding to the user ID of the game company transmitted from the server 1, the position information associated with the display image, and the sound effect. The control unit 21 of the player's terminal 2 specifies the ball position in the photographed image of the earth ball 3 corresponding to the received position information. Via the display unit 26 of the player's terminal 2, the control unit 21 of the player's terminal 2 superimposes and displays the received display image on the identified ball position. The control unit 21 of the player's terminal 2 reproduces the received sound effect via the speaker 20 of the player's terminal 2.

  FIG. 28 is an explanatory diagram for explaining the operation of the AR system of the third embodiment. For example, G1, G2 and G3 illustrated are display images showing spots that can be filled with treasures. G4 illustrated is a display image of the hint map. G5 illustrated is a display image of treasures.

  As shown in FIG. 28A, the control unit 21 of the player's terminal 2 specifies the ball position in the photographed image of the ground ball 3 corresponding to the position information associated with the received display images G1, G2 and G3. Through the display unit 26 of the player's terminal 2, the control unit 21 of the player's terminal 2 displays the display images G1, G2, and G3 superimposed on the identified ball position. When the player's terminal 2's input unit 25 accepts a tap operation on the display image G2, the control unit 21 of the player's terminal 2 obtains an instruction corresponding to the display image G2 based on the game scenario. For example, when the control unit 21 of the player's terminal 2 determines that the instruction according to the display image G2 is an instruction of the hint map, the control unit 21 acquires the display image G4. The game scenario may be built in the application or may be received by the communication unit 23 of the terminal 2 from the server 1 or an external device (not shown).

  Subsequently, as shown in FIG. 28B, the control unit 21 of the player's terminal 2 specifies the ball position in the photographed image of the ground ball 3 corresponding to the position information associated with the display image G4. Through the display unit 26 of the player's terminal 2, the control unit 21 of the player's terminal 2 superimposes and displays the display image G4 on the identified ball position. The control unit 21 of the player's terminal 2 reproduces the sound effect of the hint map (not shown) via the speaker 20 of the player's terminal 2. After the display image G4 is superimposed and displayed, it is possible to switch to the thumbnail of the display image G4 and display it. For example, the control unit 21 of the player's terminal 2 receives the tap operation of the display image G4 again by the input unit 25 of the player's terminal 2. The control unit 21 of the player's terminal 2 may switch the display image G4 out of the photographed image of the earth ball 3 to the thumbnail of the display image G4 and display it. Also, a timer function may be used. In the case of using the timer function, the control unit 21 of the player's terminal 2 activates a timer by the clock unit 28 of the player's terminal 2 and calculates an elapsed time. For example, after 5 seconds, the control unit 21 of the player's terminal 2 may automatically switch the display image G4 to the thumbnail of the display image G4 from among the photographed image of the earth ball 3 and display it. After switching to the thumbnail of the display image G4 and displaying it, the control unit 21 of the player's terminal 2 ends the timer. Although the example of the thumbnail of the display image is shown in the present embodiment, the present invention is not limited to this. For example, the display image G4 may be cleared from the photographed image of the earth ball 3, or the display image G4 may be displayed as it is on the photographed image of the earth ball 3.

  Continuing, as shown in FIG. 28C, depending on the hint presented, it keeps searching for treasure at spots that can be filled with other treasures. For example, the control unit 21 of the player's terminal 2 receives the tap operation of the display image G3 indicating a spot that can be filled with another treasure by the input unit 25 of the player's terminal 2. The control unit 21 of the player's terminal 2 acquires an instruction according to the display image G3 based on the scenario of the game. For example, when the control unit 21 of the player's terminal 2 determines that the instruction according to the display image G3 is a treasure instruction, the control unit 21 acquires the display image G5.

  Subsequently, as shown in FIG. 28D, the control unit 21 of the player's terminal 2 specifies the ball position in the captured image corresponding to the position information associated with the display image G5. Via the display unit 26 of the player's terminal 2, the control unit 21 of the player's terminal 2 superimposes and displays the display image G5 on the identified ball position. The control unit 21 of the player's terminal 2 reproduces the sound effect of treasure discovery (not shown) via the speaker 20 of the player's terminal 2. Note that the display image G4 of the hint map or the display image G5 of the treasure may be displayed on the entire screen on the display unit 26 of the player's terminal 2 or may be displayed on a partial area of the screen.

  In addition, when there is explanatory information associated with the display image, the control unit 21 of the player's terminal 2 displays the display image and the display image on the photographed image of the earth ball 3 via the display unit 26 of the player's terminal 2 The explanatory information associated with may be superimposed and displayed. For example, as shown in FIG. 28B, the explanation information associated with the display image G4 of the hint map is "Hint: Go to the sea and search!" As shown in FIG. 28D, the explanation information associated with the display image G5 of treasure is “Congratulations! Acquisition of treasure!”.

  FIG. 29 is a flowchart showing a processing procedure when the terminal 2 of the third embodiment transmits a display image and music. About the content which overlaps with FIG. 11, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 21 of the terminal 2 receives the selection of music for reproduction in accordance with the display of the specific position of the photographed image of the earth ball 3 (step S261). The control unit 21 of the terminal 2 associates the selected display image, the music, and the acquired position information with the user ID. The communication unit 23 of the terminal 2 transmits the user ID, the display image, the music, and the position information to the server 1 (step S262). The communication unit 13 of the server 1 receives the user ID, the display image, the music, and the position information transmitted from the terminal 2 (step S161). The control unit 11 of the server 1 associates the received display image, music, and position information with the user ID, and stores the information in the large-capacity storage unit 14 (step S162).

  FIG. 30 is a flowchart showing a processing procedure when the terminal 2 of the third embodiment receives a display image and music. The same reference numerals as in FIG. 12 denote the same parts as those in FIG. The control unit 11 of the server 1 acquires the display image ID, the music ID, and the position information corresponding to the user ID transmitted from the terminal 2 from the user DB 141 of the large capacity storage unit 14 (step S163). The control unit 11 of the server 1 acquires the music corresponding to the acquired music ID from the music DB 145 of the large-capacity storage unit 14 (step S164). The communication unit 13 of the server 1 transmits the acquired display image, the music, and the position information associated with the display image to the terminal 2 (step S165). The communication unit 23 of the terminal 2 receives the display image, the music, and the position information associated with the display image transmitted from the server 1 (step S263). The control unit 21 of the terminal 2 performs a game process according to the received display image, the music, and the position information associated with the display image (step S264). The process flow of the game will be described next.

  FIG. 31 is a flowchart showing the game processing procedure of the third embodiment. The control unit 21 of the terminal 2 specifies the ball position in the captured image corresponding to the position information associated with the display image indicating the spot that can be filled with the treasure (step S901). In the following, for the sake of brevity, the display image showing the spots that can be filled with the treasure is replaced with the display image of the spots. The control unit 21 of the terminal 2 superimposes and displays the display image of the spot on the specified spherical position through the display unit 26 of the terminal 2 (step S902). The control unit 21 of the terminal 2 receives the selection of the display image of the spot by the input unit 25 of the terminal 2 (step S903). Based on the game scenario, the control unit 21 of the terminal 2 determines whether it is a treasure instruction according to the display image of the selected spot (step S904). When determining that the instruction is a treasure instruction (YES in step S904), control unit 21 of terminal 2 specifies the position of the sphere in the captured image corresponding to the position information associated with the display image of the treasure (step S905). . The control unit 21 of the terminal 2 superimposes and displays the display image of the treasure on the specified sphere position via the display unit 26 of the terminal 2 (step S906). The control unit 21 of the terminal 2 reproduces the sound effect of treasure discovery via the speaker 20 of the terminal 2 (step S 907). The control unit 21 of the terminal 2 superimposes and displays the explanatory information associated with the display image of the treasure on the specified sphere position through the display unit 26 of the terminal 2 (step S908), and ends the processing. If the control unit 21 of the terminal 2 determines that it is not an instruction of treasure (NO in step S904), it determines whether it is an instruction of a hint map according to the display image of the selected spot (step S909). When controller 21 of terminal 2 determines that the instruction is a hint map instruction (YES in step S 909), control unit 21 specifies the position of the sphere in the captured image corresponding to the position information associated with the display image of the hint map (step S910). The control unit 21 of the terminal 2 superimposes and displays the display image of the hint map on the specified sphere position through the display unit 26 of the terminal 2 (step S911). The control unit 21 of the terminal 2 reproduces the sound effect of the hint map via the speaker 20 of the terminal 2 (step S 912). The control unit 21 of the terminal 2 superimposes and displays the explanatory information associated with the display image of the hint map on the specified spherical position through the display unit 26 of the terminal 2 (step S913). After the process of step S913, and when it is determined that the instruction of the hint map is not issued (NO in step S909), the control unit 21 of the terminal 2 returns to the process of accepting the selection of the display image of the spot (step S903).

  According to the present embodiment, the display image and the music can be enjoyed together. When there is a scenario such as a treasure hunt game, the mood can be easily excited using sound effects and the like. Also, only music may be provided. For example, when acquiring the location information of the birthplace of singers from around the world and transmitting the music of the singer and the acquired location information to the server 1, the user acquires the music corresponding to the location information of the birthplace of the singers around the world It becomes possible to play back.

(Embodiment 4)
The present embodiment takes, as an example, an embodiment implementing an AR system for different types of spheres. There are various types of spheres depending on the display language, the size of the sphere, the manufacturer, etc. Different marker recognition systems are used to recognize markers for different types of spheres.

  FIG. 32 is a block diagram showing a configuration example of the server 1 of the fourth embodiment. About the content which overlaps with FIG. 2, the same code | symbol is attached | subjected and description is abbreviate | omitted. The large capacity storage unit 14 includes a sphere DB 146. The sphere DB 146 stores sphere information.

  FIG. 33 is an explanatory diagram of an example of the record layout of the user DB 141 of the fourth embodiment. Descriptions of contents overlapping with FIG. 3 will be omitted. The user DB 141 includes a sphere ID sequence. The sphere ID column stores the ID of a sphere that can uniquely identify sphere information.

  FIG. 34 is an explanatory view showing an example of the record layout of the sphere DB 146. As shown in FIG. The sphere DB 146 includes a sphere ID row, a maker row, a size row, and a language row. The sphere ID string stores a sphere ID uniquely identified to identify the sphere. The maker row stores the name of the maker. The size column stores the size of the sphere. The language string stores the language type printed on the sphere.

  FIG. 35 is an explanatory diagram for explaining the operation of the AR system of the fourth embodiment. For example, for overseas travel or domestic travel, an airline provides a passenger with a viewing service such as movies and music. The airline distributes or sells different types of earth balls 3 to the passenger based on the attributes such as the nationality, age and gender of the passenger. For example, the English version of Earth Ball 3 is distributed to passengers in western countries, and the Japanese version of Earth Ball 3 is distributed to Japanese passengers. Alternatively, the small earth ball 3 may be distributed to children.

  Specifically, the control unit 21 of the terminal 2 of the airline reads the QR code (registered trademark) photographed by the photographing unit 27 of the terminal 2 of the airline, and specifies the sphere ID of the ground ball 3. The means for specifying the sphere ID is not limited to the QR code, but may be specified by a one-dimensional code or another two-dimensional code, or may be specified by manual input. Further, it may be specified by AR marker information in the photographed image of the earth ball 3. For example, the control unit 21 of the terminal 2 of the airline acquires a photographed image of the ground ball 3 with an AR marker photographed by the photographing unit 27 of the terminal 2 of the airline. The control unit 21 of the terminal 2 of the airline may recognize the AR marker in the acquired captured image, and specify the sphere ID according to the recognized AR marker. When letters with names given to seas, continents, countries or cities in the map constitute an AR marker, specify the sphere ID according to the type of the language of the letter by the sphere DB 146 of the mass storage 14 Is possible. In addition, although the example which discriminate | determines spherical object ID by the QR code, the character, or AR marker provided to the earth ball 3 was shown, it does not restrict to this. Individual applications may be prepared according to the type of sphere. In this case, the sphere ID can be specified by the application ID transmitted from the terminal 2.

  The terminal 2 of the airline transmits the user ID of the airline, the display image, and the position information associated with the display image to the server 1 in association with the specified sphere ID of the earth ball 3. The server 1 stores the user ID of the airline transmitted from the terminal 2 of the airline, the display image, and the position information associated with the display image in the large-capacity storage unit 14 in association with the sphere ID.

  As a result, the display image provided by the airline can be superimposed and displayed on the photographed image of the distributed earth ball 3 through the display unit 26 of the terminal 2 of the passenger. For example, the photographing unit 27 of the terminal 2 of the passenger photographs the QR code of the spherical surface of the distributed earth ball 3. The control unit 21 of the passenger's terminal 2 reads the captured QR code and specifies the sphere ID. The communication unit 23 of the passenger's terminal 2 transmits the specified sphere ID and the user ID of the airline to the server 1. The control unit 11 of the server 1 acquires the display image corresponding to the received sphere ID and the user ID of the airline, and the position information associated with the display image. The communication unit 13 of the server 1 transmits the display image and the position information associated with the display image to the passenger's terminal 2. The communication unit 23 of the passenger's terminal 2 receives the display image transmitted from the server 1 and the position information associated with the display image. The control unit 21 of the passenger's terminal 2 specifies the ball position in the photographed image of the earth ball 3 corresponding to the position information associated with the received display image. Through the display unit 26 of the passenger's terminal 2, the control unit 21 of the passenger's terminal 2 superimposes and displays the received display image on the specified spherical position.

  FIG. 36 is a flowchart showing a processing procedure when the terminal 2 of the fourth embodiment transmits a display image based on the sphere ID. About the content which overlaps with FIG. 11, the same code | symbol is attached | subjected and description is abbreviate | omitted. The control unit 21 of the terminal 2 identifies the sphere ID (step S 281), and associates the user ID, the selected display image, and the position information associated with the display image with the identified sphere ID. The communication unit 23 of the terminal 2 transmits, to the server 1, the sphere ID, the user ID, the display image, and the position information associated with the display image (step S282). The communication unit 13 of the server 1 receives the sphere ID, the user ID, the display image and the position information associated with the display image transmitted from the terminal 2 (step S181). The control unit 11 of the server 1 stores the received user ID, the display image, and the position information associated with the display image in the large-capacity storage unit 14 in association with the received sphere ID (step S182).

  FIG. 37 is a flowchart showing a processing procedure when the terminal 2 of the fourth embodiment receives a display image based on the sphere ID. About the content which overlaps with FIG. 12, the same code | symbol is attached | subjected and description is abbreviate | omitted. A processing flow will be described as an example of specifying the sphere ID by the QR code. The photographing unit 27 of the terminal 2 photographs the QR code on the surface of the earth ball 3. The control unit 21 of the terminal 2 acquires a QR code from the photographed image of the QR code on the surface of the spherical surface of the ground ball 3 that has been photographed (step S283). The control unit 21 of the terminal 2 specifies the sphere ID by the acquired QR code (step S284). The communication unit 23 of the terminal 2 transmits the specified sphere ID and user ID to the server 1 (step S285). The communication unit 13 of the server 1 receives the sphere ID and the user ID transmitted from the terminal 2 (step S183). The control unit 11 of the server 1 acquires the display image ID corresponding to the received sphere ID and user ID and the position information from the user DB 141 of the large capacity storage unit 14 (step S184).

  According to this embodiment, it is possible to use an AR system for a plurality of types of spheres. By transmitting and receiving to a server such as a display image or music for each sphere ID, it is possible to provide diversified or versatile services. It becomes possible to provide a unique earth ball 3 as a novelty or a product of another company.

  It should be understood that the embodiments disclosed herein are illustrative in all respects and not restrictive. The scope of the present invention is indicated not by the meaning described above but by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

1 Information processing device (server)
11 control unit 12 storage unit 13 communication unit 14 large capacity storage unit 141 user DB
142 Display image DB
143 explanatory information DB
144 Public Information DB
145 Music DB
146 Sphere DB
1a Portable storage medium 1b Semiconductor memory 1P Control program 2 Information processing terminal (terminal)
21 control unit 22 storage unit 23 communication unit 24 album 25 input unit 26 display unit 27 imaging unit 28 clock unit 29 microphone 20 speaker 2P control program 3 earth ball B bus N network

Claims (4)

On the computer
There are multiple types of spheres with markers on the surface,
Identify a sphere ID identifying the sphere,
Accept a selection of content to be displayed superimposed on the captured image of the sphere,
Obtain location information corresponding to the content,
A program that executes processing for transmitting the received content and the acquired position information in association with the specified sphere ID and user ID. On the computer
There are multiple types of spheres with markers on the surface,
Receiving a sphere ID identifying the sphere, a user ID associated with the sphere ID, a content to be displayed superimposed on the photographed image of the sphere, and position information associated with the content;
Storing the content and position information in association with the received sphere ID and the user ID;
When the sphere ID and the user ID are received, the content corresponding to the sphere ID and the user ID, and the position information associated with the content are read out.
A program for executing a process of transmitting the read content and the position information. There are multiple types of spheres with markers on the surface,
A sphere ID for identifying the sphere, a user ID associated with the sphere ID, a content to be displayed superimposed on the captured image of the sphere, and a receiving unit for receiving position information associated with the content;
A storage unit for storing the content and the position information in association with the sphere ID and the user ID received by the receiving unit;
A second receiver that receives the sphere ID and the user ID;
A reading unit that reads the content stored by the storage unit and the position information associated with the content based on the sphere ID and the user ID received by the second receiving unit;
An information processing apparatus, comprising: a transmission unit that transmits the content read by the reading unit and the position information. There are multiple types of spheres with markers on the surface,
Identify a sphere ID identifying the sphere,
Accept a selection of content to be displayed superimposed on the captured image of the sphere,
Obtain location information corresponding to the content,
An information processing method comprising: causing a computer to execute processing for transmitting the received content and the acquired position information in association with the specified sphere ID and user ID.

Images