JP2022185235A - Communication system, communication method, and communication program - Google Patents

Abstract

To provide a user with an AR experience with reality.SOLUTION: A communication system 1 comprises: AR glasses 10; a smart device 30 carried by a user; and a server 40 for communication with the smart device. The server 40 identifies a peripheral situation of the user on the basis of information on the position of the smart device 30, updates an area in which an object within a virtual map is displayed on the map in accordance with the identified peripheral situation, and transmits information on the virtual map including information on the inside of the area to the smart device 30. The smart device 30 causes the AR glasses 10 to output the virtual map by using the information on the virtual map transmitted from the server 40.SELECTED DRAWING: Figure 3

Description

The present invention relates to a communication system, communication method and communication program.

In recent years, HMDs (Head Mounted Displays) and the like have been proposed that allow users to experience virtual reality (VR) and augmented reality (AR). Among these, as an HMD using AR technology, AR glasses, which are worn on the user's head and used in the same manner as general eyeglasses, have been proposed.

The AR glasses capture an image of the real space in front of the user, and display the captured image of the real space on a display corresponding to the lens portion of the AR glasses. In addition, AR glasses display virtual content generated by CG on a display by superimposing it on a captured image of the real space. As a result, a user wearing AR glasses can experience an AR space in which virtual content is superimposed on a real space.

Japanese Patent No. 6655751

A system has been proposed that allows a user to experience an RPG (Role-Playing Game) by displaying RPG (Role-Playing Game) content on an AR glass display. There has been a demand for this system to provide users with a realistic AR experience.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a communication system, a communication method, and a communication program capable of providing users with a realistic AR experience.

In order to solve the above-described problems and achieve the object, a communication system according to the present invention includes a display device, a smart device carried by a user, and a server that communicates with the smart device. The server includes an acquisition unit that acquires the location information of the smart device, an identification unit that identifies the surrounding situation of the user based on the location information acquired by the acquisition unit, and the surrounding situation identified by the identification unit. an updating unit for updating an area in which an object in a virtual map is displayed on the map according to a transmission unit for transmitting map information to the smart device, the smart device detecting location information of the smart device and transmitting the location information to the server; and a location detection unit for transmitting the location information from the server. and a control unit for outputting a virtual map to the display device using the obtained virtual map information.

According to the present invention, it is possible to provide users with a realistic AR experience.

FIG. 1 is a diagram showing an example of an image displayed on a display of AR glasses. FIG. 2 is a diagram illustrating the configuration of the communication system according to the embodiment. FIG. 3 is a block diagram showing an example of the configuration of the communication system according to the embodiment. 4 is a block diagram showing an example of the configuration of the AR glasses shown in FIG. 3. FIG. FIG. 5 is a block diagram showing an example of the configuration of the smart device shown in FIG. 3. As shown in FIG. FIG. 6 is a block diagram showing an example of the configuration of the server shown in FIG. 3; FIG. 7 is a diagram for explaining the map update range. FIG. 8 is a diagram illustrating an example of updating a map in a location with a wide update range. FIG. 9 is a diagram for explaining an example of map update in a place where the update range is narrow. FIG. 10 is a sequence diagram illustrating a procedure of communication processing in the communication system according to the embodiment; FIG. 11 is a diagram showing a computer executing a program.

Hereinafter, embodiments of a communication system, a communication method, and a communication program according to the present application will be described in detail with reference to the drawings. Note that the communication system, communication method, and communication program according to the present application are not limited by this embodiment.

In the following embodiments, a communication system, a communication method, and a processing flow of a communication program according to the embodiments will be described in order, and finally effects of the embodiments will be described.

[Embodiment]
First, an embodiment will be described. In the embodiment, a user who is sightseeing in a predetermined area, specifically, a sightseeing spot (for example, an island) wears AR glasses capable of displaying virtual RPG content, and while sightseeing in the tourist spot, A system that allows RPGs to be experienced will be explained.

FIG. 1 is a diagram showing an example of an image displayed on a display of AR glasses. As illustrated in FIG. 1, a monster content C1 and a coin content C2 are superimposed on an image of an actual island (for example, an image of a forest) and displayed on the AR glass, and the user defeats the monster and collects the coin. It enables game experiences such as collecting and working on missions. In addition, communication is possible between users wearing AR glasses.

In the embodiment, the entire small island is utilized, and while viewing a virtual map displayed on AR glasses, the user moves his or her own body, and furthermore, while communicating with friends, an RPG experience that is unique to that area can be enjoyed. make it possible. In the virtual map displayed on the AR glasses, the area where the objects in the map are displayed expands according to the range visited by the user. Furthermore, in the embodiment, a realistic AR experience is provided by outputting virtual content to the AR glasses according to the surrounding conditions of the user who is actually sightseeing by the user wearing the AR glasses. The virtual contents are, for example, animal and plant character images created by CG or the like, sounds such as animal cries and footsteps, or maps. Note that the character image is not limited to a game character-like image, and may be an image created from a photograph or the like, and includes moving images.

In addition, a virtual map M is displayed on the AR glass as a map of tourist spots such as islands, and when the user moves, the area where the objects in the map M are displayed on the map is updated. That is, for example, each time the user moves to a new area, the map M is updated, and objects such as roads, buildings, mountains, and rivers in the vicinity of the map M are displayed on the map M according to the range visited by the user. The area displayed above expands. Note that the map M does not have to be based on an actual map of a tourist spot such as an island, and the objects displayed on the map may not actually exist. It can be a simple map. In other words, with AR glasses, for example, even if a castle or a town that does not actually exist is displayed on the map M, or an area that the user does not want to move to, such as a place that is not a tourist spot, is not displayed on the map M. good. Also, in the example of FIG. 1, the map M is displayed in a small size on the upper right, but when the user performs a predetermined operation, the map M is enlarged and displayed. Note that the device for displaying the map M is not limited to AR glasses, and the map M may be displayed on another device.

[Configuration of communication system]
A configuration of a communication system according to an embodiment will be described. FIG. 2 is a diagram illustrating the configuration of the communication system according to the embodiment. FIG. 3 is a block diagram showing an example of the configuration of the communication system according to the embodiment.

As shown in FIGS. 2 and 3, the communication system 1 according to the embodiment includes an AR glass 10 (display device), a motion controller 20 held by a user, a smart device 30 carried by a user, a data center, and the like. has a server 40 located in

The AR glasses 10 are worn on the user's head like general eyeglasses. The AR glasses 10 have an imaging function, an audio input function, a communication function, a display function, and an audio output function. The AR glasses 10 capture an image of the real space, and display the captured image of the real space on a display corresponding to the lens portion of the AR glasses 10 . In addition, under the control of the smart device 30, the AR glasses 10 superimpose the virtual content generated by CG on the captured image of the real space and display it on the display. Under the control of the smart device 30, the AR glasses 10 also output virtual content according to the user's surroundings.

The motion controller 20 is a controller held by the user. The motion controller 20 has, for example, an acceleration sensor and a communication function for transmitting detection results of the acceleration sensor to the smart device 30 .

The smart device 30 is a device carried by a user, for example, a smart device such as a tablet or a smartphone, and has a position detection function, an image recognition function, and a voice recognition function. The smart device 30 controls the AR glasses 10 . The smart device 30 acquires the motion of the user's hand based on the detection result of the acceleration sensor transmitted from the motion controller 20 . The smart device 30 connects to the server 40 via the network N and transmits the user's position detection result, image recognition result, and voice recognition result to the server 40 . The smart device 30 receives virtual content output from the AR glasses 10 via the network N from the server 40 . The smart device 30 controls output of virtual content provided by the server 40 to the AR glasses 10 .

The server 40 holds map information and virtual content in the AR space, and determines the user's surroundings based on the user's position detection results, image recognition results, and voice recognition results transmitted from the smart device 30. do. The server 40 then selects virtual content according to the user's surroundings and transmits it to the smart device 30 .

[AR glass]
Next, the AR glasses 10 will be explained. FIG. 4 is a block diagram showing an example of the configuration of the AR glasses 10 shown in FIG. 3. As shown in FIG.

As shown in FIG. 4 , the AR glasses 10 have an imaging unit 11 , an audio input unit 12 (recording unit), a communication unit 13 , a display unit 14 and an audio output unit 15 .

The imaging unit 11 images the real space. For example, when the imaging direction of the imaging unit 11 is in front of the AR glasses 10 , the imaging unit 11 images the real space in front of the AR glasses 10 . The imaging unit 11 generates an image of a real space by processing a photoelectric conversion element such as a CCD or a CMOS and a photoelectric conversion signal output from the imaging element. The imaging unit 11 acquires a two-dimensional image of three colors of R (red), G (green), and B (blue), for example.

The voice input unit 12 is configured by a device that receives input of voice and records the input voice.

The communication unit 13 performs wireless communication with the smart device 30 . The communication unit 13 transmits the image captured by the imaging unit 11 to the smart device 30 . The communication unit 13 receives virtual content transmitted from the smart device 30 . The virtual contents are images of maps, character images of animals and plants, sounds such as cries and footsteps of animals.

The display unit 14 is configured by a display. The display unit 14 displays the image of the real space captured by the imaging unit 11 . The display unit 14 displays a virtual map image and animal and plant character images among the virtual contents received by the communication unit 13 by superimposing them on the real space image. The display unit 14 may be a display capable of transmissive display, and may display character images of animals and plants among the virtual contents.

The audio output unit 15 outputs audio. The audio output unit 15 outputs audio such as animal barks and footsteps from the virtual content received by the communication unit 13 .

[Smart device]
Next, the smart device 30 is described. FIG. 5 is a block diagram showing an example of the configuration of the smart device 30 shown in FIG. 3. As shown in FIG. As shown in FIG. 5 , the smart device 30 has a communication section 31 , a storage section 32 and a control section 33 .

The communication unit 31 performs wireless or wired communication with other devices. The communication unit 21 is a communication interface that transmits and receives various information to and from another device connected via a network or the like. The communication unit 31 is realized by a NIC (Network Interface Card) or the like, and performs communication between another device and the control unit 33 (described later) via an electric communication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit 31 receives images from the AR glasses 10 and transmits virtual content to the AR glasses 10 . The communication unit 31 transmits position detection results and image recognition results to the server 40 and receives virtual content from the server 40 .

The storage unit 32 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), an optical disc, or the like. Note that the storage unit 32 may be a rewritable semiconductor memory such as a RAM (Random Access Memory), a flash memory, or an NVSRAM (Non Volatile Static Random Access Memory). The storage unit 32 stores an OS (Operating System) and various programs executed by the smart device 30 . Furthermore, the storage unit 32 stores various information used in executing the program. The storage unit 32 stores image data 321 transmitted from the AR glasses 10 and content 322 including virtual content transmitted from the server 40 .

The control unit 33 controls the smart device 30 as a whole. The control unit 33 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 33 also has an internal memory for storing programs defining various processing procedures and control data, and executes each processing using the internal memory. Further, the control unit 33 functions as various processing units by running various programs. The control unit 33 has a position detection unit 331, an image recognition unit 332, a motion detection unit 333, and an AR glasses control unit 334 (control unit).

The position detector 331 detects the position of the smart device 30 and transmits it to the server 40 . The position detection unit 331 can acquire the position information of the smart device 30 by GPS (Global Positioning System), for example. The position detection unit 331 periodically acquires the position of the smart device 30 and transmits the position information represented by latitude and longitude to the server 40 via the communication unit 31 .

The image recognition unit 332 processes the image captured by the AR glasses 10 and recognizes features of the image such as an object appearing in the image. The image recognition unit 332 recognizes animals, plants, landscapes, and buildings appearing in images captured by the AR glasses 10 . The image recognition unit 332 performs image recognition that has already learned image features such as animals, plants, landscapes, and buildings that are specific to a predetermined area, specifically, a sightseeing spot that the user is currently sightseeing, by, for example, machine learning. Using the model, the features of the image captured by the AR glasses 10 are recognized. The image recognition unit 332 may perform processing to recognize an animal, a plant, a landscape, or a building when there is a predetermined change or more in the image by comparing the images before and after the image. The image recognition section 332 transmits the image recognition processing result to the server 40 via the communication section 31 .

Note that the server 40 may have the image recognition function. At that time, the smart device 30 transmits the image captured by the AR glasses 10 to the server 40 .

The motion detection unit 333 acquires the user's hand motion based on the detection result of the acceleration sensor transmitted from the motion controller 20 .

The AR glasses control unit 334 controls the processing of the AR glasses 10 . For example, the AR glasses control unit 334 uses virtual map information transmitted from the server 40 to cause the AR glasses 10 to output a virtual map. Also, for example, the AR glasses control unit 334 controls image display processing on the display of the lens unit of the AR glasses 10 . The AR glasses control section 334 has a content acquisition section 335 and a content synthesis section 336 .

The content acquisition unit 335 acquires virtual content transmitted from the server 40 . This virtual content is a virtual map, a character image, etc. superimposed on the image of the real space.

When the content acquired by the content acquiring unit 335 is a virtual map or character image superimposed on the image of the real space, the content synthesizing unit 336 combines the virtual map or character image with the image of the real space. Perform synthesis processing. After the composition processing by the content composition unit 336, the AR glasses control unit 334 causes the display unit 14 of the AR glasses 10 to display an image in which the character image is superimposed on the image of the real space. The superimposed positions of the virtual map and character images are set in advance, for example. Alternatively, the superimposed position of the virtual map or character image is set by an instruction from the server 40 .

[server]
Next, the server 40 will be explained. FIG. 6 is a block diagram showing an example of the configuration of server 40 shown in FIG. As shown in FIG. 6 , the server 40 has a communication section 41 , a storage section 42 and a control section 43 .

The communication unit 41 performs wireless or wired communication with other devices. The communication unit 41 is a communication interface that transmits and receives various information to and from another device connected via a network or the like. The communication unit 41 is realized by a NIC (Network Interface Card) or the like, and performs communication between another device and the control unit 43 (described later) via an electric communication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit 41 receives location information, image recognition results, and voice recognition results from the smart device 30 and transmits virtual content to the smart device 30 .

The storage unit 42 is a storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), an optical disc, or the like. Note that the storage unit 42 may be a rewritable semiconductor memory such as a RAM (Random Access Memory), a flash memory, or an NVSRAM (Non Volatile Static Random Access Memory). The storage unit 42 stores an OS (Operating System) and various programs executed by the server 40 .

Further, the storage unit 42 stores various information used in executing the program. The storage unit 42 stores map information 421 including forests, rivers, mountains, plains, sights, etc. of sightseeing spots such as islands visited by the user, and content information 422 including various types of virtual content. It is assumed that the map information 421 includes, for example, a map including contour lines, and includes information capable of specifying the altitude corresponding to each position. Also, the map information 421 includes attribute information indicating forests, rivers, mountains, plains, famous places, etc., or attribute information such as good view or bad view for each position. The various virtual contents are character images of animals and plants, or sounds of footsteps or cries of animals, and are provided in predetermined areas such as sightseeing spots, forests, rivers, mountains, plains, famous places, and the like. The content may be unique to each predetermined location of the tourist spot, which is created in advance in association with the location. Also, various contents may be created separately for enemies and allies. The content information 422 includes, for example, setting conditions for various types of content, which are setting information such as the display position and size of character images.

The control unit 43 controls the server 40 as a whole. The control unit 43 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 43 also has an internal memory for storing programs defining various processing procedures and control data, and executes each processing using the internal memory. Further, the control unit 43 functions as various processing units by running various programs. The control unit 43 includes a position information acquisition unit (acquisition unit) 431, an image recognition result acquisition unit 432 that acquires the image recognition result transmitted from the smart device 30, a surrounding situation identification unit (specification unit) 433, an area update unit (update unit) 434 and a content transmission control unit (control unit) 435 .

The location information acquisition unit 431 acquires location information of the smart device 30 . For example, the location information acquisition unit 431 detects the location of the user based on the location information received from the smart device 30 and the map information 421 . The location information acquisition unit 431 may acquire the location information of the smart device 30 periodically at predetermined time intervals, or may acquire the location information of the smart device 30 at predetermined timings.

The surrounding situation identification unit 433 identifies the surrounding situation of the user based on the position information acquired by the position information acquisition unit 431 . For example, the surrounding situation identifying unit 433 refers to the map information 421 and identifies the altitude of the position corresponding to the position information of the smart device 30 as the user's surrounding situation.

In addition, the surrounding situation identification unit 433 may, for example, identify a position attribute corresponding to the position information of the smart device 30 as the user's surrounding situation. For example, the surrounding situation identification unit 433 refers to the map information 421 based on the user's position and identifies mountains, forests, plains, etc. as attributes corresponding to the position information of the smart device 30 . Note that the surrounding situation identification unit 433 may determine attributes of mountains and plains based on the contour lines included in the map information 421 .

Further, the surrounding situation identification unit 433 may identify the user's surrounding situation based on the position information of the smart device 30 and the image of the real space. For example, when the image recognition result obtained by the image recognition result obtaining unit 432 includes a plurality of trees, the surrounding situation identifying unit 433 identifies that the user's surrounding situation is a forest.

The region updating unit 434 updates the region in which the objects in the virtual map are displayed on the map according to the surrounding conditions determined by the surrounding condition specifying unit 433 . For example, the region updating unit 434 updates the region in which the objects in the map are displayed on the map so that the higher the altitude of the position identified by the surrounding situation identifying unit 433, the wider the update range.

To explain with a specific example, for example, if the altitude of the position specified by the surrounding situation specifying unit 433 is equal to or higher than a predetermined threshold, the region updating unit 434 updates the user's position as the center of 100 meters around the user's position. For the area of the range, update so that the object in the virtual map is displayed on the map, and if it is equal to or less than the predetermined threshold, for the area of 50 meters around the user's position, Update so that the objects in the virtual map are displayed on the map. Note that the processing of the area updating unit 434 is not limited to this. For example, the area updating unit 434 may set a plurality of thresholds so that the update range becomes wider step by step as the altitude increases. . Also, the area updating unit 434 may calculate the size of the update range according to the altitude using a predetermined algorithm, and the update range may gradually widen as the altitude increases.

Here, it is a figure explaining the update range of a map using FIG. FIG. 7 is a diagram for explaining the map update range. As illustrated in FIG. 7, the server 40 updates the map in a wide range when the user goes to a high altitude place such as the top of a mountain, and updates the map when the user goes to a low altitude place. The virtual map displayed on the AR glasses 10 is updated so that the update range becomes narrower.

For example, as exemplified in FIG. 8, in the AR glasses 10, when the update range of the map is wide, an object in a wide range (for example, 100 meters around) around the user's position P is displayed on the map. A virtual map is displayed with the updated state. FIG. 8 is a diagram illustrating an example of updating a map in a location with a wide update range. In addition, as illustrated in FIG. 9, in the AR glasses 10, when the update range of the map is narrow, compared to the example of FIG. A virtual map is displayed with the objects updated to appear on the map. FIG. 9 is a diagram for explaining an example of map update in a place where the update range is narrow.

Further, the area updating unit 434 may update the area where the objects in the virtual map are displayed on the map according to the attribute of the position specified by the surrounding situation specifying unit 433 . To give a specific example, when the attribute of the position specified by the surrounding situation specifying unit 433 is “mountain” or “the view is good”, the region updating unit 434 updates the position of the user. Update so that the objects in the virtual map are displayed on the map in a range of 100 meters around the center. For example, when the attribute of the position specified by the surrounding situation specifying unit 433 is "forest" or "poor view", the region updating unit 434 updates the area around the user's position by 50 meters around the user's position. update so that the objects in the virtual map are displayed on the map.

Also, the area update unit 434 may update the area where the objects in the virtual map are displayed on the map, according to both the altitude and the attribute information. As a specific example, if the altitude is greater than or equal to a predetermined threshold value and the attribute information is "good view", the area update unit 434 updates the area around the user's position by 150 meters. The area of the range is updated so that the objects in the virtual map are displayed on the map, and if the altitude is greater than or equal to a predetermined threshold and the attribute information is "poor visibility", the user Update so that the objects in the virtual map are displayed on the map for a range of 75 meters around the position of .

Further, the area updating unit 434 may update the area where the objects in the virtual map are displayed on the map, taking into consideration the image recognition result acquired by the image recognition result acquiring unit 432. . For example, if the image recognition result includes a tree, the region update unit 434 narrows the range of the region to be updated by 10 meters, and if the image recognition result includes a wall or a fence, the range of the region to be updated is narrowed by 10 meters. may be narrowed by 30 meters.

When the region is updated by the region updater 434 , the content transmission control unit 435 transmits to the smart device 30 virtual map information including information within the region. For example, when the area is updated by the area updating unit 434, the content transmission control unit 435 generates image information of a virtual map in which objects in the updated area are displayed on the map. Image information is sent to the smart device 30 . In this case, the content transmission control unit 435 may transmit image information of the entire virtual map to the smart device 30 or may transmit image information of only the updated portion to the smart device 30 . Also, for example, the content transmission control unit 435 transmits the output timing of the content and the display position of the content to the smart device 30 together with the virtual content in the AR space.

[Processing procedure of communication processing]
FIG. 10 is a sequence diagram illustrating a procedure of communication processing in the communication system according to the embodiment;

As illustrated in FIG. 10, each time the smart device 30 detects the location of the smart device 30 (step S1), it transmits location information to the server 40 (step S2).

Then, when the image is transmitted from the AR glasses 10 (step S3), the smart device 30 executes image recognition processing on the image (step S4), and transmits the image recognition result to the server 40 (step S5).

The server 40 identifies the surrounding situation of the user based on the position information acquired by the position information acquisition unit 431 (step S6). Then, the server 40 updates the area where the objects in the virtual map are displayed on the map according to the surrounding conditions (step S7). Subsequently, when the area is updated by the area update unit 434, the server 40 transmits virtual map information including information on the area to the smart device 30 (step S8).

The smart device 30 synthesizes the content and the image of the real space (step S9). Then, the smart device 30 instructs the AR glasses 10 to output content (step S10), and causes the AR glasses to output virtual content in the AR space (for example, virtual maps and character images of animals and plants). (Step S11).

[Effects of Embodiment]
As described above, in the present embodiment, the server 40 of the communication system 1 identifies the surrounding situation of the user based on the location information of the smart device 30, and displays the virtual map in accordance with the identified surrounding situation. The object updates the area displayed on the map, and transmits virtual map information including information within the area to the smart device 30 . The smart device 30 uses the virtual map information transmitted from the server 40 to cause the AR glasses 10 to output the virtual map. Therefore, in the communication system 1, the update range of the map changes not only according to the user's walking range but also according to the surrounding conditions of the user, so that it is possible to provide the user with a more realistic AR experience.

Further, in the present embodiment, the map update range is changed according to the user's surroundings in consideration of the altitude and attributes of the user's position. It is possible to expand the range where you can see the , and if you are in the forest, it is possible to prevent the range where you can see objects in the map from expanding.

[System configuration, etc.]
Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device is realized by a CPU or GPU and a program analyzed and executed by the CPU or GPU, or as hardware by wired logic can be realized.

Further, among the processes described in this embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be done automatically by known methods. In addition, information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

[program]
It is also possible to create a program in which the processes executed by the AR glasses 10, the smart device 30, and the server 40 described in the above embodiments are described in a computer-executable language. For example, it is possible to create a program in which the processes executed by the AR glasses 10, the smart device 30, and the server 40 in the embodiment are described in a computer-executable language. In this case, the same effects as those of the above embodiments can be obtained by having the computer execute the program. Further, such a program may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read by a computer and executed to realize processing similar to that of the above embodiments.

FIG. 11 is a diagram showing a computer executing a program. As illustrated in FIG. 11, computer 1000 includes, for example, memory 1010, CPU 1020, hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. , and these units are connected by a bus 1080 .

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012 as illustrated in FIG. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). Hard disk drive interface 1030 is connected to hard disk drive 1090 as illustrated in FIG. A disk drive interface 1040 is connected to the disk drive 1100 . A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100 . Serial port interface 1050 is connected to mouse 1110 and keyboard 1120, for example. Video adapter 1060 is connected to display 1130, for example.

Here, as illustrated in FIG. 11, the hard disk drive 1090 stores an OS 1091, application programs 1092, program modules 1093, and program data 1094, for example. That is, the above program is stored, for example, in hard disk drive 1090 as a program module in which instructions to be executed by computer 1000 are described.

Various data described in the above embodiments are stored as program data in the memory 1010 or the hard disk drive 1090, for example. Then, the CPU 1020 reads the program modules 1093 and program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as necessary, and executes various processing procedures.

Note that the program module 1093 and program data 1094 related to the program are not limited to being stored in the hard disk drive 1090. For example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive or the like. . Alternatively, the program module 1093 and program data 1094 related to the program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.), and are transmitted via the network interface 1070. It may be read by CPU 1020 .

The above-described embodiments and modifications thereof are included in the scope of the invention described in the claims and their equivalents, as well as in the technology disclosed in the present application.

10 AR glasses 11 imaging unit 12 audio input unit 13, 31, 41 communication unit 14 display unit 15 audio output unit 20 motion controller 30 smart device 32, 42 storage unit 33, 43 control unit 40 server 321 image data 322 content 331 position detection Unit 332 Image recognition unit 333 Motion detection unit 334 AR glasses control unit 335 Content acquisition unit 336 Content synthesis unit 421 Map information 422 Content information 431 Position information acquisition unit 432 Image recognition result acquisition unit 433 Surrounding situation identification unit 434 Area update unit 435 Content Transmission control part

Claims (6)

A communication system having a display device, a smart device carried by a user, and a server communicating with the smart device,
The server is
an acquisition unit that acquires location information of the smart device;
an identifying unit that identifies the surrounding situation of the user based on the location information obtained by the obtaining unit;
an updating unit that updates an area in which an object in a virtual map is displayed on the map according to the surrounding situation specified by the specifying unit;
a transmitting unit configured to transmit virtual map information including information within the area to the smart device when the area is updated by the updating unit;
has
The smart device is
a position detection unit that detects position information of its own device and transmits the position information to the server;
a control unit for outputting a virtual map to the display device using the virtual map information transmitted from the server;
A communication system characterized by comprising: The identification unit identifies an altitude of a location corresponding to the location information of the smart device as the surrounding situation of the user,
The updating unit updates the area where the objects in the virtual map are displayed on the map so that the higher the altitude of the position specified by the specifying unit, the wider the update range. The communication system according to claim 1, wherein The specifying unit specifies a location attribute corresponding to the location information of the smart device as the surrounding situation of the user,
2. The method according to claim 1, wherein the updating unit updates a region in which the object in the virtual map is displayed on the map according to the attribute of the position specified by the specifying unit. Communications system. The display device
an imaging unit that captures an image of a real space;
a communication unit that communicates with the smart device and transmits an image of the real space to the smart device;
has
2. The communication system according to claim 1, wherein the identification unit identifies the surrounding situation of the user based on the location information of the smart device and the image of the real space. A communication method executed by a communication system having a display device, a smart device carried by a user, and a server communicating with the smart device,
a location detection step in which the smart device detects location information of itself and transmits the location information to the server;
an acquisition step in which the server acquires the location information of the smart device;
an identifying step in which the server identifies a surrounding situation of the user based on the location information obtained in the obtaining step;
an updating step of updating an area in which an object in the virtual map is displayed on the map according to the surrounding situation specified by the specifying step;
a transmission step in which the server transmits virtual map information including information in the area to the smart device when the area is updated in the updating step;
a control step in which the smart device outputs a virtual map to the display device using the virtual map information transmitted from the server;
A communication method comprising: A communication program for causing a computer to execute a communication method,
computer as a server,
an acquisition step of acquiring location information of the smart device carried by the user;
a specifying step of specifying the surrounding situation of the user based on the position information acquired by the acquiring step;
an updating step of updating an area in which an object in the virtual map is displayed on the map, according to the surrounding situation identified by the identifying step;
a sending step of sending virtual map information including information in the area to the smart device when the area is updated by the updating step;
and
a computer as the smart device;
a position detection step of detecting position information of the device itself and transmitting the position information to the server;
a control step of outputting a virtual map to a display device using the virtual map information transmitted from the server;
A communication program characterized by executing

Images