JP7219862B2 - Communication system, communication method and communication program - Google Patents

Description

The present invention relates to a communication system, communication method and communication program.

In recent years, HMDs (Head Mounted Displays) and the like have been proposed that allow users to experience virtual reality (VR) and augmented reality (AR). Among these, as an HMD using AR technology, AR glasses, which are worn on the user's head and used in the same manner as general eyeglasses, have been proposed.

The AR glasses capture an image of the real space in front of the user, and display the captured image of the real space on a display corresponding to the lens portion of the AR glasses. In addition, AR glasses display virtual content generated by CG on a display by superimposing it on a captured image of the real space. As a result, a user wearing AR glasses can experience an AR space in which virtual content is superimposed on a real space.

Japanese Patent No. 6655751

A system has been proposed that allows a user to experience an RPG (Role-Playing Game) by displaying RPG (Role-Playing Game) content on an AR glass display. There has been a demand for this system to provide users with a realistic AR experience.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a communication system, a communication method, and a communication program capable of providing a user with a realistic virtual space experience.

In order to solve the above problems and achieve the object, the communication system of the present invention communicates with the display device used by the user, the motion controller held by the user, the smart device carried by the user, and the smart device. and a server for displaying virtual content on a display device to provide a game experience to a user, wherein the server displays the virtual content of a type of weapon according to the user's selection on the smart device. a determination unit that determines the surrounding conditions of the user based on at least location information of the smart device, map information, and weather information; and a second setting unit for setting the attack power of the virtual content of the set weapon, wherein the smart device detects at least location information of the smart device and transmits the detected location information to the server. and a controller for outputting the virtual content of the weapon set by the first setting unit to the display device so as to have the attack power set by the second setting unit. Characterized by

According to the present invention, it is possible to provide a user with an experience in virtual space.

FIG. 1 is a diagram showing an example of an image displayed on a display of AR glasses. FIG. 2 is a diagram illustrating the configuration of the communication system according to the embodiment. FIG. 3 is a block diagram showing an example of the configuration of the communication system according to the embodiment. 4 is a block diagram showing an example of the configuration of the AR glasses shown in FIG. 3. FIG. FIG. 5 is a block diagram showing an example of the configuration of the smart device shown in FIG. 3. As shown in FIG. FIG. 6 is a block diagram showing an example of the configuration of the server shown in FIG. 3; FIG. 7 is a diagram illustrating an example of weapon setting by the user. FIG. 8 is a diagram for explaining an example of a user's peripheral situation. FIG. 9 is a diagram for explaining an example of a user's peripheral situation. FIG. 10 is a sequence diagram illustrating a processing procedure of communication processing according to the embodiment. FIG. 11 is a diagram showing a computer executing a program.

Hereinafter, embodiments of a communication system, a communication method, and a communication program according to the present application will be described in detail with reference to the drawings. Note that the communication system, communication method, and communication program according to the present application are not limited by this embodiment.

In the following embodiments, a communication system, a communication method, and a processing flow of a communication program according to the embodiments will be described in order, and finally effects of the embodiments will be described.

[Embodiment]
First, an embodiment will be described. In the embodiment, AR glasses capable of displaying virtual RPG content, for example, are used as the display device used by the user who is sightseeing in a predetermined area, specifically, a sightseeing spot (for example, an island). We will explain a system that allows you to experience RPG while wearing it and sightseeing in tourist spots.

FIG. 1 is a diagram showing an example of an image displayed on a display of AR glasses. As illustrated in FIG. 1, a monster content C1 and a coin content C2 are superimposed on an image of an actual island (for example, an image of a forest) and displayed on the AR glass, and the user defeats the monster and collects the coin. It enables game experiences such as collecting and working on missions. In addition, communication is possible between users wearing AR glasses. In the embodiment, the entire small island is used to enable the user to have an RPG experience unique to the area while moving their bodies and communicating with their friends.

Furthermore, in the embodiment, the motion controller held by the user is regarded as a weapon, and the user's movement is reflected in the game world by displaying the content of the weapon that moves according to the movement of the motion controller on the AR glasses. At this time, in the embodiment, in accordance with the movement of the user, virtual content is displayed on the AR glasses so that the attack power and the type of attack change, and in accordance with the surrounding conditions of the user, the weapon attack By changing the force, it provides a realistic AR experience.

[Configuration of communication system]
A configuration of a communication system according to an embodiment will be described. FIG. 2 is a diagram illustrating the configuration of the communication system according to the embodiment. FIG. 3 is a block diagram showing an example of the configuration of the communication system according to the embodiment.

As shown in FIGS. 2 and 3, the communication system 1 according to the embodiment includes an AR glass 10 (display device), a motion controller 20 held by a user, a smart device 30 carried by a user, a data center, and the like. has a server 40 located in

The AR glasses 10 are worn on the user's head like general eyeglasses. The AR glasses 10 have an imaging function, an audio input function, a communication function, a display function, and an audio output function. The AR glasses 10 capture an image of the real space, and display the captured image of the real space on a display corresponding to the lens portion of the AR glasses 10 . In addition, under the control of the smart device 30, the AR glasses 10 superimpose the virtual content generated by CG on the captured image of the real space and display it on the display. Under the control of the smart device 30, the AR glasses 10 also output virtual content according to the user's surroundings.

The motion controller 20 is a controller held by the user. The motion controller 20 has, for example, an acceleration sensor and a communication function for transmitting detection results of the acceleration sensor to the smart device 30 .

The smart device 30 is a device carried by a user, for example, a smart device such as a tablet or a smartphone, and has a position detection function and an image recognition function. The smart device 30 controls the AR glasses 10 . The smart device 30 connects to the server 40 via the network N and transmits the user's location information, image recognition results, and temperature information to the server 40 . The smart device 30 receives virtual content such as enemy character images and weapons output from the AR glasses 10 from the server 40 via the network N. FIG.

The smart device 30 acquires the motion of the user's hand based on the detection result of the acceleration sensor transmitted from the motion controller 20 . The smart device 30 controls output of virtual content provided by the server 40 to the AR glasses 10 . For example, the smart device 30 causes the AR glasses 10 to display the image content of the weapon so that the user holds the weapon in his/her hand, and the image content of the weapon moves according to the movement of the motion controller 20 held by the user. is displayed on the AR glass 10. At this time, the smart device 30 causes the AR glasses 10 to output the image content of the weapon so as to achieve the attack power set by the server 40 .

The server 40 holds map information and virtual content on the AR space. The server 40 transmits virtual content to be output to the AR glasses 10 to the smart device 30 . The server 40 sets for the user the virtual content of the type of weapon selected by the user. The server 40 determines the user's surroundings based on the location information of the smart device 30, map information, weather information, image recognition results from the smart device 30, and temperature information. Then, the server 40 sets the attack power of the virtual content of the weapon set for the user according to the user's surroundings.

[AR glass]
Next, the AR glasses 10 will be explained. FIG. 4 is a block diagram showing an example of the configuration of the AR glasses 10 shown in FIG. 3. As shown in FIG.

As shown in FIG. 4 , the AR glasses 10 have an imaging unit 11 , an audio input unit 12 (recording unit), a communication unit 13 , a display unit 14 and an audio output unit 15 .

The imaging unit 11 images the real space. For example, when the imaging direction of the imaging unit 11 is in front of the AR glasses 10 , the imaging unit 11 images the real space in front of the AR glasses 10 . The imaging unit 11 generates an image of a real space by processing a photoelectric conversion element such as a CCD or a CMOS and a photoelectric conversion signal output from the imaging element. The imaging unit 11 acquires a two-dimensional image of three colors of R (red), G (green), and B (blue), for example.

The voice input unit 12 is configured by a device that receives input of voice and records the input voice.

The communication unit 13 performs wireless communication with the smart device 30 . The communication unit 13 transmits the image captured by the imaging unit 11 and the audio recorded by the audio input unit 12 to the smart device 30 . The communication unit 13 receives virtual content transmitted from the smart device 30 . The virtual content includes character images of enemies and allies, images of weapons, voices of enemies and allies, and voices associated with actions of the user. Note that the character image is not limited to a game character-like image, and may be an image created from a photograph or the like, and includes moving images.

The display unit 14 is configured by a display. The display unit 14 displays the image of the real space captured by the imaging unit 11 . The display unit 14 displays the image content of the weapon among the virtual content received by the communication unit 13 by superimposing it on the image of the real space. The image content of the weapon is displayed by replacing it with the motion controller held by the user. Note that the display unit 14 may be a display capable of transmissive display, and may display image content.

The audio output unit 15 outputs audio. The audio output unit 15 outputs sounds such as animal barks and footsteps from the virtual content received by the communication unit.

[Smart device]
Next, the smart device 30 is described. FIG. 5 is a block diagram showing an example of the configuration of the smart device 30 shown in FIG. 3. As shown in FIG. As shown in FIG. 5 , the smart device 30 has a communication section 31 , a storage section 32 and a control section 33 .

The communication unit 31 performs wireless or wired communication with other devices. The communication unit 31 is a communication interface that transmits and receives various information to and from another device connected via a network or the like. The communication unit 31 is realized by a NIC (Network Interface Card) or the like, and performs communication between another device and the control unit 33 (described later) via an electric communication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit 31 receives images and sounds from the AR glasses 10 and transmits virtual content to the AR glasses 10 . The communication unit 31 transmits position information, image recognition results, and temperature information to the server 40 and receives virtual content from the server 40 .

The storage unit 32 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), an optical disc, or the like. Note that the storage unit 32 may be a rewritable semiconductor memory such as a RAM (Random Access Memory), a flash memory, or an NVSRAM (Non Volatile Static Random Access Memory). The storage unit 32 stores an OS (Operating System) and various programs executed by the smart device 30 . Furthermore, the storage unit 32 stores various information used in executing the program. The storage unit 32 stores image data 321 transmitted from the AR glasses 10 and content information 322 including virtual content such as enemies and weapons transmitted from the server 40 .

The control unit 33 controls the smart device 30 as a whole. The control unit 33 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 33 also has an internal memory for storing programs defining various processing procedures and control data, and executes each processing using the internal memory. Further, the control unit 33 functions as various processing units by running various programs. The control unit 33 has a position detection unit 331, an image recognition unit 332, a temperature detection unit 333, a motion detection unit 334, and an AR glasses control unit 335 (control unit).

The position detection unit 331 detects the position of the smart device 30 . The position detection unit 331 can acquire the position information of the smart device 30 by GPS (Global Positioning System), for example. The position detection unit 331 periodically acquires the position of the smart device 30 and transmits the position information represented by latitude and longitude to the server 40 via the communication unit 31 .

The image recognition unit 332 processes the image captured by the AR glasses 10 and recognizes features of the image such as an object appearing in the image. The image recognition unit 332 recognizes animals, plants, landscapes, and buildings appearing in images captured by the AR glasses 10 . The image recognition unit 332 performs image recognition that has already learned image features such as animals, plants, landscapes, and buildings that are specific to a predetermined area, specifically, a sightseeing spot that the user is currently sightseeing, by, for example, machine learning. Using the model, the features of the image captured by the AR glasses 10 are recognized. The image recognition unit 332 recognizes the brightness of the user's location from the image captured by the AR glasses 10 . The image recognition unit 332 may perform processing to recognize an animal, a plant, a landscape, or a building when there is a predetermined change or more in the image by comparing the images before and after the image. The image recognition section 332 transmits the image recognition processing result to the server 40 via the communication section 31 .

Note that the server 40 may have the image recognition function. At that time, the smart device 30 transmits the image captured by the AR glasses 10 to the server 40 .

The temperature detection unit 333 detects the temperature of the location where the smart device 30 is located, and transmits the detected temperature information to the server 40 via the communication unit 31 .

The motion detection unit 334 acquires the user's action, specifically, the motion of the user's hand, based on the detection result of the acceleration sensor transmitted from the motion controller 20 .

The AR glasses control unit 335 controls the processing of the AR glasses 10 . For example, the AR glasses control unit 335 controls image display processing on the display of the lens portion of the AR glasses 10 and audio output processing by the AR glasses 10 . The AR glasses control unit 335 transmits selection information such as weapons and equipment selected by the user to the server 40 via the communication unit 31 in the RPG experience. The AR glasses control section 335 has a content acquisition section 336 and a content synthesis section 337 .

The content acquisition unit 336 acquires virtual content transmitted from the server 40 . This virtual content is content of character images of enemies and allies to be superimposed on images in real space, content of weapon images, or content of voices of enemies and allies, etc., and voices accompanying actions of the user. When the content acquired by the content acquisition unit 336 is audio, the AR glasses control unit 335 causes the audio output unit 15 of the AR glasses 10 to output audio in accordance with the movement of the character images of enemies and allies and the user's actions. .

When the content acquired by the content acquisition unit 336 is a character image superimposed on a real space image, the content synthesizing unit 337 performs synthesis processing of the character image and the real space image. After the composition processing by the content composition unit 337, the AR glasses control unit 335 causes the display unit 14 of the AR glasses 10 to display an image in which the character image is superimposed on the image of the real space. The superimposition position of the character image is set in advance, for example. Alternatively, the character image superimposition position is set by an instruction from the server 40 .

The AR glasses control unit 335 controls the synthesis processing of the content synthesis unit 337, and displays the weapon image content on the AR glasses 10 so that the motion controller 20 held by the user can be seen as a weapon. Then, the AR glasses control unit 335 causes the AR glasses 10 to display the content of the weapon image so that the weapon moves according to the movement of the motion controller 20 . The AR glasses control unit 335 causes the AR glasses 10 to display virtual content such that the attack power and the type of attack change according to the movement of the user.

At that time, the AR glasses control unit 335 causes the AR glasses 10 to display virtual content that directs the user's action. For example, when fighting an enemy with a sword, the AR glasses control unit 335 causes the AR glasses 10 to display image content such that sparks appear when the sword collides with the enemy. The AR glasses control unit 335 causes the AR glasses 10 to output a sound as if the sword hit the enemy when fighting with the sword.

The AR glasses control unit 335 provides a realistic AR experience by changing the attack power of the weapon according to the user's surroundings. That is, the AR glasses control unit 335 causes the AR glasses 10 to output the virtual content of the weapon so as to have the attack power set by the attack power setting unit 433 (described later).

[server]
Next, the server 40 will be explained. FIG. 6 is a block diagram showing an example of the configuration of server 40 shown in FIG. As shown in FIG. 6 , the server 40 has a communication section 41 , a storage section 42 and a control section 43 .

The communication unit 41 performs wireless or wired communication with other devices. The communication unit 41 is a communication interface that transmits and receives various information to and from another device connected via a network or the like. The communication unit 41 is realized by a NIC (Network Interface Card) or the like, and performs communication between another device and the control unit 43 (described later) via an electric communication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit 41 receives location information, image recognition results, and voice recognition results from the smart device 30 and transmits virtual content to the smart device 30 .

The storage unit 42 is a storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), an optical disc, or the like. Note that the storage unit 42 may be a rewritable semiconductor memory such as a RAM (Random Access Memory), a flash memory, or an NVSRAM (Non Volatile Static Random Access Memory). The storage unit 42 stores an OS (Operating System) and various programs executed by the server 40 . The storage unit 42 stores various types of information used in program execution. The storage unit 42 stores map information 421 , content information 422 including various virtual content, and reference information 423 .

The map information 421 is a map including forests, rivers, mountains, plains, famous places, etc. of sightseeing spots such as islands that the user visits. The map information 421 indicates the elevation of the land by contour lines. The map information 421 may describe places that tend to be muddy, or may display roads by distinguishing between mountain roads and paved roads.

Various types of virtual content are character images of enemies and allies, images of weapons, voices of enemies and allies, or voices accompanying actions of the user. The image content of the weapon and the audio content accompanying the user's action are set according to each degree of offensive power. These contents may be contents peculiar to each predetermined place of a tourist spot, which are created in advance in association with each predetermined place such as a predetermined area, for example, a sightseeing spot or a famous place. For example, in the case of a famous place, the image of the symbol character of this famous place may be set as the enemy or ally content. The content information 422 includes, for example, setting conditions for various types of content, which are setting information such as the display position and size of character images.

The reference information 423 includes information indicating a reference situation in determination by the surrounding situation determination unit 432 (described later). The reference information 423 includes information indicating the reference attack power, which is the reference attack power. The reference conditions are, for example, a dry road, no slope, cloudy light, average temperature, and no rain. Also, the reference attack power is level "2" when the level is "1" to "3", for example.

The control unit 43 controls the server 40 as a whole. The control unit 43 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 43 also has an internal memory for storing programs defining various processing procedures and control data, and executes each processing using the internal memory. Further, the control unit 43 functions as various processing units by running various programs. The control unit 43 has a weapon content setting unit 431 (first setting unit), a surrounding situation determination unit 432 (determination unit), and an attack power setting unit 433 (second setting unit).

When the weapon content setting unit 431 acquires the user's weapon selection information from the smart device 30 , the weapon content setting unit 431 sets the virtual content of the type of weapon according to the user's selection in the user's smart device 30 .

The weapon content setting unit 431 sets, in the smart device 30, weapon image content that is set according to each level of attack power. The weapon content setting unit 431 transmits to the smart device 30 the image content of the weapon selected by the user and the content for rendering when the weapon is used. The image content of this weapon is set so that the attack power and type of attack are changed according to the user's movement. In addition, the image content of this weapon is set according to the attack power, for example, for each level of the attack power. For example, if the attack power level is "3", it is an image of a golden sword, if the attack power level is "2", it is an image of a metal sword, and if the attack power level is " 1", the image is of a rusted and chipped sword.

The peripheral situation determination unit 432 determines the user's peripheral situation based on at least the location information of the smart device 30, the map information 421, and the weather information. The surrounding situation determination unit 432 further uses the image recognition result of the image recognition unit 332 of the smart device 30 or the temperature information detected by the image recognition result and the temperature detection unit 333 to more accurately determine the user's surrounding situation. You can judge. The peripheral situation determination unit 432 determines whether the user's peripheral situation is better or worse than the reference situation.

For example, the reference conditions are a dry road, no slopes, cloudy weather, average temperature, and no rain. In the following cases, the peripheral situation determination unit 432 determines that the user's peripheral situation is worse than the reference situation. Specifically, this is the case where the surrounding situation determination unit 432 determines that the location where the user is located is a muddy and bad road based on the map information 421, weather information, and image recognition results. Alternatively, the surrounding situation determination unit 432 determines that the location where the user is located is a sloped location based on the contour lines of the map information 421 . Alternatively, the surrounding situation determination unit 432 determines that the location where the user is located has poor visibility and is dark based on the image recognition result, and the temperature is lower than the average temperature based on the temperature information. Alternatively, it is the case where the surrounding situation determination unit 432 determines that it is raining based on the weather information and the image recognition result.

In addition, the peripheral situation determination unit 432 determines that the user's peripheral situation is better than the reference situation in the following cases. For example, this is the case where the surrounding situation determination unit 432 determines that the location where the user is located is a paved road based on the map information 421, weather information, and image recognition results. Alternatively, it is a case where the surrounding situation determination unit 432 determines that the place is flat based on the contour lines of the map information 421 . Alternatively, based on the image recognition result, the surrounding situation determination unit 432 determines that the visibility is good, it is bright, and the temperature is the average temperature based on the temperature information, or the weather information and the image recognition result indicate that it is sunny. This is the case when the surrounding situation determination unit 432 determines.

The attack power setting unit 433 sets the attack power of the virtual content of the weapon set for the user according to the surrounding situation of the user. The attack power setting unit 433 sets the attack power of the weapon set for the user lower than the reference attack power when the surrounding situation of the user is worse than the reference situation. Also, if the user's peripheral situation is better than the reference situation, the attack power setting unit 433 sets the attack power of the weapon set for the user higher than the reference attack power.

[Weapon setting example by user]
FIG. 7 is a diagram illustrating an example of weapon setting by the user. For example, when the user U1 selects a sword and shield as weapons, the server 40 transmits image content of the sword and shield selected by the user U1 to the smart device 30 of the user U1. The server 40 also transmits contents of other equipment to the smart device 30 of the user U1 according to the user U1's selection. It should be noted that the user has two motion sensors 20 when selecting sword and shield. Alternatively, if the user has one motion sensor 20, the movement of the one motion sensor 20 may be used separately for the sword and the shield with a predetermined setting. Alternatively, if the user has one motion sensor 20 , either the sword or the shield may be configured not to be linked with the motion sensor 20 .

In the example of FIG. 7, the smart device 30 causes the AR glasses 10 to display image content of a sword and a shield so that the user U1 holds a sword in his right hand and a shield in his left hand. Then, when the user U1 waves his or her right hand vertically, the smart device 30 causes the AR glasses 10 to display an action image in which a character such as a monster is cut vertically. At this time, when the user U1 swings the sword quickly, the smart device 30 increases the attack power of the sword, and furthermore, as an effect, displays image content for effect such as sparks or glittering light. Display on the AR glass 10. Also, when the user U1 waves his or her right hand sideways, the smart device 30 causes the AR glasses 10 to display an action image in which a character such as a monster is mowed down.

Then, the communication system 1 changes the offensive power of the weapon in accordance with the surrounding circumstances of the user U1. 8 and 9 are diagrams for explaining an example of the surrounding situation of the user. As shown in FIG. 8, when the user U1 is on a road in a dark, muddy forest, the communication system 1 determines that the user's surrounding conditions are worse than the reference conditions. Then, in the communication system 1, the attack power of the sword selected by the user U1 is set to level "1", which is lower than the reference attack power (for example, level "2"), according to the degree of deterioration of the surrounding situation. As a result, the image content C11 of a sword with only an attack power of level "1" is displayed on the AR glasses 10 of the user U1 so as to be held in the hand of the user U1.

Also, as shown in FIG. 9, when the user U1 is on a paved road with a good view under fine weather, the communication system 1 determines that the surrounding conditions of the user are better than the reference conditions. Then, in the communication system 1, the attack power of the sword selected by the user U1 is set to level "3", which is higher than the reference attack power, according to the surrounding situation. As a result, the image content C12 of the golden sword of level "3" is displayed on the AR glasses 10 of the user U1 so as to be held in the hand of the user U1.

In this way, the communication system 1 changes the attack power of the weapon held by the user in the AR space according to the user's actual surroundings.

[Processing procedure of communication processing]
FIG. 10 is a sequence diagram illustrating a processing procedure of communication processing according to the embodiment.

As shown in FIG. 10, when the user selects a weapon by operating the smart device 30 (step S1), the smart device 30 transmits weapon information indicating the weapon selected by the user to the server 40. (step S2). In response to this, the server 40 sets the virtual content of the weapon type according to the user's selection to the user's smart device 30 (step S3), and sets the virtual content of this weapon to the smart device 30. Send (step S4).

Each time the smart device 30 detects the location of the smart device 30 (step S5), it transmits the location information to the server 40 (step S6). Then, when the image is transmitted from the AR glasses 10 (step S7), the smart device 30 executes image recognition processing for the image (step S8), and transmits the image recognition result to the server 40 (step S9). The smart device 30 detects the temperature (step S10), and transmits temperature information indicating the detected temperature to the server 40 (step S11).

Server 40 further refers to weather information (step S12). The server 40 determines the surrounding situation of the user (step S13). The server 40 uses at least the location information of the smart device 30, the map information 421, and the weather information for determination. The server 40 may also use the image recognition result of the image recognition unit 332 of the smart device 30 or the temperature information detected by the image recognition result and the temperature detection unit 333 . The server 40 sets the attack power of the virtual content of the weapon set for the user according to the user's surroundings (step S14). The server 40 transmits information indicating the set attack power to the smart device 30 (step S15).

The motion controller 20 detects acceleration using the acceleration sensor (step S16), and transmits the detected acceleration information to the smart device 30 (step S17). The smart device 30 detects the motion of the user's hand based on the acceleration information transmitted from the motion controller 20 (step S18).

The smart device 30 synthesizes the transmitted image content of the weapon with the image of the real space according to the movement detected in the movement detection process (step S18) so as to achieve the attack power set by the server 40. Synthesis processing is performed (step S19). Then, the smart device 30 instructs the AR glasses 10 to output content (step S20), and virtual content in the AR space (for example, image content of enemies, allies, and weapons, content for production, audio content, etc.) ) is output from the AR glasses 10 (step S21).

[Effects of Embodiment]
As described above, in the present embodiment, by changing the attack power of the weapon held by the user in the AR space according to the actual surrounding situation of the user, it is possible to make the game more realistic as if the user were actually fighting an enemy in the real environment. A user can be provided with a realistic AR experience in a virtual space.

[System configuration, etc.]
Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device is realized by a CPU or GPU and a program analyzed and executed by the CPU or GPU, or as hardware by wired logic can be realized.

Further, among the processes described in this embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be done automatically by known methods. In addition, information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

[program]
It is also possible to create a program in which the processes executed by the AR glasses 10, the smart device 30, and the server 40 described in the above embodiments are described in a computer-executable language. For example, it is possible to create a program in which the processes executed by the AR glasses 10, the smart device 30, and the server 40 in the embodiment are described in a computer-executable language. In this case, the same effects as those of the above embodiments can be obtained by having the computer execute the program. Further, such a program may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read by a computer and executed to realize processing similar to that of the above embodiments.

FIG. 11 is a diagram showing a computer executing a program. As illustrated in FIG. 11, computer 1000 includes, for example, memory 1010, CPU 1020, hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. , and these units are connected by a bus 1080 .

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012 as illustrated in FIG. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). Hard disk drive interface 1030 is connected to hard disk drive 1090 as illustrated in FIG. A disk drive interface 1040 is connected to the disk drive 1100 . A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100 . Serial port interface 1050 is connected to mouse 1110 and keyboard 1120, for example. Video adapter 1060 is connected to display 1130, for example.

Here, as illustrated in FIG. 11, the hard disk drive 1090 stores an OS 1091, application programs 1092, program modules 1093, and program data 1094, for example. That is, the above program is stored, for example, in hard disk drive 1090 as a program module in which instructions to be executed by computer 1000 are described.

Various data described in the above embodiments are stored as program data in the memory 1010 or the hard disk drive 1090, for example. Then, the CPU 1020 reads the program modules 1093 and program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as necessary, and executes various processing procedures.

Note that the program module 1093 and program data 1094 related to the program are not limited to being stored in the hard disk drive 1090. For example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive or the like. . Alternatively, the program module 1093 and program data 1094 related to the program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.), and are transmitted via the network interface 1070. It may be read by CPU 1020 .

The above-described embodiments and modifications thereof are included in the scope of the invention described in the claims and their equivalents, as well as in the technology disclosed in the present application.

1 communication system 10 AR glasses 11 imaging unit 12 audio input unit 13, 31, 41 communication unit 14 display unit 15 audio output unit 20 motion controller 30 smart device 32, 42 storage unit 33, 43 control unit 40 server 321 image data 322, 422 content information 331 position detection unit 332 image recognition unit 333 temperature detection unit 334 motion detection unit 335 AR glasses control unit 336 content acquisition unit 337 content synthesis unit 421 map information 431 weapon content setting unit 432 surrounding situation determination unit 433 attack power setting unit

Claims (4)

A spectacles-type display device worn by a user who is sightseeing in a predetermined area, a motion controller held by the user, a smart device carried by the user, and a server communicating with the smart device, wherein virtual content is provided. is displayed on the display device to provide the user with a game experience,
The display device
an imaging unit that captures an image of a real space;
a communication unit that communicates with the smart device and transmits an image of the real space to the smart device;
a display unit that superimposes and displays virtual content on the image of the real space;
has
The server is
a first setting unit that sets virtual content of a type of weapon selected by the user in the smart device;
a determination unit that determines a surrounding situation of the user based on at least location information of the smart device, map information, and weather information;
a second setting unit that sets the offensive power of the virtual content of the weapon set for the user according to the surrounding situation of the user;
has
The smart device is
a location detection unit that detects at least location information of the smart device and transmits the detected location information to the server;
a control unit that causes the display device to output the virtual content of the weapon set by the first setting unit so as to have the attack power set by the second setting unit;
has
The communication system shall:
an image recognition unit that recognizes features of the image in the real space;
has
The determination unit determines the surrounding situation of the user based on location information of the smart device, map information, weather information, and an image recognition result of the image recognition unit,
When the user is on a road in a dark and muddy forest, the determination unit determines that the surrounding situation of the user is worse than a reference situation, and reduces the attack power of the weapon set for the user. , set lower than the standard attack power,
When the user is on a paved road with a good view under fine weather, the determination unit determines that the surrounding situation of the user is better than the reference situation, and sets the attack power of the weapon set for the user to Set higher than the standard attack power
A communication system characterized by: The smart device is
a temperature detection unit that detects the temperature of a location where the smart device is located and transmits the detected temperature information to the server;
2. The communication according to claim 1 , wherein the judgment unit judges the surrounding situation of the user based on at least location information of the smart device, map information, weather information, and the temperature information. system. A communication method executed by a communication system having a spectacle-type user-used display device worn by a user who is sightseeing in a predetermined area, a smart device carried by the user, and a server that communicates with the smart device. ,
a step in which the display device captures an image of a real space;
a step of the display device communicating with the smart device and transmitting an image of the real space to the smart device;
a step in which the display device superimposes and displays virtual content on the image of the real space;
a first setting step in which the server sets virtual content of a type of weapon selected by the user in the smart device;
a step in which the server determines a surrounding situation of the user based on at least location information of the smart device, map information, and weather information;
a second setting step in which the server sets the attack power of the virtual content of the weapon set for the user according to the surrounding circumstances of the user;
the smart device detecting at least location information of the smart device and transmitting the detected location information to the server;
a step of causing the smart device to output the virtual content of the weapon set in the first setting step to the display device so as to have the attack power set in the second setting step;
the communication system recognizing features of the real space image;
including
the determining step determines the user's surroundings based on location information of the smart device, map information, weather information, and an image recognition result in the recognizing step;
In the determining step, when the user is on a road in a dark and muddy forest, it is determined that the surrounding situation of the user is worse than a reference situation, and the attack power of the weapon set for the user is determined. is set lower than the standard attack power,
In the step of determining, when the user is on a paved road with a fine view under fine weather, it is determined that the surrounding situation of the user is better than the reference situation, and the offensive power of the weapon set for the user is reduced. , a communication method characterized in that the attack power is set higher than the reference attack power . A communication program that causes a computer to perform the method, comprising:
A computer as a spectacle-type display device worn by a user who is sightseeing in a predetermined area,
capturing an image of real space;
communicating with a smart device and transmitting the image of the real space to the smart device;
a step of superimposing and displaying virtual content on the image of the real space;
computer as a server,
a first setting step of setting, in a smart device carried by the user, virtual content of the type of weapon selected by the user;
determining the surrounding situation of the user based on at least location information of the smart device, map information, and weather information;
a second setting step of setting the attack power of the virtual content of the weapon set for the user according to the user's surroundings;
recognizing features of the real space image;
and
a computer as the smart device;
detecting location information of at least the smart device and transmitting the detected location information to the server;
a step of outputting the virtual content of the weapon set in the first setting step to a display device used by a user so as to have the attack power set in the second setting step;
and
the determining step determines the surrounding situation of the user based on location information of the smart device, map information, weather information, and an image recognition result in the recognizing step;
In the determining step, when the user is on a road in a dark and muddy forest, it is determined that the surrounding situation of the user is worse than a reference situation, and the attack power of the weapon set for the user is determined. is set lower than the standard attack power,
In the determining step, if the user is on a paved road with a good view under fine weather, it is determined that the surrounding situation of the user is better than the reference situation, and the attack power of the weapon set for the user is reduced. , a communication program characterized in that the attack power is set higher than the standard attack power .

Images