JP7158649B2 - Server device, information processing system and program - Google Patents

Description

The present disclosure relates to a server device, an information processing system, and a program.

In recent years, technologies for displaying objects including real estate and movable properties such as buildings using AR (Augmented Reality) and VR (Virtual Reality) technologies for user terminals such as smartphones have been used. . For example, Patent Literature 1 discloses a technique for displaying an object in an augmented reality space on a user terminal. Further, Patent Literature 2 discloses a technique for displaying an object in a virtual reality space on a user terminal.

Further, Patent Literature 3 discloses an AR display management system capable of regulating AR display.

JP 2013-84313 A JP 2020-149399 A JP 2019-87091 A

In conventional augmented reality and virtual reality technologies, there is a problem that an additional image cannot be added to an image of a real space or a virtual space and displayed.

An object of the present disclosure is to provide a server device, an information processing system, and a program for synthesizing an additional image and displaying it on a user terminal.

The first aspect is
A server device comprising a computer that functions as reception means, setting means, and instruction means by executing a program,
The receiving means receives setting information related to setting of display of the additional image from the first terminal,
The setting means sets a display mode of the additional image on the user's second terminal based on the setting information received by the receiving means,
The instruction means is a server device that causes the second terminal to display an image obtained by selectively synthesizing the additional image in the display mode set by the setting means.

Also, in the first aspect,
When the image of the surrounding environment detected by the detection unit of the second terminal is displayed on the second terminal, the instruction means displays the additional image in the display mode set by the setting means. The image can be selectively combined with the image and displayed on the second terminal.

Also, in the first aspect,
the setting information includes information about an object to which the additional image is to be added;
The setting means adds the additional image to the object in the display mode set by the setting means when the object is included in the image of the surrounding environment detected by the detection unit of the second terminal. can be displayed on the second terminal in this state.

Also, in the first aspect,
the receiving means receives from the second terminal information about the surrounding environment of the second terminal detected by the detecting unit of the second terminal as surrounding environment information;
The setting means can set a display mode of the additional image on the second terminal based on the setting information and the surrounding environment information received by the receiving means.

Also, in the first aspect,
the reception means receives information about the current location of the second terminal from the second terminal as current location information;
The setting means can set the display mode of the additional image on the second terminal based on the setting information and the current position information received by the receiving means.

Also, in the first aspect,
The receiving means receives information about the virtual space,
The instruction means selects the additional image as the image of the virtual space in the display mode set by the setting means when causing the second terminal to display the image of the virtual space received by the reception means. , and displayed on the second terminal.

Also, in the first aspect,
The setting information includes information on setting whether or not to display the additional image on the second terminal,
When the setting information is information for displaying the additional image on the second terminal, the setting means may set the display mode to display the additional image on the second terminal. .

Also, in the first aspect,
the receiving means receives attribute information about the user who owns the second terminal;
The setting means can also refer to the attribute information received by the receiving means to set the display mode of the additional image on the second terminal.

Also, in the first aspect,
The setting means can also refer to date and time information regarding the current date and time to set the display mode of the additional image on the second terminal.

The second aspect is
A server device comprising a computer that functions as reception means, setting means, determination means, and instruction means by executing a program,
The reception means receives intellectual property information related to the intellectual property of the object,
The setting means sets a display mode of the additional image added to the object on the user's second terminal,
It is appropriate that the determination means causes the second terminal to display the additional image added to the object in the display mode set by the setting means based on the received intellectual property information. determine whether there is
The display mode set by the setting means when the determination means determines that it is appropriate to display the additional image on the second terminal in a state where the additional image is added to the object. and displaying the additional image added to the object on the second terminal.

Also, in the second aspect,
When the image of the surrounding environment detected by the detection unit of the second terminal is displayed on the second terminal, the instruction means displays the additional image in the display mode set by the setting means. The image can be synthesized with the image and displayed on the second terminal.

Also, in the second aspect,
the receiving means receives from the second terminal information about the surrounding environment of the second terminal detected by the detecting unit of the second terminal as surrounding environment information;
The determining means adds the additional image to the object in the surrounding environment information received by the receiving means in the display mode set by the setting means based on the intellectual property information. It is possible to determine whether or not it is appropriate to display on the second terminal in this state.

Also, in the second aspect,
The determination means refers to the extent to which the object is included in the surrounding environment information received by the reception means, and adds the additional image to the object in the display mode set by the setting means. , it can be determined whether or not it is appropriate to display on the second terminal.

Also, in the second aspect,
the reception means receives information about the current location of the second terminal from the second terminal as current location information;
Based on the intellectual property information and the current position information received by the receiving means, the determining means adds the additional image to the object in the display mode set by the setting means, and It can be determined whether or not it is appropriate to display on two terminals.

Also, in the second aspect,
The receiving means receives information about the virtual space,
The instruction means, when causing the second terminal to display the image of the virtual space received by the reception means, synthesizes the additional image with the image of the virtual space in the display mode set by the setting means. can be displayed on the second terminal.

Also, in the second aspect,
the receiving means receives information about a display mode of the second terminal of the user as display mode information;
Based on the intellectual property information and the display mode information received by the receiving means, the determining means adds the additional image to the object in the display mode set by the setting means, and It can be determined whether or not it is appropriate to display on two terminals.

The third aspect is
An information processing system comprising reception means, setting means, and instruction means,
The receiving means receives setting information related to setting of display of the additional image from the first terminal,
The setting means sets a display mode of the additional image on the user's second terminal based on the setting information received by the receiving means,
The instruction means is an information processing system that causes the second terminal to display an image obtained by selectively synthesizing the additional image in the display mode set by the setting means.

The fourth aspect is
An information processing system comprising reception means, setting means, determination means, and instruction means,
The reception means receives intellectual property information related to the intellectual property of the object,
The setting means sets a display mode of the additional image added to the object on the user's second terminal,
It is appropriate that the determination means causes the second terminal to display the additional image added to the object in the display mode set by the setting means based on the received intellectual property information. determine whether there is
The display mode set by the setting means when the determination means determines that it is appropriate to display the additional image on the second terminal in a state where the additional image is added to the object. and displaying the added image on the second terminal in a state in which the added image is added to the object.

The fifth aspect is
A program that causes a computer to function as reception means, setting means, and instruction means,
The receiving means receives setting information related to setting of display of the additional image from the first terminal,
The setting means sets a display mode of the additional image on the user's second terminal based on the setting information received by the receiving means,
The instruction means is a program that causes the second terminal to display an image obtained by selectively synthesizing the additional image in the display mode set by the setting means.

The sixth aspect is
A program that causes a computer to function as reception means, setting means, determination means, and instruction means,
The reception means receives intellectual property information related to the intellectual property of the object,
The setting means sets a display mode of the additional image added to the object on the user's second terminal,
It is appropriate that the determination means causes the second terminal to display the additional image added to the object in the display mode set by the setting means based on the received intellectual property information. determine whether there is
The display mode set by the setting means when the determination means determines that it is appropriate to display the additional image on the second terminal in a state where the additional image is added to the object. and causing the second terminal to display the additional image added to the object.

According to the present disclosure, an additional image can be synthesized and displayed on a user terminal.

1 is a block diagram showing the configuration of an information processing system in this embodiment; FIG. FIG. 10 is a diagram showing the surrounding environment and an image displayed on the AR device when the user wears the AR device; FIG. 4 is a diagram showing an image of virtual space displayed on the monitor of the computer used by the user; FIG. 10 is a flowchart showing a series of operations when an additional image is displayed on a user's AR device or computer in a display mode set by setting means; FIG. 10 is a flowchart showing a series of operations when displaying an additional image that is determined to be appropriate based on intellectual property information on a user's AR device or computer;

[Embodiment]
An information processing system 1 according to an embodiment of the present disclosure will be described with reference to the drawings.

<Description of information processing system 1>
In the information processing system 1 shown in FIG. 1, the server device 2 (computer), the AR device 3 (second terminal) used by the user, the computer 4 (second terminal) used by the user, and the display of additional images are set. A computer 5 (first terminal) used by a person (hereinafter also referred to as a setter) is connected via a communication network 6 so as to be able to communicate with each other.

Objects include buildings existing in the real space, billboards for buildings, monuments, movable properties, and the like. Buildings include, for example, detached houses, commercial buildings, condominiums, and the like. Advertising signs for buildings include those installed on the roofs of buildings, those installed on side walls of buildings, and the like. Monuments include monuments, bronze statues, and the like. Movable property includes vehicles, clothing, ornaments, and the like.

Moreover, the target object is not limited to an object that exists in the real space, and includes a virtual building that exists in the virtual space, an advertising billboard for the building, a monument, a movable property, and the like. Objects that exist in the virtual space may be replicas of real estate such as buildings or movable properties such as cars that actually exist, or may be real or movable properties that do not actually exist.

The additional image is an image of the real space (including an image recognized by the naked eye and an image captured by a camera) displayed on the display unit 38 of the AR device 3, which will be described later, when the user uses the AR device 3. This is an image to be synthesized. The additional image is an image that does not exist in the real space, and includes, for example, an image displayed on the side of a building on the display unit 38 of the AR device 3, an image of an advertisement posted on an advertising signboard, or the like. Also, the additional image may be an image of an independent advertising signboard that does not exist in the real space.

The additional image is an image that is synthesized with the image of the virtual space displayed on the display unit 48 of the computer 4 (to be described later) when the user uses the computer 4 . The additional image is an image that is formed from various data such as map data and building data and does not exist in the virtual space. Includes images of advertisements, etc., or images of independent billboards themselves.

The setter generally holds the rights to the object. For example, when the object is a building, the setter is the building owner, lessee, or the like. If the building is, for example, a commercial building or condominium, a designated person may be assigned to each floor.

Further, when the object is an advertising signboard for a building or a vehicle, the setter is a person who has the right to post an advertisement on the advertising signboard. Also, when the object is a monument, the setter is the owner or manager of the monument. Also, when the object is a movable property, the setter is the owner or lessee of the movable property. Each setter is assigned identification information such as an identification number of several digits or several tens of digits.

Also, when the target object is an object in the virtual space (for example, a building object, an advertising billboard object), the setter is a person who has the right to post an advertisement on the object. Specifically, the creator who created and provided the virtual space gives the setter the right to post advertisements by transferring the right to present advertisements for objects in the virtual space.

Also, even if the person does not have the right to the object, the person who sets the display of the additional image may be the setter. For example, when synthesizing an image of an independent advertising signboard that does not exist in the real space or the virtual space as an additional image and displaying it on the user's AR device 3 or computer 4, the configurator determines the display of such an additional image. You are the one who makes the settings. That is, the target object may be a movable property existing in the physical space, or a predetermined area in the physical space.

The AR device 3 is portable by the user and can display an image of the surrounding environment for the user to view. Specifically, the AR device 3 captures an image of the surrounding environment by means of the surrounding detection unit 35, which will be described later, and acquires the image. to display. The AR device 3 includes, for example, AR glasses that can be worn by the user, or a smart phone, a tablet terminal, or the like that can be carried by the user.

<Overview of information processing system 1>
The server device 2 (computer) accepts setting information related to setting of display of the additional image from the computer 5 of the configurator, and sets the display mode of the additional image on the user's AR device 3 or computer 4 based on the received setting information. do. Specifically, as the display mode of the additional image, setting is made as to whether or not to display the additional image on the user's AR device 3 or the computer 4 by synthesizing the additional image with the image of the real space or the virtual space.

Then, the server device 2 selectively synthesizes the additional image in the set display mode and displays it on the user's AR device 3 or computer 4 . At this time, if a display mode is set in which the additional image is synthesized and not displayed on the user's AR device 3 or computer 4, the user's AR device 3 or computer 4 may display the real space or the virtual space itself. An image is displayed.

The information processing system 1 of this embodiment is applicable to both augmented reality (AR) and virtual reality (VR) technologies. When the information processing system 1 of the present embodiment is applied to augmented reality (AR) technology, an image obtained by synthesizing an additional image with an image of the surrounding environment of the AR device 3 is displayed on the display unit 38 of the AR device 3 owned by the user. can be displayed.

Further, when the information processing system 1 of the present embodiment is applied to virtual reality (VR) technology, when the image of the virtual space is displayed on the display unit 48 of the computer 4, an additional image is synthesized with the image of the virtual space. The resulting image can be displayed on the display unit 48 .

<Configuration of information processing system 1>
The hardware configurations of the server device 2, the user's AR device 3 and computer 4, and the setter's computer 5 will be described below with reference to FIG.

<Configuration of Server Apparatus 2>
As shown in FIG. 1 , the server device 2 has a network interface 21 , a control section 22 and a storage section 23 . Network interface 21 and storage unit 23 are electrically connected to control unit 22 via bus 24 .

The network interface 21 is communicably connected to the user's AR device 3 and computer 4 and the configurator's computer 5 via a communication network 6 such as the Internet and a LAN.

The control unit 22 is composed of a microcomputer including a CPU and semiconductor memory, and controls the operation of the server device 2 .

The control unit 22 functions as a reception unit 221, a setting unit 222, a determination unit 223, and an instruction unit 224 by executing various programs stored in the storage unit 23, which will be described later.

The receiving unit 221 receives information from the user's AR device 3 and computer 4, and from the configurator's computer 5, respectively. In the present embodiment, the reception unit 221 receives setting information regarding display settings of the additional image from the computer 5 of the configurator. The setting information includes information on setting whether or not to display the additional image on the user's AR device 3 or computer 4 . The setting information also includes a region where an additional image is added to the object and the content of the additional image.

In addition, the receiving means 221 also receives the identification information of the configurator when receiving the setting information from the computer 5 of the configurator. The configurator identification information includes at least one of the configurator's name, identification number, telephone number, and address.

Also, the receiving unit 221 receives attribute information about the user from the computer 5 of the configurator or the AR device 3 or computer 4 of the user. User attribute information includes, for example, information such as the user's age, gender, occupation, hobbies, and preferences.

The setting unit 222 sets the display mode of the additional image on the user's AR device 3 or computer 4 based on the setting information received by the receiving unit 221 . Specifically, the setting means 222 sets whether or not to display the additional image on the user's AR device 3 or computer 4, based on the setting information sent from the computer 5 of the configurator. The content of the additional image displayed on the computer 4 is set.

The determination means 223 determines whether or not the identification information received by the reception means 221 corresponds to the object to which the additional image is added, based on correspondence information stored in the storage unit 23, which will be described later. .

The instruction means 224 causes the user's AR device 3 or the computer 4 to display an image obtained by selectively synthesizing the additional image with the image of the surrounding environment in the display mode set by the setting means 222 .

Specifically, when the user uses the AR device 3, the instruction unit 224 displays an image of the surrounding environment detected by the surrounding detection unit 35 of the AR device 3, which will be described later, on the display unit 38, which will be described later. , the additional image is selectively combined with the image of the surrounding environment in the display mode set by the setting means 222 and displayed on the AR device 3 .

Also, when the user uses the computer 4, the reception means 221 receives information on the virtual space in advance. Information received by the receiving means 221 is stored in the storage unit 23 . Then, the instructing means 224 causes the computer 4 to display the image of the virtual space based on the information of the virtual space received by the receiving means 221 (that is, the information stored in the storage unit 23). The computer 4 is caused to display an image obtained by selectively synthesizing the additional image with the image of the virtual space in the set display mode.

The storage unit 23 includes a HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), or the like.

The storage unit 23 stores information on objects such as buildings existing in the real space and information on objects in the virtual space. Specifically, the storage unit 23 stores various data such as map data and building data in the real world, and various data such as map data and building data in the virtual space.

The storage unit 23 also stores information about correspondence between the identification information of the configurator and the object as correspondence information. Such correspondence information is stored in the storage unit 23 by being received from the external device by the receiving means 221 .

<Configuration of user's AR device 3>
As shown in FIGS. 1 and 2, the user's AR device 3 includes a network interface 31, a control unit 32, a graphics processing unit 33, an operation unit 34, a surrounding detection unit 35, a position detection unit 36, a storage unit 37, and a display unit. 38. The network interface 31 , graphic processing unit 33 , operation unit 34 , peripheral detection unit 35 , position detection unit 36 and storage unit 37 are electrically connected to control unit 32 via bus 39 .

The network interface 31 is communicably connected to the communication network 6 in order to transmit and receive various data between the AR device 3 and the server device 2 .

The graphics processing unit 33 creates an image by selectively synthesizing an additional image with an image of the surrounding environment detected by the surrounding environment detection unit 35 (described later) according to image information output from the control unit 32 (described later). The created image is displayed on the display unit 38 and can be visually recognized by the user.

The operation unit 34 is a switch or the like operated by the user, and the content of the operation by the user is input to the control unit 32 . The contents of the operation by the user are a command to turn on/off the power, and a command to switch whether or not to synthesize the additional image with the image of the surrounding environment in the display mode set by the setting means 222 .

The surrounding detection unit 35 captures an image of the surrounding environment of the AR device 3, and is composed of, for example, a camera, a sensor, and the like. An image of the surrounding environment of the AR device 3 captured by the surrounding detection unit 35 is input to the control unit 32 .

The position detection unit 36 detects the current position of the AR device 3, and is composed of GPS or the like, for example. Information on the current position of the AR device 3 detected by the position detection unit 36 is input to the control unit 32 .

The storage unit 37 includes a HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), or the like. The storage unit 37 stores identification information of the AR device 3 and attribute information about users. Further, the storage unit 37 stores programs executed by the control unit 32 .

When the AR device 3 is used as AR glasses (smart glasses), the display unit 38 includes a small monitor corresponding to the binocular lenses of the AR glasses. Also, the display unit 38 includes a monitor such as a touch panel when the AR device 3 is used as a smartphone or a tablet terminal.

The control unit 32 is composed of a microcomputer including a CPU and a semiconductor memory, and controls operations of the AR device 3 . The control unit 32 functions as communication means 321 , reception means 322 and display control means 323 by executing various programs stored in the storage unit 37 .

The communication means 321 is a function of communicating with the server device 2 via the network interface 31 . The communication unit 321 transmits to the server device 2 an image of the surrounding environment of the AR device 3 detected by the surrounding detection unit 35 and information about the current position of the AR device 3 detected by the position detection unit 36 . The communication unit 321 also receives information about the display mode of the additional image sent from the server device 2 .

The receiving unit 322 receives an image of the surrounding environment of the AR device 3 detected by the surrounding detection unit 35 and information about the current position of the AR device 3 detected by the position detection unit 36 . Further, the receiving unit 322 receives information regarding the display mode of the additional image transmitted from the server device 2 to the AR device 3 .

The display control unit 323 controls the image displayed on the display unit 38 by controlling the graphics processing unit 33 based on the information regarding the display mode of the additional image received by the receiving unit 322 .

<Configuration of User's Computer 4>
As shown in FIGS. 1 and 3, the user's computer 4 has a network interface 41 , a control section 42 , a graphics processing section 43 , an audio processing section 44 , an operation section 45 , a storage section 46 and a display section 48 . The network interface 41 , graphic processing section 43 , audio processing section 44 , operation section 45 and storage section 46 are electrically connected to the control section 42 via a bus 49 .

The network interface 41 is communicably connected to the communication network 6 in order to transmit and receive various data between the computer 4 and the server device 2 .

The graphics processing unit 43 creates an image by selectively synthesizing the additional image with the image of the surrounding environment transmitted from the server device 2 according to the image information output from the control unit 42, which will be described later. The created image is displayed on the display unit 48 and can be visually recognized by the user.

The audio processing unit 44 is connected to a speaker (not shown) of the computer 4, creates sound according to instructions from the control unit 42, which will be described later, and outputs the sound from the speaker.

The operation unit 45 is a keyboard or the like operated by the user, and the contents of operations by the user are input to the control unit 42 . The content of the operation by the user includes, for example, an operation of instructing movement within the virtual space displayed on the display unit 48 .

The storage unit 46 includes a HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), or the like. The storage unit 46 stores identification information of the computer 4 and attribute information about users. Further, the storage unit 46 stores programs to be executed by the control unit 42 .

The display unit 48 includes a monitor such as a head mounted display. Note that the display unit 48 is not limited to a head-mounted display, and other types of monitors (for example, desktop displays) may be used as long as they can display a virtual space.

The control unit 42 is composed of a microcomputer including a CPU and semiconductor memory, and controls the operation of the computer 4 . The control unit 42 functions as communication means 421 , reception means 422 and display control means 423 by executing various programs stored in the storage unit 46 .

The communication means 421 has a function of communicating with the server device 2 via the network interface 41 . The communication unit 421 transmits to the server device 2 information about the content of the user's operation input through the operation unit 45 . The communication unit 421 also receives map data and building data of the virtual space sent from the server device 2 and information on the display mode of the additional image.

The accepting unit 422 accepts the content of the user's operation input through the operation unit 45 . In addition, the receiving means 422 receives map data and building data of the virtual space transmitted from the server device 2 to the computer 4 and information regarding the display mode of the additional image.

The display control unit 423 controls the graphics processing unit 43 based on the map data and building data of the virtual space received by the receiving unit 422 and the information regarding the display mode of the additional image, thereby controlling the image displayed on the display unit 48. to control.

The configurator's computer 5 has substantially the same configuration as the user's computer 4 . Therefore, description of the detailed configuration of the configurator's computer 5 will not be repeated.

<Flow of Series of Operations of Information Processing System 1>
Next, the flow of a series of operations of the information processing system 1 will be described with reference to FIG.

First, a case where the information processing system 1 of the present embodiment is applied to augmented reality (AR) technology will be described. First, the setting person uses the computer 5 to set the display of the additional image. As a setting for displaying such an additional image, adding an additional image to an object existing in the real space, and adding a new additional image regardless of the object existing in the real space. There are two methods of

Specifically, when the setting person inputs a command to start setting the display of the additional image to the computer 5 , the computer 5 transmits a setting start command to the server device 2 . When the server device 2 receives the setting start command, the server device 2 transmits to the computer 5 the information on the building or the like existing in the physical space stored in the storage unit 23 . As a result, the information about the building or the like existing in the real space is displayed on the monitor of the computer 5 .

Then, when the configurator selects an object to which the additional image is to be added from buildings, etc. existing in the real space displayed on the monitor, the setting information for adding the additional image to the selected object is sent to the computer. 5 can be entered. Specifically, when the configurator selects a building as an object, the computer 5 can add an image such as an advertisement to the side of the building as an additional image.

In addition, when the configurator wishes to add a new additional image regardless of the object existing in the real space, the area to which the additional image is to be added and the area to which the additional image is to be added can be Select the content of the attached image. The content of the additional image is an image of a building, an advertising signboard, or the like that does not exist in the real space.

The setting information input to the computer 5 by the setting person is transmitted from the computer 5 to the server device 2 . Further, when the setting information input to the computer 5 by the configurator is transmitted from the computer 5 to the server device 2 , the identification information of the configurator is also transmitted to the server device 2 . When the receiving means 221 receives the setting information and the identification information of the configurator (YES in step st1), the received setting information and the identification information of the configurator are stored in the storage unit 23 (step st2).

At this time, the determination means 223 determines whether or not the identification information received by the reception means 221 corresponds to the object to which the additional image is added, based on the correspondence information stored in the storage unit 23. to judge whether Then, only when the determining means 223 determines that the two match, the instructing means 224 causes the storage unit 23 to store the received setting information and the identification information of the configurator. That is, even if a person other than the configurator who has the right to the object inputs setting information to the computer 5, this setting information is not stored in the storage unit 23. FIG.

Thereafter, when the user goes out while carrying the AR device 3 and turns on the AR device 3 using the operation unit 34 , a display instruction is transmitted from the AR device 3 to the server device 2 . At this time, an image of the surrounding environment of the AR device 3 detected by the surrounding detection unit 35 and information regarding the current position of the AR device 3 detected by the position detection unit 36 are also transmitted to the server device 2 . Attribute information about the user stored in the storage unit 37 is also transmitted from the AR device 3 to the server device 2 .

When the reception unit 221 of the server device 2 receives the display instruction (YES in step st3), the setting unit 222 sets the user's AR The display mode of the additional image on the device 3 is set (step st4).

Specifically, when the server device 2 accepts setting information for synthesizing an additional image with an image of the physical space, the setting unit 222 adds the additional image to the image of the physical space detected by the peripheral detection unit 35 . is displayed on the AR device 3.

More specifically, the setting unit 222 stores setting information, information about the surrounding environment detected by the surrounding detection unit 35 (surrounding environment information), and information about the current position of the AR device 3 detected by the position detection unit 36 (current position information), the display mode of the additional image is set. Specifically, the setting means 222 uses at least one of the surrounding environment information and the current position information to determine whether or not the object or area to which the additional image should be added exists near the AR device 3. Identify. At this time, information about the direction in which the AR device 3 is facing may be transmitted from the AR device 3 to the server device 2 . In this case, the direction in which the AR device 3 is facing is also referred to, and the determining means determines whether or not the object exists in the area to which the AR device 3 is facing (that is, whether or not the object can be visually recognized by the user). 223 may be determined.

Then, when it is specified that the object or area to which the additional image is to be added exists near the AR device 3, the instruction means 224 displays the image synthesized with the additional image in the set display mode as an AR image. Displayed on the display section 38 of the device 3 (step st5). Specifically, the image of the surrounding environment detected by the surrounding detection unit 35 of the AR device 3 is combined with the additional image, and the combined image is transmitted from the server device 2 to the AR device 3 . In this manner, the instruction means 224 causes the display unit 38 of the AR device 3 to synthesize and display the additional image.

Note that when a command not to synthesize the additional image with the image of the surrounding environment is input from the operation unit 34 of the AR device 3, the instruction unit 224 causes the image of the additional image to be synthesized with the image of the surrounding environment to be displayed on the AR device. 3 is not displayed on the display unit 38. In this case, the image of the surrounding environment detected by the surrounding detection unit 35 is displayed as it is on the display unit 38 of the AR device 3 .

The setting unit 222 also refers to the attribute information regarding the user when setting the display mode of the additional image. Specifically, for example, when an advertisement image is displayed as an additional image on the AR device 3, an advertisement image that matches the user's attribute information is preferentially displayed on the display unit 38, and an advertisement image that does not match the user's attribute information is preferentially displayed. To prevent an advertisement image from being displayed on a display part 38. - 特許庁For example, if the user is a child as attribute information of the user, the display unit 38 is caused to preferentially display images of advertisements relating to children, and the display unit 38 is prevented from displaying images of advertisements unsuitable for children. .

The setting unit 222 also refers to the date and time information regarding the current date and time when setting the display mode of the additional image. The date and time information includes the current month, day of the week, date, time, and time zone. For example, when an advertisement image is displayed on the AR device 3 as an additional image, an advertisement image that matches the current time is preferentially displayed on the display unit 38 . Specifically, the display unit 38 displays images of advertisements of restaurants and foods for lunch in the morning, and images of advertisements of restaurants and alcoholic beverages for dinner in the afternoon.

A specific example of the additional image displayed on the AR device 3 will be described with reference to FIG. FIG. 2 shows an example in which AR glasses are used as the AR device 3 . A first building 102 and a second building 106 exist in the surrounding environment 100 of the AR device 3 , and an advertising signboard 104 is installed on the roof of the first building 102 . In the example shown in FIG. 2, the advertising signboard 104 is in a state where no advertisement is posted.

In such a surrounding environment 100, for example, if the setter is the owner or lessee of the first building 102 or the advertising signboard 104, and wants to display an advertisement image on the side surface 103 of the first building 102, When it is desired to display an advertisement image on the advertisement signboard 104, the computer 5 is set so as to synthesize the advertisement image with the image of the surrounding environment. As a result, in the image 100a of the surrounding environment displayed on the display unit 38 of the AR device 3, the image of the first building 102a on which the advertisement image is displayed on the side and the image of the advertising signboard 104a on which the advertisement image is displayed is displayed. can be displayed.

In addition, the setting person can use the computer 5 to set where in the first building 102 and the advertising signboard 104 the advertisement image is to be synthesized. For example, if the configurator is the owner or lessee of the first building 102, the configurator may select, for example, the area where the advertisement image on the side of the first building 102 is synthesized (for example, the right half, above the fifth floor, the entire area, a portion other than a predetermined area, etc.) can be selected. Further, when a setter is assigned to each floor of the first building 102, the setter can select the area where the advertisement image is synthesized for each floor.

Also, when the setter is the owner or lessee of the advertising signboard 104, the setter can select the area where the advertisement image on the advertising signboard 104 is synthesized. At this time, the advertisement signboard 104 can be divided into a plurality of areas, and an advertisement image can be synthesized in each area.

Further, even if an advertisement is posted on the billboard 104 in the real space, the image of the advertisement set by the setter in the image 100a displayed on the display unit 38 of the AR device 3 is synthesized with the advertisement existing in the real space. can be displayed. At this time, the AR device 3 can synthesize and display an advertisement that matches the content of the advertisement on the billboard 104 in the real space, but cannot synthesize and display an advertisement that does not match. can be performed on the computer 5.

In addition, when the setter is the owner or lessee of the first building 102 or the advertising signboard 104, at a predetermined time or date (for example, every few seconds, morning, afternoon, weekdays, holidays, holidays, anniversaries or months) In addition, the computer 5 can be set to switch and display a plurality of advertisement images in the same area of the first building 102 or the advertisement signboard 104 .

Also, the setting person synthesizes an additional image that matches the image of the surrounding environment detected by the surrounding detection unit 35 and displays it on the display unit 38, but does not display the additional image that does not match the image of the surrounding environment on the display unit 38. You can make settings like this. For example, when religious facilities (such as shrines and torii gates) in the surrounding environment detected by the surrounding detection unit 35 are included, an image related to another religion (such as an image of Christmas illuminations) is added as an additional image. ) can be combined and not displayed on the display unit 38 . Also, if public facilities such as school facilities, medical institutions, police stations, fire departments, military facilities, etc., are included in the surrounding environment detected by the surrounding detection unit 35, the additional image is not synthesized and displayed on the display unit 38. It is possible to make settings such as settings that do not display advertising images of private companies.

Further, when the image of the surrounding environment detected by the surrounding detection unit 35 includes an advertising signboard of a company, the image of the advertising signboard of a competitor of this company is combined with the image of the surrounding environment and displayed on the display unit 38. The configurator can make a setting that does not allow it.

Further, for example, when the configurator is the owner of the second building 106, even if no advertising signboard is actually installed on the rooftop of the second building 106, the configurator of the second building 106 in the AR device 3 If it is desired to display an advertising sign on the roof, settings can be made by the computer 5. - 特許庁As a result, in the image 100a of the surrounding environment displayed on the display unit 38 of the AR device 3, the image of the advertising signboard 108a on which the advertisement is displayed can be displayed while being installed on the roof of the second building 106a.

As shown in FIG. 2, by synthesizing an advertisement image with an image of the actual surrounding environment and displaying it on the AR device 3, it is possible to obtain advertisement income without spending the cost of installing an advertisement signboard on a building or the like. can. In particular, it is not limited to commercial buildings, but can also be used for residential facilities such as individual houses and condominiums, structures such as bridges (including pedestrian bridges) and utility poles, municipal offices, mountains, rivers, etc., without actually installing advertising signs. can earn income. In addition, it is possible to synthesize only the image of the advertisement suitable for the building or the surrounding environment where this building is installed with the image of the actual surrounding environment and display it on the AR device 3 .

Next, a case where the information processing system 1 of the present embodiment is applied to virtual reality (VR) technology will be described. First, the setting person uses the computer 5 to set the display of the additional image. Specifically, when the setting person inputs a command to start setting the display of the additional image to the computer 5 , the computer 5 transmits a setting start command to the server device 2 .

When receiving the setting command, the server device 2 transmits the virtual space information stored in the storage unit 23 to the computer 5 , and an image of the virtual space is displayed on the monitor of the computer 5 .

Then, when the configurator selects an object to which the additional image is to be added from the image of the virtual space displayed on the monitor of the computer 5, the setting information of the additional image can be input to the computer 5. Specifically, when the configurator selects an object (building) from the image of the virtual space, for example, an image such as an advertisement can be added as an additional image to the side of the object.

Also, when the configurator wishes to add a new additional image to the image of the virtual space displayed on the monitor of the computer 5, he/she selects an area where the new additional image is to be synthesized. After that, the setting person can input the setting information as to what content of the additional image is to be added to the computer 5 .

The setting information input to the computer 5 by the setting person is transmitted from the computer 5 to the server device 2 . Further, when the setting information input to the computer 5 by the configurator is transmitted from the computer 5 to the server device 2 , the identification information of the configurator is also transmitted to the server device 2 . When the receiving means 221 receives the setting information and the identification information of the configurator transmitted to the server device 2 (YES in step st1), the received setting information and the identification information of the configurator are stored in the storage unit 23 ( step st2).

At this time, the determination means 223 determines whether or not the identification information received by the reception means 221 corresponds to the object to which the additional image is added, based on the correspondence information stored in the storage unit 23. to judge whether Then, only when the determining means 223 determines that the two match, the instructing means 224 causes the storage unit 23 to store the received setting information and the identification information of the configurator. That is, even if a person other than the configurator who has the right to the object inputs setting information to the computer 5, this setting information is not stored in the storage unit 23. FIG.

Thereafter, when the user moves within the virtual space displayed on the display unit 48 of the computer 4 using the operation unit 45 or the like, a display instruction is transmitted from the computer 4 to the server device 2 . At this time, information about the current position in the virtual space is also transmitted to the server device 2 . The attribute information about the user stored in the storage unit 46 is also transmitted from the computer 4 to the server device 2 .

When the reception unit 221 of the server device 2 receives the display instruction (YES in step st3), the setting unit 222 sets the display mode of the additional image in the user's computer 4 based on the received setting information (step st4). .

Specifically, when the server device 2 accepts setting information for adding an additional image to the image in the virtual space, the setting means 222 selects an image obtained by synthesizing the additional image with the image in the virtual space as the display mode. It is set to be displayed on the display unit 48 of the computer 4 .

More specifically, the setting means 222 sets the display mode of the additional image based on the setting information and information on the current position in the virtual space (current position information). Specifically, the setting means 222 identifies whether or not the object or area to which the additional image is to be added exists near the current position in the virtual space.

Then, when it is specified that the object or area to which the additional image is to be added exists near the current position in the virtual space, the instruction means 224 displays the image to which the additional image is added in the set display mode. is displayed on the display unit 48 of the user's computer 4 (step st5). Specifically, the image of the virtual space stored in the storage unit 23 is combined with the additional image in a set display mode, and the combined image is transmitted from the server device 2 to the user's computer 4 . Then, the synthesized image is displayed on the display unit 48 of the user's computer 4 .

The setting unit 222 also refers to the attribute information regarding the user when setting the display mode of the additional image. Specifically, for example, when an advertisement image is displayed as an additional image on the display unit 48 of the user's computer 4, the advertisement image that matches the user's attribute information is preferentially displayed on the display unit 48, and the user's To prevent a display part 48 from displaying an image of an advertisement that does not match attribute information.

The setting unit 222 also refers to the date and time information regarding the current date and time when setting the display mode of the additional image. Specifically, for example, when an advertisement image is displayed as an additional image on the display unit 48 of the user's computer 4, an advertisement suitable for the current time is preferentially displayed on the display unit 38. FIG.

Further, when an image synthesized with the additional image in the display mode set by the setting means 222 is displayed on the display unit 48 of the computer 4, an image of a two-dimensional code indicating the URL of the website introducing the additional image is synthesized. You may be able to do so. In this case, by reading the two-dimensional code attached to the additional image in the virtual space displayed on the display unit 48 of the computer 4, it is possible to view a website that introduces products related to the additional image. Become.

A specific example of the image displayed on the user's computer 4 will be described with reference to FIG. As shown in FIG. 3, in a virtual space image 200 displayed on the display unit 48 of the computer 4, there are a first building 202 and a second building 206, and an advertising signboard 204 is installed on the roof of the first building 202. It is

In such a virtual space image 200 , for example, if the configurator wishes to display an advertisement on the side 203 of the first building 202 or an advertisement on the billboard 204 , the computer 5 makes settings. As a result, in the image 200 of the virtual space displayed on the display unit 48 of the computer 4, it is possible to display the image of the first building 202 on which the advertisement is displayed and the image of the advertising signboard 204 on which the advertisement is displayed. .

Even if no advertising signboard is installed on the roof of the second building 206 in the image 200 of the virtual space, if the setting person wants to display the advertising signboard on the roof of the second building 206, the computer 5 can set I do. As a result, in the virtual space image 200 displayed on the display unit 48 of the computer 4 , the image of the advertising signboard 208 on which the advertisement is displayed can be displayed in a state of being installed on the roof of the second building 206 .

In this way, as shown in FIG. 3, by synthesizing and displaying an advertisement image on the image of the virtual space on the user's computer 4, the owner of the object in the virtual space can also obtain advertising income in the virtual space. be able to. In addition, the owner of the object in the virtual space sells the right to place an advertisement on the object to, for example, an advertising agency, so that the advertising agency can lend the area of the object to a person wishing to place an advertisement.

To summarize the above, the server apparatus 2 includes a computer that functions as reception means 221, setting means 222, and instruction means 224 by executing programs. The accepting unit 221 accepts setting information regarding display setting of the additional image from the computer 5 (first terminal) of the setting person.

The setting unit 222 sets the display mode of the additional image on the user's AR device 3 or computer 4 (second terminal) based on the setting information received by the receiving unit 221 . The instruction means 224 causes the user's AR device 3 or the computer 4 to display an image obtained by synthesizing the additional image in the display mode set by the setting means 222 .

<effect>
According to this embodiment, by setting the display mode of the additional image based on the received setting information, an image obtained by synthesizing the additional image with the image of the real space or virtual space is displayed on the user's AR device 3 or computer 4. can be made

<Other embodiments>
The various control means and processing procedures described in the above embodiments are examples, and are not intended to limit the scope of the present invention, its applications, or its uses. Various control means and processing procedures can be appropriately modified in design without changing the gist of the present invention.

For example, in the above description, a setting person inputs setting information using the computer 5, the input setting information is transmitted from the computer 5 to the server device 2, and the accepting means 221 accepts the setting information. However, as another aspect, the configurator may be able to input the setting information using a reception-only terminal prepared by the platformer such as the administrator of the server device 2 .

Further, whether or not to display an image synthesized with the additional image on the user's AR device 3 or computer 4 in the display mode set by the setting unit 222 is determined by the user's AR device 3 or computer 4, not the server device 2. It may be performed on the side.

Also, the additional image is not limited to an advertisement image. As additional images, images that decorate the walls of buildings, etc., names and addresses of buildings, etc., explanations, directions, messages (e.g., danger ahead, steps, →, ↑, welcome to 〇〇, take the elevator to the 3rd floor Please use a variety of types of images, such as images that include characters or icons indicating such things as, images that decorate movable properties, images that indicate specific bases for location information games using AR technology, etc. . In addition, it may be a guidance image for evacuating residents in the event of a disaster, or a guidance image for an event such as a fireworks display.

It should be noted that the right of the setter to set the additional image may be traded or lent. For example, when a given business operator conducts an event, this right is lent by the setter, or when the business owner displays an advertisement for its own product or service, this right is lent by the setter. You can receive it. Then, the setting means 222 stores and manages the information about the setting person. Also, in an emergency such as a disaster, regardless of the setting information, necessary information may be displayed.

Further, as another embodiment, the receiving means 221 may receive intellectual property information related to the intellectual property of the object from the terminal of the intellectual property right holder. In this case, based on the received intellectual property information, the determination unit 223 displays the image in which the additional image is added to the object in the display mode set by the setting unit 222. ) to determine whether it is appropriate to display

Then, only when the determination means 223 determines that it is appropriate, the instruction means 224 displays the image in which the additional image is added to the object in the display mode set by the setting means 222 on the user's AR device 3 or the like. Let

In other such embodiments, the intellectual property of the subject matter includes patent rights, design rights, trademark rights, copyrights, etc. of the subject matter. The intellectual property information also includes information as to whether the object is protected by intellectual property, and identification information of the owner of the intellectual property if the object is protected by intellectual property. In addition, intellectual property information is, for example, the registration number and registration details of patent rights, design rights, and trademark rights registered with the Patent Office, or registered with the Agency for Cultural Affairs, if the object is protected by intellectual property. This includes the registered copyright registration number and registered content.

In addition, the intellectual property information indicates that an additional image is added to the image of the object displayed on the user's AR device 3 or computer 4 when the object is protected by intellectual property. Contains information on whether or not the rights holder approves.

Intellectual property rights holders include copyright holders of works, trademark rights holders such as logos attached to objects, design right holders when objects are protected by design rights, and objects with patent rights Including patent owners where protected by

Details of such other embodiments are described below. The information processing system 1 of another embodiment is applicable to both augmented reality (AR) and virtual reality (VR) technologies. When the information processing system 1 of another embodiment is applied to augmented reality (AR) technology, the image of the object is displayed on the display unit 38 of the AR device 3 owned by the user as the image of the surrounding environment of the AR device 3. It can be displayed in a combined state.

Further, when the information processing system 1 of another embodiment is applied to virtual reality (VR) technology, when an image of an object in the virtual space is displayed on the display unit 48 of the computer 4, the additional image is displayed in the virtual space. The image can be displayed on the display unit 48 in a state of being synthesized with the image.

A configuration of the information processing system 1 of such another embodiment will be described. Note that the description of the same contents as in the present embodiment described above will be omitted.

The reception means 221 of the server device 2 receives intellectual property related to the intellectual property of the object from a terminal (for example, a computer) of the intellectual property right holder or a reception dedicated terminal prepared by the platformer such as the administrator of the server device 2. accept information. The intellectual property information received by the receiving means 221 is stored in the storage unit 23 . Also, the receiving unit 221 receives information about the display mode of the AR device 3 or computer 4 from the user's AR device 3 or computer 4 as display mode information.

The setting unit 222 sets the display mode of the additional image on the user's AR device 3 or computer 4 . Since the details of the display mode of the additional image have been described above, the description thereof will be omitted.

Based on the intellectual property information received by the receiving means 221, the determining means 223 causes the user's AR device 3 or the computer 4 to display the additional image added to the object in the display mode set by the setting means 222. to determine whether it is appropriate. In addition, when the display mode information is received by the reception unit 221, the judgment unit 223 can refer to the display mode information and cause the user's AR device 3 or the computer 4 to display the additional image added to the object. determine if it is appropriate. The details of the determination made by the determination means 223 will be described later.

If the determination unit 223 determines that it is appropriate to display the addition image on the object on the user's AR device 3 or the computer 4, the instruction unit 224 instructs the user to display the image combined with the addition image. displayed on the AR device 3 or the computer 4. Specifically, a command including the display mode of the additional image is transmitted from the server device 2 to the AR device 3 and the computer 4 via the network interface 21 .

Next, the flow of a series of operations of the information processing system 1 of another embodiment will be described using FIG.

First, a case where an information processing system 1 of another embodiment is applied to augmented reality (AR) technology will be described. In another embodiment, intellectual property information about the intellectual property of the object is sent to the server device 2 . Such intellectual property information is transmitted from the terminal to the server device 2 when the intellectual property right holder inputs the intellectual property information of the object to a terminal such as a computer. Moreover, the intellectual property information may be transmitted to the server apparatus 2 from a terminal other than the intellectual property right holder, specifically, for example, a computer of an organization or group that has jurisdiction over the intellectual property information.

When the intellectual property information transmitted to the server device 2 is accepted by the accepting means 221 (YES in step st11), the accepted intellectual property information is stored in the storage unit 23 (step st12).

After that, the setting person uses the computer 5 to set the display of the additional image. The setting information and the identification information of the configurator input to the computer 5 by the configurator are transmitted from the computer 5 to the server device 2 and are received by the receiving means 221 (YES in step st13). The setting information and the identification information of the configurator received by the receiving means 221 are stored in the storage unit 23 (step st14). Since the display setting of the additional image has been described above, the details are omitted.

When the user goes out while carrying the AR device 3 and turns on the power of the AR device 3 using the operation unit 34 , a display instruction is transmitted from the AR device 3 to the server device 2 . At this time, an image of the surrounding environment of the AR device 3 detected by the surrounding detection unit 35 and information regarding the current position of the AR device 3 detected by the position detection unit 36 are also transmitted to the server device 2 . User attribute information is also transmitted from the AR device 3 to the server device 2 .

When the receiving unit 221 of the server device 2 receives the display instruction (YES in step st15), the setting unit 222 sets the display mode of the additional image on the user's AR device 3 based on the received setting information (step st16). ). Since the setting of the display mode of the additional image has been described above, the details will be omitted.

Then, based on the intellectual property information stored in the storage unit 23, the determination unit 223 displays the additional image on the user's AR device 3 in the display mode set by the setting unit 222 while adding the additional image to the object. It is determined whether or not it is appropriate to allow the operation (step st17). For example, if the object to which the additional image is to be added is protected by intellectual property, the additional image shall be included only if the addition of the additional image is permitted by the intellectual property right holder. It is judged appropriate to display the image attached to the object on the AR device 3 .

When the determination unit 223 determines that it is appropriate to display the additional image on the user's AR device 3 (YES in step st17), the instruction unit 224 causes the image of the surrounding environment detected by the surrounding detection unit 35 to be displayed. The additional image is synthesized in the set display mode. The instruction means 224 then transmits the combined image from the server device 2 to the AR device 3 . In this way, the instruction means 224 causes the display section 38 of the AR device 3 to display this synthesized image (step st18).

On the other hand, if it is determined that it is not appropriate to display the image in which the additional image is added to the object on the user's AR device 3 (NO in step st17), the instructing means 224 instructs that the additional image is added to the object. is not displayed on the AR device 3 (step st19). In this case, the image of the surrounding environment detected by the surrounding detection unit 35 is displayed on the display unit 38 as it is.

In addition, it is appropriate for the determination unit 223 to refer to the extent to which the target object is included in the surrounding environment information received by the reception unit 221 and cause the AR device 3 to display the additional image in the display mode set by the setting unit 222. Determine whether or not For example, regarding an object protected by intellectual property, the determination means 223 will It may be determined whether or not it is appropriate to display the additional image on the AR device 3 .

In this case, when the time period during which the object is continuously detected by the surroundings detection unit 35 is shorter than the predetermined time, the judgment unit 223 does not make a judgment, and the surrounding environment detected by the surroundings detection unit 35 is detected. is displayed on the display unit 38 as it is. On the other hand, when the time for which the object is continuously detected by the periphery detection unit 35 is longer than or equal to the predetermined time, the additional image is added to the object in the set display mode after the predetermined time has passed. The judgment means 223 judges whether it is appropriate to display on the AR device 3 in the state.

Moreover, when the target object is included in the surrounding environment information received by the receiving means 221, the determining means 223 may immediately start determination. In this case, even if it is determined that it is appropriate for the AR device 3 to display the additional image added to the object, the time during which the object is continuously detected by the periphery detection unit 35 is longer than the predetermined time. is short, the image of the surrounding environment detected by the surrounding detection unit 35 is displayed on the display unit 38 as it is.

Further, the receiving means 221 receives information about the current position of the AR device 3 from the AR device 3 as current position information, and the determining means 223 sets based on the intellectual property information and the current position information received by the receiving means 221. It is determined whether or not it is appropriate to display on the AR device 3 the image synthesized with the additional image in the display mode set by means 222 . Specifically, the country or state where the AR device 3 is located is identified based on the current location information, and the intellectual property of the object is certified according to the intellectual property system of the identified country or state. .

Also, in the AR device 3 , the display mode of the image displayed on the display section 38 may be set by the operation section 34 . In this case, the display mode of the AR device 3 set by the operation unit 34 is transmitted from the AR device 3 to the server apparatus 2, and the receiving unit 221 receives information regarding the display mode of the AR device 3 as display mode information. Display modes include, for example, a trademark display mode, a trademark non-display mode, a copyright display mode, a copyright non-display mode, and the like.

In this case, the determining means 223 causes the AR device 3 to display an image synthesized with the additional image in the display mode set by the setting means 222 based on the intellectual property information and the display mode information received by the receiving means 221. is appropriate. For example, when the trademark non-display mode is set as the display mode, the image of the object with the trademark is not displayed on the AR device 3, or the image of the object with the trademark is not displayed. Only the image of the object excluding the is displayed on the AR device 3.

When the copyrighted material non-display mode is set as the display mode, the image of the object protected by copyright is not displayed on the AR device 3 .

A specific example of an image displayed on the user's AR device 3 in such another embodiment will be described. For example, if an object that actually exists, such as a monument or a bronze statue, is protected by copyright, whether or not the copyright owner of this object permits the addition of an additional image to this object. can be set, and the set intellectual property information is transmitted to the server device 2 . Also, at this time, the copyright holder can set a region in which the addition of the additional image is permitted or not permitted in the object.

Specifically, for example, if the target object is a bronze statue and the target object is protected by copyright, the image displayed on the AR device 3 is synthesized with an image of clothing as an additional image on the bronze statue. A copyright holder can set a setting to allow the face to be displayed by the face, but not to display another face synthesized.

In addition, the copyright holder can make a setting that does not allow the image displayed on the AR device 3 to be combined with an advertisement image as an additional image on the object. In addition, for example, when an object is protected by copyright, the copyright holder can make a setting that does not allow addition of an additional image that makes the object appear damaged.

In addition, in the image displayed on the AR device 3, the intellectual property right holder does not set whether or not to allow the addition of the additional image to the object itself protected by the intellectual property. The intellectual property owner may be able to set whether or not to allow additional images to be added around the object. For example, when a bronze statue as an object is protected by copyright, whether or not to allow the image displayed on the AR device 3 to be displayed by synthesizing an image of wings as an additional image around the statue. can be set by the intellectual property right holder.

Also, for example, when the target object is a movable property such as a vehicle, a bag, or clothing, and the target object is protected by a copyright, a trademark right, a design right, a patent right, etc., the AR device 3 displays In the image, another image such as an advertisement can be combined with this movable property as an additional image and displayed. In this case, the intellectual property right holder can use the terminal to set whether or not to allow the addition of the additional image to the object displayed on the AR device 3 .

In addition, when the intellectual property owner makes such settings, the terminal can be set so as not to allow additional images to be overwritten on images such as brand names and logos displayed on the object. It's like Addition of additional images that damage the brand of the object, addition of images that cause a misunderstanding of quality, and addition of images that can replace the brand name or logo, etc. of competitors of the manufacturer of the object. The intellectual property right holder can make a setting that the addition is not permitted through the terminal.

Next, a case where the information processing system 1 of another embodiment is applied to virtual reality (VR) technology will be described. Also in this case, when the intellectual property information transmitted to the server device 2 is accepted by the accepting means 221 (YES in step st11), the accepted intellectual property information is stored in the storage unit 23 (step st12).

After that, the setting person uses the computer 5 to set the display of the additional image. The setting information and the identification information of the configurator input to the computer 5 by the configurator are transmitted from the computer 5 to the server device 2 and are received by the receiving means 221 (YES in step st13). The setting information and the identification information of the configurator received by the receiving means 221 are stored in the storage unit 23 (step st14). Since the display setting of the additional image has been described above, the details are omitted.

Then, when the user moves within the virtual space displayed on the display unit 48 of the computer 4 using the operation unit 45 or the like, a display instruction is transmitted from the computer 4 to the server device 2 . At this time, information about the current position in the virtual space is also transmitted to the server device 2 . The attribute information about the user stored in the storage unit 46 is also transmitted from the computer 4 to the server device 2 .

When the receiving means 221 of the server device 2 receives the display instruction (YES in step st15), the setting means 222 sets the display mode of the additional image in the user's computer 4 based on the received setting information (step st16). . Since the setting of the display mode of the additional image has been described above, the details will be omitted.

Based on the intellectual property information stored in the storage unit 23, the determination unit 223 determines whether it is appropriate to display the image synthesized with the additional image in the set display mode on the computer 4 of the user. (step st17). When it is determined that it is appropriate to display the image synthesized with the additional image on the user's computer 4 (YES in step st17), the instruction means 224 displays the additional image in the display mode set by the setting means 222. is displayed on the computer 5 (step st18).

On the other hand, when it is determined that it is not appropriate to display the image obtained by adding the additional image to the object on the user's computer 4 (NO in step st17), the instruction means 224 causes the setting means 222 to set The image synthesized with the additional image in the display mode is not displayed on the computer 4 (step st19). In this case, the display unit 48 of the computer 4 displays an image of the virtual space without the additional image.

To summarize the above, according to another embodiment, the server device 2 includes a computer that functions as reception means 221, setting means 222, determination means 223, and instruction means 224 by executing a program. It is. The receiving means 221 receives intellectual property information regarding the intellectual property of the object. The setting unit 222 sets the display mode of the additional image on the user's AR device 3 or computer 4 (second terminal).

Based on the received intellectual property information, the determination unit 223 can cause the user's AR device 3 or the computer 4 to display the image of the additional image added to the object in the display mode set by the setting unit 222. determine whether it is appropriate. The instruction means 224 synthesizes the additional image in the display mode set by the setting means 222 when the determination means 223 determines that it is appropriate to display the image of the additional image added to the object. The image is displayed on the user's AR device 3 or computer 4 .

According to another embodiment, by determining whether it is appropriate to display the additional image attached to the object based on the intellectual property information, only the additional image appropriate for the intellectual property is displayed. can be displayed on the user's AR device 3 or computer 4. This makes it possible to protect the intellectual property rights of the intellectual property right holders.

Moreover, in each of the above-described embodiments, the case where the server device 2 causes the program to function has been exemplified, but the present invention is not limited to this. All the means of the program, that is, part of the means such as the reception means 221, the setting means 222, the judgment means 223, the instruction means 224, the user's AR device 3 and computer 4, the setter's computer 5, the intellectual property It may be provided in another terminal such as the terminal of the right holder. In this case, it is also possible to appropriately change which means functions in which device.

In addition, the setting information, intellectual property information, and the like received by the receiving means 221 are not stored in the storage unit 23 of the server device 2, but in a cloud or an external storage medium provided separately from the server device 2. It may be configured to be stored.

Further, the setting means 222 of the server device 2 may create a map or a list that allows confirmation of where and what kind of object is in the real space or virtual space and what kind of additional image can be synthesized. good. Maps and lists created by the setting means 222 may be displayed on the display section 38 of the user's AR device 3 or the display section 48 of the computer 4 .

In addition, the setting means 222 of the server device 2 sets, for each user, the number of times and display time of the object displayed on the display unit 38 of the AR device 3 and the display unit 48 of the computer 4, the number of times and display time of the additional image, the object The number of additional images added to the object and the display time may be tallied. Then, the setting unit 222 may give a privilege to the user when the number of times or the display time of the object or the additional image to be displayed reaches a predetermined value set in advance. For example, when the additional image is an advertisement image, the privilege may be a tangible object such as a product related to the advertisement, or an intangible object such as a coupon.

It is also possible to hold an event using the total number of times the objects and additional images displayed on the display unit 38 of the AR device 3 and the display unit 48 of the computer 4 are displayed. Specifically, the events include a game in which the user discovers a predetermined number or more or all of the additional images in a predetermined area within a predetermined time on the AR device 3 or the computer 4, or a puzzle-solving task as an additional image on the AR device. A puzzle-solving game that is displayed on the display unit 38 of No. 3 or the like may be executed.

In addition, information about the number of times and display time of objects and additional images displayed on the display unit 38 of the AR device 3 and the display unit 48 of the computer 4 for each user, which is aggregated by the setting unit 222, is sent from the server device 2 to the setter. may be transmitted to the computer 5 of the advertiser, the computer of the advertiser, or the like. This allows the setter, advertiser, etc., to recognize the effectiveness of the advertisement for the user.

Further, in the above description, a plurality of advertisement images are displayed at the first building 102 or the advertisement signboard at predetermined times or dates (for example, every few seconds, in the morning, afternoon, weekdays, holidays, holidays, anniversaries, or months). Although it has been described that the computer 5 can be set to switch and display in the same area 104, the present invention is not limited to this. For example, when an additional image (for example, an advertisement image) is displayed on the user's AR device 3 or computer 4 for a predetermined number of times or for a predetermined period of time, the advertisement image is switched to the same area of the first building 102 or the advertising signboard 104. You may make the computer 5 perform the setting which displays.

Further, the receiving means 221 of the server device 2 may receive information about the user's My Number as the user's identification information. Then, when the information about the My Number is associated with the information that the user has a specific addiction, the setting means 222 determines whether the item or event (for example, game, alcohol, gambling, etc.) subject to addiction The user's AR device 3 or computer 4 may not display the advertisement image as an additional image, or the user's AR device 3 or computer 4 may display an additional image related to the cure of addiction. Moreover, the right of the setter to the target object may be confirmed based on the information about the My Number.

Also, in the above description, an example has been described in which the receiving unit 221 receives the setting information regarding the setting of the display of the additional image from the computer 5 (first terminal) of the configurator, but the present invention is not limited to this. In this embodiment, the setting for displaying the additional image may be arbitrarily performed in the server device 2 .

In addition, for buildings in the real space and virtual space that are not related to this system, the display of advertisements can be controlled without the setting person using the computer 5 to set the display of additional images. good too. In other words, the fact that the display of the additional image is not set by the setting person can also be treated as part of the setting by the setting unit 222 .

An example will be described in which the target object is not a movable property or the like existing in the physical space, but a predetermined area in the physical space. For example, regardless of whether or not there is a building on the land, the owner (setting person) of the land sets the display of the additional image for the object, with the space above the land as the object. can be Also, the predetermined area (object) in the physical space may be the area around a predetermined building or the distance from a predetermined point (starting point) to the building (destination). For example, an area (target object) is set in the public open space of a given building, and an additional image showing information such as tenants living in the building in that area, or a route from the starting point to the destination with ←, ↑, etc. An additional image showing directions or an additional image such as XX meters to the destination may be displayed.

Furthermore, when a plurality of additional images are set in one area (object), the arrangement is adjusted so that these additional images do not overlap when viewed from the user. In addition, according to the attributes and settings of the user, the arrangement is adjusted so that a part of the plurality of additional images is emphasized and displayed. Further, the additional image may be displayed in an emphasized manner according to a predetermined value such as the amount of money paid by the setting person or the like for displaying the additional image, or the newer the display setting, the more emphasized the image may be displayed.

Here, the emphasis display means that the size, position, color, luminosity, blinking, etc. of the additional image are made different from other additional images. The display mode also includes a mention of a specific user. For example, in the setting of the additional image, a specific user can be specified as a mention partner, and the mentioned additional image is displayed on the display unit of the terminal corresponding to the user. Moreover, the display mode of these additional images may be controlled by the server device or by the user's terminal.

Moreover, the present invention is applicable not only to AR and VR but also to XR such as MR.

2 Server device (computer)
221 reception means 222 setting means 223 determination means 224 instruction means

Claims (19)

A server device comprising a computer that functions as reception means, setting means, and instruction means by executing a program,
The receiving means receives setting information related to setting of display of the additional image from the first terminal,
The setting means sets a display mode of the additional image on the user's second terminal based on the setting information received by the receiving means,
The instruction means causes the second terminal to display an image obtained by selectively synthesizing the additional image in the display mode set by the setting means.,
When the image of the surrounding environment detected by the detection unit of the second terminal is displayed on the second terminal, the instruction means displays the additional image in the display mode set by the setting means. The image is selectively combined with the image and displayed on the second terminal, and at this time, the additional image that matches the image of the surrounding environment detected by the detection unit is combined and displayed on the second terminal. A setting can be made so that the additional image that does not match the image of the environment is not displayed on the second terminal. , server equipment. the setting information includes information about an object to which the additional image is to be added;
The setting means adds the additional image to the object in the display mode set by the setting means when the object is included in the image of the surrounding environment detected by the detection unit of the second terminal. 2. The server device according to claim 1 , wherein the second terminal is caused to display on the second terminal. the receiving means receives from the second terminal information about the surrounding environment of the second terminal detected by the detecting unit of the second terminal as surrounding environment information;
3. The server device according to claim 1 , wherein said setting means sets a display mode of said additional image on said second terminal based on said setting information and said surrounding environment information received by said receiving means. the reception means receives information about the current location of the second terminal from the second terminal as current location information;
4. The setting unit according to any one of claims 1 to 3 , wherein the setting unit sets a display mode of the additional image on the second terminal based on the setting information and the current location information received by the reception unit. server equipment. The setting information includes information on setting whether or not to display the additional image on the second terminal,
2. When said setting information is information for displaying said additional image on said second terminal, said setting means sets said display mode to display said additional image on said second terminal. 5. The server device according to any one of 1 to 4 . the receiving means receives attribute information about the user who owns the second terminal;
The server device according to any one of claims 1 to 5 , wherein said setting means also refers to said attribute information received by said receiving means to set said display mode of said additional image on said second terminal. . 7. The server device according to any one of claims 1 to 6 , wherein said setting means also refers to date and time information relating to current date and time to set the display mode of said additional image on said second terminal. A server device comprising a computer that functions as reception means, setting means, determination means, and instruction means by executing a program,
The reception means receives intellectual property information related to the intellectual property of the object,
The setting means sets a display mode of the additional image added to the object on the user's second terminal,
It is appropriate that the determination means causes the second terminal to display the additional image added to the object in the display mode set by the setting means based on the received intellectual property information. determine whether there is
The display mode set by the setting means when the determination means determines that it is appropriate to display the additional image on the second terminal in a state where the additional image is added to the object. and causing the second terminal to display the additional image added to the object, and causing the second terminal to display the additional image added to the object by the determining means. A server device that does not display the additional image on the second terminal when determined . When the image of the surrounding environment detected by the detection unit of the second terminal is displayed on the second terminal, the instruction means displays the additional image in the display mode set by the setting means. 9. The server device according to claim 8 , which is combined with an image and displayed on the second terminal. the receiving means receives from the second terminal information about the surrounding environment of the second terminal detected by the detecting unit of the second terminal as surrounding environment information;
The determining means adds the additional image to the object in the surrounding environment information received by the receiving means in the display mode set by the setting means based on the intellectual property information. 10. The server device according to claim 9 , wherein the server device determines whether or not it is appropriate to display on the second terminal in a state where the content is displayed. The determination means refers to the extent to which the object is included in the surrounding environment information received by the reception means, and adds the additional image to the object in the display mode set by the setting means. 11. The server device according to claim 10 , wherein the server device determines whether or not it is appropriate to display on the second terminal. the reception means receives information about the current location of the second terminal from the second terminal as current location information;
Based on the intellectual property information and the current position information received by the receiving means, the determining means adds the additional image to the object in the display mode set by the setting means, and 12. The server device according to any one of claims 9 to 11 , which determines whether or not it is appropriate to display on two terminals. The receiving means receives information about the virtual space,
The instruction means, when causing the second terminal to display the image of the virtual space received by the reception means, synthesizes the additional image with the image of the virtual space in the display mode set by the setting means. 9. The server device according to claim 8 , wherein the second terminal is caused to display on the second terminal. the receiving means receives information about a display mode of the second terminal of the user as display mode information;
Based on the intellectual property information and the display mode information received by the receiving means, the determining means adds the additional image to the object in the display mode set by the setting means, and 14. The server device according to any one of claims 8 to 13 , which determines whether or not it is appropriate to display on two terminals. Based on the received intellectual property information, the determination means determines that, if the object to which the additional image is to be added is protected by intellectual property, the additional image is determined by an intellectual property right holder. 15. Only when it is permitted to add the additional image, it is determined that it is appropriate to display the image in which the additional image is added to the object on the second terminal. The server device according to the item. An information processing system comprising reception means, setting means, and instruction means,
The receiving means receives setting information related to setting of display of the additional image from the first terminal,
The setting means sets a display mode of the additional image on the user's second terminal based on the setting information received by the receiving means,
The instruction means causes the second terminal to display an image obtained by selectively synthesizing the additional image in the display mode set by the setting means.,
When the image of the surrounding environment detected by the detection unit of the second terminal is displayed on the second terminal, the instruction means displays the additional image in the display mode set by the setting means. The image is selectively combined with the image and displayed on the second terminal, and at this time, the additional image that matches the image of the surrounding environment detected by the detection unit is combined and displayed on the second terminal. A setting can be made so that the additional image that does not match the image of the environment is not displayed on the second terminal. , information processing system. An information processing system comprising reception means, setting means, determination means, and instruction means,
The reception means receives intellectual property information related to the intellectual property of the object,
The setting means sets a display mode of the additional image added to the object on the user's second terminal,
It is appropriate that the determination means causes the second terminal to display the additional image added to the object in the display mode set by the setting means based on the received intellectual property information. determine whether there is
The display mode set by the setting means when the determination means determines that it is appropriate to display the additional image on the second terminal in a state where the additional image is added to the object. and causing the second terminal to display the additional image added to the object, and causing the second terminal to display the additional image added to the object by the determining means. An information processing system that does not display the additional image on the second terminal when it is determined . A program that causes a computer to function as reception means, setting means, and instruction means,
The receiving means receives setting information related to setting of display of the additional image from the first terminal,
The setting means sets a display mode of the additional image on the user's second terminal based on the setting information received by the receiving means,
The instruction means causes the second terminal to display an image obtained by selectively synthesizing the additional image in the display mode set by the setting means.,
When the image of the surrounding environment detected by the detection unit of the second terminal is displayed on the second terminal, the instruction means displays the additional image in the display mode set by the setting means. The image is selectively combined with the image and displayed on the second terminal, and at this time, the additional image that matches the image of the surrounding environment detected by the detection unit is combined and displayed on the second terminal. A setting can be made so that the additional image that does not match the image of the environment is not displayed on the second terminal. ,program. A program that causes a computer to function as reception means, setting means, determination means, and instruction means,
The reception means receives intellectual property information related to the intellectual property of the object,
The setting means sets a display mode of the additional image added to the object on the user's second terminal,
It is appropriate that the determination means causes the second terminal to display the additional image added to the object in the display mode set by the setting means based on the received intellectual property information. determine whether there is
The display mode set by the setting means when the determination means determines that it is appropriate to display the additional image on the second terminal in a state where the additional image is added to the object. and causing the second terminal to display the additional image added to the object, and causing the second terminal to display the additional image added to the object by the determining means. A program that, if determined, does not display the additional image on the second terminal .

Images