JP2022102549A - Server device, information processing system, and program - Google Patents

Abstract

To provide a server device configured to display an image with additional image synthesized thereon, on a user terminal, an information processing system, and a program.SOLUTION: A server device 2 includes a computer which functions as receiving means, 221, setting means 222, and instruction means 224 by executing a program. The receiving means 221 receives setting information on settings for displaying an additional image, from a first terminal. The setting means 222 sets a mode for displaying the additional image on a second terminal of a user on the basis of the setting information received by the receiving means 221. The instruction means 224 causes the second terminal to display an image with the additional image selectively synthesized thereon in the display mode set by the setting means 222.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to server devices, information processing systems and programs.

In recent years, a technique for displaying an object including real estate such as a building or an animal by using a technique of augmented reality (AR) or virtual reality (VR) on a user terminal such as a smartphone has been used. .. For example, Patent Document 1 discloses a technique for displaying an object in an augmented reality space on a user terminal. Further, Patent Document 2 discloses a technique for displaying an object in a virtual reality space on a user terminal.

Further, Patent Document 3 discloses an AR display management system capable of regulating AR display.

Japanese Unexamined Patent Publication No. 2013-8413 Japanese Unexamined Patent Publication No. 2020-149399 Japanese Unexamined Patent Publication No. 2019-87091

In the conventional augmented reality and virtual reality techniques, there is a problem that an additional image cannot be added and displayed on an image in a real space or a virtual space.

It is an object of the present disclosure to provide a server device, an information processing system and a program for synthesizing an additional image and displaying it on a user terminal.

The first aspect is
A server device provided with a computer that functions as a reception means, a setting means, and an instruction means by executing a program.
The reception means receives setting information regarding the display setting of the additional image from the first terminal, and receives the setting information.
The setting means sets the display mode of the additional image on the user's second terminal based on the setting information received by the reception means.
The instruction means is a server device that causes the second terminal to display an image in which the additional image is selectively combined in the display mode set by the setting means.

Also, in the first aspect,
When the instruction means displays an image of the surrounding environment detected by the detection unit of the second terminal on the second terminal, the additional image is displayed on the peripheral environment in the display mode set by the setting means. It can be selectively combined with an image and displayed on the second terminal.

Also, in the first aspect,
The setting information includes information on an object to which the additional image should be added.
When the object is included in the image of the surrounding environment detected by the detection unit of the second terminal, the setting means adds the additional image to the object in the display mode set by the setting means. It can be displayed on the second terminal in this state.

Also, in the first aspect,
The receiving means receives information about the surrounding environment of the second terminal detected by the detection unit of the second terminal as peripheral environment information from the second terminal.
The setting means can set the display mode of the additional image on the second terminal based on the setting information received by the reception means and the surrounding environment information.

Also, in the first aspect,
The receiving means receives information about the current position of the second terminal as the current position information from the second terminal.
The setting means can set the display mode of the additional image on the second terminal based on the setting information and the current position information received by the receiving means.

Also, in the first aspect,
The reception means receives information about the virtual space and receives information.
When the instruction means displays the image of the virtual space received by the reception means on the second terminal, the instruction means selectively displays the additional image as the image of the virtual space in the display mode set by the setting means. Can be combined with and displayed on the second terminal.

Also, in the first aspect,
The setting information includes information regarding setting whether or not to display the additional image on the second terminal.
When the setting information is information to display the additional image on the second terminal, the setting means can set to display the additional image on the second terminal as the display mode. ..

Also, in the first aspect,
The receiving means receives attribute information about the user who owns the second terminal, and receives the attribute information.
The setting means can set the display mode of the additional image in the second terminal with reference to the attribute information received by the receiving means.

Also, in the first aspect,
The setting means can set the display mode of the additional image on the second terminal with reference to the date and time information regarding the current date and time.

The second aspect is
A server device equipped with a computer that functions as a reception means, a setting means, a judgment means, and an instruction means by executing a program.
The reception means receives intellectual property information regarding the intellectual property of the object and receives it.
The setting means sets a display mode of an additional image added to the object on the user's second terminal.
It is appropriate for the determination means to display the additional image on the second terminal in a state of being attached to the object in the display mode set by the setting means based on the received intellectual property information. Judge if there is,
When it is determined by the determination means that it is appropriate for the instruction means to display the additional image on the second terminal in a state of being attached to the object, the display mode set by the setting means. It is a server device that displays the additional image on the second terminal in a state of being attached to the object.

Also, in the second aspect,
When the instruction means displays an image of the surrounding environment detected by the detection unit of the second terminal on the second terminal, the additional image is displayed on the peripheral environment in the display mode set by the setting means. It can be combined with an image and displayed on the second terminal.

Also, in the second aspect,
The receiving means receives information about the surrounding environment of the second terminal detected by the detection unit of the second terminal as peripheral environment information from the second terminal.
The determination means adds the additional image to the object in the display mode set by the setting means based on the intellectual property information about the object in the surrounding environment information received by the reception means. It is possible to determine whether or not it is appropriate to display the information on the second terminal in this state.

Also, in the second aspect,
The determination means also refers to the extent to which the object is included in the surrounding environment information received by the reception means, and the additional image is added to the object in the display mode set by the setting means. It is possible to determine whether or not it is appropriate to display the image on the second terminal.

Also, in the second aspect,
The receiving means receives information about the current position of the second terminal as the current position information from the second terminal.
The determination means is the first in a state where the additional image is added to the object in the display mode set by the setting means based on the intellectual property information and the current position information received by the reception means. 2 It is possible to determine whether or not it is appropriate to display on the terminal.

Also, in the second aspect,
The reception means receives information about the virtual space and receives information.
When the instruction means displays the image of the virtual space received by the reception means on the second terminal, the instruction means synthesizes the additional image with the image of the virtual space in the display mode set by the setting means. Can be displayed on the second terminal.

Also, in the second aspect,
The receiving means receives information about the display mode of the second terminal of the user as display mode information, and receives the information.
The determination means is the first in a state where the additional image is added to the object in the display mode set by the setting means based on the intellectual property information and the display mode information received by the reception means. 2 It is possible to determine whether or not it is appropriate to display on the terminal.

The third aspect is
An information processing system including a reception means, a setting means, and an instruction means.
The reception means receives setting information regarding the display setting of the additional image from the first terminal, and receives the setting information.
The setting means sets the display mode of the additional image on the user's second terminal based on the setting information received by the reception means.
The instruction means is an information processing system that causes the second terminal to display an image in which the additional image is selectively synthesized in the display mode set by the setting means.

The fourth aspect is
An information processing system including a reception means, a setting means, a judgment means, and an instruction means.
The reception means receives intellectual property information regarding the intellectual property of the object and receives it.
The setting means sets a display mode of an additional image added to the object on the user's second terminal.
It is appropriate for the determination means to display the additional image on the second terminal in a state of being attached to the object in the display mode set by the setting means based on the received intellectual property information. Judge if there is,
When it is determined by the determination means that it is appropriate for the instruction means to display the additional image on the second terminal in a state of being attached to the object, the display mode set by the setting means. It is an information processing system that displays the additional image on the second terminal in a state of being attached to the object.

The fifth aspect is
A program that causes a computer to function as a reception means, a setting means, and an instruction means.
The reception means receives setting information regarding the display setting of the additional image from the first terminal, and receives the setting information.
The setting means sets the display mode of the additional image on the user's second terminal based on the setting information received by the reception means.
The instruction means is a program that causes the second terminal to display an image in which the additional image is selectively combined in the display mode set by the setting means.

The sixth aspect is
A program that causes a computer to function as a reception means, a setting means, a judgment means, and an instruction means.
The reception means receives intellectual property information regarding the intellectual property of the object and receives it.
The setting means sets a display mode of an additional image added to the object on the user's second terminal.
It is appropriate for the determination means to display the additional image on the second terminal in a state of being attached to the object in the display mode set by the setting means based on the received intellectual property information. Judge if there is,
When it is determined by the determination means that it is appropriate for the instruction means to display the additional image on the second terminal in a state of being attached to the object, the display mode set by the setting means. This is a program for displaying the additional image on the second terminal in a state of being attached to the object.

According to the present disclosure, additional images can be combined and displayed on a user terminal.

It is a block diagram which shows the structure of the information processing system in this embodiment. It is a figure which shows the surrounding environment when a user wears an AR device, and the image displayed on the AR device. It is a figure which shows the image of the virtual space displayed on the monitor of the computer used by the user. It is a flow diagram which shows the flow of a series of operations at the time of displaying an additional image on a user's AR device or a computer in the display mode set by the setting means. It is a flow diagram which shows the flow of a series of operations at the time of displaying an additional image judged to be appropriate based on intellectual property information on a user's AR device or a computer.

[Embodiment]
The information processing system 1 of the embodiment of the present disclosure will be described with reference to the drawings.

<Explanation of information processing system 1>
In the information processing system 1 shown in FIG. 1, the server device 2 (computer), the AR device 3 used by the user (second terminal), the computer 4 used by the user (second terminal), and the display of additional images are set. Computers 5 (first terminals) used by a person (hereinafter, also referred to as a setter) are connected to each other so as to be able to communicate with each other via a communication network 6.

Objects include buildings existing in real space, billboards for buildings, monuments, animals, and the like. Buildings include, for example, single-family homes, commercial buildings, condominiums, and the like. Advertising signs for buildings include those installed on the roof of a building and those installed on the side walls of a building. Monuments include monuments, statues, etc. Local property includes vehicles, clothing, ornaments, etc.

In addition, the object is not limited to the one existing in the real space, and includes the virtual building existing in the virtual space, the advertising signboard of the building, the monument, the movable property, and the like. The object existing in the virtual space may be a copy of a real estate such as a building or a moving property such as a car that actually exists, or may be a real estate or a moving property that does not actually exist.

The additional image is an image in the real space (including an image recognized by the naked eye and an image captured by the camera) displayed on the display unit 38 of the AR device 3 described later when the user uses the AR device 3. It is an image to be combined. The additional image is an image that does not exist in the real space, and includes, for example, an image displayed on the side surface of the building on the display unit 38 of the AR device 3 and an image such as an advertisement posted on an advertising signboard. Further, the additional image may be an image of an independent advertising signboard itself that does not exist in the real space.

Further, the additional image is an image synthesized with an image of the virtual space displayed on the display unit 48 of the computer 4, which will be described later, when the user uses the computer 4. The additional image is an image that does not exist in the virtual space formed from various data such as map data and building data, and is, for example, posted on an image displayed on the side surface of the building or an advertising signboard on the display unit 48 of the computer 4. Includes images such as advertisements or images of independent advertising signs themselves.

The setter generally has the right to the object. For example, when the object is a building, the setter is the owner, the lessee, or the like of the building. When the building is, for example, a commercial building or a condominium, a setter may be assigned to each floor.

Further, when the object is an advertising signboard of a building or a vehicle, the setter is a person or the like who has the right to post an advertisement on the advertising signboard. When the object is a monument, the setter is the owner or manager of the monument. If the object is a movable property, the setter is the owner or the borrower of the movable property. Identification information such as an identification number of several digits or several tens of digits is assigned to each setter.

Further, when the object is an object in the virtual space (for example, a building object, an advertising signboard object), the setter is a person who has the right to post an advertisement on the object. Specifically, the creator who created and provided the virtual space is given the right to post the advertisement to the setter by transferring the right to present the advertisement to the object in the virtual space.

Further, even if the object does not have the right, the person who sets the display of the additional image may be the set person. For example, in the case of synthesizing an image of an independent billboard itself that does not exist in the real space or virtual space as an additional image and displaying it on the user's AR device 3 or computer 4, the setter can display such an additional image. Become the person who makes the settings. That is, the object may be a predetermined area of the real space as an object in addition to the animals and the like existing in the real space.

The AR device 3 is portable to the user and can display an image of the surrounding environment so that the user can visually recognize the AR device 3. Specifically, the AR device 3 captures the surrounding environment by the peripheral detection unit 35 described later to acquire an image, and the display unit 38 described later with the acquired image as it is or with an additional image synthesized. To display. The AR device 3 includes, for example, AR glasses that can be worn by the user, a smartphone or tablet terminal that can be carried by the user, and the like.

<Overview of Information Processing System 1>
The server device 2 (computer) receives setting information regarding the display setting of the additional image from the computer 5 of the setter, and sets the display mode of the additional image on the user's AR device 3 or the computer 4 based on the received setting information. do. Specifically, as a display mode of the additional image, it is set whether or not the additional image is combined with the image in the real space or the virtual space and displayed on the user's AR device 3 or the computer 4.

Then, the server device 2 selectively synthesizes the additional images in the set display mode and displays them on the user's AR device 3 or the computer 4. At this time, if a display mode is set in which additional images are combined and not displayed on the user's AR device 3 or computer 4, the user's AR device 3 or computer 4 has the real space or virtual space itself. The image is displayed.

The information system 1 of the present embodiment can be applied to both augmented reality (AR) and virtual reality (VR) techniques. When the information processing system 1 of the present embodiment is applied to the augmented reality (AR) technique, an image obtained by synthesizing an additional image with an image of the surrounding environment of the AR device 3 on the display unit 38 of the AR device 3 possessed by the user. Can be displayed.

Further, when the information processing system 1 of the present embodiment is applied to the technique of virtual reality (VR), when the image of the virtual space is displayed on the display unit 48 of the computer 4, the additional image is synthesized with the image of the virtual space. The image can be displayed on the display unit 48.

<Configuration of information processing system 1>
Hereinafter, each hardware configuration of the server device 2, the user's AR device 3, the computer 4, and the setter's computer 5 will be described with reference to FIG.

<Configuration of server device 2>
As shown in FIG. 1, the server device 2 has a network interface 21, a control unit 22, and a storage unit 23. The network interface 21 and the storage unit 23 are electrically connected to the control unit 22 via the bus 24.

The network interface 21 is communicably connected to the user's AR device 3, the computer 4, and the setter's computer 5 via a communication network 6 such as the Internet and a LAN.

The control unit 22 is composed of a microcomputer including a CPU and a semiconductor memory, and controls the operation of the server device 2.

The control unit 22 functions as a reception unit 221, a setting unit 222, a determination unit 223, and an instruction unit 224 by executing various programs stored in the storage unit 23 described later.

The reception means 221 receives information from the user's AR device 3, the computer 4, and the setter's computer 5, respectively. In the present embodiment, the receiving means 221 receives the setting information regarding the setting of the display of the additional image from the computer 5 of the setter. The setting information includes information regarding setting whether or not to display the additional image on the user's AR device 3 or computer 4. Further, the setting information includes a region to which the additional image is added to the object and the content of the additional image.

Further, the receiving means 221 also receives the identification information of the setting person when receiving the setting information from the computer 5 of the setting person. The configurator's identification information includes at least one of the configurator's name, identification number, telephone number, and address.

Further, the receiving means 221 receives attribute information about the user from the computer 5 of the setter, or the AR device 3 or the computer 4 of the user. The attribute information about the user includes, for example, information such as the user's age, gender, occupation, hobbies, and preferences.

The setting means 222 sets the display mode of the additional image on the user's AR device 3 or the computer 4 based on the setting information received by the receiving means 221. Specifically, the setting means 222 sets whether or not to display the additional image on the user's AR device 3 or computer 4 based on the setting information sent from the computer 5 of the setter, or the AR device 3 or the like. Set the content of the additional image displayed on the computer 4.

The determination means 223 determines whether or not the identification information received by the reception means 221 corresponds to the object to which the additional image is added, based on the correspondence information stored in the storage unit 23 described later. ..

The instruction means 224 causes the user's AR device 3 or the computer 4 to display an image in which an additional image is selectively combined with the image of the surrounding environment in the display mode set by the setting means 222.

Specifically, when the user uses the AR device 3, the instruction means 224 displays an image of the surrounding environment detected by the peripheral detection unit 35 of the AR device 3 described later on the display unit 38 described later. , The additional image is selectively combined with the image of the surrounding environment and displayed on the AR device 3 in the display mode set by the setting means 222.

Further, when the user uses the computer 4, the reception means 221 receives the information of the virtual space in advance. The information received by the reception means 221 is stored in the storage unit 23. Then, the instruction means 224 uses the setting means 222 to display an image of the virtual space on the computer 4 based on the information of the virtual space received by the receiving means 221 (that is, the information stored in the storage unit 23). The computer 4 displays an image in which an additional image is selectively combined with the image in the virtual space in the set display mode.

The storage unit 23 is composed of an HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), or the like.

The storage unit 23 stores information about objects such as buildings existing in the real space and information about the objects in the virtual space. Specifically, the storage unit 23 stores various data such as map data and building data in the real world, and various data such as map data and building data in the virtual space.

Further, the storage unit 23 stores information regarding the correspondence between the identification information of the setter and the object as correspondence information. Such correspondence information is stored in the storage unit 23 by being received from the external device by the reception means 221.

<Configuration of user AR device 3>
As shown in FIGS. 1 and 2, the user's AR device 3 includes a network interface 31, a control unit 32, a graphic processing unit 33, an operation unit 34, a peripheral detection unit 35, a position detection unit 36, a storage unit 37, and a display unit. Has 38. The network interface 31, the graphic processing unit 33, the operation unit 34, the peripheral detection unit 35, the position detection unit 36, and the storage unit 37 are electrically connected to the control unit 32 via the bus 39.

The network interface 31 is communicably connected to the communication network 6 in order to transmit and receive various data between the AR device 3 and the server device 2.

The graphic processing unit 33 creates an image in which an additional image is selectively combined with the image of the surrounding environment detected by the peripheral detection unit 35 described later according to the image information output from the control unit 32 described later. The created image is displayed on the display unit 38 and can be visually recognized by the user.

The operation unit 34 is a switch or the like operated by the user, and the operation content by the user is input to the control unit 32. The operation content by the user is a command for turning the power on / off and switching whether or not to combine the additional image with the image of the surrounding environment in the display mode set by the setting means 222.

The peripheral detection unit 35 captures the surrounding environment of the AR device 3, and is composed of, for example, a camera, a sensor, or the like. The image of the surrounding environment of the AR device 3 captured by the peripheral detection unit 35 is input to the control unit 32.

The position detection unit 36 detects the current position of the AR device 3, and is composed of, for example, GPS or the like. Information on the current position of the AR device 3 detected by the position detection unit 36 is input to the control unit 32.

The storage unit 37 is composed of an HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), or the like. The storage unit 37 stores the identification information of the AR device 3 and the attribute information related to the user. Further, the storage unit 37 stores a program executed by the control unit 32.

When the AR device 3 is used as AR glasses (smart glasses), the display unit 38 includes a small monitor corresponding to both eye lenses of the AR glasses. Further, the display unit 38 includes a monitor such as a touch panel when the AR device 3 is used as a smartphone or a tablet terminal.

The control unit 32 is composed of a microcomputer including a CPU and a semiconductor memory, and controls the operation of the AR device 3. The control unit 32 functions as the communication means 321 and the reception means 322 and the display control means 323 by executing various programs stored in the storage unit 37.

The communication means 321 is a function of communicating with the server device 2 via the network interface 31. The communication means 321 transmits to the server device 2 an image of the surrounding environment of the AR device 3 detected by the peripheral detection unit 35 and information on the current position of the AR device 3 detected by the position detection unit 36. Further, the communication means 321 receives the information regarding the display mode of the additional image sent from the server device 2.

The receiving means 322 receives an image of the surrounding environment of the AR device 3 detected by the peripheral detection unit 35 and information on the current position of the AR device 3 detected by the position detecting unit 36. Further, the receiving means 322 receives information regarding the display mode of the additional image transmitted from the server device 2 to the AR device 3.

The display control means 323 controls the image displayed on the display unit 38 by controlling the graphic processing unit 33 based on the information regarding the display mode of the additional image received by the reception means 322.

<Configuration of user's computer 4>
As shown in FIGS. 1 and 3, the user's computer 4 has a network interface 41, a control unit 42, a graphic processing unit 43, an audio processing unit 44, an operation unit 45, a storage unit 46, and a display unit 48. The network interface 41, the graphic processing unit 43, the audio processing unit 44, the operation unit 45, and the storage unit 46 are electrically connected to the control unit 42 via the bus 49.

The network interface 41 is communicably connected to the communication network 6 in order to transmit and receive various data between the computer 4 and the server device 2.

The graphic processing unit 43 creates an image in which an additional image is selectively combined with the image of the surrounding environment transmitted from the server device 2 according to the image information output from the control unit 42 described later. The created image is displayed on the display unit 48 and can be visually recognized by the user.

The audio processing unit 44 is connected to a speaker (not shown) of the computer 4, creates voice according to the instruction of the control unit 42 described later, and outputs the voice from the speaker.

The operation unit 45 is a keyboard or the like operated by the user, and the operation content by the user is input to the control unit 42. The operation content by the user includes, for example, an operation instructing to move in the virtual space displayed on the display unit 48.

The storage unit 46 is composed of an HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), or the like. The storage unit 46 stores the identification information of the computer 4 and the attribute information related to the user. Further, the storage unit 46 stores a program executed by the control unit 42.

The display unit 48 includes a monitor such as a head-mounted display. The display unit 48 is not limited to the head-mounted display, and another type of monitor (for example, a desktop display) may be used as long as it can display a virtual space.

The control unit 42 is composed of a microcomputer including a CPU and a semiconductor memory, and controls the operation of the computer 4. The control unit 42 functions as a communication means 421, a reception means 422, and a display control means 423 by executing various programs stored in the storage unit 46.

The communication means 421 is a function of communicating with the server device 2 via the network interface 41. The communication means 421 transmits information regarding the operation content by the user input by the operation unit 45 to the server device 2. Further, the communication means 421 receives information regarding the display mode of the virtual space map data, the building data, and the additional image sent from the server device 2.

The reception means 422 receives the operation content by the user input by the operation unit 45. Further, the receiving means 422 receives information regarding the display mode of the virtual space map data, the building data, and the additional image transmitted from the server device 2 to the computer 4.

The display control means 423 controls the graphic processing unit 43 based on the map data and building data of the virtual space received by the reception means 422 and the information regarding the display mode of the additional image, so that the image displayed on the display unit 48 is displayed. To control.

The setter's computer 5 has substantially the same configuration as the user's computer 4. Therefore, the description of the detailed configuration of the computer 5 of the setter will not be repeated.

<Flow of a series of operations of the information processing system 1>
Next, the flow of a series of operations of the information processing system 1 will be described with reference to FIG.

First, a case where the information processing system 1 of the present embodiment is applied to the augmented reality (AR) technique will be described. First, the setter sets the display of the additional image by the computer 5. As a setting for displaying such an additional image, an additional image is added to an object existing in the real space, and a new additional image is added regardless of the object existing in the real space. There are two methods.

Specifically, when the setter inputs a command to the computer 5 to start setting the display of the additional image, the computer 5 transmits the setting start command to the server device 2. When the server device 2 receives the setting start command, it transmits information about a building or the like existing in the real space stored in the storage unit 23 to the computer 5. As a result, information about buildings and the like existing in the real space is displayed on the monitor of the computer 5.

Then, when the setter selects an object to which an additional image is to be added from the buildings existing in the real space displayed on the monitor, the setting information that the additional image is added to the selected object is input to the computer. You will be able to enter in 5. Specifically, when the setter selects a building as an object, an image such as an advertisement can be added as an additional image to the side surface of the building on the computer 5.

Further, when the setter wants to add a new additional image regardless of the object existing in the real space, the area where the additional image is to be added on the screen of the real space displayed on the monitor on the computer 5 and the area where the additional image is to be added. Select the content of the additional image. The content of the additional image is an image of a building, an advertising signboard, or the like that does not exist in the real space.

The setting information input to the computer 5 by the setter is transmitted from the computer 5 to the server device 2. Further, when the setting information input to the computer 5 by the setter is transmitted from the computer 5 to the server device 2, the identification information of the setter is also transmitted to the server device 2. When the setting information and the identification information of the setter are received by the receiving means 221 (YES in step st1), the accepted setting information and the identification information of the setter are stored in the storage unit 23 (step st2).

Further, at this time, the determination means 223 determines whether or not the identification information received by the reception means 221 and the object to which the additional image is added correspond to each other based on the correspondence information stored in the storage unit 23. To judge. Then, only when it is determined by the determination means 223 that the two match, the instruction means 224 stores the received setting information and the identification information of the setter in the storage unit 23. That is, even if a person other than the setter who has the right to the object inputs the setting information to the computer 5, the setting information is not stored in the storage unit 23.

After that, when the user goes out with the AR device 3 in possession and the user turns on the power of the AR device 3 by the operation unit 34, the display instruction is transmitted from the AR device 3 to the server device 2. At this time, the image of the surrounding environment of the AR device 3 detected by the peripheral detection unit 35 and the information regarding the current position of the AR device 3 detected by the position detection unit 36 are also transmitted to the server device 2. Further, the attribute information about the user stored in the storage unit 37 is also transmitted from the AR device 3 to the server device 2.

When the receiving means 221 of the server device 2 receives the display instruction (YES in step st3), the setting means 222 receives the set information (that is, the setting information stored in the storage unit 23), and the user's AR The display mode of the additional image on the device 3 is set (step st4).

Specifically, when the server device 2 accepts the setting information of synthesizing the additional image with the image in the real space, the setting means 222 adds the image to the image in the real space detected by the peripheral detection unit 35. It is set to display the combined image on the AR device 3.

More specifically, the setting means 222 provides setting information, information on the surrounding environment detected by the peripheral detection unit 35 (peripheral environment information), and information on the current position of the AR device 3 detected by the position detection unit 36 (current position). Information), the display mode of the additional image is set. Specifically, the setting means 222 uses at least one of the surrounding environment information and the current position information to determine whether or not an object or region to which an additional image should be added exists near the AR device 3. Identify. At this time, information regarding the direction in which the AR device 3 is facing may be transmitted from the AR device 3 to the server device 2. In this case, whether or not the object exists in the area where the AR device 3 is facing (that is, whether or not the object can be visually recognized by the user) is determined by referring to the direction in which the AR device 3 is facing. It may be judged by 223.

Then, when it is specified that the object or region to which the additional image is to be added exists near the AR device 3, the instruction means 224 ARs the image obtained by synthesizing the additional image in the set display mode. It is displayed on the display unit 38 of the device 3 (step st5). Specifically, an additional image is synthesized with an image of the surrounding environment detected by the peripheral detection unit 35 of the AR device 3, and the combined image is transmitted from the server device 2 to the AR device 3. In this way, the instruction means 224 synthesizes and displays the additional image on the display unit 38 of the AR device 3.

When the operation unit 34 of the AR device 3 inputs a command not to synthesize the additional image with the image of the surrounding environment, the instruction means 224 combines the image of the peripheral environment with the image of the surrounding environment into the AR device. It is not displayed on the display unit 38 of 3. In this case, the image of the surrounding environment detected by the peripheral detection unit 35 is displayed as it is on the display unit 38 of the AR device 3.

The setting means 222 also refers to the attribute information about the user when setting the display mode of the additional image. Specifically, for example, when the image of the advertisement is displayed on the AR device 3 as an additional image, the image of the advertisement that matches the attribute information of the user is preferentially displayed on the display unit 38 and does not match the attribute information of the user. The image of the advertisement is not displayed on the display unit 38. For example, when the user is a child as the attribute information of the user, the image of the advertisement related to the child is preferentially displayed on the display unit 38, and the image of the advertisement unsuitable for the child is not displayed on the display unit 38. ..

The setting means 222 also refers to date and time information regarding the current date and time when setting the display mode of the additional image. The date and time information includes the current month, day of the week, date, time, and time zone. For example, when the image of the advertisement is displayed on the AR device 3 as an additional image, the image of the advertisement that matches the current time is preferentially displayed on the display unit 38. Specifically, an image of an advertisement for a restaurant or food related to lunch is displayed on the display unit 38 in the morning time, and an image of an advertisement for a restaurant or alcoholic beverage related to dinner is displayed on the display unit 38 in the afternoon time.

A specific example of the additional image displayed on the AR device 3 will be described with reference to FIG. FIG. 2 shows an example in which AR glasses are used as the AR device 3. In the peripheral environment 100 of the AR device 3, the first building 102 and the second building 106 exist, and the advertising signboard 104 is installed on the roof of the first building 102. In the example shown in FIG. 2, the advertising signboard 104 is in a state in which no advertisement is posted.

In such a surrounding environment 100, for example, when the setter is the owner or the borrower of the first building 102 or the advertising signboard 104, when it is desired to display the image of the advertisement on the side surface 103 of the first building 102, or When it is desired to display the image of the advertisement on the advertisement signboard 104, the computer 5 is set to synthesize the image of the advertisement with the image of the surrounding environment. As a result, in the image 100a of the surrounding environment displayed on the display unit 38 of the AR device 3, the image of the first building 102a in which the image of the advertisement is displayed on the side surface and the image of the advertising signboard 104a in which the image of the advertisement is displayed are displayed. It can be displayed.

Further, the setter can set where to synthesize the image of the advertisement in the first building 102 or the advertising signboard 104 by the computer 5. For example, when the setter is the owner or the lessee of the first building 102, the setter is, for example, an area where the image of the advertisement on the side surface of the first building 102 is synthesized (for example, the right half, above the fifth floor, etc. The whole, a place other than a predetermined area, etc.) can be selected. Further, when the setter is assigned to each floor of the first building 102, the setter can select the area where the image of the advertisement is combined for each floor.

Further, when the setter is the owner or the borrower of the advertising sign 104, the setting person can select the area where the image of the advertisement on the advertising sign 104 is combined. At this time, the advertising signboard 104 can be divided into a plurality of areas, and the image of the advertisement can be combined in each area.

Further, even if the advertisement is posted on the advertisement signboard 104 in the real space, the image of the advertisement set by the setter in the image 100a displayed on the display unit 38 of the AR device 3 is combined with the advertisement existing in the real space. Can be displayed. At this time, the person who set the AR device 3 can synthesize and display an advertisement that matches the content of the advertisement of the advertisement signboard 104 in the real space, but cannot synthesize and display an advertisement that does not match. Can be done on the computer 5.

In addition, when the setter is the owner or the lessee of the first building 102 or the advertising signboard 104, at a predetermined time or date and time (for example, every few seconds, in the morning, afternoon, weekdays, holidays, holidays, anniversaries or months). In addition, the computer 5 can be set to switch and display a plurality of advertisement images in the same area of the first building 102 or the advertising signboard 104.

Further, the setter synthesizes an additional image that matches the image of the surrounding environment detected by the peripheral detection unit 35 and displays it on the display unit 38, but does not display the additional image that does not match the image of the peripheral environment on the display unit 38. It is possible to make such settings. For example, when a religion-related facility (for example, a shrine or a torii) in the surrounding environment detected by the peripheral detection unit 35 is included, an image related to another religion (for example, an image of Christmas illuminations) is added as an additional image. ) Can be combined and set so as not to be displayed on the display unit 38. In addition, when public facilities such as school facilities, medical institutions, police stations, fire departments, and military facilities in the surrounding environment detected by the peripheral detection unit 35 are included, additional images should not be combined and displayed on the display unit 38. Settings can be made so that advertisement images of private companies are not displayed.

Further, when the image of the surrounding environment detected by the peripheral detection unit 35 includes the advertising signboard of the company, the image of the advertising signboard of the competitor of this company is combined with the image of the surrounding environment and displayed on the display unit 38. The setter can make settings that do not allow it.

Further, for example, when the setter is the owner of the second building 106, even if the advertising signboard is not installed on the roof of the actual second building 106, the setter is the second building 106 in the AR device 3. If you want to display an advertising sign on the roof, you can set it with the computer 5. As a result, in the image 100a of the surrounding environment displayed on the display unit 38 of the AR device 3, the image of the advertising signboard 108a on which the advertisement is displayed can be displayed in a state of being installed on the roof of the second building 106a.

As shown in FIG. 2, by synthesizing the image of the advertisement with the image of the actual surrounding environment in the AR device 3 and displaying it, it is possible to obtain advertising revenue without spending the cost of installing an advertising signboard on a building or the like. can. In particular, advertising is not limited to commercial buildings, but also in residential facilities such as private houses and condominiums, structures such as bridges (including pedestrian bridges) and electric poles, municipal offices, mountains, rivers, etc. without actually installing advertising signs. You can earn income. Further, only the image of the advertisement suitable for the building and the surrounding environment in which the building is installed can be combined with the image of the actual surrounding environment and displayed on the AR device 3.

Next, a case where the information processing system 1 of the present embodiment is applied to a virtual reality (VR) technique will be described. First, the setter sets the display of the additional image by the computer 5. Specifically, when the setter inputs a command to the computer 5 to start setting the display of the additional image, the computer 5 transmits the setting start command to the server device 2.

When the server device 2 receives the setting command, the server device 2 transmits the information of the virtual space stored in the storage unit 23 to the computer 5, and the image of the virtual space is displayed on the monitor of the computer 5.

Then, when the setter selects an object to which the additional image is to be added from the image of the virtual space displayed on the monitor of the computer 5, the setting information of the additional image can be input to the computer 5. Specifically, when the setter selects an object (building) from, for example, an image in a virtual space, an image such as an advertisement can be added as an additional image to the side surface of the object.

Further, when the setter wants to add a new additional image to the image of the virtual space displayed on the monitor of the computer 5, the area where the new additional image should be combined is selected. After that, the setter can input the setting information of what kind of additional image is to be added to the computer 5.

The setting information input to the computer 5 by the setter is transmitted from the computer 5 to the server device 2. Further, when the setting information input to the computer 5 by the setter is transmitted from the computer 5 to the server device 2, the identification information of the setter is also transmitted to the server device 2. When the setting information and the identification information of the setter transmitted to the server device 2 are received by the receiving means 221 (YES in step st1), the accepted setting information and the identification information of the setter are stored in the storage unit 23 (YES in step st1). Step st2).

Further, at this time, the determination means 223 determines whether or not the identification information received by the reception means 221 and the object to which the additional image is added correspond to each other based on the correspondence information stored in the storage unit 23. To judge. Then, only when it is determined by the determination means 223 that the two match, the instruction means 224 stores the received setting information and the identification information of the setter in the storage unit 23. That is, even if a person other than the setter who has the right to the object inputs the setting information to the computer 5, the setting information is not stored in the storage unit 23.

After that, when the user moves in the virtual space displayed on the display unit 48 of the computer 4 by the operation unit 45 or the like, the display instruction is transmitted from the computer 4 to the server device 2. At this time, information about the current position in the virtual space is also transmitted to the server device 2. Further, the attribute information about the user stored in the storage unit 46 is also transmitted from the computer 4 to the server device 2.

When the receiving means 221 of the server device 2 receives the display instruction (YES in step st3), the setting means 222 sets the display mode of the additional image on the user's computer 4 based on the received setting information (step st4). ..

Specifically, when the server device 2 accepts the setting information of adding an additional image to the image of the virtual space, the setting means 222 uses an image obtained by synthesizing the additional image with the image of the virtual space as a display mode. It is set to be displayed on the display unit 48 of the computer 4.

More specifically, the setting means 222 sets the display mode of the additional image based on the setting information and the information regarding the current position in the virtual space (current position information). Specifically, the setting means 222 specifies whether or not the object or region to which the additional image is to be added exists near the current position in the virtual space.

Then, when it is specified that the object or region to which the additional image is to be added exists near the current position in the virtual space, the instruction means 224 is an image to which the additional image is added in the set display mode. Is displayed on the display unit 48 of the user's computer 4 (step st5). Specifically, an additional image is combined with the image of the virtual space stored in the storage unit 23 in a set display mode, and the combined image is transmitted from the server device 2 to the user's computer 4. Then, the combined image is displayed on the display unit 48 of the user's computer 4.

The setting means 222 also refers to the attribute information about the user when setting the display mode of the additional image. Specifically, for example, when the image of the advertisement is displayed on the display unit 48 of the user's computer 4 as an additional image, the image of the advertisement that matches the attribute information of the user is preferentially displayed on the display unit 48 of the user. The display unit 48 is prevented from displaying the image of the advertisement that does not match the attribute information.

The setting means 222 also refers to date and time information regarding the current date and time when setting the display mode of the additional image. Specifically, for example, when the image of the advertisement is displayed on the display unit 48 of the user's computer 4 as an additional image, the display unit 38 preferentially displays the advertisement suitable for the current time.

Further, when displaying the image obtained by synthesizing the additional image in the display mode set by the setting means 222 on the display unit 48 of the computer 4, the image of the two-dimensional code indicating the URL of the website introducing the additional image is synthesized. You may be able to do it. In this case, by reading the two-dimensional code attached to the additional image in the virtual space displayed on the display unit 48 of the computer 4, it is possible to browse a website that introduces products and the like related to the additional image. Become.

A specific example of the image displayed on the user's computer 4 will be described with reference to FIG. As shown in FIG. 3, in the image 200 of the virtual space displayed on the display unit 48 of the computer 4, the first building 202 and the second building 206 are present, and the advertising signboard 204 is installed on the roof of the first building 202. Has been done.

In the image 200 of such a virtual space, for example, when the setter wants to display the advertisement on the side surface 203 of the first building 202 or wants to display the advertisement on the advertising signboard 204, the setting is made by the computer 5. As a result, in the image 200 of the virtual space displayed on the display unit 48 of the computer 4, the image of the first building 202 in which the advertisement is displayed on the side surface and the image of the advertising signboard 204 in which the advertisement is displayed can be displayed. ..

Further, even if the advertising signboard is not installed on the rooftop of the second building 206 in the image 200 of the virtual space, if the setter wants to display the advertising signboard on the rooftop of the second building 206, the setting is made by the computer 5. I do. As a result, in the image 200 of the virtual space displayed on the display unit 48 of the computer 4, the image of the advertising signboard 208 on which the advertisement is displayed can be displayed in a state of being installed on the roof of the second building 206.

In this way, as shown in FIG. 3, by synthesizing and displaying the image of the advertisement on the image of the virtual space on the user's computer 4, the owner of the object in the virtual space also obtains the advertisement revenue in the virtual space. be able to. Further, the owner of the object in the virtual space can sell the right to place an advertisement on the object to, for example, an advertising agency, so that the advertising agency can lend the area of the object to the person who wants to place the advertisement.

To summarize the above, the server device 2 includes a receiving means 221, a setting means 222, and a computer that functions as an instruction means 224 by executing a program. The reception means 221 receives setting information regarding the display setting of the additional image from the computer 5 (first terminal) of the setter.

The setting means 222 sets the display mode of the additional image on the user's AR device 3 or the computer 4 (second terminal) based on the setting information received by the receiving means 221. The instruction means 224 causes the user's AR device 3 or the computer 4 to display an image obtained by synthesizing the additional image in the display mode set by the setting means 222.

<Effect>
According to the present embodiment, by setting the display mode of the additional image based on the received setting information, the image obtained by synthesizing the additional image with the image in the real space or the virtual space is displayed on the user's AR device 3 or the computer 4. Can be made to.

<Other embodiments>
The various control means and processing procedures described in the embodiments are examples, and are not intended to limit the scope of the present invention, its applications or its uses. Various control means and processing procedures can be appropriately redesigned without changing the gist of the present invention.

For example, in the above description, the mode in which the setter inputs the setting information by the computer 5, the input setting information is transmitted from the computer 5 to the server device 2, and the receiving means 221 receives the setting information has been described. However, as another aspect, the setting person may be able to input the setting information by the reception-only terminal prepared by the platformer such as the administrator of the server device 2.

Further, it is not the server device 2 but the user's AR device 3 or computer 4 that determines whether or not to display the combined image of the additional image on the user's AR device 3 or computer 4 in the display mode set by the setting means 222. It may be done on the side.

Further, the additional image is not limited to the image of the advertisement. As additional images, images that decorate the walls of buildings, etc., names and addresses of buildings, explanations, directions, messages (for example, danger in the future, steps, →, ↑, welcome to 〇〇, 3rd floor by elevator Various types of images may be used, such as images containing characters and icons that indicate (such as), images that decorate animals, images that show specific locations of location-based games using AR technology, and so on. .. In addition, it may be a guidance image for evacuating residents in the event of a disaster, or a guidance image for an event such as a fireworks display.

The right to set the additional image of the setter may be bought, sold, or lent. For example, when a given business owner conducts an event, the setter may lend this right, or the business owner may lend this right from the setter to display advertisements for his or her products or services. You can get it. Then, the setting means 222 stores and manages the information about the setter. Further, in the case of an emergency such as a disaster, the information necessary for the emergency may be displayed regardless of the setting information.

Further, as another embodiment, the reception means 221 may receive intellectual property information regarding the intellectual property of the object from the terminal of the intellectual property right holder. In this case, the determination means 223 attaches an image in which the additional image is added to the object in the display mode set by the setting means 222 based on the received intellectual property information, and the user's AR device 3 or the computer 4 (second terminal). ) Is appropriate to display.

Then, the instruction means 224 displays an image in which the additional image is added to the object in the display mode set by the setting means 222 on the user's AR device 3 or the like only when it is determined by the determination means 223 to be appropriate. Let me.

In such other embodiments, the intellectual property of the object includes the patent right, design right, trademark right, copyright, etc. of the object. In addition, the intellectual property information includes information on whether or not the object is protected by the intellectual property, and if the object is protected by the intellectual property, identification information of the right holder of the intellectual property. In addition, if the object is protected by intellectual property, the intellectual property information is registered with, for example, the registration number or registration content of the patent right, design right, trademark right registered with the Japan Patent Office, or with the Cultural Agency. Includes the registration number and registration details of the copyright.

Further, the intellectual property information is such that when the object is protected by the intellectual property, an additional image is added to the image of the object displayed on the user's AR device 3 or the computer 4. Includes information on whether or not the right holder accepts it.

The intellectual property right holder is the copyright holder of the copyrighted work, the right holder of the trademark such as the logo attached to the object, the design right holder when the object is protected by the design right, and the patent right of the object. Includes patentees when protected by.

Details of such other embodiments will be described below. The information processing system 1 of the other embodiment can be applied to both augmented reality (AR) and virtual reality (VR) techniques. When the information processing system 1 of another embodiment is applied to the augmented reality (AR) technique, the image of the object is displayed on the display unit 38 of the AR device 3 possessed by the user as the image of the surrounding environment of the AR device 3. It can be displayed in a synthesized state.

Further, when the information processing system 1 of another embodiment is applied to the technique of virtual reality (VR), when the image of the object in the virtual space is displayed on the display unit 48 of the computer 4, the additional image is displayed in the virtual space. It can be displayed on the display unit 48 in a state of being combined with the image.

The configuration of the information processing system 1 of such another embodiment will be described. The same contents as those of the present embodiment described above will be omitted.

The reception means 221 of the server device 2 is an intellectual property related to the intellectual property of the object from the terminal of the intellectual property right holder (for example, a computer) or the reception-only terminal prepared by a platform such as the administrator of the server device 2. Accept information. The intellectual property information received by the reception means 221 is stored in the storage unit 23. Further, the receiving means 221 receives information about the display mode of the AR device 3 or the computer 4 from the user's AR device 3 or the computer 4 as the display mode information.

The setting means 222 sets the display mode of the additional image on the user's AR device 3 or the computer 4. Since the details of the display mode of the additional image are described above, they will be omitted.

The determination means 223 causes the user's AR device 3 or the computer 4 to display the additional image in the state of being attached to the object in the display mode set by the setting means 222 based on the intellectual property information received by the reception means 221. Determine if this is appropriate. Further, when the display mode information is received by the reception means 221, the determination means 223 may display the additional image on the user's AR device 3 or computer 4 in a state of being attached to the object with reference to the display mode information. Determine if it is appropriate. The details of the content of the judgment by the judgment means 223 will be described later.

When the instruction means 224 determines that it is appropriate for the determination means 223 to display the additional image on the user's AR device 3 or computer 4 with the additional image added to the object, the user uses the combined image of the additional image. It is displayed on the AR device 3 or the computer 4. Specifically, a command including a display mode of an additional image is transmitted from the server device 2 to the AR device 3 and the computer 4 via the network interface 21.

Next, a series of operation flows of the information processing system 1 of another embodiment will be described with reference to FIG.

First, a case where the information processing system 1 of another embodiment is applied to the technique of augmented reality (AR) will be described. In another embodiment, intellectual property information regarding the intellectual property of the object is transmitted to the server device 2. As for such intellectual property information, when the intellectual property right holder inputs the intellectual property information of the object into a terminal such as a computer, the intellectual property information is transmitted from this terminal to the server device 2. Further, the intellectual property information may be transmitted to the server device 2 from a terminal other than the intellectual property right holder, specifically, for example, a computer of an organization or group having jurisdiction over the intellectual property information.

When the intellectual property information is received by the receiving means 221 transmitted to the server device 2 (YES in step st11), the received intellectual property information is stored in the storage unit 23 (step st12).

After that, the setter sets the display of the additional image by the computer 5. The setting information input to the computer 5 by the setter and the identification information of the setter are transmitted from the computer 5 to the server device 2 and accepted by the receiving means 221 (YES in step st13). The setting information and the identification information of the setter received by the receiving means 221 are stored in the storage unit 23 (step st14). Since the setting of the display of the additional image is described above, the details will be omitted.

Then, when the user goes out with the AR device 3 in possession and the operation unit 34 turns on the power of the AR device 3, the display instruction is transmitted from the AR device 3 to the server device 2. At this time, the image of the surrounding environment of the AR device 3 detected by the peripheral detection unit 35 and the information regarding the current position of the AR device 3 detected by the position detection unit 36 are also transmitted to the server device 2. In addition, attribute information about the user is also transmitted from the AR device 3 to the server device 2.

When the receiving means 221 of the server device 2 receives the display instruction (YES in step st15), the setting means 222 sets the display mode of the additional image in the user's AR device 3 based on the received setting information (step st16). ). Since the setting of the display mode of the additional image is described above, the details will be omitted.

Then, the determination means 223 displays the additional image on the user's AR device 3 in a state where the additional image is added to the object in the display mode set by the setting means 222 based on the intellectual property information stored in the storage unit 23. It is determined whether or not it is appropriate to make it (step st17). For example, if the object to which the additional image should be attached is protected by intellectual property, the additional image is targeted only when the right holder of the intellectual property permits the addition of the additional image. It is determined that it is appropriate to display the image attached to the object on the AR device 3.

When the determination means 223 determines that it is appropriate to display the additional image on the user's AR device 3 (YES in step st17), the instruction means 224 displays the image of the surrounding environment detected by the peripheral detection unit 35. The additional image is combined in the set display mode. Then, the instruction means 224 transmits the combined image from the server device 2 to the AR device 3. In this way, the instruction means 224 causes the display unit 38 of the AR device 3 to display the combined image (step st18).

On the other hand, when it is determined that it is not appropriate to display the image added to the object by the additional image on the user's AR device 3 (NO in step st17), the instruction means 224 uses the additional image as the object. The image added to is not displayed on the AR device 3 (step st19). In this case, the image of the surrounding environment detected by the peripheral detection unit 35 is displayed on the display unit 38 as it is.

Further, it is appropriate that the determination means 223 displays the additional image on the AR device 3 in the display mode set by the setting means 222 with reference to the degree to which the object is included in the surrounding environment information received by the receiving means 221. Judge whether or not it is. For example, for an object protected by intellectual property, the determination means 223 may be used only when the time for which the object is continuously detected by the peripheral detection unit 35 or the position detection unit 36 is longer than a predetermined time. It may be determined whether or not it is appropriate to display the additional image on the AR device 3.

In this case, if the time for which the object is continuously detected by the peripheral detection unit 35 is shorter than the predetermined time, the determination means 223 does not make a judgment, and the peripheral environment detected by the peripheral detection unit 35 is not performed. The image of is displayed on the display unit 38 as it is. On the other hand, when the time for continuous detection of the object by the peripheral detection unit 35 is longer than a predetermined time, the additional image is added to the object in the set display mode after the predetermined time has elapsed. The determination means 223 determines whether or not it is appropriate to display the AR device 3 in the state.

Further, when the surrounding environment information received by the reception means 221 includes an object, the judgment means 223 may immediately start the judgment. In this case, even if it is determined that it is appropriate to display the additional image on the object 3 in a state of being attached to the object, the time during which the object is continuously detected by the peripheral detection unit 35 is longer than a predetermined time. If it is too short, the image of the surrounding environment detected by the peripheral detection unit 35 is displayed on the display unit 38 as it is.

Further, the reception means 221 receives information regarding the current position of the AR device 3 from the AR device 3 as the current position information, and the determination means 223 is set based on the intellectual property information and the current position information received by the reception means 221. It is determined whether or not it is appropriate to display the image obtained by synthesizing the additional image on the AR device 3 in the display mode set by the means 222. Specifically, the country or state where the AR device 3 exists is specified based on the current location information, and the intellectual property of the object is certified according to the intellectual property system of this specified country or state. ..

Further, in the AR device 3, the display mode of the image displayed on the display unit 38 may be set by the operation unit 34. In this case, the display mode of the AR device 3 set by the operation unit 34 is transmitted from the AR device 3 to the server device 2, and the receiving means 221 accepts the information regarding the display mode of the AR device 3 as the display mode information. The display mode includes, for example, a trademark display mode, a trademark non-display mode, a copyrighted work display mode, a copyrighted work non-display mode, and the like.

In this case, the determination means 223 causes the AR device 3 to display an image obtained by synthesizing the additional image in the display mode set by the setting means 222 based on the intellectual property information and the display mode information received by the receiving means 221. To determine if is appropriate. For example, when the trademark non-display mode is set as the display mode, the image of the object with the trademark is not displayed on the AR device 3, or the trademark among the objects with the trademark is attached. Only the image of the object excluding the above may be displayed on the AR device 3.

When the copyrighted work non-display mode is set as the display mode, the image of the object protected by the copyright is not displayed on the AR device 3.

A specific example of the image displayed on the user's AR device 3 in such another embodiment will be described. For example, if an object such as a monument or bronze statue that actually exists is protected by copyright, whether or not the copyright holder of this object allows the addition of an additional image to this object. Can be set, and the set intellectual property information is transmitted to the server device 2. Further, at this time, the copyright holder can set an area that allows or does not allow the addition of the additional image in the object.

Specifically, for example, when the object is a bronze image and the object is protected by copyright, the image of clothing is combined with the bronze image as an additional image in the image displayed on the AR device 3. The copyright holder can make settings that allow the display to be displayed, but not to combine and display another face.

Further, the copyright holder can make a setting that does not allow the image of the advertisement to be combined with the object as an additional image in the image displayed on the AR device 3. Further, for example, when the object is protected by a copyright, the copyright holder can make a setting that does not allow the addition of an additional image in which the object appears to be damaged or the like.

Further, in the image displayed on the AR device 3, the intellectual property right holder does not set whether or not to allow the addition of the additional image to the object itself protected by the intellectual property, but the object. The intellectual property right holder may be able to set whether or not to allow the addition of an additional image around the object. For example, when the bronze statue is copyrighted as an object, whether or not to allow the image of the wing to be combined and displayed as an additional image around the bronze statue in the image displayed on the AR device 3. Can be set by the intellectual property right holder.

Further, for example, when the object is an animal such as a vehicle, a bag, or clothing, and the object is protected by a copyright, a trademark right, a design right, a patent right, etc., it is displayed on the AR device 3. In the image, another image such as an advertisement can be combined and displayed as an additional image to this animal. In this case, the intellectual property right holder can set whether or not to allow the addition of the additional image to the object displayed on the AR device 3 by the terminal.

In addition, when the intellectual property right holder makes such a setting, the terminal can be set so as not to allow the additional image to be overwritten on the image such as the brand name or logo displayed on the object. It has become like. In addition, the addition of an additional image that damages the brand of the object, the addition of an additional image that misleads the quality, and the addition of an additional image that can be replaced with the brand name or logo of a competitor of the manufacturer of the object. The intellectual property right holder can make a setting that the addition is not allowed by the terminal.

Next, a case where the information processing system 1 of another embodiment is applied to a virtual reality (VR) technique will be described. Also in this case, when the intellectual property information is received by the reception means 221 transmitted to the server device 2 (YES in step st11), the received intellectual property information is stored in the storage unit 23 (step st12).

After that, the setter sets the display of the additional image by the computer 5. The setting information input to the computer 5 by the setter and the identification information of the setter are transmitted from the computer 5 to the server device 2 and accepted by the receiving means 221 (YES in step st13). The setting information and the identification information of the setter received by the receiving means 221 are stored in the storage unit 23 (step st14). Since the setting of the display of the additional image is described above, the details will be omitted.

Then, when the user moves in the virtual space displayed on the display unit 48 of the computer 4 by the operation unit 45 or the like, the display instruction is transmitted from the computer 4 to the server device 2. At this time, information about the current position in the virtual space is also transmitted to the server device 2. Further, the attribute information about the user stored in the storage unit 46 is also transmitted from the computer 4 to the server device 2.

When the receiving means 221 of the server device 2 receives the display instruction (YES in step st15), the setting means 222 sets the display mode of the additional image on the user's computer 4 based on the received setting information (step st16). .. Since the setting of the display mode of the additional image is described above, the details will be omitted.

Then, it is appropriate for the determination means 223 to display the image obtained by synthesizing the additional image in the set display mode on the user's computer 4 based on the intellectual property information stored in the storage unit 23. (Step st17). When it is determined that it is appropriate to display the combined image of the additional image on the user's computer 4 (YES in step st17), the instruction means 224 uses the display mode set by the setting means 222 to display the additional image. Is displayed on the computer 5 (step st18).

On the other hand, when it is determined that it is not appropriate to display the image obtained by adding the added image to the object on the user's computer 4 (NO in step st17), the instruction means 224 is set by the setting means 222. The computer 4 does not display the combined image of the additional image in the display mode (step st19). In this case, the display unit 48 of the computer 4 displays an image of the virtual space in which the additional image is not combined.

Summarizing the above, according to another embodiment, the server device 2 includes a reception means 221, a setting means 222, a determination means 223, and a computer that functions as an instruction means 224 by executing a program. It is a thing. The reception means 221 receives intellectual property information regarding the intellectual property of the object. The setting means 222 sets the display mode of the additional image on the user's AR device 3 or the computer 4 (second terminal).

Based on the received intellectual property information, the determination means 223 may display the image of the additional image on the object in the display mode set by the setting means 222 on the user's AR device 3 or computer 4. Determine if it is appropriate. When the instruction means 224 determines that it is appropriate for the determination means 223 to display the image of the additional image in a state of being added to the object, the instruction means 224 synthesizes the additional image in the display mode set by the setting means 222. The image is displayed on the user's AR device 3 or computer 4.

According to another embodiment, only the additional image appropriate for the intellectual property is determined by determining whether it is appropriate to display the additional image attached to the object based on the intellectual property information. Can be displayed on the user's AR device 3 or computer 4. This makes it possible to protect the intellectual property rights of the intellectual property right holder.

Further, in each of the above embodiments, the case where the server device 2 functions the program is illustrated, but the present invention is not limited to this. Some of the means of the program, that is, the reception means 221, the setting means 222, the judgment means 223, the instruction means 224, etc., are the user's AR device 3, the computer 4, the setter's computer 5, and the intellectual property. It may be provided in another terminal such as the terminal of the right holder. In this case, it is possible to appropriately change which means is to be operated by which device.

Further, the setting information, the intellectual property information, and the like received by the receiving means 221 are not stored in the storage unit 23 of the server device 2, but are stored in a cloud or an external storage medium provided separately from the server device 2. It may be memorized.

Further, even if the setting means 222 of the server device 2 creates a map or a list that can confirm where and what object is in the real space or virtual space and what kind of additional image can be synthesized. good. Then, the map or list created by the setting means 222 may be displayed on the display unit 38 of the user's AR device 3 or the display unit 48 of the computer 4.

Further, the setting means 222 of the server device 2 has, for each user, the number of times and display time of the object displayed on the display unit 38 of the AR device 3 and the display unit 48 of the computer 4, the number of times and display time of the additional image, and the target. The number of times and the display time of the additional image added to the object may be totaled. Then, the setting means 222 may give a privilege to the user when the number of times of the displayed object or the additional image and the display time reach a preset predetermined value. When the additional image is, for example, an advertisement image, the privilege may be a tangible object such as a product related to the advertisement, or an intangible object such as a coupon.

It is also possible to hold an event using the total of the number of objects and additional images displayed on the display unit 38 of the AR device 3 and the display unit 48 of the computer 4. As an event, specifically, a game in which a user discovers a predetermined number or more or all of additional images in a predetermined area on an AR device 3 or a computer 4 within a predetermined time, or an AR device that solves a mystery as an additional image. You may execute a mystery solving game that is displayed on the display unit 38 or the like of 3.

Further, the information regarding the number of times and the display time of the object and the additional image displayed on the display unit 38 of the AR device 3 and the display unit 48 of the computer 4 for each user, which are aggregated by the setting means 222, is transmitted from the server device 2 to the setter. It may be transmitted to the computer 5 of the above, the computer of the advertiser, or the like. As a result, the setter, the advertiser, and the like can recognize the advertising effect on the user.

Further, in the above description, images of a plurality of advertisements are displayed on the first building 102 or an advertising signboard at a predetermined time or date and time (for example, every few seconds, in the morning, afternoon, weekdays, holidays, holidays, anniversaries, or months). It has been explained that the computer 5 can be set to switch and display in the same area of 104, but the present invention is not limited to this. For example, when a certain additional image (for example, an advertisement image) is displayed on the user's AR device 3 or computer 4 a predetermined number of times or for a predetermined time, the image of this advertisement is switched to the same area of the first building 102 or the advertisement signboard 104. The computer 5 may be used to make settings for displaying.

Further, the receiving means 221 of the server device 2 may accept the information about the user's My Number as the user's identification information. Then, when the information about the My Number is associated with the information that the user has a specific addiction, the setting means 222 of the article or event (for example, a game, alcohol, gambling, etc.) that is the target of the addiction. The advertisement image may not be displayed on the user's AR device 3 or computer 4 as an additional image, or the additional image related to the healing of the addiction may be displayed on the user's AR device 3 or computer 4. In addition, the setter's right to the object may be able to be confirmed based on the information regarding the My Number.

Further, in the above description, an example in which the receiving means 221 receives the setting information regarding the setting of the display of the additional image from the computer 5 (first terminal) of the setter has been described, but the present invention is not limited to this. In the present embodiment, the setting of the display of the additional image may be arbitrarily performed in the server device 2.

Further, for buildings in real space or virtual space that are not related to this system, the setting person can control the display of advertisements without setting the display of additional images by the computer 5. May be good. That is, the fact that the display of the additional image is not set by the setter can also be treated as a part of the setting by the setting means 222.

An example will be described in which an object is not an animal or the like existing in the real space, but a predetermined area in the real space is the object. For example, the owner (setter) of the land sets the display of an additional image for the object, regardless of whether or not there is a building on the land, with the space above the land as the object. May be. Further, the predetermined area (object) in the real space may be the periphery of the predetermined building or the path from the predetermined point (starting point) to the building (destination). For example, an area (object) is set in a public open space of a predetermined building, and additional images showing information such as tenants moving into the area in that area, or on the way from the starting point to the destination, such as ←, ↑, etc. An additional image showing directions or an additional image such as 000 meters to the destination may be displayed.

Further, when a plurality of additional images are set in one area (object), the arrangement is adjusted so that these additional images do not overlap when viewed from the user. In addition, the arrangement is adjusted so that a part of a plurality of additional images is emphasized and displayed according to the user's attributes and settings. Further, the additional image may be highlighted according to a predetermined value such as an amount of money associated with the display of the additional image paid by the setter or the like, or the newer display setting may be highlighted.

Here, the highlighting is to make the display mode such as the size, position, color, luminous intensity, and blinking of the additional image different from that of other additional images. The display mode also includes mentions to a particular user. For example, in the setting of an additional image, a specific user can be specified as a mention partner, and the mentioned additional image is displayed on the display unit of the terminal corresponding to the user. Further, the display mode of these additional images may be controlled by the server device or may be controlled by the user's terminal.

Further, the present invention can be applied not only to AR and VR but also to XR such as MR.

2 Server device (computer)
221 Reception means 222 Setting means 223 Judgment means 224 Instruction means

Claims (20)

A server device provided with a computer that functions as a reception means, a setting means, and an instruction means by executing a program.
The reception means receives setting information regarding the display setting of the additional image from the first terminal, and receives the setting information.
The setting means sets the display mode of the additional image on the user's second terminal based on the setting information received by the reception means.
The instruction means is a server device that causes the second terminal to display an image in which the additional image is selectively combined in the display mode set by the setting means. When the instruction means displays an image of the surrounding environment detected by the detection unit of the second terminal on the second terminal, the additional image is displayed on the peripheral environment in the display mode set by the setting means. The server device according to claim 1, wherein the server device is selectively combined with an image and displayed on the second terminal. The setting information includes information on an object to which the additional image should be added.
When the object is included in the image of the surrounding environment detected by the detection unit of the second terminal, the setting means adds the additional image to the object in the display mode set by the setting means. The server device according to claim 2, which is displayed on the second terminal in the above-mentioned state. The receiving means receives information about the surrounding environment of the second terminal detected by the detection unit of the second terminal as peripheral environment information from the second terminal.
The server device according to claim 2 or 3, wherein the setting means sets a display mode of the additional image in the second terminal based on the setting information received by the receiving means and the surrounding environment information. The receiving means receives information about the current position of the second terminal as the current position information from the second terminal.
The setting means according to any one of claims 2 to 4, wherein the setting means sets the display mode of the additional image in the second terminal based on the setting information received by the receiving means and the current position information. Server device. The reception means receives information about the virtual space and receives information.
When the instruction means displays the image of the virtual space received by the reception means on the second terminal, the instruction means selectively displays the additional image as the image of the virtual space in the display mode set by the setting means. The server device according to claim 1, which is synthesized in the above and displayed on the second terminal. The setting information includes information regarding setting whether or not to display the additional image on the second terminal.
When the setting information is information to display the additional image on the second terminal, the setting means sets the display mode to display the additional image on the second terminal. The server device according to any one of 1 to 6. The receiving means receives attribute information about the user who owns the second terminal, and receives the attribute information.
The server device according to any one of claims 1 to 7, wherein the setting means sets the display mode of the additional image in the second terminal with reference to the attribute information received by the receiving means. .. The server device according to any one of claims 1 to 8, wherein the setting means sets a display mode of the additional image in the second terminal with reference to date and time information regarding the current date and time. A server device equipped with a computer that functions as a reception means, a setting means, a judgment means, and an instruction means by executing a program.
The reception means receives intellectual property information regarding the intellectual property of the object and receives it.
The setting means sets a display mode of an additional image added to the object on the user's second terminal.
It is appropriate for the determination means to display the additional image on the second terminal in a state of being attached to the object in the display mode set by the setting means based on the received intellectual property information. Judge if there is,
When it is determined by the determination means that it is appropriate for the instruction means to display the additional image on the second terminal in a state of being attached to the object, the display mode set by the setting means. A server device for displaying the additional image on the second terminal in a state of being attached to the object. When the instruction means displays an image of the surrounding environment detected by the detection unit of the second terminal on the second terminal, the additional image is displayed on the peripheral environment in the display mode set by the setting means. The server device according to claim 10, which is combined with an image and displayed on the second terminal. The receiving means receives information about the surrounding environment of the second terminal detected by the detection unit of the second terminal as peripheral environment information from the second terminal.
The determination means adds the additional image to the object in the display mode set by the setting means based on the intellectual property information about the object in the surrounding environment information received by the reception means. The server device according to claim 11, wherein it is determined whether or not it is appropriate to display the information on the second terminal in the above-mentioned state. The determination means also refers to the degree to which the object is included in the surrounding environment information received by the reception means, and the additional image is added to the object in the display mode set by the setting means. 12. The server device according to claim 12, wherein it is determined whether or not it is appropriate to display the information on the second terminal. The receiving means receives information about the current position of the second terminal as the current position information from the second terminal.
The determination means is the first in a state where the additional image is added to the object in the display mode set by the setting means based on the intellectual property information and the current position information received by the reception means. 2. The server device according to any one of claims 11 to 13, which determines whether or not it is appropriate to display the information on a terminal. The reception means receives information about the virtual space and receives information.
When the instruction means displays the image of the virtual space received by the reception means on the second terminal, the instruction means synthesizes the additional image with the image of the virtual space in the display mode set by the setting means. 10. The server device according to claim 10, which is displayed on the second terminal. The receiving means receives information about the display mode of the second terminal of the user as display mode information, and receives the information.
The determination means is the first in a state where the additional image is added to the object in the display mode set by the setting means based on the intellectual property information and the display mode information received by the reception means. 2. The server device according to any one of claims 10 to 15, which determines whether or not it is appropriate to display the information on a terminal. An information processing system including a reception means, a setting means, and an instruction means.
The reception means receives setting information regarding the display setting of the additional image from the first terminal, and receives the setting information.
The setting means sets the display mode of the additional image on the user's second terminal based on the setting information received by the reception means.
The instruction means is an information processing system that causes the second terminal to display an image in which the additional image is selectively combined in the display mode set by the setting means. An information processing system including a reception means, a setting means, a judgment means, and an instruction means.
The reception means receives intellectual property information regarding the intellectual property of the object and receives it.
The setting means sets a display mode of an additional image added to the object on the user's second terminal.
It is appropriate for the determination means to display the additional image on the second terminal in a state of being attached to the object in the display mode set by the setting means based on the received intellectual property information. Judge if there is,
When it is determined by the determination means that it is appropriate for the instruction means to display the additional image on the second terminal in a state of being attached to the object, the display mode set by the setting means. An information processing system for displaying the additional image on the second terminal in a state of being attached to the object. A program that causes a computer to function as a reception means, a setting means, and an instruction means.
The reception means receives setting information regarding the display setting of the additional image from the first terminal, and receives the setting information.
The setting means sets the display mode of the additional image on the user's second terminal based on the setting information received by the reception means.
The instruction means is a program that causes the second terminal to display an image in which the additional image is selectively combined in the display mode set by the setting means. A program that causes a computer to function as a reception means, a setting means, a judgment means, and an instruction means.
The reception means receives intellectual property information regarding the intellectual property of the object and receives it.
The setting means sets a display mode of an additional image added to the object on the user's second terminal.
It is appropriate for the determination means to display the additional image on the second terminal in a state of being attached to the object in the display mode set by the setting means based on the received intellectual property information. Judge if there is,
When it is determined by the determination means that it is appropriate for the instruction means to display the additional image on the second terminal in a state of being attached to the object, the display mode set by the setting means. A program for displaying the additional image on the second terminal in a state of being attached to the object.

Images