WO2022123922A1

WO2022123922A1 - Information processing system

Info

Publication number: WO2022123922A1
Application number: PCT/JP2021/038963
Authority: WO
Inventors: 泰士山本; 宏樹林; 真治木村; 幹生岩村; 江利子大関; 修後藤
Original assignee: 株式会社Ｎｔｔドコモ
Priority date: 2020-12-11
Filing date: 2021-10-21
Publication date: 2022-06-16
Also published as: JPWO2022123922A1

Abstract

This information processing system comprises: a storage unit which stores content data; an acquisition unit which acquires surrounding space information; an area determination unit which determines, on the basis of the surrounding space information, a public area in which AR content is depicted and a private area in which private content is depicted; a first specification unit which specifies the AR content associated with an object in a real space included in the public area by referring to the content data; a second specification unit which specifies the private content depicted in the private area; an image generation unit which generates a depiction image in which the AR content is depicted in the public area and the private content specified by the second specification unit is depicted in the private area; and an output unit which outputs the depiction image.

Description

Information processing system

One aspect of the present invention relates to an information processing system.

Patent Document 1 describes that private content owned by the communication terminal and public content not owned by the communication terminal are simultaneously displayed on the same screen on the communication terminal of the user.

Japanese Unexamined Patent Publication No. 2007-294068

Here, in a system that simultaneously displays private content and public content on the same screen as described above, it is preferable for the user to mainly display either content according to the surrounding situation of the user, for example. In some cases, it is easy to process the content, but usually, the display area of each content is fixed, and there may be a situation where it is not easy for the user to process the content.

One aspect of the present invention has been made in view of the above circumstances, and an object thereof is to improve the processability of contents.

The information processing system according to one aspect of the present invention acquires a storage unit that stores content information in which an object in real space and AR content are associated with each other, and peripheral space information including at least an image of real space around the user. It includes an acquisition unit and an area determination unit that determines a display area of content including AR content based on peripheral spatial information.

In the information processing system according to one aspect of the present invention, the display area of the content is determined based on the peripheral space information including at least the image of the real space around the user. According to such an information processing system, the display area changes according to the change of the peripheral spatial information (that is, the spatial information around the user changes). As a result, each area can be adjusted according to the surrounding spatial information, and the ease of content processing by the user can be improved. As described above, according to the information processing system according to one aspect of the present invention, the ease of processing the content can be improved.

According to one aspect of the present invention, the ease of processing the content can be improved.

It is a figure explaining the outline of the information processing system which concerns on this embodiment. It is a block diagram which shows the functional structure of the information processing system which concerns on this embodiment. It is a figure explaining an example of the area determination processing by an image generation server. It is a figure explaining an example of the area determination processing by an image generation server. It is a figure explaining an example of the area determination processing by an image generation server. It is a flowchart which shows the process executed by an image generation server. It is a figure which shows the hardware configuration of the communication terminal, the positioning server, and the image generation server server included in the information processing system which concerns on this embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same reference numerals are used for the same or equivalent elements, and duplicate description is omitted.

FIG. 1 is a diagram illustrating an outline of the information processing system 1 according to the present embodiment. In the information processing system 1, the image generated by the image generation server 50 is displayed on the communication terminal 10. That is, the user visually recognizes the image generated by the image generation server 50 on the screen of the communication terminal 10. 1 (a) to 1 (c) show an example of an image generated by the image generation server 50 and displayed on the screen of the communication terminal 10. The image generation server 50 generates an image in which AR content or the like is superimposed on the image captured by the communication terminal 10.

The information processing system 1 has a public area in which AR content is depicted (superimposed) in an image in real space based on peripheral space information, and a private area in which private content related to a user carrying a communication terminal 10 is depicted (superimposed). Is determined, and an image (depicted image) in which the content is depicted in each area is generated. The public area is an area for depicting AR content associated with an object in real space. AR content in the public domain is, for example, content related to signs, advertisements, art works, and various entertainment works. The private area is an area that describes the user's private content (private content). The private content is, for example, content such as a search screen, a game, an e-mail, an SNS, a map, etc. using the Internet.

More specifically, in the information processing system 1, the public area, the private area, and the non-depiction area are determined based on the peripheral space information. That is, in the information processing system 1, the sizes of the public area, the private area, and the non-depiction area change according to the peripheral space information. The non-depiction area is an area that secures the user's field of view without depicting the content.

As shown in FIG. 1A, at least the public area A1 and the private area A2 are set in the image (screen of the communication terminal 10) displayed on the communication terminal 10. In the example shown in FIG. 1B, a plurality of AR contents F associated with an object in the real space are depicted in the public area A1. Specifically, in FIG. 1B, in the public area A1, the AR content F as a sign associated with the building related to the station, the AR content F as the sign associated with the building related to the supermarket, and the AR content F. AR content F as a sign associated with a building related to a cafe and AR content F as an advertisement associated with a building related to the sale of clothing are depicted. Further, in the example shown in FIG. 1B, the content M related to the map and the content S related to the SNS are depicted in the private area A2. In this case, in consideration of the surrounding space information, the area where there are many buildings depicting AR content (upper part in FIG. 1B) is designated as the public area A1, and the area where there are few buildings depicting AR content (FIG. 1 (c)). In), the lower part) is the private area A2. Then, in the example shown in FIG. 1B, it is specified that the user is walking by considering, for example, information on the moving speed of the user, and the areas of the public area A1 and the private area A2 are relatively large. It is set and the non-depicted area is not set. This is because a non-depicted area for ensuring the user's field of view is unnecessary when the user is walking.

In the example shown in FIG. 1 (c), in the public area A1, the AR content F as a sign associated with the building related to the restaurant and the AR content F as the sign associated with the building related to the cafe are Each is depicted. Further, in the example shown in FIG. 1 (c), the content M related to the map is depicted in the private area A2. Then, in the example shown in FIG. 1 (c), it is specified that the user is in a car by considering, for example, information on the moving speed of the user, and the areas of the public area A1 and the private area A2 are compared. The non-depiction area A3 is set to be small. In this case, in consideration of the surrounding space information, the area where there are many buildings depicting AR content (upper part in FIG. 1B) is designated as the public area A1, and the area where there are few buildings depicting AR content (FIG. 1 (c)). In), the lower part) is set as the private area A2, and the area where the most visibility is desired during driving (the central part in FIG. 1C) is set as the non-depiction area A3.

According to such an information processing system 1, the public area A1, the private area A2, and the non-depiction area A3 change according to the change of the peripheral spatial information and the like (that is, the spatial information around the user changes). It will be. As a result, for example, when it is a tourist spot or the like and the AR content corresponding to the surrounding buildings or the like is to be displayed mainly, the public area A1 is enlarged and the user is more private than the AR contents related to the surrounding buildings or the like. If you want to display the content in the center, increase the private area A2, and if you should give priority to ensuring the user's view, increase the non-depiction area A3, etc., depending on the surrounding spatial information. While ensuring the ease of content processing by the user, it is possible to prevent the user's visibility from becoming poor.

Next, with reference to FIG. 2, the functional configuration of the information processing system 1 according to the present embodiment will be described. FIG. 2 is a block diagram showing a functional configuration of the information processing system 1 according to the present embodiment.

As shown in FIG. 2, the information processing system 1 includes a communication terminal 10, a positioning server 30, and an image generation server 50. Although only one communication terminal 10, a positioning server 30, and an image generation server 50 are shown in FIG. 2, the information processing system 1 may be configured to include a plurality of each.

The communication terminal 10 is a communicable terminal having a display, and is, for example, a smartphone, a tablet-type terminal, a PC, a glasses-type wearable terminal, or the like. The communication terminal 10 has a camera and is configured to be capable of capturing an image of the surrounding area. For example, when an application related to content depiction is started, the communication terminal 10 starts imaging by a camera to be mounted. Further, the communication terminal 10 has a sensor such as an acceleration sensor, and is configured to be able to derive a moving speed (moving speed of a user carrying the communication terminal 10). Further, the communication terminal 10 is configured to be able to acquire the user's gaze point (the point at which the user gazes through the display of the eyeglass-type wearable terminal). The communication terminal 10 transmits an image (captured image) captured by the camera to the positioning server 30. The communication terminal 10 acquires a positioning result based on the captured image from the positioning server 30. The communication terminal 10 transmits an captured image, a user's position (positioning result), a user's gaze point, and a user's movement speed to the image generation server 50.

The positioning server 30 has a storage unit 31 and a positioning unit 32. The storage unit 31 stores the map data 300. In the map data 300, the feature amount (for example, the luminance direction vector) of the feature points included in the captured image acquired in advance is associated with the global position information which is the absolute position information associated with the feature points. There is. The map data 300 is, for example, a 3D point cloud. The map data 300 is preliminarily imaged by a stereo camera (not shown) or the like capable of simultaneously capturing an object from a plurality of different directions, and is generated based on a large number of captured images. The feature point is a point that is prominently detected in the image, and is, for example, a point where the brightness (intensity) is large (or small) as compared with other regions. The global position information of the feature point is the global position information set in association with the feature point, and is the global position information in the real world about the region indicated by the feature point in the image. It should be noted that the association of the global position information with each feature point can be performed by a conventionally known method. The storage unit 31 stores three-dimensional global position information as the global position information of the feature points of the map data 300. The storage unit 31 stores, for example, the latitude, longitude, and height of the feature points as three-dimensional global position information of the feature points.

The positioning unit 32 is based on the captured image captured by the communication terminal 10 and the map data 300 stored in the storage unit 31, and the global position information (three-dimensional) of the communication terminal 10 at the time of imaging by the communication terminal 10. Position information) is estimated. Specifically, the positioning unit 32 matches the feature points of the map data 300 with the feature points of the captured image captured by the communication terminal 10, and obtains a region of the map data 300 corresponding to the captured captured image. Identify. Then, the positioning unit 32 estimates the imaging position of the captured image (that is, the global position information of the communication terminal 10 at the time of imaging) based on the global position information associated with the feature points of the map data 300 related to the specified area. .. The positioning unit 32 transmits the positioning result to the communication terminal 10.

The positioning result includes information on the direction estimated from the captured image (direction in the three-dimensional coordinates of roll, pitch, and yaw) in addition to the global position information. Further, the positioning unit 32 may acquire global position information based on the captured images captured by the communication terminal 10 at a fixed cycle, or may be captured by the communication terminal 10 at the timing of receiving an instruction from the user. Global position information may be acquired based on the captured image. The positioning result may be acquired without relying on the positioning server 30. That is, the communication terminal 10 may acquire the user's position (positioning result) by a conventionally known positioning method regardless of the captured image.

The image generation server 50 includes an acquisition unit 51, an area determination unit 52, a first specific unit 53, a second specific unit 54, an image generation unit 55, an output unit 56, and a storage unit 57. ing.

The storage unit 57 stores content data 500 (content information) in which an object in real space and AR content are associated with each other. That is, the storage unit 57 stores the information of the AR content to be drawn in the public area A1 in association with the object in the real space. The storage unit 57 may store, for example, the content (display content) of the AR content, the size of the AR content, and the like in association with the shape and position information of an object (building, etc.) in the real space.

The acquisition unit 51 acquires peripheral space information including at least an image of the real space around the user. The acquisition unit 51 acquires the captured image (image of the real space around the user) transmitted from the communication terminal 10 and the position (positioning result) of the user corresponding to the captured image as the peripheral space information. The acquisition unit 51 may acquire the metadata of the map showing the shape of the object as the peripheral space information. Further, the acquisition unit 51 may further acquire information indicating the state of the user. The acquisition unit 51 acquires the user's viewpoint (gaze point) and movement speed transmitted from the communication terminal 10 as information indicating the state of the user. The acquisition unit 51 outputs each acquired information to the area determination unit 52.

The area determination unit 52 describes a public area for describing the AR content of the content data 500 in a real space image (captured image) based on the peripheral space information acquired by the acquisition unit 51, and a private area for describing the private content related to the user. A region and a non-depicted region, which is an region for ensuring the user's view, are determined. Specifically, the area determination unit 52 may determine a public area, a private area, and a non-depiction area based on the type of the surrounding building specified from the image (captured image) included in the surrounding space information. good. For example, the area determination unit 52 may specify a building area such as a traffic light or a traffic sign that the user needs to clearly see from the captured image, and may use the area as a non-depiction area. Further, the area determination unit 52 sets the area of the building type in which the AR content of the content data 500 is likely to be displayed as a public area, and the area of the building type in which the AR content of the content data 500 is unlikely to be displayed is private. It may be an area.

Further, the area determination unit 52 may determine a public area, a private area, and a non-depiction area based on the shape of the surrounding building specified from the surrounding space information. For example, the area determination unit 52 is a place where many people come and go from the captured image, such as the shape of the entrance of the building (a place where there is a high possibility of collision with the user and the user needs to clearly see it). A region showing the shape of the above may be specified, and the region may be set as a non-depiction region. Further, the area determination unit 52 sets the area of the building shape in which the AR content of the content data 500 is likely to be displayed as a public area, and the area of the building shape in which the AR content of the content data 500 is unlikely to be displayed is private. It may be an area.

Further, the area determination unit 52 may determine the public area, the private area, and the non-depiction area in consideration of the information indicating the state of the user acquired by the acquisition unit 51. Specifically, the area determination unit 52 may determine each area based on the user's gaze point. For example, the area determination unit 52 may determine that the user's gaze point and the area around the gaze point are the areas that the user wants to see, and may set them as non-depiction areas. Alternatively, the area determination unit 52 may determine that the user's gaze point and the area around the gaze point are areas in which the AR content related to each building or the like can be effectively shown to the user, and may be set as a public area.

Further, the area determination unit 52 may determine each area based on the movement mode of the user specified from the movement speed of the user. For example, the area determination unit 52 determines that it is necessary to sufficiently secure the user's field of view when the user's movement mode is specified to be a car (the user is driving) from the user's movement speed. However, the non-depiction area may be enlarged. Further, when the movement mode of the user is specified to be walking (the user is walking) from the movement speed of the user, the area determination unit 52 may make the public area and the private area relatively large. .. Further, the area determination unit 52 determines from the movement speed of the user that it is not necessary to provide a non-depiction area from the viewpoint of safety or the like when the user's movement mode is stopped, which indicates that the user is not moving. , The non-depicted area may be determined without.

3 to 5 are diagrams illustrating an example of area determination processing by the area determination unit 52 of the image generation server 50.

In the example shown in FIG. 3, the area determination unit 52 identifies that the user is driving from the moving speed of the user. In this case, the area determination unit 52 sets the non-depiction area A3 to be relatively large (for example, larger than the public area A1 and the private area A2). Further, the area determination unit 52 determines the size and position of the public area A1, the private area A2, and the non-depiction area A3 based on the type and shape of the surrounding buildings specified from the surrounding space information. Specifically, as shown in FIG. 3, the area determination unit 52 is a area such as a sign or a signal necessary for driving, an area near a road, a pedestrian jumping out, or a car from a building (parking lot, etc.). The area of the sidewalk where is expected to appear is determined as the non-depiction area. Further, the area determination unit 52 sets the area of the building shape in which the AR content of the content data 500 is likely to be displayed as the public area A1, and the area (lower area) in which the AR content of the content data 500 is not likely to be displayed. ) Is the private area A2.

In the example shown in FIG. 4, the area determination unit 52 specifies that the user is walking based on the moving speed of the user. In this case, the area determination unit 52 may set the non-depiction area A3 smaller than the state shown in FIG. Further, the area determination unit 52 determines the entrance portion of the building, the area near the sidewalk, and the area where the vehicle may enter the sidewalk from the roadway, based on the type and shape of the surrounding building specified from the surrounding space information. , The non-depiction area A3 is determined. Further, the area determination unit 52 sets the area of the building shape in which the AR content of the content data 500 is likely to be displayed as the public area A1, and the area (lower area) in which the AR content of the content data 500 is not likely to be displayed. ) Is the private area A2.

In the example shown in FIG. 5, the area determination unit 52 specifies that the user is stopped based on the moving speed of the user. In this case, the area determination unit 52 may determine the non-depiction area A3 without it. Further, since the area determination unit 52 is stopped, it is determined that the user is watching the communication terminal 10 (operating the private content) because the user is in a meeting or the like, and the private area A2 is relatively large. It may be (larger than the public area A1). Also in this case, for example, the area determination unit 52 may set the area of the object having many associated AR contents (public contents) as the public area A1.

The public area and private area are set as follows, for example, according to the state of the user. If the user seems to be waiting, the public area is normal, the private area is large, and if the user seems to be waiting for a signal, the public area is large, the private area is normal, and the user is walking. If you think it is, make the public area larger, make the private area smaller (or none), and if you think the user is on a train or bus, make the public area smaller (or none), and make the private area smaller. Larger, if the user seems to be driving, the public area may be set to be smaller (or none) and the private area may be set to be smaller (or none).

Returning to FIG. 2, the first specifying unit 53 identifies the AR content associated with the object in the real space included in the public area by referring to the content data 500. The first specific unit 53 narrows down the AR content in the content data 500 according to, for example, the position of the user and the shape of the object.

The second specifying unit 54 specifies the private content depicted in the private area based on the instruction from the user. The second specifying unit 54 specifies, for example, an application designated by the user on the display of the communication terminal 10 as private content.

The image generation unit 55 generates a depiction image in which the AR content specified by the first specific unit 53 is depicted in the public area and the private content specified by the second specific unit 54 is depicted in the private area. The image generation unit 55 may change the depiction mode of the AR content in consideration of the distance to the AR content, the content size, the movement speed of the user, and the like. In this case, the image generation unit 55 may determine the depiction mode of the AR content by, for example, the following formula. In the following equation, w1, w2, and w3 are weights that change according to the screen size of the communication terminal 10, the visual acuity of the user, and the like.
Depiction mode = (w1 x distance) x (w2 x content size) x (w3 x speed)

The output unit 56 outputs a depiction image generated by the image generation unit 55. The output unit 56 displays a depiction image on the display of the communication terminal 10.

Next, the process executed by the image generation server 50 will be described with reference to FIG. FIG. 6 is a flowchart showing a process executed by the image generation server 50.

As shown in FIG. 6, the image generation server 50 first acquires peripheral space information and information indicating a user's state (step S1). Subsequently, the image generation server 50 determines the sizes and positions of the public area, the private area, and the non-depicted area based on the acquired information (step S2).

Subsequently, the image generation server 50 specifies the AR content to be displayed in the public area (step S3). Further, the image generation server 50 specifies the private content to be displayed in the private area (step S4).

Finally, the image generation server 50 generates a depiction image in which the AR content is depicted in the public area and the private content is depicted in the private area (step S5), and outputs the depiction image (step S6).

Next, the operation and effect of the information processing system 1 according to the present embodiment will be described.

The information processing system 1 according to the present embodiment acquires the storage unit 57 that stores the content data 500 in which the object in the real space and the AR content are associated with each other, and the peripheral space information including at least the image of the real space around the user. The acquisition unit 51 is provided, and the area determination unit 52 that determines the display area of the content including the AR content based on the peripheral space information is provided.

In such an information processing system 1, the display area of the content is determined based on the peripheral space information including at least the image of the real space around the user. According to such an information processing system 1, the display area changes according to the change of the peripheral space information (that is, the change of the spatial information around the user). As a result, each area can be adjusted according to the surrounding spatial information, and the ease of content processing by the user can be improved. As described above, according to the information processing system 1 according to the present embodiment, the ease of processing the contents can be improved. Further, by improving the ease of processing, unnecessary processing by the user is reduced, so that the processing load as a system can be reduced.

The area determination unit 52 determines a public area for describing the AR content of the content data 500 in the image in the real space and a private area for describing the private content related to the user, and the information processing system 1 refers to the content data 500. By doing so, the first specifying unit 53 that specifies the AR content associated with the object in the real space included in the public area, and the second specifying the private content that is drawn in the private area based on the instruction from the user. Image generation that generates a depiction image in which the specific unit 54 and the AR content specified by the first specific unit 53 are depicted in the public area, and the private content specified by the second specific unit 54 is depicted in the private area. A unit 55 and an output unit 56 that outputs a depiction image are provided.

In the information processing system 1 according to the present embodiment, the public area in which the AR content is depicted (superimposed) on the image in the real space based on the peripheral space information including at least the image in the real space around the user, and the private area related to the user. A private area in which the content is described is determined, and a depiction image in which the content corresponding to each area is described is generated and output in each area. According to such an information processing system 1, the public area and the private area change according to the change of the peripheral space information (that is, the change of the spatial information around the user). As a result, for example, when the AR content corresponding to the surrounding buildings, etc. is to be displayed mainly in a tourist spot, the public area is enlarged, and the user's private content is larger than the AR content related to the surrounding buildings, etc. If you want to display the content in the center, you can adjust each area according to the surrounding spatial information, such as enlarging the private area, and improve the ease of content processing by the user. As described above, according to the information processing system according to the present embodiment, the ease of processing the content can be improved.

The area determination unit 52 may determine an area that secures the user's field of view as a non-depiction area that does not depict the content, based on the peripheral space information. According to such a configuration, since the content is not drawn in the area where the user's field of view should be secured, it is possible to prevent the user's field of view from becoming poor due to the drawing of the content.

The area determination unit 52 may determine a public area, a private area, and a non-depiction area based on the type of the surrounding building specified from the surrounding space information. According to such a configuration, each area can be determined more appropriately according to the type of building. That is, for example, a building area that needs to be clearly seen by the user, such as a traffic light or a traffic sign, can be made into a non-depiction area, and the ease of processing the content and the securing of the user's field of view are more appropriately implemented. be able to.

The area determination unit 52 may determine a public area, a private area, and a non-depiction area based on the shape of the surrounding building specified from the surrounding space information. According to such a configuration, each area can be determined more appropriately according to the shape of the building. That is, it is possible to make a non-depiction area for a place where many people come and go (a place where there is a high possibility of collision with the user and the user needs to clearly see) such as an entrance of a building. , Content processing ease and user visibility can be secured more appropriately.

The acquisition unit 51 may further acquire information indicating the user's state, and the area determination unit 52 may determine the public area, the private area, and the non-depiction area in consideration of the user's state. According to such a configuration, not only the peripheral space information but also the state of the user is taken into consideration, so that it is possible to determine a more appropriate area from the viewpoint of the ease of content processing of the user.

The acquisition unit 51 may acquire information indicating the state of the user including at least the user's viewpoint, and the area determination unit 52 may determine the public area, the private area, and the non-depiction area based on the user's viewpoint. .. According to such a configuration, it is possible to determine each area so as to further improve the user's content processing ease in consideration of where the user is visually recognizing.

The acquisition unit 51 acquires information indicating the state of the user including the movement speed of the user, and the area determination unit 52 acquires the public area, the private area, and the area based on the movement mode of the user specified from the movement speed of the user. The non-depicted area may be determined. According to such a configuration, each area can be appropriately determined according to the movement mode of the user, for example, the non-depiction area is increased during driving and the public area is increased during walking.

The area determination unit 52 may determine the non-depiction area without the non-depiction area when the movement mode of the user is stopped indicating that the user is not moving. Since it is not necessary to provide a non-depicted area for safety reasons during the stop, each area can be determined more appropriately by controlling in this way.

Next, the hardware configurations of the communication terminal 10, the positioning server 30, and the image generation server 50 included in the information processing system 1 will be described with reference to FIG. 7. The communication terminal 10, the positioning server 30, and the image generation server 50 physically include a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like. It may be configured as a device.

In the following description, the word "device" can be read as a circuit, device, unit, etc., and the hardware configurations of the communication terminal 10, the positioning server 30, and the image generation server 50 are shown in the figure. It may be configured to include one or more devices, or it may be configured not to include some devices.

For each function of the communication terminal 10, the positioning server 30, and the image generation server 50, the processor 1001 performs calculations and communicates by loading predetermined software (programs) on hardware such as the processor 1001 and the memory 1002. It is realized by controlling communication by the device 1004 and reading and / or writing of data in the memory 1002 and the storage 1003.

The processor 1001 operates, for example, an operating system to control the entire computer. The processor 1001 may be configured by a central processing unit (CPU: Central Processing Unit) including an interface with peripheral devices, a control device, an arithmetic unit, a register, and the like. For example, the control function of the area determination unit 52 of the image generation server 50 may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), a software module and data from the storage 1003 and / or the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above-described embodiment is used.

For example, the control function of the area determination unit 52 of the image generation server 50 or the like may be realized by a control program stored in the memory 1002 and operated by the processor 1001, or may be realized in the same manner for other functional blocks. .. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be mounted on one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, or the like that can be executed to implement the wireless communication method according to the embodiment of the present invention.

The storage 1003 is a computer-readable recording medium, and is, for example, an optical disk such as a CDROM (Compact Disc ROM), a hard disk drive, a flexible disk, an optical magnetic disk (for example, a compact disk, a digital versatile disk, or a Blu-ray (registration)). It may consist of at least one such as a (trademark) disk), a smart card, a flash memory (eg, a card, stick, key drive), a floppy (registered trademark) disk, a magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server or other suitable medium containing memory 1002 and / or storage 1003.

The communication device 1004 is hardware (transmission / reception device) for communicating between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by the bus 1007 for communicating information. The bus 1007 may be composed of a single bus or may be composed of different buses between the devices.

Further, the communication terminal 10, the positioning server 30, and the image generation server 50 include a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field). It may be configured to include hardware such as Programmable Gate Array), and a part or all of each functional block may be realized by the hardware. For example, the processor 1001 may be implemented on at least one of these hardware.

Although the present embodiment has been described in detail above, it is clear to those skilled in the art that the present embodiment is not limited to the embodiment described in the present specification. This embodiment can be implemented as an amendment or modification without departing from the spirit and scope of the present invention as determined by the description of the scope of claims. Therefore, the description of the present specification is for the purpose of illustration and does not have any limiting meaning to the present embodiment.

Each aspect / embodiment described in the present specification includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA. (Registered Trademark), GSM (Registered Trademark), CDMA2000, UMB (Ultra Mobile Broad-band), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-Wide) Band), Bluetooth®, and other systems that utilize suitable systems and / or extended next-generation systems based on them may be applied.

The order of the processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in the present specification may be changed as long as there is no contradiction. For example, the methods described herein present elements of various steps in an exemplary order and are not limited to the particular order presented.

The input / output information and the like may be saved in a specific location (for example, memory) or may be managed by a management table. Information to be input / output may be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (Boolean: true or false), or by comparing numerical values (for example, a predetermined value). It may be done by comparison with the value).

Each aspect / embodiment described in the present specification may be used alone, in combination, or may be switched and used according to the execution. Further, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit one, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.

Software, whether called software, firmware, middleware, microcode, hardware description language, or other names, instructions, instruction sets, codes, code segments, program codes, programs, subprograms, software modules. , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, features, etc. should be broadly interpreted.

Further, software, instructions, etc. may be transmitted and received via a transmission medium. For example, the software may use wired technology such as coaxial cable, fiber optic cable, twist pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave to website, server, or other. When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

The information, signals, etc. described herein may be represented using any one of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.

Note that the terms described in the present specification and / or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

Further, the information, parameters, etc. described in the present specification may be represented by an absolute value, a relative value from a predetermined value, or another corresponding information. ..

The communication terminal may be a mobile communication terminal, a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communication device, a remote device, a mobile subscriber station, an access terminal, etc. It may also be referred to as a mobile device, wireless device, remote device, handset, user agent, mobile client, client, or some other suitable term.

The statement "based on" as used herein does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

When the terms "first", "second", etc. are used herein, any reference to that element does not generally limit the quantity or order of those elements. These designations can be used herein as a convenient way to distinguish between two or more elements. Therefore, references to the first and second elements do not mean that only two elements can be adopted there, or that the first element must somehow precede the second element.

As long as "include", "including", and variations thereof are used herein or within the scope of the claims, these terms are similar to the term "comprising". In addition, it is intended to be inclusive. Moreover, the term "or" as used herein or in the claims is intended to be non-exclusive.

In the present specification, a plurality of devices shall be included unless the device has only one clearly present in context or technically.

In the whole of this disclosure, if it does not clearly indicate the singular from the context, it shall include multiple things.

1 ... Information processing system, 51 ... Acquisition unit, 52 ... Area determination unit, 53 ... First specific unit, 54 ... Second specific unit, 55 ... Image generation unit, 56 ... Output unit, 57 ... Storage unit, 500 ... Content data.

Claims

A storage unit that stores content information in which an object in real space and AR content are associated with each other,
An acquisition unit that acquires peripheral space information including at least an image of the real space around the user,
An area determination unit that determines a display area of content including the AR content based on the peripheral space information, and an area determination unit.
Information processing system equipped with.
The area determination unit determines a public area for describing the AR content of the content information in the image in the real space and a private area for describing the private content related to the user.
The information processing system is
A first specific unit that identifies the AR content associated with an object in the real space included in the public area by referring to the content information.
Based on the instruction from the user, the second specific part that specifies the private content depicted in the private area, and
An image generation unit that generates a depiction image in which the AR content specified by the first specific unit is depicted in the public area and the private content specified by the second specific unit is depicted in the private area. When,
The information processing system according to claim 1, further comprising an output unit for outputting the depiction image.
The information processing system according to claim 2, wherein the area determination unit determines an area for securing the user's field of view as a non-depiction area that does not depict the content based on the peripheral space information.
The information processing system according to claim 3, wherein the area determination unit determines the public area, the private area, and the non-depiction area based on the type of the surrounding building specified from the surrounding space information.
The information processing system according to claim 3 or 4, wherein the area determination unit determines the public area, the private area, and the non-depiction area based on the shape of the surrounding building specified from the peripheral space information. ..
The acquisition unit further acquires information indicating the state of the user, and obtains information.
The information processing system according to any one of claims 3 to 5, wherein the area determination unit determines the public area, the private area, and the non-depiction area in consideration of the state of the user.
The acquisition unit acquires information indicating the state of the user including at least the viewpoint of the user, and obtains information.
The information processing system according to claim 6, wherein the area determination unit determines the public area, the private area, and the non-depiction area based on the viewpoint of the user.
The acquisition unit acquires information indicating the state of the user, including the movement speed of the user, and obtains information indicating the state of the user.
The information processing according to claim 6 or 7, wherein the area determination unit determines the public area, the private area, and the non-depiction area based on the movement mode of the user specified from the movement speed of the user. system.
The information processing system according to claim 8, wherein the area determination unit determines the non-depiction area without the non-depiction area when the movement mode of the user is stopped indicating that the user is not moving.