JP2019061519A - Information processing apparatus - Google Patents

Abstract

To inform a user of a place worth paying attention in a real space in a visually easily understandable manner.SOLUTION: An information processing apparatus 10 acquires a picked-up image of a real space picked up by an imaging display device 20, and controls the imaging display device 20 to display an additional image superimposed on the real space. Specifically, the information processing apparatus 10 displays, above respective areas included in the real space, floating bodies floating at an altitude according to information on the areas in a superimposed manner. The floating bodies may have the shape of a sphere or a polygon, and may have any shape, such as a shape simulating a balloon, a hot-air balloon, or an airship. The altitudes of the floating bodies are determined according to the information on the respective areas, and for example, determined according to the number of people in the respective areas grasped by human behavior grasping equipment 30. As the number of people in the area is larger, the altitude of the floating body displayed in a superimposed manner as if floating above the area becomes higher.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for superimposing an image on a physical space.

  The so-called AR (Augmented Reality) extends the real world seen by humans by adding, deleting, emphasizing and attenuating information on a captured image of the real space. For example, in Patent Document 1, by superimposing a spherical icon on a captured image, the user is notified of the presence of point information, and when the icon is selected by the user, the captured image is viewed from inside the sphere. It is disclosed to change the image.

JP, 2014-6582, A

  An object of the present invention is to visually and clearly indicate a place where a user deserves attention in real space.

  In order to solve the above problems, the present invention relates to a display control unit that causes a floating body to be superimposed and displayed above each area included in the real space, and a height of the floating body to be superimposed and displayed on the real space. And an altitude determining unit that determines the information according to the information on the area corresponding to the information processing apparatus.

  The height determination unit may determine the height of the floating body in accordance with the number of persons in the area or the number or amount of actions by persons in the area.

  The height determination unit may determine the height of the floating body in accordance with the volume of sound in the area.

  The height determination unit may determine the height of the floating body in accordance with the remaining time of the event being held in the area.

  The height determining unit may determine the height of the floating body in accordance with the possibility of participating in an event held in the area.

  The height determination unit may determine the height of the floating body in accordance with the number of imaging devices imaging the physical space including the area.

  The height determination unit may determine the height of the floating body in accordance with money paid as an advertisement cost for goods or services provided in the area. The display control unit may superimpose a moving image in which a plurality of floating bodies float from the floating bodies while the floating bodies are being displayed in a superimposed manner in a superimposed manner on the real space. .

  The display control unit may display information on the area corresponding to the floating body when the floating body displayed in a superimposed manner is selected by the user.

  When the floating body displayed in a superimposed manner is selected, a processing unit may be provided which performs processing for sharing information on an area corresponding to the floating body among a plurality of users.

  When the floating body displayed in a superimposed manner is selected, a processing unit may be provided that performs processing for providing the user with value information on an area corresponding to the floating body.

  According to the present invention, it is possible to visually and clearly indicate in a real space a place that the user deserves to pay attention to.

It is a figure which illustrates the composition of the information processing system concerning one embodiment of the present invention. It is a block diagram showing an example of the hardware constitutions of an imaging display. It is a block diagram showing an example of the hardware constitutions of an information processor. It is a figure which shows an example of area DB memorize | stored in the information processing apparatus. It is a block diagram showing an example of functional composition of an information processing system. It is a sequence chart which shows an example of operation of an information processing system. It is a figure which illustrates the physical space picturized by an imaging display. It is a figure which illustrates a mode that a floating body is superimposed and displayed on the real space imaged by the imaging display apparatus.

[Constitution]
FIG. 1 is a diagram showing an example of the information processing system 1 of the present embodiment. The information processing system 1 includes an information processing apparatus 10 that performs processing for realizing so-called AR (Augmented Reality), an imaging display apparatus 20 that captures and displays an image, and a person who grasps the number of people in each area. A trend grasping facility 30 and a network 90 for communicably connecting the information processing apparatus 10, the imaging display apparatus 20 and the human trend grasping facility 30 are provided. The network 90 is, for example, a Local Area Network (LAN) or a Wide Area Network (WAN), or a combination thereof, and includes a wired section or a wireless section. In addition, although the information processing apparatus 10, the imaging display apparatus 20, and the human movement grasping | ascertainment apparatus 30 are shown 1 each at FIG. 1, each of these may be plural.

  The information processing device 10 acquires the captured image of the physical space captured by the imaging display device 20, and controls the imaging display device 20 to display an additional image superimposed on the physical space. Specifically, the information processing apparatus 10 superimposes and displays a floating body floating at an altitude according to the information on the area above the areas included in the real space. The floating body may have, for example, a shape such as a sphere or a polyhedron, or may have any shape such as a shape simulating a balloon, a balloon, an airship or the like. The height of the floating body is determined in accordance with the information on each area, but in the present embodiment, it is determined in accordance with the number of persons present in each area grasped by the human trend grasping facility 30. The higher the number of people in the area, the higher the height of the floating body displayed superimposed so as to float above the area. When a user using the imaging display device 20 holds the imaging display device 20 in real space, the floating body is displayed at a high position above the sky as the area where many people gather is more prominent. As a result, it becomes possible to visually understand in an easy-to-understand manner which area is a place worth noting attention.

  FIG. 2 is a diagram illustrating the hardware configuration of the imaging display device 20. The imaging display device 20 is, for example, a wearable computer of a glass type, a head mounted display type, or a contact lens type, or a portable computer such as a smartphone or a tablet. The device for realizing AR includes a type in which an additional image called an AR object is superimposed on a captured image and displayed on a display surface that displays the captured image in real space, and light transmission disposed in front of the user's eyes There is a type that displays an AR object at a position on the display surface that overlaps with a predetermined position of the real space seen by the user via the display surface on the type display surface, but the imaging display device 20 is any type. It is also good. In any case, the imaging display device 20 functions as a device that displays the AR object superimposed on the real space.

  The imaging display device 20 includes a central processing unit (CPU) 201, a read only memory (ROM) 202, a random access memory (RAM) 203, an auxiliary storage device 204, a communication IF 205, an imaging unit 206, a display unit 207, an operation unit 208, and the like. The positioning unit 209 and the direction detection unit 210 are included.

  The CPU 201 is a processor that performs various operations. The ROM 202 is, for example, a non-volatile memory storing programs and data used to start the imaging display device 20. The RAM 203 is a volatile memory that functions as a work area when the CPU 201 executes a program. The auxiliary storage device 204 is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD), for example, and stores programs and data used in the imaging display device 20. When the CPU 201 executes this program, the imaging display device 20 functions as a computer device, and the function shown in FIG. 5 described later is realized. The communication IF 205 is an interface for performing communication via the network 90 in accordance with a predetermined communication standard. This communication standard may be a standard of wireless communication or a standard of wired communication. In the case of wireless communication, the communication IF 205 includes, for example, an antenna and a communication circuit operating according to a communication standard such as LTE (Long Term Evolution) or Wi-Fi (registered trademark). The imaging unit 206 includes a lens and an optical element, and generates captured image data indicating a captured image. The display unit 207 includes, for example, a liquid crystal panel, a liquid crystal drive circuit, and the like, and displays an image according to the image data. The operation unit 208 includes, for example, an operator such as a key or a touch sensor, receives an operation of the user, and supplies a signal corresponding to the operation to the CPU 201. The positioning unit 209 is, for example, a GPS (Global Positioning System) unit, and measures the position of the imaging display device 20. The orientation detection unit 210 is, for example, a gyro sensor or an acceleration sensor, and detects the orientation (orientation) of the imaging display device 20.

  FIG. 3 is a diagram illustrating the hardware configuration of the information processing apparatus 10. The information processing device 10 is a computer device having a CPU 101, a ROM 102, a RAM 103, an auxiliary storage device 104, and a communication IF 105. The CPU 101 is a processor that performs various operations. The ROM 102 is, for example, a non-volatile memory storing programs and data used to start the information processing apparatus 10. The RAM 103 is a volatile memory that functions as a work area when the CPU 101 executes a program. The auxiliary storage device 104 is a non-volatile storage device such as an HDD or an SSD, and stores programs and data used in the information processing apparatus 10. Execution of this program by the CPU 101 realizes the function shown in FIG. 5 described later. The communication IF 105 is an interface for performing communication via the network 90 in accordance with a predetermined communication standard. The information processing apparatus 10 may include, for example, other configurations such as a display unit and an operation unit, in addition to the configuration illustrated in FIG. 3.

  The auxiliary storage device 104 includes an area ID for identifying each area partitioned by a predetermined standard, the number of persons present in the area, and the height of the floating body determined according to the number of persons. An area database (hereinafter, database is abbreviated as DB) is stored. FIG. 4 is a diagram showing an example of the area DB. For example, in the area of the area ID “A01”, there are “2500 people”, and the height of the floating body determined according to the number is “100 m”. The number of persons present in each area is notified from the human trend grasping equipment 30 to the information processing apparatus 10 and stored in the auxiliary storage device 104. The human movement grasping equipment 30 estimates the number of persons present in each area, for example, from the in-zone information of the mobile phone carried by each person. In this estimation, concealment processing such as removal or modification of ID information or the like for identifying the user is performed so that the user of the mobile telephone can not be identified.

FIG. 5 is a block diagram showing an example of a functional configuration of the information processing system 1. As shown in FIG. 5, in the imaging display device 20, functions of an imaging unit 21 for imaging a physical space and a display unit 22 for displaying an image according to image data under the control of the information processing apparatus 10 are realized. Ru. In the information processing apparatus 10, the height of the floating body floating above the corresponding area is determined based on the acquisition unit 11 that acquires the number of persons present in each area from the human trend grasping facility 30 and the acquired number. A function of a height determination unit 12 and a display control unit for superimposing and displaying a floating body above each area included in the real space in the imaging display device 20 is realized.
The determination unit determines the height of the floating body according to the number of persons in the area or the number or amount of actions by persons in the area.

  In the imaging display device 20, the imaging unit 21 is realized by the imaging unit 206 and the communication IF 205 as hardware, and the display unit 22 is realized by the display unit 207 as hardware. In the information processing apparatus 10, the acquisition unit 11 is realized by the communication IF 105 as hardware, the height determination unit 12 is realized by the CPU 101 as hardware, and the display control unit 13 is realized by the CPU 101 and the communication IF 105 as hardware. Ru.

[Operation]
An operation example of the information processing system 1 will be described with reference to FIGS. 6 to 8. First, the user activates the imaging unit 21 of the imaging display device 20, and starts imaging of a physical space around itself (step S11). At this time, the positioning unit 209 and the orientation detection unit 210 respectively perform positioning and orientation detection (step S12). The imaging unit 21 transmits captured image data obtained by imaging the physical space to the information processing apparatus 10 together with information indicating the determined position and the detected direction (step S13).

  Here, FIG. 7 is a diagram illustrating an image captured by the imaging display device 20. The captured image includes a plurality of buildings B and a road R.

  When acquiring the captured image data and the like, the display control unit 13 of the information processing device 10 recognizes the position and orientation of the imaging display device 20, and specifies the range imaged by the imaging display device 20 (step S14). Then, the display control unit 13 compares the range imaged by the imaging display device 20 with the area DB, and specifies the area included in the range. Furthermore, the display control unit 13 specifies the height of the floating body corresponding to the specified area, transmits information including the position and the height of the area corresponding to the floating body to the imaging display device 20, and The imaging display device 20 is instructed to display a floating body at high altitude (steps S15 and S16).

  Under the control of the display control unit 13 of the information processing apparatus 10, the display unit 22 of the imaging display device 20 displays the floating body at the designated altitude in the designated area (step S17). For example, in the case where the imaging display device 20 is of a type that displays an AR object at a position on the display surface overlapping a predetermined position of the real space viewed by the user via the transmissive display surface, Display the floating body at a position on the display surface overlapping the position.

  Here, FIG. 8 is a view exemplifying a state in which the floating body is superimposed and displayed in the real space imaged by the imaging display device 20. In addition to a plurality of buildings B and roads R, floating bodies V1 to V4 are displayed in the captured image. Among the floating bodies, since the floating body V1 has the highest height, many people are gathered on the ground below the floating body V1. A user who sees this visits the area by himself or searches for information about the area, depending on the position of the floating body.

  According to the embodiment described above, the user of the imaging display device 20 can use the image display device 20 because the altitude of the floating object displayed so as to float above the area is higher as the number of people in the area is larger. It is possible to know in an easy-to-understand manner which area is a place worthy of attention.

[Modification]
The present invention is not limited to the embodiments described above. You may deform | transform the embodiment mentioned above as follows. Also, the following two or more modifications may be implemented in combination.
[Modification 1]
The altitude determination unit 12 may determine the altitude of the floating body according to conditions other than the number of people in each area. For example, the height determination unit 12 may determine the height of the floating body in accordance with the number or amount of human actions in each area. Here, the number or amount of human actions in each area is, for example, the number or amount of articles or comments posted by a person in the area on SNS (Social Networking Service) or the like. Since the function by which the post position is recorded when posting to SNS etc. is known, the altitude determination unit 12 uses this function to identify the altitude according to the number or amount of posts by people in each area. Just do it. In this way, an area where the number or amount of human actions is large can be used as a place worth noting.

  Further, the height determination unit 12 may determine the height of the floating body according to the volume of the sound in each area. Specifically, the volume of the sound in each area is detected by a microphone (for example, a microphone of a mobile phone carried by a human may be used) arranged in each area, and the height determination unit 12 uses the detection result. To determine the height of the floating body. For example, the higher the volume, the higher the floating body. In this way, an area with a large volume can be used as a place worth noting.

  Further, the height determination unit 12 may determine the height of the floating body in accordance with the remaining time of the event being held in each area. Specifically, the holding time and the ending time of the event held in each area may be described in the area DB, and the height determination unit 12 may specify the height of the floating body using the description content. . For example, the height of the floating body is higher as the remaining time of the event is longer. In this way, an area where the remaining time of the event is long can be used as a place worth noting.

  Further, the height determination unit 12 may determine the height of the floating body according to the possibility of participating in the event held in each area. Specifically, the holding time and the ending time of the event held in each area are described in the area DB, and the height determination unit 12 picks up the position where the event is held, the holding time, the ending time, and imaging The position of the user who is using the display device 20 is compared, and from the position of the user, the time that can participate in the event toward the area of the event is calculated, and the time is used to specify the height of the floating body do it. For example, the higher the floatable time for the event, the higher the floating body height. In this way, it is possible to use an area that is likely to participate in the event as a place worth noting.

  Further, the height determination unit 12 may determine the height of the floating body in accordance with the number of imaging devices that are imaging the real space including each area. Specifically, the altitude determination unit 12 specifies the position of the imaging range of the imaging and display device 20, and when each area is included in the imaging range, counts as the number of the areas being imaged. The height of the floating body may be specified using the counted number. The number of areas imaged may be counted, for example, at predetermined intervals of one minute. Thereby, the number of imaging devices that are imaging the real space including each area is specified. For example, the height of the floating body is higher as the number of imaging display devices 20 imaging the real space including each area is larger. In this way, it is possible to use an area in which the number of imaging devices imaging the physical space including each area is large as a place worth noting.

  Further, the height determination unit 12 may determine the height of the floating body in accordance with the money to be paid as the advertisement cost of the product or service provided in each area. Specifically, the advertisement cost of the product or service provided in each area may be described in the area DB, and the height determination unit 12 may specify the height of the floating body using the advertisement cost. For example, the higher the advertising cost, the higher the floating body height. In this way, it is possible to use an area paid a lot for advertising as a place worth noting.

[Modification 2]
The display control unit 13 may, for example, rupture the floating body when an act is performed such that the floating body being superimposed and displayed in the real space is gripped by the user (that is, an action such as breaking a balloon). A moving image in which a plurality of small floating bodies float from the floating body may be superimposed and displayed on the real space. Furthermore, when the display control unit 13 performs an action to touch the small floating body by the user, the floating body transmits information on the area (for example, a position of the area or a route to the area). Alternatively, the event name etc. held in the area may be superimposed and displayed.

  In addition, when the floating body is selected, for example, by the user's act of touching the floating body superimposed and displayed in the real space, the information processing apparatus 10 information on the area corresponding to the floating body (for example, the area A processing unit may be provided which performs processing for sharing a plurality of users with the position of the path, the route to the area, or the name of an event held in the area. For example, the processing unit performs the process of posting the above information on the SNS or the like used by the user.

  In addition, when the floating body is selected, for example, by the user's act of touching the floating body superimposed and displayed in the real space, the information processing apparatus 10 sends the user the value information on the area corresponding to the floating body. You may make it provide the process part which performs the process for provision. For example, the processing unit transmits information on the area (for example, the position of the area, a route to the area, an event name held in the area, or the like) to the imaging display device 20 by e-mail.

[Modification 3]
The size, color and shape of the floating body are arbitrary. Also, the size, color and shape of the floating body may be changed according to some conditions.

[Other modifications]
The block diagram used for the description of the said embodiment has shown the block of a functional unit. These functional blocks (components) are realized by any combination of hardware and / or software. Moreover, the implementation means of each functional block is not particularly limited. That is, each functional block may be realized by one physically and / or logically coupled device, or directly and / or indirectly two or more physically and / or logically separated devices. It may be connected by (for example, wired and / or wireless) and realized by the plurality of devices. For example, at least a part of the functions of the information processing apparatus 10 may be implemented in the imaging display apparatus 20. When all the functions of the information processing device 10 are implemented in the imaging display device 20, the imaging display device 20 functions as the information processing device according to the present invention.

  Each aspect / embodiment described in the present specification is LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA (Registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-Wide Band), The present invention may be applied to a system utilizing Bluetooth (registered trademark), other appropriate systems, and / or an advanced next-generation system based on these.

As long as there is no contradiction, the processing procedure, sequence, flow chart, etc. of each aspect / embodiment described in this specification may be reversed. For example, for the methods described herein, elements of the various steps are presented in an exemplary order and are not limited to the particular order presented.
Each aspect / embodiment described in this specification may be used alone, may be used in combination, and may be switched and used along with execution. In addition, notification of predetermined information (for example, notification of "it is X") is not limited to what is explicitly performed, but is performed by implicit (for example, not notifying of the predetermined information) It is also good.

  The terms "system" and "network" as used herein are used interchangeably.

  The information or parameters described in the present specification may be represented by absolute values, may be represented by relative values from predetermined values, or may be represented by corresponding other information. For example, radio resources may be indexed.

  The names used for the parameters described above are in no way limiting. In addition, the formulas etc. that use these parameters may differ from those explicitly disclosed herein. Since various channels (eg PUCCH, PDCCH etc.) and information elements (eg TPC etc.) can be identified by any suitable names, the various names assigned to these various channels and information elements can be Is not limited.

  The terms "determining", "determining" as used herein may encompass a wide variety of operations. For example, “judgment” and “decision” may be judging, calculating, calculating, processing, processing, deriving, investigating, looking up (for example, a table) (Searching in a database or another data structure), ascertaining may be regarded as “decision” or “decision”. Also, "determination" and "determination" are receiving (e.g. receiving information), transmitting (e.g. transmitting information), input (input), output (output), access (Accessing) (for example, accessing data in a memory) may be regarded as "determined" or "determined". In addition, "determination" and "decision" are to be considered as "determination" and "determination" that they have resolved (resolving), selecting (selecting), choosing (choosing), establishing (establishing), etc. May be included. That is, "determination" "determination" may include considering that some action is "determination" "determination".

  The present invention may be provided as an information processing method including steps of processing performed in the information processing system 1. Further, the present invention may be provided as a program executed by the imaging display device 20 or the information processing device 10. Such a program may be provided in the form of being recorded in a recording medium such as an optical disk, or may be provided in the form of being downloaded to a computer via a network such as the Internet and installed and made available. It is possible.

  Software, instructions, etc. may be sent and received via a transmission medium. For example, software may use a wireline technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or a website, server or other using wireless technology such as infrared, radio and microwave When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

  The information, signals, etc. described herein may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips etc that may be mentioned throughout the above description may be voltage, current, electromagnetic waves, magnetic fields or particles, optical fields or photons, or any of these May be represented by a combination of

  The terms described herein and / or the terms necessary for the understanding of the present specification may be replaced with terms having the same or similar meanings. For example, the channels and / or symbols may be signals. Also, the signal may be a message. Also, the component carrier (CC) may be called a carrier frequency, a cell or the like.

  Any reference to an element using the designation "first," "second," etc. as used herein does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient way of distinguishing between two or more elements. Thus, reference to the first and second elements does not mean that only two elements can be taken there, or that in any way the first element must precede the second element.

  The “means” in the configuration of each device described above may be replaced with a “unit”, a “circuit”, a “device” or the like.

  As long as "including", "comprising", and variations thereof are used in the present specification or claims, these terms as well as the term "comprising" are inclusive. Intended to be Further, it is intended that the term "or" as used in the present specification or in the claims is not an exclusive OR.

  Throughout the disclosure, for example, when articles are added by translation, such as a, an, and the in English, these articles are not clearly indicated by the context, unless the article clearly indicates otherwise. It shall contain several things.

  Although the present invention has been described above in detail, it is apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be embodied as modifications and determinations without departing from the spirit and scope of the present invention defined by the description of the claims. Accordingly, the description in the present specification is for the purpose of illustration and does not have any limiting meaning on the present invention.

DESCRIPTION OF SYMBOLS 1 ... Information processing system, 10 ... Information processing apparatus, 11 ... Acquisition part, 12 ... Altitude determination part, 13 ... Display control part, 20 ... Imaging display apparatus, 21 ... Imaging part, 22 ... Display part, 30 ... Human trend grasp Equipment 90: Network 101: CPU 102: RAM 103: ROM 104: auxiliary storage device 105: communication IF 201: CPU 202: RAM 203: ROM 204: auxiliary storage device 205: communication IF 206: Imaging unit, 207: Display unit, 208: Operation unit, 209: Positioning unit, 210: Direction detection unit.

Claims (10)

A display control unit for displaying a floating body superimposed on the sky above each area included in the real space;
An altitude determining unit configured to determine the altitude of the floating body superimposed and displayed on the real space according to the information on the area corresponding to the floating body. The information processing according to claim 1, wherein the height determination unit determines the height of the floating body according to the number of persons in the area or the number or amount of actions by persons in the area. apparatus. The information processing apparatus according to claim 1, wherein the height determination unit determines the height of the floating body in accordance with the volume of sound in the area. The information processing apparatus according to claim 1, wherein the height determination unit determines the height of the floating body according to a remaining time of an event held in the area. The information processing apparatus according to claim 1, wherein the height determination unit determines the height of the floating body according to a possibility of participating in an event held in the area. The information processing apparatus according to claim 1, wherein the height determination unit determines the height of the floating body according to the number of imaging devices imaging the physical space including the area. The information processing apparatus according to claim 1, wherein the height determination unit determines the height of the floating body according to money paid as an advertisement cost of a product or service provided in the area. The said display control part displays the information regarding the said area corresponding to the said floating body, when the said floating body currently displayed by superposition is selected by the user. The information processing apparatus according to claim 1. The processor according to any one of claims 1 to 8, further comprising: a processing unit that performs processing for sharing information on an area corresponding to the floating body among a plurality of users when the floating body displayed in a superimposed manner is selected. The information processing apparatus according to any one of the items. The processor according to any one of claims 1 to 9, further comprising: a processing unit that performs processing for providing the user with value information on an area corresponding to the floating body when the floating body displayed in a superimposed manner is selected. An information processing apparatus according to any one of the preceding items.

Images