JP2016048192A - Information processing system, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology of achieving a beneficial function using information on a detected user behavior.SOLUTION: An information processing system acquires current location information of a user, acquires information on azimuth and elevation/depression angle of a line of sight of the user detected by a wearable device, identifies a structure of an object that the user views, by use of the current location information and the information on azimuth and elevation/depression angle, and acquires information on the structure from map information.SELECTED DRAWING: Figure 3

Description

  The present invention relates to information processing technology.

  In recent years, various wearable devices have been developed. A wearable device is a device that is assumed to be worn by a user on a body (for example, an arm or a head). For example, Patent Document 1 discloses a glasses-type wearable device worn on the head.

JP 2014-086905 A

  Since the wearable device is worn on the user's body, it is also possible to detect the user's movement using a sensor provided in the wearable device. A technology that can realize useful functions using such features of wearable devices is expected.

  The present invention has been made in view of the above, and an object of the present invention is to provide a technology capable of realizing a useful function using information about detected user actions.

  An information processing system according to the present invention includes current location information acquisition means for acquiring user's current location information, angle information acquisition means for acquiring information on the azimuth angle and elevation angle of the user's line of sight detected by a wearable device, It comprises gaze point information acquisition means for specifying a structure to be visually recognized by the user using current location information and information on the azimuth angle and elevation angle, and acquiring information about the structure from map information.

  An information processing method according to the present invention includes a current location information acquisition step of acquiring user's current location information, an angle information acquisition step of acquiring information on an azimuth angle and an elevation angle of the user's line of sight detected by a wearable device, Using the present location information and the information on the azimuth angle and the elevation angle, it is possible to identify a structure to be visually recognized by the user, and to include a point-of-gaze information acquisition step of acquiring information about the structure from map information.

  The program according to the present invention includes a computer, a current location information acquisition unit that acquires current location information of a user, an angle information acquisition unit that acquires information on an azimuth angle and an elevation angle of the user's line of sight detected by a wearable device, the current location Using the information and the information on the azimuth angle and the elevation angle, the structure to be visually recognized by the user is specified, and it functions as gaze information acquisition means for acquiring information about the structure from map information.

  The program of the present invention can be installed or loaded on a computer through various recording media such as an optical disk such as a CD-ROM, a magnetic disk, and a semiconductor memory, or via a communication network. .

  ADVANTAGE OF THE INVENTION According to this invention, the technique which can implement | achieve the useful function using the information about the detected user's operation | movement can be provided.

It is a conceptual diagram which shows the hardware constitutions of the information processing system in one Embodiment. It is a figure which shows the external appearance structure of the spectacles type device in one Embodiment. It is a figure which shows the function structure of the server apparatus in one Embodiment. It is a figure for demonstrating the specific method of the estimated attention point in one Embodiment. It is a figure for demonstrating the specific method of the estimated attention point in one Embodiment. It is a figure which shows the function structure of the spectacles type device in one Embodiment. It is a figure which shows the example of a display displayed on a display part in one Embodiment. It is a flowchart which shows the example of the process performed by the server apparatus. It is a flowchart which shows the example of the process performed by a spectacles type device.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the scope of the invention is not limited to these.

[Hardware configuration]
With reference to FIG. 1, an example of a hardware configuration of an information processing system according to an embodiment will be described. The information processing system 1 is a system for providing various services using wearable devices. The information processing system 1 mainly includes a server device 10, a glasses-type device 20, and a mobile terminal 30. The glasses-type device 20 and the portable terminal 30 can communicate with the server apparatus 10 via the network N. The eyeglass-type device 20 can communicate with the mobile terminal 30 (for example, via Bluetooth (registered trademark) or Wi-Fi (registered trademark)). For example, the information processing system 1 can provide various information providing services based on map information to a user who wears the glasses-type device 20 that is a wearable device and carries the mobile terminal 30.

  The network N is a communication line for transmitting and receiving information between the server apparatus 10 and the glasses-type device 20 and the mobile terminal 30. The network N may be, for example, the Internet, a LAN, a dedicated line, a packet communication network, a telephone line, a corporate network, another communication line, a combination thereof, or the like, and is wired or wireless ( Or any combination thereof).

  The server device 10 is constituted by an information processing device such as a dedicated or general-purpose server computer. The server device 10 may be constituted by a single information processing device or may be constituted by a plurality of information processing devices distributed on a network.

  The server device 10 mainly includes a control unit 11, a communication unit 12, and a storage unit 13. The control unit 11 mainly includes a CPU (Central Processing Unit) 11a and a memory 11b. The description of each configuration will be described later. For example, the server device 10 functions as various function implementing means by the CPU 11a executing a predetermined program stored in the memory 11b or the like. Note that FIG. 1 only shows a main configuration included in the server apparatus 10, and the server apparatus 10 includes other configurations included in a general information processing apparatus.

  The spectacle-type device 20 is a wearable device having a spectacle-type shape. The wearable device is a device (information processing apparatus) that is assumed to be used by being worn on the body (for example, an arm or a head) by a user. That is, since the spectacle-type device 20 has a spectacle-type shape, the user can use the spectacle-type device 20 while wearing the spectacle-type device 20 on the head like wearing spectacles. The eyeglass-type device 20 mainly includes a control unit 21, a communication unit 22, a storage unit 23, an operation unit 24, a display unit 25, a camera 26, a sensor 27, and a microphone 28. The control unit 21 mainly includes a CPU 21a and a memory 21b. For example, the glasses-type device 20 functions as various function realizing means when the CPU 21a executes a predetermined program stored in the memory 21b or the like. Since the user wears each of these configurations included in the glasses-type device 20 as a result when wearing the glasses-type device 20, each of these configurations can be regarded as a wearable device in a narrow sense. Each of these components will be described later.

  FIG. 2 shows an example of the external appearance of the eyeglass-type device 20. In the example of the figure, it is shown that the glasses-type device 20 includes an operation unit 24, a display unit 25, and a camera 26 in appearance. The operation unit 24 is located on the outer side of the eyeglass-type device 20. The user can issue an operation instruction to the glasses-type device 20 by swiping or tapping the operation unit 24. The display unit 25 is configured to be installed at a position where a user wearing the spectacle-type device 20 can visually recognize display contents by a prism or the like. The camera 26 is directed in the direction of the line of sight of the user wearing the spectacle-type device 20, and is installed at a position where it is possible to capture / capture what is estimated to be viewed by the user (present at the estimated gaze point). Configured as follows.

  Returning to the description of FIG. The mobile terminal 30 is a portable information processing device such as a mobile phone (including a smartphone), a tablet terminal, a PDA (Personal Digital Assistants), a navigation device, and a personal computer. The portable terminal 30 mainly includes a control unit 31, a communication unit 32, a storage unit 33, an operation unit 34, a display unit 35, and a positioning unit 36. The control unit 31 mainly includes a CPU 31a and a memory 31b. The description of each configuration will be described later. For example, the portable terminal 30 functions as various function realizing means by the CPU 31a executing a predetermined program stored in the memory 31b or the like.

  The control units 11, 21, 31 control operations of various configurations of the information processing apparatuses including these, and control execution of various processes. Details of processing executed in the control units 11, 21, 31 will be described later. The communication units 12, 22, and 32 are communication interfaces for communicating with other information processing apparatuses. The storage units 13, 23, and 33 are configured by a storage device such as a hard disk. The storage units 13, 23, and 33 store various programs and various information necessary for executing processing in each information processing apparatus. The operation units 24 and 34 are user interfaces for receiving operation instructions from the user. The display units 25 and 35 are user interfaces for displaying the processing results by each information processing apparatus.

The camera 26 is a device that can capture / capture still images / moving images.
The sensor 27 is composed of various sensors. The sensor 27 can include, for example, a gyro sensor, an acceleration sensor, a geomagnetic sensor, and / or an image sensor. By the sensor 27, for example, based on the direction in which the face of the user wearing the spectacle-type device 20 is facing, the azimuth of the direction of the line of sight of the user (the angle in the horizontal direction between the reference azimuth) An estimated value can be detected, or an estimated value of an elevation angle (an angle in the vertical direction with respect to the horizontal) in the direction of the user's line of sight can be detected based on the inclination of the user's head. The sensor 27 may be configured to detect the estimated values of the azimuth angle and the elevation angle in the user's line-of-sight direction from the movement of the user's eyeball. The sensor 27 can also detect blinking based on the movement of the eyelid of the user wearing the glasses-type device 20.
The microphone 28 is a device that converts voice / sound produced by a user wearing the glasses-type device 20 into an electrical signal.

  The positioning unit 36 specifies the position (for example, latitude and longitude) of the mobile terminal 30 based on a signal received from a GPS satellite or a communication device (for example, a base station), and specifies the date and time when the position is specified (positioning). Process to identify information).

  As described above, in the present embodiment, the information processing system 1 includes the server device 10, the glasses-type device 20, and the mobile terminal 30, but is not limited thereto, and can include any device. In addition, a configuration having a part or all of the configurations of the server device 10, the glasses-type device 20, and the mobile terminal 30 can be grasped as an information processing system. For example, a part of the configuration of the glasses-type device 20 can be grasped as an information processing system, and a configuration in which a part of the configuration of the glasses-type device 20 and a part of the configuration of the portable terminal 30 are combined. Can be grasped as an information processing system.

[Function configuration]
With reference to FIG. 3, a functional configuration in one embodiment of the server device 10 will be described. The server device 10 includes a current location information acquisition unit 111, an angle information acquisition unit 112, a gazing point information acquisition unit 113, a route search unit 114, and a database 120 as main functional configurations. Details of each functional configuration will be described below.

  The database 120 stores various types of information such as information necessary for processing executed in the server device 10 and information generated by the processing. The database 120 stores, for example, map information and facility information. The map information includes road information, railway line information, building information, terrain information, address information, and other geographical information. The facility information includes information about facilities such as convenience stores, cafes, stations, fast food, restaurants, buildings (buildings), and the like. The facility information includes information on products and / or services handled at the facility. Furthermore, the facility information includes information on the position of each facility associated with the map information. The facility information may be information included in the map information.

  The current location information acquisition unit 111 acquires the current location information of the user wearing the glasses-type device 20 and carrying the portable terminal 30 from the portable terminal 30 (or from the portable terminal 30 via the glasses-type device 20). Note that the current location information of the user is information obtained based on the positioning process of the positioning unit 36 of the mobile terminal 30.

  The angle information acquisition unit 112 wears the glasses-type device 20 and obtains information on the azimuth and elevation angles of the line of sight of the user who carries the portable terminal 30 from the glasses-type device 20 (or from the glasses-type device 20 to the portable terminal 30). To get through). Note that the information on the azimuth angle and the elevation angle is information obtained based on processing of the sensor 27 of the eyeglass-type device 20, as will be described later.

  The gazing point information acquisition unit 113 refers to the database 120, the current location information of the user acquired by the current location information acquisition unit 111, the azimuth angle of the user's line of sight acquired by the angle information acquisition unit 112, and Using the information on the elevation angle, the structure (object such as a facility) to be visually recognized by the user is specified, and the estimated gazing point in the structure is specified. The gazing point information acquisition unit 113 acquires information about the specified structure and information about the position of the estimated gazing point from the map information. The information about the position of the estimated gazing point includes information on the type of the specified structure (object such as a facility) (for example, a multi-purpose building, a station, a restaurant, a mountain, etc.), a name, and a position (address). In addition, when the specified structure is a building such as a building and one or more facilities are included in each floor of the building, the information on the position of the estimated gaze point includes the estimated gaze point. Information about the floor corresponding to the position of the floor (for example, including the number of floors of the floor and information of one or more facilities existing on the floor), or information on one facility existing at the position of the estimated gaze point Is included. Information about the position of the identified estimated gaze point is transmitted to the eyeglass-type device 20.

  With reference to FIG.4 and FIG.5, the example of the acquisition method of the information about the position of an estimated gaze point of a user is demonstrated. 4 and 5 show a user A wearing the glasses-type device 20 and carrying the portable terminal 30 and a structure (for example, a building) B. FIG. The point-of-gaze information acquisition unit 113 includes the current location information of the user A, the azimuth angle a of the line of sight of the user A (in this example, the azimuth angle when the north is the reference azimuth), the elevation angle b, and the map information. By using this, an object (an object to be visually recognized by the user) existing from the current location of the user A in the user A's line-of-sight direction is specified. In addition, the gazing point information acquisition unit 113 identifies, as the estimated gazing point, a position corresponding to the azimuth angle and the elevation angle of the line of sight of the user A in the identified object. In this example, the gazing point information acquisition unit 113 specifies the point C as the estimated gazing point. The target indicated by the estimated gazing point is not limited to a point, but includes an area having an arbitrary size and shape (for example, a circular area having a radius of 1 m, a rectangular area having 10 m square, etc.). The gaze point information acquisition unit 113 acquires information about the position of the specified estimated gaze point from the database 120.

  As described above, according to the present embodiment, the orientation of the direction in which the user's line of sight is detected by the wearable device (for example, the eyeglass-type device 20 or the sensor 27) when the user moves the head. Information on the position of the estimated gaze point of the user can be acquired using the information on the angle and the elevation angle. In other words, a useful function can be realized by using information about the user's operation detected by the wearable device. In particular, since the object is identified by using the elevation angle information of the user's line of sight, an object that cannot be seen unless the user looks up (for example, an object on the back side of a building in front of the user, It is possible to specify an object that can be seen only from the upper part of the building.

  Returning to the description of FIG. The route search unit 114 uses the specified position (for example, the current location of the user wearing the glasses-type device 20) as the departure point, and uses the position of the estimated gazing point specified by the gazing point information acquisition unit 113 as the destination. Perform search processing. Information about the searched route is transmitted to the eyeglass-type device 20.

  With reference to FIG. 6, the example of a function structure of the spectacles type device 20 which concerns on one Embodiment is demonstrated. The glasses-type device 20 includes a current location information acquisition unit 211, an angle information acquisition unit 212, a gazing point information acquisition unit 213, an action detection unit 214, a voice recognition unit 215, and a display control unit 216 as main functional configurations. Details of each functional configuration will be described below.

  The current location information acquisition unit 211 acquires current location information specified by the mobile terminal 30 from the mobile terminal 30. The acquired current location information is transmitted to the server device 10. Note that the current location information is acquired from the mobile terminal 30, but the present location information is not limited to this, and the current location information may be specified by the eyeglass-type device 20 including a positioning processing unit and performing a positioning process.

  The angle information acquisition unit 212 identifies the azimuth angle and the elevation angle in the direction in which the face of the user wearing the eyeglass device 20 is facing based on the detection result of the sensor 27. The angle information acquisition unit 212 identifies the estimated values of the azimuth angle and the elevation angle in the direction of the user's line of sight as angle information based on the azimuth angle and the elevation angle. The specified angle information is transmitted to the server device 10. Note that the user wears the sensor 27 included in the glasses-type device 20 as a result of wearing the glasses-type device 20 that is a wearable device, and thus the sensor 27 can also be regarded as a wearable device.

  The gazing point information acquisition unit 213 obtains information about the position of the estimated gazing point specified by the server device 10 (for a user wearing the spectacle-type device 20 and carrying the mobile terminal 30) as map information of the server device 10 Get from. In addition, although the information about the position of the estimated gazing point is specified by the server device 10, the present invention is not limited to this, and the gazing point information acquisition unit 213 performs a process of specifying information about the position of the estimated gazing point. You may go. That is, the gazing point information acquisition unit 213 is based on the current location information acquired by the current location information acquisition unit 211, the angle information specified by the angle information acquisition unit 212, and the map information acquired from the server device 10. Information on the position of the estimated gazing point can be specified by the same process as described above.

  The action detection unit 214 detects a predetermined action of the user of the eyeglass-type device 20 in addition to the input to the operation unit 24. The predetermined action includes, for example, the blinking of the user or the movement of the predetermined hand by the user in front of the glasses-type device 20. An arbitrary method can be used as a method for detecting these predetermined actions, and for example, a conventional detection technique can be used.

  The voice recognition unit 215 recognizes a voice uttered by the user of the eyeglass-type device 20 and specifies an operation instruction from the user.

  The display control unit 216 displays various types of information such as information on processing results by the eyeglass-type device 20 and information received from an external device (for example, current location information and information on the position of the estimated gazing point) on the display unit 25 (that is, Control is performed so as to display on a monitor located near the eyes of the user of the eyeglass-type device 20. In addition, the display control unit 216 estimates the estimated note acquired by the gazing point information acquisition unit 213 in response to a predetermined operation by the user via the operation unit 24 or a predetermined action detected by the action detection unit 214. It is possible to control to display information on the position of the viewpoint on the display unit 25.

  FIG. 7 shows an example of information about the position of the estimated gazing point displayed on the display unit 25 as a result of the control by the display control unit 216. In this example, the information about the position of the estimated point of gaze displayed on the display unit 25 includes information on the type of facility, the name of the facility, and the product (handled product) handled in the facility.

  As described above, according to the present embodiment, the azimuth angle in the direction in which the user's line of sight is detected when the user moves the head, detected by the wearable device (for example, the eyeglass-type device 20 or the sensor 27). And information about the position of the estimated gaze point of the user can be acquired using the information of the elevation angle and the information can be displayed on a monitor located in the vicinity of the user's eyes. In other words, a useful function can be realized by using information about the user's operation detected by the wearable device.

[Processing flow]
Next, a flow of processing executed in the server device 10 and the eyeglass device 20 will be described. Each processing step included in the processing flow described below can be executed in any order or in parallel as long as there is no contradiction in the processing contents. Steps may be added. Further, a step described as one step for convenience can be executed by being divided into a plurality of steps, while a step described as being divided into a plurality of steps for convenience can be grasped as one step. Note that details of the processing of each processing step have already been described, and are omitted here.

First, the flow of processing executed in the server device 10 will be described with reference to FIG.
In step S <b> 11, the control unit 11 acquires the current location information of the user wearing the glasses-type device 20 and carrying the portable terminal 30 from the portable terminal 30 (or from the portable terminal 30 via the glasses-type device 20). .

  In step S <b> 12, the control unit 11 wears the glasses-type device 20 and carries information on the azimuth and elevation angles of the line of sight of the user carrying the mobile terminal 30 from the glasses-type device 20 (or from the glasses-type device 20). Via terminal 30).

  In step S13, the control unit 11 refers to the database 120, and the information on the current location of the user acquired in step S11 and the information on the azimuth and elevation angles of the user's line of sight acquired in step S12. Is used to identify the structure (object such as a facility) that is to be visually recognized by the user, and the estimated gaze point in the structure is identified. Furthermore, the control part 11 acquires the information about the identified structure and the information about the position of the estimated gaze point from the map information.

  In step S <b> 14, the control unit 11 transmits information about the position of the estimated gazing point acquired in step S <b> 13 to the glasses-type device 20.

Next, with reference to FIG. 9, the flow of processing executed in the eyeglass-type device 20 will be described.
In step S <b> 21, the control unit 21 acquires the current location information of the user specified by the mobile terminal 30 from the mobile terminal 30. The acquired current location information is transmitted to the server device 10.

  In step S <b> 22, the control unit 21 specifies the azimuth angle and the elevation angle in the direction in which the face of the user wearing the eyeglass device 20 is facing based on the detection result by the sensor 27. The angle information acquisition unit 212 identifies the estimated values of the azimuth angle and the elevation angle in the direction of the user's line of sight as angle information based on the azimuth angle and the elevation angle. The specified angle information is transmitted to the server device 10.

  In step S <b> 23, the control unit 21 obtains information about the position of the estimated gazing point (of the user wearing the eyeglass device 20 and carrying the mobile terminal 30) specified by the server device 10 on the map of the server device 10. Obtained from the information via the network N.

  In step S24, in response to the detection of a predetermined action by the sensor 27, information on the position of the estimated gazing point acquired in step S23 is displayed on the display unit 25.

  As described above, according to the present embodiment, the azimuth angle in the direction in which the user's line of sight is detected when the user moves the head, detected by the wearable device (for example, the eyeglass-type device 20 or the sensor 27). And information about the position of the estimated gaze point of the user can be acquired using the information of the elevation angle and the information can be displayed on a monitor located in the vicinity of the user's eyes. In other words, a useful function can be realized by using information about the user's operation detected by the wearable device.

[Other Embodiments]
The present invention is not limited to the above-described embodiment, and can be implemented in various other forms without departing from the gist of the present invention. The above-described embodiment is merely an example in all respects, and is not construed as limiting.

  For example, in the above embodiment, the estimated gazing point is specified by the server device 10 or the spectacle-type device 20, but the present invention is not limited to this and may be performed by the mobile terminal 30. Specifically, the mobile terminal 30 specifies the current location information of the user by the positioning unit 36, acquires information on the azimuth angle and elevation angle of the user's line of sight as angle information from the eyeglass-type device 20, and acquires the current location information and angle Using the information, it is possible to specify the estimated gazing point by the method described above.

  10 server device, 20 glasses-type device, 30 portable terminal, 11, 21, 31 control unit, 11a, 21a, 31a CPU, 11b, 21b, 31b memory, 12, 22, 32 communication unit, 13, 23, 33 storage unit 24, 34 Operation unit, 25, 35 Display unit, 26 Camera, 27 Sensor, 28 Microphone, 36 Positioning unit

Claims (7)

Current location information acquisition means for acquiring the current location information of the user;
Angle information acquisition means for acquiring information on the azimuth angle and elevation angle of the user's line of sight detected by the wearable device;
Gaze point information acquisition means for specifying the structure to be visually recognized by the user using the current location information and the azimuth and elevation angle information, and acquiring information about the structure from map information. Information processing system. The gazing point information acquisition means specifies the user's estimated gazing point in the identified structure, acquires information about the position of the estimated gazing point from map information,
The information acquired by the gazing point information acquisition unit includes acquiring information on a floor located at the estimated gazing point in the building when the specified structure is a building. 1. The information processing system according to 1.   The information processing system according to claim 2, wherein the acquired floor information includes information on a facility existing on the floor located at the estimated gaze point. Detecting means for detecting a predetermined action of the user;
4. The display control unit according to claim 1, further comprising: a display control unit configured to control the information acquired by the gazing point information acquisition unit to be displayed on the wearable device in response to detection of the predetermined action. Information processing system according to item.   5. The information processing system according to claim 1, further comprising a search unit that performs a route search using the target position specified by the gazing point information acquisition unit as a destination. A current location information acquisition step for acquiring the current location information of the user;
An angle information acquisition step of acquiring information on the azimuth angle and elevation angle of the user's line of sight detected by the wearable device;
Using the current location information and the azimuth and elevation angle information to identify a structure to be visually recognized by the user, and a point-of-gaze information acquisition step of acquiring information about the structure from map information. Information processing method. Computer
Current location information acquisition means for acquiring the current location information of the user;
Angle information acquisition means for acquiring information on the azimuth and elevation angles of the user's line of sight detected by the wearable device;
Using the current location information and the information on the azimuth angle and the elevation angle, the gaze point information acquisition means for identifying the structure to be visually recognized by the user and acquiring information about the structure from map information,
Program to function as.

Images