JP7025480B2 - Information processing equipment, information processing methods and information processing programs - Google Patents

Description

The present invention relates to an information processing apparatus, an information processing method and an information processing program.

A technique for constructing a sound statistical database in which sounds in a predetermined space are collected for each type is disclosed.

Japanese Unexamined Patent Publication No. 2016-180791

However, in the above-mentioned conventional technique, fixed point observation using a recording device or the like installed in a predetermined space is assumed. Therefore, even if it is possible to analyze the environment of a specific place such as the interior of a building, there are obstacles (shields, etc.) such as ceilings and walls such as the upper and lower floors of the building and rooms next to each other, and they are spatially continuous. It was not possible to recognize the presence of obstacles such as spatial continuity and walls in places where there was no space.

The present application has been made in view of the above and aims to recognize the spatial continuity of the presence or absence of obstacles.

The information processing device according to the present application includes an acquisition unit that acquires sensor information and position information acquired at the same time by a user's terminal device having sensors that detect environmental sounds, pressure pressure, and illuminance, and the sensor information of each user. If the sensor information and the sensor information expected to be obtained at the position indicated by the position information are different from the determination unit that determines the spatial continuity based on the position information and the position information, the position information The correction unit is provided with a correction unit for correcting the above, and the correction unit exists within a predetermined range from the sensor information acquired in the past at the position indicated by the position information or the position indicated by the position information based on the position information. Using the sensor information acquired from the terminal device of another user, the sensor information expected to be obtained at the position indicated by the position information is estimated, and the sensor information expected to be obtained at the position indicated by the position information is used. When the sensor information is different from the sensor information acquired from the user's terminal device, the position information is modified to the position information corresponding to the sensor information acquired from the user's terminal device .

According to one aspect of the embodiment, it is possible to recognize the spatial continuity regarding the presence or absence of obstacles.

FIG. 1 is an explanatory diagram showing an outline of an information processing method according to an embodiment. FIG. 2 is a diagram showing a configuration example of an information processing system according to an embodiment. FIG. 3 is a diagram showing a configuration example of the terminal device according to the embodiment. FIG. 4 is a diagram showing a configuration example of the information providing device according to the embodiment. FIG. 5 is a diagram showing an example of a user information database. FIG. 6 is a diagram showing an example of a sensor information database. FIG. 7 is a diagram showing an example of a map information database. FIG. 8 is a flowchart showing a processing procedure according to the embodiment. FIG. 9 is a diagram showing an example of a hardware configuration.

Hereinafter, the information processing apparatus, the information processing method, and the embodiment for implementing the information processing program (hereinafter referred to as “the embodiment”) according to the present application will be described in detail with reference to the drawings. It should be noted that this embodiment does not limit the information processing apparatus, information processing method, and information processing program according to the present application. Further, in the following embodiments, the same parts are designated by the same reference numerals, and duplicate explanations are omitted.

[1. Overview of information processing method]
First, with reference to FIG. 1, an outline of an information processing method performed by the information processing apparatus according to the embodiment will be described. FIG. 1 is an explanatory diagram showing an outline of an information processing method according to an embodiment. In FIG. 1, a case where the spatial continuity regarding the presence or absence of obstacles (shields, etc.) is determined based on environmental information will be described as an example.

As shown in FIG. 1, the information processing system 1 includes a terminal device 10 and an information providing device 100. The terminal device 10 and the information providing device 100 are connected to each other via a network N (see FIG. 2) so as to be able to communicate with each other by wire or wirelessly.

The terminal device 10 is a smart device such as a smartphone or tablet used by the user U, and can communicate with an arbitrary server device via a wireless communication network such as 4G (Generation) or LTE (Long Term Evolution). It is a portable terminal device that can be used. Further, the terminal device 10 is a screen of a liquid crystal display or the like and has a screen having a touch panel function, and various types of display data such as contents such as tap operation, slide operation, scroll operation, etc. from the user by a finger, a stylus, or the like. Accepts the operation of. The operation performed on the area of the screen on which the content is displayed may be an operation on the content. Further, the terminal device 10 may be not only a smart device but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC. Here, a smartphone will be described as an example.

The information providing device 100 acquires sensor information (sensor data), user information, and position information, which are outputs of various sensors (Sensors) mounted on the terminal device 10, from the terminal device 10 of the user U. It is a processing device and is realized by a server device, a cloud system, or the like. Here, the information providing device 100 acquires sensor information, user information, and position information from each of an unspecified number of terminal devices 10, and stores the sensor information, user information, and position information in a database in association with each other. .. Then, the information providing device 100 determines the spatial continuity regarding the presence or absence of an obstacle based on the sensor information and the position information. For example, the sensor information of users within a predetermined range (position information is within a specific range) is compared, and if it is determined that they are different, it is determined that there is an obstacle such as a wall separating the space, and the space is determined. Judge that it is not continuous. Obstacles such as walls are not limited to permanent ones and may be temporary ones.

The user information is identification information or the like for uniquely identifying the user U or the terminal device 10. The information providing device 100 does not need to specify the user, and does not need to acquire the user information when only the sensor information and the position information are sufficient.

[1-1. Outline of information processing method from data collection to judgment of spatial continuity]
First, with reference to FIG. 1, an outline of an information processing method from data collection to determination of spatial continuity will be described.

As shown in FIG. 1, the information providing device 100 receives sensor information, user information, and position information from each of the terminal devices 10 of a plurality of user U located in various places (step S1).

At this time, the terminal device 10 may be moving or may stay in one place. Further, the information providing device 100 may receive sensor information, user information, and position information from the same terminal device 10 that moves around each place. The sensor information is, for example, environmental sound, atmospheric pressure, illuminance, and the like. The information providing device 100 extracts environmental sounds from the sounds included in the sensor information, and classifies (categorizes) the environmental sounds according to the type of sound. Further, the position information is the position information at the time when the terminal device 10 acquires the sensor information. The position information also corresponds to the sensor information in a broad sense. Further, the sensor information and the position information also include information regarding the date and time when the terminal device 10 acquired the sensor information and the position information.

In the example of FIG. 1, the information providing device 100 receives sensor information, user information, and position information from each of the terminal devices 10A to 10E possessed by the users U1 to U5. Of these, the terminal device 10A of the user U1 and the terminal device 10B of the user U2 exist in the same space. The terminal device 10C of the user U3, the terminal device 10D of the user U4, and the terminal device 10E of the user U5 exist in different spaces. For example, the terminal device 10C of the user U3, the terminal device 10D of the user U4, and the terminal device 10E of the user U5 exist on the upper and lower floors or adjacent rooms in the same building.

The building is, for example, an apartment house such as an apartment or an apartment, or a detached house. Actually, the building is an office building such as a company, a commercial facility such as a store, an accommodation facility such as a hotel, an educational institution such as a school, a medical institution such as a hospital, a research institution such as a research institute, and an industry such as a factory. It may be a distribution base such as a plant or a distribution center. In addition, large commercial facilities (shopping centers / outlet malls / underground streets), entertainment facilities (theme parks / amusement parks / amusement parks / zoos / aquariums / pools / bathing facilities), cultural facilities (halls / theaters / libraries / museums / museums) , Complex facilities, sports facilities, temples and shrines, service areas (SA) and parking areas (PA), railway stations, roadside stations, airports, ports (boarding areas), etc.

Further, the building has a space partitioned by, for example, a wall, a partition, a floor, a ceiling, or the like. These spaces are, for example, rooms in buildings, warehouses, passageways, and the like. The entrance, corridor, bath, toilet, garage, etc. may be included. That is, a space is a place where a person stays, goes in and out, or passes through. The space may be a place where a person lives or works, or may be a place where goods used by a person are stored. Further, in the building, the space may be one or a plurality. Further, the space is not limited to, for example, a room in a building, but may be a cabin of a vehicle, a ship, an aircraft, or the like. That is, the space is not limited to the space where people live indoors such as a house, but may be a space where passengers and passengers spend time in automobiles, high-speed buses, railroads, large passenger ships, airplanes, and the like. In other words, the space may be a living space inside a building or moving object.

The terminal device 10 of the user U is not limited to a building, but may exist in a space such as a tourist spot, a park, a waterside, a campsite, a forest, a tunnel, or a cave. That is, the space is not limited to indoors, but may be outdoors. As a matter of course, even outdoors, there are spaces partitioned by obstacles such as walls, buildings and other structures, and natural objects. Further, the space is not limited to the partitioned space, and may be a space within a predetermined range centered on a certain point.

Then, the information providing device 100 stores the sensor information received from the terminal device 10, the user information, and the position information in association with each other (step S2).

For example, the information providing device 100 stores sensor information, user information, and location information in a database in association with each other, and holds the information for a predetermined period (for example, one week / one month / six months / one year). In reality, the period may not be set and the accumulation may continue.

When the environmental sounds are classified by sound type, the information providing device 100 stores the user information and the position information for each sound type in association with each other.

Here, if the sensor information is different even though the position information is the same, the information providing device 100 determines that there is a problem (trouble, error, etc.) in acquiring any of the position information, and eventually The position information is corrected (step S3).

For example, the information providing device 100 uses the sensor information acquired in the past at the position based on the position information and the sensor information acquired from the terminal device 10 of another user existing within a predetermined range from the position. , Estimate the sensor information expected to be obtained at that position. That is, the information providing device 100 estimates accurate sensor information by majority voting based on the combination of the position information and the sensor information at that position. Alternatively, the information providing device 100 causes the model to learn a set of the position information and the sensor information expected to be obtained at that position as correct answer data. Then, the information providing device 100 acquires the sensor information expected to be obtained at the position by inputting the position information to the trained model. That is, the information providing device 100 acquires the sensor information output by the model when the position information is input as the estimation result. Then, the information providing device 100 corrects the position information associated with the sensor information different from the sensor information expected to be obtained at that position according to the sensor information. That is, the information providing device 100 corrects the position information presumed to be incorrect according to the sensor information.

Then, the information providing device 100 determines the spatial continuity regarding the presence or absence of an obstacle based on the sensor information and the position information (step S4).

For example, the information providing device 100 compares sensor information between users within a predetermined range, and if it is determined that they are different, it is determined that there is an obstacle such as a wall separating the space, and the information providing device 100 is spatially continuous. Judge that there is no. Specifically, the environmental sounds included in the sensor information of users within a predetermined range are compared, and if there is a difference in the volume of the environmental sounds above a predetermined threshold value, between the spaces where each user is located. It is judged that there is an obstacle such as a wall in the wall, and it is judged that the space is not continuous. That is, the information providing device 100 determines the spatial continuity regarding the presence or absence of an obstacle based on the sensor information and the position information.

At this time, the information providing device 100 may estimate the position of an obstacle such as a wall based on the sensor information and the position information.

Then, the information providing device 100 generates a map (map) regarding the space (target space) where each user is located based on the spatial continuity (step S5).

For example, when the information providing device 100 determines that there is an obstacle such as a wall, the information providing device 100 generates a map including the obstacle such as a wall. At this time, the information providing device 100 may generate a map in a specific space based on the estimated position of an obstacle such as a wall.

Then, the information providing device 100 updates the existing map based on the generated map (step S6).

The information providing device 100 may update the map information provided on the map search site or the like based on the generated map.

Then, the information providing device 100 provides a map in response to a request from the terminal device 10 of each user U (step S7).

At this time, the information providing device 100 may provide only the generated map (map including obstacles such as walls). An updated existing map may be provided.

[2. Information processing system configuration example]
Next, the configuration of the information processing system 1 including the information providing device 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment. As shown in FIG. 2, the information processing system 1 according to the embodiment includes a terminal device 10 and an information providing device 100. These various devices are connected so as to be communicable by wire or wirelessly via the network N. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

Further, the number and types of each device included in the information processing system 1 shown in FIG. 2 are not limited to those shown in the figure. For example, in FIG. 2, for simplification of the illustration, only one type of terminal device 10 is shown, but this is merely an example and is not limited, and may be two or more types.

The terminal device 10 is an information processing device used by the user U, and is a small portable device or mobile device having various sensors and capable of being carried or moved. For example, the terminal device 10 is a smart device such as a smartphone or a tablet terminal. Actually, feature phones, PCs (Personal Computers), PDAs (Personal Digital Assistants), car navigation systems, wearable devices such as smart watches and head-mounted displays, smart glasses, smart speakers, and 360-degree cameras. And so on.

Further, the terminal device 10 includes a wireless communication network such as LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation: 5th generation mobile communication system), Bluetooth (registered trademark), and wireless LAN (Local). It can be connected to the network N via short-range wireless communication such as Area Network) and communicate with the information providing device 100.

The information providing device 100 is, for example, a PC, a server device, a mainframe, a workstation, or the like. The information providing device 100 may be realized by cloud computing.

[3. Configuration example of terminal device]
Next, the configuration of the terminal device 10 will be described with reference to FIG. FIG. 3 is a diagram showing a configuration example of the terminal device 10. As shown in FIG. 3, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40. Be prepared.

(Communication unit 11)
The communication unit 11 is connected to the network N (see FIG. 2) by wire or wirelessly, and transmits / receives information to / from the information providing device 100 via the network N. For example, the communication unit 11 is realized by a NIC (Network Interface Card), an antenna, or the like.

(Display unit 12)
The display unit 12 is a display device that displays various information such as position information. For example, the display unit 12 is a liquid crystal display (LCD) or an organic EL display (Organic Electro-Luminescent Display). Further, the display unit 12 is a touch panel type display, but the display unit 12 is not limited thereto.

(Input unit 13)
The input unit 13 is an input device that receives various operations from the user U. The input unit 13 has, for example, a button for inputting characters, numbers, and the like. When the display unit 12 is a touch panel type display, a part of the display unit 12 functions as an input unit 13. The input unit 13 may be a microphone or the like that accepts voice input from the user U. The microphone may be wireless.

(Positioning unit 14)
The positioning unit 14 receives a signal (radio wave) transmitted from a GPS (Global Positioning System) satellite, and based on the received signal, position information (for example, latitude) indicating the current position of the terminal device 10 which is its own device. And longitude). That is, the positioning unit 14 positions the terminal device 10. GPS is only an example of GNSS (Global Navigation Satellite System).

In addition, the positioning unit 14 can position the position by various methods other than GPS. For example, the positioning unit 14 may position the position by using various communication functions of the terminal device 10 as an auxiliary positioning means for position correction or the like as described below.

(Wi-Fi positioning)
For example, the positioning unit 14 positions the position of the terminal device 10 by using the Wi-Fi (registered trademark) communication function of the terminal device 10 or the communication network provided by each communication company. Specifically, the positioning unit 14 determines the position of the terminal device 10 by performing Wi-Fi communication or the like and measuring the distance to a nearby base station or access point.

(Beacon positioning)
Further, the positioning unit 14 may position the position by using the Bluetooth (registered trademark) function of the terminal device 10. For example, the positioning unit 14 positions the terminal device 10 by connecting to a beacon transmitter connected by the Bluetooth function.

(Geomagnetic positioning)
Further, the positioning unit 14 positions the position of the terminal device 10 based on the geomagnetic pattern of the structure measured in advance and the geomagnetic sensor included in the terminal device 10.

(RFID positioning)
Further, for example, when the terminal device 10 has an RFID (Radio Frequency Identification) tag function equivalent to a contactless IC card used in a station ticket gate, a store, or the like, or has a function of reading an RFID tag. In this case, the used position is recorded together with the information that the payment or the like was made by the terminal device 10. The positioning unit 14 may position the position of the terminal device 10 by acquiring such information. Further, the position may be determined by an optical sensor included in the terminal device 10, an infrared sensor, or the like.

The positioning unit 14 may position the position of the terminal device 10 by using one or a combination of the above-mentioned positioning means, if necessary.

(Sensor unit 20)
The sensor unit 20 includes various sensors mounted on the terminal device 10. In the example shown in FIG. 3, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, a pressure sensor 23, a temperature sensor 24, a sound sensor 25, an optical sensor 26, a magnetic sensor 27, and an image sensor ( It is equipped with a camera) 28.

It should be noted that the above-mentioned sensors 21 to 28 are merely examples and are not limited thereto. That is, the sensor unit 20 may be configured to include a part of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or in place of the sensors 21 to 28. ..

The acceleration sensor 21 is, for example, a 3-axis acceleration sensor, and detects the moving direction, speed, acceleration, and other physical movements of the terminal device 10. The gyro sensor 22 detects the physical movement of the terminal device 10 such as the inclination in the triaxial direction based on the angular velocity of the terminal device 10 and the like. The barometric pressure sensor 23 detects, for example, the barometric pressure around the terminal device 10.

Since the terminal device 10 includes the above-mentioned acceleration sensor 21, gyro sensor 22, pressure sensor 23, etc., technologies such as pedestrian autonomous navigation (PDR) using each of these sensors 21 to 23, etc. It becomes possible to determine the position of the terminal device 10 by using. This makes it possible to acquire indoor position information, which is difficult to acquire with a positioning system such as GPS.

For example, a pedometer using an acceleration sensor 21 can calculate the number of steps, the walking speed, and the walking distance. Further, the gyro sensor 22 can be used to know the traveling direction of the user U, the direction of the line of sight, and the inclination of the body. Further, from the atmospheric pressure detected by the atmospheric pressure sensor 23, it is possible to know the altitude at which the terminal device 10 of the user U exists and the number of floors of the floor.

The air temperature sensor 24 detects, for example, the air temperature around the terminal device 10. The sound sensor 25 detects, for example, the sound around the terminal device 10. The optical sensor 26 detects the illuminance around the terminal device 10. The magnetic sensor 27 detects, for example, the geomagnetism around the terminal device 10. The image sensor 28 captures an image of the surroundings of the terminal device 10.

The pressure sensor 23, the temperature sensor 24, the sound sensor 25, the optical sensor 26, and the image sensor 28 described above detect the pressure, temperature, sound, and illuminance, respectively, and capture an image of the surroundings to obtain the terminal device 10. It is possible to detect the surrounding environment and situation of. Further, it is possible to improve the accuracy of the position information of the terminal device 10 from the surrounding environment and the situation of the terminal device 10.

(Control unit 30)
The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM, an input / output port, and various circuits. Further, the control unit 30 may be configured by hardware such as an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 includes a transmission unit 31, a reception unit 32, and a processing unit 33.

(Transmitting unit 31)
The transmission unit 31 is, for example, various information input by the user U using the input unit 13, various information detected by the sensors 21 to 28 mounted on the terminal device 10, and a terminal device positioned by the positioning unit 14. The position information and the like of 10 can be transmitted to the information providing device 100 via the communication unit 11.

(Receiver 32)
The receiving unit 32 can receive various information provided by the information providing device 100 and a request for sensor information from the information providing device 100 via the communication unit 11.

(Processing unit 33)
The processing unit 33 controls the entire terminal device 10 including the display unit 12 and the like. For example, the processing unit 33 can output various information transmitted by the transmitting unit 31 and various information received by the receiving unit 32 from the information providing device 100 to the display unit 12 for display.

(Memory unit 40)
The storage unit 40 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an optical disk. To. Various programs, various data, and the like are stored in the storage unit 40.

[4. Configuration example of information providing device]
Next, the configuration of the information providing device 100 according to the embodiment will be described with reference to FIG. FIG. 4 is a diagram showing a configuration example of the information providing device 100 according to the embodiment. As shown in FIG. 4, the information providing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Further, the communication unit 110 is connected to the network N (see FIG. 2) by wire or wirelessly.

(Memory unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. As shown in FIG. 4, the storage unit 120 has a user information database 121, a sensor information database 122, and a map information database 123.

(User information database 121)
The user information database 121 stores various information about the user U. For example, the user information database 121 stores various information such as the attributes of the user U. FIG. 5 is a diagram showing an example of the user information database 121. In the example shown in FIG. 5, the user information database 121 has items such as "user ID (Identifier)", "age", "gender", "home", "work location", and "terminal ID".

The "user ID" indicates identification information for identifying the user U. Further, "age" indicates the age of the user U identified by the user ID. The "age" may be a specific age of the user U identified by the user ID, for example, 35 years old. Further, "gender" indicates the gender of the user U identified by the user ID.

Further, "home" indicates the location information of the home of the user U identified by the user ID. In the example shown in FIG. 5, "home" is illustrated with an abstract reference numeral such as "LC11", but may be latitude / longitude information or the like. Further, for example, "home" may be a region name or an address.

Further, the "work location" indicates the location information of the work location (school in the case of a student) of the user U identified by the user ID. In the example shown in FIG. 5, the “work location” is illustrated with an abstract code such as “LC12”, but may be latitude / longitude information or the like. Further, for example, the "work location" may be a region name or an address.

Further, the "terminal ID" indicates identification information for identifying the terminal device 10 (operating terminal device) currently used by the user U identified by the user ID. The number of "terminal IDs" may be plural. That is, one user U may use a plurality of terminal devices at the same time. Further, the "terminal ID" may include information indicating the type of the terminal device 10.

For example, in the example shown in FIG. 5, the age of the user U identified by the user ID "U1" is "20's", and the gender is "male". Further, for example, the user U identified by the user ID "U1" indicates that his / her home is "LC11". Further, for example, the user U identified by the user ID "U1" indicates that the work location is "LC12". Further, for example, it is shown that the user U identified by the user ID "U1" is using the terminal device 10 identified by the terminal ID "D1".

Here, in the example shown in FIG. 5, although illustrated using abstract values such as “U1”, “LC11”, “LC12” and “D1”, “U1”, “LC11”, “LC12” and “ It is assumed that information such as a specific character string or a numerical value is stored in "D1". Hereinafter, abstract values may be illustrated also in figures relating to other information.

The user information database 121 is not limited to the above, and various information may be stored depending on the purpose. For example, the user information database 121 may store various information about the terminal device 10 of the user U. Further, the user information database 121 relates to attributes such as demographic (demographic attribute), psychographic (psychological attribute), geographic (geographical attribute), and behavioral (behavioral attribute) of the user U. Information may be stored. For example, the user information database 121 includes name, family structure, occupation, position, income, qualification, living style (detached house, apartment, etc.), presence / absence of a car, commuting time, commuting route, commuter pass section (station). , Lines, etc.), Frequently used stations (other than the nearest station to your home / work location), Behavior history (Frequently used location information), Lessons (location, time zone, etc.), Hobbies, interests, lifestyle, etc. Information may be stored. Further, the user information database 121 may store information about a search query (search keyword) input by the user U to a search engine or the like.

(Sensor information database 122)
The sensor information database 122 stores various information related to the sensor output. For example, the sensor information database 122 stores the sensor information from the terminal device 10 of the user U, the user information, and the position information in association with each other. FIG. 6 is a diagram showing an example of the sensor information database 122. In the example shown in FIG. 6, the sensor information database 122 has items such as "user ID", "acquisition date and time", "sensor information", "position information", "context", "proximity position", and "spatial continuity". Has.

The "user ID" is user information acquired from the terminal device 10, and indicates identification information for identifying the user U who uses the terminal device 10. The "user ID" may be identification information for identifying the terminal device 10. That is, the "user ID" may be a "terminal ID".

The "acquisition date and time" indicates the date and time when the terminal device 10 acquired the sensor information, the user information, and the position information. When the information providing device 100 acquires the sensor information, the user information, and the position information from the terminal device 10 in real time, the information providing device 100 obtains the sensor information, the user information, and the position information from the terminal device 10. It may be the date and time of acquisition. Further, the "acquisition date and time" is not limited to the year, month, day, hour, minute, and second, and may include information regarding the day of the week, daytime / nighttime, and the like.

"Sensor information" indicates the output (detection result) of various sensors of the terminal device 10. The "sensor information" is, for example, environmental sound, atmospheric pressure, illuminance, and the like. The environmental sounds are classified (categorized) and stored for each sound type.

The "position information" indicates the position information at the time when the terminal device 10 acquires the sensor information. The "position information" is, for example, position information measured by GPS, PDR, or the like.

The "context" indicates the environment or situation in which the user U is placed. "Context" means, for example, that user U is near a river or on a mountain, in a city or in a store, playing sports, playing games, driving a car, or sunshine. Indicates that you are in a room where your rights have been infringed.

The "proximity position" indicates other position information within a predetermined range (within 10 m, etc.) from the above "position information". If a plurality of position information corresponds to other position information within a predetermined range from the above "position information", the "proximity position" may be plural.

The "spatial continuity" is the presence or absence of spatial continuity obtained as a result of collating the above-mentioned "sensor information" at the same time with the sensor information associated with the position information indicated by the above-mentioned "proximity position". Is shown. Here, when there is spatial continuity, it is indicated as "○ (Yes)", and when there is no spatial continuity, it is indicated as "× (No)". Further, when there is no data regarding the proximity position or when the sensor information has not been collated yet, it is indicated as "-(not input)". However, these are just examples. In addition, when there are a plurality of the above-mentioned "proximity positions", the "spatial continuity" may be associated with each position information indicated by the proximity positions.

For example, in the example shown in FIG. 6, the terminal device 10 of the user U identified by the user ID "U1" has the sensor information "sensor information # 11" and the position information "position information" in the acquisition date and time "acquisition date and time # A". It indicates that "# 11" is acquired. Further, the user U identified by the user ID "U1" indicates that he / she is placed in the environment or situation indicated by the context "context # 11". Further, the sensor information "sensor information # 11" in the acquisition date and time "acquisition date and time # A" and the sensor information "position information # 21" existing around the position information "position information # 11" are associated with the proximity position "position information # 21". As a result of collating with "Sensor information # 21", it is shown that there is no obstacle such as a wall separating the space and there is spatial continuity "○ (Yes)".

The sensor information database 122 is not limited to the above, and various information may be stored depending on the purpose. For example, the sensor information database 122 may store information regarding an expiration date for storing sensor information. Further, the sensor information database 122 may store information regarding the analysis result (environmental evaluation result) of the sensor information. Further, the sensor information database 122 may store information regarding comments regarding a situation or the like at the time of acquisition of sensor information input by the user U of the terminal device 10. The comment may be text or voice.

(Map information database 123)
The map information database 123 stores various information related to the map. FIG. 7 is a diagram showing an example of the map information database 123. In the example shown in FIG. 7, the map information database 123 has items such as "map ID", "map information", "change location", "change date and time", and "obstacle".

The "map ID" indicates identification information for identifying the map. The map may be an existing map or a newly generated map.

“Map information” indicates map information provided to user U in response to a request from user U. Here, "map information" indicates map information of a place where the presence or absence of spatial continuity is confirmed.

"Changed part" indicates a changed part based on the generated map in the map information. That is, the changed part of the map information is shown. If there is no change, it is indicated as "-(not entered)". Here, the change includes new creation.

"Change date and time" indicates the date and time changed based on the generated map in the map information. That is, it indicates the date and time when the map information was changed. If there is no change, it is indicated as "-(not entered)".

"Obstacle" indicates the presence or absence of an obstacle such as a wall separating the space. Here, if there is an obstacle, it is indicated as "○ (Yes)", and if there is no obstacle, it is indicated as "× (No)". If there is no data on obstacles or if the presence or absence of obstacles is unknown, it is indicated as "-(not entered)".

For example, in the example shown in FIG. 7, the map identified by the map ID "U1" has an obstacle in the changed part "changed part # 1" in the map information "map information # 1" at the changed date and time "change date and time # 1". "○ (Yes)" indicates that the change has been made.

The map information database 123 is not limited to the above, and various information may be stored depending on the purpose. For example, the map information database 123 may store the area name indicated by the map, information on the topography and geography on the map, other detailed information, supplementary information, and the like. Further, the map information database 123 may store comments and the like regarding the map input by the user U.

(Control unit 130)
Returning to FIG. 4, the explanation will be continued. The control unit 130 is a controller, and is an information providing device 100 by means of, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. It is realized by executing various programs (corresponding to an example of an information processing program) stored in the internal storage device of the ASIC with a storage area such as a RAM as a work area. In the example shown in FIG. 4, the control unit 130 includes an acquisition unit 131, an integration unit 132, a correction unit 133, a determination unit 134, a generation unit 135, and an output unit 136.

(Acquisition unit 131)
The acquisition unit 131 acquires sensor information (sensor data), user information, and position information from each of the terminal devices 10 of the plurality of users U located in various places via the communication unit 110. If it is not necessary to specify the user and only the sensor information and the position information are sufficient, it is not necessary to acquire the user information.

(Aggregation unit 132)
The collecting unit 132 collects the sensor information received from the terminal device 10, the user information, and the position information in a database in association with each other. The integrated unit 132 may be a part of the acquisition unit 131.

Further, when the sensor information received from the terminal device 10 contains sound, the integration unit 132 extracts only the environmental sound from the sound. For example, the integration unit 132 may filter the sounds included in the sensor information to extract only the environmental sounds. Further, the integration unit 132 classifies (categorizes) environmental sounds according to the type of sound.

The integrated unit 132 may extract environmental sounds from the sounds included in the sensor information received from the terminal device 10 by using a model based on machine learning. In this case, the integration unit 132 may train the model as correct answer data, for example, a set of a sound included in the sensor information received from the terminal device 10 and an environmental sound classified (categorized) for each type of sound. Then, the integration unit 132 extracts the environmental sounds classified by the sound type by inputting the sounds included in the sensor information to the trained model. That is, the integration unit 132 may use the environmental sound output by the model when the sound included in the sensor information is input as the extraction result.

Any type of model can be adopted as the model. For example, the integration unit 132 may adopt an SVM (Support Vector Machine) or a DNN (Deep Neural Network) as a model. Here, the DNN may be a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). Further, the RNN may be an LSTM (Long short-term memory) or the like. That is, any type of model can be adopted as the model. Further, the model may be a model realized by combining a plurality of models, for example, a model in which a CNN and an RNN are combined.

The learning is, for example, deep learning (deep learning) using a deep neural network (DNN). Data mining and other machine learning algorithms may also be used. The integration unit 132 learns the model by the various learning methods described above.

(Correction part 133)
If the sensor information is different even though the position information is the same, the correction unit 133 determines that there is a problem (trouble, error, etc.) in acquiring any of the position information, and determines that there is a problem (trouble, error, etc.) in either position information. To fix. For example, the correction unit 133 uses the sensor information acquired in the past at the position based on the position information and the sensor information acquired from the terminal device 10 of another user existing within a predetermined range from the position. Estimate the sensor information expected to be obtained at that position. That is, the correction unit 133 estimates accurate sensor information by majority voting based on the combination of the position information and the sensor information at that position. Alternatively, the correction unit 133 causes the model to learn a set of the position information and the sensor information expected to be obtained at that position as correct answer data. Then, the information providing device 100 acquires the sensor information expected to be obtained at the position by inputting the position information to the trained model. Then, the correction unit 133 corrects the position information associated with the sensor information different from the sensor information expected to be obtained at that position according to the sensor information. That is, the correction unit 133 may have a function as an estimation unit for estimating sensor information.

(Judgment unit 134)
The determination unit 134 determines the spatial continuity regarding the presence or absence of an obstacle based on the sensor information and the position information. For example, the determination unit 134 compares the sensor information of users within a predetermined range based on the sensor information and the position information, and determines whether or not they are different. If it is determined that they are different, it is determined that there is an obstacle such as a wall separating the space, and it is determined that the space is not continuous.

Specifically, the determination unit 134 compares the environmental sounds included in the sensor information of the users within a predetermined range based on the sensor information and the position information, and makes a difference of the volume of the environmental sound by a predetermined threshold value or more. If there is, it is determined that there is an obstacle such as a wall between the spaces where each user is located, and it is determined that they are not spatially continuous.

At this time, the determination unit 134 may estimate the position where there is an obstacle such as a wall based on the sensor information and the position information. That is, the determination unit 134 may have a function as an estimation unit for estimating the position of an obstacle such as a wall.

Further, the determination unit 134 may determine whether the user U is indoors or outdoors based on the environmental sound, atmospheric pressure, illuminance, and the like.

(Generator 135)
The generation unit 135 generates a map (map) based on the result of the determination by the determination unit 134. For example, when it is determined that there is an obstacle such as a wall, the generation unit 135 generates a map including the obstacle such as a wall.

(Output unit 136)
The output unit 136 transmits information about the map to the terminal device 10 of the user U via the communication unit 110. The map may be an existing map or a newly generated map. Further, the output unit 136 may be a part of the generation unit 135.

[5. Processing procedure]
Next, the processing procedure by the information providing apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing a processing procedure according to the embodiment. The processing procedure shown below is repeatedly executed by the control unit 130 of the information providing device 100.

As shown in FIG. 8, the information providing device 100 acquires sensor information (sensor data), user information, and position information from each of the terminal devices 10 of the plurality of users U located in various places (step S101).

Then, the information providing device 100 associates the sensor information, the user information, and the position information and stores them in the database (step S102).

For example, the information providing device 100 extracts environmental sounds from the sounds included in the sensor information, classifies (categorizes) the extracted environmental sounds by sound type, and users classify the classified environmental sounds by sound type. It is stored in association with information and location information.

Then, the information providing device 100 determines whether or not the sensor information at the same position matches (step S103).

If the sensor information at the same position does not match (step S103; No), the information providing device 100 corrects any of the position information (step S104).

For example, if the information providing device 100 determines that there is a problem (trouble, error, etc.) in acquiring any of the position information when the sensor information is different even though the position information is the same, the information providing device 100 determines that there is a problem (trouble, error, etc.). Correct the location information of.

When the information providing device 100 matches the sensor information at the same position from the terminal device 10 of the user U (step S103; Yes), or when any of the position information is corrected (step S104), the information providing device 100 determines. Based on the sensor information and the position information, the spatial continuity regarding the presence or absence of an obstacle is determined (step S105).

For example, the information providing device 100 compares the sensor information of users within a predetermined range based on the sensor information and the position information, and if it is determined that they are different, there is an obstacle such as a wall separating the space. It is judged that it is not spatially continuous.

Then, the information providing device 100 generates a map (map) regarding the space (target space) where each user is located based on the spatial continuity (step S106).

For example, when the information providing device 100 determines that there is an obstacle such as a wall, the information providing device 100 generates a map including the obstacle such as a wall. At this time, the information providing device 100 may generate a map in a specific space based on the estimated position of an obstacle such as a wall.

Then, the information providing device 100 determines whether or not it is necessary to change the existing map (step S107). When the information providing device 100 does not need to change the existing map (step S107; No), the information providing device 100 does not have to change the existing map (does nothing).

When the information providing device 100 needs to change the existing map (step S107; Yes), the information providing device 100 updates the existing map based on the generated map (step S108).

The information providing device 100 may update the map information provided on the map search site or the like based on the generated map.

Then, the information providing device 100 provides a map in response to a request from the terminal device 10 of each user U (step S109).

At this time, the information providing device 100 may provide only the generated map (map including obstacles such as walls). An updated existing map may be provided.

[6. Modification example]
The terminal device 10 and the information providing device 100 described above may be implemented in various different forms other than the above-described embodiment. Therefore, a modified example of the embodiment will be described below.

In the above embodiment, the terminal device 10 may be stored in a pocket or bag (bag, backpack, pouch) of the user U's clothes. Therefore, the terminal device 10 may transmit the sensor information to the information providing device 100 only when it is used (operation, video viewing, etc.) by the user U. For example, the information providing device 100 acquires sensor information when the terminal device 10 is activating some application program (application) or the like, and does not acquire sensor information when no application or the like is activated. To. Alternatively, the information providing device 100 acquires the sensor information when the terminal device 10 is in the non-sleep state, and does not acquire the sensor information when the terminal device 10 is in the sleep state. In addition, "do not acquire sensor information" includes "do not store even if sensor information is acquired".

Further, the information providing device 100 acquires the operation history of the terminal device 10 together with the sensor information, and when the terminal device 10 has not been used by the user U for a predetermined time or more (for example, 5 minutes or more), the terminal device 10 It is presumed that the user U is stored in the pocket or bag of the clothes of the user U, and the sensor information from the terminal device 10 may not be acquired.

Further, the information providing device 100 presumes that the terminal device 10 is stored in the pocket or bag of the user U's clothes based on the sensor information (environmental sound, pressure, illuminance, etc.) or its change. In that case, the sensor information from the terminal device 10 may not be acquired.

The information providing device 100 may be implemented as a function in the terminal device 10 of the user U. For example, the information providing device 100 may be one of the terminal devices 10 of the user U.

[7. effect〕
As described above, the information providing device 100 according to the present application includes an acquisition unit 131 and a determination unit 134. The acquisition unit 131 acquires the sensor information and the position information acquired by the terminal device 10 of the user U having the sensor at the same time. The determination unit 134 determines spatial continuity based on the sensor information and the position information. As described above, the information providing device 100 according to the present application can recognize the spatial continuity regarding the presence or absence of obstacles by utilizing HaaS (Hardware as a Service).

[8. Hardware configuration]
Further, the terminal device 10 and the information providing device 100 according to the above-described embodiment are realized by, for example, a computer 1000 having a configuration as shown in FIG. Hereinafter, the information providing device 100 will be described as an example. FIG. 9 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to the output device 1010 and the input device 1020, and the arithmetic unit 1030, the primary storage device 1040, the secondary storage device 1050, the output I / F (Interface) 1060, the input I / F 1070, and the network I / F 1080 are bused. It has a form connected by 1090.

The arithmetic unit 1030 operates based on a program stored in the primary storage device 1040 or the secondary storage device 1050, a program read from the input device 1020, or the like, and executes various processes. The arithmetic unit 1030 is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used by the arithmetic unit 1030 for various calculations. Further, the secondary storage device 1050 is a storage device in which data used by the calculation device 1030 for various calculations and various databases are registered, and is a ROM (Read Only Memory), an HDD (Hard Disk Drive), and an SSD (Solid). State Drive), flash memory, etc. The secondary storage device 1050 may be an internal storage or an external storage. Further, the secondary storage device 1050 may be a removable storage medium such as a USB memory or an SD (Secure Digital) memory card. Further, the secondary storage device 1050 may be a cloud storage (online storage), NAS (Network Attached Storage), a file server, or the like.

The output I / F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various information such as a display, a projector, and a printer. For example, USB (Universal Serial Bus) or DVI. (Digital Visual Interface), HDMI (Registered Trademark) (High Definition Multimedia Interface), etc. Further, the input I / F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, a keypad, a button, a scanner, and the like, and is realized by, for example, USB.

Further, the output I / F 1060 and the input I / F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

Further, the output device 1010 and the input device 1020 may be integrated like a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated as input / output I / F.

The input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like.

The network I / F 1080 receives data from another device via the network N and sends it to the arithmetic unit 1030, and also transmits the data generated by the arithmetic unit 1030 to the other device via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I / F 1060 and the input I / F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 100, the arithmetic unit 1030 of the computer 1000 realizes the function of the control unit 130 by executing the program loaded on the primary storage device 1040. Further, the arithmetic unit 1030 of the computer 1000 may load a program acquired from another device via the network I / F 1080 onto the primary storage device 1040 and execute the loaded program. Further, the arithmetic unit 1030 of the computer 1000 may cooperate with other devices via the network I / F 1080 to call the functions and data of the program from other programs of the other devices and use them.

[9. others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Further, the above-mentioned components include those that can be easily assumed by those skilled in the art, those that are substantially the same, that is, those in a so-called equal range. Furthermore, the components described above can be combined as appropriate. Further, various omissions, replacements or changes of the components can be made without departing from the gist of the above-described embodiment.

Further, among the processes described in the above-described embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or part of it can be done automatically by a known method. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown in the figure.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

For example, the above-mentioned information providing device 100 may be realized by a plurality of server computers, or may be realized by calling an external platform or the like by API (Application Programming Interface), network computing, or the like depending on the function. Can be changed flexibly.

Further, the above-described embodiments and modifications can be appropriately combined as long as the processing contents do not conflict with each other.

Further, the above-mentioned "section, module, unit" can be read as "means", "circuit" and the like. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

1 Information processing system 10 Terminal equipment 14 Positioning unit 20 Sensor unit 100 Information provision device 110 Communication unit 120 Storage unit 121 User information database 122 Sensor information database 123 Map information database 130 Control unit 131 Acquisition unit 132 Integration unit 133 Correction unit 134 Judgment unit 135 Generator 136 Output

Claims (10)

An acquisition unit that acquires sensor information and position information acquired at the same time by a user's terminal device that has sensors that detect environmental sound, atmospheric pressure, and illuminance .
A judgment unit that determines spatial continuity based on the sensor information and the position information of each user ,
When the sensor information and the sensor information expected to be obtained at the position indicated by the position information are different, a correction unit for correcting the position information and a correction unit.
Equipped with
The correction part
Based on the position information, the sensor information acquired in the past at the position indicated by the position information or the sensor information acquired from the terminal device of another user existing within a predetermined range from the position indicated by the position information is used. Estimate the sensor information expected to be obtained at the position indicated by the position information, and
When the sensor information expected to be obtained at the position indicated by the position information and the sensor information acquired from the user's terminal device are different, the position information is converted into the position information according to the sensor information acquired from the user's terminal device. To fix
An information processing device characterized by this. The information processing apparatus according to claim 1, wherein the determination unit determines the presence or absence of an obstacle that separates a space based on the sensor information and the position information. The information processing apparatus according to claim 2, wherein the determination unit estimates the position of the obstacle that separates the space based on the sensor information and the position information. A generator that generates a map of the space where the user is located based on spatial continuity,
The information processing apparatus according to any one of claims 1 to 3, further comprising. The information processing apparatus according to claim 4, wherein the generation unit updates an existing map based on the generated map. A provider that provides the map in response to a request from the user,
The information processing apparatus according to claim 4 or 5, further comprising. If the sensor information is different even though the position information of each user is the same, the correction unit corrects the position information of any of the users.
The information processing apparatus according to any one of claims 1 to 6. An extraction unit that extracts environmental sounds from the sounds included in the sensor information,
Further prepare
The determination unit compares the environmental sounds of users within a predetermined range, and if there is a difference in the volume of the environmental sounds by a predetermined threshold value or more, it determines that they are not spatially continuous. The information processing apparatus according to any one of claims 1 to 7. It is an information processing method executed by an information processing device.
An acquisition process for acquiring sensor information and position information acquired at the same time by a user's terminal device having sensors for detecting environmental sound, atmospheric pressure, and illuminance .
A judgment process for determining spatial continuity based on the sensor information and the position information of each user, and
When the sensor information and the sensor information expected to be obtained at the position indicated by the position information are different, a correction step of correcting the position information and a correction step of correcting the position information.
Including
In the correction step,
Based on the position information, the sensor information acquired in the past at the position indicated by the position information or the sensor information acquired from the terminal device of another user existing within a predetermined range from the position indicated by the position information is used. Estimate the sensor information expected to be obtained at the position indicated by the position information, and
When the sensor information expected to be obtained at the position indicated by the position information and the sensor information acquired from the user's terminal device are different, the position information is converted into the position information according to the sensor information acquired from the user's terminal device. To fix
An information processing method characterized by that. An acquisition procedure for acquiring sensor information and position information acquired at the same time by a user's terminal device having sensors for detecting environmental sound, atmospheric pressure, and illuminance, and
A judgment procedure for determining spatial continuity based on the sensor information and the position information of each user, and
When the sensor information and the sensor information expected to be obtained at the position indicated by the position information are different, a correction procedure for correcting the position information and a correction procedure for correcting the position information.
Let the computer run
In the above correction procedure,
Based on the position information, the sensor information acquired in the past at the position indicated by the position information or the sensor information acquired from the terminal device of another user existing within a predetermined range from the position indicated by the position information is used. Estimate the sensor information expected to be obtained at the position indicated by the position information, and
When the sensor information expected to be obtained at the position indicated by the position information and the sensor information acquired from the user's terminal device are different, the position information is converted into the position information according to the sensor information acquired from the user's terminal device. To fix
An information processing program characterized by this .

Images