JP2022132273A - Information providing service program and information distribution device for information providing service - Google Patents

Abstract

To provide software, a program, and an information distribution device for enjoying general-purpose location-linked information support services without the need for complicated information user registration, information provider registration, and construction of a dedicated distribution server in an information distribution system that supports location-linked information terminals.SOLUTION: An intermediary information distribution device acquires predetermined condition information such as terminal location information from an information terminal device, collects information that is open to the public on a network based on the acquired condition information, records information necessary for display from the acquired information, and provides the necessary information to a user. Thereby, the intermediary information distribution device collects, analyzes, and shares the information on the existing network necessary for a user information terminal.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information providing service program and a distribution apparatus as an information processing apparatus for information providing service.

The present invention detects an image captured by an image capturing unit associated with an information terminal and uses any position detection unit (GNSS, GPS, quasi-zenith satellite, wireless LAN position estimation, acceleration sensor, etc.) for arbitrary latitude and longitude. Any method that can acquire position information) and direction detection unit (any method that detects the direction, orientation, and rotation status of the information terminal by any method, such as an electronic compass or an electronic gyro sensor that detects geomagnetism) Then, a CG image is generated by a CG (computer graphic) generation unit, and the imaged image obtained by the imaging unit (real scenery, etc., regardless of whether it is outdoors or indoors) and the CG image generated by the image generation unit are combined. It is an invention of a terminal position of a user information terminal to be combined and a distribution device used for operation of an information support device.

More specific content of the application described later is, for example, a service that acquires advertisements and advertisements from a predetermined distribution server with image features based on location, and performs advertisements and advertisements. In addition, there is a service that acquires necessary information from a predetermined distribution server for information to be combined in the imaged screen, or a service that sets CG information to be combined based on the ID acquired from the two-dimensional marker and transmits the necessary information to the predetermined information. It is configured in the form of a service that acquires from a distribution server, or a service that acquires management information from a predetermined server as to which object the CG to be synthesized is licensed. Special delivery devices are required to provide users with structured proprietary information, in the form of dedicated servers or the use of dedicated markup languages for these services.

JP-A-2002-027529 JP 2010-118019 A JP 2013-149029 A JP 2013-196616 A JP 2014-142847 A JP 2019-020162 A Japanese Patent Publication No. 2014-505896 WO2013/108549

Written by Dieter Schmalstieg et al., Translated by Kiyoshi Ikeda et al.

The main object of the present invention is to provide a sufficient amount of content on a global scale by using information on existing networks to fill the lack of content in the network service infrastructure that is being newly developed. It is to make it possible to build an environment for providing a new network service infrastructure. More specifically, the problem to be solved by the present invention relates to AR (Augmented Reality) / MR (Mixed Reality) services linked to the location of the terminal, and is directed to user terminals without using a dedicated distribution system. There is a problem that appropriate information cannot be provided.

In new Internet services, for new information spaces such as outdoors where content is scarce, servers will be installed as dedicated distribution devices, and an information distribution environment using a dedicated markup language will be prepared. There are existing markup languages such as ARML and KML as XML for AR space in the case of services for AR/MR as described in the prior literature, and existing AR solutions such as Wikitube (registered trademark) are available. However, since it uses a dedicated markup language, there are problems that the dedicated markup language is not widespread and that it is not popular because it requires the installation of a dedicated server.

Such an essential problem is to provide sufficient content and/or sufficient information for provision at the start of service provision in order to configure the information service system and/or information service infrastructure by position interlocking used in the present invention. Therefore, the fundamental problem arises as to whether the chicken (information user) or the egg (information provider) is needed first, that the user cannot prepare sufficiently.

In order to solve the above-mentioned fundamental problem, the present invention collects information configured on a general network without relying on the expansion of information provided by the service provider itself related to the present invention, The information generated by extracting necessary information from the collected information and combining the extracted information appropriately according to the situation of the user information terminal on the user information terminal is synthesized and displayed on the actual image on the user terminal By doing so, by reconstructing the information acquired by searching existing network information, the above-mentioned problem of lack of dedicated information for AR / MR services is solved. /MR can be applied.

According to the present invention, it is possible to generate information to be developed in an AR space without content information by scraping processing or mashup processing from existing content information on the Internet, and furthermore, convert to ARML or KML at the time of mashup of the present invention. By doing so, the display information can be handled easily as a processing system, and by registering the mashed-up information in the database (structure is shown in FIG. It is also possible to provide a caching system used for efficient information providing services and contents that are lacking in newly provided services, thereby improving user convenience in new services.

In addition, the above-mentioned mashup processing collects predetermined contents into a predetermined format according to the search conditions used in the scraping process and the user's personal information, and extracts information that is likely to give serendipity to the user who specified the search conditions. Recommendation information and personalized information are generated and registered in the databases of user terminals, information distribution devices, and information relay mediation devices, so that non-three-dimensional By appropriately superimposing information on the network, it is possible to realize information generation processing and the like that can improve the user's convenience.

More specifically, various implementation methods such as collaborative filtering have been proposed for mashup and recommendation methods. The relay server of the present invention performs scraping and mashup processing in response to a request from a user terminal in which the relay server of the present invention receives the mashed-up information generated by the client application, and caches the information for the client application. It is characterized in that it is implemented by converting it into customized information.

In addition, the CG information provided in this section and other sections may be two-dimensional images, three-dimensional images, or moving images, and the image format does not have to be a specific one. Alternatively, other arbitrary information such as voice may be provided at the same time, or, of course, a method of giving different images to both eyes of an HMD (head mounted display) for stereoscopic display or a method of smartphone display may be used. .

In addition, the information terminal can connect to the information intermediary distribution device and obtain information from the terminal without receiving an exclusive service provided by obtaining information from a special information distribution device and without registering as a user. This allows users to obtain more convenient location-linked information support for their own information terminal device from any information intermediation distribution device, while maintaining sufficient privacy. By sharing information with other information terminals and information intermediary distribution devices if necessary, users can enjoy more convenient services. It is also possible to enjoy non-centralized services by configuring the communication environment.

According to the present invention, in the new network service infrastructure, existing information is used, and without preparing information specifically configured for the new technology, the new network service infrastructure can be fully developed on a global level. Content can be provided. The service application user of the present invention can enjoy the optimum location-linked information support service without special user registration, and the information sender who wants to provide information by the service is also a user of the location-linked information distribution system. Since information can be provided to service app users without registration or distribution information registration, location-linked services can be provided in an extremely convenient state without registering users or distributors for location-linked services.・There is an effect that you can enjoy it. In addition, as a secondary effect, there is an advantage that the service provider does not have to install a dedicated distribution server for distribution to the user information terminal. As a result, content can be enriched in a new environment with little content.

In this way, it is possible to deploy existing content information on the Internet in an AR/MR space where there is no content information by scraping processing and mashup processing linked to the location, and it can be used in information spaces such as new outdoors where content is scarce. On the other hand, without installing servers as dedicated distribution devices or preparing information distribution information and distribution environments using a dedicated markup language, it is possible to suddenly provide sufficient content on a global level. In addition to being able to define the configuration of computer graphics for displaying on the user information terminal at the time of mashup processing, it is also possible to store the database in the user terminal (the structure is shown in FIG. 6) and the information By registering the mashed-up information in the database (structure shown in FIG. 6) of the distribution device and/or the information distribution relay device, it is possible to sufficiently collect and provide necessary information for new services without content.

There are existing markup languages such as ARML and KML as XML for AR space. is not popular because it uses these dedicated markup languages. Furthermore, the display information may be converted into ARML or KML at the time of the mashup of the present invention to make it easier to handle as a processing system, or may be expressed in markdown language and effects may be added to the display content.

In addition, even if information is provided in cooperation with the information terminal and the service information providing and distributing system, the service information providing and distributing system does not require that the person wishing to provide information register predetermined information in the service information providing and distributing system. By being able to provide information terminal users with service information, there is an effect that users can enjoy convenience with higher added value. In order to make these effects optimal and reliable, various techniques such as statistical processing, machine learning, and AI methods such as deep learning may be combined to improve the effects of the present invention. It is also possible to realize advertisement by /MR.

As described in the above section, according to the present invention, the distribution device itself collects appropriate information in advance, so the chicken (information distributor) and the egg (information distributor) that prioritize infrastructure or content There is an effect of the invention that the convenience of the service system in the present invention is greatly improved by solving the fundamental problem of the information user). In the case of software implementing the present invention, it may be stored in a storage medium associated with a storage unit or the like, or may be distributed over a network. It may be stored in an associated storage medium, and as a result, it is possible to realize the merits of being able to synthesize and display virtual CG objects in an appropriate real space three-dimensional coordinate system while enriching content in a new environment with little content.

An example of the configuration of a single user information terminal used as a client is shown. 1 shows an example configuration of an information distribution device used as a distribution device. 4 shows a processing procedure according to the present invention by a single information terminal. 1 shows a processing procedure of an embodiment according to the present invention by interlocking an information distribution server and an information terminal; An example of a system processing procedure by cooperation between a user information terminal for a client and an information distribution server is shown. FIG. 10 shows an example of a table configuration when position-linked information is configured as a database; FIG.

Next, in an information distribution device used in an information distribution service system that obtains information of any combination of information linked to information such as position, direction, inclination, etc. from a user information terminal and distributes information, the information distribution device is used. If a person who wishes to distribute information to a general information terminal user tries to realize an information distribution service system for general information terminal users without special information distribution provider registration (for example, registration procedure such as store registration) Whether information is provided to the user information terminal on the consumer side using the information distribution device having such a device configuration is more specifically based on the step configuration as the information distribution system, and the information distribution A step-by-step explanation will be given as to what kind of step configuration the software to be operated is executed when the hardware configuration similar to that of the information distribution device in the system is used.

As shown in FIG. 2, the information distribution device (2001) of the information distribution system of the present invention comprises at least a calculation section (2010), a storage section (2020), and a communication section (2030). Furthermore, the storage unit may be regarded as having a volatile storage unit (2021) and a nonvolatile storage unit (2022). The computing unit may be composed of a CPU, a GPGPU, or any other computing device (TPU, ReRAM analog computing device, etc.), and the storage unit may be a storage medium such as a RAM, a cache memory, or a GPGPU memory as a volatile storage unit. It may be provided, and the nonvolatile memory is preferably composed of a storage medium including arbitrary electronic storage devices such as ROM, flash memory, and ReRAM, and arbitrary storage such as HDD, SSD, and optical storage devices. In addition, it may be possible to provide a keyboard, a mouse, a display section, etc. to enable operation, or to optionally provide other accompanying sections.

Then, the user information terminal can detect various information such as position, direction, inclination, etc. provided in the user terminal for information on position and/or direction and/or tilt and/or captured image and/or captured image features. Various detection units (1060) detect and acquire respective information. More specifically, GPS (including GNSS such as quasi-zenith satellites), accelerometer, gyro sensor, electronic compass (1064), etc., as well as imaging units and depth sensors (depth cameras). Based on the information, the user terminal device (1001) transmits the first location-linked information stored in the information distribution device of the information distribution system of the present invention, which includes position information, etc. (3005) of issuing a request for searching and/or extracting and/or sorting from.

Then, in the information distribution service using the information distribution apparatus according to the information distribution system of the present invention, a step of receiving first information such as position-related information of the user information terminal and search condition information from the user terminal based on the request. (3006) is executed. The step of receiving the first information searches and/or extracts and/or selects the second location-linked information necessary for the user information terminal from the first location-linked information collected in advance in the information distribution system. By doing so, the information necessary for providing to the user information terminal can be transmitted.

As described above, the information distribution device (and/or information distribution system software) used in the information distribution system as a service in the present invention to provide the second position-linked information to the user information terminal can be any user information terminal. (or software running on the terminal), information that can be retrieved from other information distribution devices connected in advance via communication lines by the information distribution system, and/or extraction and/or editing and / Or the first location-linked information group generation step used for searching from an arbitrary user terminal by listing (crawling or scraping) is performed by the information distribution device (2002) used in the information distribution system, and the first The user information terminal (1001) retrieves the second location-linked information group from the first location-linked information acquired by performing the location-linked information group generation step of , thereby generating the second location-linked information group for the user terminal. A step of acquiring a group of location-linked information is performed.

Then, it can be delivered to the storage unit of the information delivery device (2001) used in the information delivery system of the present invention regardless of the presence or absence of a request issued from the user information terminal by the steps of collecting, extracting, editing and listing. By saving the distribution information as the first location-linked information group obtained as a search result, it is possible to prepare information that is likely to be needed in advance even if there is no particular request from the user terminal. , In response to a request for location-related information and search target selection information provided from the user information terminal (of course, application software of the terminal may be used), information near the location of the user terminal A step (in 5102) of transmitting from the information distribution device side in the information distribution system of the present invention to the information terminal is executed.

As described above, the information distribution device used in the information distribution system of the present invention autonomously connects to other information distribution devices by using the communication unit provided in the information distribution device without any special request from the user information terminal. It is possible to collect and store information necessary for distribution. At this time, the purpose is to collect and store in advance the information necessary for the information distribution system to distribute the location-linked information to the user information terminal (1001), and search according to the geographical position to which the information belongs. A step of selecting conditions (4001, 5102) is performed, and a step of determining predetermined location-linked search conditions (4002) is performed in the above step. It should be noted that it is possible to collect information in advance without performing such special geographical attribution, or to configure it from geographical information included in information obtained based on general keyword search results.

Then, in order to collect information to be distributed, the information distribution terminal searches and collects information on the general network according to the conditions decided in the decision step (4002). Performed on the device. In response to the request of the search and collection step (4003), information as a search result is transmitted from an external information distribution device to the information distribution device of the present invention, and the information distribution device of the present invention receives necessary information (4004). is carried out.
The external information distribution device performs search processing (4101) in response to step (4003), which is a request of the information distribution device of the present invention.

After performing the step (4004) of receiving search target information as a position-linked search result, the information distribution apparatus of the present invention performs a step (4005) of classifying necessary information contents from the received search target information. . At this time, a database (DB: It is preferable to organize and store them in FIG. 6) or include them in the storage items.

Subsequently, a step (4006) is executed in which the information classified in the step (4005) is sorted and recorded in, for example, a DB (database) or the like according to required specifications. This step uses location information attached to the information as described above (any physical location-identifiable information such as latitude and longitude, postal code, address, telephone number, intra-company location-related information such as a delivery company). It is preferable to organize and store them in a database (DB) or the like, or include them in the storage items, and furthermore, the database update of the information distribution device from the user terminal is not packet communication of the smartphone, but Wi-Fi connection. Preferably.

The process of mashup processing as collection, extraction, editing, and listing of these information, more specifically, the information that is specifically handled when it is executed in step (4005), is handled by the information user using the user information terminal. Specific keywords that can be handled within (information related to service provision such as product information, store information, inventory information, screening performance information) and location information (latitude and longitude, address, postal code, telephone number, other places Identification code: For example, a step of extracting predetermined information in the form of searching for an area code used internally by a delivery company is executed.

In addition, the above-mentioned mashup process summarizes predetermined contents into a predetermined format according to the search conditions used in the scraping process and the user's personal information, and extracts information that is likely to give serendipity to the user who specified the search conditions. It includes information generation processing that can improve user convenience by generating, generating recommendation information and personalized information, and specific mashups and recommendations Various implementation methods such as collaborative filtering have been proposed. The server performs scraping and mashup processing, and the relay server of the present invention (relay information distribution device of the present invention) receives the mashed-up information generated by the client application and converts it into cached information for the client application. It is characterized by being implemented by

More specifically, if the information acquired in the above step is configured in XML, the information surrounded by predetermined tags such as <ADDRESS>JAPAN</ADDRESS> and the target selection information are used. By extracting the information necessary for the terminal user, it is determined whether the information is necessary for the user. Of course, this extraction process may be from HTML, described in any markup language, etc., or configured in a data description language (JSON, etc.) with a list-up structure like an appropriate database structure. Alternatively, by registering mashed-up information in the database in the user terminal (structure is shown in FIG. 6) or in the database of the information distribution device or information distribution relay device (structure is shown in FIG. 6), a new service without content can be created. It is also possible to sufficiently collect and provide necessary information to service users.

For example, when the user's information terminal is located in Tokyo, cases such as <ADDRESS>TOKYO</ADDRESS> or <address>Tokyo</address> are conceivable. Similarly, a postal code such as <ZIP>1060837</ZIP> may be used as an extraction condition, or a telephone number such as <TEL>03-XXXX-XXXX</TEL> may be used. As notations other than XML, information on location-linked search results is acquired in various formats such as JSON and CSV, and the necessary parts are extracted and organized (scraping and mashup) on the user information terminal and displayed. The information obtained from the distribution device may be edited into possible configurations, and a received information display processing step is performed to configure the information so that it can be displayed on the user information terminal based on the edited information. In this step, it is also possible to preferentially configure information only in the direction in which the imaging section of the display device of the user terminal device is directed.

For example, in this way, the information distribution system and the information distribution device used in the information distribution system of the present invention acquire the first location-linked information. Any information can be collected by any method as long as the information terminal of the present invention can retrieve and/or collect any information without being limited to such a method. In addition, autonomous collection on the user information terminal (1001) side is also possible. A configuration for efficiently collecting information required by the location-linked information distribution device may be realized by providing the information to the location-linked information distribution device.

Regarding the search target selection information as the search condition information described above, the search condition information is configured by selecting an appropriate icon or entering characters, and the search target selection information as the configured search condition information is used for the user information terminal. It can also be configured to perform information retrieval. In this case, it can be applied to a case where a search input screen or the like is provided from the information distribution device side like a Web browser.

Based on the search target selection information as the search condition information, the above-mentioned information terminal user service system generates the first information in the information distribution device (2001) arranged for the user information terminal from the information distribution device (2001). By providing the second position-linked information obtained from the position-linked information to the user information terminal (1001), it is possible to use it by performing it in advance in the service system for information terminal users instead of processing it in the user terminal. This has the effect of reducing the processing load in the user terminal and making it possible to provide CG configuration information for the user terminal as an efficient position-linked information terminal, thus improving convenience.

Then, based on the information acquired from the service system for user information terminal users, the user information terminal provides CG configuration information for the user terminal for providing information in the direction of the imaging unit based on the current position of the user information terminal. A CG image is composed appropriately from the user information terminal, and synthesized and displayed in the user information terminal based on the spatial coordinate system optimized according to the search result information and the position and direction of the user terminal. By synthesizing the CG image with the actual scenery captured by the imaging unit, it is possible to realize a CG image synthesis information providing service for the actual scenery by the user terminal using the information for the information terminal user service system according to the present invention.

Therefore, even in a situation where a person who wishes to register as a distributor in the service infrastructure does not pre-register in the information distribution system in such a new service infrastructure, the information distribution apparatus of the present invention can collect and provide information in advance. By doing so, it is possible to solve the fundamental problem of lack of information provided in the initial stage of the new information infrastructure.

Further, the present invention is defined as an information delivery system by cooperating with a plurality of information delivery devices to realize the position-linked information delivery of the present invention, and is realized by one information delivery device (2001). is assumed to be carried out in cooperative distributed processing by a group of information distribution devices, it is summarized as an example of an information distribution system. In the case of software that implements the present invention, it may be stored in a storage medium associated with a storage unit or the like. is where you configure scraping and mashups.

Based on another embodiment, the user information terminal (1001) that has acquired necessary information transmits the information acquired from the communication line by the communication unit (1050) in the user information terminal to the user information terminal (1001) Arbitrary information generated by processing within and/or collected, organized, and processed (crawling, scraping, etc.) from various communication lines in the information distribution device (2001) of the present invention in advance for user terminals An information generation step (5006) is performed to extract and generate necessary information from arbitrary information.

First, the user information processing terminal (FIG. 1-1001) handled by the user will be described. More specifically, this information processing terminal is not only an information processing device such as a mobile phone, a smart phone, a tablet, or a notebook PC, but also an HMD (head mounted display) connected to an appropriate information processing device. Alternatively, the HMD may have a built-in information processing device. And the position detection unit of the information terminal (GPS as GNSS, quasi-zenith satellite, DGPS, eLORAN, wireless LAN position estimation, arbitrary method that can acquire arbitrary position information including latitude and longitude such as acceleration sensor, etc. ) and a direction detection unit (an arbitrary method for detecting the direction, direction, and rotation status of the information terminal by an arbitrary method such as an electronic compass (1064) or an electronic gyro sensor (1062) by detecting geomagnetism). A CG image is generated by a CG (computer graphic) generation unit as a processing unit that performs image processing generation unit, etc. from information configured by the storage unit, and a captured image (such as a real landscape, outdoor・By combining the CG image generated by the image generation unit with the CG image generated by the image generation unit, a composite image of the captured image and the CG image linked to the terminal position, imaging direction, and captured image of the user information terminal is displayed on the information terminal display unit. By displaying , the invention relates to the operation of an information support device relating to city/indoor/outdoor use of a user information terminal.

Furthermore, these information terminals generally include electronic devices that can acquire images, such as imaging devices, three-dimensional object imaging devices that can capture three-dimensional objects by imaging the reflection of structure lights, depth cameras, depth sensors, infrared rays, etc. An imaging unit (1030) by an imaging device including imaging devices other than visible light such as an imaging device and an ultraviolet imaging device, and generally various sensors such as GPS (1061), gyro (1062), acceleration (1063), electronic It may be various detectors (1060) composed of any sensing device that can electronically collect information around various terminals such as a compass (1064), GNSS, ambient brightness, and a microphone for voice input. Then, based on the information obtained from these imaging devices and sensors, arithmetic processing is performed by the arithmetic unit (1010) in cooperation with the storage unit (1020).

In the storage unit (1020), storage using arbitrary storage media such as volatile memory (1021) such as RAM and cache memory, nonvolatile memory (1022) such as ROM, flash memory, ReRAM, and external storage such as HDD and SSD The imaging unit (1030) may be provided with ToF as a depth sensor, structure light, multi-camera, light coding, lidar, infrastructure depth, and the like.

Further, the display unit (1040) displays information on the display unit (1040) based on the information processed by the storage unit and the calculation unit described above, or displays the information on the image generation unit (1070) based on the processed information described above. ) to generate display information such as 3DCG, 2D graphics, and moving images, and display the visual information according to the user's terminal on the display unit (1040). Specifically, arbitrary display such as 3D display may be performed on the screen of the smartphone in the case of a smartphone, and 3D display in the case of an HMD (head mounted display).

Furthermore, the communication unit (1050) can cooperate and exchange information by communicating with other information processing devices regardless of whether they are wired or wireless.・Area Network), NFC (Near Field Communication), or a device that communicates with any short-range small device or a device that can communicate with a non-powered small device.

Further, the operation unit (1080) may be a touch panel, a cursor key, a physical keyboard, a touch pad, or a gesture recognition device such as an imaging or combination of operations on the touch pad. .

As the first step of the present invention, the user information terminal is equipped with a position detection unit and/or an acceleration output unit and/or a direction (an orientation (1064) by a geomagnetic sensor as an electronic compass, a gyro sensor (1062), etc.). A step (3001) of generating terminal position-related information by specifying the current position and direction (azimuth) of the information terminal is performed by various detection units (1060) such as a relative change in direction due to a change in direction. Here, the device that detects the direction may be configured to extract a more accurate direction by, for example, an electronic compass (1064) or cooperation between the electronic compass and the gyro (1062). The acceleration detection unit (1063) may be able to detect the tilt by detecting the downward direction of the user terminal from gravitational acceleration, or detect the moving speed of the user terminal from changes in acceleration.

Various detection units (1060) as the position detection unit described above, more specifically, position identification by GNSS, GPS (1061), A-GPS, WiFi, wireless communication base station (eg, mobile phone line base station), etc. Various methods can be used as long as the location can be specified in any way, such as latitude and longitude, address information based on latitude and longitude, postal code, telephone number (for example, area code). Conversely, the latitude and longitude may be calculated by extracting the address, postal code, etc. from the text information. Can tilt detection be realized by a gyro sensor (1062) or an acceleration sensor (1063)? etc. can be determined, and appropriate display can be performed on the display unit (1040) of the user information terminal (1001). Also, if the terminal seems to be moving, it is possible to continuously perform positional correction using an acceleration sensor. In this way, the user terminal first executes the step (3001) of acquiring the current location information.

Subsequently, the user information terminal performs a step (3002) of capturing an image of the surroundings of the terminal using the imaging unit (1030). Then, a feature extraction step (3003) is performed to acquire information about the captured image in the storage unit (1020) and extract features from the acquired image information through calculation in the calculation unit (1010).

When performing this feature extraction step (3003), if the image generation unit (1070) is a GPGPU (General-Purpose Computing on Graphics Processing Units) specification capable of performing computation and storage processing for purposes other than CG generation, the computation for feature extraction is performed. If so, it is also possible to perform feature extraction and the like using the storage section and the calculation section configured in the image generation section (1070). In addition, if it is not a GPGPU, it is possible to use a tensor processor dedicated to tensor processing, a matrix by analog arithmetic processing using ReRAM, etc., and a calculation unit for realizing neural network processing. Feature extraction as an application such as machine learning etc. may be performed, and depth information generated by depth image information obtained by a depth camera or the like may be included in this image feature extraction step.

Next, as a target selection information input step for selecting the target of desired information, input is performed by tapping a user interface such as an icon displayed on the screen from the operation unit (1080) combined with a display unit such as a touch screen. , enter appropriate text as a keyword, specify contraindications for the user (it may be religious (such as Haram) or constitutional (allergy)), hobby By specifying or doing, menus and icons for configuring personalized information composed of recommendations and curation based on information obtained from applications and servers can be selected and operated to search targets related to search conditions. A step (3004) is performed in which screening information is set. This operation unit (1080) may be a cursor operation key, a physical keyboard, or the like other than the touch screen. When the input from the operation unit is composed of a GUI and a touch screen, it is possible to configure instruction information based on the user's operation by performing operations such as text input, icon selection, and menu selection by tapping. can.

Note that the two steps of the imaging step (3002) and the imaging feature extraction step (3003), the sensor information acquisition step (3001), and the user's operation input step affect the overall processing even if the order is different. This image feature extraction step may include depth information generated by depth image information obtained by a depth camera or the like. The sequence of steps 3001 to 3004 in FIG. 3 corresponds to 5002 and 5003 in FIG.

The search of the present invention is carried out based on any combination of the three pieces of information of the location information of the user information terminal and/or the image feature information and/or the search target selection information acquired in the four steps in FIG. A step of providing location information and/or image feature information and/or search target selection information of a user information terminal to a general information providing/distributing device connected to a possible communication line, and requesting a search (3005 , 5004). These steps are the steps for transmitting information from the communication unit (1050) for requesting acquisition of location-linked information.

Then, based on the information provided by the user information terminal in the search request step (3005, 5004), the search request step (3005, 5004) is linked to the information distribution apparatus of the present invention connected by the communication line. A step (3101) is performed in which information retrieval is performed by an external information distribution device. Thereafter, the user information terminal carries out steps (3006, 5005) in which the communication unit (1050) receives the information group obtained according to the step (3101) performed by the information distribution device. These steps are steps for receiving location-linked information.

Furthermore, as information suitable for AR/MR, surrounding image characteristics may be provided to the user terminal. The image features may include depth information associated with imaging. Depth information may be included.

Then, in the information generation step (5006), any of the generated information is displayed on the display unit (1040) in the user information terminal (1001) based on search target selection information for selecting utilization target information. For this purpose, the information is processed by the calculation unit (1010), the storage unit (1020), and the display information generation unit (1070), the display information is determined, and the information to be provided to the user is sent to the user information terminal ( 1001), the calculation unit (1010), the storage unit (1020), and the display information generation unit (1070) are used to form CG image information.

Also, the description so far has been made on the premise that the user information terminal (1001) is a non-transmissive device. Transmissive devices do not necessarily require a surrounding image that serves as a background, but detect and calculate subtle deviations in the line of sight between the camera position line of sight and the line of sight in the transparent screen from the imaged surrounding image, and synthesize a CG image according to the deviation. is corrected and the CG image is projected onto a transmissive screen to realize CG image synthesis and display of a real scene that is not a captured image (3010). In that case, it may be stored in a storage medium associated with a storage unit or the like.

In the examples so far, there is a description of the flow of providing the image features acquired by the information terminal to the distribution device. image, and their image feature information), there is no need to send image features from the terminal device to the distribution device, but if necessary, normal captured images, invisible light images, depth images, and their image features can be further added. By adding it, it may be provided to the more convenient relay information distribution apparatus of the present invention.

Furthermore, as information suitable for AR/MR, the user terminal may be provided with image characteristics of the surroundings. Depth information accompanying imaging may be added to the image features, and depth information generated by depth image information obtained by a depth camera or the like may be included in the image feature extraction step. good too.

At this time, the information as CG images configured and provided by the calculation unit, storage unit, and display processing unit in the user information terminal (1001) is converted into text, 2D images, 3DCG images, moving images, and further from text to speech. It may be any information provided to the user's audiovisual, such as including music and voice using a voice information generation unit that performs conversion of , or when using the information delivery device (2001), delivery The above-described information may be configured within the device.

As described above, position-linked information that can be provided by any information providing method. By displaying the provided location-linked information or using audio signals that sound like they are being heard from the center in stereo, the user of the user information terminal is linked to the current location and direction of the terminal. It is possible to acquire, generate, and output image, video, and audio information. Also, if it is outside the screen, the direction of the position-linked information that can be the target may be presented by an arrow or the like.

In the examples so far, the shape of the user information terminal is not particularly specified, but the structure of conventional smartphones, smart glasses, HMD (head mounted display), AR (augmented reality), MR (mixed reality) A glasses-type device such as glasses may be used. In addition, neither the drawings nor the examples clearly state that there is a voice output unit, but the voice output unit is omitted because it is always installed in many of the current general information terminals, and AR / as an image assistant In addition to MR, it may be combined with a position-linked voice support system, and augmented/detailed display, voice guidance, etc., by AR/MR of the target gazed at by line-of-sight detection may be used.

Furthermore, in the case of glasses-type transmissive devices such as smart glasses, the background image is transmitted without arithmetically synthesizing the captured CG image for synthesis with the captured image of the real scene by information processing. The configuration may be such that the image for synthesis can be displayed.

More specifically, the composite display contents of the actual scenery and the CG image displayed in the display unit (1040) are obtained by extracting the latitude and longitude of the display target based on the position-linked information obtained from the latitude and longitude of the user information terminal. , determine the direction to the user terminal from the extracted latitude and longitude, determine whether the direction of the imaging unit of the user terminal matches the direction of the target display object, and determine whether the determined direction of the imaging unit of the user terminal User location-linked information for the user information terminal (1001) in which the actual scenery and CG image suitable for the user terminal are synthesized by changing the size and display format of the display object according to the direction and the distance of the display object. is displayed, it will be executed. As for the method of implementation, more methods have been presented in the past, so by combining them appropriately, it is possible to realize a more reliable location-linked information provision service.

A more specific linking process between the user information terminal, the information distributing apparatus of the present invention, and the general distributing apparatus in the overall invention will be described with reference to FIG. As in other embodiments, the user information terminals (5001, 1001) can be used for various detections of the user information terminals (5001, 1001) such as captured images, captured image features, GPS information, tilt information, compass direction, acceleration information, etc. Information related to retrieval is acquired by input operations from the unit (1060), the imaging unit (1030), and the user's operation unit (1080).

A step (5002) of acquiring various kinds of search condition information necessary for making a search request based on the information related to the search obtained by a series of user information terminals (5001, 1001) is performed. A search condition setting step (5003) is performed to set conditions in accordance with the search request destination.

Subsequently, based on the search conditions set here, a search processing request step (5004) for making a search request to the information distribution apparatus of the present invention in the case of FIG. , 2001) receives the search request, searches for the first position-linked information in the information distribution device (5101, 2001) of the present invention, and if there is no appropriate search result, the external general information distribution device A further step of requesting a search for external location-linked information corresponding to the first location-linked information (5112) is performed, and the external general distribution device performs the step of requesting a search for the external location-linked information A step (5202) of searching for information in an external general distribution device according to the request of the above is executed, and a step (5203) of returning the obtained search result to the information distribution device (5101, 2001) of the present invention is executed. Then, the distribution device (5101, 2001) of the present invention temporarily additionally saves the position-linked information obtained as a reply result to the first position-linked information, and the user information terminal ( 5001, 1001) may receive the second location-linked information as a search result by executing the search result acquisition step (5005).

In FIG. 5, only the information distribution device (5101, 2001) of the present invention is targeted. the general distribution device receives the request (5202), the general distribution device performs the search result reply step (5203), and the user information terminal (5001) performs the search result acquisition step (5005). You may receive search results based on your search criteria.

Then, the user information terminal (5001, 1001) executes a search result content extraction step (5006) as a step of extracting necessary information from the obtained search result information. The information extracted in the extraction step (5006) and the state of the user information terminal (for example, the information specified in the search condition, the captured image obtained by the imaging device, the feature information obtained from the captured image, the terminal tilt information, the terminal acceleration information, A step (5007) of generating CG corresponding to the extracted content is performed according to terminal gyro information, terminal GPS information, terminal compass information, etc.), and the generated CG image is captured by the imaging unit of the user information terminal. A step (5008) of synthesizing the CG image with the image is performed, and as a result, a step (5009) of displaying a synthesized image in which the CG is synthesized with the surrounding image is performed on the display unit of the user information terminal.

As a result, on the user information terminal (5001, 1001), it becomes possible to synthesize and display a CG image based on the information obtained from the general information distribution device and the information obtained from the dedicated information distribution device with the surrounding image. To enable a user information end user to enjoy sufficient service even if a distribution applicant does not have to perform distribution information registration or distribution applicant registration in a dedicated information distribution system.

Also, if the information distribution device (5101, 2001) of the present invention collects, classifies, and stores (saves) information from any general information distribution device as in other embodiments, (5001, 1001) makes a search request to the information distribution device (5101, 2001) of the present invention, whereby the information distribution device (5101, 2001) of the present invention searches the storage unit in the device search processing step ( 5102) is performed. At this time, in step (5102), the information can be registered in the database of the delivery device.

The search processing step (5102) searches for information related to the location information of the user's information terminal with respect to the information stored in the storage unit of the information distribution device (5101, 2001) of the present invention based on the received search conditions. If no suitable search result is found in the internal search, the external general information distribution device is searched via the communication line as in other embodiments. A location-linked information general search step (5122) is performed.

A step (5123) of acquiring position-linked information search results based on the first group of position-linked information obtained from the information distribution device (5101, 2001) of the present invention by the above-described search method is performed. / Or using the first position-linked information group generated at the time of request for search, external general information distribution as return information for the subsequent search processing step (5102) for the user terminal (5001, 1001) A search result can be configured as second location-linked information including information from the device.

Then, a step (5103) of temporarily storing (temporarily storing) the position-linked search results as the first group of position-linked information obtained from an external general information distribution device as reply information in the search processing step (5102). is implemented, and the information distribution device (5101, 2001) of the present invention retrieves search result information using the temporarily stored (temporarily stored) first location-linked information group if necessary in response to access from another terminal. The problem is solved by providing it to arbitrary user information terminals (5001, 1001) as the second location-linked information group. If the temporarily stored information is frequently used, the storage state of the information may be changed to long-term storage. In addition, in the case of software that implements the present invention, it may be stored in a storage medium associated with a storage unit or the like.

Next, the first location-linked information and the second location-linked information will be described with reference to FIG. First, the first location-linked information and the first location-linked information group and the second location-linked information and the second location-linked information group are stored in the user information terminal (5001, 1001) according to another embodiment of the present invention. This is a concept required when using the information distribution device (5101, 2001) of the present invention in .

The second location-linked information and the second location-linked information group are information obtained as a result of processing such as search and/or extraction and/or selection from the first location-linked information and the first location-linked information group. Since it refers to a group, it may be considered that the requirement is included in the first location-linked information and the first location-linked information group. Furthermore, with the progress of steps (5004, 5005, 5102), the second position-linked information group collected from (5201) is stored in the storage unit (FIG. 2) of the information distribution device of (5101) for the user terminal. may be recorded as information of the database (FIG. 6) structure.

The first location-linked information and the first location-linked information group are roughly composed of the following information. More specifically, it has a database table structure as shown in FIG. 6, and has position-related information such as latitude, longitude, postal code, telephone number, and address as first search keys. Next, it has as a key information indicating attributes related to location information, such as whether an object existing at that location is a store, a house, a school, or a park, and then has preliminary classification information of attribute information related to location information as a key. These pieces of preliminary information may be layered in multiple layers or configured to have multiple items in the same layer.

More specifically, if it is a store, it is a "restaurant", "electronic store", "sporting goods store", "sports gym", etc. They are "elementary school", "junior high school", "high school", "university", etc., and item information is attached based on the classification related to each attribute. The item information may be of any configuration as long as it is in a form that can be discriminated whether it is numerical information or text information.

The first position-linked information configured in this way is appropriately registered in the single information distribution device of the present invention or in a plurality of linked distribution systems to constitute the position-linked information distribution device (5101, 2001) as a system. Then, the user information terminal (5001, 1001) uses the information acquired and generated in the terminal as a key to make a search request to the first location-linked information group stored in the location-linked information distribution device (5101, 2001). I do. As a result of a request such as a search request to this position-linked information distribution device (5101, 2001), the position-linked information distribution device (5101, 2001) of the present invention sends the second group of position-linked information to the user information terminal (5001, 1001). ).

As a result, the user information terminals (5001, 1001) that have received the second group of location-linked information appropriately change the user information terminal (5001) based on the received second location-linked information as in other embodiments. , 1001), information can be provided to the visual sense of the user in the form of appropriately synthesizing an image with the actual scenery or indoor/outdoor information. In addition, in the case of software that implements the present invention, it may be stored in a storage medium associated with a storage unit or the like.

Furthermore, based on the collection, classification, analysis, and editing of location-linked information in other embodiments, a location-linked information database (FIG. 6) can be appropriately associated with locations as an example of the information configuration of records in the database according to predetermined specifications. (The detailed numerical values in each field are rough values without any particular intention, so there may be cases where values that do not match the original latitude and longitude are described). Information to be classified, analyzed, and stored in the storage unit (2020) of the location-linked information distribution device (2001) as a storage location of this information using AI methods such as statistical methods, machine learning methods, and deep learning. It generates a group of information such as variables, arrays, matrices, and tensors that are indicators of the physical position, and converts physical position-related information into arithmetic elements such as latitude and longitude, appropriate road route distance information by foot or vehicle, and appropriate public Fields such as information providing stores that process information provided by users and user intentions inferred from operations as information that can be evaluated by converting it into transportation network distance information, etc. It can also be configured as a position-linked information delivery device (2001) that appropriately provides information for service users by appropriately associating information.

More specifically, information search in the user information terminal (1001) if it is information to be provided by the user information terminal, or in the position-linked information distribution device (2001) as the present invention in other embodiments. By using the statistical processing, machine learning, and deep learning described above in the steps of collecting, classifying, analyzing, editing, and extracting, the optimization of the information provided to the user of the user information terminal (1001) can be performed by a machine. It is intended to be realized by learning and AI. By optimizing the information provided to these users, it is possible to realize a curation service for location-linked information that is effective for the user's aha experience and serendipity. In addition to sites, it is also possible to extract and optimize based on information linked to location from restaurants, lodging, product review sites, etc., and provide them to users.

Further, although not described in FIG. 6, if described as an example, image information and/or image characteristics and/or information indicating the location of image information (for example, file name, URL, etc.) and/or information indicating the location of the image feature information (for example, file name, URL, etc.) may be used by analyzing, recording, and storing in association with position-linked information. It is also possible to provide more efficient location-linked information to service users from personal information terminals and location-linked information distribution devices. Of course, if location information and image information and/or image feature information are associated with each other by an arbitrary method, the implementation method will be a special specification or implementation method as long as it is appropriately associated information rather than information in the same record. It doesn't have to be.

At this time, image information and image feature information classified by position linkage are classified according to direction information obtained from the user information terminal, and AI such as machine learning and deep learning based on direction information is used to improve processing efficiency and optimization. By doing so, the information delivery environment of the present invention can be configured so that it can be applied to performance improvement for providing position-linked information more appropriately according to orientation, such as direction information and tilt information. More specifically, position information and/or inclination information and/or direction information and/or movement speed information and/or image information and/or image feature information obtained from the above-mentioned user information terminal or any information distribution device Using SLAM (Simultaneous Localization and Mapping), which associates image information with location information, etc., forms a three-dimensional map by AI processing, etc., and uses a system that assists the self-position estimation of the user terminal by SLAM interlocking An AR/MR service or the like may be configured.

Also, in the user information terminal (1001), the information collected by the user information terminal itself is stored in the storage unit (1020) of the user information terminal (1001), and the position-linked information delivery device (2001) stores the position-linked information in the user information terminal (1001). A database having the same configuration as the information database (FIG. 6) is built in the storage unit (1020) of the user information terminal (1001), and part of the information is shared or used with the location-linked information distribution device (2001). By uploading from the personal information terminal (1001) to the database (FIG. 6) of the location-linked information distribution device (2001), a series of service systems may be operated more efficiently and optimally.

Furthermore, the position-linked information distribution device (2001) and the user information terminal (1001) in the present invention each have a communication unit, and this communication unit can be wired LAN, wired WAN, It is also possible to use various arbitrary communication techniques such as communication carrier lines, in-store wireless LANs, wireless LANs via PANs (Personal Area Networks), and information communication via communication carrier lines. In addition, in the case of software that implements the present invention, it may be stored in a storage medium associated with a storage unit or the like.

In the device and/or system and/or software/program as the above embodiment, the position information and/or direction information and/or tilt information and/or image information and/or image feature information and/or collected by the information delivery device Alternatively, any text information collected based on the location of the user information terminal is selected by any method, combined, hashed or appropriately encrypted, and stored in any ledger information (distributed ledger information etc.), the information distribution system and user information terminal of the present invention may be configured so that the information reliability and information management status can be grasped by blockchain etc., and the user Calculation resources required for distributed computation such as blockchain may be secured using calculation resources of terminals and distribution systems. In the case of software that implements the present invention, it may be stored in a storage medium associated with a storage unit or the like, and position-linked data related to the installation location of the distribution device may be generated by parallel distributed processing for wide-area deployment. An appropriate location-linked service may be realized through collection and parallel distributed processing.

Services provided by these user information terminals, information distribution devices, and information distribution systems are based on location, such as referring to past historical information, games linked to location information related to content such as arbitrary copyrighted works, etc. Provide information as a service, provide real estate trading information and open house information linked to location, provide restaurant evaluation and vacant seat status linked to location, inventory information and It is possible to provide services by associating any information linked to location, such as bargain sale information, sales fair information, sales event information, price information, seminar information by any operator, and event venue information. It is also possible to introduce related contents or to provide information related to historical locations.

According to the present invention, CG information required for display on the user terminal is generated by scraping processing and mashup processing linked to the position in this way, and by synthesizing it with the actual scenery, existing content information on the Internet can be used as content information. For information spaces such as new outdoors where content is scarce, servers can be installed as dedicated distribution devices, or an information distribution environment using a dedicated markup language. It is possible to suddenly provide sufficient contents at the world level without preparing

There are existing markup languages such as ARML and KML as XML for AR space. is not popular because it uses these dedicated markup languages. Furthermore, the display information may be easily handled as a processing system by converting it to ARML or KML at the time of mashup of the present invention, or simple notation for AR using markdown notation may be implemented. Templates and display specifications at the time of computer graphic generation may be defined based on such information so that they can be displayed on the terminal. By registering the mashed-up information in the database of the information distribution device and/or the information distribution relay device, it is possible to sufficiently collect and provide necessary information to users in a new service without content.

To summarize the processing procedure as an embodiment of the client side of the user information terminal device (1001) of the present invention, Based on the position information and/or direction information and/or instruction information obtained from the operation unit (1080) (which may be configured as text information or selected icon selection information input from 1080), the user information Via the communication unit (1050) of the terminal device (1001), the first information is sent from any information distribution device (for example, a web search site, SNS site, any evaluation site such as a hotel or gourmet) and/or an information distribution device system. step of collecting (3005, 3006), and the user information for the first information based on the position information and the instruction information of the user information terminal device (1001) from the collected first information; A step (3007) of performing a scraping process using the arithmetic unit (1010) and the storage unit (1020) of the terminal device (1001) to generate the second information; Based on the positional information and/or the instruction information, the second information is subjected to mashup processing using the computing unit (1010) and the storage unit (1020) of the user information terminal device (1001) to obtain the third information. a step of generating (3008), and based on the third information and the imaging direction of the user information terminal device (1001), the user information terminal a step of synthesizing and displaying the computer graphic image information generated by the image generation unit (1070) of the information terminal device (1001) and the captured image information obtained by the imaging unit (1030) of the user information terminal device (1001); (3009, 3010) is configured as a user information terminal device (1001) characterized by implementing (3009, 3010), and can also be implemented as a program.

The first information is the information that serves as the selection criteria for generating the second information in the scraping process that extracts the necessary information from the search conditions requested to the external distribution device and the acquired search results, and the information that is displayed after the selection. The condition setting information of the third information mashed up together is used as a reference for displaying a CG image on the display unit of the user information terminal based on the third information. At this time, by registering the mashed-up information in the database in the user terminal (the structure is shown in Fig. 6) or in the database of the information distribution device or the information distribution relay device (the structure is shown in Fig. 6), it can be used in new services without content. It is also possible to collect and provide sufficient information necessary for the person.

The second information is information organized and processed based on the first condition after processing for collecting unprocessed information from an external information distribution device or the like based on the condition of the first information, generated. As a specific example, an HTML file is extracted from a predetermined Web server, further access is made to the link destination of the file to collect more detailed information, and address information is extracted from the collected HTML file. Then, by searching for the address information, the latitude/longitude information reversely extracted from the address information and the link address such as the predetermined URL are set as the second information, and such processing is repeated. In this way, the second information itself may be configured as an information group having a list structure, or furthermore, an HTML title tag or the like may be extracted and associated with a link to form a list structure, or HTML By associating the file itself as a list structure, the second information has a list structure such as a link to the display content in the third information generated by the mashup process, a file name, and direct HTML content. Information may be generated. Then, based on the list structure as described above, a database (FIG. 6) is constructed and appropriately stored in the storage unit. Stored in non-volatile storage. The non-volatile storage unit is illustrated as an abstract concept, and more specifically, any storage medium can be used as a form of implementation, including rewritable storage such as HDD and SSD and non-rewritable storage such as ROM. It may be, and its implementation is not particularly constrained.

The third information is information configured based on the second information, various conditions for synthesizing with the photographed image are added based on the first information, and any direction and/or relative coordinates are added in the display. Information according to the distance and information necessary for display (for example, 3D model information, text information for display, voice information, coloring information, CSS information, and link information and files that describe the relevance of those information information, URL information, etc.) is configured.

The captured image information is configured as image information obtained by the imaging unit (1030) of the user information terminal. Computer graphic (CG) image information is configured as image information generated by an image generation unit (1070) to be synthesized on the captured image information based on the third information. In addition, the scraping process extracts specific keywords and HTML tags, arranges them in a predetermined format and stores them in the extraction process, and the mashup process organizes the collected information in a form that is easy to provide to users. Shows processing for designating a display format and enabling display as organized information. Providing such processing in line with user preference analysis is sometimes called curation. At this time, by registering the mashed-up information in the database in the user terminal (the structure is shown in Fig. 6) or in the database of the information distribution device or the information distribution relay device (the structure is shown in Fig. 6), it can be used in new services without content. It is also possible to collect and provide sufficient information necessary for the person.

In other words, based on position information and/or direction information obtained from various detection units of the user information terminal device and/or instruction information obtained from the operation unit, the user information terminal device a step of collecting first information from an arbitrary information distribution device and/or an information distribution device system via a communication unit; and location information and/or instruction information of the user information terminal device from the collected first information. a step of scraping the first information using the computing unit and the storage unit of the user information terminal device to generate the second information based on the above; a step of performing a mashup process on the second information using a computing unit and a storage unit of the user information terminal device based on the positional information and/or the instruction information to generate the third information; Based on the information and the imaging direction of the user information terminal device, the display unit of the user information terminal device displays the computer graphic image information generated by the drawing calculation unit of the user information terminal device and the user information. It can also be said to be a user information terminal device characterized by carrying out a step of synthesizing and displaying captured image information obtained by an imaging unit of the terminal device. Furthermore, whether it is a database configured in a smartphone as a user terminal or a database of a distribution device, it is preferable that each database is updated not by packet communication of the smartphone but by Wi-Fi connection.

The information scraped and mashed up from these user terminal devices is acquired by the imaging unit when the AR/MR information is displayed at an appropriate three-dimensional position accompanying the captured image displayed in the user terminal display unit. SLAM is performed by accumulating the image feature information extracted from the imaging information in the database of FIG. Information, date and time information, celestial body information such as the sun and the moon may be recorded in association with each other, and based on such information, the efficiency and quality of scraping and mashup may be improved. In the corresponding database, as well as the database of the user terminal, the number of items may be increased to improve the convenience of the user. User convenience may be improved by recording information in a database for SLAM.

According to the present invention, in a new network service infrastructure, the information on the existing network is used, and the new network service infrastructure is suddenly developed on a global level without preparing the information specifically configured for the new technology. will be able to provide sufficient content to Furthermore, according to the present invention, it is necessary to urge users who wish to use the information support system to register using a conventionally complicated processing system, and to urge system users who wish to provide information in the position-linked information support service system to register. , and a system that generally supports location-linked information by synthesizing actual images and CG images by registering and setting them, autonomously collects location-linked information using the method of the present invention By using information terminals and distribution systems, it is possible to provide location-linked information support for the user's information terminal by eliminating the need to perform complicated user registration procedures as in the past. It is possible to build an AR/MR browser service system application software environment that makes it possible to realize an appropriate and highly convenient position-linked information support service system according to the direction.

Furthermore, in addition to the possibilities described in the previous section, when providing a completely new service infrastructure, it is possible to build an information distribution infrastructure without newly registering users on the distribution side and on the receiving side. By resolving the problem of which comes first, the chicken (service infrastructure) or the egg (content within the service infrastructure), and by accelerating the spread of new service infrastructure, we can dramatically increase the availability of augmented reality in the information industry. be able to improve.

1001 User information terminal device 1010 Calculation unit 1020 Storage unit 1030 Imaging unit 1040 Display unit 1050 Communication unit 1060 Various detection units 1061 GPS
1062 gyro sensor 1063 acceleration sensor 1064 electronic compass 1070 image generator 1080 operation unit

2001 Information distribution device 2010 Calculation unit 2020 Storage unit 2030 Communication unit

3001 Various detection step 3002 Imaging step 3003 Image feature extraction step 3004 Search condition setting operation step 3005 Search processing request step 3006 Search result acquisition step 3007 Search result content extraction step 3008 CG display content generation step 3009 Combining CG with captured image Step 3010 to display the composite image

4001 Location-linked storage target selection step 4002 Location-linked search condition determination step 4003 Location-linked search request step 4004 Location-linked search acquisition step 4005 Location-linked search result content classification step 4006 Classified search result DB storage step

5001 Processing system 5101 of the information processing terminal of the present invention Processing system 5201 of the information distribution apparatus of the present invention Processing system 5002 of a general information distribution apparatus Various sensors/imaging and their feature extraction step 5003 of the information terminal Search condition setting step 5004 Information search process request step 5005 related to the position of the information terminal Information search result acquisition step 5006 related to the position of the information terminal Content extraction step 5007 from the search result suitable for the information terminal Step 5008 of generating CG for information terminal Step 5009 of synthesizing the generated CG with the background image according to the situation of the information terminal Step 5102 of displaying the synthesized image on the information terminal Step 5102 of searching in the information distribution device Step 5111 Position within the information distribution device Linked information search step 5112 External search step 5113 when suitable information is not found in the information distribution device Search result return step 5103 Information obtained by external search Temporary storage (cache) processing step 5202 General Search processing to external distribution device 5203 Information provision to this information distribution device of search results obtained from a general external distribution device

Claims (3)

Necessary information acquisition condition information based on position information and direction information obtained from various detection units of the user information terminal device and/or instruction information obtained from the operation unit,
Via the communication unit of the user information terminal device,

a step in which the mediation information distribution device receives first information as the necessary information acquisition condition information;

Based on said first information,
Arbitrary information generated in the user information terminal device,
By uploading from the user information terminal to the mediation information distribution device,
While performing the step of recording in the storage unit of the mediation information distribution device,

In response to a request from a user information terminal device other than the user information terminal device for the arbitrary information uploaded to the database,
An intermediation information distribution device, characterized by carrying out the step of providing to said other user information terminal device.
Necessary information acquisition condition information based on position information and direction information obtained from various detection units of the user information terminal device and/or instruction information obtained from the operation unit,
Via the communication unit of the user information terminal device,

a step in which the mediation information distribution device receives first information as the necessary information acquisition condition information;

Based on said first information,
Arbitrary information generated in the user information terminal device,
By uploading from the user information terminal to the mediation information distribution device,
While performing the step of recording in the storage unit of the mediation information distribution device,

In response to a request from a user information terminal device other than the user information terminal device for the arbitrary information uploaded to the database,
A program characterized by executing a step of providing to the other user information terminal device.
Necessary information acquisition condition information based on position information and direction information obtained from various detection units of the user information terminal device and/or instruction information obtained from the operation unit,
Via the communication unit of the user information terminal device,

a step in which the mediation information distribution device receives first information as the necessary information acquisition condition information;

Based on said first information,
Arbitrary information generated in the user information terminal device,
By uploading from the user information terminal to the mediation information distribution device,
While performing the step of recording in the storage unit of the mediation information distribution device,

In response to a request from a user information terminal device other than the user information terminal device for the arbitrary information uploaded to the database,
A storage medium recording a program characterized by executing the step of providing to the other user information terminal device.

Images