JP7365145B2 - Information provision device, information provision method, and program - Google Patents

Description

The present invention relates to an information providing device, an information providing method, and a program.

Conventionally, feature images that analyze feature image data at the same location, determine whether the content of the feature image data differs, and notify changes in the content of feature image data at the same location based on the determination results. A data change notification device has been disclosed (for example, see Patent Document 1).

Patent No. 4895123

However, the conventional technology notifies users of changes in the content of feature image data, and may not be able to provide useful information to users.

The present invention has been made in consideration of such circumstances, and one of the objects is to provide an information providing device, an information processing method, and a program that can provide more useful information to users. do.

One aspect of the present invention includes an acquisition unit that acquires, from another device having a position identification unit and an imaging unit, an image in which the position information specified by the position identification unit is associated and a landscape is captured by the imaging unit. , extracting a semantic region expressing a specific meaning related to a feature included in the landscape from the image acquired by the acquisition unit, and selecting one of the images acquired by the acquisition unit from among a plurality of specific models. a recognition unit that recognizes a specific meaning within the semantic area using a specific model applied to a landscape; and a recognition unit that recognizes a specific meaning within the semantic area using a specific model that is applied to a landscape; The information providing device includes a providing unit that provides the meaning of the information to the user.

According to one aspect of the present invention, more useful information can be provided to the user.

1 is a diagram showing an example of a functional configuration of an information processing system 1. FIG. 3 is a sequence diagram illustrating the flow of processing executed by the imaging device 20 and the information processing device 100. FIG. 4 is a diagram showing an example of the contents of reference information 44. FIG. 4 is a diagram showing an example of the contents of correspondence information 42. FIG. 4 is a diagram showing an example of updated reference information 44. FIG. 5 is a flowchart illustrating an example of the flow of processing executed by the imaging device 20. FIG. 2 is a flowchart illustrating an example of the flow of specific processing executed by the information processing device 100. FIG. FIG. 3 is a diagram illustrating an example of how a rectangle is derived in a target image. 7 is a diagram showing an example of how a rectangular image IM is input to a specific model 126, and the specific model 126 outputs the price of regular gasoline. FIG. 3 is a diagram showing an example of the content of recognition information 128. FIG. 3 is a diagram showing an example of an image displayed on the terminal device 300. FIG. It is a figure showing an example of functional composition of imaging device 20A of a 2nd embodiment. It is a flowchart which shows an example of the flow of the process performed by 20 A of imaging devices. FIG. 3 is a conceptual diagram of a process in which the imaging device 20A determines whether an image is to be transmitted or an image not to be transmitted. It is a flowchart which shows an example of the flow of processing performed by 20A of imaging devices of 3rd Embodiment. FIG. 6 is a diagram for explaining a process in which images before and after refueling are specified. FIG. 6 is a diagram for explaining a process in which images before and after the vehicle is parked in a parking lot are identified. 12 is a diagram illustrating an example of the contents of a plurality of specific models 126. FIG. 12 is a flowchart illustrating an example of the flow of processing executed by the information processing apparatus 100 of the fourth embodiment. 12 is a diagram illustrating an example of the contents of a plurality of specific models 126. FIG. 12 is a flowchart illustrating an example of the flow of processing executed by the information processing apparatus 100 of the fifth embodiment. It is a figure showing an example of functional composition of information processing system 1A of a 6th embodiment. 4 is a diagram showing an example of the contents of learning data 412. FIG. 4 is a diagram showing another example of the content of learning data 412. FIG.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of an information providing apparatus, an information providing method, and a program of the present invention will be described with reference to the drawings.

[Summary (Part 1)]
The information processing device of the embodiment is realized by one or more processors. The information processing device (server) includes a communication unit that communicates with other devices, an imaging control unit that causes an imaging unit to image a landscape based on reference information indicating a standard for imaging a landscape, and a position acquisition unit that acquires a position. An image of the landscape associated with location information is acquired from a terminal device including an acquisition unit, the meaning of the captured landscape is recognized from the acquired image, and the reference information is transmitted to the terminal device. and causes the terminal device to transmit the image.

The "terminal device (edge)" may be a terminal device mounted on a vehicle, or may be a portable terminal device that can be attached to a vehicle. For example, the terminal device is a drive recorder, a smartphone, or the like. Further, the terminal device may acquire an image captured by an image capturing unit mounted on the vehicle, or may acquire a determination result from a position identifying unit mounted on the vehicle. In the following description, the terminal device will be explained as a drive recorder.

"Reference information" includes, for example, information indicating the location at which a landscape is imaged, information based on the location where a predetermined event has occurred, information based on the location of a predetermined feature, and information indicating vehicle behavior (such as speed). information indicating the state of the vehicle (for example, a stopped state or a refueling state). The predetermined feature is, for example, a gas station or a parking lot. The "meaning expressed by a landscape" is, for example, a feature, a character, a symbol, etc. that can be recognized from the landscape.

[Summary (Part 2)]
The information processing device of the embodiment is realized by one or more processors. The information processing device includes a communication unit that communicates with other devices, an imaging control unit that causes an imaging unit that captures a landscape to capture an image, a position acquisition unit that acquires a position, and transmits information based on the captured image and model information. An image in which the landscape associated with position information is captured is acquired from a terminal device including a determination unit that determines the image, and from the acquired image, a meaning expressed by the captured landscape is recognized, and the Model information is transmitted to the terminal device, and the terminal device is caused to transmit the image.

The "image of a landscape" is an image of a predetermined feature such as a gas station or a parking lot. "Model information" is information that, when a captured image is input to the imaging unit, outputs information indicating whether the image is a transmission target image. The model information is generated by machine learning of learning data in which an image to be transmitted is associated with information indicating that the image is to be transmitted.

[Overview (Part 3)]
The information processing device (an example of an “information providing device”) of the embodiment is realized by one or more processors. The information processing device acquires, from another device having a position specifying unit and an imaging unit, an image in which the position information specified by the position specifying unit is associated and a scenery is captured by the image capturing unit, and from the acquired image, Extracting a semantic region expressing a specific meaning related to a feature included in a landscape, recognizing the specific meaning within the semantic region, and associating it with a position where an image including the semantic region was captured, Providing the user with a specific recognized meaning.

A "local feature" includes, for example, one or both of a gas station and a parking lot. The "semantic area" is, for example, a gas station sign, a parking sign, or the like. The "specific meaning" is, for example, information indicating the price of fuel on a gas station sign or the availability of a parking sign. "Providing the user with the recognized specific meaning in association with the location" means, for example, providing the user with the recognized specific meaning in association with the location or things at that location (or in the vicinity). It is to provide. Furthermore, "providing a specific meaning to a user" means, for example, providing a specific meaning to a terminal device held by a user, or specifying a meaning in a database in order to provide a specific meaning to a terminal device held by a user. The purpose of this is to associate the meanings of the objects with the features and to memorize them.

Further, the information processing device recognizes a specific meaning within the semantic area using a specific model that is applied to each position where the image acquired by the acquisition unit is captured, from among the plurality of specific models. Good too. The specific model applied to each location is generated by machine learning of learning data in which a learning image taken at the location and a specific meaning in the semantic area in the learning image are associated with each other. It is what was done.

Further, the information processing device may recognize the specific meaning within the meaning area using a specific model applied to the scenery of the image acquired by the acquisition unit from among the plurality of specific models. The specific model applied to the landscape of the image is obtained by machine learning learning data in which a learning image in which the landscape is captured and a specific meaning in the semantic area in the learning image are associated with each other. It was generated.

Further, the information processing device may recognize a specific meaning within the meaning area by using a specific model applied to a signboard included in the image acquired by the acquisition unit from among a plurality of specific models. . The specific model to which the signboard is applied is generated by machine learning of learning data in which a learning image of the signboard and a specific meaning indicated by the signboard are associated with each other.

Further, the information processing device extracts a semantic region representing the meaning of the specific thing in the image from the image including the specific thing acquired by the acquisition unit at a first time, and the information processing device extracts a semantic region expressing the meaning of the specific thing in the image, and A semantic region expressing the meaning of the specific thing in the image is extracted from the image including the specific thing acquired by the above-mentioned image, and information on the semantic region of the image acquired by the acquisition unit at a first time and a second A specific meaning within the semantic area may be recognized based on a difference between the semantic area and the information acquired by the acquisition unit at the time. The specific meaning is, for example, the price of fuel for a vehicle or the availability of parking spaces.

<First embodiment>
[composition]
FIG. 1 is a diagram showing an example of the functional configuration of an information processing system 1. As shown in FIG. The information processing system 1 includes, for example, a vehicle 10, an information processing device 100, a news server 200, and a terminal device 300. These devices communicate with each other via the network NW. The network NW includes, for example, a WAN (Wide Area Network), a LAN (Local Area Network), the Internet, a dedicated line, a wireless base station, a provider, and the like. Note that some or all of the configurations or functions included in the information processing device 100 may be included in the vehicle 10. Further, some or all of the functions or configurations included in the vehicle 10 may be included in the information processing device 100.

[vehicle]
An imaging device 20 is mounted on the vehicle 10. The imaging device 20 includes, for example, a communication section 22, an imaging section 24, a position specifying section 26, an imaging control section 28, a transmission control section 30, and a storage section 40. The imaging control unit 28 and the transmission control unit 30 are realized by a hardware processor such as a CPU (Central Processing Unit) executing an imaging application 46 stored in the storage unit 40. The imaging control unit 28 and the transmission control unit 30 cooperate with the information processing device 100 to cause the imaging unit 24 to capture an image and to transmit the captured image to the information processing device 100. The storage unit 40 is realized by, for example, a ROM (Read Only Memory), a flash memory, an SD card, a RAM (Random Access Memory), or the like. The storage unit 40 stores correspondence information 42, reference information 44, and an imaging application 46. Details of the contents of the correspondence information 42 and reference information 44 will be described later.

The communication unit 22 performs wireless communication to connect to the network NW, and communicates with other devices via the network NW. The imaging unit 24 images the scenery. For example, the imaging unit 24 captures a landscape as a moving image. The imaging unit 24 is, for example, a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera.

The position specifying unit 26 specifies the position of the own device based on signals received from GNSS (Global Navigation Satellite System) satellites.

The imaging control unit 28 causes the imaging unit 24 to image the scenery based on the reference information 44. The transmission control unit 30 uses the communication unit 22 to transmit the image captured by the imaging unit 24 to the information processing device 100. Note that the image captured by the imaging unit 24 may be transmitted to the information processing device 100 using the communication unit of the vehicle 10.

[Information processing device]
The information processing device 100 includes, for example, a communication section 102, a first processing section 104, a second processing section 106, a third processing section 108, a tracking processing section 110, an information management section 112, and an updating section 114. , a providing section 116, and a storage section 120. The first processing section 104, the second processing section 106, the third processing section 108, the tracking processing section 110, the information management section 112, the updating section 114, and the providing section 116 are stored in a storage device by a hardware processor such as a CPU. This is achieved by executing the program. Further, these functional units may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may also be realized by cooperation of software and hardware. Further, the above program may be stored in advance in a storage device or in a removable storage medium such as a DVD or CD-ROM, and when the storage medium is installed in the drive device of the information processing device 100. It may be installed on the storage device by

The storage unit 120 is realized by, for example, a ROM, flash memory, SD card, RAM, HDD (Hard Disc Drive), register, or the like. Further, part or all of the storage unit 120 may be a NAS (Network Attached Storage), an external storage server device, or the like. The storage unit 120 stores, for example, correspondence information 122, reference information 124, specific model 126, and recognition information 128. Details of each piece of information will be described later.

The communication unit 102 communicates with the imaging device 20 via the network NW. The communication unit 102 includes, for example, a communication interface such as a NIC (Network Interface Card).

The first processing unit 104 divides the moving image captured by the imaging unit 24 into images, and generates a target image to be processed.

The second processing unit 106 derives a rectangle estimated to include a predetermined feature or a rectangle estimated to have a predetermined feature from the image. The second processing unit 106 derives a rectangle based on a predetermined rectangle detection algorithm.

The third processing unit 108 inputs the rectangular image derived by the second processing unit 106 to a predetermined specific model 126, and recognizes the meaning of the scenery included in the rectangle based on the output result of the specific model 126. The specific model 126 is a model for classifying images, such as a neural network generated by machine learning or the like.

The tracking processing unit 110 tracks the same feature in a plurality of target images and integrates processing results for the same feature. The tracking processing unit 110 extracts the meaning of the same feature, for example, based on the recognition results for the same feature in a predetermined number of images. For example, the tracking processing unit 110 recognizes the meaning of a feature using a predetermined number of consecutive images (frames). The tracking processing unit 110 also determines movement of the position of the feature between multiple frames. More specifically, in the process of recognizing meaning using a plurality of images, the tracking processing unit 110 recognizes that a feature is recognized twice as a gas station sign, and once as a restaurant sign. If recognized, the feature is identified as a gas station sign, for example.

The information management unit 112 manages information acquired by the own device and processing results of the own device. The updating unit 114 updates the reference information 124 and transmits the updated reference information 124 to the imaging device 20. The providing unit 116 provides the information included in the recognition information 128 to the terminal device 300 based on a request from the terminal device 300.

The news server 200 provides information related to news to the information processing device 100. Information related to news is, for example, information that includes events that attract attention from the public, such as incidents, events, and accidents, and the places and positions where the events occur.

The terminal device 300 is a terminal device held by a user. The terminal device 300 is, for example, a smartphone, a tablet terminal, a navigation device, or the like.

[Processing executed by the information processing system]
FIG. 2 is a sequence diagram showing the flow of processing executed by the imaging device 20 and the information processing device 100. In this process, the information processing device 100 corrects and instructs the location and time for capturing a moving image, and the imaging device 20 instructs the imaging unit 24 to capture a landscape.

First, the information processing device 100 transmits the reference information 124 to the imaging device 20 (S10). Next, the imaging device 20 acquires the transmitted reference information 124 and stores the acquired reference information 124 (44) in the storage unit 40 (S12). Next, the imaging device 20 causes the imaging unit 24 to capture an image based on the position information specified by the position specifying unit 26 and the reference information 44 (S14), and displays the captured image and the location where the image was captured. The correspondence information 42 associated with the information is transmitted to the information processing device 100 (S16). Next, the information processing device 100 stores the acquired correspondence information 42 in the storage unit 120 (S18).

(Standard information)
FIG. 3 is a diagram showing an example of the contents of the reference information 44. The reference information 44 is information in which an imaging area for imaging a landscape is associated with a time period of the imaging target. The reference information 44 is, for example, information that defines the location and time at which images are captured by a moving object such as the imaging device 20, based on information extracted from the previous image. For example, if enough images of that position have not been collected so far, the reference information 44 is generated so that that position will be imaged. The reference information 44 shown in FIG. 3 includes imaging areas AR1 to AR3, and dates and times are associated with the imaging areas AR1 to AR3. For example, the imaging areas AR1 to AR3 are areas indicated by circles having a predetermined radius and having a predetermined position as the center. For example, the imaging device 20 causes the imaging unit 24 to start imaging when entering the imaging regions AR1 to AR3. Note that the reference information 44 may be associated with only the imaging area.

(Compatible information)
FIG. 4 is a diagram showing an example of the contents of the correspondence information 42. The correspondence information 42 is, for example, information in which a captured moving image (image) and the date and time at which the moving image was captured are associated with each other based on the reference information 44. For example, in the correspondence information 42, position information is associated with each imaging time of a moving image.

Returning to the explanation of FIG. 2. The information processing device 100 executes specific processing based on the correspondence information 122 (step S20). The specific process is a process that recognizes the meaning of scenery in an image. The details of the identification process will be explained later with reference to FIGS. 7 to 9.

Next, the information processing device 100 determines whether the timing to update the reference information 124 (44) has arrived (S22). When the timing to update the reference information 124 has arrived, the information processing device 100 updates the reference information 124 (S24), and transmits the updated reference information 124 to the imaging device 20 (S26). The imaging device 20 then acquires the updated reference information 124 (44) and acquires an image based on the acquired reference information 124 (S28).

(Updated standard information)
FIG. 5 is a diagram showing an example of the updated reference information 44. For example, when the information processing device 100 acquires information regarding news from the news server 200, the reference information 124 is updated based on the acquired information regarding the news. The information processing device 100 adds the imaging area AR4 including the position of news-related information to the imaging areas AR1 to AR3, and updates the reference information 124. For example, the imaging area AR4 is an area indicated by a circle with a predetermined radius centered on the position where the incident occurred. Note that the update may be performed when a predetermined period of time has passed since the last time the reference information 44 was transmitted.

Furthermore, the updating unit 114 may generate the reference information 44 in which the imaging position and time are corrected based on one or more of the time, latitude and longitude, or speed of the previous imaging. In this case, for example, the updating unit 114 updates (corrects) the reference information 44 so that an image of a place or time period that was sufficiently imaged last time is imaged. Further, since it was not possible to sufficiently capture an image of a location where the vehicle traveled at a speed higher than a predetermined speed, the reference information 44 is generated so as to capture an image of that location again. In this case, the information processing device 100 acquires, for example, information for specifying the moving speed of the vehicle 10 or the imaging device 20 (for example, the speed itself or position information acquired at predetermined intervals).

As described above, the information processing device 100 corrects the location and position that the imaging device 20 images, and instructs the imaging device 20 to do so. The imaging device 20 then images the scenery based on the instruction. In this way, the imaging device 20 and the information processing device 100 can obtain necessary information while suppressing the amount of communication by performing processing in cooperation.

[Processing executed by the imaging device]
FIG. 6 is a flowchart illustrating an example of the flow of processing executed by the imaging device 20. First, the imaging control unit 28 determines whether the current position is within the imaging area to be imaged, based on the reference information 44 and the position specified by the position specifying unit 26 (S100). If it is not the imaging area, the processing of one routine in this flowchart ends.

If it is the imaging area, the imaging control unit 28 causes the imaging unit 24 to start imaging (S102). Next, the imaging control unit 28 causes the storage unit 40 to store correspondence information 42 that associates the image captured by the imaging unit 24 with position information (S104). Next, the transmission control unit 30 transmits the correspondence information 42 to the information processing device 100 (S106). The information processing device 100 stores the transmitted correspondence information 42 in the storage unit 120 as the correspondence information 122. This completes the processing of one routine in this flowchart.

For example, in order to utilize a video captured by a drive recorder or the like mounted on a vehicle, if all the video is transmitted to the information processing device 100, the amount of communication may exceed the allowable amount. On the other hand, through the process described above, the imaging device 20 transmits the image captured based on the reference information 44 to the information processing device 100, thereby suppressing the amount of communication and transmitting necessary information to the information processing device 100. Can be sent.

[Specific processing executed by information processing device]
FIG. 7 is a flowchart illustrating an example of the flow of specific processing executed by the information processing apparatus 100. First, the first processing unit 104 of the information processing device 100 determines whether an unprocessed video exists in the correspondence information 122 (S200). If there is no unprocessed moving image, the processing of one routine in this flowchart ends.

If an unprocessed video exists, the first processing unit 104 divides the video (S202) and generates target images to be processed (S204). Next, the second processing unit 106 derives a rectangle in the target image (S206). FIG. 8 is a diagram illustrating an example of how a rectangle is derived in a target image. In the target image, a rectangle that is estimated to include a predetermined feature (feature) is derived.

Next, the third processing unit 108 inputs the rectangular image to the specific model 126 (S208). For example, the third processing unit 108 extracts all the rectangles derived from the target image, selects images of the extracted rectangles one by one, and inputs the selected images to the specific model 126.

Next, the third processing unit 108 identifies a feature included in the image based on the output result output to the specific model 126 (S210). FIG. 9 is a diagram showing an example of how a rectangular image IM is input to the specific model 126, and the specific model 126 outputs the price of regular gasoline. For example, if a rectangular image includes a signboard with the price of regular gasoline written on it, and this image (or features of the image) is input to specific model 126, specific model 126 derives the price of regular gasoline. The image features are, for example, the brightness of each pixel, the brightness gradient, or a feature amount derived based on a plurality of pixels in a predetermined range.

Returning to the explanation of FIG. 7. Next, the third processing unit 108 determines whether all the divided images have been processed (S212). When all images have been processed, the processing of one routine in this flowchart is completed. If all target images have not been processed, the process returns to S204. In addition, in the process of this flowchart, when multiple target images are processed, the tracking processing unit 110 executes processing to integrate the processing results of the multiple target images, and recognizes the meaning of the same feature in the target images. You may.

Through the processing described above, the information management unit 112 of the information processing device 100 generates, for example, recognition information 128. FIG. 10 is a diagram showing an example of the contents of the recognition information 128. The recognition information 128 is, for example, information defined for each video. The recognition information 128 includes, for example, image identification information, location information where the image was captured, rectangle identification information, information indicating a feature, semantic information derived from the feature, the date and time of image acquisition, and the date and time when the image was captured. This information is associated with identification information (ID) of the imaging device 20.

Based on the recognition information 128, the information processing device 100 can recognize the gasoline price at a gas station located at a predetermined location and the availability of a predetermined parking lot. The information processing device 100 can then transmit the above recognition result to the terminal device 300. Thereby, the user of the terminal device 300 can obtain information regarding a desired feature.

FIG. 11 is a diagram showing an example of an image displayed on the terminal device 300. For example, the providing unit 116 of the information processing device 100 provides the recognition information 128 to the terminal device 300 in response to a request from the terminal device 300. For example, when the providing unit 116 acquires from the terminal device 300 a request to provide the location information Pc of the terminal device 300 as well as the price of gasoline in the vicinity of the location information Pc, the providing unit 116 acquires from the recognition information 128 a request for providing gasoline prices in the vicinity of the location information Pc. The price is extracted, and information associating the extracted gas station location with the gasoline price is provided to the terminal device 300. Thereby, as shown in FIG. 11, the gasoline prices at gas stations around the location information Pc are displayed on the display section of the terminal device 300.

Further, the communication unit 102 may acquire the route (designated route) on which the vehicle 10 is scheduled to travel. In this case, the providing unit 116 may provide the user with information indicating the fuel price or parking availability recognized through the above-described process based on the acquired route. For example, the providing unit 116 may provide the user with the price of fuel at gas stations located along or along the planned route, or the availability of gas stations near the destination. Further, the providing unit 116 may provide the user with the location of the gas station where the price of fuel is lowest.

According to the first embodiment described above, the information processing device 100 obtains an image of the scenery associated with position information from the imaging device 20, and determines the meaning of the captured scenery from the obtained image. By recognizing the reference information 124, transmitting the reference information 124 to the imaging device 20, and having the imaging device 20 transmit the image, necessary information can be acquired while suppressing the amount of communication.

<Second embodiment>
The second embodiment will be described below. In the first embodiment, the imaging device 20 is described as capturing an image based on the reference information 44 when the position of the imaging device 20 is within the imaging region. In contrast, in the second embodiment, the imaging device 20 acquires the determination model 45 from the information processing device 100 instead of (or in addition to) the reference information 44 . Then, when the imaging device 20 determines that the captured image is a transmission target based on the determination model 45, the imaging device 20 transmits the image to the information processing device 100. The second embodiment will be described below.

FIG. 12 is a diagram illustrating an example of the functional configuration of an imaging device 20A according to the second embodiment. The imaging device 20A includes a storage section 40A instead of the storage section 40. A determination model 45 is stored in the storage unit 40A. The judgment model 45 is a model that, when an image is input, derives a score for the image. For example, the determination model 45 is a model that derives a score indicating that the image includes a predetermined feature (for example, a gas station or a parking lot). The determination model 45 is a model for classifying images, such as a neural network whose learning data is generated by machine learning or the like. The learning data is, for example, information in which an image including a predetermined feature and information for identifying the feature are associated with each other.

In addition to the functional configuration of the imaging device 20, the imaging device 20A further includes a determining unit 29. The determining unit 29 inputs the image captured by the imaging unit 24 to the determination model 45, and obtains the score output by the determination model 45. Then, the determining unit 29 determines whether or not to transmit the image to the information processing device 100 based on the acquired score. Note that the imaging control unit 28 of the second embodiment causes the imaging unit 24 to constantly capture images, for example.

[Processing executed by the imaging device]
FIG. 13 is a flowchart illustrating an example of the flow of processing executed by the imaging device 20A. First, the imaging control unit 28 causes the imaging unit 24 to capture a moving image (S300). Next, the imaging control unit 28 causes the storage unit 40 to store correspondence information 42 that associates the image captured by the imaging unit 24 with position information (S302). Next, the determining unit 29 inputs the captured image (an image obtained by dividing the video) into the judgment model 45 (S304), and based on the result output by the judgment model 45, the image input to the judgment model 45 is transmitted. It is determined whether the image is the target image (S306). If the image input to the determination model 45 is not the image to be transmitted, the processing of one routine in this flowchart ends. If the image input to the determination model 45 is the image to be transmitted, the transmission control unit 30 transmits the image input to the determination model 45 to the information processing apparatus 100 (S308). This completes the processing of one routine in this flowchart.

FIG. 14 is a conceptual diagram of a process in which the imaging device 20A determines whether an image is an image to be transmitted or an image not to be transmitted. In the illustrated example, the determination model 45 is a model that outputs information indicating the degree to which an image includes a feature that resembles a gas station. If the determination model 45 outputs a result indicating that the input image includes a feature that looks like a gas station, as shown in the upper diagram of FIG. 14, this image is determined to be the image to be transmitted. When the determination model 45 outputs a result indicating that the input image does not include any features that are typical of a gas station, as shown in the lower diagram of FIG. 14, this image is determined to be an image that is not to be transmitted.

Note that the number of determination models 45 is not limited to one, and a plurality of determination models may be prepared. For example, the imaging device 20 may input an image to a plurality of determination models 45, and if any of the determination models 45 outputs a result to be transmitted, the image may be designated as a transmission target.

Further, the determining unit 29 uses the plurality of determination models 45 to send to the information processing device 100 images captured after the first feature is recognized and until the second feature is recognized. You can also send it. For example, the determining unit 29 may transmit to the information processing device 100 an image captured after the guardrail is recognized and until the red building is recognized.

According to the second embodiment described above, the information processing device 100 transmits the determination model 45 to the imaging device 20A. When the imaging device 20A determines based on the determination model 45 that the image captured by the imaging unit 24 is the image to be transmitted, the imaging device 20A transmits the image to the information processing device 100. In this way, the information processing apparatus 100 does not need to set an imaging area over a wide range, and therefore can more easily acquire necessary information while suppressing communication traffic.

<Third embodiment>
The third embodiment will be described below. In the third embodiment, the information processing device 100 transmits predetermined conditions to the imaging device 20A instead of the reference information 44. If the timing at which the image was captured satisfies a predetermined condition, the imaging device 20A transmits the image captured at that timing to the information processing device 100. The predetermined condition is a condition set for information regarding a vehicle. The third embodiment will be described below, focusing on the differences from the first embodiment.

FIG. 15 is a flowchart illustrating an example of the flow of processing executed by the imaging device 20A of the third embodiment. This process is, for example, a process executed at predetermined intervals. First, the imaging control unit 28 causes the imaging unit 24 to capture a moving image (S400). Next, the imaging control unit 28 causes the storage unit 40 to store correspondence information 42 that associates the image captured by the imaging unit 24 with position information (S402). It is assumed that the imaging device 20 communicates with the vehicle and acquires the state of the vehicle at each time.

Next, the determining unit 29 refers to the correspondence information 42 and the information regarding the vehicle, identifies an image to be transmitted (S404), and transmits the identified image to the information processing device 100 (S406). This completes the processing of this flowchart.

Hereinafter, a method for identifying an image to be transmitted will be described with reference to the correspondence information 42 and information regarding the vehicle. For example, the determining unit 29 determines an image captured at a timing when the vehicle is controlled according to a preset control procedure as the image to be transmitted. The vehicle is controlled according to a preset control procedure, for example, when the acceleration is controlled to be above a threshold value, or when the speed is controlled to be above a first predetermined speed and then below a second predetermined speed. , the steering has been operated to a predetermined degree or more.

Further, the determining unit 29 may determine, as the image to be transmitted, an image captured at a timing when the position information and speed information of the vehicle satisfy a predetermined condition. For example, images before and after the speed has changed may be transmitted to the information processing device 100, such as when the vehicle was traveling at around 80 km on an expressway but traveled at 50 km only in a predetermined section. This is because in this case, there may be an accident or an obstacle.

Further, for example, the determining unit 29 may determine the image to be transmitted based on the state of the vehicle, as described below. For example, (1) images before and after refueling, and (2) images before and after stopping at a parking lot may be transmitted to the information processing device 100. (1) Images before and after refueling are identified based on changes in the fuel in the fuel tank. FIG. 16 is a diagram for explaining a process in which images before and after refueling are specified. The vertical axis in FIG. 16 indicates the level of fuel in the fuel tank, and the horizontal axis in FIG. 16 indicates time. For example, when the level of fuel increases at a predetermined slope as shown at time P1 in the figure, the determining unit 29 determines that fuel has been refilled at that time (for example, determines that gasoline has been refilled at a gas station). ), the images at times P2 and P3 before and after that are determined to be images to be transmitted.

As described above, when an image of a gas station is to be transmitted, the imaging device 20A can more reliably transmit the image of the gas station to the information processing device 100 through the process described above.

(2) Images before and after the vehicle is parked are identified based on the speed of the vehicle and the state of the parking brake. FIG. 17 is a diagram for explaining a process in which images before and after the vehicle is parked in a parking lot are identified. The vertical axis in FIG. 17 shows the speed of the vehicle, and the horizontal axis in FIG. 17 shows time. For example, if the speed of the vehicle is zero for a predetermined time and the parking brake is in an on state, as shown at time P11 in the figure, the determining unit 29 determines that the vehicle has stopped in the parking lot at that time. At least the image at time P13 after the specific state is determined to be the image to be transmitted.

As described above, when a parking image is to be transmitted, the imaging device 20A can more reliably transmit the parking image to the information processing device 100 through the processing described above.

According to the third embodiment described above, the information processing device 100 transmits the predetermined conditions to the imaging device 20A. Then, when the imaging device 20A determines that the image captured by the imaging unit 24 is the image to be transmitted based on a predetermined condition, the imaging device 20A transmits the image to the information processing device 100. In this way, the information processing apparatus 100 does not need to set an imaging area over a wide range, and therefore can more easily acquire necessary information while suppressing communication traffic.

<Fourth embodiment>
The fourth embodiment will be described below. The fourth embodiment will be described based on the processing of the first embodiment. In the fourth embodiment, the information processing device 100 includes a plurality of specific models 126. Then, the information processing apparatus 100 selects a specific model 126 from among the plurality of specific models 126 based on the position where the image was captured, inputs the image to the selected specific model 126, and analyzes the image. Hereinafter, differences from the first embodiment will be mainly explained.

A specific model 126 is stored in the storage unit 120 of the information processing device 100 of the fourth embodiment. FIG. 18 is a diagram showing an example of the contents of the plurality of specific models 126. For example, each of the plurality of specific models 126 is information in which a position (or region), a type of the specific model 126, and a recognition target are associated with each other. The specific model 126 associated with a position is a specific model that can more accurately recognize the meaning of the scenery included in the rectangle of the image in which the associated position is captured than the other specific models 126. Note that, as illustrated, the recognition target of the specific model 126 is not limited to one, but may be multiple.

FIG. 19 is a flowchart illustrating an example of the flow of processing executed by the information processing apparatus 100 of the fourth embodiment. Description of processes similar to those in FIG. 7 will be omitted. After the processing in S200, the first processing unit 104 divides the video (S202). Next, the first processing unit 104 identifies the target image and the position where the target image is identified (S203). Next, the first processing unit 104 identifies the specific model 126 based on the identified position (S205). Next, the second processing unit 106 derives a rectangle in the target image (S206). Next, the third processing unit 108 inputs a rectangular image to the specific model 126 specified in S205 (S208). Then, the process proceeds to the subsequent steps.

As described above, the information processing apparatus 100 can more accurately recognize the meaning contained in an image by specifying the specific model 126 to be used based on the position of the scenery contained in the image.

According to the fourth embodiment described above, the information processing apparatus 100 applies one of the plurality of specific models 126 to each position where the acquired image is taken, from among the plurality of specific models 126 applied to each position where the image is taken. By using the specific model 126 to recognize a specific meaning within a semantic domain that represents a specific meaning related to a feature included in a landscape, and by providing the recognized specific meaning to the user, can provide more useful information.

Note that, for example, if the target image is an image of a gas station signboard located at a predetermined position, the third processing unit 108 may combine a previous image of the same gas station signboard with a previous image of the same gas station signboard. The difference in price may be extracted from the difference between this image and the current image of the signboard. For example, the third processing unit 108 derives the difference between the images within the rectangle including the gasoline price of the previously captured image and the currently captured image, and based on the derived difference and a preset template. Then, the price of gasoline in the currently captured image may be derived. In this way, the third processing unit 108 can extract the price of gasoline based on the difference from the previously captured image when performing image recognition.

<Fifth embodiment>
The fifth embodiment will be described below. The fifth embodiment will be described based on the processing of the second embodiment. In the fifth embodiment, the first processing unit 104 selects a specific model 126 from among the plurality of specific models 126 based on the type of image transmitted from the imaging device 20, and assigns the image to the selected specific model 126. Enter and analyze images. Hereinafter, differences from the first embodiment will be mainly explained.

A plurality of specific models 126 are stored in the storage unit 120 of the information processing device 100 of the fifth embodiment. FIG. 20 is a diagram showing an example of the contents of the plurality of specific models 126. For example, the plurality of specific models 126 are information in which image types, model types, and recognition targets are associated with each other. The specific model 126 associated with the type of image is a specific model that can more accurately recognize the meaning of the scenery included in the rectangle of the image of the associated type than the other specific models 126. Note that, for example, the type of image is not limited to the type classified as gas station, parking lot, etc. as shown in the figure, but may be classified as a building, signboard, etc. Further, as illustrated, the recognition target of the specific model 126 is not limited to one, but may be multiple.

FIG. 21 is a flowchart illustrating an example of the flow of processing executed by the information processing apparatus 100 of the fifth embodiment. Description of processes similar to those in FIG. 7 will be omitted. After the processing in S200, the first processing unit 104 divides the video (S202) and generates target images to be processed (S204). Next, the first processing unit 104 identifies the specific model 126 based on the type of the target image (S205#). Next, the second processing unit 106 derives a rectangle in the target video (S206). Next, the third processing unit 108 inputs a rectangular image to the specific model 126 specified in S205# (S208). Then, the process proceeds to the subsequent steps.

As described above, the information processing apparatus 100 can more accurately recognize the meaning contained in an image by specifying the specific model 126 to be used based on the type of image.

According to the fifth embodiment described above, the information processing apparatus 100 selects the specific model 126 that is applied to each landscape of the acquired image from among the multiple specific models 126 that are applied to each landscape (type) of the image. is used to recognize a specific meaning within a semantic domain that represents a specific meaning related to a feature included in the landscape, and provide the user with the recognized specific meaning, thereby making it more useful to the user. information.

<Sixth embodiment>
The sixth embodiment will be described below. The information processing system 1A of the sixth embodiment differs from each embodiment in that it further includes a learning device 400. Hereinafter, differences from the first embodiment will be mainly explained.

FIG. 22 is a diagram illustrating an example of the functional configuration of an information processing system 1A according to the sixth embodiment. The information processing system 1A, for example, further includes a learning device 400 in addition to the functional configuration of the information processing system 1 of the first embodiment.

The learning device 400 includes, for example, a learning section 410 and learning data 412. Learning unit 410 generates specific model 126 based on learning data 412. The learning device 400 then transmits the generated specific model 126 to the information processing device 100.

FIG. 23 is a diagram showing an example of the contents of the learning data 412. For example, the learning data 412 is information in which an image showing the price of fuel at a gas station is associated with correct data. The image showing the price of fuel at a gas station may be, for example, an image of a signboard displaying the price of fuel at the same gas station (or an image of the same location or the same scenery). However, the image may be a signboard displaying fuel prices at different gas stations. The correct data includes the identification information of the feature included in the image (for example, a signboard displaying the price of regular gasoline), the information expressed on the feature (for example, the price of regular gasoline), and the area corresponding to the feature. are information that are associated with each other.

The learning unit 410 performs machine learning on the learning data 412 to generate a specific model 126 that, when an image is input, outputs the price of regular gasoline or information indicating that the price of regular gasoline is not included in the image. That is, since processing is performed to recognize the same type of signboard, learning processing is performed exclusively for the signboard, and a highly accurate specific model 126 is generated.

FIG. 24 is a diagram showing another example of the contents of the learning data 412. For example, the learning data 412 is information in which an image representing parking availability and correct answer data are associated with each other. The image showing the parking availability may be, for example, an image of a signboard displaying the same parking availability (or an image of the same location or the same scenery), or an image of a different parking lot. The image may be a captured image of a signboard displaying the availability status of. Correct data includes the identification information of a feature included in the image (for example, a signboard indicating parking availability), information indicating whether the signboard is empty or full, and the area corresponding to the feature. It is information. The learning unit 410 performs machine learning on the learning data 412 to generate a specific model 126 that, when an image is input, outputs information indicating that the image does not include parking availability or a signboard representing parking availability information. do.

Note that the predetermined feature may be a closed gas station, a closed convenience store, or an opened store. In this case, the correct data is information in which information such as closed gas stations, closed convenience stores, and opened stores is associated with information indicating these areas. The learning unit 410 learns this learning data 412 and when an image is input, information that the image includes a closed gas station, information that the image includes a closed convenience store, and information that the image includes a store that has opened. A specific model 126 that outputs information indicating this is generated. Thereby, it is possible to easily recognize the opening or closing of a store based on the image.

Furthermore, the predetermined feature may be a vehicle number, a person's face, or the like. In this case, the correct data may include information in which an image containing a vehicle number plate is associated with information indicating the area of the vehicle number plate, or information in which an image containing a person's face is associated with information indicating the area of the person's face. This is the information given. The learning unit 410 learns this learning data 412 and, when an image is input, generates a specific model 126 that outputs information indicating whether the image includes a person's face and information indicating the area of the person's face. Thereby, a person's face can be easily recognized based on the image.

In this way, by learning information such as differences in information between individual signboards and images taken at the same location, it is possible to more accurately recognize individual signboards and the semantic information of a given location. A model 126 is generated. Since the specific model 126 is a model generated by learning the difference in image information, the information processing device 100 performs the process of recognizing a specific meaning in the semantic region using the specific model 126 at the first time. This is an example of a process of recognizing a specific meaning within the semantic region based on the difference between the information on the semantic region of the image acquired at the second time and the information on the semantic region acquired at the second time.

According to the sixth embodiment described above, the learning device 400 learns the learning data 412 and generates the specific model 126. Using this specific model 126, the information processing device 100 can accurately recognize a specific meaning within a semantic area that represents a specific meaning related to a feature. As a result, more useful information can be provided to the user.

Further, as described above, the learning data 412 may be subjected to machine learning to generate a rectangular derivation model. The rectangle derivation model is, for example, a model used by the second processing unit 106 in the process of deriving a rectangle. Thereby, the information processing apparatus 100 can accurately derive a rectangle including the predetermined feature.

Further, as described above, the learning data 412 may be subjected to machine learning to generate the determination model 45. Thereby, the information processing apparatus 100 can provide the imaging device 20 with a model that allows the imaging device 20 to determine whether the image is to be transmitted.

According to each of the embodiments described above, the information processing device 100 causes the communication unit 22 that communicates with other devices and the imaging unit 24 that images a landscape to capture an image based on reference information indicating a standard for capturing a landscape. A communication unit 102 that acquires an image of a landscape associated with position information from an imaging device 20 including an imaging control unit 28 and a position specifying unit 26 that acquires a position, and an image acquired by the communication unit 102. The third processing unit 108 recognizes the meaning of the captured scenery, and the updating unit 114 transmits reference information to the imaging device 20 and causes the imaging device 20 to transmit images, thereby suppressing the amount of communication. You can obtain the necessary information while doing so.

Further, according to each of the embodiments described above, the information processing device 100 associates the position information specified by the position specifying unit with the other device having the position specifying unit and the imaging unit, and the image capturing unit 24 detects the scenery. A communication unit 102 that acquires a captured image extracts a semantic region expressing a specific meaning related to a feature included in the landscape from the image acquired by the communication unit 102, and It includes a third processing unit 108 that recognizes the meaning, and a providing unit 116 that provides the user with the specific meaning recognized by the third processing unit 108 in association with the position where the image including the semantic region was captured. This makes it possible to provide more useful information to users.

Although the mode for implementing the present invention has been described above using embodiments, the present invention is not limited to these embodiments in any way, and various modifications and substitutions can be made without departing from the gist of the present invention. can be added.

DESCRIPTION OF SYMBOLS 10...Vehicle, 20...Imaging device, 22...Communication section, 24...Imaging section, 26...Position specifying section, 28...Imaging control section, 29...Determining section, 30...Transmission control section, 40, 40A...Storage section, 42 .. Correspondence information, 44.. Reference information, 45.. Judgment model, 46.. Imaging application, 100.. Information processing device, 102.. Communications department, 104.. First processing section, 106.. Second processing section, 108.. Third processing section. , 110... Tracking processing unit, 112... Information management unit, 114... Updating unit, 116... Providing unit, 120... Storage unit, 122... Correspondence information, 124... Reference information, 126... Specific model, 128... Recognition information, 200... News server, 300‥Terminal device, 400‥Learning device

Claims (20)

A communication unit that communicates with other devices, an imaging control unit that causes an imaging unit that captures a landscape to capture an image, a position acquisition unit that acquires a position, and a determination unit that determines an image to be transmitted based on the image and model information. an acquisition unit that acquires, from a terminal device including, an image that is determined to be a transmission target by the determination unit from among the images that capture the landscape, and that is associated with position information; and,
The determining unit extracts a semantic region expressing a specific meaning related to a feature included in the landscape from the image acquired by the acquiring unit, and selects a semantic region that is acquired by the acquiring unit from among the plurality of specific models. a recognition unit that recognizes a specific meaning within the semantic area using a specific model applied to the specific meaning type of the semantic area of the image determined to be transmitted by;
a providing unit that provides a user with the specific meaning recognized by the recognition unit in association with the position where the image including the meaning area was captured;
An information providing device comprising: An image of a first type including a first type of feature whose landscape is imaged by the imaging unit, from another device having a position identification unit and an imaging unit, with which positional information identified by the location identification unit is associated, or an acquisition unit that acquires a second type of image including a second type of feature;
When the acquisition unit acquires the first type of image, extracts a semantic region representing a specific meaning related to the first type of feature included in the landscape from the first type of image; , out of a plurality of specific models, a first specific model that is applied to the first type of scenery with a specific meaning in the semantic area of the image acquired by the acquisition unit is used to determine the landscape in the semantic area. recognize a specific meaning,
When the acquisition unit acquires the second type of image, extracts a semantic region representing a specific meaning related to the second type of feature included in the landscape from the second type of image; , out of a plurality of specific models, a second specific model that is applied to the second type of landscape of the specific meaning in the semantic area of the image acquired by the acquisition unit is used to determine the landscape within the semantic area. a recognition part that recognizes a specific meaning;
a providing unit that provides a user with the specific meaning recognized by the recognition unit in association with the position where the image including the meaning area was captured;
An information providing device comprising: The specific model is generated by machine learning of learning data in which a learning image in which the scenery is captured and a specific meaning in the semantic area in the learning image are associated with each other. ,
The information providing device according to claim 1 or 2. A communication unit that communicates with other devices, an imaging control unit that causes an imaging unit that captures a landscape to capture an image, a position acquisition unit that acquires a position, and a determination unit that determines an image to be transmitted based on the image and model information. an acquisition unit that acquires, from a terminal device including, an image that is determined to be a transmission target by the determination unit from among the images that capture the landscape, and that is associated with position information; and,
The determining unit extracts a semantic region expressing a specific meaning related to a feature included in the landscape from the image acquired by the acquiring unit, and selects a semantic region that is acquired by the acquiring unit from among the plurality of specific models. a recognition unit that recognizes a specific meaning in the semantic area using a specific model applied to the type of signboard that is the feature in the semantic area of the image determined to be transmitted by;
a providing unit that provides a user with the specific meaning recognized by the recognition unit in association with the position where the image including the meaning area was captured;
An information providing device comprising: An image of a first type including a first type of feature whose landscape is imaged by the imaging unit, from another device having a position identification unit and an imaging unit, with which positional information identified by the location identification unit is associated, or an acquisition unit that acquires a second type of image including a second type of feature;
When the acquisition unit acquires the first type of image, extracts a semantic region representing a specific meaning related to the first type of feature included in the landscape from the first type of image; , from among a plurality of specific models, using a first specific model that is applied to a signboard that is the feature of the specific meaning of the first type in the semantic region of the image acquired by the acquisition unit, recognizing a specific meaning within said semantic domain;
When the acquisition unit acquires the second type of image, extracts a semantic region representing a specific meaning related to the second type of feature included in the landscape from the second type of image; , from among a plurality of specific models, using a second specific model that is applied to a signboard that is the feature of the specific meaning of the second type in the semantic area of the image acquired by the acquisition unit, a recognition unit that recognizes a specific meaning within the semantic area;
a providing unit that provides a user with the specific meaning recognized by the recognition unit in association with the position where the image including the meaning area was captured;
An information providing device comprising: The specific model is generated by machine learning of learning data in which a learning image of the signboard and a specific meaning indicated by the signboard are associated with each other.
The information providing device according to claim 4 or 5. The feature is a gas station or a parking lot.
The information providing device according to any one of claims 1 to 6. The recognition unit extracts a semantic region representing a vehicle fuel price within the image from the image acquired by the acquisition unit, and recognizes the fuel price within the semantic region,
The providing unit provides the fuel price recognized by the recognition unit to the user in association with the position where the image including the semantic region was captured.
The information providing device according to any one of claims 1 to 6. The acquisition unit acquires the specified route,
The providing unit provides the fuel price recognized by the recognition unit to the user based on the route acquired by the acquisition unit.
The information providing device according to claim 8. The recognition unit extracts a semantic region representing parking availability in the image from the image acquired by the acquisition unit, and recognizes the availability in the semantic region,
The providing unit provides the user with the parking availability status recognized by the recognition unit in association with the position where the image including the semantic region was captured.
The information providing device according to any one of claims 1 to 6. The acquisition unit acquires the specified route,
The providing unit provides the availability status recognized by the recognition unit to the user based on the route acquired by the acquisition unit.
The information providing device according to claim 10. further comprising an information management unit that manages recognition information that associates a position where the image including the meaning region is captured with a specific meaning recognized by the recognition unit,
The provision unit provides the user with the recognition information managed by the information management unit.
The information providing device according to any one of claims 1 to 10. The computer is
A communication unit that communicates with other devices, an imaging control unit that causes an imaging unit that captures a landscape to capture an image, a position acquisition unit that acquires a position, and a determination unit that determines an image to be transmitted based on the image and model information. acquiring from a terminal device including an image that is determined to be a transmission target by the determining unit from among images that capture the landscape, and that is an image that captures the landscape that is associated with position information;
A semantic region expressing a specific meaning related to a feature included in the landscape is extracted from the acquired image, and the acquired image is determined to be transmitted by the determining unit from among the plurality of specific models. recognizing a specific meaning within the semantic region using a specific model applied to the specific meaning type of the semantic region of the image;
providing the recognized specific meaning to the user in association with the position where the image including the meaning area was captured;
Information provision method. to the computer,
A communication unit that communicates with other devices, an imaging control unit that causes an imaging unit that captures a landscape to capture an image, a position acquisition unit that acquires a position, and a determination unit that determines an image to be transmitted based on the image and model information. acquiring from a terminal device including an image that is determined to be a transmission target by the determining unit among images that capture the landscape, and that is associated with position information, and that captures the landscape;
A semantic region expressing a specific meaning related to a feature included in the landscape is extracted from the acquired image, and the acquired image is transmitted by the determination unit from among a plurality of specific models. recognizing a specific meaning within the semantic area using a specific model applied to the specific meaning type of the semantic area of the determined image;
providing the user with the recognized specific meaning in association with the position where the image including the meaning area was captured;
program. The computer is
An image of a first type including a first type of feature whose landscape is imaged by the imaging unit, from another device having a position identification unit and an imaging unit, with which positional information identified by the location identification unit is associated, or Obtaining a second type of image including a second type of feature;
When the first type of image is acquired, a semantic region representing a specific meaning related to the first type of feature included in the landscape is extracted from the first type of image, and a plurality of specific meanings are extracted from the first type of image. Recognizing a specific meaning within the semantic area using a first specific model from among the models that is applied to the specific semantic landscape of the first type in the semantic area of the acquired image;
When the second type of image is acquired, a semantic region expressing a specific meaning related to the second type of feature included in the landscape is extracted from the second type of image, and a plurality of specific meanings are extracted from the second type of image. Recognizing a specific meaning within the semantic area using a second specific model from among the models that is applied to the second type of the specific semantic landscape of the semantic area of the acquired image;
providing the recognized specific meaning to the user in association with the position where the image including the meaning area was captured;
Information provision method. to the computer,
An image of a first type including a first type of feature whose landscape is imaged by the imaging unit, from another device having a position identification unit and an imaging unit, with which positional information identified by the location identification unit is associated, or acquiring a second type of image including a second type of feature;
When the first type of image is acquired, a semantic region expressing a specific meaning related to the first type of feature included in the landscape is extracted from the first type of image, and a plurality of specific From among the models, a first specific model that is applied to the landscape of the first type of the specific meaning in the semantic area of the acquired image is used to recognize a specific meaning in the semantic area. ,
When the second type of image is acquired, a semantic region expressing a specific meaning related to the second type of feature included in the landscape is extracted from the second type of image, and a plurality of specific meanings are extracted from the second type of image. From among the models, a second specific model that is applied to the second type of landscape of the specific meaning in the semantic area of the acquired image is used to recognize the specific meaning in the semantic area. ,
providing the user with the recognized specific meaning in association with the position where the image including the meaning area was captured;
program. The computer is
A communication unit that communicates with other devices, an imaging control unit that causes an imaging unit that captures a landscape to capture an image, a position acquisition unit that acquires a position, and a determination unit that determines an image to be transmitted based on the image and model information. acquiring from a terminal device including an image that is determined to be a transmission target by the determining unit from among images that capture the landscape, and that is an image that captures the landscape that is associated with position information;
A semantic region expressing a specific meaning related to a feature included in the landscape is extracted from the acquired image, and the acquired image is determined to be transmitted by the determining unit from among a plurality of specific models. recognizing a specific meaning in the semantic area using a specific model applied to the type of signboard that is the feature in the semantic area of the image,
providing the recognized specific meaning to the user in association with the position where the image including the meaning area was captured;
Information provision method. to the computer,
A communication unit that communicates with other devices, an imaging control unit that causes an imaging unit that captures a landscape to capture an image, a position acquisition unit that acquires a position, and a determination unit that determines an image to be transmitted based on the image and model information. acquiring from a terminal device including an image that is determined to be a transmission target by the determining unit among images that capture the landscape, and that is associated with position information, and that captures the landscape;
A semantic region expressing a specific meaning related to a feature included in the landscape is extracted from the acquired image, and the acquired image is transmitted by the determining unit from among a plurality of specific models. Recognizing a specific meaning within the semantic area using a specific model applied to the type of signboard that is the feature in the semantic area of the determined image;
providing the user with the recognized specific meaning in association with the position where the image including the meaning area was captured;
program. The computer is
An image of a first type including a first type of feature whose landscape is imaged by the imaging unit, from another device having a position identification unit and an imaging unit, with which positional information identified by the location identification unit is associated, or Obtaining a second type of image including a second type of feature;
When the first type of image is acquired, a semantic region representing a specific meaning related to the first type of feature included in the landscape is extracted from the first type of image, and a plurality of specific meanings are extracted from the first type of image. From among the models, a first specific model that is applied to a signboard that is the feature of the first type and the specific meaning in the semantic area of the acquired image is used to identify a specific model in the semantic area. recognize the meaning,
When the second type of image is acquired, a semantic region expressing a specific meaning related to the second type of feature included in the landscape is extracted from the second type of image, and a plurality of specific meanings are extracted from the second type of image. From among the models, a second specific model is applied to a signboard that is a feature of the second type and the specific meaning in the semantic area of the acquired image. recognize the meaning,
providing the recognized specific meaning to the user in association with the position where the image including the meaning area was captured;
Information provision method. to the computer,
An image of a first type including a first type of feature whose landscape is imaged by the imaging unit, from another device having a position identification unit and an imaging unit, with which positional information identified by the location identification unit is associated, or acquiring a second type of image including a second type of feature;
When the first type of image is acquired, a semantic region expressing a specific meaning related to the first type of feature included in the landscape is extracted from the first type of image, and a plurality of specific From among the models, a first specific model that is applied to a signboard that is the feature of the first type and the specific meaning in the semantic area of the acquired image is used to specify the specific meaning within the semantic area. Recognize the meaning of
When the second type of image is acquired, a semantic region expressing a specific meaning related to the second type of feature included in the landscape is extracted from the second type of image, and a plurality of specific meanings are extracted from the second type of image. From among the models, a second specific model is applied to a signboard that is a feature of the second type and the specific meaning in the semantic area of the acquired image. Recognize the meaning,
providing the user with the recognized specific meaning in association with the position where the image including the meaning area was captured;
program.

Images