JP2020119515A - Installation object recognition system and program thereof - Google Patents

Abstract

To provide an installation object recognition system and a program thereof that identify a type of installation object provided along a road and analyze contents of the installation object.SOLUTION: In an installation object recognition system and a program thereof, when an instruction to search for a specific installation object is input for a road (path) designated from image data of a landscape photographed along the road, an installation object recognition device 1 searches for image data of the installation object, determines whether the installation object exists, and when the installation object exists, determines a type to analyze contents, learns the characteristic of the installation object, and displays an icon corresponding to the type or the like on the map.SELECTED DRAWING: Figure 1

Description

The present invention relates to a system for recognizing an installation provided along a road, and more particularly, to an installation recognition system and a program therefor capable of identifying the type of installation and analyzing the content of the installation.

[Conventional technology]
Conventionally, there has been a system for recognizing an object from an image taken by a camera.
The technology of image recognition is that the pattern of the image to be detected is recognized in advance by the image processing device of the system, the correlation between the image pattern and the image captured by the camera is measured, and the captured image with high correlation is detected as the image to be detected. There is something to judge.

[Related technology]
As related prior art documents, Japanese Patent Laid-Open No. 2018-163654 “System and method for telecommunications event management” (Patent Document 1) and Japanese Patent No. 5462609 “Stop line recognition device” (Patent Document 2) ), Japanese Patent No. 6411933 "Vehicle state determination device" (Patent Document 3).

Patent Document 1 describes that a street view image and GPS (Global Positioning System) position information are acquired, a target image is model-learned by deep learning, and the target object is recognized.
Patent Document 2 describes a device that recognizes a stop line on a road when a vehicle-mounted camera recognizes a red traffic light.
Patent Document 3 describes a device that recognizes the number of lanes on a road from map information and determines a vehicle that stops at a roadside zone on the road.

JP, 2008-163654, A Japanese Patent No. 5462609 Japanese Patent No. 6411933

However, although the above-mentioned conventional system recognizes an object from an image captured by a camera, it does not analyze the type and content of the object and cannot easily grasp the characteristics of the object. There was a problem.

It should be noted that Patent Documents 1 to 3 do not describe how to recognize the contents of an installed object and grasp its characteristics.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an installation object recognition system and a program thereof that specify the type of installation object provided along a road and analyze the content of the installation object. To do.

(Installation object recognition system)
The present invention for solving the problems of the above conventional example is an installation object recognition system including an installation object recognition device that searches for a specific installation object from image data of a landscape taken along a route. When the route in the map information is designated and an instruction to search for a specific installation is given, the recognition device determines whether the image data of the specific installation exists along the route in the image data of the landscape. When the image data of the specific installation object is present, the type of the specific installation object is determined, and the type of the specific installation object is stored together with the position information.

(Determine the size of the installation)
In the above installation object recognition system, the installation object recognition device determines the size of the specific installation object when image data of the specific installation object is present, and determines the size of the specific installation object together with the position information. It is characterized by storing.

(Icon display of installed objects)
The present invention is characterized in that, in the installation object recognition system, the installation object recognition device displays an icon corresponding to a type and size of a specific installation object on a map displayed on a display unit according to position information. ..

(3D image display of installation)
The present invention is characterized in that, in the installation object recognition system, the installation object recognition device displays a three-dimensional image corresponding to a type and size of a specific installation object on a map displayed on a display unit according to position information. And

(Use of machine learning)
According to the present invention, in the installation object recognition system, when the installation object recognition device determines that image data of a specific installation object exists in image data of a landscape, the image data portion of the image data of the scenery is stored. It is characterized in that it is machine-learned and used in the determination process.

(Image data of the image capturing device that acquires the position information)
The present invention is characterized in that, in the installation object recognition system, the image data of the installation object is taken by a shooting device that acquires position information.

(Installation object recognition program)
The present invention is a program that operates in an installation object recognition device that searches for a specific installation object from image data of a landscape taken along a path, wherein the installation object recognition device specifies a path in map information and specifies the installation object. When the instruction to search the installation object is given, it is judged whether the image data of the specific installation object exists in the image data of the scenery along the route, and when the image data of the specific installation object exists. Is characterized by determining the type of a specific installation and causing the function to store the type of the specific installation together with the position information.

(Program for determining the size of the installation)
According to the present invention, in the above program, the installation object recognizing device determines the size of the specific installation object when the image data of the specific installation exists, and stores the size of the specific installation object together with the position information. It is characterized by making it function as follows.

(Program with icon display)
The present invention is characterized in that, in the above program, the installation object recognition device is caused to function so as to display an icon corresponding to a type and size of a specific installation object on a map displayed on a display unit according to position information. ..

(3D image display program)
According to the present invention, in the above program, the installation object recognition device is caused to function so as to display a three-dimensional image corresponding to the type and size of a specific installation object on a map displayed on the display unit according to position information. And

(Program for using machine learning)
According to the present invention, in the above program, when the installation object recognition device determines that the image data of a specific installation object exists in the image data of a landscape, the image data portion of the image data of the landscape is stored and machine learning is performed. It is characterized in that it is made to function so as to be used for the judgment processing.

(Image data of the image capturing device that acquires the position information)
The present invention is characterized in that, in the above program, the image data of the installation object is captured by an image capturing device that acquires position information.

According to the present invention, when the installation recognizing device specifies the route in the map information and is instructed to search for the specific installation, the image data of the specific installation becomes image data of the landscape along the route. As an installation object recognition system that determines whether or not there is image data of a specific installation object, determines the type of the specific installation object, and stores the type of the specific installation object together with position information. Since it is possible to easily understand the characteristics of the installation from the image of the landscape, it has the effect of being able to use it for marketing strategies.

It is a schematic diagram of this system. It is a figure which shows the example of a display of the installation object in this system. It is a flowchart of the installation object search process in this system. It is a flowchart of the icon display process in this system.

Embodiments of the present invention will be described with reference to the drawings.
[Outline of Embodiment]
An installation object recognition system (this system) according to an embodiment of the present invention inputs an instruction to search for a specific installation object on a designated road (route) from image data of a landscape photographed along the road. Then, the image data of the installation is searched, it is determined whether there is the installation, and if there is the installation, the type is determined and the content is analyzed to grasp the characteristics of the installation. Then, the icon according to the type etc. is displayed on the map, and the characteristics of the installation object can be displayed on the map with the icon to easily grasp the characteristics of the installation object, which can be used for the marketing strategy. ..

Further, the present system detects a specific installation object from the image data taken together with the position information by the imaging device having the GPS function, analyzes the type and characteristics of the installation object, and displays the icon of the installation object on the two-dimensional map. Is displayed or the 3D image data of the installation object is displayed on a three-dimensional map, and the arrangement characteristics of the installation object can be easily recognized and can be used in a marketing strategy.

[This system: Fig. 1]
This system will be described with reference to FIG. FIG. 1 is a schematic diagram of the present system.
As shown in FIG. 1, the present system includes an installation object recognition device 1, an image database (DB) 2, a map database (DB) 3, a network 4, and an installation object database (DB) 5. ..

The image DB 2 is a storage unit that stores image data of a landscape photographed along a road corresponding to coordinate data.
The map DB 3 is a storage unit that stores map information (map data) corresponding to coordinate data.
The image DB 2 and the map DB 3 may be integrated.

The network 4 connects the installation recognizing device 1, the image DB 2, the map DB 3, and the installation DB 5, and assumes the Internet.
The installation object DB 5 is a storage unit that stores image data of an installation object, which will be described later, and related information (installation object type, size, etc.) in association with position information as installation object information.

The installed object recognition device 1 is a processing device of a computer, and includes a control unit 11, a storage unit 12, an interface unit 13, a display unit 14, and an input unit 15.
The control unit 11 operates the processing program stored in the storage unit 12 and executes the processing described below.
The storage unit 12 stores a processing program and other necessary data.
The interface unit 13 is connected to the network 4, the display unit 14, and the input unit 15.
The display unit 14 displays the processing result and the like.
The input unit 15 inputs an instruction or the like for performing processing.

[Processing content]
The processing contents in the installed object recognition device 1 will be described.
The control unit 11 reads the processing program from the storage unit 12 and causes the display unit 14 to display map data of the corresponding area from the map DB 3 according to an instruction from the input unit 15.
When a road to be searched is specified for the displayed map data and an installation object to be searched (for example, a vending machine for soft drinks) is set, the control unit 11 causes the scenery image data corresponding to the road. Is read from the image DB 2 and it is determined whether or not there is an installation object to be searched.
Since the road is specified in the map data by specifying the coordinate data, the corresponding landscape image data is also specified in the image DB 2 based on the coordinate data.

Next, if there is an installation object to be searched as a result of the determination, the control unit 11 analyzes the type of the installation object (beverage maker) and the size of the installation object (large, medium, and small), and further, the accuracy of image recognition. If it can be raised, the handled product can be recognized and analyzed from the image pattern of the installation.

In the above processing, the search route was specified on the map and the installation object was searched along the route. However, the area was specified on the map and the installation route was set along the road in the area. You may make it search for an installation comprehensively.
In this case, specifically, in the designated area, for example, while scanning the coordinate data on the line in the horizontal direction, it is searched whether or not the image of the corresponding installation object exists.

[Display of installation: Figure 2]
Next, a screen display of the searched installation object will be described with reference to FIG. FIG. 2 is a diagram showing a display example of installation objects in the present system.
As shown in FIG. 2, the control unit 11 of the installation object recognition device 1 displays the icon of the installation object searched for on the map data. The icons may be color-coded according to type and the size may be adjusted according to the size of the installation.
In FIG. 2, the letters A, B, and C in the icon indicate the company name or abbreviation of the beverage manufacturer, and the size of the icon indicates the size difference of the vending machine. The icons may be displayed in different colors for easy identification.

AI (Artificial Intelligence) is used to determine the installation object to be searched, and a large amount of image data of the installation installation object that serves as a sample is read in advance for learning to improve the recognition accuracy.

[Installation object search processing: Fig. 3]
Next, a process of searching for an installation in this system will be described with reference to FIG. FIG. 3 is a flowchart of the installation object search processing in this system.
When the control unit 11 of the installation object recognition device 1 reads the processing program from the storage unit 12 and executes it, the installation object search process is started.

First, the installation object to be searched is specified from the input unit 15 of the installation object recognition apparatus 1 (S11). Specifically, "soft drink water vending machine" or the like is designated. In the storage unit 12 of the installed object recognition device 1, a plurality of images of the vending machine for soft drinks are stored in advance as a search target, and the images are matched with the images on the route to be the same or similar. The image to be extracted will be extracted.

Next, the installation object recognition device 1 reads the map information of the search target from the map DB 3, displays it on the display unit 14, and the input unit 15 specifies the search route on the displayed map (S12).
The control unit 11 advances the search position along the route on the map (S13), reads the corresponding image data from the image DB2 from the coordinate data thus advanced, and executes the image recognition processing of the installation object on the route. (S14).

The image recognition processing of the installation object on the route determines whether or not the image corresponding to the coordinate data includes an image that is the same as or similar to the image of the installation object for search stored in advance in the storage unit 12. To do. This determination process may be performed using an AI learning model.
Specifically, deep learning is performed using the data searched and stored in the installation DB 5 as the image data of the installation as teacher data to generate an AI learning model for determining the installation. After that, if the determination process is performed using the learned learning model, the accuracy of the determination can be improved.

Then, by image recognition processing, it is determined whether or not there is a corresponding installation object (S15), and if there is a corresponding installation object (in the case of Yes), the image data of the corresponding installation object is stored in the installation object DB5 as position information ( It is stored in association with the coordinate data), the type and size of the installation object are further specified, and stored in the installation DB 5 as information of the installation object in association with the coordinate data (S16).
The type of the installed product is a manufacturer of soft drinks, and if the accuracy of the image recognition technology is improved, the type of the product sold and displayed by the pattern recognition of the image is included.
Further, the size of the installed object is the size of the vending machine.

Then, the control unit 11 determines whether or not the designated route is finished (S17), and when it is finished (Yes), the process is finished.
If the designated route has not been completed (No), the process returns to step S13.
Further, in the determination processing S15, if there is no corresponding installation object (No), the processing S16 is not performed and the processing proceeds to the processing S17.

[Icon display processing: FIG. 4]
Next, a process of displaying the corresponding icon on the map as a result of performing the installation object search process will be described with reference to FIG. FIG. 4 is a flowchart of the icon display processing in this system.
When the control unit 11 of the installation object recognition device 1 reads the processing program from the storage unit 12 and executes it, the processing is started, and the acquired installation object information stored in the storage unit 12 is read out to perform the classification processing (S21). ).

In the storage unit 12, icons are created and stored corresponding to the information of the installation items.
Since the installation information is the type and size of the installation, the control unit 11 selects the corresponding (corresponding) icon from the storage unit 12 (S22).

Then, the control unit 11 displays the icon of the selected (classified) installation object on the map displayed on the display unit 14 based on the position information (S23). The display example is as shown in FIG.

Further, the control unit 11 determines whether or not all the installation objects are displayed on the map (S24), and if all the installation objects are displayed (Yes), the process ends and all the installation objects are displayed. When is not displayed (in the case of No), the process returns to step S21, the information of the installation that has not been read is read, and the classification process is performed.

[application]
Although the example in which the installation object is a "vending machine" is shown, the installation object may be an "automobile". In that case, the shape of the car (sedans, coupe, trucks, etc.), manufacturing company, vehicle type, etc. are analyzed.
When the installed item is a “store”, the type of store (convenience store, restaurant, supermarket, sports gym, etc.), the scale of the store, the size of the parking lot, etc. are analyzed. Store analysis can be used in store opening strategies.

Further, the election poster can be analyzed by designating the election period. If the period is not designated, the poster that is always stuck can be identified and the distribution of the supporters can be analyzed.
Furthermore, instead of simply recognizing only the store, information around the store (people, cars, bicycles, motorcycles, bus stops, parking lots, vending machines, etc.) can be comprehensively analyzed and used for store opening strategies. ..
Alternatively, the route may be set in the order in which people walk on the road, the store image may be recognized and analyzed according to the route, and a store opening plan may be created based on the result.

In addition, it can be applied to grasp the objects installed on the road in a tourist area.
For example, when installing a rental cycle or the like, by searching for the installation space, the presence or absence of competing installations, etc., it can be used for the installation strategy of the rental cycle or the like.
Also, by searching for directions to major tourist facilities, it is possible to know whether the signs for directions are properly installed, and it can be used for the development of tourist areas.

[Collection of image data]
Collection of image data in this system will be specifically described.
The image data is captured by an image capturing device having a GPS function, and the image capturing device is mounted on a vehicle to capture an image along a road.
The imaging device has a GPS function, periodically acquires position information and time information based on a GPS reception signal, and acquires moving image data or still image (photo) data taken at specific intervals.
Although the position information is acquired from the GPS reception signal, the position information may be acquired by other communication means.

As the imaging device, a smartphone having a GPS function, a tablet, a drive recorder, or the like is assumed.
The photographing apparatus further has a connection function to the network 4 of FIG. 1, transmits photographed image data to the image DB 2 via the network 4, and the image data is stored in the image DB 2 together with time information and position information.
Then, the installation object recognizing apparatus 1 executes the installation object search process shown in FIG.

[Awareness of installation]
The image recognition of the installation object is performed by machine learning the image data of the still image of the installation object as teacher data and using the learned AI model from the image data of the still image or the image data of the moving image.
Since image data captured from a vehicle changes from a small shape in the distance to a large shape in the vicinity, the AI may learn the series of shape changes to recognize the installation object.

[Display of installation]
In FIG. 2, an example is shown in which the installation objects are displayed as icons by type, color, and size. However, a three-dimensional (3D: three dimensions) image is generated from the image data that recognizes the installation object, and the 3D image is displayed. You may use and display on a map. In this case, the map is a 3D simple map in accordance with the 3D image of the installation.
Since the 3D image has a depth, it is possible to measure the distance from the vehicle in which the image was taken. Therefore, a more accurate position of the installed object can be calculated based on the distance information and the direction of the installed object from the position information of the image capturing device.

[Layer management of each installation]
A plurality of types of target installations, for example, a vending machine, a gas station, a fast food restaurant, etc. are searched separately, and each set (layer) is stored in the installation DB 5, and a layer is arbitrarily set to 1 One or more may be selected and an icon or 3D image may be displayed on the map.
Accordingly, various installations can be selectively displayed on the map, and a plurality of types of installations can be combined and displayed on the map. This function can be used for marketing such as when installing a vending machine or a store.

[Recognition of non-installed objects]
Up to now, it has been shown that the installed object is recognized, but even if the pattern of the moving person, the bicycle, the vehicle, etc. is recognized instead of the installed object, it is possible to grasp the flow of people and the flow of vehicles. Good.
As a result, the place where people can easily gather is known from the flow of people, and it can be used for marketing. In addition, it is possible to recognize traffic congestion from the flow of cars and use it to alleviate traffic congestion.

[Effect of Embodiment]
According to this system, when the installation object recognition device 1 inputs an instruction to search for a specific installation object on a specified road (route) from image data of a landscape taken along the road, The image data of the installed object is searched to determine whether or not the installed object exists. When the installed object exists, the type is determined and the content is analyzed, and the characteristics of the installed object are grasped and displayed on the map. Since the icon corresponding to the type is displayed, the characteristic of the installed object can be displayed on the map by the icon, which is effective for the marketing strategy.

Further, according to the present system, a specific installation object is searched from the image data captured by the imaging device having the GPS function together with the position information, the type and characteristics of the installation object are analyzed, and the installation object is displayed on the two-dimensional map. Is displayed, or a 3D image of the installation object is displayed on a three-dimensional map, and the arrangement characteristics of the installation object can be easily recognized, and it is effective in marketing strategy.

INDUSTRIAL APPLICABILITY The present invention is suitable for an installation object recognition system and its program for identifying the type of installation object provided along a road and analyzing the content of the installation object.

DESCRIPTION OF SYMBOLS 1... Installed object recognition device, 2... Image database (image DB), 3... Map database (map DB), 4... Network, 11... Control part, 12... Storage part, 13... Interface part, 14... Display part, 15 … Input section

Claims (12)

An installation object recognition system including an installation object recognition device that searches for a specific installation object from image data of a landscape taken along a route,
When the route in the map information is designated and an instruction to search for the specific installation is given to the installation object recognition device, image data of the specific installation object exists in the image data of the landscape along the path. It is determined whether or not to do so, and when the image data of the specific installation object exists, the type of the specific installation object is determined, and the type of the specific installation object is stored together with position information. Installation recognition system. The installation object recognition device determines the size of the specific installation object when image data of the specific installation object is present, and stores the size of the specific installation object together with position information. The installation recognition system according to Item 1. The installation object recognition system according to claim 2, wherein the installation object recognition device displays an icon corresponding to the type and size of the specific installation object on the map displayed on the display unit according to the position information. The installation object recognition system according to claim 2, wherein the installation object recognition device displays a three-dimensional image corresponding to the type and size of the specific installation object on the map displayed on the display unit according to the position information. .. When the installation object recognition device determines that the image data of the specific installation object is present in the image data of the landscape, the installation object recognition device stores the image data portion of the image data of the landscape, performs machine learning, and uses the image data for the determination processing. The installation object recognition system according to any one of claims 1 to 4, characterized in that. The installation object recognition system according to any one of claims 1 to 5, wherein the image data of the installation object is captured by an imaging device that acquires position information. A program that operates with an installation object recognition device that searches for a specific installation object from image data of a landscape taken along a route,
When the route in the map information is designated by the installation object recognition device and an instruction to search for the specific installation object is given, image data of the specific installation object exists in the image data of the landscape along the path. If there is image data of the specific installation object, the type of the specific installation object is determined, and the type of the specific installation object is stored together with the position information. A program characterized by that. A feature is that the installation object recognizing device functions to determine the size of the specific installation object when image data of the specific installation object exists and store the size of the specific installation object together with position information. The program according to claim 7. 9. The program according to claim 8, wherein the installation object recognition device is caused to function so as to display an icon corresponding to the type and size of the specific installation object on the map displayed on the display unit according to the position information. 9. The program according to claim 8, wherein the installation object recognition device is caused to function so as to display a three-dimensional image corresponding to the type and size of a specific installation object on the map displayed on the display unit according to position information. .. When the installation object recognition device determines that the image data of the specific installation object exists in the image data of the landscape, the image data portion of the image data of the landscape is stored, machine learning is performed, and used in the determination processing. The program according to any one of claims 7 to 10, which is caused to function. The program according to any one of claims 7 to 11, wherein the image data of the installation object is imaged by an imaging device that acquires position information.

Images