JP6951782B2 - Installation object recognition system and its program - Google Patents

Description

The present invention relates to a system for recognizing an installed object provided along a road, and more particularly to an installed object recognition system capable of identifying the type of the installed object and analyzing the contents of the installed object and a program thereof.

[Conventional technology]
Conventionally, there has been a system that recognizes an object from an image taken by a camera.
In the image recognition technology, the pattern of the image to be detected is recognized in advance by the image processing device of the system, the correlation between the image pattern and the image taken by the camera is measured, and the captured image having a high correlation is the image to be detected. There is something to judge.

[Related technology]
As related prior art documents, Japanese Patent Application Laid-Open No. 2018-163654 “System and Method for Management of Telecommunications Event” (Patent Document 1) and Japanese Patent No. 5462609 “Stop Line Recognition Device” (Patent Document 2). ), Patent No. 6411933, "Vehicle Condition Determination Device" (Patent Document 3).

Patent Document 1 describes that a street view image and GPS (Global Positioning System) position information are acquired, the target image is model-learned by deep learning, and the target object is recognized.
Patent Document 2 describes a device that recognizes a stop line on a road when an in-vehicle camera recognizes a red light.
Patent Document 3 describes a device that recognizes the number of lanes on a road from map information and determines a vehicle that stops in a roadside zone on the road.

JP-A-2018-163654 Japanese Patent No. 5462609 Japanese Patent No. 6411933

However, in the above-mentioned conventional system, although the object is recognized from the image taken by the camera, the type and contents of the object are not analyzed, and the characteristics of the object cannot be easily grasped. There was a problem.

It should be noted that Patent Documents 1 to 3 do not describe recognition of the contents of the installed object and grasping its characteristics.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an installation object recognition system and a program thereof for identifying the type of installation object provided along the road and analyzing the contents of the installation object. do.

(Installed object recognition system)
The present invention for solving the above-mentioned problems of the conventional example is an installation object recognition system including an installation object recognition device that searches for a specific installation object from image data of a landscape taken along a route, and is an installation object. When a route in the map information is specified and an instruction to search for a specific installation is given, the recognition device determines whether or not the image data of the specific installation exists in the image data of the landscape along the route. When image data of a specific installation object exists, the type of the specific installation object is determined, and the type of the specific installation object is stored together with the position information.

(Determine the size of the installation)
In the above-mentioned installation object recognition system, when the installation object recognition device has image data of a specific installation object, the present invention determines the size of the specific installation object, and together with the position information, the size of the specific installation object. It is characterized by memorizing.

(Icon display of installation object)
The present invention is characterized in that, in the above-mentioned installation object recognition system, the installation object recognition device displays an icon corresponding to a specific installation object type and size on a map displayed on a display unit according to position information. ..

(3D image display of the installed object)
The present invention is characterized in that, in the above-mentioned installation object recognition system, the installation object recognition device displays a three-dimensional image according to the type and size of a specific installation object on a map displayed on a display unit according to position information. And.

(Use of machine learning)
In the above-mentioned installation object recognition system, when the installation object recognition device determines that the image data of a specific installation object exists in the landscape image data, the present invention stores the image data portion in the landscape image data. It is characterized by being machine-learned and used for judgment processing.

(Image data from a shooting device that acquires location information)
The present invention is characterized in that, in the installation object recognition system, the image data of the installation object is photographed by an imaging device that acquires position information.

(Installation recognition program)
The present invention is a program that operates in an installation object recognition device that searches for a specific installation object from image data of a landscape taken along a route, and specifies the installation object recognition device by designating a route in map information. When an instruction to search for an installation is given, it is made to determine whether or not the image data of the specific installation exists in the image data of the landscape along the route, and when the image data of the specific installation exists. Is characterized in that it functions to determine the type of a specific installation object and store the type of the specific installation object together with the position information.

(Program for determining the size of the installed object)
In the above program, the present invention causes the installation object recognition device to determine the size of a specific installation object when image data of the specific installation object exists, and to store the size of the specific installation object together with the position information. It is characterized by making it function.

(Icon display program)
The present invention is characterized in that, in the above program, the installation object recognition device functions to display an icon corresponding to a specific installation type and size on a map displayed on a display unit according to position information. ..

(3D image display program)
The present invention is characterized in that, in the above program, the installation object recognition device functions to display a three-dimensional image corresponding to the type and size of a specific installation object according to the position information on the map displayed on the display unit. And.

(Program for using machine learning)
In the above program, when the installation object recognition device determines that the image data of a specific installation object exists in the landscape image data, the present invention causes the installation object recognition device to store the image data portion of the landscape image data and perform machine learning. , It is characterized in that it functions to be used for judgment processing.

(Image data from a shooting device that acquires location information)
The present invention is characterized in that, in the above program, the image data of the installed object is photographed by an imaging device that acquires position information.

According to the present invention, when the installation object recognition device specifies a route in the map information and is instructed to search for a specific installation object, the image data of the specific installation object becomes the image data of the landscape along the route. As an installation recognition system that determines whether or not it exists, and if image data of a specific installation exists, determines the type of the specific installation and stores the type of the specific installation together with the position information. Therefore, the characteristics of the installation can be easily grasped from the landscape image, which has the effect of being utilized in marketing strategies.

It is a schematic diagram of this system. It is a figure which shows the display example of the installation object in this system. It is a flow chart of the installation object search process in this system. It is a flow chart of the icon display processing in this system.

Embodiments of the present invention will be described with reference to the drawings.
[Outline of Embodiment]
In the installation object recognition system (this system) according to the embodiment of the present invention, an instruction to search for a specific installation object is input for a designated road (route) from image data of a landscape taken along the road. Then, the image data of the installation object is searched, it is determined whether or not the installation object exists, and if there is the installation object, the type is determined and the contents are analyzed, and the characteristics of the installation object are grasped. Then, icons according to the type etc. are displayed on the map, and the characteristics of the installed object can be displayed on the map with the icon so that the characteristics of the installed object can be easily grasped, which can be utilized for marketing strategy. ..

In addition, this system detects a specific installation object from the image data taken with the position information by a photographing device having a GPS function, analyzes the type and characteristics of the installation object, and displays the installation object icon on a two-dimensional map. Is displayed, or 3D image data of the installation object is displayed on a three-dimensional map, and the arrangement characteristics of the installation object can be easily recognized and can be used for marketing strategies.

[This system: Fig. 1]
This system will be described with reference to FIG. FIG. 1 is a schematic view of this system.
As shown in FIG. 1, this system includes an installation object recognition device 1, an image database (DB) 2, a map database (DB) 3, a network 4, and an installation object database (DB) 5. ..

The image DB 2 is a storage unit that stores image data of a landscape taken along a road, corresponding to coordinate data.
The map DB 3 is a storage unit that stores map information (map data) corresponding to coordinate data.
The image DB 2 and the map DB 3 may be integrated.

The network 4 connects the installation object recognition device 1, the image DB 2, the map DB 3, and the installation object DB 5, and assumes the Internet.
The installation object DB 5 is a storage unit that stores the image data of the searched installation object and related information (type, size, etc. of the installation object) described later as information on the installation object in association with the position information.

The installation object recognition device 1 is a computer processing device, and includes a control unit 11, a storage unit 12, an interface unit 13, a display unit 14, and an input unit 15.
The control unit 11 operates the processing program stored in the storage unit 12 to execute the processing described later.
The storage unit 12 stores a processing program and other necessary data.
The interface unit 13 is connected to the network 4, the display unit 14, and the input unit 15.
The display unit 14 displays the processing result and the like.
The input unit 15 inputs an instruction or the like for performing the process.

[Processing content]
The processing contents in the installation object recognition device 1 will be described.
The control unit 11 reads the processing program from the storage unit 12 and causes the display unit 14 to display the map data of the corresponding area from the map DB 3 according to the instruction from the input unit 15.
When the road to be searched is specified for the displayed map data and the installation object to be searched (for example, a vending machine for soft drinks) is set, the control unit 11 determines the landscape image data corresponding to the road. Is read from the image DB2, and it is determined whether or not there is an installation object to be searched.
Since the road is designated in the map data by specifying the coordinate data, the corresponding landscape image data is also specified in the image DB2 based on the coordinate data.

Next, when the control unit 11 determines that there is an installation object to be searched, the control unit 11 analyzes the type of the installation object (beverage maker) and the size of the installation object (large, medium, and small), and further, the accuracy of image recognition. If it can be raised, it is possible to recognize and analyze the products handled from the image pattern of the installed object.

In the above process, the search route was specified on the map and the installation was searched along the route. However, the area was specified on the map and the search route was provided along the road in the area. You may try to search the installation comprehensively.
In this case, specifically, in the designated area, for example, while scanning the coordinate data in the lateral direction on the line, it is searched for whether or not the image of the corresponding installation object exists.

[Display of installation: Fig. 2]
Next, the screen display of the searched installation object will be described with reference to FIG. FIG. 2 is a diagram showing a display example of an installation object in this system.
As shown in FIG. 2, the control unit 11 of the installation object recognition device 1 displays the icon of the installed object searched on the map data. The icons may be color-coded according to type and the size may be adjusted according to the size of the installation object.
In FIG. 2, the letters A, B, and C in the icon indicate the company name or abbreviation of the beverage manufacturer, and the size of the icon indicates the difference in the size of the vending machine. The icons may be color-coded to make them easier to identify.

AI (Artificial Intelligence) is used to determine the installation object to be searched by reading a large amount of image data of the installation object as a sample in advance for learning to improve the recognition accuracy.

[Search process for installations: Fig. 3]
Next, the search process for the installed object in this system will be described with reference to FIG. FIG. 3 is a flow chart of an installation object search process in this system.
When the control unit 11 of the installation object recognition device 1 reads the processing program from the storage unit 12 and executes it, the installation object search process is started.

First, the input unit 15 of the installation object recognition device 1 specifies the installation object to be searched (S11). Specifically, a designation such as "a vending machine for soft drinks" is made. A plurality of images of the soft drink vending machine are stored in advance in the storage unit 12 of the installation object recognition device 1 for search targets, and these images are matched with the images on the route to be the same or similar. The image to be used will be extracted.

Next, the installation object recognition device 1 reads the map information to be searched from the map DB3, displays it on the display unit 14, and specifies the search route from the input unit 15 on the displayed map (S12).
The control unit 11 advances the search position along the route on the map (S13), reads the corresponding image data from the advanced coordinate data from the image DB 2, and executes the image recognition process of the installation object on the route. (S14).

The image recognition process of the installation object on the route determines whether or not the image corresponding to the coordinate data includes an image that is the same as or similar to the image of the installation object for the search target stored in advance in the storage unit 12. do. This determination process may be performed using the learning model of AI.
Specifically, the data that has been searched and stored in the installation object DB5 as image data of the installation object is deep-learned as teacher data, and an AI learning model for determining the installation object is generated. After that, if the determination process is performed using the trained learning model, the accuracy of the determination can be improved.

Then, by the image recognition process, it is determined whether or not there is a corresponding installation object (S15), and if there is a corresponding installation object (in the case of Yes), the image data of the corresponding installation object is stored in the installation object DB 5 as position information (in the case of Yes). It is stored in association with the coordinate data), further, the type and size of the installation object are specified, and is stored in the installation object DB 5 as information on the installation object in association with the coordinate data (S16).
The type of installation is a manufacturer of soft drinks, and if the accuracy of image recognition technology is improved, the type of product sold and displayed by image pattern recognition is included.
The size of the installed object is the size of the vending machine.

Then, the control unit 11 determines whether or not the designated route has been completed (S17), and when it has ended (in the case of Yes), the process ends.
If the specified route has not been completed (No), the process returns to process S13.
Further, in the determination process S15, when there is no corresponding installation object (in the case of No), the process S16 is not performed and the process proceeds to the process S17.

[Icon display processing: Fig. 4]
Next, the process of displaying the corresponding icon on the map as a result of the search process of the installed object will be described with reference to FIG. FIG. 4 is a flow chart of icon display processing in this system.
When the control unit 11 of the installation object recognition device 1 reads the processing program from the storage unit 12 and executes it, the processing is started, the acquired installation information stored in the storage unit 12 is read out, and the classification process is performed (S21). ).

An icon is created and stored in the storage unit 12 in accordance with the information of the installed object.
Since the information on the installation object is the type and size of the installation object, the control unit 11 selects the corresponding (corresponding) icon from the storage unit 12 (S22).

Then, the control unit 11 displays the icon of the installed object selected (classified) on the map displayed on the display unit 14 based on the position information (S23). A display example is as shown in FIG.

Further, the control unit 11 determines whether or not all the installed objects are displayed on the map (S24), and if all the installed objects are displayed (in the case of Yes), the process is completed and all the installed objects are displayed. Is not displayed (in the case of No), the process returns to process S21, and the information of the installed objects that have not been read is read and the classification process is performed.

[application]
Although the installation object has shown an example of a "vending machine", the installation object may be an "automobile". In that case, the shape of the car (sedan, coupe, truck, etc.), manufacturer, car model, etc. are analyzed.
If the installed object is a "store", the type of store (convenience store, restaurant, supermarket, sports gym, etc.), the size of the store, the size of the parking lot, etc. are analyzed. Store analysis can be used for store opening strategies.

Election posters can also be analyzed by specifying the election period, and if the period is not specified, the posters that are always posted may be identified and the distribution of supporters may be analyzed.
Furthermore, it is possible to comprehensively analyze information around the store (people, cars, bicycles, motorcycles, bus stops, parking lots, vending machines, etc.) and use it for store opening strategies, rather than simply recognizing the store. ..
Further, the route may be set in the order in which people walk on the road, the store image may be recognized and analyzed according to the route, and the store opening plan may be created based on the result.

It can also be applied to grasping road installations in tourist spots.
For example, when installing a rental cycle or the like, it can be utilized for an installation strategy such as a rental cycle by searching for the presence or absence of an installation space, the presence or absence of competing installations, and the like.
In addition, by searching for directions to major tourist facilities, it is possible to grasp whether or not the signboards for directions are properly installed, which can be used for the development of tourist spots.

[Collection of image data]
In this system, the collection of image data will be specifically described.
The image data is taken by a shooting device having a GPS function, and the shooting device is mounted on a vehicle to shoot along the road.
The photographing device has a GPS function, periodically acquires position information and time information based on a GPS received signal, and acquires moving image data or still image (photograph) data taken at specific intervals.
Although the position information is acquired from the GPS received signal, the position information may be acquired by other communication means.

As the photographing device, a smartphone, a tablet, a drive recorder, or the like having a GPS function is assumed.
The photographing device further includes a connection function to the network 4 of FIG. 1, transmits the photographed image data to the image DB 2 via the network 4, and stores the image data in the image DB 2 together with the time information and the position information.
Then, the installation object recognition device 1 executes the installation object search process shown in FIG.

[Recognition of installed objects]
Image recognition of the installation object is performed by machine learning the image data of the still image of the installation object as teacher data by AI and using the learned AI model from the image data of the still image or the image data of the moving image.
Since the image data taken from the car changes from a small shape in the distance of the installation object to a large shape in the vicinity, AI may be made to learn the series of shape changes to recognize the installation object.

[Display of installation]
In FIG. 2, an example in which the installed objects are displayed as icons by type, color, and size is shown. However, a three-dimensional (3D) image is generated from the image data recognizing the installed object, and the 3D image is displayed. It may be used and displayed on a map. In this case, the map is a simple 3D map according to the 3D image of the installed object.
Since the 3D image has a depth, it is possible to measure the distance from the photographed car. Therefore, it is possible to calculate a more accurate position of the installation object based on the distance and direction from the position information of the photographing device that has been photographed to the installation object.

[Layer management of each installation]
Multiple types of target installations, such as vending machines, gas stations, fast food stores, etc., are searched separately, and each set (layer) is stored in the installation DB5, and one layer is arbitrarily set. One or more may be selected to display an icon or a 3D image on the map.
As a result, various installation objects can be selectively displayed on the map, and a plurality of types of installation objects can be combined and displayed on the map. With this function, it can be used for marketing such as when installing vending machines and stores.

[Recognition other than installation]
So far, we have shown that we recognize installed objects, but even if we try to recognize patterns of moving people, bicycles, vehicles, etc. instead of installed objects, we can grasp the flow of people and the flow of cars. good.
As a result, the place where people can easily gather can be known from the flow of people, and it can be used for marketing. In addition, it is possible to recognize traffic congestion from the flow of vehicles and use it to alleviate traffic congestion.

[Effect of Embodiment]
According to this system, when the installation object recognition device 1 inputs an instruction to search for a specific installation object on a specified road (route) from the image data of the landscape taken along the road, the said Search the image data of the installation object, determine whether or not there is the installation object, if there is the installation object, determine the type and analyze the contents, grasp the characteristics of the installation object and put it on the map Since the icons are displayed according to the type, etc., the characteristics of the installed object can be displayed on the map as icons, which has the effect of being useful for marketing strategies.

In addition, according to this system, a specific installation object is searched for from image data taken with position information by a photographing device having a GPS function, the type and characteristics of the installation object are analyzed, and the installation object is displayed on a two-dimensional map. The icon is displayed or a 3D image of the installation object is displayed on a three-dimensional map, and the arrangement characteristics of the installation object can be easily recognized, which has the effect of being used in marketing strategies.

The present invention is suitable for an installation recognition system and a program thereof that identifies the type of installation installed along the road and analyzes the contents of the installation.

1 ... Installation object recognition device, 2 ... Image database (image DB), 3 ... Map database (map DB), 4 ... Network, 11 ... Control unit, 12 ... Storage unit, 13 ... Interface unit, 14 ... Display unit, 15 … Input section

Claims (12)

It is an installation object recognition system equipped with an installation object recognition device that searches for a specific installation object from image data of landscapes taken along the route.
When a route in the map information is specified in the installation object recognition device and an instruction to search for the specific installation object is given, the image data of the specific installation object exists in the image data of the landscape along the route. It is characterized in that it is determined whether or not to perform, and if the image data of the specific installation object exists, the type of the specific installation object is determined, and the type of the specific installation object is stored together with the position information. Installation object recognition system. The claim is characterized in that the installation object recognition device determines the size of the specific installation object when the image data of the specific installation object exists, and stores the size of the specific installation object together with the position information. Item 1. Installation object recognition system according to item 1. The installation object recognition system according to claim 2, wherein the installation object recognition device displays an icon according to the type and size of a specific installation object on a map displayed on a display unit according to position information. The installation object recognition system according to claim 2, wherein the installation object recognition device displays a three-dimensional image according to the type and size of a specific installation object on a map displayed on a display unit according to position information. .. When the installation object recognition device determines that the image data of a specific installation object exists in the landscape image data, the installation object recognition device stores the image data portion of the landscape image data, performs machine learning, and uses it for the determination process. The installation object recognition system according to any one of claims 1 to 4, wherein the installation object recognition system is characterized. The installation object recognition system according to any one of claims 1 to 5, wherein the image data of the installation object is taken by a photographing device that acquires position information. A program that runs on an installation recognition device that searches for a specific installation from image data of landscapes taken along the route.
When a route in the map information is specified for the installation object recognition device and an instruction to search for the specific installation object is given, the image data of the specific installation object exists in the image data of the landscape along the route. If there is image data of the specific installation object, the type of the specific installation object is determined, and the function is to store the type of the specific installation object together with the position information. A program characterized by that. The feature is that the installation object recognition device functions to determine the size of the specific installation object when image data of the specific installation object exists and to store the size of the specific installation object together with the position information. The program according to claim 7. The program according to claim 8, wherein the installation object recognition device functions to display an icon corresponding to a specific installation type and size on a map displayed on a display unit according to position information. The program according to claim 8, wherein the installation object recognition device functions to display a three-dimensional image corresponding to a specific installation type and size on a map displayed on a display unit according to position information. .. When the installation object recognition device determines that the image data of a specific installation object exists in the image data of the landscape, the image data portion of the image data of the landscape is stored and machine-learned to be used for the determination process. The program according to any one of claims 7 to 10, characterized in that it functions. The program according to any one of claims 7 to 11, wherein the image data of the installed object is taken by a photographing device that acquires position information.

Images