CN114756634A

CN114756634A - Method and device for discovering interest point change, electronic equipment and storage medium

Info

Publication number: CN114756634A
Application number: CN202110025582.9A
Authority: CN
Inventors: 淦小健; 何俊; 宋凡; 章恒
Original assignee: Fengtu Technology Shenzhen Co Ltd
Current assignee: Fengtu Technology Shenzhen Co Ltd
Priority date: 2021-01-08
Filing date: 2021-01-08
Publication date: 2022-07-15

Abstract

The application provides a method and a device for discovering interest point change, electronic equipment and a computer readable storage medium. The interest point change discovery method comprises the following steps: acquiring a scene image of a street area and an acquisition position of the scene image; detecting a first image feature of a target interest point in the scene image; acquiring a second image characteristic of a reference interest point of which the geographic position is within a preset distance range of the acquisition position from a preset database; acquiring target similarity between the target interest point and the reference interest point according to the first image characteristic and the second image characteristic; and if the target similarity is smaller than a preset threshold value, determining that the interest point change exists in the street area in the pre-constructed interest point map. The method and the device can improve the detection precision of the interest point change discovery.

Description

Method and device for discovering change of interest point, electronic equipment and storage medium

Technical Field

The present application relates to the field of computer vision technologies, and in particular, to a method and an apparatus for discovering a change of a point of interest, an electronic device, and a computer-readable storage medium.

Background

A Point Of Interest (POI) may refer to a place such as a shop, a supermarket, an organization, and the like. In a real environment, the interest points are likely to be added or changed every day, so that the related information of the new interest points needs to be added in the interest point database so as to be presented on the electronic map.

In the prior art, when determining whether a newly found interest point is repeated, it depends on judging whether the name and address of the newly found interest point are repeated with the name and address of the interest point database, but because there are still a lot of interest points with the same name and errors in address acquisition, there may be a case that the address and name of the newly found interest point a are completely the same as those of the interest point B in the database, but the interest point a and the interest point B are not actually the same interest point. There may be a case where the newly found point of interest C is the same name and different address from the point of interest D in the database, but the point of interest C and the point of interest D are actually the same point of interest.

It can be seen that, because the new interest points cannot be found due to slight differences of addresses, the existing interest point finding method for determining whether nouns and addresses are repeated has a high requirement on the accuracy of obtaining addresses, and simply depends on whether comparison names and addresses are the same to detect whether interest points change, so that the detection accuracy is low.

Disclosure of Invention

The application provides a method and a device for discovering interest point changes, electronic equipment and a computer readable storage medium, aiming at solving the problem of low interest point change discovery precision because whether interest points change is repeatedly judged depending on names and addresses.

In a first aspect, the present application provides a method for discovering a change in a point of interest, the method comprising:

acquiring a scene image of a street area and an acquisition position of the scene image;

detecting a first image feature of a target interest point in the scene image;

acquiring a second image characteristic of a reference interest point of which the geographic position is within a preset distance range of the acquisition position from a preset database;

acquiring target similarity between the target interest point and the reference interest point according to the first image characteristic and the second image characteristic;

and if the target similarity is smaller than a preset threshold value, determining that the interest point change exists in the street area in the pre-constructed interest point map.

In a possible implementation manner of the present application, the detecting a first image feature of a target interest point in the scene image includes:

calling a pre-trained instance segmentation model, and carrying out interest point instance segmentation on the scene image to obtain an instance segmentation result, wherein the instance segmentation result comprises a target segmentation graph of the target interest point;

and performing feature extraction on the target segmentation graph to obtain a first feature vector of the target segmentation graph, wherein the first feature vector is used for representing a first image feature of the target interest point.

In a possible implementation manner of the present application, the example segmentation model is obtained by training a PSE-Net model, and the method further includes:

acquiring a sample data set, wherein the sample data set comprises a plurality of sample images and label data of each sample image, and the label data is used for indicating interest point information in the sample images;

and training the PSE-Net model according to the sample data set to obtain the example segmentation model.

In a possible implementation manner of the present application, the representing a second image feature of the reference interest point by a second feature vector, and the obtaining a target similarity between the target interest point and the reference interest point according to the first image feature and the second image feature includes:

detecting a cosine distance between the first feature vector and the second feature vector;

and taking the cosine distance as the target similarity.

In a possible implementation manner of the present application, if the target similarity is smaller than a preset threshold, it is determined that there is an interest point change in the along-street area in a pre-constructed interest point map, and then the method further includes:

adding the acquisition location and the first feature vector association to the database.

In one possible implementation manner of the present application, the method further includes:

and when the interest point change exists in the street area, updating the interest point map according to the target interest point and the acquisition position, wherein the interest point map is constructed by a plurality of interest points in the database.

when the interest point change in the street area is determined, acquiring a target reference interest point, wherein the target reference interest point is the reference interest point of which the target similarity with the target interest point is smaller than a preset threshold value;

and updating the second image characteristic of the target reference interest point in the database according to the first image characteristic.

In a possible implementation manner of the present application, the obtaining a target similarity between the target interest point and the reference interest point further includes:

and if the target similarity is greater than or equal to a preset threshold value, determining that no interest point change occurs in the street area in the interest point map, wherein the interest point map is constructed by a plurality of interest points in the database.

In a second aspect, the present application provides a point of interest change discovery apparatus, including:

the acquisition unit is used for acquiring a scene image of a street area and an acquisition position of the scene image;

the detection unit is used for detecting a first image characteristic of a target interest point in the scene image;

the acquisition unit is further used for acquiring a second image characteristic of a reference interest point of which the geographic position is within a preset distance range of the acquisition position from a preset database;

the acquiring unit is further configured to acquire a target similarity between the target interest point and the reference interest point according to the first image feature and the second image feature;

and the judging unit is used for determining that the interest point change exists in the along-street area in the pre-constructed interest point map if the target similarity is smaller than a preset threshold value.

In a possible implementation manner of the present application, the detection unit is specifically configured to:

calling a pre-trained example segmentation model, and carrying out interest point example segmentation on the scene image to obtain an example segmentation result, wherein the example segmentation result comprises a target segmentation graph of the target interest point;

In a possible implementation manner of the present application, the example segmentation model is obtained by training a PSE-Net model, and the interest point change discovery apparatus further includes a training unit, where the training unit is specifically configured to:

and training the PSE-Net model according to the sample data set to obtain the instance segmentation model.

In a possible implementation manner of the present application, the second image feature of the reference interest point is represented by a second feature vector, and the obtaining unit is specifically configured to:

and taking the cosine distance as the target similarity.

In a possible implementation manner of the present application, the device for discovering a change of an interest point further includes an updating unit, and after the step of determining that there is a change of an interest point along the street area in the pre-constructed interest point map is determined if the target similarity is smaller than a preset threshold, the updating unit is specifically configured to:

In a possible implementation manner of the present application, the updating unit is specifically configured to:

when the interest point change exists in the street area, a target reference interest point is obtained, wherein the target reference interest point is the reference interest point of which the target similarity with the target interest point is smaller than a preset threshold value;

In a possible implementation manner of the present application, the determining unit is specifically configured to:

In a third aspect, the present application further provides an electronic device, where the electronic device includes a processor and a memory, where the memory stores a computer program, and the processor executes, when calling the computer program in the memory, any of the steps in the method for discovering a change of point of interest provided in the present application.

In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is loaded by a processor to execute the steps in the method for discovering a change of interest.

According to the method and the device, the target similarity between the target interest point and the reference interest point is compared according to the first image characteristic of the target interest point and the second image characteristic of the reference interest point (the geographic position of which is within the preset distance range of the acquisition position of the target interest point), and whether the interest point in the street area in the pre-constructed interest point map changes is determined. On one hand, the method does not need to depend on whether the name and the address are repeatedly found, but judges whether the interest points are the same or not based on the image characteristics, so that the detection precision of the interest point change finding is improved. On the other hand, the interest points with the geographic positions within the preset distance range of the acquisition positions of the target interest points in the database are searched for similarity comparison, so that the problem of low detection precision of the change discovery of the interest points due to high requirement on the acquisition precision of the addresses is solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of a scene of a point of interest change discovery system according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a method for discovering a change of an interest point according to an embodiment of the present application;

FIG. 3 is a scene diagram of a point of interest in a scene image according to an embodiment of the present disclosure;

FIG. 4 is a schematic view of another scene of a point of interest in a scene image according to an embodiment of the present application;

FIG. 5 is a scene diagram of a point of interest with a geographic location within a preset distance range of a collection location according to an embodiment of the present disclosure;

FIG. 6 is a flowchart illustrating an embodiment of step 202 provided by embodiments of the present application;

FIG. 7 is a schematic diagram of a point of interest change discovery process provided by an embodiment of the present application;

fig. 8 is a schematic structural diagram of an embodiment of a point of interest change discovery apparatus provided in an embodiment of the present application;

fig. 9 is a schematic structural diagram of an embodiment of an electronic device provided in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the embodiments of the present application, it should be understood that the terms "first", "second", and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.

The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known processes have not been described in detail so as not to obscure the description of the embodiments of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the embodiments herein.

First, before describing the embodiments of the present application, the following description will be given of the embodiments of the present application with respect to the context of the application.

In the embodiment Of the present application, a Point Of Interest (POI) may refer to a place such as a bar, a gas station, a hospital, a station, a store, a supermarket, or a unit. For convenience of description, the interest point is also referred to as POI in the embodiments of the present application.

With the increasing level of material consumption of people, the POI along the street can be out of order and generate a great amount of changes, so that the POI address service along the street can be accurately achieved, and the demand is strong. In the existing POI acquisition process, a large amount of manpower and material resources are needed for supporting, POI along streets are efficiently acquired and updated in time, and the POI becomes a foundation stone for constructing a high-precision map system.

However, in the prior art, such map building often depends on the way of visiting and recording in the field, and reporting by merchants and complaints by users. The prior art has difficulty in automatically discovering new POI and online verification.

Based on the above-mentioned defects of the prior art, the embodiments of the present application provide a method for finding a change in a point of interest, which overcomes the defects of the prior art to some extent at least.

An execution main body of the method for discovering change of an interest point according to the embodiment of the present application may be the apparatus for discovering change of an interest point provided by the embodiment of the present application, or different types of electronic devices such as a server device, a physical host, or a User Equipment (UE) integrated with the apparatus for discovering change of an interest point, where the apparatus for discovering change of an interest point may be implemented in a hardware or software manner, and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a notebook computer, a palm computer, a desktop computer, or a Personal Digital Assistant (PDA).

The electronic device can adopt a working mode of independent operation or a working mode of a device cluster, and the detection accuracy of the interest point change discovery can be improved by applying the interest point change discovery method provided by the embodiment of the application.

Referring to fig. 1, fig. 1 is a schematic view of a scene of a point of interest change discovery system according to an embodiment of the present application. The interest point change discovery system may include an electronic device 100, and an interest point change discovery apparatus is integrated in the electronic device 100. For example, the electronic device may acquire a scene image of a street area, and an acquisition position of the scene image; detecting a first image feature of a target interest point in the scene image; acquiring a second image characteristic of a reference interest point of which the geographic position is within a preset distance range of the acquisition position from a preset database; acquiring target similarity between the target interest point and the reference interest point according to the first image characteristic and the second image characteristic; and if the target similarity is smaller than a preset threshold value, determining that the interest point change exists in the street area in the pre-constructed interest point map.

In addition, as shown in fig. 1, the system for discovering change of interest may further include a memory 200 for storing data, such as image data and video data.

It should be noted that the scene schematic diagram of the interest point change discovery system shown in fig. 1 is only an example, and the interest point change discovery system and the scene described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not form a limitation on the technical solution provided in the embodiment of the present application.

In the following, a point of interest change discovery method provided in an embodiment of the present application is described, where an electronic device is used as an execution subject, and for simplicity and convenience of description, the execution subject will be omitted in subsequent embodiments of the method, and the method includes: acquiring a scene image of a street area and an acquisition position of the scene image; detecting a first image feature of a target interest point in the scene image; acquiring a second image characteristic of a reference interest point of which the geographic position is within a preset distance range of the acquisition position from a preset database; acquiring target similarity between the target interest point and the reference interest point according to the first image characteristic and the second image characteristic; and if the target similarity is smaller than a preset threshold value, determining that the interest point change exists in the street area in the pre-constructed interest point map.

Referring to fig. 2, fig. 2 is a schematic flowchart of a method for discovering a change of a point of interest according to an embodiment of the present application. It should be noted that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in an order different from that shown or described herein. The interest point change discovery method comprises the following steps of 201-205:

201. and acquiring a scene image of the street area and an acquisition position of the scene image.

Maps are graphics or images that represent, according to certain rules, several phenomena of the earth (or other stars) on a plane or sphere, optionally in two or more dimensions and means. For convenience of description, in the embodiments of the present application, a map that can be used to display a POI is referred to as a POI map for short.

In order to facilitate building of the POI map, in the embodiment of the present application, images of each area are collected in advance, POIs of each area are detected, and the collected POIs and geographical positions of the POIs are recorded in a preset database. For example, in order to display the POI in the shenzhen southern mountain area on the POI map, a camera and a position acquisition device, such as a GPS (Global Positioning System) acquisition device, a beidou System acquisition device, etc., may be installed on the patrol car, and the patrol car is used to take a picture or a video in the shenzhen southern mountain area and record the acquisition position of the picture or the video through the GPS, so as to complete the acquisition of the POI and the geographic position of the POI in the shenzhen southern mountain area, and store the acquired POI and the geographic position of the POI in a preset database in an associated manner.

Then, a point-of-interest map (in the embodiment of the present application, the point-of-interest map is also referred to as a POI map for short) is constructed through the POIs recorded in the preset database and the geographical location of each POI.

After the POI map is constructed, POI changes, such as POI addition, POI removal or POI change, may still occur in a certain street area. Therefore, whether the POI changes or not is detected by acquiring the scene image of the street area again in the embodiment of the application.

Along a street area should be understood as a geographical area, and not as a street in the narrow sense.

The acquisition position refers to a geographical position where the image acquisition equipment is located when acquiring the scene image.

Specifically, in practical application, by applying the electronic device provided by the embodiment of the present application, a camera (the camera is mainly used for collecting images along a street area) can be directly included in hardware, and the images shot by the camera are locally stored and can be directly read in the electronic device; or the electronic equipment can also establish network connection with the camera and acquire the image obtained by the camera on line from the camera according to the network connection; alternatively, the electronic device may also read the image captured by the camera from a related storage medium storing the image captured by the camera, and the specific acquisition mode is not limited herein.

The camera can shoot images according to a preset shooting mode, for example, shooting height, shooting direction or shooting distance can be set, the specific shooting mode can be adjusted according to the camera, and the camera is not limited specifically. The multi-frame images shot by the camera can form a video through a time line.

Furthermore, a position acquisition device, such as a GPS acquisition device, may be integrated into the camera so as to record a GPS position of scene image acquisition while acquiring a scene image of the street area.

Further, in order to ensure the accuracy of the POI map, an orientation acquisition device can be further integrated in the camera, so that the orientation of the POI can be recorded when the images of the area along the street are acquired.

202. A first image feature of a target point of interest in the scene image is detected.

The target interest point refers to a POI carried in a scene image. The first image feature refers to a spatial feature of an example segmentation map of the target point of interest.

In some embodiments, since one scene image may include multiple POIs, the capturing and recording of multiple POIs may be implemented by using one scene image, where the target interest point refers to each POI carried in the scene image. As shown in fig. 3, fig. 3 is a scene schematic diagram of a point of interest in a scene image according to an embodiment of the present application. For example, if a certain street area includes a mall a, a supermarket b, a hospital c, and an office building d in the scene image, the mall a, the supermarket b, the hospital c, and the office building d all belong to the target interest point, and 4 POIs in the mall a, the supermarket b, the hospital c, and the office building d can be detected respectively.

In some embodiments, one scene image is only used for recording one POI, and in this case, the target point of interest refers to one POI closest to the camera in geographic position from among one or more POIs carried in the scene image, so as to improve accuracy of the geographic position of the recorded POI. As shown in fig. 4, fig. 4 is another scene schematic diagram of a point of interest in a scene image according to an embodiment of the present application. For example, 4 POIs including bank a, florist B, supermarket C and supermarket D are carried in the scene image in fig. 4, where one POI closest to the camera is bank a, bank a may be used as the target point of interest.

For convenience of description, in the embodiments of the present application, an example in which one scene image is used to record only one POI is described. It can be understood that, in the case where one scene image is used to record a plurality of POIs, the purpose of detecting whether the POIs change can be achieved by performing similar data processing.

In particular, the scene image may be segmented by an example segmentation algorithm to detect various points of interest in the scene image. The detection method of the target interest point will be described in detail later, and will not be described herein again.

203. And acquiring a second image characteristic of the reference interest point of which the geographic position is within a preset distance range of the acquisition position from a preset database.

The database records a plurality of points of interest collected in advance and the geographic position of each point of interest. The second image feature refers to a spatial feature of an example segmentation map of the reference point of interest.

The reference interest point refers to an interest point of which the geographic position is within a preset distance range of the acquisition position of the scene image.

As shown in fig. 5, fig. 5 is a scene schematic diagram of a point of interest whose geographic location is within a preset distance range of an acquisition location according to an embodiment of the present application. For example, the capture position of the scene image is position a, and the preset distance range is within 50 meters, then the points of interest (POI1, POI3, POI5) whose geographic positions are within 50 meters of position a are all reference points of interest.

Here, the preset distance range is only an example, and may be specifically adjusted according to actual requirements, and is not limited thereto. For example, the distance may be within 5 meters, within 10 meters, or within 20 meters.

204. And acquiring the target similarity between the target interest point and the reference interest point according to the first image characteristic and the second image characteristic.

The target similarity refers to the similarity between the target interest point and the reference interest point.

In particular, a target similarity between the target point of interest and the reference point of interest may be determined by a similarity measure. For example, the cosine distance of the image features between the target interest point and the reference interest point may be calculated, and the similarity between the target interest point and the reference interest point may be determined by calculating the cosine distance of the image features between the target interest point and the reference interest point.

Similarity measure, a measure that comprehensively assesses how close two things are. The similarity measure may have various expressions, for example, cosine distance, jaccard similarity, etc.

205. And if the target similarity is smaller than a preset threshold value, determining that the interest point change exists in the street area in the pre-constructed interest point map.

In some embodiments, the number of the reference interest points is one, and at this time, if the target similarity is smaller than the preset threshold, it is proved that the target interest point does not exist in the interest point map, and it may be directly determined that the interest point change exists along the street area in the pre-constructed interest point map (the target interest point may be a new interest point or an original interest point is replaced). If the similarity of the target is greater than or equal to the preset threshold value, the fact that the target interest point exists in the interest point map is proved, and then it can be directly determined that the interest points along the street area in the preset interest point map are not changed.

For example, the acquisition place of a scene image a along a street area is a place a, and the geographic position of the POI3 in the POI recorded in the database is within 50 meters of the place a. If the similarity (i.e., the target similarity) between the point of interest 1 in the scene image a and the POI3 is 90% and the preset threshold is 95%, it may be determined that there is a change in the POI along the street area in the pre-constructed point of interest map.

In some embodiments, if the number of the reference interest points is greater than one, then in step 204, the target similarity between the target interest point and each reference interest point is respectively compared. In step 205, if the target similarity between the target interest point and each reference interest point is smaller than the preset threshold, it is proved that no target interest point exists in the interest point map, and it is determined that there is an interest point change (the target interest point may be a new or replaced original interest point) along the street area in the pre-constructed interest point map. If the target similarity between the target interest point and at least one reference interest point in the plurality of reference interest points is larger than or equal to a preset threshold value, the fact that the target interest point exists in the interest point map is proved, and it is determined that the interest points along the street area in the pre-constructed interest point map are not changed.

For example, the acquisition location of the scene image a along the street area is a place a, and the geographic positions of the POI1, the POI2 and the POI3 in the POIs recorded in the database are within 50 meters of the place a. If the similarity (i.e., the target similarity) between the point of interest 1 in the scene image a and the POI1, 2, and 3 is 90%, 80%, and 10%, respectively, and the preset threshold is 95%, it may be determined that there is a change in the POI along the street area in the pre-constructed point of interest map. If the similarity (i.e., the target similarity) between the point of interest 1 in the scene image a and the POI1, 2, and 3 is 96%, 80%, and 10%, respectively, and the preset threshold is 95%, it may be determined that there is a change in the POI along the street area in the pre-constructed point of interest map.

From the above, according to the first image feature of the target interest point and the second image feature of the reference interest point (the geographic location of which is within the preset distance range of the acquisition location of the target interest point), the target similarity between the target interest point and the reference interest point is compared, so as to determine whether interest points along the street area in the pre-constructed interest point map change. On one hand, the method does not need to depend on whether the name and the address are repeatedly found, but judges whether the interest points are the same or not based on the image characteristics, so that the detection precision of the interest point change finding is improved. On the other hand, the interest points with the geographic positions within the preset distance range of the acquisition positions of the target interest points in the database are searched for similarity comparison, so that the problem of low detection precision of the change discovery of the interest points due to high requirement on the acquisition precision of the addresses is solved.

Further, after it is determined that there is a change in the interest point along the street area in the pre-constructed interest point map (i.e., the POI map), in order to facilitate the next detection of whether the POI has changed, the collection position of the scene image is taken as the geographic position where the target interest point is located, and the target interest point and the collection position of the scene image are stored in the database in an associated manner. In order to improve the accuracy of the POI map, the POI map may be updated according to the target interest points and the acquisition positions of the scene images.

Further, in order to improve the accuracy of the POI map, the POI map may be updated according to the target interest points and the acquisition positions of the scene images. Namely, the method for discovering the change of the interest point may further include: and when the interest point change exists in the street area, updating the interest point map according to the target interest point and the acquisition position, wherein the interest point map is constructed by a plurality of interest points in the database.

Further, in order to improve the accuracy of the POI map, whether a POI with the same geographic location as the acquisition location exists in the database may be further detected, and if the POI exists, it is proved that the change of the point of interest is: and if the original POI is replaced by the target interest point, adding the target interest point and the collection position of the scene image to the database in a correlation manner, and deleting the POI recorded in the collection position in the database so as to prevent the same geographical position on the POI map from reflecting a plurality of POIs. If not, the change of the interest point is proved as follows: and if the target interest point is a newly added POI in the street area, directly adding the target interest point and the acquisition position of the scene image into the database in a correlated manner. Further, when the target interest point and the acquisition position of the scene image are added to the database in a correlated manner, the scene image itself or the scene image subjected to processing such as image distortion correction can be added to the database in a correlated manner together with the acquisition position of the target interest point and the acquisition position of the scene image, so that feature comparison can be performed on the basis of the scene image recorded in the database and a new scene image on the subsequent aspect. On the other hand, when the map is constructed by using the interest points in the database, the map can be constructed by using the scene images in the database, and the scene images of the interest points can be shown to the user in the constructed map.

Further, in order to improve the efficiency of acquiring the target similarity between the target interest point and the reference interest point, when it is determined that the interest point change exists in the street area, the target reference interest point is acquired; and updating the second image characteristic of the target reference interest point in the database according to the first image characteristic. The target reference interest point refers to a reference interest point of which the target similarity with the target interest point is smaller than a preset threshold. Specifically, the second image feature of the target reference interest point stored in the database is replaced by the first image feature, so that the updated image feature of the target interest point is obtained. Therefore, the similarity comparison can be directly carried out between the updated image feature of the target reference interest point in the database and the first image feature of the new target interest point in the follow-up process, and the similarity between the target reference interest point and the new target interest point can be rapidly determined. And image feature extraction is not required to be carried out on the target reference interest points again, so that the speed of finding the change of the interest points is improved to a certain extent.

In some embodiments of the present application, the points of interest may be represented by feature vectors. At this time, as shown in fig. 6, step 202 may specifically include steps 601 to 602, where:

601. and calling a pre-trained example segmentation model, and carrying out interest point example segmentation on the scene image to obtain an example segmentation result, wherein the example segmentation result comprises a target segmentation graph of the target interest point.

The example segmentation result refers to a segmentation map of interest points in the scene image. The target segmentation graph is a segmentation graph obtained after example segmentation is carried out on the target interest points.

In one embodiment, the scene image contains only one interest point, and the example segmentation result refers to a segmentation map of one POI in the scene image. For example, only one POI1 in the scene image is found, and a segmentation map of the POI1 can be obtained by example segmentation.

In one embodiment, the scene image includes a plurality of interest points, and the example segmentation result refers to a segmentation map of each POI in the scene image. For example, if there are two interest points of POI2 and POI3 in the scene image, a segmentation map of POI2 and a segmentation map of POI3 can be obtained by example segmentation, respectively.

In this embodiment, only one interest point (i.e. only one target interest point) is included in the scene image as an example, unless otherwise specified.

Illustratively, the example segmentation model may be trained by:

(1) the method comprises the steps of obtaining a sample data set, wherein the sample data set comprises a plurality of sample images and label data of each sample image, and the label data is used for indicating interest point information in the sample images.

Specifically, a plurality of images along the street area may be acquired as sample images by the image acquisition manner in step 201, and the POI in each sample image is labeled to obtain a labeled sample data set.

(2) And constructing an initial model.

Specifically, an open source model (such as a Mask R-CNN model, a PSE-Net model, and the like) which can be used for an instance segmentation task can be adopted as an initial model to be trained.

(3) And training the initial model by adopting a sample data set to obtain a trained embodiment segmentation model. At this time, the trained example segmentation model may perform POI example segmentation according to the image, so as to segment the POI in the image.

The training process of the example segmentation model is similar to that of the conventional network model, and reference may be made to the existing training mode of the network model, which is not described in detail herein.

For example, the example segmentation model is obtained by training a PSE-Net model by adopting a sample data set, and then the trained PSE-Net model is used for carrying out interest point example segmentation on a scene image, so that a target segmentation graph of a target interest point is obtained.

The PSE-Net model is that a plurality of kernel (namely, kernel) with different scales of each POI instance are obtained through semantic segmentation based on the pixel level, then the kernel with the minimum scale is gradually expanded to the POI instance with a complete shape through a progressive scale expansion algorithm, and finally a segmentation graph of each POI instance is obtained through segmentation. Where kernels represent feature maps (i.e., segmentation maps), each kernels shares a similar shape with the original POI instance and they are all located at the same center point but in different proportions.

Because the mask regions (namely mask regions) of adjacent POIs are often partially overlapped, and the shrinking mask regions (PSE-Net model is based on gradually expanding the minimum-scale kernel to the POI examples with complete shapes through a progressive scale expanding algorithm, and can realize the shrinking of the mask regions of the POIs) of the adjacent POIs are not overlapped, the problem of overlapping the mask regions is well solved by adopting the trained PSE-Net model to segment the examples, the segmentation precision of the target interest points is improved, and the detection precision of the change discovery of the interest points is further improved.

602. And performing feature extraction on the target segmentation graph to obtain a first feature vector of the target segmentation graph, wherein the first feature vector is used for representing a first image feature of the target interest point.

In one embodiment, feature extraction is performed on the target segmentation graph through an Encoding model to obtain a first feature vector of the target segmentation graph.

The Encoding model is based on a fastreID (fastreID, Pythrch toolbox for re-identifying general instances) re-identification toolbox, and global features extracted from CNN are subjected to metric learning by using a tiplet loss function and a cirle loss function, so that feature vectors with a good POI distinguishing effect are learned as Encoding representation of each POI.

For example, the target segmentation graph of the target interest point is a segmentation graph a, feature extraction is performed on the segmentation graph a through an Encoding model, and feature vectors of the segmentation graph a are respectively: a feature vector a. The first feature vector is referred to as feature vector a.

It can be seen from the above contents that after the example segmentation is performed on the target interest point, the feature vector is adopted to perform feature representation, so that the features of the target interest point can be accurately reflected; by converting the similarity detection between the interest points into the similarity detection between the feature vectors, the detection precision of the target similarity can be improved to a certain extent, and the precision of the change discovery of the interest points is further improved.

Correspondingly, in some embodiments of the present application, each POI point in the preset database may also be represented by a feature vector, similar to the representation of the target POI point. I.e. the first image feature of the reference point of interest is represented by the second feature vector. At this time, the target similarity between the target interest point and the reference interest point is obtained by comparing the similarities of the first feature vector and the second feature vector. In this case, step 204 may specifically include: detecting a cosine distance between the first feature vector and the second feature vector; and taking the cosine distance as the target similarity.

For example, a cosine distance between the first feature vector and the second feature vector may be detected as a target similarity between the target interest point and the reference interest point. The cosine distance is calculated as shown in the following formula (1):

a, B respectively represents a first feature vector and a second feature vector, dist (A, B) represents the cosine distance between the first feature vector and the second feature vector, the range of dist (A, B) is [0,2], the closer dist (A, B) is to 0, the higher the similarity between A and B is.

In the prior art, when determining whether a newly found interest point is repeated, it is determined whether names and addresses of the newly found interest point and the interest point database are repeated, but there are still a plurality of interest points with similar or identical addresses, and it is determined whether the interest point is changed depending on whether the names and the addresses are repeated, so that the accuracy of finding the change of the interest point is low.

According to the method and the device, the target segmentation graph of the target interest point is obtained by segmenting the scene image by the interest point instance, whether the interest point changes or not is judged by comparing the similarity between the feature vector of the target segmentation graph and the feature vector of the interest point in the database, whether the interest point changes or not can be avoided depending on whether the name and the address of the newly found interest point and the interest point database are repeated or not, and the accuracy of finding the change of the interest point is improved to a certain extent.

Further, in order to detect whether the POI is changed or not next time, the similarity comparison can be directly performed between the feature vector of the interest point and the feature vector of the target interest point without repeatedly determining the feature vector of the interest point in the database every time the similarity comparison is performed, and after it is determined that the interest point changes along the street area in the pre-constructed interest point map (i.e., POI map), the collection position of the scene image is taken as the geographic position where the target interest point is located, and the collection position of the scene image and the target interest point are added to the database in a correlated manner.

In order to better understand the embodiments of the present application, a specific scenario example is described. Referring to fig. 7, fig. 7 is a schematic diagram of a process for discovering a change of interest according to an embodiment of the present application. In this example, the process of finding a change in a point of interest will be described by taking an example of "identifying a POI located in an image from an image collected by an external acquisition device, then searching POIs stored in a database and located within 50 meters near the position according to the position information of the acquisition device, determining whether the currently located POI matches with a POI in the database through an image matching algorithm, if so, indicating that the POI is verified (still), and if not, indicating that the current POI is newly added".

1. And acquiring a scene image of a street area through external acquisition equipment, and taking the GPS position when the external acquisition equipment acquires the scene image as the acquisition position of the scene image.

2. And detecting a target interest point in the scene image through a PSE-Net model, and carrying out example segmentation on the target interest point to obtain a segmentation graph of the target interest point. The detailed implementation process of the example segmentation by the PSE-Net model can refer to the above description, and is not described herein again.

3. And performing characteristic vector representation on the target segmentation graph through an Encoding model to obtain a first characteristic vector for representing the target interest point.

4. And searching out a feature vector, namely a second feature vector, of which the geographic position is within 50 meters of the GPS position from a POI feature vector library (namely a preset database).

5. And (4) calculating cosine distances between the first characteristic vector and the second characteristic vector respectively determined in the step (3) and the step (4) to obtain the similarity between the target interest point and each interest point in the database.

6. And if the similarity between the target interest point and each interest point in the database is smaller than a preset threshold, the target interest point is proved to be the newly added POI. If the similarity between the target interest point and at least one interest point in the database is greater than or equal to a preset threshold value, the existence of the target interest point is proved, and if yes, whether the POI existing in the database is changed or not can be verified in a similarity comparison mode.

In order to better implement the method for discovering a change in an interest point in the embodiment of the present application, on the basis of the method for discovering a change in an interest point, an apparatus for discovering a change in an interest point is further provided in the embodiment of the present application, as shown in fig. 800, which is a schematic structural diagram of an embodiment of the apparatus for discovering a change in an interest point in the embodiment of the present application, and the apparatus 800 for discovering a change in an interest point includes:

an obtaining unit 801, configured to obtain a scene image of a street area and an acquisition position of the scene image;

a detecting unit 802, configured to detect a first image feature of a target interest point in the scene image;

the obtaining unit 801 is further configured to obtain, from a preset database, a second image feature of a reference interest point whose geographic position is within a preset distance range of the acquisition position;

the obtaining unit 801 is further configured to obtain a target similarity between the target interest point and the reference interest point according to the first image feature and the second image feature;

a determining unit 803, configured to determine that there is an interest point change in the along-street area in the pre-constructed interest point map if the target similarity is smaller than a preset threshold.

In some embodiments of the present application, the detecting unit 802 is specifically configured to:

In some embodiments of the present application, the example segmentation model is obtained by training a PSE-Net model, and the interest point change discovering device 800 further includes a training unit (not shown in the figure), and the training unit is specifically configured to:

In some embodiments of the present application, the second image feature of the reference interest point is represented by a second feature vector, and the obtaining unit 801 is specifically configured to:

and taking the cosine distance as the target similarity.

In some embodiments of the present application, the apparatus 800 for discovering a change of interest point further includes an updating unit, and after the step of determining that there is a change of interest point along the street area in the pre-constructed map of interest points if the target similarity is smaller than a preset threshold, the updating unit (not shown in the figure) is specifically configured to:

In some embodiments of the present application, the updating unit is specifically configured to:

In some embodiments of the present application, the determining unit 803 is specifically configured to:

In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.

Since the interest point change discovery apparatus may execute the steps in the interest point change discovery method in any embodiment corresponding to fig. 1 to 7, the beneficial effects that can be achieved by the interest point change discovery method in any embodiment corresponding to fig. 1 to 7 in the present application can be achieved, which are detailed in the foregoing description and will not be repeated herein.

In addition, in order to better implement the method for discovering a change of an interest point in the embodiment of the present application, on the basis of the method for discovering a change of an interest point, an embodiment of the present application further provides an electronic device, referring to fig. 9, where fig. 9 shows a schematic structural diagram of the electronic device in the embodiment of the present application, specifically, the electronic device provided in the embodiment of the present application includes a processor 901, and when the processor 901 is used to execute a computer program stored in a memory 902, the steps of the method for discovering a change of an interest point in any embodiment corresponding to fig. 1 to 7 are implemented; alternatively, the processor 901 is configured to implement the functions of the units in the corresponding embodiment of fig. 8 when executing the computer program stored in the memory 902.

Illustratively, a computer program may be partitioned into one or more modules/units, which are stored in the memory 902 and executed by the processor 901 to implement embodiments of the present application. One or more modules/units may be a series of computer program instruction segments capable of performing certain functions, the instruction segments being used to describe the execution of a computer program in a computer device.

The electronic device may include, but is not limited to, a processor 901, a memory 902. Those skilled in the art will appreciate that the illustration is merely an example of an electronic device, and does not constitute a limitation of the electronic device, and may include more or less components than those illustrated, or combine some components, or different components, for example, the electronic device may further include an input output device, a network access device, a bus, etc., and the processor 901, the memory 902, the input output device, the network access device, etc., are connected via the bus.

The Processor 901 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being the control center for the electronic device and various interfaces and lines connecting the various parts of the overall electronic device.

The memory 902 may be used to store computer programs and/or modules, and the processor 901 implements various functions of the computer device by running or executing the computer programs and/or modules stored in the memory 902 and invoking data stored in the memory 902. The memory 902 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like; the storage data area may store data (such as audio data, video data, etc.) created according to the use of the electronic device, etc. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.

It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described interest point change discovering device, the electronic device and the corresponding units thereof may refer to the description of the interest point change discovering method in any embodiment corresponding to fig. 1 to 7, and are not repeated herein.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer-readable storage medium, where a plurality of instructions are stored, where the instructions can be loaded by a processor to execute steps in the method for discovering a change of an interest point in any embodiment of the present application, as shown in fig. 1 to 7, for specific operations, reference may be made to descriptions of the method for discovering a change of an interest point in any embodiment of fig. 1 to 7, which are not described herein again.

Wherein the computer-readable storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

Because the instructions stored in the computer-readable storage medium can execute the steps in the method for discovering a change in point of interest in any embodiment of the present application, as shown in fig. 1 to 7, the beneficial effects that can be achieved by the method for discovering a change in point of interest in any embodiment of the present application, as shown in fig. 1 to 7, can be achieved, which are detailed in the foregoing description and will not be repeated herein.

The method, the apparatus, the electronic device, and the computer-readable storage medium for discovering a change of an interest point provided in the embodiments of the present application are described in detail above, and a specific example is applied in the description to explain the principles and embodiments of the present application, and the description of the embodiments is only used to help understanding the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method for discovering changes in a point of interest, the method comprising:

detecting a first image feature of a target interest point in the scene image;

2. The method of finding a change in interest point according to claim 1, wherein the detecting a first image feature of a target interest point in the scene image comprises:

3. The method of claim 2, wherein the instance segmentation model is trained from a PSE-Net model, the method further comprising:

4. The method for finding change in interest points according to claim 2, wherein the second image feature of the reference interest point is represented by a second feature vector, and the obtaining of the target similarity between the target interest point and the reference interest point according to the first image feature and the second image feature comprises:

and taking the cosine distance as the target similarity.

5. The method for discovering changes in interest points according to claim 2, wherein if the target similarity is smaller than a preset threshold, it is determined that there is a change in interest points along the street area in the pre-constructed interest point map, and then the method further comprises:

6. The method of finding a change in a point of interest according to claim 1, further comprising:

7. The method of finding a change in a point of interest according to claim 1, further comprising:

8. The method according to claim 1, wherein the obtaining of the target similarity between the target interest point and the reference interest point further comprises:

9. A point of interest change discovery apparatus, characterized in that said point of interest change discovery apparatus comprises:

a detection unit, configured to detect a first image feature of a target interest point in the scene image;

the obtaining unit is further configured to obtain a target similarity between the target interest point and the reference interest point according to the first image feature and the second image feature;

10. An electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the method for discovering a change in point of interest according to any one of claims 1 to 8 when calling the computer program in the memory.