CN109214280B - Shop identification method and device based on street view, electronic equipment and storage medium - Google Patents
Shop identification method and device based on street view, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN109214280B CN109214280B CN201810844680.3A CN201810844680A CN109214280B CN 109214280 B CN109214280 B CN 109214280B CN 201810844680 A CN201810844680 A CN 201810844680A CN 109214280 B CN109214280 B CN 109214280B
- Authority
- CN
- China
- Prior art keywords
- shop
- image
- street view
- store
- operation state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The shop identification method based on the street view belongs to the technical field of computers and is beneficial to improving the efficiency of determining the shop state. The shop identification method based on the street view provided by the embodiment of the disclosure comprises the following steps: street view information acquired by street view information acquisition equipment is acquired, wherein the street view information at least comprises: geographic location and street view images; acquiring a first shop image in the street view image; and matching the acquired first shop image with a second shop image in a preset shop image set according to the geographic position, and determining the operation state of the corresponding shop according to the matching result. According to the shop identification method based on the street view, the shop image acquired by the street view information acquisition equipment is used for identifying the shop based on multi-modal characteristics such as geographic position, shop image characteristics and the like, so that the efficiency of determining the shop state is improved.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to a shop identification method and device based on street view, electronic equipment and a storage medium.
Background
With the development of take-out business, merchant store management is one of important business modules at a take-out platform end, and the platform needs to update the current state of a store in time, such as the on-line state and the off-line state of the store. Through research on the prior art, the inventor of the present disclosure finds that shop management is mainly submitted by a merchant, and then an operator at a take-out platform end periodically visits or checks, updates the state of the shop in time, and determines whether the shop exists. In the prior art, a large amount of labor and time cost are consumed in a mode of manually determining the shop state, the efficiency is low, and the shop state is not updated timely.
Disclosure of Invention
The embodiment of the disclosure provides a shop identification method based on a street view, which can automatically identify the state of a shop and is beneficial to improving the efficiency of determining the state of the shop.
In a first aspect, a shop identification method based on street view includes:
street view information acquired by street view information acquisition equipment is acquired, wherein the street view information at least comprises: geographic location and street view images;
acquiring a first shop image in the street view image;
according to the geographic position, matching the acquired first shop image with a second shop image in a preset shop image set;
and determining the operation state of the corresponding shop according to the matching result.
In a second aspect, an embodiment of the present disclosure provides a street view-based shop recognition apparatus, including:
street view information acquisition module for acquire the street view information that street view information acquisition equipment gathered, street view information includes at least: geographic location and street view images;
the shop image acquisition module is used for acquiring a first shop image in the street view image;
the shop identification module is used for matching the acquired first shop image with a second shop image in a preset shop image set according to the geographic position;
and the shop management module is used for determining the operation state of the corresponding shop according to the matching result.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the street view-based store identification method according to the embodiment of the present disclosure.
In a fourth aspect, the disclosed embodiments provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the street view-based store identification method described in the disclosed embodiments.
According to the shop identification method based on street view provided by the embodiment of the disclosure, street view information acquired by street view information acquisition equipment is acquired, and the street view information at least comprises the following steps: geographic location and street view images; then, acquiring a first shop image in the street view image; and according to the geographic position, the acquired first shop image is matched with a second shop image in a preset shop image set, and the operation state of the corresponding shop is determined according to the matching result, so that the state of the shop can be automatically identified, and the efficiency of determining the shop state is improved.
Drawings
To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
Fig. 1 is a flowchart of a street view-based store identification method according to a first embodiment of the present disclosure;
fig. 2 is a flowchart of a street view-based store identification method according to a second embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a shop recognition device based on street view according to a third embodiment of the present disclosure;
fig. 4 is a second schematic structural diagram of a shop recognition apparatus based on a street view according to a third embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
Example one
As shown in fig. 1, a shop identification method based on street view according to an embodiment of the present disclosure includes: step 110 to step 140.
With the development of take-out business, each street can see that take-out distributors take and deliver take-out by riding a bicycle or a motorcycle, and the street view information acquisition equipment in the embodiment of the disclosure can be image acquisition equipment carried by the take-out distributors in the current take-out distribution scene, and can also be take-out distribution robots and the like which may appear in the future along with the development of the take-out business. For convenience of understanding, in the embodiment of the present disclosure, a take-away delivery person carries an image capturing device to capture street view images, which exemplifies a specific scheme of the present disclosure. Correspondingly, the street view information acquisition equipment of the take-out delivery personnel can be equipment with an image acquisition function, such as a camera, a camera and an intelligent terminal, and the image acquisition equipment can be in the form of a head-mounted type, a vehicle-mounted type or a handheld type.
In some embodiments of the present disclosure, the street view information collecting device may collect street view information according to the control of the takeaway distributor, for example, collecting street view information after receiving the key trigger of the takeaway distributor; street view information may also be collected based on the motion status of the take-away shipper, for example, street view information may be collected automatically after detecting that the movement speed of the take-away shipper has decreased to a predetermined speed threshold (e.g., 1 meter per second).
In other embodiments of the disclosure, the obtaining street view information collected by the street view information collecting device includes: acquiring the real-time geographical position of street view information acquisition equipment; and triggering the street view information acquisition equipment to acquire the street view image when the real-time geographic position is determined to be in the preset range.
For example, the takeout platform first controls the street view information collecting equipment to collect relevant information of the shop requiring the state updating when the street view information collecting equipment arrives at the street or the geographical position according to the geographical position or street information of the shop requiring the state updating in the shop database to be identified. The preset range is determined according to the geographic position or the street of the shop with the state needing to be updated. Then, the takeout platform acquires the real-time geographic position of street view information acquisition equipment of the takeout rider, matches the acquired real-time geographic position with a preset range, and when the real-time geographic position is within the preset range, the takeout platform sends a control instruction to the street view information acquisition equipment to control the street view information acquisition equipment to acquire street view information of the current position, namely to acquire the street view information of the preset range. The street view information includes, but is not limited to, a captured street view image and a real-time geographic location of the street view information capture device (i.e., a store location in the captured street view image).
And step 120, acquiring a first shop image in the street view image.
In some embodiments of the disclosure, the street view image may include images of a plurality of shops or may not include images of shops, and before performing shop matching, an image of each shop to be matched in the street view image needs to be acquired first, which is referred to as a "first shop image" in this embodiment.
In some embodiments of the present application, the first shop image in the street view image may be obtained by using edge detection, image recognition, and other technologies in the prior art.
And step 130, matching the acquired first shop image with a second shop image in a preset shop image set according to the geographic position.
In the implementation of the present disclosure, a store that needs to be managed, that is, a store that needs to be updated, is first identified. In some embodiments of the present disclosure, the shop in which the status needs to be updated is determined by means of a preset shop image set, wherein the preset shop image set includes: a full store image; or each shop image matched with the geographic position. The whole amount of shops refer to images of all shops needing to be checked on the takeout platform; each store image matching the geographic position may be an image of a store at the geographic position, or may be an image of a store within a preset range around the geographic position. In the embodiment of the disclosure, the images in the preset shop image set are referred to as "second shop images", and each second shop image is associated with a corresponding geographic location and has a geographic location attribute. The geographic position attribute of the second shop image is the geographic position of the shop corresponding to the second shop image. The shop images matched with the geographic positions are selected for matching, so that the matching data volume can be reduced, and the efficiency of determining the shop states is improved. The matching of the acquired first shop image with a second shop image in a preset shop image set according to the geographic position comprises: determining a position characteristic distance between a geographic position in the street view information and a geographic position corresponding to a second shop image and an image characteristic distance between the first shop image and the second shop image for each acquired first shop image and each second shop image in a preset shop image set; and determining a second shop image matched with the first shop image through a combined distance obtained by weighting and summing the position characteristic distance and the image characteristic distance. Wherein the image feature distance between the first store image and the second store image comprises: a visual feature distance and/or a text feature distance between the first store image and the second store image.
In some embodiments of the present disclosure, the matching, according to the geographic location, the acquired first shop image with a second shop image in a preset shop image set includes: determining a position characteristic distance between a geographic position in the street view information and a geographic position corresponding to a second shop image, a visual characteristic distance between the first shop image and the second shop image, and a text characteristic distance between the first shop image and the second shop image for each acquired first shop image and each second shop image in a preset shop image set; and determining a second shop image matched with the first shop image by a combined distance obtained by weighting and summing the position characteristic distance, the visual characteristic distance and the text characteristic distance. For example, the joint distance between a first store image in the street view image and a second store image in the preset set of store images may be determined by the following formula:
D(Object1,Object2)＝λpDposition(Object1,Object2)+λvDvisual(Object1,Object2)+λeDedit(Object1,Object2)；
wherein Object is1And Object2Respectively a first shop image and a second shop image; dposition(Object1,Object2) Representing a first store image and a second store in the street view imageA location feature distance; dvisual(Object1,Object2) A visual feature distance representing the first and second store images; dedit(Object1,Object2) A text feature distance representing the first and second store images; lambda [ alpha ]p、λvAnd λeAnd respectively representing the weight values of the position characteristic distance, the visual characteristic distance and the text characteristic distance, and determining the value of each weight value according to the test result.
In some embodiments of the present disclosure, the location-feature distance may be a straight-line distance (i.e., L2 distance) between a geographic location of the first store image in the street view image (i.e., a geographic location where the street view image is captured and a geographic location of a store corresponding to the first store image) and a geographic location of the second store image (i.e., a geographic location associated with the second store image and a geographic location where the store is located in the second store image), or may be a street-block distance (i.e., L1 distance) between geographic locations of the first store image and the second store image in the street view image. For example, the geographic location characteristics of the first store image and the second store image in the street view image are calculated by the following formula:
wherein, LatobjectAnd LngobjectRepresenting the latitude and longitude coordinates of the store.
In some embodiments of the present disclosure, the visual feature distance may be calculated by a cosine distance or a euclidean distance. For example, the visual feature distance between the visual feature of the first store image and the visual feature of the second store image in the street view image is calculated by the following formula:
wherein, Vobject,iRepresenting image objecti-dimensional visual features.
In some embodiments of the present disclosure, the visual feature is a feature expressing the overall visual effect of the image, and may be any one or combination of color feature, texture feature, shape feature, and the like; the visual features may also be other image features. Wherein, the visual features can pass through the extraction algorithm of the corresponding features in the prior art. Such as: extracting texture features in the street view image through a wavelet-based texture feature extraction algorithm, and acquiring color features in the street view image by calculating distribution features of a color space.
In some preferred embodiments of the present disclosure, the visual features of the first store image in the street view image are obtained by a convolutional neural network-based. For example, the street view image to be processed is first divided into 7 × 7 grids, and then the convolutional neural network is trained using the following loss function:
wherein the content of the first and second substances,
indicates whether there is a store, x, in the first gridiyiwihiciThe deviation of the abscissa, the ordinate, the width and the height of the ith in-grid shop and the probability of containing the shop are represented; x is the number ofi′yi′wi′hi′ci' is the output of the convolutional neural network. Wherein x isi、yi、wi、hiAnd ciMay be determined by manual measurements on training samples.
In the training process of the convolutional neural network, the convolutional neural network can be trained by taking an image labeled with the coordinates and the size of the shop as a training sample and taking the minimum loss function as a target. After the convolutional neural network training is completed, the output of any one feature representation layer of the convolutional neural network can be selected as the visual feature of the input streetscape image.
In the embodiment of the disclosure, for a street view image to be identified, firstly, dividing the street view image into a plurality of grids, and determining whether the street view image comprises a shop image; then, the convolutional neural network is trained based on the loss function. When the trained convolutional neural network is used to acquire the visual feature of the first shop image in the input street view image, preferably, the output of the penultimate feature representation layer of the convolutional neural network is selected as the visual feature of the input street view image, so that the detailed visual feature can be acquired, and the acquired visual feature data volume is not too large, and the operation efficiency is not reduced.
In other embodiments of the present disclosure, the street view image may be divided into other number of grids, or a convolutional neural network may be trained by using other loss functions, so as to obtain the visual feature of the first shop image in the street view image, which is not limited by the present disclosure.
In some embodiments of the present disclosure, the text feature distance may be represented by a minimum edit distance. For example, the text feature distance between the text feature of the first shop image and the text feature of the second shop image in the street view image is calculated by the following formula:
Because OCR algorithms have errors and it is not suitable to use text features directly for matching, the disclosed embodiments improve the stability of text feature metrics by using the minimum edit distance.
In other embodiments of the present disclosure, the textual feature may be a store name, address, phone, etc. of the store. Wherein the text feature can be obtained by recognizing a text region in the street view image through an OCR (Optical Character Recognition) Recognition algorithm. For example, a shop name recognition model is first trained by an image labeled with a shop name; then, determining the area of the shop name in the street view image through the shop name identification model; and finally, identifying the region where the shop name is located in the street view image through an OCR algorithm, and identifying the shop name in the street view image. In other embodiments of the present disclosure, the Chinese text feature of the first shop image may also be obtained by other methods, and the present disclosure does not limit the technical means for obtaining the text feature of the first shop image in the street view image.
In determining the joint distance between the first shop image in the street view image collected by the street view image collecting device of the takeout distributor and each second shop image in the preset database, the second shop image in which the joint distance between the second shop image and the first shop image in the collected street view image is the smallest and the joint distance satisfies the preset distance threshold may be further determined as the second shop image in the street view image, that is, the shop in the first shop image is the same as the shop in the second shop image.
And step 140, determining the operation state of the corresponding shop according to the matching result.
In some embodiments of the present application, the determining an operation status of the corresponding shop according to the matching result includes: when the matching result is that the matching is successful, determining that the shop corresponding to the second shop image matched with the first shop image is in a normal operation state; or when the matching failure times of the first shop images collected at the same geographic position reach preset times, determining that the shop corresponding to the first shop images and/or the shop corresponding to the second shop images at the same geographic position are in an abnormal operation state. For example, when it is determined by matching that a first shop image in a street view image is successfully matched with a certain second shop image, it may be determined that a shop corresponding to the second shop image in a preset shop image set is a shop in a normal operation state, and the operation state of the shop corresponding to the second shop image may be further updated to be normal. If the minimum joint distance does not satisfy the preset distance threshold, it may be assumed that the first shop image in the street view images currently acquired by the image acquisition device of the take-out distributor is not in the preset shop image set. And further, whether the operation state of the shop corresponding to the second shop image matched with the geographic position is an abnormal operation state or not is determined by counting the matching results of the shops in the shop images collected at the geographic position.
In some embodiments of the present application, in the case where knowledge matching fails in the matching result, that is, the second shop image is not matched, the case where the second shop image is not matched may be counted based on the geographic location. For example, when the first store image captured at the geographic position a is matched and identified with the second store image, every time the first store image captured at the geographic position a does not match with the second store image, 1 is added to the case where the geographic position a does not match with the second store image, and when the cumulative number of times of the case where the geographic position a does not match with the second store image exceeds a preset number of times (for example, 10 times), it is considered that there is no store at the geographic position a in the second store image, and therefore, there is a possibility that a store at the geographic position a in the store corresponding to the second store image is a store in an abnormal operation state. In a specific application process, the reasons for the abnormal state include: the shop corresponding to the second shop image located at the geographic position A is already closed, and is replaced by another shop; alternatively, the store located at the geographic position a (i.e., the store corresponding to the first store image) is a new-operation store, and store information is not registered in the takeout platform. In some embodiments of the application, when it is determined that the second shop with the matched geographic position is an abnormal shop, prompt information that the second shop with the matched geographic position is the abnormal shop may be output, so that staff can check and update the state of the shop in time.
In other embodiments of the present disclosure, the second shop image matched with the first shop image in the collected street view images may be determined only according to a joint distance obtained by weighted summation of the position feature distance and the visual feature distance, or the second shop image matched with the first shop image in the collected street view images may be determined only according to a joint distance obtained by weighted summation of the position feature distance and the text feature distance.
According to the shop identification method based on street view provided by the embodiment of the disclosure, street view information acquired by street view information acquisition equipment is acquired, and the street view information at least comprises the following steps: geographic location and street view images; then, acquiring a first shop image in the street view image; and according to the geographic position, the acquired first shop image is matched with a second shop image in a preset shop image set, and the operation state of the corresponding shop is determined according to the matching result, so that the state of the shop can be automatically determined, and the efficiency of determining the shop state is improved. Furthermore, due to the fact that the abnormal stores are determined in time, states of the abnormal stores can be checked and updated in time, and timeliness of updating of the states of the stores is improved. After the abnormal shop is determined, only the state of the abnormal shop needs to be checked, so that the workload is reduced, and the efficiency of updating the shop state can be improved. According to the shop identification method based on the street view, the shop images collected by the street view information collection equipment of the takeout distributor automatically identify the shop states based on multi-modal characteristics such as geographic positions and the characteristics of the shop images, and the shop identification method based on the street view is beneficial to improving the efficiency of determining the shop states. Furthermore, the similarity distance between shops is determined based on multi-modal characteristics such as geographic positions, visual characteristics and/or text characteristics, the considered characteristic dimension is more comprehensive, the distance measurement mode is more stable, and the recognition is more accurate.
Example two
The embodiment of the disclosure provides a shop identification method based on street view, as shown in fig. 2, the method includes: step 210 to step 270.
For obtaining street view information collected by the street view information collection device, refer to the first embodiment, which is not described in detail in this embodiment.
In some embodiments of the disclosure, before the acquiring the first shop image in the street view image, the method further includes: inputting the street view image into a pre-trained image classifier; when the determination result of the image classifier indicates that the street view image includes the shop image, the step of acquiring the first shop image in the street view image is performed.
Since the street view image acquisition device of the takeout distributor may acquire street view images during the movement and may continuously acquire the street view images, street view images not including shop images may exist in the acquired street view images, and in order to improve the identification efficiency, it is first determined whether a shop image is included in the acquired street view images before a first shop image in the street view images is acquired.
In some embodiments of the present disclosure, a certain number of street view image samples may be collected by street view image collection equipment of a takeaway distributor, and manually labeled, and then classification features may be extracted based on the labeled street view image samples to train a classification model. For example, the LBP feature of each street view image sample, that is, the size relationship between each pixel and its surrounding pixels, can be extracted; then, an SVM classifier is trained based on the extracted LBP features and sample labels corresponding to the LBP features to determine whether the shop is included in the streetscape image.
In order to improve the efficiency of store detection in the images, a linear SVM classifier is adopted in the embodiment of the disclosure, and training data of the classifier is derived from street view images and non-street view images which are already labeled with samples. Wherein the linear SVM model can be represented as:
subject to yn(wTxn+b)≥1for all n；
where n denotes the index of the training sample, ynRepresents the nth trainingClass of training sample (i.e., street view image including store or non-street view image not including store), xnFor the LBP feature of the nth sample, w and b are model parameters, respectively the weight and bias of the feature combination, i.e. the model parameters.
After the classifier training is completed, the obtained street view image collected by the street view information collection equipment of the take-out delivery staff is input into the SVM classifier, and then whether the street view image comprises the classification result of the shop or not can be output. When the shop is included in the street view image, the characteristic extraction and identification of the highlighted shop image can be further performed on the street view image; and when the street view image does not comprise the shop, discarding the currently acquired street view image, and re-acquiring the street view image in the street view information acquired by the street view information acquisition equipment of the take-out distributor.
And step 230, performing histogram equalization processing on the street view image.
In some embodiments of the disclosure, before the performing acquiring the first shop image in the street view image, the performing further includes: and carrying out histogram equalization processing on the street view image.
Because the illumination conditions of the scenes are varied when street view images are collected by street view image collecting equipment of a takeaway distributor, in order to adapt to possible different illumination conditions in a real scene, before the characteristics of the shop images are extracted, histogram equalization operation needs to be carried out on the collected street view images, and the stability of the extracted characteristics of the shop images can be improved.
The histogram equalization processing on the street view image may adopt a histogram equalization method in the prior art, and this embodiment is not described in detail.
For a specific implementation of obtaining the first shop image in the street view image, reference is made to the first embodiment, and details are not described in this embodiment.
And step 250, matching the acquired first shop image with a second shop image in a preset shop image set according to the geographic position.
According to the geographic position, a specific implementation manner of matching the acquired first store image with a second store image in a preset store image set is referred to in the first embodiment, and details are not repeated in this embodiment.
And step 260, determining the operation state of the corresponding shop according to the matching result.
For a specific implementation manner of determining the operation state of the corresponding store according to the matching result, reference is made to the first embodiment, and details are not described in this embodiment.
And 270, outputting the first operation state information through the man-machine interaction interface.
In some embodiments of the present disclosure, after the step of determining the operation status of the corresponding shop according to the matching result, the method further includes: when the matching result is that the matching fails, outputting first operation state information through a man-machine interaction interface; the first operation state information is used for indicating the abnormal operation of the shop corresponding to the matching result.
For example, a first shop image in the street view images collected at the geographic location a for multiple times is not matched with a second shop image in the preset shop image set, and at this time, the shop corresponding to the first shop image in the street view images collected at the geographic location a may be considered as a newly-opened shop, that is, the shop is not recorded in the database of the platform, so that the shop is preliminarily determined to be in an abnormal operation state, and the shop operation state abnormality information may be output through the human-computer interaction interface to notify an operator to check and update the operation state of the shop in which the abnormal operation state is preliminarily determined in time.
For another example, a first shop image in the street view image collected many times at the geographic location a is not matched with each second shop image at the geographic location a in the preset shop image set, and at this time, the shop corresponding to all the second shop images at the geographic location a can be considered as a closed shop, so that the shop is preliminarily determined to be in an abnormal operation state, and the abnormal information of the shop operation state can be output through the human-computer interaction interface, so as to notify an operator to check and update the operation state of the shop preliminarily determined to be in the abnormal operation state in time.
In other embodiments of the present disclosure, after the step of determining the operation status of the corresponding shop according to the matching result, the method further includes: when the matching result is that the matching is successful, outputting second operation state information through a man-machine interaction interface; and the second operation state information is used for indicating the corresponding abnormal shop operation in the successfully matched second shop image.
According to the shop identification method based on street view provided by the embodiment of the disclosure, street view information acquired by street view information acquisition equipment is acquired, and the street view information at least comprises the following steps: geographic location and street view images; then, acquiring a first shop image in the street view image; and according to the geographic position, the acquired first shop image is matched with a second shop image in a preset shop image set, and the operation state of the corresponding shop is determined according to the matching result, so that the efficiency of determining the shop state is improved. According to the shop identification method based on the street view, the shop images acquired by the street view information acquisition equipment of the takeout distributor are used for identifying the shop based on multi-mode characteristics such as geographic positions and shop image characteristics, so that the real-time performance is higher, and the timeliness of determining the shop state is improved. Meanwhile, the abnormal information of the shop operation state is output through the man-machine interaction interface so as to inform an operator to check and update the state of the abnormal shop in time, and the timeliness of the update of the shop state is improved.
Furthermore, the similarity distance between shops is determined based on multi-modal characteristics such as geographic positions, visual characteristics and/or text characteristics, the considered characteristic dimension is more comprehensive, the distance measurement mode is more stable, and the recognition is more accurate.
On the other hand, whether the shop is included in the collected street view image is judged before the first shop image in the street view image is acquired, and only when the shop is included, the follow-up operation is executed, so that the shop identification time when the shop is not included in the collected street view image can be effectively shortened, and the shop identification efficiency is improved. By performing histogram equalization processing on the street view image before the first shop image in the street view image is acquired, the stability of acquiring the characteristics of the shop image can be improved, and the improvement of the identification accuracy is facilitated.
EXAMPLE III
The embodiment of the present disclosure provides a shop recognition device based on street view, as shown in fig. 3, the device includes:
the streetscape information acquiring module 310 is configured to acquire streetscape information acquired by streetscape information acquiring equipment, where the streetscape information at least includes: geographic location and street view images;
a shop image acquisition module 320, configured to acquire a first shop image in the street view image;
the store identification module 330 is configured to match the acquired first store image with a second store image in a preset store image set according to the geographic position;
and the shop management module 340 is configured to determine an operation state of the corresponding shop according to the matching result.
In some embodiments of the present disclosure, the store identification module 330 is further configured to:
determining a position characteristic distance between a geographic position in the street view information and a geographic position corresponding to a second shop image and an image characteristic distance between the first shop image and the second shop image for each acquired first shop image and each second shop image in a preset shop image set;
and determining a second shop image matched with the first shop image through a combined distance obtained by weighting and summing the position characteristic distance and the image characteristic distance.
Wherein the image feature distance between the first store image and the second store image comprises: a visual feature distance and/or a text feature distance between the first store image and the second store image.
In some embodiments of the present disclosure, the preset shop image set includes:
a full store image; or the like, or, alternatively,
each store image matching the geographic location.
In some embodiments of the present disclosure, the store management module 340 is further configured to:
when the matching result is that the matching is successful, determining that the shop corresponding to the second shop image matched with the first shop image is in a normal operation state; or the like, or, alternatively,
when the matching failure times of the first shop images collected at the same geographic position reach preset times, determining that the shop corresponding to the first shop images and/or the shop corresponding to the second shop images at the same geographic position are in an abnormal operation state.
As shown in fig. 4, in some embodiments of the present disclosure, the apparatus further comprises:
a shop image classification module 350, configured to input the street view image into a pre-trained image classifier;
the shop image classification module 350 is further configured to execute the street view information obtaining module when the determination result of the image classifier indicates that the street view image includes a shop image.
As shown in fig. 4, in some embodiments of the present disclosure, the apparatus further comprises:
and the image preprocessing module 360 is configured to perform histogram equalization processing on the street view image.
In some embodiments of the present disclosure, the streetscape information obtaining module 310 is further configured to:
acquiring the real-time geographical position of street view information acquisition equipment;
and triggering the street view information acquisition equipment to acquire the street view image when the real-time geographic position is determined to be in the preset range.
As shown in fig. 4, in some embodiments of the present disclosure, the apparatus further comprises:
an operation state information output module 370, configured to output first operation state information through a human-computer interaction interface when the matching result is a matching failure; the first operation state information is used for indicating the abnormal operation of the shop corresponding to the matching result. The abnormal information of the shop operation state is output through the man-machine interaction interface so as to inform an operator to check and update the state of the abnormal shop in time, and the timeliness of the update of the shop state is improved.
The shop identification device based on street view provided by the embodiment of the present disclosure is used to implement each step of the shop identification method based on street view described in the first and second embodiments of the present disclosure, and specific implementation of each module of the device refers to the corresponding step, which is not described herein again.
The shop recognition device based on street view that this disclosed embodiment provided, through acquireing the street view information that street view information acquisition equipment gathered, street view information includes at least: geographic location and street view images; then, acquiring a first shop image in the street view image; and according to the geographic position, the acquired first shop image is matched with a second shop image in a preset shop image set, and the operation state of the corresponding shop is determined according to the matching result, so that the efficiency of determining the shop state is improved. According to the shop recognition device based on the street view, the shop images collected by the street view information collection equipment are used for recognizing the shop based on multi-modal characteristics such as geographic positions and shop image characteristics, so that the real-time performance is higher, and the timeliness for determining the shop state is improved. Furthermore, the similarity distance between shops is determined based on multi-modal characteristics such as geographic positions, visual characteristics and/or text characteristics, the considered characteristic dimension is more comprehensive, the distance measurement mode is more stable, and the recognition is more accurate.
On the other hand, whether the shop is included in the collected street view image is judged before the first shop image in the street view image is acquired, and only when the shop is included, the follow-up operation is executed, so that the shop identification time when the shop is not included in the collected street view image can be effectively shortened, and the shop identification efficiency is improved. By performing histogram equalization processing on the street view image before the first shop image in the street view image is acquired, the stability of acquiring the characteristics of the shop image can be improved, and the improvement of the identification accuracy is facilitated. Accordingly, the present disclosure also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the street view-based shop identification method according to any one of the first embodiment and the second embodiment of the present disclosure is implemented. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer and the like.
The present disclosure also provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor, implements the steps of the streetscape-based shop identification method according to any one of the first to second embodiments of the present disclosure.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The shop identification method and device based on street view provided by the present disclosure are introduced in detail, and specific examples are applied in the text to explain the principle and the implementation of the present disclosure, and the description of the above embodiments is only used to help understanding the method and the core idea of the present disclosure; meanwhile, for a person skilled in the art, based on the idea of the present disclosure, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present disclosure should not be construed as a limitation to the present disclosure.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Claims (16)
1. A shop identification method based on street view is characterized by comprising the following steps:
controlling street view information acquisition equipment to acquire the street view information of the shop needing to be updated when the street view information acquisition equipment arrives at the street or the geographical position according to the geographical position or the street information of the shop needing to be updated in the shop database to be identified;
obtaining street view information collected by the street view information collection equipment, wherein the street view information at least comprises: geographic location and street view images;
acquiring a first shop image in the street view image;
according to the geographic position, matching the acquired first shop image with a second shop image in a preset shop image set;
and determining the operation state of the corresponding shop as a normal operation state or an abnormal state according to the matching result, and updating the operation state of the shop corresponding to the second shop image in the preset shop image set.
2. The method of claim 1, wherein the step of matching the acquired first store image with a second store image from a preset set of store images according to the geographic location comprises:
determining a position characteristic distance between a geographic position in the street view information and a geographic position corresponding to a second shop image and an image characteristic distance between the first shop image and the second shop image for each acquired first shop image and each second shop image in a preset shop image set;
and determining a second shop image matched with the first shop image through a combined distance obtained by weighting and summing the position characteristic distance and the image characteristic distance.
3. The method of claim 2, wherein the image feature distance between the first store image and the second store image comprises: a visual feature distance and/or a text feature distance between the first store image and the second store image.
4. The method of claim 1, wherein the set of preset store images comprises:
a full store image; or the like, or, alternatively,
each store image matching the geographic location.
5. The method of claim 1, wherein the determining that the operation state of the corresponding shop is a normal operation state or an abnormal state according to the matching result comprises:
when the matching result is that the matching is successful, determining that the shop corresponding to the second shop image matched with the first shop image is in a normal operation state;
when the matching failure times of the first shop images collected at the same geographic position reach preset times, determining that the shop corresponding to the first shop images and/or the shop corresponding to the second shop images at the same geographic position are in an abnormal operation state.
6. The method of any of claims 1 to 5, wherein the step of obtaining the first shop image in the street view image is preceded by the step of:
inputting the street view image into a pre-trained image classifier;
when the determination result of the image classifier indicates that the street view image includes the shop image, the step of acquiring the first shop image in the street view image is performed.
7. The method of claim 6, wherein prior to the step of obtaining the first store image in the street view image, further comprising:
and carrying out histogram equalization processing on the street view image.
8. The method according to any one of claims 1 to 5, wherein the step of acquiring street view information acquired by the street view information acquisition device comprises:
acquiring the real-time geographical position of street view information acquisition equipment;
and triggering the street view information acquisition equipment to acquire the street view image when the real-time geographic position is determined to be in the preset range.
9. The method of claim 1, further comprising:
when the matching result is that the matching fails, outputting first operation state information through a man-machine interaction interface; the first operation state information is used for indicating the abnormal operation of the shop corresponding to the matching result.
10. A street view-based store identification device, comprising:
the street view information acquisition module is used for controlling street view information acquisition equipment to acquire the street view information of the shop needing to be updated when the street view information acquisition equipment arrives at the street or the geographic position according to the geographic position or the street information of the shop needing to be updated in the shop database to be identified
The street view information acquisition module is used for acquiring street view information acquired by the street view information acquisition equipment, and the street view information at least comprises: geographic location and street view images;
the shop image acquisition module is used for acquiring a first shop image in the street view image;
the shop identification module is used for matching the acquired first shop image with a second shop image in a preset shop image set according to the geographic position;
and the shop management module is used for determining the operation state of the corresponding shop as a normal operation state or an abnormal state according to the matching result, and updating the operation state of the shop corresponding to the second shop image in the preset shop image set.
11. The apparatus of claim 10, wherein the store identification module is further configured to:
determining a position characteristic distance between a geographic position in the street view information and a geographic position corresponding to a second shop image and an image characteristic distance between the first shop image and the second shop image for each acquired first shop image and each second shop image in a preset shop image set;
and determining a second shop image matched with the first shop image through a combined distance obtained by weighting and summing the position characteristic distance and the image characteristic distance.
12. The apparatus of claim 11, wherein the image feature distance between the first store image and the second store image comprises: a visual feature distance and/or a text feature distance between the first store image and the second store image.
13. The apparatus of claim 10, wherein the store management module is further configured to:
when the matching result is that the matching is successful, determining that the shop corresponding to the second shop image matched with the first shop image is in a normal operation state;
when the matching failure times of the first shop images collected at the same geographic position reach preset times, determining that the shop corresponding to the first shop images and/or the shop corresponding to the second shop images at the same geographic position are in an abnormal operation state.
14. The apparatus of claim 10, further comprising:
the operation state information output module is used for outputting first operation state information through the man-machine interaction interface when the matching result is that the matching fails; the first operation state information is used for indicating the abnormal operation of the shop corresponding to the matching result.
15. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the streetscape-based store identification method of any one of claims 1 to 9 when executing the computer program.
16. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the street view-based shop identification method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844680.3A CN109214280B (en) | 2018-07-27 | 2018-07-27 | Shop identification method and device based on street view, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844680.3A CN109214280B (en) | 2018-07-27 | 2018-07-27 | Shop identification method and device based on street view, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109214280A CN109214280A (en) | 2019-01-15 |
| CN109214280B true CN109214280B (en) | 2021-10-01 |
Family
ID=64990858
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810844680.3A Active CN109214280B (en) | 2018-07-27 | 2018-07-27 | Shop identification method and device based on street view, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109214280B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110135245B (en) * | 2019-04-02 | 2021-11-19 | 北京三快在线科技有限公司 | Store arrival confirmation method and device, electronic equipment and readable storage medium |
| CN110223050A (en) * | 2019-06-24 | 2019-09-10 | 广东工业大学 | A kind of verification method and relevant apparatus of merchant store fronts title |
| CN111047355A (en) * | 2019-11-28 | 2020-04-21 | 口碑(上海)信息技术有限公司 | Abnormal entity object monitoring method and device |
| CN111311316B (en) * | 2020-02-03 | 2023-05-23 | 支付宝(杭州)信息技术有限公司 | Method and device for depicting merchant portrait, electronic equipment, verification method and system |
| CN111461762A (en) * | 2020-03-05 | 2020-07-28 | 支付宝(杭州)信息技术有限公司 | Merchant detection method and device and electronic equipment |
| CN111368761B (en) * | 2020-03-09 | 2022-12-16 | 腾讯科技（深圳）有限公司 | Shop business state recognition method and device, readable storage medium and equipment |
| CN111832658B (en) * | 2020-07-20 | 2023-08-18 | 北京百度网讯科技有限公司 | Point-of-interest information processing method and device, electronic equipment and storage medium |
| CN112613891B (en) * | 2020-12-24 | 2023-10-03 | 支付宝(杭州)信息技术有限公司 | Shop registration information verification method, device and equipment |
| CN112801078A (en) * | 2020-12-25 | 2021-05-14 | 北京百度网讯科技有限公司 | Point of interest (POI) matching method and device, electronic equipment and storage medium |
| CN113065016A (en) * | 2021-03-23 | 2021-07-02 | 支付宝(杭州)信息技术有限公司 | Offline store information processing method, device, equipment and system |
| CN114170301B (en) * | 2022-02-09 | 2022-05-17 | 城云科技（中国）有限公司 | Abnormal municipal facility positioning method and device and application thereof |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011055250A (en) * | 2009-09-02 | 2011-03-17 | Sony Corp | Information providing method and apparatus, information display method and mobile terminal, program, and information providing system |
| CN104820718B (en) * | 2015-05-22 | 2018-01-30 | 哈尔滨工业大学 | Image classification and search method based on geographic location feature Yu overall Vision feature |
| US9594984B2 (en) * | 2015-08-07 | 2017-03-14 | Google Inc. | Business discovery from imagery |
| JP6496671B2 (en) * | 2016-01-13 | 2019-04-03 | 株式会社ぐるなび | Information processing apparatus, terminal apparatus, information processing method, and program |
| US10970896B2 (en) * | 2016-11-02 | 2021-04-06 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
| CN107944934A (en) * | 2017-12-26 | 2018-04-20 | 中山市优罗网络科技有限公司 | Method, apparatus, computer device and storage medium for providing shop object information |
-
2018
- 2018-07-27 CN CN201810844680.3A patent/CN109214280B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| 《基于环境富指纹的室内位置识别系统设计与实现》;安家琪;《中国优秀硕士学位论文全文数据库 信息科技辑》;20180315(第3期);第I136-918页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109214280A (en) | 2019-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109214280B (en) | Shop identification method and device based on street view, electronic equipment and storage medium | |
| CN112418117B (en) | Small target detection method based on unmanned aerial vehicle image | |
| CN110084165B (en) | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation | |
| CN112541395A (en) | Target detection and tracking method and device, storage medium and electronic device | |
| CN111914813A (en) | Power transmission line inspection image naming method and system based on image classification | |
| CN112434566B (en) | Passenger flow statistics method and device, electronic equipment and storage medium | |
| CN113723157B (en) | Crop disease identification method and device, electronic equipment and storage medium | |
| CN112613569A (en) | Image recognition method, and training method and device of image classification model | |
| CN111881958A (en) | License plate classification recognition method, device, equipment and storage medium | |
| CN111091057A (en) | Information processing method and device and computer readable storage medium | |
| CN113780145A (en) | Sperm morphology detection method, sperm morphology detection device, computer equipment and storage medium | |
| CN112836683A (en) | License plate recognition method, device, equipment and medium for portable camera equipment | |
| CN115690545B (en) | Method and device for training target tracking model and target tracking | |
| CN111898418A (en) | Human body abnormal behavior detection method based on T-TINY-YOLO network | |
| CN114399729A (en) | Monitoring object movement identification method, system, terminal and storage medium | |
| CN112738725B (en) | Real-time identification method, device, equipment and medium for target crowd in semi-closed area | |
| CN117765348A (en) | Target detection model deployment method, target detection method and electronic equipment | |
| CN112966762A (en) | Wild animal detection method and device, storage medium and electronic equipment | |
| CN117372956A (en) | Method and device for detecting state of substation screen cabinet equipment | |
| CN115131826B (en) | Article detection and identification method, and network model training method and device | |
| CN115953744A (en) | Vehicle identification tracking method based on deep learning | |
| CN114445787A (en) | Non-motor vehicle weight recognition method and related equipment | |
| CN114494355A (en) | Trajectory analysis method and device based on artificial intelligence, terminal equipment and medium | |
| CN114842183A (en) | Convolutional neural network-based switch state identification method and system | |
| CN113569600A (en) | Method and device for identifying weight of object, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |