CN109214280A - Shop recognition methods, device, electronic equipment and storage medium based on streetscape - Google Patents
Shop recognition methods, device, electronic equipment and storage medium based on streetscape Download PDFInfo
- Publication number
- CN109214280A CN109214280A CN201810844680.3A CN201810844680A CN109214280A CN 109214280 A CN109214280 A CN 109214280A CN 201810844680 A CN201810844680 A CN 201810844680A CN 109214280 A CN109214280 A CN 109214280A
- Authority
- CN
- China
- Prior art keywords
- shop
- image
- streetscape
- geographical location
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The shop recognition methods based on streetscape of the embodiment of the present disclosure, belongs to field of computer technology, facilitates the efficiency for promoting determining shop state.The shop recognition methods based on streetscape that the embodiment of the present disclosure provides includes: to obtain the streetscape information of streetscape information collecting device acquisition, and the streetscape information includes at least: geographical location and street view image；Obtain the first shop image in the street view image；According to the geographical location, the first shop image that will acquire is matched with the second shop image in default shop image collection, and the operation state in corresponding shop is determined according to matching result.The shop recognition methods based on streetscape that the embodiment of the present disclosure provides, the shop image acquired by streetscape information collecting device, and shop is identified based on the multi-modal feature such as geographical location, shop characteristics of image, improve the efficiency of determining shop state.
Description
Technical field
The embodiment of the present disclosure is related to field of computer technology, more particularly to a kind of shop recognition methods based on streetscape,
Device, electronic equipment and storage medium.
Background technique
With the development of takeout service, the management of trade company shop is to take out one of platform end important service module, and platform needs
Timely update the current state in shop, such as online, the offline state in shop.The inventor of the disclosure passes through to the prior art
The study found that shop management is submitted mainly by trade company oneself, the operation personnel's regular return visit of platform end or selective examination are then taken out, and
The state in the shop Shi Gengxin determines that shop whether there is.Manually determine that the mode of shop state needs to expend greatly in the prior art
The manpower and time cost of amount, efficiency is lower, and will lead to shop state and update not in time.
Summary of the invention
The embodiment of the present disclosure provides a kind of shop recognition methods based on streetscape, can automatically identify shop state, have
Help be promoted the efficiency of determining shop state.
In a first aspect, the embodiment of the present disclosure provides a kind of shop recognition methods based on streetscape includes:
The streetscape information of streetscape information collecting device acquisition is obtained, the streetscape information includes at least: geographical location and street
Scape image；
Obtain the first shop image in the street view image；
According to the geographical location, the first shop image and the second shop in default shop image collection that will acquire
Image is matched；
The operation state in corresponding shop is determined according to matching result.
Second aspect, the embodiment of the present disclosure provide a kind of shop identification device based on streetscape, comprising:
Streetscape data obtaining module, for obtaining the streetscape information of streetscape information collecting device acquisition, the streetscape information
It includes at least: geographical location and street view image；
Shop image collection module, for obtaining the first shop image in the street view image；
Shop identification module, the first shop image and default shop figure for will acquire according to the geographical location
The second shop image in image set conjunction is matched；
Shop management module, for determining the operation state in corresponding shop according to matching result.
The third aspect, the embodiment of the present disclosure additionally provide a kind of electronic equipment, including memory, processor and are stored in institute
The computer program that can be run on memory and on a processor is stated, when the processor executes the computer program, is realized
Shop recognition methods described in the embodiment of the present disclosure based on streetscape.
Fourth aspect, the embodiment of the present disclosure provide a kind of computer readable storage medium, are stored thereon with computer journey
Sequence, when which is executed by processor, the step of realizing shop recognition methods based on streetscape described in the embodiment of the present disclosure.
The shop recognition methods based on streetscape that the embodiment of the present disclosure provides, by obtaining the acquisition of streetscape information collecting device
Streetscape information, the streetscape information includes at least: geographical location and street view image；Then, it obtains in the street view image
First shop image；And according to the geographical location, in the first shop image that will acquire and default shop image collection
Second shop image is matched, and the operation state in corresponding shop is determined according to matching result, is capable of automatic identification shop
State improves the efficiency of determining shop state.
Detailed description of the invention
It, below will be in embodiment or description of the prior art in order to illustrate more clearly of the technical solution of the embodiment of the present disclosure
Required attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is only some realities of the disclosure
Example is applied, it for those of ordinary skill in the art, without any creative labor, can also be attached according to these
Figure obtains other attached drawings.
Fig. 1 is the shop recognition methods flow chart based on streetscape of the embodiment of the present disclosure one；
Fig. 2 is the flow chart of the shop recognition methods based on streetscape of the embodiment of the present disclosure two；
Fig. 3 is one of the structural schematic diagram of the shop identification device based on streetscape of the embodiment of the present disclosure three；
Fig. 4 is the second structural representation of the shop identification device based on streetscape of the embodiment of the present disclosure three.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present disclosure, the technical solution in the embodiment of the present disclosure is carried out clear, complete
Site preparation description, it is clear that described embodiment is disclosure a part of the embodiment, instead of all the embodiments.Based on this public affairs
Embodiment in opening, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example belongs to the range of disclosure protection.
Embodiment one
A kind of shop recognition methods based on streetscape that the embodiment of the present disclosure provides, as shown in Figure 1, this method comprises: step
Rapid 110 to step 140.
Step 110, the streetscape information of streetscape information collecting device acquisition is obtained, the streetscape information includes at least: geographical
Position and street view image.
With the development of takeout service, each block can see that take-away dispatching person rides bicycle or motorcycle takes, send outside
It sells, streetscape information collecting device described in disclosure example can be the current take-away dispatching person carrying taken out in dispatching scene
Image capture device, or with the development of takeout service, in the future it is possible that send dispensing machine people etc. to some other department.For
It is easy to understand, carries image capture device acquisition street view image in the embodiment of the present disclosure to take out dispatching person and illustrate this public affairs
The concrete scheme opened.Correspondingly, the streetscape information collecting device for taking out dispatching person can be camera, camera, intelligent terminal
Deng the equipment with image collecting function, described image, which acquires equipment, to be the forms such as wear-type, vehicular or hand-held.
In some embodiments of the present disclosure, the streetscape information collecting device can be adopted according to the control for taking out dispatching person
Collect streetscape information, for example, acquiring streetscape information after receiving the key triggering for taking out dispatching person；It can also be according to take-away dispatching person
Motion state acquire streetscape information, for example, detect take out dispatching person movement speed be reduced to pre-set velocity threshold value (such as 1
Metre per second (m/s)) after, automatic collection streetscape information.
In other embodiments of the disclosure, the streetscape information for obtaining the acquisition of streetscape information collecting device, comprising:
Obtain the real-time geographical locations of streetscape information collecting device；When determining that the real-time geographical locations are located at preset range, triggering
The streetscape information collecting device acquires street view image.
For example, take out platform first according to the geographical location in the shop for needing more new state in shop database to be identified or
Street information needs more new state to control to acquire when streetscape information collecting device reaches the street or the geographical location
The relevant information in shop.Wherein, the geographical location in the shop of more new state or place street determine preset range as needed.So
Afterwards, it takes out platform and obtains the real-time geographical locations for taking out the streetscape information collecting device of jockey, and the real-time geographic position that will acquire
It sets and is matched with preset range, when the real-time geographical locations are located at the preset range, take out platform to streetscape information
It acquires equipment and sends control instruction, control the streetscape information of streetscape information collecting device acquisition current location, i.e. acquisition institute
State the streetscape information of preset range.Wherein, the streetscape information includes but is not limited to the street view image acquired and streetscape letter
The real-time geographical locations (store location in the street view image acquired) of breath acquisition equipment.
Step 120, the first shop image in the street view image is obtained.
In some embodiments of the present disclosure, the image in multiple shops may include in the street view image, it is also possible to no
Including shop image, before carrying out shop matching, it is necessary first to get each of described street view image shop to be matched
Image, be known as " the first shop image " in the present embodiment.
In some embodiments of the present application, it can be obtained using technologies such as edge detection in the prior art, image recognitions
Take the first shop image in the street view image.
Step 130, according to the geographical location, in the first shop image that will acquire and default shop image collection
Second shop image is matched.
The disclosure needs the shop of more new state when it is implemented, the shop for needing to manage determining first.In the disclosure
Some embodiments in, the shop for needing more new state is determined by way of presetting shop image collection, wherein described default
Shop image collection includes: full dose shop image；Or, with the matched each shop image in the geographical location.Wherein, full dose shop
Paving refer to take out platform the shop of verification state in need image；It can with the matched each shop image in the geographical location
Think the image in the shop of the geographical location, or the figure in the shop around the geographical location in preset range
Picture.In the embodiment of the present disclosure, the image preset in the image collection of shop is known as " the second shop image ", each second shop image
With corresponding geographic location association, there is geographical location attribute.The geographical location attribute of second shop image is second shop
The locating geographical location in the corresponding shop of image.Selection match can reduce with the matched shop image in the geographical location
Matched data volume promotes the efficiency for determining shop state.The first shop figure that according to the geographical location, will acquire
It include: to be directed to each the first shop figure got as match with the second shop image in default shop image collection
The second shop image of each of picture and default shop image collection, determines the geographical location and second in the streetscape information
Position feature distance between geographical location corresponding to the image of shop, and, the first shop image and the second shop image
Between characteristics of image distance；By being weighted what summation obtained to the position feature distance, described image characteristic distance
Joint distance, determining the second shop image with the first shop images match.Wherein, first shop image and second shop
Characteristics of image distance between paving image include: visual signature between first shop image and the second shop image away from
From and/or text feature with a distance from.
In some embodiments of the present disclosure, the first shop image that according to the geographical location, will acquire with
The second shop image in default shop image collection is matched, comprising: for the first shop of each got image
With the second shop image of each of default shop image collection, geographical location and the second shop in the streetscape information are determined
Spread the view between position feature distance, the first shop image and the second shop image between geographical location corresponding to image
Feel characteristic distance, and, the text feature distance between the first shop image and the second shop image；By to the position
Characteristic distance, the visual signature distance and the text feature distance are weighted the obtained joint distance of summation, determine with
Second shop image of the first shop images match.For example, can determine the first shop in the street view image by following formula
Spread the joint distance of image and the second shop image in default shop image collection:
D(Object1,Object2)=λpDposition(Object1,Object2)+λvDvisual(Object1,Object2)+
λeDedit(Object1,Object2)；
Wherein, Object1And Object2First shop image and the second shop image respectively；Dposition(Object1,
Object2) indicate the first shop image in the street view image and the position feature distance in the second shop；Dvisual
(Object1,Object2) indicate first shop image and the second shop image visual signature distance；Dedit
(Object1,Object2) indicate first shop image and the second shop image text feature distance；λp、λvAnd λeRespectively
The weight for indicating the position feature distance, visual signature distance and text feature distance, determines each weight according to test result
Value.
In some embodiments of the present disclosure, the position feature distance can be the first shop figure in the street view image
Picture geographical location (i.e. acquisition street view image geographical location and the corresponding shop of the first shop image geographical location) and
(i.e. in the associated geographical location of the second shop image and the second shop image shop in the geographical location of second shop image
The locating geographical location of paving) between linear distance (i.e. L2 distance), or the first shop image in the street view image
City block distance (i.e. L1 distance) between the geographical location of second shop image.For example, calculating institute by following formula
State the geographic location feature of the first shop image and second shop image in street view image:
Wherein, LatobjectAnd LngobjectIndicate the latitude and longitude coordinates in shop.
In some embodiments of the present disclosure, the visual signature distance can pass through COS distance or Euclidean distance meter
It calculates.For example, calculating the visual signature of the first shop image and second shop figure in the street view image by following formula
Visual signature distance between the visual signature of picture:
Wherein, Vobject,iIndicate the i-th dimension visual signature of image object.
In some embodiments of the present disclosure, the visual signature is the feature for expressing the whole visual effect of image, can
Think the combination of any one or more feature such as color characteristic, textural characteristics, shape feature；The visual signature can also be
Other characteristics of image.Wherein, the visual signature can pass through the extraction algorithm of individual features in the prior art.Such as: passing through base
The textural characteristics in the street view image, the distribution spy by calculating color space are extracted in the Texture Segmentation Algorithm of small echo
Sign obtains the color characteristic etc. in the street view image.
In some preferred embodiments of the disclosure, by obtaining the first shop in street view image based on convolutional neural networks
Spread the visual signature of image.For example, street view image to be processed to be divided into the grid of 7*7 first, then, using following loss
Function is trained convolutional neural networks:
Wherein,Indicate whether there is shop, x in first gridiyiwihiciIndicate the cross in shop in i-th of grid
Coordinate, ordinate, width and the offset of height, the probability containing shop；xi′yi′wi′hi′ci' it is the defeated of convolutional neural networks
Out.Wherein, xi、yi、wi、hiAnd ciIt can be for by carrying out what manual measurement determined to training sample.
It, can be by being labelled with the label of shop coordinate and size in advance in the training process of above-mentioned convolutional neural networks
Image as training sample, with the minimum target of above-mentioned loss function, training convolutional neural networks.It is instructed in convolutional neural networks
Practice after completing, can choose the output streetscape figure as input of any one character representation layer of the convolutional neural networks
The visual signature of picture.
In embodiment of the disclosure, for street view image to be identified, firstly, the street view image is divided into several
Grid determines in the street view image whether include shop image；Then, then based on above-mentioned loss function training convolutional nerve net
Network.It is excellent when obtaining the visual signature of the first shop image in the street view image of input using trained convolutional neural networks
Choosing, select the vision of the output street view image as input of the penultimate character representation layer of the convolutional neural networks
Feature, it is not only available to arrive careful visual signature, but also the visual signature data volume got will not be made excessive, reduce operation
Efficiency.
In the other embodiments of the disclosure, street view image can also be divided into the grid of other quantity, or use
Unknown losses function training convolutional neural networks, to obtain the visual signature of the first shop image in street view image, the disclosure pair
This is without limitation.
In some embodiments of the present disclosure, the text feature distance can be indicated by smallest edit distance.Example
Such as, the text of the text feature of the first shop image and second shop image in the street view image is calculated by following formula
Text feature distance between eigen:
Wherein, Editn()
For editing distance calculation formula.
Since there are errors for OCR algorithm, is directly matched and be not suitable for using text feature, the embodiment of the present disclosure passes through
The stability of text feature measurement is improved using smallest edit distance.
In other embodiments of the disclosure, the text feature can be trade name, address, the phone etc. in shop.Its
In, the text feature can be identified by OCR (Optical Character Recognition, optical character identification) and be calculated
Method is identified to obtain to text filed in the street view image.For example, being instructed first by being labeled with the image of shop title
Practice shop title identification model；Then, the shop title in the street view image is determined by the shop title identification model
Region；Finally, identifying by OCR algorithm to the shop title region in the street view image, it is identified
In shop title.In the other embodiments of the disclosure, the Chinese of the first shop image can also be obtained by other methods
Eigen, the disclosure are not limited technological means used by the text feature of the first shop image in the street view image is obtained
It is fixed.
Determined take out dispatching person street view image acquisition equipment acquisition street view image in the first shop image and
The joint distance of each second shop image in presetting database may further determine that and first in the street view image of acquisition
It is described that joint distance minimum and the joint distance between the image of shop, which meet the second shop image of pre-determined distance threshold value,
The second shop image in street view image, i.e. shop in the first shop image are identical as the shop in the second shop image.
Step 140, the operation state in corresponding shop is determined according to matching result.
In some embodiments of the present application, the operation state that corresponding shop is determined according to matching result, comprising:
When matching result is successful match, determine that shop corresponding with the second shop image of the first shop images match is normal fortune
Battalion's state；Or, the matching result in the first shop image of same geographical location acquisition is that the number that it fails to match reaches pre-
If when number, determining the second shop image of the corresponding shop of first shop image and/or the same geographical location
Corresponding shop is abnormal operation state.For example, when by matching the first shop image and a certain the determined in street view image
When the success of two shop images match, it can determine that the corresponding shop of second shop image in default shop image collection is
The shop of normal operation state, it is normal for can further updating the operation state in the corresponding shop of second shop image.
If the smallest joint distance is unsatisfactory for pre-determined distance threshold value, it may be considered that the image capture device for taking out dispatching person is worked as
The first shop image is not in the image collection of default shop in the street view image of preceding acquisition.Further by the geographical position
The matching result for setting the shop in the shop image of acquisition is counted, and determines the matched second shop image in the geographical location
Whether the operation state in corresponding shop is abnormal operation state.
In some embodiments of the present application, in matching result knowledge, it fails to match, i.e., is not matched to the second shop image
In the case where, it can be counted to geographical location is based on the case where not being matched to the second shop image.For example, in geographical location
When the first shop image and the second shop image acquired at A carries out match cognization, often present in the A of geographical location acquisition the
The case where one shop image is not matched to the second shop image, the case where the second shop image is not matched to geographical location A into
It row plus 1 counts, (such as 10 when the cumulative number when geographical location A is not matched to the second shop image the case where is more than preset times
It is secondary), then it is assumed that the shop of geographical location A is not in the second shop image, therefore, in the corresponding shop of the second shop image
Shop positioned at geographical location A may be the shop of abnormal operation state.During concrete application, the reason of abnormality, is wrapped
Include: the corresponding shop of the second shop image at the A of geographical location has been closed down, other shops are changed to；Alternatively, being located at ground
Shop (i.e. the corresponding shop of the first shop image) at reason position A is shop of newly starting business, and is not taking out platform registration shop
Information.It, can be defeated when determining matched second shop in the geographical location is abnormal shop in some embodiments of the present application
Matched second shop in the geographical location is the prompt information in abnormal shop out, verifies and update shop in time convenient for staff
The state of paving.
In the other embodiments of the disclosure, it can also be weighted according only to position feature distance and visual signature distance
The joint distance that summation obtains determines second shop figure with the first shop images match in the street view image of acquisition
Picture, or it is weighted the joint distance that summation obtains according only to position feature distance and text feature distance, it determines and acquires
The street view image in the first shop images match second shop image, can reach and identify shop in the disclosure
Technical effect.
The shop recognition methods based on streetscape that the embodiment of the present disclosure provides, by obtaining the acquisition of streetscape information collecting device
Streetscape information, the streetscape information includes at least: geographical location and street view image；Then, it obtains in the street view image
First shop image；And according to the geographical location, in the first shop image that will acquire and default shop image collection
Second shop image is matched, and the operation state in corresponding shop is determined according to matching result, can automatically determine shop
State facilitates the efficiency for promoting determining shop state.Further, since abnormal shop being determined in time, may further and
When verify and update abnormal shop state, facilitate promoted shop state update timeliness.Also, abnormal shop is being determined
After paving, it is only necessary to which the state for verifying abnormal shop reduces workload, can also promote the efficiency for updating shop state.This
The shop recognition methods based on streetscape that open embodiment provides, what the streetscape information collecting device by taking out dispatching person acquired
Shop image, and based on multi-modal Automatic feature recognition shop states such as geographical location, shop characteristics of image, help to be promoted true
Determine the efficiency of shop state.Further, it is determined based on the multi-modal feature such as geographical location, visual signature and/or text feature
Similarity distance between shop, the characteristic dimension of consideration is more comprehensive, and distance metric mode is more stable, identifies more acurrate.
Embodiment two
The embodiment of the present disclosure provides a kind of shop recognition methods based on streetscape, as shown in Fig. 2, this method comprises: step
Rapid 210 to step 270.
Step 210, the streetscape information of streetscape information collecting device acquisition is obtained, the streetscape information includes at least: geographical
Position and street view image.
The streetscape information of streetscape information collecting device acquisition is obtained referring to embodiment one, this embodiment is not repeated.
Step 220, it determines in the street view image whether include shop image, if so, thening follow the steps 230, otherwise, jumps
Go to the streetscape information that step 210 reacquires the acquisition of streetscape information collecting device.
In some embodiments of the present disclosure, before the first shop image obtained in the street view image, also wrap
It includes: the street view image is input to Image Classifier trained in advance；When the definitive result of described image classifier indicates institute
When stating in street view image including shop image, the step of obtaining the first shop image in the street view image is executed.
Street view image acquisition equipment due to taking out dispatching person may acquire street view image during the motion, and may
Therefore continuous acquisition street view image can have the street view image not comprising shop image, in order to be promoted in the street view image of acquisition
Recognition efficiency, before obtaining the first shop image in the street view image, first determine whether in the street view image of acquisition whether
Including shop image.
In some embodiments of the present disclosure, equipment acquisition one can be acquired by taking out the street view image of dispatching person first
The street view image sample of fixed number amount, and manually marked, the street view image sample extraction characteristic of division after being then based on mark,
With train classification models.For example, the LBP feature of every width street view image sample can be extracted, i.e., each pixel and picture around it
The size relation of element；Then, the LBP feature based on extraction and the corresponding sample label training SVM classifier of the LBP feature,
Whether to determine in street view image including shop.
In order to promote the efficiency that shop in image is detected, Linear SVM classifier is used in the embodiment of the present disclosure, classifier
Training data marks street view image and non-street view image from sample has been completed.Wherein, the writeable expression of Linear SVM model
Are as follows:
subject to yn(wTxn+b)≥1for all n；
Wherein, n indicates the index of training sample, ynThe classification for indicating n-th of training sample (includes the streetscape figure in shop
As or do not include shop non-street view image), xnFor the LBP feature of n-th of sample, w and b are model parameter, are characterized respectively
Combined weight and biasing, i.e. model parameter.
After classifier training is completed, the street view image of the streetscape information collecting device acquisition for the take-away dispatching person that will acquire
Be input to the SVM classifier, i.e., in the exportable street view image whether include shop classification results.When the streetscape figure
When including shop as in, shop image characteristics extraction and identification further can be highlighted to street view image execution；When described
When not including shop in street view image, then the street view image currently acquired, the streetscape letter of the take-away dispatching person of reacquisition are abandoned
Street view image in the streetscape information of breath acquisition equipment acquisition.
Step 230, histogram equalization processing is carried out to the street view image.
In some embodiments of the present disclosure, before the first shop image executed in the acquisition street view image,
Further include: histogram equalization processing is carried out to the street view image.
The illumination condition of scene changes multiplicity when street view image acquisition equipment due to taking out dispatching person acquires street view image,
In order to adapt to possible different illumination conditions in reality scene, before extracting shop characteristics of image, it is also necessary to the streetscape of acquisition
Image does histogram equalization operation, can increase the stability of the shop characteristics of image of extraction.
Histogram equalizing method in the prior art can be used by carrying out histogram equalization processing to the street view image, this
Embodiment no longer elaborates.
Step 240, the first shop image in the street view image is obtained.
The specific embodiment of the first shop image in the street view image is obtained referring to embodiment one, the present embodiment is not
It repeats again.
Step 250, according to the geographical location, in the first shop image that will acquire and default shop image collection
Second shop image is matched.
According to the geographical location, the first shop image and the second shop in default shop image collection that will acquire
Image carries out matched specific embodiment referring to embodiment one, and this embodiment is not repeated.
Step 260, the operation state in corresponding shop is determined according to matching result.
Determine the specific embodiment of the operation state in corresponding shop referring to embodiment one, this implementation according to matching result
Example repeats no more.
Step 270, the first operation state information is exported by man-machine interactive interface.
In some embodiments of the present disclosure, the step of operation state in corresponding shop is determined according to matching result it
Afterwards, the method also includes: matching result be it fails to match when, pass through man-machine interactive interface export the first operation state letter
Breath；It is abnormal that the first operation state information is used to indicate the operation of shop corresponding to matching result.
For example, working as the first shop image and default shop image set at the A of geographical location in the street view image of multi collect
The second shop image in conjunction mismatches, at this point it is possible to think the first shop in the street view image acquired at the A of geographical location
The corresponding shop of image is the shop newly started business, i.e. the shop is not recorded in the database of platform, therefore preliminary judgement should
Shop is abnormal operation state, shop operation state exception information can be exported by man-machine interactive interface, to notify operation people
Member verifies and updates in time the operation state in the shop of preliminary judgement exception operation state.
For another example working as the first shop image and default shop image at the A of geographical location in the street view image of multi collect
The second shop image mismatches each of at the A of geographical location in set, at this point it is possible to think all at the A of geographical location
The corresponding shop of two shop images is the shop closed down, therefore the preliminary judgement shop is abnormal operation state, can pass through people
Machine interactive interface exports shop operation state exception information, to notify operation personnel to verify in time and update preliminary judgement to be abnormal
The operation state in the shop of operation state.
In other embodiments of the disclosure, the step of operation state in corresponding shop is determined according to matching result it
Afterwards, the method also includes: matching result be successful match when, pass through man-machine interactive interface export the second operation state letter
Breath；It is abnormal that the second operation state information is used to indicate corresponding shop operation in the second shop image of successful match.
The shop recognition methods based on streetscape that the embodiment of the present disclosure provides, by obtaining the acquisition of streetscape information collecting device
Streetscape information, the streetscape information includes at least: geographical location and street view image；Then, it obtains in the street view image
First shop image；And according to the geographical location, in the first shop image that will acquire and default shop image collection
Second shop image is matched, and the operation state in corresponding shop is determined according to matching result, helps to promote determining shop
The efficiency of paving state.The shop recognition methods based on streetscape that the embodiment of the present disclosure provides, the streetscape by taking out dispatching person are believed
The shop image of breath acquisition equipment acquisition, and shop is identified based on the multi-modal feature such as geographical location, shop characteristics of image, in real time
Property is stronger, improves the timeliness of determining shop state.Meanwhile shop operation state is exported by man-machine interactive interface and is believed extremely
Breath helps to promote the timeliness that shop state updates to notify operation personnel to verify in time and the state in update abnormal shop.
Further, it is determined between shop based on the multi-modal feature such as geographical location, visual signature and/or text feature
The characteristic dimension of similarity distance, consideration is more comprehensive, and distance metric mode is more stable, identifies more acurrate.
On the other hand, by first determining whether the streetscape acquired before obtaining the first shop image in the street view image
It whether include shop in image, only in the case where including shop, just execution subsequent operation can effectively shorten the street of acquisition
Shop recognition time when not including shop in scape image, and then promote shop recognition efficiency.By obtaining the streetscape figure
Histogram equalization processing is carried out to street view image before the first shop image as in, can be promoted and obtain shop characteristics of image
Stability helps to promote recognition accuracy.
Embodiment three
The embodiment of the present disclosure provides a kind of shop identification device based on streetscape, as shown in figure 3, described device includes:
Streetscape data obtaining module 310, for obtaining the streetscape information of streetscape information collecting device acquisition, the streetscape letter
Breath includes at least: geographical location and street view image；
Shop image collection module 320, for obtaining the first shop image in the street view image；
Shop identification module 330, the first shop image and default shop for will acquire according to the geographical location
The second shop image in image collection is matched；
Shop management module 340, for determining the operation state in corresponding shop according to matching result.
In some embodiments of the present disclosure, the shop identification module 330 is further used for:
For the second shop figure of each of each got the first shop image and default shop image collection
Picture, determine position feature between geographical location corresponding to the geographical location in the streetscape information and the second shop image away from
From, and, the characteristics of image distance between the first shop image and the second shop image；
By being weighted the joint distance that summation obtains to the position feature distance, described image characteristic distance, really
Fixed the second shop image with the first shop images match.
Wherein, the characteristics of image distance between first shop image and the second shop image includes: described first
Visual signature distance and/or text feature distance between shop image and the second shop image.
In some embodiments of the present disclosure, the default shop image collection includes:
Full dose shop image；Or,
With the matched each shop image in the geographical location.
In some embodiments of the present disclosure, the shop management module 340 is further used for:
When matching result is successful match, shop corresponding with the second shop image of the first shop images match is determined
For normal operation state；Or,
It is that the number that it fails to match reaches default in the matching result of the first shop image of same geographical location acquisition
When number, the second shop image pair of the corresponding shop of first shop image and/or the same geographical location is determined
The shop answered is abnormal operation state.
As shown in figure 4, in some embodiments of the present disclosure, described device further include:
Shop image classification module 350, for the street view image to be input to Image Classifier trained in advance；
The shop image classification module 350 is also used to indicate the streetscape when the definitive result of described image classifier
When including shop image in image, the streetscape data obtaining module is executed.
As shown in figure 4, in some embodiments of the present disclosure, described device further include:
Image pre-processing module 360, for carrying out histogram equalization processing to the street view image.
In some embodiments of the present disclosure, the streetscape data obtaining module 310 is further used for:
Obtain the real-time geographical locations of streetscape information collecting device；
When determining that the real-time geographical locations are located at preset range, the streetscape information collecting device acquisition streetscape is triggered
Image.
As shown in figure 4, in some embodiments of the present disclosure, described device further include:
Operation state message output module 370 is used for when matching result is that it fails to match, defeated by man-machine interactive interface
First operation state information out；It is abnormal that the first operation state information is used to indicate the operation of shop corresponding to matching result.
Shop operation state exception information is exported by man-machine interactive interface, to notify operation personnel to verify in time and update abnormal shop
State, facilitate promoted shop state update timeliness.
The shop identification device based on streetscape that the embodiment of the present disclosure provides for realizing the embodiment of the present disclosure one and is implemented
Each step of shop recognition methods described in example two based on streetscape, the specific embodiment of each module of device is referring to corresponding
Step, details are not described herein again.
The shop identification device based on streetscape that the embodiment of the present disclosure provides, by obtaining the acquisition of streetscape information collecting device
Streetscape information, the streetscape information includes at least: geographical location and street view image；Then, it obtains in the street view image
First shop image；And according to the geographical location, in the first shop image that will acquire and default shop image collection
Second shop image is matched, and the operation state in corresponding shop is determined according to matching result, helps to promote determining shop
The efficiency of paving state.The shop identification device based on streetscape that the embodiment of the present disclosure provides, is adopted by streetscape information collecting device
The shop image of collection, and shop is identified based on the multi-modal feature such as geographical location, shop characteristics of image, real-time is stronger, is promoted
The timeliness of determining shop state.Further, based on multi-modal spies such as geographical location, visual signature and/or text features
The similarity distance determined between shop is levied, the characteristic dimension of consideration is more comprehensive, and distance metric mode is more stable, and identification is more
Accurately.
On the other hand, by first determining whether the streetscape acquired before obtaining the first shop image in the street view image
It whether include shop in image, only in the case where including shop, just execution subsequent operation can effectively shorten the street of acquisition
Shop recognition time when not including shop in scape image, and then promote shop recognition efficiency.By obtaining the streetscape figure
Histogram equalization processing is carried out to street view image before the first shop image as in, can be promoted and obtain shop characteristics of image
Stability helps to promote recognition accuracy.Correspondingly, the disclosure additionally provides a kind of electronic equipment, including memory, processing
Device and it is stored in the computer program that can be run on the memory and on a processor, the processor executes the computer
The shop identification side based on streetscape as described in the embodiment of the present disclosure one and embodiment two any one embodiment is realized when program
Method.The electronic equipment can be PC machine, mobile terminal, personal digital assistant, tablet computer etc..
The disclosure additionally provides a kind of computer readable storage medium, is stored thereon with computer program, which is located
It manages and is realized when device executes such as the embodiment of the present disclosure one to the shop based on streetscape as described in any one embodiment of embodiment two
The step of recognition methods.
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.For Installation practice
For, since it is basically similar to the method embodiment, so being described relatively simple, referring to the portion of embodiment of the method in place of correlation
It defends oneself bright.
A kind of shop recognition methods and device based on streetscape provided above the disclosure is described in detail, herein
In apply specific case the principle and embodiment of the disclosure be expounded, the explanation of above example is only intended to sides
Assistant solves disclosed method and its core concept；At the same time, for those skilled in the art, the think of according to the disclosure
Think, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not be construed as pair
The limitation of the disclosure.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It realizes by means of software and necessary general hardware platform, naturally it is also possible to pass through hardware realization.Based on such reason
Solution, substantially the part that contributes to existing technology can embody above-mentioned technical proposal in the form of software products in other words
Come, which may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, CD, including
Some instructions are used so that a computer equipment (can be personal computer, server or the network equipment etc.) executes respectively
Method described in certain parts of a embodiment or embodiment.
Claims (16)
1. a kind of shop recognition methods based on streetscape characterized by comprising
The streetscape information of streetscape information collecting device acquisition is obtained, the streetscape information includes at least: geographical location and streetscape figure
Picture；
Obtain the first shop image in the street view image；
According to the geographical location, the first shop image and the second shop image in default shop image collection that will acquire
It is matched；
The operation state in corresponding shop is determined according to matching result.
2. the method according to claim 1, wherein first according to the geographical location, will acquire
The step of shop image is matched with the second shop image in default shop image collection, comprising:
For the second shop image of each of each got the first shop image and default shop image collection, really
Position feature distance between geographical location corresponding to geographical location and the second shop image in the fixed streetscape information, with
And the first characteristics of image distance between shop image and the second shop image；
By being weighted the joint distance that summation obtains to position feature distance, described image characteristic distance, determine with
Second shop image of the first shop images match.
3. according to the method described in claim 2, it is characterized in that, between first shop image and the second shop image
Characteristics of image distance include: between first shop image and the second shop image visual signature distance and/or text
Eigen distance.
4. the method according to claim 1, wherein the default shop image collection includes:
Full dose shop image；Or,
With the matched each shop image in the geographical location.
5. the method according to claim 1, wherein the operation for determining corresponding shop according to matching result
State, comprising:
When matching result is successful match, determine that shop corresponding with the second shop image of the first shop images match is positive
Normal operation state；Or,
It is that the number that it fails to match reaches preset times in the matching result of the first shop image of same geographical location acquisition
When, determine that the second shop image of the corresponding shop of first shop image and/or the same geographical location is corresponding
Shop is abnormal operation state.
6. method according to any one of claims 1 to 5, which is characterized in that the obtained in the street view image
Before the step of one shop image, further includes:
The street view image is input to Image Classifier trained in advance；
When the definitive result of described image classifier indicates in the street view image to include shop image, executes and obtain the street
The step of the first shop image in scape image.
7. according to the method described in claim 6, it is characterized in that, described execute the first shop obtained in the street view image
Before the step of image, further includes:
Histogram equalization processing is carried out to the street view image.
8. method according to any one of claims 1 to 5, which is characterized in that the acquisition streetscape information collecting device is adopted
The step of streetscape information of collection, comprising:
Obtain the real-time geographical locations of streetscape information collecting device；
When determining that the real-time geographical locations are located at preset range, the streetscape information collecting device acquisition streetscape figure is triggered
Picture.
9. the method according to claim 1, wherein the method also includes:
When matching result is that it fails to match, the first operation state information is exported by man-machine interactive interface；First operation
It is abnormal that status information is used to indicate the operation of shop corresponding to matching result.
10. a kind of shop identification device based on streetscape characterized by comprising
Streetscape data obtaining module, for obtaining the streetscape information of streetscape information collecting device acquisition, the streetscape information is at least
It include: geographical location and street view image；
Shop image collection module, for obtaining the first shop image in the street view image；
Shop identification module, the first shop image and default shop image set for will acquire according to the geographical location
The second shop image in conjunction is matched；
Shop management module, for determining the operation state in corresponding shop according to matching result.
11. device according to claim 10, which is characterized in that the shop identification module is further used for:
For the second shop image of each of each got the first shop image and default shop image collection, really
Position feature distance between geographical location corresponding to geographical location and the second shop image in the fixed streetscape information, with
And the first characteristics of image distance between shop image and the second shop image；
By being weighted the joint distance that summation obtains to position feature distance, described image characteristic distance, determine with
Second shop image of the first shop images match.
12. device according to claim 11, which is characterized in that first shop image and the second shop image it
Between characteristics of image distance include: between first shop image and the second shop image visual signature distance and/or
Text feature distance.
13. device according to claim 10, which is characterized in that the shop management module is further used for:
When matching result is successful match, determine that shop corresponding with the second shop image of the first shop images match is positive
Normal operation state；Or,
It is that the number that it fails to match reaches preset times in the matching result of the first shop image of same geographical location acquisition
When, determine that the second shop image of the corresponding shop of first shop image and/or the same geographical location is corresponding
Shop is abnormal operation state.
14. the method according to claim 1, wherein described device further include:
Operation state message output module, for passing through man-machine interactive interface output first when matching result is that it fails to match
Operation state information；It is abnormal that the first operation state information is used to indicate the operation of shop corresponding to matching result.
15. a kind of electronic equipment, including memory, processor and it is stored on the memory and can runs on a processor
Computer program, which is characterized in that the processor realizes claim 1 to 9 any one when executing the computer program
The shop recognition methods based on streetscape.
16. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The step of shop recognition methods described in claim 1 to 9 any one based on streetscape is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844680.3A CN109214280B (en) | 2018-07-27 | 2018-07-27 | Shop identification method and device based on street view, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844680.3A CN109214280B (en) | 2018-07-27 | 2018-07-27 | Shop identification method and device based on street view, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109214280A true CN109214280A (en) | 2019-01-15 |
| CN109214280B CN109214280B (en) | 2021-10-01 |
Family
ID=64990858
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810844680.3A Active CN109214280B (en) | 2018-07-27 | 2018-07-27 | Shop identification method and device based on street view, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109214280B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110135245A (en) * | 2019-04-02 | 2019-08-16 | 北京三快在线科技有限公司 | To shop confirmation method, device, electronic equipment and readable storage medium storing program for executing |
| CN110223050A (en) * | 2019-06-24 | 2019-09-10 | 广东工业大学 | A kind of verification method and relevant apparatus of merchant store fronts title |
| CN111047355A (en) * | 2019-11-28 | 2020-04-21 | 口碑(上海)信息技术有限公司 | Abnormal entity object monitoring method and device |
| CN111311316A (en) * | 2020-02-03 | 2020-06-19 | 支付宝(杭州)信息技术有限公司 | Method and device for depicting merchant portrait, electronic equipment, verification method and system |
| CN111368761A (en) * | 2020-03-09 | 2020-07-03 | 腾讯科技（深圳）有限公司 | Shop business state recognition method and device, readable storage medium and equipment |
| CN111461762A (en) * | 2020-03-05 | 2020-07-28 | 支付宝(杭州)信息技术有限公司 | Merchant detection method and device and electronic equipment |
| CN111832658A (en) * | 2020-07-20 | 2020-10-27 | 北京百度网讯科技有限公司 | Interest point information processing method and device, electronic equipment and storage medium |
| CN112613891A (en) * | 2020-12-24 | 2021-04-06 | 支付宝(杭州)信息技术有限公司 | Shop registration information verification method, device and equipment |
| CN112801078A (en) * | 2020-12-25 | 2021-05-14 | 北京百度网讯科技有限公司 | Point of interest (POI) matching method and device, electronic equipment and storage medium |
| CN113065016A (en) * | 2021-03-23 | 2021-07-02 | 支付宝(杭州)信息技术有限公司 | Offline store information processing method, device, equipment and system |
| CN114170301A (en) * | 2022-02-09 | 2022-03-11 | 城云科技（中国）有限公司 | Abnormal municipal facility positioning method and device and application thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102006548A (en) * | 2009-09-02 | 2011-04-06 | 索尼公司 | Information providing method and apparatus, information display method and mobile terminal and information providing system |
| CN104820718A (en) * | 2015-05-22 | 2015-08-05 | 哈尔滨工业大学 | Image classification and searching method based on geographic position characteristics and overall situation vision characteristics |
| CN107015724A (en) * | 2016-01-13 | 2017-08-04 | 株式会社咕嘟妈咪 | The non-transitorycomputer readable storage medium of information processor, information processing method and storage program |
| CN107690657A (en) * | 2015-08-07 | 2018-02-13 | 谷歌有限责任公司 | Trade company is found according to image |
| CN107944934A (en) * | 2017-12-26 | 2018-04-20 | 中山市优罗网络科技有限公司 | Method, apparatus, computer device and storage medium for providing shop object information |
| US20180122117A1 (en) * | 2016-11-02 | 2018-05-03 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
-
2018
- 2018-07-27 CN CN201810844680.3A patent/CN109214280B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102006548A (en) * | 2009-09-02 | 2011-04-06 | 索尼公司 | Information providing method and apparatus, information display method and mobile terminal and information providing system |
| CN104820718A (en) * | 2015-05-22 | 2015-08-05 | 哈尔滨工业大学 | Image classification and searching method based on geographic position characteristics and overall situation vision characteristics |
| CN107690657A (en) * | 2015-08-07 | 2018-02-13 | 谷歌有限责任公司 | Trade company is found according to image |
| CN107015724A (en) * | 2016-01-13 | 2017-08-04 | 株式会社咕嘟妈咪 | The non-transitorycomputer readable storage medium of information processor, information processing method and storage program |
| US20180122117A1 (en) * | 2016-11-02 | 2018-05-03 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and storage medium |
| CN107944934A (en) * | 2017-12-26 | 2018-04-20 | 中山市优罗网络科技有限公司 | Method, apparatus, computer device and storage medium for providing shop object information |
Non-Patent Citations (1)
| Title |
|---|
| 安家琪: "《基于环境富指纹的室内位置识别系统设计与实现》", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110135245B (en) * | 2019-04-02 | 2021-11-19 | 北京三快在线科技有限公司 | Store arrival confirmation method and device, electronic equipment and readable storage medium |
| CN110135245A (en) * | 2019-04-02 | 2019-08-16 | 北京三快在线科技有限公司 | To shop confirmation method, device, electronic equipment and readable storage medium storing program for executing |
| CN110223050A (en) * | 2019-06-24 | 2019-09-10 | 广东工业大学 | A kind of verification method and relevant apparatus of merchant store fronts title |
| CN111047355A (en) * | 2019-11-28 | 2020-04-21 | 口碑(上海)信息技术有限公司 | Abnormal entity object monitoring method and device |
| CN111311316A (en) * | 2020-02-03 | 2020-06-19 | 支付宝(杭州)信息技术有限公司 | Method and device for depicting merchant portrait, electronic equipment, verification method and system |
| CN111311316B (en) * | 2020-02-03 | 2023-05-23 | 支付宝(杭州)信息技术有限公司 | Method and device for depicting merchant portrait, electronic equipment, verification method and system |
| CN111461762A (en) * | 2020-03-05 | 2020-07-28 | 支付宝(杭州)信息技术有限公司 | Merchant detection method and device and electronic equipment |
| CN111368761A (en) * | 2020-03-09 | 2020-07-03 | 腾讯科技（深圳）有限公司 | Shop business state recognition method and device, readable storage medium and equipment |
| CN111832658A (en) * | 2020-07-20 | 2020-10-27 | 北京百度网讯科技有限公司 | Interest point information processing method and device, electronic equipment and storage medium |
| CN111832658B (en) * | 2020-07-20 | 2023-08-18 | 北京百度网讯科技有限公司 | Point-of-interest information processing method and device, electronic equipment and storage medium |
| CN112613891A (en) * | 2020-12-24 | 2021-04-06 | 支付宝(杭州)信息技术有限公司 | Shop registration information verification method, device and equipment |
| CN112613891B (en) * | 2020-12-24 | 2023-10-03 | 支付宝(杭州)信息技术有限公司 | Shop registration information verification method, device and equipment |
| CN112801078A (en) * | 2020-12-25 | 2021-05-14 | 北京百度网讯科技有限公司 | Point of interest (POI) matching method and device, electronic equipment and storage medium |
| CN113065016A (en) * | 2021-03-23 | 2021-07-02 | 支付宝(杭州)信息技术有限公司 | Offline store information processing method, device, equipment and system |
| CN114170301A (en) * | 2022-02-09 | 2022-03-11 | 城云科技（中国）有限公司 | Abnormal municipal facility positioning method and device and application thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109214280B (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109214280A (en) | Shop recognition methods, device, electronic equipment and storage medium based on streetscape | |
| CN109766872B (en) | Image recognition method and device | |
| CN108304835A (en) | character detecting method and device | |
| CN105426870B (en) | A kind of face key independent positioning method and device | |
| CN107247947A (en) | Face character recognition methods and device | |
| CN109784269A (en) | One kind is based on the united human action detection of space-time and localization method | |
| CN109241871A (en) | A kind of public domain stream of people's tracking based on video data | |
| CN109948425A (en) | A kind of perception of structure is from paying attention to and online example polymerize matched pedestrian's searching method and device | |
| CN106845357A (en) | A kind of video human face detection and recognition methods based on multichannel network | |
| CN109993734A (en) | Method and apparatus for output information | |
| CN110490238A (en) | A kind of image processing method, device and storage medium | |
| CN107808358A (en) | Image watermark automatic testing method | |
| CN112668375B (en) | Tourist distribution analysis system and method in scenic spot | |
| CN106897681A (en) | A kind of remote sensing images comparative analysis method and system | |
| CN109935080A (en) | The monitoring system and method that a kind of vehicle flowrate on traffic route calculates in real time | |
| CN110335099B (en) | Method for recommending ticket purchase line based on user historical data | |
| CN109242047A (en) | Bank card number detection and recognition methods based on K-means++ cluster and residual error network class | |
| CN108932712A (en) | A kind of rotor windings quality detecting system and method | |
| CN110909195A (en) | Picture labeling method and device based on block chain, storage medium and server | |
| CN109670517A (en) | Object detection method, device, electronic equipment and target detection model | |
| CN109697392A (en) | Draw the method and device of target object thermodynamic chart | |
| CN109614990A (en) | A kind of object detecting device | |
| CN110163060A (en) | The determination method and electronic equipment of crowd density in image | |
| CN110399804A (en) | A kind of food inspection recognition methods based on deep learning | |
| CN112233770B (en) | Gymnasium intelligent management decision-making system based on visual perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |