CN105608230A - Image retrieval based business information recommendation system and image retrieval based business information recommendation method - Google Patents

Image retrieval based business information recommendation system and image retrieval based business information recommendation method Download PDF

Info

Publication number
CN105608230A
CN105608230A CN201610078235.1A CN201610078235A CN105608230A CN 105608230 A CN105608230 A CN 105608230A CN 201610078235 A CN201610078235 A CN 201610078235A CN 105608230 A CN105608230 A CN 105608230A
Authority
CN
China
Prior art keywords
image
global characteristics
module
businessman
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610078235.1A
Other languages
Chinese (zh)
Other versions
CN105608230B (en
Inventor
张真
曹骝
秦恩泉
刘鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Innovative Data Technologies Inc
Original Assignee
Nanjing Innovative Data Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Innovative Data Technologies Inc filed Critical Nanjing Innovative Data Technologies Inc
Priority to CN201610078235.1A priority Critical patent/CN105608230B/en
Publication of CN105608230A publication Critical patent/CN105608230A/en
Application granted granted Critical
Publication of CN105608230B publication Critical patent/CN105608230B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/51—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
    • G06K17/0022—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations arrangements or provisious for transferring data to distant stations, e.g. from a sensing device

Abstract

The invention discloses an image-retrieval based business information recommendation system and an image-retrieval based business information recommendation method. An image is captured in a mobile client by a user and then is transmitted to the recommendation system, the recommendation system firstly detects whether a two-dimension code exists on the image, when the two-dimension code exists, the two-dimension code is segmented and identified, an advertising business URL (Uniform Resource Locator) is returned and is opened automatically, or an identification failed information is returned; when the two-dimension code does not exist, the system realizes object area segmentation by utilizing a visual attention mechanism, and then performs retrieval by utilizing the hierarchical matching strategy; the image is matched by a global feature I to obtain an image set S1, then a target and an image in the image set S1 are matched by a global feature II to obtain an image set S2; and finally, the target and an image in the image set S2 are matched by a local feature and a successful image retrieval information with related business information or an image retrieval failed information is returned. The system is capable of effectively increasing the retrieval operating efficiency on the premise of ensuring the retrieval precision.

Description

A kind of Business Information commending system and method based on image retrieval
Technical field
The present invention relates to a kind of Business Information commending system and method based on image retrieval, belong to information retrieval technique neckTerritory.
Background technology
Large-scale advertisement is an important competitive strategy of enterprise, is to improve commodity and the indispensable force of popularity of enterpriseDevice. When after a kind of new commodity listing, if consumer does not understand its title, purposes, purchase place, purchasing method,Just be difficult to expand market access, particularly, in the situation that Market competition, model change are accelerated greatly, enterprise passes throughLarge-scale advertising, can make consumer produce attraction to the product of this enterprise, and this exploits market for enterprise is tenDivide favourable.
Consumer wants to understand more information about manufacturer by advertisement or product at present just needs by scanning twoDimension code, but scanning Quick Response Code also has certain limitation, and the jumbo advertisements in such as roadside etc. just cannot be by scanning two dimensionCode is understood and is paid close attention to more product information.
Summary of the invention
The present invention is directed to the deficiency of the problems referred to above, propose a kind of Business Information commending system and method based on image retrieval,Consumer by utilize mobile phone by newspaper, magazine, TV programme and the jumbo advertisement in roadside etc. take pictures and upload,A series of processing such as background server process Quick Response Code location and identification or Target Segmentation, Image Feature Matching, return to thisThe main information of the manufacturing enterprise of product. The method can avoid scanning the limitation of Quick Response Code mode. Ensureing inspection simultaneouslyUnder Suo Jingdu prerequisite, effectively promote the operational efficiency of retrieval.
The present invention is that the technical scheme that solves the problems of the technologies described above proposition is:
A Business Information recommend method based on image retrieval, comprises the following steps:
Step 1, prestored information processing: obtain Quick Response Code, advertising pictures, video and businessman's relevant information of businessman,Using the key frame of advertising pictures and/or video as matching image, according to the key-frame extraction of advertising pictures and/or videoTwo kinds of global characteristics (can be color, shape or Texture eigenvalue) of figure picture and local feature (can be SIFT,SURF, ORB or BRIEF feature), and matching image and 2 D code information are carried out associated with businessman relevant information respectively.
Step 2, user takes advertisement photo by user side, and is uploaded to commending system.
Step 3, commending system detects the Quick Response Code in the advertisement photo of uploading, if Quick Response Code detected, by this Quick Response CodeThe businessman's Quick Response Code obtaining while processing with prestored information in step 1 is identified, if identify successfully, returns extensively to user sideAccuse businessman's relevant information corresponding to photo; If recognition failures returns and does not retrieve relevant information or forward step to user side4; If Quick Response Code do not detected, forward step 4 to.
Step 4, commending system employing vision noticing mechanism and algorithm of region growing carry out target to the advertisement photo of uploading and divideCut, extract the advertising image region in the advertisement photo of uploading, simultaneously using the advertising area in upload pictures as target figurePicture.
In advertising area, extract global characteristics one, global characteristics two and the local feature of target image.
When coupling, first utilize the global characteristics one of matching image and the global characteristics of target image one that step 1 obtains to carry outPreliminary search, obtains a preliminary search result; Then in the preliminary search result obtaining, utilize the overall situation of matching imageThe global characteristics two of feature two and target image carries out quadratic search, obtains a quadratic search result; Finally at this secondaryOn result for retrieval, utilize the local feature of matching image and the local feature of target image to retrieve coupling, obtain finalJoin result.
According to matching result, return to user side businessman's relevant information that advertising pictures is corresponding.
Preferred: image matching method is as follows:
Step 441, the similitude threshold value T of given global characteristics one1. The overall situation spy of the matching image that calculation procedure 1 obtainsLevy one and the similitude of the global characteristics one of target image, similitude is greater than to threshold value T1Image add S set1, collectionClose S1For preliminary search result. If S set1For sky, return to the information that does not retrieve corresponding merchant or similar businessman. IfS set1Be not empty, jump to step 442.
Step 442, the similitude threshold value T of given global characteristics two2. In S set1The global characteristics of middle calculating matching imageTwo and the similitude of the global characteristics two of target image, similitude is greater than to threshold value T2Image add S set2, setS2For quadratic search result. If S set2For sky, return to the information that does not retrieve corresponding merchant or similar businessman. If collectionClose S2Be not empty, jump to step 443.
Step 443, is used S set2The local feature of middle matching image mates with the local feature of target image, noteThe maximum of record local feature similitude.
Step 444, given local feature similitude threshold value TL. If the similitude maximum that step 443 obtains is greater than thresholdValue TL, represent to search for successfully, businessman's relevant information of this image association is returned to user side.
If the similitude maximum that step 443 obtains is less than threshold value TL, by S set2The image similarity of middle correspondenceDescending sort is also returned to the similar businessman of front n bar relevant information.
Preferred: the similitude of described global characteristics one, global characteristics two is all weighed by cosine angle.
Preferred: described businessman relevant information comprises businessman's title, address, contact method, the homepage URL of manufacturer, this is wideAccuse corresponding product and type.
Preferred: described global characteristics one is color global characteristics, and global characteristics two is shape global characteristics. Or the overall situationFeature one is shape global characteristics, and global characteristics two is color global characteristics. Described local feature is surf local feature.
A Business Information commending system based on image retrieval, carries out Business Information for the photo sending according to user sideRecommend, comprise input, system receiving module, system sending module, merchant advertisement picture storage module, businessman's featureMemory module, businessman's relevant information memory module, advertising image region extraction module, global characteristics one extraction module, completeOffice's feature two extraction modules, local feature extraction module, matching module, Quick Response Code extraction module and Quick Response Code identification mouldPiece, wherein:
Described input is used for inputting global characteristics one similitude threshold value, global characteristics two similitude threshold values, local feature phaseLike property threshold value, and by global characteristics one similitude threshold value, global characteristics two similitude threshold values, local feature similitude threshold valueBe pushed to matching module.
Described merchant advertisement picture storage module is used for receiving advertising pictures and/or video and the Quick Response Code of storage businessman,And be pushed to respectively global characteristics one extraction module, the overall situation using the key frame of advertising pictures and/or video as matching imageFeature two extraction modules, local feature extraction module; Merchant advertisement picture storage module is also for pushing away this Quick Response Code respectivelyGive Quick Response Code identification module and businessman's relevant information memory module.
Described businessman characteristic storage module is used for receiving global characteristics one extraction module, global characteristics two extraction modules, partThe information of global characteristics one, global characteristics two and the local feature of the matching image that characteristic extracting module pushes, and storageGlobal characteristics one, global characteristics two and the local feature of matching image.
Described businessman relevant information memory module is used for storing businessman's relevant information that advertising pictures is corresponding, and by businessman's featureGlobal characteristics one, global characteristics two, local feature and the merchant advertisement picture-storage mould of the matching image in memory moduleQuick Response Code in the piece respectively businessman relevant information corresponding with it carries out associated.
The picture that described system receiving module sends for receiving user side, and this picture is pushed to Quick Response Code extraction mouldPiece; The command information pushing for receiving Quick Response Code identification module, picture user side being sent according to this command information pushes awayGive advertising image region extraction module.
Quick Response Code on the picture that described Quick Response Code extraction module sends for detection of extraction user side, and by this Quick Response Code inspectionSurvey object information and send to Quick Response Code identification module.
Described Quick Response Code identification module is for according to Quick Response Code testing result information and merchant advertisement picture storage moduleQuick Response Code is identified, if identified successfully, pushes businessman's relevant information of this Quick Response Code association to system sending module;Do not retrieve relevant information or send to advertising image extracted region mould to system receiving module if recognition failures returnsPiece sends the order of information, if Quick Response Code do not detected, sends to advertising image extracted region to system receiving moduleModule sends the order of information.
Described advertising image region extraction module is communicated with for the target of the arresting power to picture extraction area maximumRegion, as advertising image region, is pushed to respectively global characteristics one extraction module, the overall situation by this advertising image region simultaneouslyFeature two extraction modules, local feature extraction module.
Described global characteristics one extraction module is for extracting in the advertising pictures that merchant advertisement picture storage module storesThe global characteristics one of matching image, and the global characteristics of this matching image one is pushed to businessman's characteristic storage module stores.Be used for the global characteristics one of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction module, and willThe global characteristics one of this target image is pushed to matching module.
Described global characteristics two extraction modules are for extracting in the advertising pictures that merchant advertisement picture storage module storesThe global characteristics two of matching image, and the global characteristics of this matching image two is pushed to businessman's characteristic storage module stores.Be used for the global characteristics two of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction module, and willThe global characteristics two of this target image is pushed to matching module.
Described local feature extraction module for extract in the advertising pictures that merchant advertisement picture storage module storesThe local feature of figure picture, and the local feature of this matching image is pushed to businessman's characteristic storage module stores. Be used for carryingGet the local feature of target image in the advertising image region that advertising image region extraction module pushes, and by this target figureThe local feature of picture is pushed to matching module.
Described matching module is used for calculating the similitude of the global characteristics one of matching image and the global characteristics one of target image,Similitude is greater than to threshold value T1Image add S set1. If S set1For sky, push not inspection to system sending moduleRope is to the information of corresponding merchant or similar businessman. If S set1Be not empty, in S set1The overall situation of middle calculating matching imageThe similitude of the global characteristics two of feature two and target image, is greater than threshold value T by similitude2Image add S set2，S set2For quadratic search result. If S set2For sky, push and do not retrieve corresponding merchant or class to system sending moduleLike the information of businessman. If S set2Be not empty, use S set2The local feature of middle matching image and the office of target imagePortion's feature is mated. If the maximum of local similarity is greater than threshold value TL, push this image to system sending moduleAssociated businessman's relevant information. If the maximum of local similarity is less than or equal to threshold value TL, by S set2In rightThe image similarity descending sort of answering also pushes the similar businessman of front n bar relevant information to system sending module.
The information that described system sending module pushes for receiving matching module, and this information is sent to user side.
Recommend a user side based on the Business Information of image retrieval, for the interaction of commending system, comprise transmission module,User's receiver module and display module, wherein:
The photo upload that described upper transmission module is used for obtaining taking pictures is to commending system.
Businessman's relevant information that described user's receiver module feeds back for receiving commending system, and this businessman's relevant information is pushed awayGive display module.
Described display module is used for showing businessman's relevant information.
A kind of Business Information commending system and method based on image retrieval of the present invention, compared to existing technology, has followingBeneficial effect:
The present invention takes an advertisement photo by user at mobile phone terminal, is transferred to commending system, and first system detects photoIn whether have Quick Response Code, if there is Quick Response Code, it cut apart and identify, return to advertisement Business URL automatically beatingOpen, or return to recognition failures information, if there is not Quick Response Code, system utilizes vision noticing mechanism realize target region to divideCut, first target is mated with imagery exploitation global characteristics one (as color characteristic) in database, will meet threshold value barSome width images of part are designated as image set S1, then by target and image set S1In imagery exploitation global characteristics two (as shapeFeature) mate, the some width images that meet threshold condition are designated as image set S2, finally by target and image set S2In imagery exploitation local feature (as surf feature) mate, and return to image retrieval success (or failure)Relevant Business Information. Algorithm of the present invention can avoid scanning the limitation of Quick Response Code, ensureing under retrieval precision prerequisite simultaneously,Effectively promote the operational efficiency of retrieval.
Brief description of the drawings
Fig. 1 is picture search algorithm flow chart of the present invention.
Fig. 2 is Business Information commending system handling process of the present invention.
Fig. 3 is the algorithm flow chart in specific embodiment of the invention.
Detailed description of the invention
Accompanying drawing discloses the structural representation of a preferred embodiment of the invention without limitation, detailed below with reference to accompanying drawingGround explanation technical scheme of the present invention.
Embodiment
A Business Information recommend method based on image retrieval, as shown in Figure 1, comprises the following steps:
Step 1, prestored information processing: obtain Quick Response Code, advertising pictures and/or video and businessman's relevant information of businessman,Businessman's relevant information comprises businessman's title, address, contact method, the homepage URL of manufacturer, corresponding product and the class of this advertisementType. Using the key frame of advertising pictures and/or video as matching image, carry according to the key frame of advertising pictures and/or videoGet two kinds of global characteristics (global characteristics can be color, shape or Texture eigenvalue) and the local feature of matching image(local feature can be SIFT, SURF, ORB or BRIEF feature), and matching image and 2 D code information are dividedNot do not carry out associated with businessman relevant information.
Step 2, user takes advertisement photo by user side, and is uploaded to commending system.
Step 3, commending system detects the Quick Response Code in the advertisement photo of uploading, if Quick Response Code detected, by this Quick Response CodeIdentify, if identify successfully, return to user side businessman's relevant information that advertisement photo is corresponding; If recognition failures toUser side returns and does not retrieve relevant information (that mainly return is the homepage URL of manufacturer) or forward step 4 to; If not inspectionMeasure Quick Response Code, forward step 4 to.
Step 4, commending system employing vision noticing mechanism and algorithm of region growing carry out target to the advertisement photo of uploading and divideCut, extract the advertising image region in the advertisement photo of uploading, simultaneously using the advertising area in upload pictures as target figurePicture.
In advertising area, extract global characteristics one, global characteristics two and the local feature of target image.
When coupling, first utilize the global characteristics one of matching image and the global characteristics of target image one that step 1 obtains to carry outPreliminary search, obtains a preliminary search result; Then in the preliminary search result obtaining, utilize the overall situation of matching imageThe global characteristics two of feature two and target image carries out quadratic search, obtains a quadratic search result; Finally at this secondaryOn result for retrieval, utilize the local feature of matching image and the local feature of target image to retrieve coupling, obtain finalJoin result.
According to matching result, return to user side businessman's relevant information that advertising pictures is corresponding.
Image matching method is as follows:
Step 441, the similitude threshold value T of given global characteristics one1. The overall situation spy of the matching image that calculation procedure 1 obtainsLevy one and the similitude of the global characteristics one of target image, similitude is greater than to threshold value T1Image add S set1, collectionClose S1For preliminary search result. If S set1For sky, return to the information that does not retrieve corresponding merchant or similar businessman. IfS set1Be not empty, jump to step 442.
Step 442, the similitude threshold value T of given global characteristics two2. In S set1The global characteristics of middle calculating matching imageTwo and the similitude of the global characteristics two of target image, similitude is greater than to threshold value T2Image add S set2, setS2For quadratic search result. If S set2For sky, return to the information that does not retrieve corresponding merchant or similar businessman. If collectionClose S2Be not empty, jump to step 443.
Step 443, is used S set2The local feature of middle matching image mates with the local feature of target image, noteThe maximum of record local feature similitude.
Step 444, given local feature similitude threshold value TL. If the similitude maximum that step 443 obtains is greater than thresholdValue TL, represent to search for successfully, businessman's relevant information of this image association is returned to user side.
If the similitude maximum that step 443 obtains is less than threshold value TL, by S set2The image similarity of middle correspondenceDescending sort is also returned to the similar businessman of front n bar relevant information.
The similitude of global characteristics one, global characteristics two is all weighed by cosine angle.
According to matching result, return to corresponding businessman's relevant information to user side.
Described global characteristics one is color global characteristics, and global characteristics two is shape global characteristics. Or global characteristics one isShape global characteristics, global characteristics two is color global characteristics. Described local feature is surf local feature.
A Business Information commending system based on image retrieval, carries out Business Information for the photo sending according to user sideRecommend, comprise input, system receiving module, system sending module, merchant advertisement picture storage module, businessman's featureMemory module, businessman's relevant information memory module, advertising image region extraction module, global characteristics one extraction module, completeOffice's feature two extraction modules, local feature extraction module, matching module, Quick Response Code extraction module and Quick Response Code identification mouldPiece, wherein:
Described input is used for inputting global characteristics one similitude threshold value, global characteristics two similitude threshold values, local feature phaseLike property threshold value, and by global characteristics one similitude threshold value, global characteristics two similitude threshold values, local feature similitude threshold valueBe pushed to matching module.
Described merchant advertisement picture storage module is used for receiving advertising pictures and/or video and the Quick Response Code of storage businessman,And be pushed to respectively global characteristics one extraction module, the overall situation using the key frame of advertising pictures and/or video as matching imageFeature two extraction modules, local feature extraction module; Merchant advertisement picture storage module is also for pushing away this Quick Response Code respectivelyGive Quick Response Code identification module and businessman's relevant information memory module.
Described businessman characteristic storage module is used for receiving global characteristics one extraction module, global characteristics two extraction modules, partThe information of global characteristics one, global characteristics two and the local feature of the matching image that characteristic extracting module pushes, and storageGlobal characteristics one, global characteristics two and the local feature of matching image.
Described businessman relevant information memory module is used for storing businessman's relevant information that advertising pictures is corresponding, and by businessman's featureGlobal characteristics one, global characteristics two, local feature and the merchant advertisement picture-storage mould of the matching image in memory moduleQuick Response Code in the piece respectively businessman relevant information corresponding with it carries out associated.
The picture that described system receiving module sends for receiving user side, and this picture is pushed to Quick Response Code extraction mouldPiece; The command information pushing for receiving Quick Response Code identification module, picture user side being sent according to this command information pushes awayGive advertising image region extraction module.
Quick Response Code on the picture that described Quick Response Code extraction module sends for detection of extraction user side, and by this Quick Response Code inspectionSurvey object information and send to Quick Response Code identification module.
Described Quick Response Code identification module is for according to Quick Response Code testing result information and merchant advertisement picture storage moduleQuick Response Code is identified, if identified successfully, pushes businessman's relevant information of this Quick Response Code association to system sending module;Do not retrieve relevant information or send to advertising image extracted region mould to system receiving module if recognition failures returnsPiece sends the order of information, if Quick Response Code do not detected, sends to advertising image extracted region to system receiving moduleModule sends the order of information.
Described advertising image region extraction module is communicated with for the target of the arresting power to picture extraction area maximumRegion, as advertising image region, is pushed to respectively global characteristics one extraction module, the overall situation by this advertising image region simultaneouslyFeature two extraction modules, local feature extraction module.
Described global characteristics one extraction module is for extracting in the advertising pictures that merchant advertisement picture storage module storesThe global characteristics one of matching image, and the global characteristics of this matching image one is pushed to businessman's characteristic storage module stores.Be used for the global characteristics one of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction module, and willThe global characteristics one of this target image is pushed to matching module.
Described global characteristics two extraction modules are for extracting in the advertising pictures that merchant advertisement picture storage module storesThe global characteristics two of matching image, and the global characteristics of this matching image two is pushed to businessman's characteristic storage module stores.Be used for the global characteristics two of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction module, and willThe global characteristics two of this target image is pushed to matching module.
Described local feature extraction module for extract in the advertising pictures that merchant advertisement picture storage module storesThe local feature of figure picture, and the local feature of this matching image is pushed to businessman's characteristic storage module stores. Be used for carryingGet the local feature of target image in the advertising image region that advertising image region extraction module pushes, and by this target figureThe local feature of picture is pushed to matching module.
Described matching module is used for calculating the similitude of the global characteristics one of matching image and the global characteristics one of target image,Similitude is greater than to threshold value T1Image add S set1. If S set1For sky, push not inspection to system sending moduleRope is to the information of corresponding merchant or similar businessman. If S set1Be not empty, in S set1The overall situation of middle calculating matching imageThe similitude of the global characteristics two of feature two and target image, is greater than threshold value T by similitude2Image add S set2，S set2For quadratic search result. If S set2For sky, push and do not retrieve corresponding merchant or class to system sending moduleLike the information of businessman. If S set2Be not empty, use S set2The local feature of middle matching image and the office of target imagePortion's feature is mated. If the maximum of local similarity is greater than threshold value TL, push this image to system sending moduleAssociated businessman's relevant information. If the maximum of local similarity is less than or equal to threshold value TL, by S set2In rightThe image similarity descending sort of answering also pushes the similar businessman of front n bar relevant information to system sending module.
The information that described system sending module pushes for receiving matching module, and this information is sent to user side.
For technical scheme of the present invention is described, the present embodiment provides a concrete example technical side of the present invention is describedCase, specifically comprises the following steps, as shown in Figure 2,3:
Step 1, prestored information processing: obtain Quick Response Code, advertising pictures and/or video and businessman's relevant information of businessman,Using the key frame of this advertising pictures and/or video as matching image, carry according to the key frame of this advertising pictures and/or videoGetting the color characteristic of matching image does as global characteristics two and surf descriptor as global characteristics one, shape facilityFor local feature, and by the color global characteristics of matching image, shape global characteristics, surf local feature and Quick Response CodeInformation is carried out associated with businessman relevant information respectively. In order to accelerate search procedure, in database, preserve in advance every width pictureThe overall situation, local feature file, when coupling, directly read in these files, omits computational process.
Businessman's relevant information comprises businessman's title, address, contact method, the homepage URL of manufacturer, product corresponding to this advertisementAnd type.
Step 2, user takes advertisement photo by user side, and is uploaded to commending system.
Step 3, commending system detects the Quick Response Code in the advertisement photo of uploading, if Quick Response Code detected, it is carried outCut apart and identify, if identify successfully, returning to user side the Business URL that advertisement photo is corresponding; If recognition failures returnsDo not retrieve relevant information; If Quick Response Code do not detected, forward step 4 to.
Step 4, feature extracting and matching
One, commending system adopts vision noticing mechanism and algorithm of region growing to carry out Target Segmentation to the advertisement photo of uploading,Advertising image region in the advertisement photo that extraction is uploaded, simultaneously using the advertisement photo of uploading as target image.
Advertising image generally has distinct color, forms stronger visual contrast with environment around, can be attractingNotice. According to this feature, the present invention adopts the Itti model based on vision noticing mechanism to obtain advertising image districtTerritory.
Step 411, picture color, brightness and 3 kinds of features of direction that extraction user side is taken advertisement photo image, colorFeature comprises red R, green G, blue B, tetra-kinds of components of yellow Y. Direction character extracts by Gabor wave filterθ=and 0 °, 45 °, 90 °, 135 ° } four direction passage. Brightness is the average of the red R of former figure, green G, tri-components of blue B.Then calculate the gaussian pyramid of every category feature figure according to the color characteristic, direction character, the brightness that extract.
Step 412, calculated characteristics disparity map: defined central authorities-periphery operator in Itti model, be used for simulating humanReceptive field mechanism, this mechanism is by realizing in the image subtraction of different scale in Gauss's gold tower. For color spyLevy and need to calculate red green and green red and blue Huang and two kinds of antagonism characteristic patterns of champac, use and then central authorities-periphery operator.It is red green and green red and blue that the gaussian pyramid of the color characteristic therefore obtaining according to step 411 adopts Itti model to calculateHuang and two kinds of antagonism characteristic patterns of champac, two kinds of antagonism characteristic patterns that obtain use respectively central authorities-periphery operator to obtain color spyLevy disparity map. The direction character obtaining according to step 411, the gaussian pyramid of brightness use respectively Itti modelCentral authorities-periphery operator obtain direction character disparity map and brightness disparity map.
RG(c,s)＝|(R(c)-G(c))Θ(G(s)-R(s))|
BY(c,s)＝|(B(c)-Y(c))Θ(Y(s)-B(s))|(2.1)
I(c,s)＝|I(c)ΘI(s)|(2.2)
O(c,s,θ)＝|O(c,θ)ΘO(s,θ),θ∈{0°,45°,90°,135°}(2.3)
In formula, RG (c, s) represents red green feature difference figure, and c is central yardstick, and c ∈ 2,3,4}, s is periphery yardstick,S=c+ δ, δ ∈ 3,4}, and R represents redness, and G represents green, and Θ represents central peripheral operator, BY (c, s) tableShow blue yellow feature difference figure, B represents blueness, and Y represents yellow, and I (c, s) represents brightness disparity map, O (c, s, θ)Represent direction character disparity map. Each pyramid can obtain 6 width feature difference figure.
Then respectively to the red green feature difference figure obtaining, blue yellow feature difference figure, brightness disparity map, direction characterDisparity map is done normalized.
Normalized processing method is as follows:
Step 4121, is mapped to the value of each pixel of feature difference figure in [0, M] interval, M be feature difference figureLarge value.
Step 4122, the mean value m of the coordinate of maximum M and other local maximum in calculated characteristics disparity map.
Step 4123, view picture feature difference image is multiplied byAnd then obtain normalized disparity map.
Step 413, generates final significantly figure: the disparity map after normalization is carried out being added across yardstick, obtain color, sideTo and significantly figure of 3 kinds of brightness, finally by these 3 kinds significantly figure distribute weights separately to superpose, generate final significantlyFigure.
Step 414, advertising area extracts. Significantly represent more can attracting notice for the upper significantly higher part of value of figure,The present embodiment obtains by regional connectivity algorithm the finally significantly target of the arresting power of the figure connection that step 413 generatesRegion.
The extracting method of the final significantly target connected region of the arresting power of figure is as follows:
Step 4141, is used OTSU maximum variance between clusters to obtain the final significantly threshold value of figure, will be less than the pixel of threshold valueAssignment is zero, remains unchanged higher than the part of threshold value. Definition set A, set B simultaneously.
Step 4142, according to the final significantly figure of sequential scanning from top to bottom from left to right, finds in final significantly figure significantlyBe worth the highest and not marked point, the Seed Points using this point as this regional connectivity, and join set A. IfCan not find Seed Points, exit step 414, forward step 415 to.
Step 4143, takes out the each point in A successively, and whether 4 neighbours point that judges each point meets remarkable value is notZero and be not labeled, if satisfy condition, this point is added to set B.
Step 4144, will be labeled as a little C in set B, and join in set A, is empty by set B assignmentCollection, forwards step 4143 to. If set A, B is all sky, forwards step 4142 to.
Step 415, the area of the each target connected region in the final significantly figure that comparison step 414 obtains successively, willThe region of area maximum, as the region that comprises advertisement, obtains advertising image region.
Two, according to the color global characteristics of advertising image extracted region target image, shape global characteristics and surf officePortion's feature.
The extracting method of described color global characteristics:
Original image is transformed into HSV space by rgb space, respectively H, S, V is quantized, then calculate respectivelyFirst three rank color moment μ of H, S, Vi、σiAnd si, i ∈ H, and S, V}, H represents tone, S represents saturation degree,V represents lightness. According to first three rank color moment μ of H, S, Vi、σiAnd siObtain color global characteristics Fc：
Fc＝[μH,σH,sH,μS,σS,sS,μV,σV,sV]。
Color global characteristics FcIt is the vector of one 9 dimension.
The extracting method of described shape global characteristics:
Step 421, is used kmeans to carry out color cluster to H component to image, and the present invention arranges k=5, clusterAfter can obtain more than one cluster areas, carry out successively above dilation erosion operation for the region of each class, remove thinThe noise of wisp.
Step 422, each marginal point in each class region that step 421 is obtained is by the ladder of its level, vertical directionDegree calculates the gradient direction of this point:
Gx＝f(x,y)-f(x-1,y)
Gy＝f(x,y)-f(x,y-1)。
θ(x,y)＝arctan(Gy/Gx)。
Wherein, θ (x, y) represents gradient direction, GxRepresent horizontal direction gradient, GyRepresent vertical gradient, f (x, y)Represent the gray value of marginal point.
Angular region is quantized into 36 grades, and statistics drops on the marginal point number in every one-level, uses marginal point sum to carry outNormalization, obtains the gradient orientation histogram of marginal point. Wherein, angular region, refers to that one 0 degree is to 360 degreeSpace.
Step 423, moves to the 0th grade by that the highest orientation angle frequency of occurrences one-level, and other are at different levels carries out same movingMove or loopy moving, and then obtain shape global characteristics, the shape facility obtaining like this possesses translation, yardstick and rotation notSex change.
The extracting method of described surf local feature:
Step 431, calculated product partial image, uses the cassette filter of different sizes in different directions image to be rolled upLong-pending, build metric space pyramid.
Step 432, tentatively judges by Hessian matrix whether this point is extreme point to the point in metric space pyramid,In 3 × 3 × 3 fields of this point, carry out non-maximum inhibition, using the point satisfying condition as candidate's extreme point, forMake extreme point more stable, accurately to locate, make extreme point reach sub-pix and sub-yardstick by the three-dimensional quadratic function of matchingThe precision of level. Obtain all extreme points in whole metric space pyramid.
Δ(H)＝DxxDyy-(0.9Dxx)2(3.4)
Wherein, (x, y) represents certain point under certain metric space, and D represents differentiate function, and Δ (H) represents hessian squareThe approximation of battle array determinant.
Step 433, in order to make Feature Descriptor have the characteristic of invariable rotary, need to calculate the direction spy of each extreme pointLevy. The extreme point obtaining taking step 432 is respectively the center of circle, in the neighborhood that 6S is radius, little with the Harr that is of a size of 4SMode plate to image processing, calculates the little wave response of this both direction, with Gaussian function pair at level, vertical both directionResponse is weighted, and S is the yardstick that this point is corresponding. Then use the whole round territory of fan-shaped traversal of 60 °, in each fan sectionCumulative institute Harr small echo response vector a little in territory, the direction of selection maximum vector is as the principal direction of this extreme point.
Step 434, centered by the extreme point obtaining by step 432, the square neighborhood that the length of side is 20S, by this neighborhoodY axle revolve the principal direction that goes to step 433 these extreme points that obtain, S is the yardstick that this point is corresponding. Square is dividedBecome 4 × 4 sub regions, calculate the interior sampled point of every sub regions with respect to the axial level of y, vertical both directionHarr small echo response vector, be designated as dx、dy, give Gauss's weights simultaneously. Then the response to sampled point andAbsolute value adds up, and is designated as ∑ dx，∑dy，∑|dx|，∑|dy|, therefore, a sub regions can obtainTo one 4 dimension vector, for whole square area, can obtain 4 × 4 × 4 dimension vectors, by this toAmount is normalized, and 64 dimensional vectors that finally obtain are the feature description vectors of this extreme point, are surfLocal feature.
Three, while coupling, at the retrieval module of image, the present embodiment adopts the strategy of hierarchical search, first utilizes step 1 to obtainTo the color global characteristics of matching image and the color global characteristics of target image carry out preliminary search, obtain one preliminaryResult for retrieval. Then in the preliminary search result obtaining, utilize the shape global characteristics of matching image and the shape of target imageShape global characteristics carries out quadratic search, obtains a quadratic search result. Finally in this quadratic search result, utilize couplingThe surf local feature of image and the surf local feature of target image are retrieved coupling, obtain final matching results.The present embodiment is in the final impregnable situation of retrieval precision, and the hierarchical search strategy of the present embodiment can significantly promote retrievalSpeed.
Matching process is as follows:
Step 441, given color similarity threshold value T1. The color global characteristics of the matching image that calculation procedure 1 obtains andThe color similarity of the color global characteristics of target image, is greater than color similarity threshold value T by color similarity1Image addEnter S set1, S set1For preliminary search result. If S set1For sky, return and do not retrieve corresponding merchant or similar businessThe information of family. If S set1Be not empty, jump to step 442.
Step 442, given shape similarity threshold value T2. In S set1Shape global characteristics and the order of middle calculating matching imageMark on a map the shape similarity of shape global characteristics of picture, is greater than shape similarity threshold value T by shape similarity2Image addEnter S set2, S set2For quadratic search result. If S set2For sky, return and do not retrieve corresponding merchant or similar businessThe information of family. If S set2Be not empty, jump to step 443.
The present embodiment is weighed the similitude of two vectors with cosine angle.
Step 443, given characteristic point threshold value Trod. In S set2Middle each surf part of calculating respectively matching imageEuclidean distance between the surf local feature point of characteristic point and target image, record minimum distance ND with time closelyNND, if ND/NND < TrodThe match is successful to represent two points, and statistical match is the logarithm of point successfully.
Step 444, given the match is successful some logarithm threshold value TL. If the maximum of step 443 logarithm that the match is successful putsThe point logarithm threshold value T that is greater than that the match is successfulL, represent to search for successfully, the homepage URL of manufacturer of this image association is returned to useFamily end.
If the maximum of step 443 logarithm that the match is successful puts is less than or equal to, the match is successful puts logarithm threshold value TL, willS set2The image similarity descending sort of middle correspondence is also returned to the similar businessman of front n bar relevant information.
Four,, according to matching result, return to user side businessman's relevant information that advertising pictures is corresponding.
According to image overall, local feature match condition, return to a kind of result in following 3 kinds of situations to user mobile phone end:1, do not retrieve corresponding merchant or similar Business Information; 2, the similar Business Information of n bar; 3, retrieve successfully corresponding businessFamily's information.
A Business Information commending system based on image retrieval, as shown in Figure 2, for the photo sending according to user sideCarry out Business Information recommendation, comprise input, system receiving module, system sending module, merchant advertisement picture-storage mouldPiece, businessman's characteristic storage module, businessman's relevant information memory module, advertising image region extraction module, color overall situation spyLevy extraction module, shape global characteristics extraction module, surf local feature extraction module, matching module, Quick Response Code extractionModule and Quick Response Code identification module, wherein:
Described input is used for inputting color similarity threshold value; Shape similarity threshold value; And the match is successful some logarithm threshold value,And by color similarity threshold value; Shape similarity threshold value; And the match is successful, and some logarithm threshold value is pushed to matching module.
Described merchant advertisement picture storage module is used for receiving advertising pictures and/or video and the Quick Response Code of storage businessman,And be pushed to respectively color global characteristics extraction module, shape using the key frame of advertising pictures and/or video as matching imageShape global characteristics extraction module, surf local feature extraction module; Merchant advertisement picture storage module is also for by this two dimensionCode is pushed to respectively Quick Response Code identification module and businessman's relevant information memory module.
Described businessman characteristic storage module be used for receiving color global characteristics extraction module, shape global characteristics extraction module,Color global characteristics, shape global characteristics and the surf part of the matching image that surf local feature extraction module pushesThe information of feature, and store color global characteristics, shape global characteristics and the surf local feature of matching image.
Described businessman relevant information memory module is used for storing businessman's relevant information that advertising pictures is corresponding, and by businessman's featureColor global characteristics, shape global characteristics, surf local feature and the merchant advertisement figure of the matching image in memory module2 D code information in the sheet memory module respectively businessman relevant information corresponding with it carries out associated.
The picture that described system receiving module sends for receiving user side, and this picture is pushed to Quick Response Code extraction mouldPiece; The command information pushing for receiving Quick Response Code identification module, picture user side being sent according to this command information pushes awayGive advertising image region extraction module.
Described Quick Response Code extraction module is for the Quick Response Code on the picture that extracts user side and send, and this Quick Response Code is detected to knotFruit information sends to Quick Response Code identification module.
Described Quick Response Code identification module is used for according to the testing result information of Quick Response Code and merchant advertisement picture storage moduleQuick Response Code identify, if identified successfully, push the relevant letter of businessman of this Quick Response Code association to system sending moduleBreath; Do not retrieve relevant information or send to advertising image extracted region to system receiving module if recognition failures returnsModule sends the order of information, if Quick Response Code do not detected, sends and carries to advertising image region to system receiving moduleDelivery piece sends the order of information.
Described advertising image region extraction module is communicated with for the target of the arresting power to picture extraction area maximumRegion, as advertising image region, is pushed to respectively color global characteristics extraction module, shape by this advertising image region simultaneouslyShape global characteristics extraction module, surf local feature extraction module.
Described color global characteristics extraction module is for extracting in the advertising pictures that merchant advertisement picture storage module storesThe color global characteristics of matching image, and the color global characteristics of this matching image is pushed to businessman's characteristic storage moduleStorage. Be used for the color overall situation spy of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction moduleLevy, and the color global characteristics of this target image is pushed to matching module.
Described shape global characteristics extraction module is for extracting in the advertising pictures that merchant advertisement picture storage module storesThe shape global characteristics of matching image, and the shape global characteristics of this matching image is pushed to businessman's characteristic storage moduleStorage. Be used for the shape overall situation spy of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction moduleLevy, and the shape global characteristics of this target image is pushed to matching module.
Described surf local feature extraction module is for extracting in the advertising pictures that merchant advertisement picture storage module storesThe surf local feature of matching image, and the surf local feature of this matching image is pushed to businessman's characteristic storage mouldPiece storage. Be used for the surf office of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction modulePortion's feature, and the surf local feature of this target image is pushed to matching module.
Described matching module is for calculating the face of the color global characteristics of matching image and the color global characteristics of target imageLook similitude, is greater than color similarity threshold value T by color similarity1Image add S set1. If S set1For sky,Push the information that does not retrieve corresponding merchant or similar businessman to system sending module. If S set1Be not empty, in setS1The shape similarity of the shape global characteristics of middle calculating matching image and the shape global characteristics of target image, by shape phaseBe greater than shape similarity threshold value T like property2Image add S set2, S set2For quadratic search result. If S set2For sky, push to system sending module the information that does not retrieve corresponding merchant or similar businessman. If S set2Be not empty,In S set2Middle each surf local feature point of matching image and the surf local feature point of target image of calculating respectivelyBetween Euclidean distance, record minimum distance ND and NND time closely, if ND/NND < TrodRepresent two pointsThe match is successful, and statistical match is the logarithm of point successfully. If the maximum of logarithm of the match is successful point is greater than that the match is successful pointLogarithm threshold value TL, push businessman's relevant information of this image association to system sending module. If statistical match is point successfullyThe maximum of the logarithm point logarithm threshold value T that is less than or equal to that the match is successfulL, by S set2The image similarity of middle correspondenceDescending sort also pushes the similar businessman of front n bar relevant information to system sending module.
The information that described system sending module pushes for receiving matching module, and this information is sent to user side.
Recommend a user side based on the Business Information of image retrieval, for the interaction of commending system, comprise transmission module,User's receiver module and display module, wherein:
The photo upload module that described upper transmission module is used for obtaining taking pictures is to commending system.
Businessman's relevant information that described user's receiver module feeds back for receiving commending system, and this businessman's relevant information is pushed awayGive display module.
Described display module is used for showing businessman's relevant information.
Above by reference to the accompanying drawings the preferred specific embodiment of described the present invention only for embodiments of the present invention are described, and notAs the restriction to aforementioned goal of the invention and claims content and scope, every foundation technical spirit of the present inventionTo any simple modification made for any of the above embodiments, equivalent variations and modification, all still belong to the technology of the present invention and rights protection modelFarmland.

Claims (8)

1. the Business Information recommend method based on image retrieval, is characterized in that, comprises the following steps:
Step 1, prestored information processing: obtain advertising pictures and/or video and businessman's relevant information, by this advertisementThe key frame of picture and/or video is as matching image, according to the key-frame extraction of this advertising pictures and/or videoGlobal characteristics one, global characteristics two and the local feature of figure picture, and matching image and businessman's relevant information are enteredRow is associated;
Step 2, user takes advertisement photo by user side, and is uploaded to commending system;
Step 3, commending system adopts vision noticing mechanism and algorithm of region growing to carry out target to the advertisement photo of uploadingCut apart, extract the advertising image region in the advertisement photo uploaded, simultaneously using the advertising area in upload pictures asTarget image;
In advertising area, extract global characteristics one, global characteristics two and the local feature of target image;
When coupling, the global characteristics one of matching image and the global characteristics one of target image that first utilize step 1 to obtainCarry out preliminary search, obtain a preliminary search result; Then in the preliminary search result obtaining, utilize match mapThe global characteristics two of picture and the global characteristics of target image two carry out quadratic search, obtain a quadratic search result;Finally in this quadratic search result, utilize the local feature of matching image and the local feature of target image to retrieveCoupling, obtains final matching results;
According to matching result, return to user side businessman's relevant information that advertising pictures is corresponding.
2. the Business Information recommend method based on image retrieval according to claim 1, is characterized in that: also comprise logicalCross Quick Response Code and recommend the method for Business Information, in the time that step 1 prestored information is processed, also obtain the Quick Response Code of businessman,2 D code information is carried out associated with businessman relevant information simultaneously; In step 3, commending system adopts vision attention machineBefore system and algorithm of region growing carry out Target Segmentation to the advertisement photo of uploading, commending system first detects upload wideAccuse the Quick Response Code in photo, if Quick Response Code detected, when prestored information in this Quick Response Code and step 1 is processed, obtainBusinessman's Quick Response Code identify, if identify successfully, return to user side businessman's relevant information that advertisement photo is corresponding;If recognition failures returns and does not retrieve relevant information or commending system employing vision noticing mechanism and region to user sideGrowth algorithm is carried out Target Segmentation to the advertisement photo of uploading; If Quick Response Code do not detected, commending system adopts and looksFeel that attention mechanism and algorithm of region growing carry out Target Segmentation to the advertisement photo of uploading.
3. the Business Information recommend method based on image retrieval according to claim 1, is characterized in that: images matchMethod is as follows:
Step 441, the similitude threshold value T of given global characteristics one1; The overall situation of the matching image that calculation procedure 1 obtainsThe similitude of the global characteristics one of feature one and target image, is greater than threshold value T by similitude1Image add set S1, S set1For preliminary search result; If S set1For sky, return and do not retrieve corresponding merchant or similar businessmanInformation; If S set1Be not empty, jump to step 442;
Step 442, the similitude threshold value T of given global characteristics two2; In S set1The overall situation of middle calculating matching imageThe similitude of the global characteristics two of feature two and target image, is greater than threshold value T by similitude2Image add setS2, S set2For quadratic search result; If S set2For sky, return and do not retrieve corresponding merchant or similar businessmanInformation; If S set2Be not empty, jump to step 443;
Step 443, is used S set2The local feature of middle matching image mates with the local feature of target image,Record the maximum of local feature similitude;
Step 444, given local feature similitude threshold value TL; If the similitude maximum that step 443 obtains is largeIn threshold value TL, represent to search for successfully, businessman's relevant information of this image association is returned to user side;
If the similitude maximum that step 443 obtains is less than threshold value TL, by S set2The image similarity of middle correspondenceDegree descending sort is also returned to the similar businessman of front n bar relevant information.
4. the Business Information recommend method based on image retrieval according to claim 3, is characterized in that: the described overall situationThe similitude of feature one, global characteristics two is all weighed by cosine angle.
5. the Business Information recommend method based on image retrieval according to claim 1, is characterized in that: described businessmanRelevant information comprises businessman's title, address, contact method, the homepage URL of manufacturer, corresponding product and the type of this advertisement.
6. the Business Information recommend method based on image retrieval according to claim 1, is characterized in that: the described overall situationFeature one is color global characteristics, and global characteristics two is shape global characteristics; Or global characteristics one is the shape overall situationFeature, global characteristics two is color global characteristics; Described local feature is surf local feature.
7. based on the arbitrary described Business Information commending system based on image retrieval of claim 1 to 6, for basisThe photo that user side sends carries out Business Information recommendation, it is characterized in that: comprise input, system receiving module,System sending module, merchant advertisement picture storage module, businessman's characteristic storage module, businessman's relevant information storage mouldPiece, advertising image region extraction module, global characteristics one extraction module, global characteristics two extraction modules, local specialLevy extraction module, matching module, Quick Response Code extraction module and Quick Response Code identification module, wherein:
Described input is used for inputting global characteristics one similitude threshold value, global characteristics two similitude threshold values, local specialLevy similitude threshold value, and by global characteristics one similitude threshold value, global characteristics two similitude threshold values, local feature phaseBe pushed to matching module like property threshold value;
Described merchant advertisement picture storage module is for receiving advertising pictures and/or video and the two dimension of storage businessmanCode, and be pushed to respectively global characteristics one using the key frame of advertising pictures and/or video as matching image and extract mouldPiece, global characteristics two extraction modules, local feature extraction module; Merchant advertisement picture storage module was also for shouldQuick Response Code is pushed to respectively Quick Response Code identification module and businessman's relevant information memory module;
Described businessman characteristic storage module be used for receiving global characteristics one extraction module, global characteristics two extraction modules,The information of global characteristics one, global characteristics two and the local feature of the matching image that local feature extraction module pushes,And store global characteristics one, global characteristics two and the local feature of matching image;
Described businessman relevant information memory module is used for storing businessman's relevant information that advertising pictures is corresponding, and by businessmanGlobal characteristics one, global characteristics two, local feature and the merchant advertisement figure of the matching image in characteristic storage moduleQuick Response Code in the sheet memory module respectively businessman relevant information corresponding with it carries out associated;
The picture that described system receiving module sends for receiving user side, and this picture is pushed to Quick Response Code extractionModule; The command information pushing for receiving Quick Response Code identification module, sends user side according to this command informationPicture is pushed to advertising image region extraction module;
Quick Response Code on the picture that described Quick Response Code extraction module sends for detection of extraction user side, and by this two dimensionCode testing result information sends to Quick Response Code identification module;
Described Quick Response Code identification module is used for according to Quick Response Code testing result information and merchant advertisement picture storage moduleQuick Response Code identify, if identified successfully, push businessman's phase of this Quick Response Code association to system sending modulePass information; Do not retrieve relevant information or send to advertising image to system receiving module if recognition failures returnsRegion extraction module sends the order of information, if Quick Response Code do not detected, sends to extensively to system receiving moduleAccuse the order of image-region extraction module transmission information;
Described advertising image region extraction module connects for the target of the arresting power to picture extraction area maximumLogical region is as advertising image region, this advertising image region is pushed to respectively simultaneously global characteristics one extraction module,Global characteristics two extraction modules, local feature extraction module;
Described global characteristics one extraction module is for extracting in the advertising pictures that merchant advertisement picture storage module storesThe global characteristics one of matching image, and the global characteristics of this matching image one is pushed to businessman's characteristic storage moduleStorage; Be used for the overall situation spy of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction moduleLevy one, and the global characteristics of this target image one is pushed to matching module;
Described global characteristics two extraction modules are for extracting in the advertising pictures that merchant advertisement picture storage module storesThe global characteristics two of matching image, and the global characteristics of this matching image two is pushed to businessman's characteristic storage moduleStorage; Be used for the overall situation spy of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction moduleLevy two, and the global characteristics of this target image two is pushed to matching module;
Described local feature extraction module is for extracting in the advertising pictures that merchant advertisement picture storage module storesThe local feature of matching image, and the local feature of this matching image is pushed to businessman's characteristic storage module stores;Be used for the local feature of the target image in the advertising image region of extracting the propelling movement of advertising image region extraction module, andThe local feature of this target image is pushed to matching module;
Described matching module is for calculating the similar of the global characteristics one of matching image and the global characteristics one of target imageProperty, similitude is greater than to threshold value T1Image add S set1; If S set1For sky, to system sending modulePush the information that does not retrieve corresponding merchant or similar businessman; If S set1Be not empty, in S set1Middle calculatingThe similitude of the global characteristics two of figure picture and the global characteristics two of target image, is greater than threshold value T by similitude2FigurePicture adds S set2, S set2For quadratic search result; If S set2For sky, push not to system sending moduleRetrieve the information of corresponding merchant or similar businessman; If S set2Be not empty, use S set2Middle matching imageLocal feature mates with the local feature of target image; If the maximum of local similarity is greater than threshold value TL，Push businessman's relevant information of this image association to system sending module; If the maximum of local similarity is less thanOr equal threshold value TL, by S set2The image similarity descending sort of middle correspondence by the similar businessman of front n bar phasePass information pushes to system sending module;
The information that described system sending module pushes for receiving matching module, and this information is sent to user side.
8. recommend a user side based on the arbitrary described Business Information based on image retrieval of claim 1 to 7, forThe interaction of commending system, is characterized in that: comprise transmission module, user's receiver module and display module, wherein:
The photo upload that described upper transmission module is used for obtaining taking pictures is to commending system;
Businessman's relevant information that described user's receiver module feeds back for receiving commending system, and by relevant this businessman letterBreath is pushed to display module;
Described display module is used for showing businessman's relevant information.
CN201610078235.1A 2016-02-03 2016-02-03 A kind of Business Information recommender system and method based on image retrieval Active CN105608230B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610078235.1A CN105608230B (en) 2016-02-03 2016-02-03 A kind of Business Information recommender system and method based on image retrieval

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610078235.1A CN105608230B (en) 2016-02-03 2016-02-03 A kind of Business Information recommender system and method based on image retrieval

Publications (2)

Publication Number Publication Date
CN105608230A true CN105608230A (en) 2016-05-25
CN105608230B CN105608230B (en) 2019-05-31

Family

ID=55988169

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610078235.1A Active CN105608230B (en) 2016-02-03 2016-02-03 A kind of Business Information recommender system and method based on image retrieval

Country Status (1)

Country Link
CN (1) CN105608230B (en)

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530007A (en) * 2016-11-08 2017-03-22 四川蜂搜科技有限公司 Advertisement pushing method based on user hobbies
CN106528662A (en) * 2016-10-20 2017-03-22 中山大学 Quick retrieval method and system of vehicle image on the basis of feature geometric constraint
CN106792004A (en) * 2016-12-30 2017-05-31 北京小米移动软件有限公司 Content item method for pushing, apparatus and system
CN106779819A (en) * 2016-11-29 2017-05-31 特斯联（北京）科技有限公司 Intelligent advertisement based on the identification of AR high in the clouds recommends method, system and server
CN107590279A (en) * 2017-09-28 2018-01-16 韦彩霞 A kind of smart travel commending system
WO2018019170A1 (en) * 2016-07-27 2018-02-01 阿里巴巴集团控股有限公司 Resource pushing method and apparatus
CN107844803A (en) * 2017-10-30 2018-03-27 中国银联股份有限公司 The method and apparatus that a kind of picture compares
CN108304847A (en) * 2017-11-30 2018-07-20 腾讯科技（深圳）有限公司 Image classification method and device, personalized recommendation method and device
CN108520058A (en) * 2018-03-30 2018-09-11 维沃移动通信有限公司 A kind of Business Information recommends method and mobile terminal
CN108921829A (en) * 2018-06-20 2018-11-30 广州晖恒广告策划有限公司 A kind of advertisement design method for objectively evaluating of view-based access control model attention mechanism
CN109032810A (en) * 2017-06-09 2018-12-18 阿里巴巴集团控股有限公司 The starting method and apparatus of business procedure
CN109416693A (en) * 2016-06-24 2019-03-01 塞吉尼亚株式会社 Server unit, terminal installation and information processing method
CN109614512A (en) * 2018-11-29 2019-04-12 亿嘉和科技股份有限公司 A kind of power equipment search method based on deep learning
CN109657083A (en) * 2018-12-27 2019-04-19 广州华迅网络科技有限公司 The method for building up and device in textile picture feature library
CN110019915A (en) * 2018-07-25 2019-07-16 北京京东尚科信息技术有限公司 Detect the method, apparatus and computer readable storage medium of picture
CN110059207A (en) * 2019-04-04 2019-07-26 Oppo广东移动通信有限公司 Processing method, device, storage medium and the electronic equipment of image information
CN110119460A (en) * 2019-05-16 2019-08-13 广东三维家信息科技有限公司 Image search method, device and electronic equipment
CN110809187A (en) * 2019-10-31 2020-02-18 Oppo广东移动通信有限公司 Video selection method, video selection device, storage medium and electronic equipment
CN111178306A (en) * 2019-12-31 2020-05-19 维沃移动通信有限公司 Display control method and electronic equipment
CN111325777A (en) * 2018-12-14 2020-06-23 北京沃东天骏信息技术有限公司 Method and apparatus for processing image
WO2020125100A1 (en) * 2018-12-21 2020-06-25 华为技术有限公司 Image search method, apparatus, and device
CN112418996A (en) * 2020-11-30 2021-02-26 珠海采筑电子商务有限公司 Recommendation method and system for elevator suppliers
CN112488049A (en) * 2020-12-16 2021-03-12 哈尔滨市科佳通用机电股份有限公司 Fault identification method for foreign matter clamped between traction motor and shaft of motor train unit
CN112597391A (en) * 2020-12-25 2021-04-02 厦门大学 Dynamic recursion mechanism-based hierarchical reinforcement learning recommendation system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080118151A1 (en) * 2006-11-22 2008-05-22 Jean-Yves Bouguet Methods and apparatus for retrieving images from a large collection of images
CN101789111A (en) * 2009-12-30 2010-07-28 中国科学院自动化研究所 Site advertisement and system for real-time interaction of electronic advertisement data
CN102542058A (en) * 2011-12-29 2012-07-04 天津大学 Hierarchical landmark identification method integrating global visual characteristics and local visual characteristics
CN105160561A (en) * 2015-09-30 2015-12-16 北京中搜网络技术股份有限公司 Situational picture advertising method and apparatus

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080118151A1 (en) * 2006-11-22 2008-05-22 Jean-Yves Bouguet Methods and apparatus for retrieving images from a large collection of images
CN101789111A (en) * 2009-12-30 2010-07-28 中国科学院自动化研究所 Site advertisement and system for real-time interaction of electronic advertisement data
CN102542058A (en) * 2011-12-29 2012-07-04 天津大学 Hierarchical landmark identification method integrating global visual characteristics and local visual characteristics
CN105160561A (en) * 2015-09-30 2015-12-16 北京中搜网络技术股份有限公司 Situational picture advertising method and apparatus

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109416693A (en) * 2016-06-24 2019-03-01 塞吉尼亚株式会社 Server unit, terminal installation and information processing method
WO2018019170A1 (en) * 2016-07-27 2018-02-01 阿里巴巴集团控股有限公司 Resource pushing method and apparatus
CN106528662A (en) * 2016-10-20 2017-03-22 中山大学 Quick retrieval method and system of vehicle image on the basis of feature geometric constraint
CN106530007A (en) * 2016-11-08 2017-03-22 四川蜂搜科技有限公司 Advertisement pushing method based on user hobbies
CN106779819A (en) * 2016-11-29 2017-05-31 特斯联（北京）科技有限公司 Intelligent advertisement based on the identification of AR high in the clouds recommends method, system and server
CN106792004A (en) * 2016-12-30 2017-05-31 北京小米移动软件有限公司 Content item method for pushing, apparatus and system
CN106792004B (en) * 2016-12-30 2020-09-15 北京小米移动软件有限公司 Content item pushing method, device and system
CN109032810A (en) * 2017-06-09 2018-12-18 阿里巴巴集团控股有限公司 The starting method and apparatus of business procedure
CN107590279A (en) * 2017-09-28 2018-01-16 韦彩霞 A kind of smart travel commending system
CN107844803A (en) * 2017-10-30 2018-03-27 中国银联股份有限公司 The method and apparatus that a kind of picture compares
CN107844803B (en) * 2017-10-30 2021-12-28 中国银联股份有限公司 Picture comparison method and device
US11238315B2 (en) 2017-11-30 2022-02-01 Tencent Technology (Shenzhen) Company Limited Image classification method, personalized recommendation method, computer device and storage medium
CN108304847A (en) * 2017-11-30 2018-07-20 腾讯科技（深圳）有限公司 Image classification method and device, personalized recommendation method and device
CN108304847B (en) * 2017-11-30 2021-09-28 腾讯科技（深圳）有限公司 Image classification method and device and personalized recommendation method and device
CN108520058A (en) * 2018-03-30 2018-09-11 维沃移动通信有限公司 A kind of Business Information recommends method and mobile terminal
CN108520058B (en) * 2018-03-30 2021-09-24 维沃移动通信有限公司 Merchant information recommendation method and mobile terminal
CN108921829A (en) * 2018-06-20 2018-11-30 广州晖恒广告策划有限公司 A kind of advertisement design method for objectively evaluating of view-based access control model attention mechanism
CN110019915A (en) * 2018-07-25 2019-07-16 北京京东尚科信息技术有限公司 Detect the method, apparatus and computer readable storage medium of picture
CN110019915B (en) * 2018-07-25 2022-04-12 北京京东尚科信息技术有限公司 Method and device for detecting picture and computer readable storage medium
CN109614512A (en) * 2018-11-29 2019-04-12 亿嘉和科技股份有限公司 A kind of power equipment search method based on deep learning
CN109614512B (en) * 2018-11-29 2022-02-22 亿嘉和科技股份有限公司 Deep learning-based power equipment retrieval method
CN111325777A (en) * 2018-12-14 2020-06-23 北京沃东天骏信息技术有限公司 Method and apparatus for processing image
WO2020125100A1 (en) * 2018-12-21 2020-06-25 华为技术有限公司 Image search method, apparatus, and device
CN109657083A (en) * 2018-12-27 2019-04-19 广州华迅网络科技有限公司 The method for building up and device in textile picture feature library
CN110059207A (en) * 2019-04-04 2019-07-26 Oppo广东移动通信有限公司 Processing method, device, storage medium and the electronic equipment of image information
CN110119460A (en) * 2019-05-16 2019-08-13 广东三维家信息科技有限公司 Image search method, device and electronic equipment
CN110809187A (en) * 2019-10-31 2020-02-18 Oppo广东移动通信有限公司 Video selection method, video selection device, storage medium and electronic equipment
CN111178306A (en) * 2019-12-31 2020-05-19 维沃移动通信有限公司 Display control method and electronic equipment
CN111178306B (en) * 2019-12-31 2024-01-26 维沃移动通信有限公司 Display control method and electronic equipment
CN112418996A (en) * 2020-11-30 2021-02-26 珠海采筑电子商务有限公司 Recommendation method and system for elevator suppliers
CN112488049A (en) * 2020-12-16 2021-03-12 哈尔滨市科佳通用机电股份有限公司 Fault identification method for foreign matter clamped between traction motor and shaft of motor train unit
CN112597391A (en) * 2020-12-25 2021-04-02 厦门大学 Dynamic recursion mechanism-based hierarchical reinforcement learning recommendation system
CN112597391B (en) * 2020-12-25 2022-08-12 厦门大学 Dynamic recursion mechanism-based hierarchical reinforcement learning recommendation system

Also Published As

Publication number Publication date
CN105608230B (en) 2019-05-31

Similar Documents

Publication Publication Date Title
CN105608230A (en) Image retrieval based business information recommendation system and image retrieval based business information recommendation method
Odone et al. Building kernels from binary strings for image matching
Huang et al. Copy-move forgery detection for image forensics using the superpixel segmentation and the Helmert transformation
US9578107B2 (en) Data capture and identification system and process
Delponte et al. SVD-matching using SIFT features
Girod et al. Mobile visual search: Architectures, technologies, and the emerging MPEG standard
CN205721777U (en) Visual search equipment and system
Xiao et al. Efficient shadow removal using subregion matching illumination transfer
CN104991954B (en) Shopping search system based on picture search and its implementation
CN106127748B (en) A kind of characteristics of image sample database and its method for building up
JP2010518507A (en) Feature matching method
Takacs et al. Rotation-invariant fast features for large-scale recognition and real-time tracking
Phan et al. Content-based retrieval of logo and trademarks in unconstrained color image databases using color edge gradient co-occurrence histograms
Zhong et al. Dense moment feature index and best match algorithms for video copy-move forgery detection
Soni et al. Keypoints based enhanced multiple copy‐move forgeries detection system using density‐based spatial clustering of application with noise clustering algorithm
Ahmed et al. A comparative analysis of image copy-move forgery detection algorithms based on hand and machine-crafted features
Wang et al. Dynamic SFM: detecting scene changes from image pairs
Iwamoto et al. BRIGHT: A scalable and compact binary descriptor for low-latency and high accuracy object identification
Khayeat et al. Improved DSIFT descriptor based copy-rotate-move forgery detection
Zhang et al. Video shadow removal using spatio‐temporal illumination transfer
JP6151908B2 (en) Learning device, identification device, and program thereof
CN113011506B (en) Texture image classification method based on deep fractal spectrum network
Liu et al. Video retrieval based on object discovery
Lee et al. Remote sensing image registration using equivariance features
Valle et al. Content-based retrieval of images for cultural institutions using local descriptors

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant