CN105184212A

CN105184212A - Image processing server

Info

Publication number: CN105184212A
Application number: CN201510159520.1A
Authority: CN
Inventors: D·K·梅热; B·A·福尔肯斯
Original assignee: IMAGE SEARCHER Inc
Current assignee: IMAGE SEARCHER Inc
Priority date: 2014-04-04
Filing date: 2015-04-03
Publication date: 2015-12-23
Also published as: CA2885835A1

Abstract

An image recognition approach employs both computer generated and manual image reviews to generate image tags characterizing an image. The computer generated and manual image reviews can be performed sequentially or in parallel. The generated image tags may be provided to a requester in real-time, be used to select an advertisement, and/or be used as the basis of an internet search. In some embodiments generated image tags are used as a basis for an upgraded image review. A confidence of a computer generated image review may be used to determine whether or not to perform a manual image review.

Description

Image processing server

the cross reference of related application

The application be on May 1st, 2014 submit there is application number 14/267, the part continuity of U.S.'s non-provisional application of by name " image procossing " of 840, it requires the right of priority of the provisional application 61/956,927 submitted on May 1st, 2013 then; The application requires right of priority and the rights and interests of following U.S. Provisional Patent Application further:

Submit on April 4th, 2014 and there is application number 61/975, " visual search " of 691;

Submit on April 7th, 2014 and there is application number 61/976, " the visual search advertisement " of 494;

Submit on May 1st, 2014 and there is application number 61/987, " image procossing " of 156;

Submit on July 31st, 2014 and there is application number 62/031, " real-time target in image procossing is selected " of 397;

Submit on October 27th, 2014 and there is application number 62/069, " the distributed image process " of 160; And

Submit on November 25th, 2014 and there is application number 62/084, " the selective image process " of 509.

All above patented claims are merged in herein by reference.

Technical field

The application is in the field of image procossing, more specifically, is in the field of the content of characterized images.

Background technology

Usually from image, information extraction is more difficult than information extraction from text data.But most information is found in the picture.The fiduciary level height of automatic image distinguishing system depends on the content of image.Such as, optical character recognition OCR is more reliable than face recognition.The target of image identification adds label to image.Label refers to the identification of the label (word) of the content of characterized images.Such as, the image of automobile can be added the label of word " automobile ", " Ford Granada (FordGranada) " or " having the white Ford Granada in 1976 damaging headlight ".These labels comprise the information of varying number, and can change in purposes thus.

Summary of the invention

The embodiment of the application comprises the method for working along both lines of adding image tag.First method is on image, perform automatic image identification.This automatic image identification causes the examination & verification (review) of image.This image audit comprises one or more label of the content of recognition image and also comprises the tolerance of the degree of confidence of the fiduciary level representing automated graphics identification alternatively.This comprises the manual interpolation label of image to second method that image adds in tagged method.Manual interpolation label comprises people and watches each image, consider the content of image and manually provide the label representing picture material.Automatic image identification has the advantage that time goes up or pecuniary cost can be relatively low analyzing each image.The manual label interpolation of image has higher accuracy and the advantage of fiduciary level.

Embodiments of the invention combine automated graphics identification and manual image identification.In certain embodiments, first automated graphics identification is performed.One or more labels that the image audit produced both had comprised characterized images usually also comprise the tolerance of the degree of confidence of these labels in accuracy.If degree of confidence is higher than predetermined threshold, so these labels associate with this image and are provided as the output adding label process.If degree of confidence is lower than predetermined threshold, so the manual examination & verification of this image is performed.Manual examination & verification causes the additional of the content of characterized images and/or different label.In certain embodiments, the manual examination & verification of automatic image identification and image is performed concurrently.If automated graphics identification causes one or more label to have degree of confidence higher than predetermined threshold, manually audit and cancelled alternatively or stop.

In certain embodiments, the identification of image can be upgraded.The upgrading of image identification process comprises the requirement of the label of the further or improvement of the content for expression image.Such as, if automated graphics identification causes label " white car ", the upgrading of this identification can cause label " white Ford Granada ".In certain embodiments, the examination & verification of upgrading utilizes expert human reviewer.Such as, above example can comprise the human reviewer utilizing and have the professional knowledge of automobile.Other example of the professional knowledge of human reviewer comes into question in other place herein.

Various embodiment of the present invention comprises guiding and improves the feature that the accuracy of image identification also minimizes cost simultaneously.By way of example, these features comprise effective utilization of population auditor, the delivered in real-time of image tag and/or the seamless upgrade of image identification.The method of image identification disclosed herein is used for generating the image tag being suitable for performing internet hunt and/or selecting advertisement alternatively.Such as, in certain embodiments, image tag be automatically used to perform Google search and/or based on Google AdWords and sell advertisement.

Various embodiment of the present invention comprises image processing system, and this image processing system comprises the I/O being configured to transmit image and image tag on a communication network; Automatic recognition interface, be configured to image to receive from the examination & verification of the Practical computer teaching of the image of automatic recognition system to automatic recognition system transmission, the examination & verification of this Practical computer teaching comprises one or more labels of the content of recognition image; Destination logic, is configured to the first destination determining to send images to, and manually audits for carrying out first by the first mankind auditor to this image; Image puts up logic, is configured to image to put up to destination; Examination & verification logic, be configured to receive the manual examination & verification from the image of destination and the examination & verification of receiving computer generation, this manually audits the one or more image tags comprising the content of recognition image; Response logic, is configured to provide the image tag of the examination & verification of Practical computer teaching and the image tag of manually examination & verification to communication network; Storer, is configured to storage figure picture; And microprocessor, be configured at least perform destination logic.

Various embodiment of the present invention comprises the method for process image, and the method comprises the image received from image source; To automated graphics recognition system distribution diagram picture; Receive the examination & verification from the Practical computer teaching of automated graphics recognition system, the examination & verification of this Practical computer teaching comprises and is assigned to one or more image tag of this image and the tolerance of degree of confidence by automated graphics recognition system, the tolerance of the image tag that the tolerance of this degree of confidence the is assigned to image correctly degree of confidence of the content of characterized images; Image is placed in image queue; Determine destination; Put up to the first destination by the image being used for manually examination & verification, this first destination comprises the display device of mankind's image censor; And reception is audited from the manual image of the image of destination, this image audit comprises the one or more image tags being assigned to image by mankind's image censor, the content of this one or more image tag characterized images.

The various embodiments of the application comprise image source, and it comprises the camera being configured to gather image; Display, is configured to present image to user; Eye trace logic, is configured to the action of the one or more eyes detecting user; Optional image tagged logic, is configured to mark to be placed on image, this mark be configured to indicating image particular subset and in response to detected action; Display logic, is configured to be presented on image by mark in real time; I/O, is configured to provide image to computer network; And processor, be configured at least perform display logic.

The various embodiments of the application comprise image source, and it comprises the camera being configured to gather image; Display, is configured to present image to user; Eye trace logic, is configured to the action of the one or more eyes detecting user; Image tagged logic, be configured to make the particular subset of user's indicating image and the object given prominence in this subset, this instruction is in response to detected action; Display logic, is configured to be presented on image by given prominence in real time; I/O, is configured to the instruction providing image and particular subset to computer network; And processor, be configured at least perform display logic.

Various embodiment of the present invention comprises image source, and it comprises the camera being configured to gather image; Display, is configured to present image to user; Select logic, be arranged to selection; Image tagged logic, be configured to make the particular subset of user's indicating image and the object given prominence in this subset, this instruction is in response to detected finger; I/O, is configured to the instruction providing image and particular subset to computer network; Display logic, is configured to show image in real time and the image tag received from computer network in response to image display, the content of this image tag characterized images; And processor, be configured to perform at least display logic.

Various embodiment of the present invention comprises image processing system, and this image processing system comprises the I/O being configured to transmit image sequence and image tag on a communication network; Optional recognition interface automatically, be configured to by image sequence to automatic recognition system transmission and the examination & verification received from the Practical computer teaching of the image of automatic recognition system, the examination & verification of this Practical computer teaching comprises one or more labels of the content of recognition image; Destination logic, is configured to the first destination determining image sequence to be sent to, and manually audits for carrying out first by the first human reviewer to this image sequence; Image puts up logic, is configured to image sequence to put up to destination; Examination & verification logic, be configured to receive the manual examination & verification from the image sequence of destination and the examination & verification of receiving computer generation alternatively, this manually audits the one or more image tags of the action comprised in recognition image sequence; Response logic, is configured to provide the image tag of the examination & verification of Practical computer teaching and the image tag of manually examination & verification to communication network; Storer, is configured to store image sequence; And microprocessor, be configured at least perform destination logic.

Various embodiment of the present invention comprises the method for process image, and the method comprises: receive one or more first descriptors from the image of Terminal Server Client at image processing server via communication network; The second descriptor the first received descriptor and this locality being stored in image processing server compares to determine whether the set of the first descriptors match second descriptor; Set in response to the first descriptor and the second descriptor matches, one or more image tags that acquisition is associated with the set of the second descriptor and stores; And provide one or more image tag to client.

Various embodiment of the present invention is included in the method for image processing server process image, and the method comprises: the data receiving image from Terminal Server Client and this image of characterization; Determine the destination for image, this destination is associated with mankind's image audit person, mating between the data of the determination feature based image of this destination and the speciality of human reviewer; Image is puted up to determined destination; Receive the one or more image tags from the characterized images of destination; And provide one or more image tag to client.

Each embodiment of the present invention comprises the method for process image, and the method comprises: the data receiving the characterized images from mobile device, the image of the feature of descriptor that is that this data characterization comprises identified image or image; The data genaration image tag of feature based image; Image tag is provided to mobile device.

Various embodiment of the present invention comprises the method for process image, and the method comprises: use portable equipment to receive image; Use the feature of the processor recognition image of portable equipment; Feature is provided to remote image processing server via communication network; Receive the image tag from the feature based of image processing server; And show image tag on the display of a portable device.

Various embodiment of the present invention comprises the method for process image, and the method comprises: use portable equipment to receive image; Use the feature of the processor recognition image of portable equipment; Based on identified feature deduced image descriptor; Symbol is provided a description to remote image processing server via communication network; Receive the image tag based on descriptor from image processing server; And show image tag on the display of a portable device.

Various embodiment of the present invention comprises the method for process image, and the method comprises: use portable equipment to receive image; Use the feature of the processor recognition image of portable equipment; Based on identified feature deduced image descriptor; Image descriptor is compared with the set of the image descriptor be stored in before on portable equipment determine whether to there is coupling between image descriptor and the set of stored image descriptor; Mate if existed between image descriptor and the set of stored image descriptor, obtain from the storer of portable equipment the one or more image tags be associated with the set of image descriptor; By obtained one or more graphical label displays on the display of a portable device.

Various embodiment of the present invention comprises the method for process image, and the method comprises: use portable equipment to receive image; Use the feature of the processor recognition image of portable equipment; Based on identified feature deduced image descriptor; Image descriptor is compared with the set of the image descriptor be stored in before on portable equipment determine whether to there is coupling between image descriptor and the set of stored image descriptor; Based on mating classified image between image descriptor and the set of stored image descriptor; The classification of image and image is sent to remote image processing server; Receive the one or more image tags based on image; And by one or more image tag display on the display of a portable device.

Various embodiment of the present invention comprises image processing system, and it comprises the I/O being configured to transmit image and image tag on a communication network; Image grading device, is configured to determine adding tagged priority to image; Destination logic, the first destination being configured to determine to send images to manually is audited first of image for by the first human reviewer; Image puts up logic, is configured to image to put up to destination; Examination & verification logic, be configured to receive the manual examination & verification from the image of destination, this manually audits the one or more image tags comprising the content of recognition image; Storer, is configured to one or more image tag to be stored in data structure; And microprocessor, be configured at least perform image grading device.

Various embodiment of the present invention comprises image processing system, and it comprises the I/O being configured to receive image on a communication network; Image grading device, is configured to determine the priority of image and determines whether to add label based on this priority to image and/or how to add label to image; For adding label to produce the device manually or automatically of one or more image tags of characterized images to image; Storer, is configured to one or more image tags of storage figure picture and characterized images in data structure; And microprocessor, be configured at least perform image grading device.

Various embodiment of the present invention comprises image processing system, and it comprises the I/O being configured to receive image on a communication network; Image grading device, is configured to determine the priority of image and selects to add tagged process to image based on this priority; For adding label to produce the device of one or more image tags of characterized images to image; Storer, is configured to one or more image tags of storage figure picture and characterized images in data structure; And be configured to the microprocessor at least performing image grading device.

Various embodiment of the present invention comprises image processing system, and it comprises the I/O being configured to transmit image and image tag on a communication network; Image grading device, is configured to determine being watched how many times based on the video comprising image and adds tagged priority to image; Destination logic, is configured to the destination determining to send images to for by the manual examination & verification of human reviewer to image; Image puts up logic, is configured to image to put up to destination; Examination & verification logic, be configured to receive the manual examination & verification from the image of destination, this manually audits the one or more image tags comprising the content of recognition image; Storer, is configured to one or more image tag to be stored in data structure; And microprocessor, be configured at least perform image grading device.

Various embodiment of the present invention comprises the method for process image, and the method comprises the image received from image source; To automated graphics recognition system distribution diagram picture; Receive the examination & verification from the Practical computer teaching of automated graphics recognition system, the examination & verification of this Practical computer teaching comprises and is assigned to one or more image tag of this image and the tolerance of degree of confidence by automated graphics recognition system, the tolerance of the image tag that the tolerance of this degree of confidence the is assigned to image correctly degree of confidence of the content of characterized images; Priority is assigned to image by the tolerance based on degree of confidence; Should by manual label based on priority determination image; Put up to the first destination by the image being used for manually examination & verification, this first destination comprises the display device of mankind's image censor; And reception is audited from the manual image of the image of destination, this image audit comprises the one or more image tags being assigned to image by mankind's image audit person, by the content of this one or more image tag characterized images that mankind's image audit person assigns.

Various embodiment of the present invention comprises the method for process image, and the method comprises the image received from image source; Microprocessor is used automatically to determine the priority of image; How label should be added based on priority determination image; Label is added to produce one or more label to image, the content of this one or more label characteristics image; And by this image and this one or more tag storage in data structure.

Accompanying drawing explanation

Fig. 1 illustrates image processing system according to various embodiments of the present invention.

Fig. 2 illustrates image acquisition screen according to various embodiments of the present invention.

Fig. 3 illustrates the Search Results based on graphical analysis according to various embodiments of the present invention.

Fig. 4 illustrates the method for process image according to various embodiments of the present invention.

Fig. 5 illustrates the alternative of process image according to various embodiments of the present invention.

Fig. 6 illustrates the method in management auditor pond according to various embodiments of the present invention.

Fig. 7 illustrates the method receiving image tag in real time according to various embodiments of the present invention.

Fig. 8 illustrates the method for upgrade image examination & verification according to various embodiments of the present invention.

Fig. 9 illustrates the example comprising the image source 120A of electronic glasses according to various embodiments of the present invention.

Figure 10 illustrates the method for the image in process image source according to various embodiments of the present invention.

Figure 11 illustrates the method based on image descriptor process image according to various embodiments of the present invention.

Figure 12 illustrates the method for use feedback processing image according to various embodiments of the present invention.

Figure 13 and 14 illustrates the method providing image tag based on image descriptor according to various embodiments of the present invention.

Figure 15 illustrates the method sorting by priority image tag according to various embodiments of the present invention.

Embodiment

Fig. 1 illustrates image processing system 110 according to various embodiments of the present invention.Image processing system 110 is arranged to image interpolation label and can comprises one or more distributive computing facility.Such as, image processing system 110 can comprise the one or more servers being positioned at geographically different location.Image processing system 110 is configured to communicate via network 115.Network 115 can comprise multiple communication network, such as internet and/or cell phone system.Network 115 is configured to use the standard agreements such as such as IP/TCP, FTP to transmit data usually.Be received from image source 120 (separate marking is 120A, 120B etc.) by the image that image processing system 110 processes.Image source 120 can comprise the calculating source being connected to internet and/or Personal mobile computing equipment.Such as, image source 120A can be the webserver being configured to provide social network sites or picture sharing service.Image source 120B can be smart phone, camera, wearable camera, electronic glasses or other portable image capture device.Image source can by URL(uniform resource locator), Internet protocol address, MAC Address, cell phone identifier and/or analog identify.In certain embodiments, image processing system 110 is configured to receive the image from great amount of images source 120.

The part of the image tag performed by image processing system 110 comprises to destination 125 (being labeled as 125A, 125B etc. respectively) transmission image.Destination 125 be mankind's image audit person computing equipment and usually geographically away from image processing system 110.Destination 125 at least comprises display and data input device, such as touch-screen, keyboard and/or microphone.Such as, destination 125 can be the buildings different from image processing system 110, city, state and/or country.Destination 125 can comprise personal computer, panel computer, smart phone etc.In certain embodiments, destination 125 comprises (calculating) application being specifically configured to the examination & verification promoting image.This application is provided to destination 125 from image processing system 110 alternatively.In certain embodiments, image processing system 110 is arranged to mankind's image audit person from destination 125 login user account.Destination 125 be usually associated with individual auditor and can by Internet protocol address, MAC Address, login sessions identifier, cell phone identifier and/or analog identify.In certain embodiments, destination 125 comprises audio frequency to text converter.The image tag data provided in some destinations 125 by mankind's image audit person are sent to image processing system 110.Image tag data can comprise text image label, comprise the non-label information of the voice data of language label and/or such as upgrade request or inappropriate (clear and definite) material mark and so on.

Image processing system 110 comprises and being arranged to and the I/O of External system communication (I/O) 130.I/O130 comprises router, switch, modulator-demodular unit, fire wall and/or analog.I/O130 is configured to receive the image from image source 120, sends image, receives the label data from destination 125, and send image tag to image source 120 alternatively to destination 125.I/O130 comprises communication hardware and comprises application programming interfaces (API) alternatively.

Image processing system 110 comprises storer 135 further.The hardware that the non-transitory that storer 135 comprises the data being arranged to such as image, image tag, computations and other data discussed in this article and so on stores.Such as, storer 135 can comprise random access memory (RAM), hard disk drive, light-memory medium and/or analog.Storer 135 is configured to by using specific data structure, index, file structure, data access routines, security protocol and/or analog and stores specific data (as described herein).

Image processing system 110 comprises at least one processor 140 further.Processor 140 is hardware devices of such as electronic microprocessor and so on.Processor 140 is configured to by hardware, firmware or software instruction is loaded into the register of processor 140 and performs specific function.Image processing system 110 comprises multiple processor 140 alternatively.Processor 140 is configured to perform the various types of logics discussed herein.

First the image received by image processing system 110 is stored in image queue 145.Image queue 145 is the ordered lists of the pending image be stored in store list.The image be stored in image queue 145 is usually stored explicitly from the image identifier for quoting image and can has different priority.Such as, the image received from picture sharing website can have the priority lower than the image received from smart phone.Usually, relative to those images of the image tag used for some other objects, just representing at wait-receiving mode that for requestor those images of the image tag of image are given higher priority in real time.Image queue 145 is stored in storer 135 alternatively.

In image queue 145, image is stored explicitly with image identifier or index and other data associated with each image alternatively.Such as, image can be associated with the source data about in image source 120.Source data can comprise the geography information of such as GPS coordinates, street and/or city name, postcode and/or analog.Source data can comprise Internet protocol address, URL(uniform resource locator), account name, the identifier of smart phone and/or analog.Source data may further include the information about the priority of the language be used on the member of image source 120, request, searching request (such as, carrying out the request of internet hunt based on the image tag produced from image) and/or analog.

In certain embodiments, the image within image queue 145 is stored explicitly by the instruction of the particular subset with image, and this subset generally includes the project of special interests.Such as, the requestor of image tag can be interested in about the image tag of the content of the particular subset of image acquisition.When image comprises several object, this can occur.In order to illustrate, consider the image having the hand of ring on a finger, user may wish that identifying this ring is interested specific region.Some embodiments of the present invention comprise and are configured to make user pass through to click object or the object that touches on the display of image source 120B and specify the application of interested specific project.This appointment occurred usually before sending image to image processing system 110.

If image is stored explicitly by the instruction that the particular subset with image has particular importance, so image tagged logical one 47 is used to mark to be placed on image alternatively.Mark is arranged with outstanding specific subset.This mark can correspond to the pixel of subset by amendment image and make, and this mark allows mankind's image audit person to be absorbed in marked subset.Such as, be posted to before one or more destination 125 at image, image can be labeled rectangle or circle.Such as, the object within the subset of outstanding image or image can comprise to object or subset apply filters (filter, filter), and/or changes the color of object or subset.In alternative embodiments, within image tagged logical one 47 is included in the application being configured to perform in one or more image source 120 or destination 125.Image tagged logical one 47 comprises the hardware be stored in non-transitory computer-readable medium, firmware and/or software.As in other place herein discussed, annotated logic 147 is configured to alternatively be generated with image and is arranged in real time on image by mark.

In certain embodiments, within annotated logic 147 is configured to be used in image, detected characteristics of image identifies the special object that can be labeled.The detection of characteristics of image comes into question in other place herein and is the part of the image procossing occurring in (such as, on image source 120A) on customer alternatively.Such as, the feature at such as edge and so on can use the processor of image source 120A to be detected.First these features can be used in the object of outstanding detection and also be sent to image processing system 110 from image source 120A subsequently, and at this place, they are used to the image descriptor of the part generated as process image subsequently.In the method, the automatic process of image is dispensed between image source 120A, image processing system 110 and/or automatic recognition system 152.

Under the control of processor 140, the image in image queue 145 is provided to automatic recognition interface 150.Thus image is provided as their priority in image queue 145 and the function of position.Automatic recognition interface 150 comprises and being configured to image, and any data be associated with this image is sent to the logic of automatic recognition system 152 alternatively.This logic is the hardware, firmware and/or the software that store on a computer-readable medium.Automatic recognition interface 150 can be configured to receive the examination & verification from the Practical computer teaching of the image of automatic recognition system 152 further, and the examination & verification of this Practical computer teaching comprises one or more image tags of the content of recognition image.In certain embodiments, automatic recognition interface 150 is configured to transmit image and data via network 115 with the form being applicable to the application programming interfaces (API) of automatic recognition system 152.In certain embodiments, automatic recognition system 152 is included in image processing system 110, and recognition interface 150 comprises the system call such as in operating system or in LAN (Local Area Network) automatically.

Automatic recognition system 152 is the computing machine automatic systems being configured to audit image and not needing the artificial input based on each picture.The output of automatic recognition system 152 is image audit (examination & verification such as, not needing the artificial input based on each picture to produce) of Practical computer teaching.The preliminary example of this system is well known in the art.Such as, see Kooaba, Clarifai, AlchemyAPI and Catchoom.Automatic recognition system 152 is configured to the object automatically identified based on the shape detected in image, character and/or pattern in two dimensional image usually.Automatic recognition system 152 is configured to perform optical character identification and/or bar code lexical or textual analysis alternatively.In certain embodiments, with the difference of the system of prior art, automatic recognition system 152 is that automatic recognition system 152 is configured to provide the examination & verification of Practical computer teaching, this examination & verification is based on the instruction of (multiple) image subset and/or image source data, discusses in its other place in this article.

Automatic recognition system 152 is configured to determine whether the copy of the image received from different image sources is added label alternatively.Such as, identical image can be included in multiple webpage.If image is extracted from the webpage of first these webpages and is added label, automatic recognition system 152 can be added label and each example be found to image automatically assigns these labels by this image of identification.Recognisable image has been added label and has comprised the part of image, image alternatively or represent that the data of image compare with the database of the image adding label before.Image automatically or manually can be added label before.

Except one or more image tag, the examination & verification of the Practical computer teaching generated by automatic recognition system 152 comprises the tolerance representing one or more image tag correctly degree of confidence of the degree of confidence of the content of recognition image alternatively.Such as, the examination & verification that mainly character maybe can be easy to the Practical computer teaching of the image of the shape of identification can have the confidence metric larger than the examination & verification of the Practical computer teaching of the image be made up of abstract or unclear shape.Different automated graphics identification systems can produce the different confidence level for dissimilar image.Automatic recognition interface 150 and automatic recognition system 152 are in the embodiment of the automatic identification performed by third party alternatively.

Image processing system 110 comprises auditor pond 155 further and is configured to manage auditor's logical one 57 in auditor pond 155.Auditor pond 155 comprises the pond (such as, group or set) of mankind's image audit person.Each mankind's image audit person is associated with the different members of destination 125 usually.Such as, each member in the different members of destination 125 can be known by different mankind's image audit persons operation or logged in by the account of different mankind's image audit person.Storer 135 is optionally configured to and stores auditor pond 155.In certain embodiments, the mankind's image audit person be included in auditor pond 155 is classified as " enlivening " and " inactive ".In order to object of the present disclosure, active mankind's image audit person is considered to providing image tag or indicating them to get out provide with minimal time delay mankind's image audit person of image tag at present.Active also comprise in the embodiment of sluggish mankind's image audit person both comprising, active auditor is provided image with examination & verification.The quantity of active auditor can in response to the requirement of image audit being become in real time to appropriateness.Such as, mankind's image audit person classification can based on the quantity of not auditing image in image queue 145 from inactive change into active.Sluggish auditor is the auditor not yet enlivened, and it has allowed the examination & verification of image lose efficacy and/or indicated them to audit image.Sluggish auditor can ask to become active auditor.When need additional enliven the mankind's image audit person time, the sluggish auditor having made this request can be reclassified as active mankind's image audit person.Which sluggish auditor is reclassified depends on auditor's mark (other local discussion in this article) alternatively as the determination of active auditor.

Auditor's logical one 57 is configured to manage auditor pond 155.This management comprises mankind's image audit person alternatively as active or sluggish classification.Such as, auditor's logical one 57 can be configured to monitor that mankind's image audit person starts to audit the time of image, and if predetermined maximum audit time (being called the picture failure time herein), by the classification of mankind's image audit person from active change into inactive.In another example, auditor's logical one 57 can be configured to calculate the examination & verification mark for mankind's image audit person.In certain embodiments, the integrality of the image audit that fractional marks is performed by specific mankind's image audit person, speed and/or accuracy is audited.This examination & verification mark can be calculated based on examination & verification number of times and accidental test pattern or be changed.Such as, these test patterns can be placed in the image queue 145 of being audited by different mankind's image audit persons before.This examination & verification mark can also be the function of the monetary cost be associated with mankind's image audit person.Auditor's logical one 57 comprises the hardware be stored in non-transitory computer-readable medium, firmware and/or software.In certain embodiments, auditor's mark is manually determined by mankind coordinator.These mankind coordinators audit image and are assigned to the label of these images by mankind's image audit person.Alternatively the statistic sampling of the image of examination & verification is sent to coordinator and coordinator by the labeling assignments mark to image.This mark is used in alternatively determines auditor's mark.

In certain embodiments, auditor's logical one 57 is configured to the state monitoring mankind's image audit person in real time.Such as, auditor's logical one 57 can be configured to the input monitoring individual word or the keystroke inputted by the auditor at destination 125A.This supervision can be used to determine which auditor audits image actively, and which auditor just completes the examination & verification of image, and/or which auditor some seconds or minute after also do not provide label to input.Use audio frequency apparatus label word input can also monitor by auditor's logical one 57.

In certain embodiments, the member in auditor pond 155 is associated with the specialty that mankind's image audit person has speciality or special knowledge.Such as, auditor can be the expert of automotive field and be associated with this specialty.Other specialty can comprise art, plant, animal, electronics, music, food medical professionalism, clothes, dress accessory, fine work etc.As other place herein discussed, the specialty of auditor can be used to select this auditor between the initial manual period under review and/or during examination & verification upgrading.

The examination & verification mark be associated with mankind's image audit person and/or specialty alternatively by auditor's logical one 57 be used for determining when need additional enliven auditor time which sluggish auditor to be changed to active.Auditor's logical one 57 comprises the hardware be stored in non-transitory computer-readable medium, firmware and/or software.

Image processing system 110 comprises destination logical one 60 further.Destination logical one 60 is configured to determine to send the one or more destinations (such as, destination 125) of image for manually examination & verification to it.Each destination 125 is associated to corresponding mankind's image audit person in auditor pond 155.The determination made by destination logical one 60 is alternatively based on the characteristic of the mankind's image audit person in determined destination.Destination can be the computing equipment, smart phone, flat computer, personal computer etc. of mankind's image audit person.In certain embodiments, destination is browser, and auditor logs in image processing system 110 from this browser.In certain embodiments, determine that destination comprises the MAC Address of the destination determined in destination 125, Session ID, Internet protocol and/or URL(uniform resource locator).Destination logical one 60 comprises the hardware be stored in non-transitory computer-readable medium, firmware and/or software.

Usually, destination logical one 60 is configured to determine and active instead of that sluggish mankind's image audit person the is associated destination 125 such as determined by auditor's logical one 57.The destination logical one 60 usually examination & verification mark be also configured to based on auditor determines destination 125.Such as, compared with the auditor with lower auditor's mark, those auditors with higher fractional can be selected for the examination & verification of higher priority.Thus, the determination of the member of destination 125 can based on auditor's mark and image audit priority.

In certain embodiments, destination logical one 60 is configured to one or more members of determining based on the real time monitoring of the input activity of the auditor be associated in destination 125.As other place herein discussed, this supervision can be performed by auditor's logical one 57, and can comprise the detection of individual word or the keystroke inputted by mankind's image audit person.In certain embodiments, destination logical one 60 is configured to the destination 125B compared with just knocking in image tag at present on keyboard mankind's image audit person, and the destination 125A just having completed the examination & verification of image mankind's image audit person is selected in favor.

In certain embodiments, destination logical one 60 is configured to use the image tag received via automatic recognition system 152 to determine one or more members of destination 125.Such as, if the image tag of " automobile " is received via automatic recognition interface 150, so destination logical one 60 can use this information to select and the member with the destination 125 be associated mankind's image audit person of automotive field specialty.

The value of image audit can also be considered in the selection of the destination for manually auditing.Such as, the image audit of high level can cause the determination of the destination be associated with mankind's image audit person with relative high examination & verification mark, and the image audit of lower value can cause the determination of the destination be associated with mankind's image audit person with relatively low examination & verification mark simultaneously.In certain embodiments, for some image audit, destination logical one 60 is configured to select between destination 125 so that minimize the time needing to audit image, such as, minimize the time until the image tag of manually examination & verification is provided to network 115.

Destination logical one 60 is configured to determine multiple destination for single image alternatively.Such as, the first destination can be selected, and then upgrade request afterwards, the second destination can be determined.Upgrade request can from the mankind's image audit person be associated with the first destination or from image source 120A.In certain embodiments, destination logical one 60 is configured to determine multiple destination, and image will be puted up concurrently to the plurality of destination.Such as, each two, three or more destinations be all associated from different mankind's image audit persons can be determined and same image is puted up concurrently to all destinations determined.As used in the present context, " walking abreast " means image and be at least posted to the second destination before any part receiving examination & verification from the first destination.

In various embodiments, exist a variety of causes make two or more destinations can determine by destination logical one 60.Such as, can ask that there is particular professional mankind's image audit person to the request of the examination & verification of upgrading.With reference to automobile example, first by upgrade request that label has the image of " white car " label can cause more information.Destination logical one 60 can be configured to select subsequently and have the destination be associated mankind's image audit person (such as, can provide the auditor of " 1976 Ford Granada " label) of automotive field specialty.Upgrade request indicating image is as the criterion further to audit, and such as, image demand is further audited or can be benefited from further audit.Upgrade request can represented by the calculating object of such as flag, order or data value etc.

When the manual examination & verification of image expends the too many time, another example of the second destination can be needed to occur.Usually, within the time period of distributing, the label of image adds and should occur, otherwise this examination & verification is considered to lose efficacy.The time period of distribution is the function of the priority of image audit alternatively.Those examination & verifications being intended to occur in real time can have compared with the lower priority examination & verification shorter time period.If the examination & verification of image was lost efficacy, additional mankind's image audit person that image processing system 110 is configured to alternatively to being associated with the destination determined by destination logical one 60 provided image.

When the first human reviewer makes upgrade request, another example of the second destination can be needed to occur.Such as, cause the request of the upgrade audit of the label of " automobile " can from the mankind's image audit person providing label " automobile ".Although this example simplifies, other example can comprise the image of more abstruse theme, the integrated circuit such as encapsulated.

Image processing system 110 comprises image further and puts up logical one 65, and it is configured to the destination 125 of being puted up by the image being used for manually examination & verification to being determined by destination logical one 60.Put up to generally include, via network 115, image is sent to one or more destination 125.In various embodiments, image is puted up logical one 65 and is further configured to and provides to destination 125 information be associated with image.Such as, image puts up logical one 65 can put up the instruction of the subset of image and image (such as, subset identification), the priority of the examination & verification of the information in the source of the image that marked by image tagged logical one 47, recognition image (such as, in this article other local source data discussed), image, the picture failure phase, the positional information be associated with image and/or analog.As other place herein discussed, source data can comprise URL(uniform resource locator), global positioning coordinates, longitude share account and/or analog with latitude, account identification symbol, Internet protocol address, social account, picture.

In certain embodiments, image is puted up logical one 65 and is configured to be provided for the image of manually examination & verification to more than one destination 125 in about same time.Such as, image can be provided to destination 125A and destination 125B concurrently.Such as, " walk abreast and send " means before being received back label information from destination 125A and 125B, and image is provided to these destinations 125.

In certain embodiments, image puts up the image that logical one 65 is configured to be provided in the one or more destination 125 of forward direction receiving image tag from automatic recognition system 152 manually examination & verification.Alternately, in certain embodiments, image is puted up logical one 65 and is configured to wait for before putting up image to one or more in destination 125 until the examination & verification for the Practical computer teaching of image is received from automatic recognition system 152.In these embodiments, what the examination & verification (comprising image tag) of Practical computer teaching was also posted in the destination 125 be associated with image alternatively is one or more.

Image puts up the identifier that logical one 65 is configured to put up together image and image alternatively.Image is puted up logical one 65 and is comprised the hardware be stored in non-transitory computer-readable medium, firmware and/or software.

Image processing system 110 comprises the examination & verification logical one 70 manually and automatically audited being configured to managing image further.This management comprise monitor examination & verification progress, receive examination & verification from automatic recognition system 152 and/or destination 125.The examination & verification received comprise as other place in this article the image tag discussed.In certain embodiments, audit logical one 70 to be configured to control to put up to the image of the destination of in destination 125 based on the tolerance of degree of confidence.The degree of confidence that the received one or more image tags of the measurement representation of degree of confidence are correct.This one or more image tag can be received from a destination automatic recognition system 152 and/or destination 125.Such as, in certain embodiments, if be greater than predetermined threshold by the degree of confidence of the image audit of automatic recognition system 152, so audit logical one 70 and can determine that the manual examination & verification of image is unnecessary.This predetermined threshold can be the value of image audit, the priority of image audit, the quality and quantity of available destination 125 and/or the function of analog.Examination & verification logical one 70 comprises the hardware be stored in non-transitory computer-readable medium, firmware and/or software.

In certain embodiments, if it is one or more that image is sent to automatic recognition system 152 concurrently and is sent in destination 125, the reception so with the examination & verification from automatic recognition system 152 of the degree of confidence higher than predetermined threshold can cause cancelling the one or more manual examination & verification in destination 125 by examination & verification logical one 70.Similarly, if image is sent to multiple destination 125 concurrently, and image audit is received from first destination these destinations 125, so audits logical one 70 and be configured to alternatively cancel the examination & verification request for image at other destination 125 place.In certain embodiments, audit logical one 70 to be configured to, once keystroke or word are received from the destination of first in destination 125, cancel the examination & verification request in other destination 125.

In certain embodiments, audit logical one 70 and be configured to the activity monitoring mankind's image audit person in real time.This supervision can comprise receive from destination 125 based on word one by one or the examination & verification input of single keystroke.As other place herein discussed, received along with word and/or keystroke audit logical one 70, they are transferred into one in image source 120 alternatively.The supervision of the activity of manual auditor can be used to determine when the examination & verification of image lost efficacy and/or completed the progress of manual image examination & verification.The state of mankind's image audit person can be provided to auditor's logical one 57 by examination & verification logical one 70 in real time.Use this state, auditor's logical one 57 can by the state of auditor from active change into sluggish, regulate the examination & verification mark stored of auditor, establishment or change for the specialty of auditor and/or similar.

In certain embodiments, examination & verification logical one 70 is configured to the tolerance (such as, the tolerance of the accuracy of image audit) by receiving degree of confidence and puts up logical one 65 to destination logical one 60 and/or image send response signal and control image putting up to destination 125.Thus, audit logical one 70 can be configured to control to put up to the image of the one or more destinations in destination 125 based on the tolerance of degree of confidence.The degree of confidence of the one or more image tag of measurement representation of the degree of confidence correctly content of recognition image.In certain embodiments, audit logical one 70 to be configured to receive the examination & verification comprising the out of Memory except image tag from manual image auditor.Such as, audit logical one 70 can receive the upgrade request from mankind's image audit person and cause the image audit of upgrading requested.Examination & verification logical one 70 is configured to process other the non-label information be received in manually or in the examination & verification of Practical computer teaching alternatively.This information can comprise the identification of incorrect (such as, obscene) image, the identification not comprising the image of identifiable design object, the identification of image sent to the auditor of mistake specialty and/or similar identification.

In certain embodiments, the degree of confidence that logical one 70 is configured to be regulated by the image audit compared from the identical image in multiple source image audit is audited.These image audit can be all Practical computer teaching, all be manual examination & verification or the examination & verification comprising at least one Practical computer teaching and at least one manual examination & verification.

In certain embodiments, audit logical one 70 be configured to be provided as the image tag of the part reception of first (Practical computer teaching or manual) examination & verification and provide received image tag to the mankind's image audit person at 125B place, destination.Agency's (such as, application of browser or specific purposes) that destination 125B performs is configured to provide to the display of destination 125B the image tag of the first examination & verification alternatively.In like fashion, the image tag of (increase, delete and/or replace) the first examination & verification can be edited mankind's image audit person at 125B place, destination.Such as, the image tag received from destination 125A can be provided to destination 125B for amendment.

In certain embodiments, audit logical one 70 to be configured to calculate examination & verification mark based on the accuracy of these image audit time used, these image audit and the result of image audit that receives from destination 125.

In certain embodiments, audit logical one 70 to be configured to use response logic 175 to provide image audit to the source (such as, in image source 120) of image.When based on character one by one or the image audit of word one by one completes time, image audit can be provided.When based on character by character or when providing one by one, be received from mankind's image audit person with character or word, image tag is provided to the source of image alternatively word.Alternatively, response logic 175 is configured to provide image audit via network 115.

Image audit need not be returned to one in image source 120.Such as, if image source 120A is picture sharing service or social networking website, the image audit from the image of image source 120A can be stored explicitly with the account in picture sharing service or social networking website.This storage can in storer 135 or in the position of image processing system 110 outside, such as at the webserver place holding this website.Image audit one of being both returned to alternatively in image source 120 is also stored in other place.

In certain embodiments, response logic 175 is configured to based on the image tag received in Practical computer teaching and/or manual image audit and performs search.The result of this search can be provided to the source of image, such as, and image source 120A or 120B.Such as, in certain embodiments, user uses smart phone to utilize the camera of image source 120A to create image.This image is provided to image processing system 110, and it uses the image audit of automatic recognition system 152 and destination 125A synthetic image.This image audit comprises the image tag of the internet hunt (such as, Google or Yahoo's search) being automatically used to subsequently perform image tag.The result of this internet hunt is provided to the smart phone of user subsequently.

In certain embodiments, response logic 175 is configured to the image tag of that provide Practical computer teaching to ad system 180 and/or manual examination & verification.Ad system 180 is configured to select advertisement based on image tag.Be provided to the source of the image for synthetic image label selected Advertisement Option.Such as, response logic 175 can provide label " to have 1976 Ford Granadas of the head lamp of damage " to ad system 180, and responsively, ad system 180 can select advertisement for replacement head lamp.If the source being used to the image generating these labels is network address, advertisement can be shown in this network address.Especially, share or the account of social networking website if the source of image is picture, so advertisement can be shown on the account.Ad system 180 is included in image processing system 110 alternatively.Ad system 180 is configured in response to specific label alternatively as providing quotation for advertisement.Ad system 180 comprises the Adwords of Google alternatively.

Image processing system 110 comprises the contents processing logical one 85 being configured to extract the image added for label from the member of image source 120 alternatively further.Contents processing logical one 85 is configured to resolve the webpage comprising image and comprise text alternatively, and from these webpages, extract the image being used for label and adding.The image tag produced can be provided to ad system 180 subsequently for the advertisement selecting can be placed in from the webpage wherein extracting image.In certain embodiments, contents processing logical one 85 is configured to emulation browser function so that load and will be usually displayed on the image on webpage.These images can be displayed on be associated with particular account webpage, social network sites, picture sharing website, Blog Website, news website, meet-a-mate site, sports site and/or and so on website.Contents processing logical one 85 is configured to analytical element data label alternatively so that recognition image.

Contents processing logical one 85 is configured to resolve the text be placed in same web page as image alternatively.The text can be automatically recognized system 152 be used in image label add, be combined with the content of image.Such as, contents processing logical one 85 can be configured to text (as determined by optical character recognition OCR (OCR)) in the captions of recognition image, the comment made about image, the text relating to image, Web page subject or title, the people of image interior label or thing, image and/or analog.The text of being resolved by contents processing logical one 85 or its subset can be used to speed and/or the quality of improving label.The text resolved is provided to automatic recognition system 152 and/or label is added for by human reviewer in the destination be provided in destination 125.In certain embodiments, automatic recognition system 152 is configured to use provided text in the generation of the label for image.Such as, the text provided can be used to provide text, identify dictionary, ontology, language and/or information, the accuracy of its image tag providing automatically and/or manually generate, degree of accuracy, counting yield and/or other quality.The text provided does not rely on the source as just generated label usually, but is used as input with the process improving image.Thus, the label of generation can comprise except find in provided text those except word.

In certain embodiments, image is puted up logical one 65 and is configured to be provided in image and text that the webpage identical with image finds to destination 125.Such as, the girl in park and the image of bicycle can have captions " mountain bike sale " or comment " happy birthday Zhu Li ".In destination 125, the text can be presented to human reviewer together with image.Human reviewer can use this information to understand emphasis and/or the context of image better, and provides better image tag thus.Similarly, in certain embodiments, automatic recognition interface 150 is configured to be provided in automatic recognition system 152 image and text that the webpage identical with image finds.At automatic recognition system 152 place, the text provided is used to the automated tag of the content improvement image based on image.In the above examples, the text provided should can be placed outstanding to automatic recognition system 152 suggestion on bicycle or on Zhu Li.This can cause totally different label, as " Schwinn bicycle " or " birthday girl ".

Image processing system 110 comprises image grading device 190 alternatively further.Image grading device 190 is configured to the grade (such as, priority) determining label image.Priority can be used to determine (as truly wanted) how label image.Such as, the determination of priority can based on the source of image, image is loaded on the number of times on webpage, the position of image on webpage, image on webpage by the number of times watched, comprise the number of the webpage of image thereon, comprise the classification of one or more webpages of image, comprise the mark of the webpage of image, the classification of the second image on the webpage comprising image, comprise the owner of the webpage of image, comprise the domain name of the webpage of image, comprise the keyword on the webpage of image, the text that the webpage comprising image finds, the metadata that the webpage comprising image finds, the number of times that image is clicked on webpage, the number of times that other image is clicked on webpage, whether image is a part for video, use the image tag that automatic recognition system 152 automatically generates, the combination in any of these examples, and/or it is similar.Image grading device 190 comprises storing the logic that hardware, firmware and/or software are on a computer-readable medium form.Image grading device 190 comprises storing the logic that hardware, firmware and/or software are on a computer-readable medium form.In various embodiments, the priority determined by image grading device 190 comprises two ranks (label or without label), three ranks (automated tag, manual label or without label), ten priority-level or some other hierarchy plans.

Destination logical one 60 is configured to the destination selecting the manual label of image to add based on the priority of image alternatively.

In those embodiments, the number of times that wherein image is loaded on webpage is used to determine priority, this quantity can be per set time section, such as every day or monthly.This quantity can be determined by the row comprising Java on webpage or html script, as known in the art.The position of image on webpage can be considered to some images can need beholder to roll downwards before image is watched.Thus, in fact image can be used to the priority of computed image by the number of times watched.Usually, larger priority is assigned to more often by the image watched.Image grading device 190 be configured to alternatively based on image on webpage or on other webpage clicked number of times and/or the number of times clicked on webpage of other image to image assigned priority.Image grading device 190 be configured to alternatively based on image on more than one webpage by the number of times determination priority of watching.Such as, if image is found on 25 different webpages, the sum so watched on all webpages can be used to determine the priority for image.In certain embodiments, image grading device 190 is configured to the number of times determination priority that is loaded in a browser based on image.

Popular image can be included in some webpages.Such as, image social media website extensively shared can be included on some websites.Image grading device 190 can be configured to be included the quantity of webpage thereon according to image and/or comprise the priority of quantity computed image of webpage of the link of leading to this image.Image grading device 190 is configured to identify (this possibility the is incoherent) image be included on multiple webpage alternatively.In certain embodiments, image grading device 190 is configured to use third party's service, and such as TinEye.com, determines the quantity of the webpage that image is placed thereon.Usually, the quantity comprising the webpage of image is thereon larger, and the priority being assigned to this image is larger.

In certain embodiments, image grading device 190 is configured to the priority of the classification computed image based on the one or more webpages comprising image.Such as, if classification is very high in a search engine, by other a large amount of web page interlinkages or good classification under some other standards, the image so on this webpage can be given the priority being classified as function with this webpage for webpage.Usually, classification is higher, and the classification that webpage has is higher, and the priority being assigned to the image on webpage is larger.Webpage classification obtains from the third party source of such as search engine and so on alternatively.

Image grading device 190 is configured to mark based on the webpage comprising image alternatively to image assigned priority.Such as, compared with the image at another webpage place for identical website, larger priority can be assigned for the image on the homepage of URL.In addition, image can based on comprise thereon image webpage particular type and be assigned priority.Such as, compared with the image in company's site or individual blog, the image in social networking website can be given higher priority.In another example, for the webpage of other type, the image on the reference webpage of such as dictionary.com or Wikipedia.com and so on can be given higher priority.The priority being assigned to image is the mark of the owner based on webpage alternatively.

In certain embodiments, image grading device 190 is configured to the priority of the second image on webpage as function determines the priority of the first image on the same web page.Such as, if the second image has high priority, the priority of the first image can correspondingly be increased.

Image grading device 190 is configured to other content based on the webpage comprising image thereon alternatively to image assigned priority.Such as, if webpage comprises text and/or metadata, the particular term in the text or metadata or the existence of keyword can be used to the priority of assigning image.Specifically, if webpage comprises valuable keyword, the image so on this webpage can be assigned higher priority.The monetary value of the estimation of keyword is associated with the value of the word for advertisement or some other objects, such as, in Google upper valuable word.Comprise and can be assigned suitably high priority by the image on the webpage of the Adwords term of valuation higher.The frequency of the use of these terms and their quantity on webpage can also be considered determine picture priority by image grading device 190.The text considered and/or metadata can be included in the URL of webpage, in the map title, in the comment made figure, in the label of being assigned by third direction image, name, brand name, trade mark, enterprise's name, relate to image text near, and/or similar.

In certain embodiments, image grading device 190 is configured to receive the text that uses optical character recognition OCR to derive from image and determines the priority for image based on the text.Such as, image grading device 190 can receive the text generated by using automatic recognition system 152 to process image, and based on the text to image assigned priority.In certain embodiments, image grading device 190 is configured to provide higher priority to the first image on webpage, relative to for the image that more below occurs of this webpage.

Image grading device 190 is configured to determine further (as truly done) how based on the priority of assigning label is added to image.Such as, the image of lowest priority can by label.The image with higher priority a little can use automatic recognition system 152 to be added label, and the image with higher priority can be located in by the object of human reviewer in destination 125 and be added label.Those have enough high priorities, tagged image to be added by human reviewer and be divided into higher priority group and lower priority group alternatively further, being given more concerns and being done outgoing label more up hill and dale or modestly by human reviewer in higher priority group is added.Image puts up the instruction that logical one 65 is configured to the priority providing image and image to the member in destination 125 alternatively.

In certain embodiments, first image is used handled by automatic recognition system 152.Subsequently, this image can be sent to the one or more members in destination 125 based on the degree of confidence in the priority for image and the automated tag performed by automatic recognition system 152 interpolation.Such as, if image has relatively low priority, the confidence criteria so for sending image to human reviewer is set to relatively low.(low confidence standard means that automated tag interpolation may be considered to enough and this image is not used in people's alanysis and is sent out.) if image has relatively high priority, the confidence criteria so for sending image to human reviewer is relatively high.Thus, the degree of confidence that high priority image request is higher is added only to rely on automated tag, and more may be sent to human reviewer.

Such as can be comprised for the process path that image is selected by image grading device 190 and a) not add label, b) automatic recognition system 152 is only used to add label, c) based on importance and/or the degree of confidence producing label, the automatic recognition system 152 with optional mankind follow-up is used to add label, d) mankind that followed by this automated tag adds of automated tag interpolation audit, e) add label by human reviewer, and/or f) add label based on the rank by the suggestion will considered by human reviewer by human reviewer.These process paths are selected based on the priority of being assigned to image by image grading device 190 at least partly.The combination in any in these process paths can be found in various different embodiment.In certain embodiments, the result controlled the type of the process being used to label image causes may having larger possibility by label by those images more valuable.As a result, mankind's label resources is applied to limit priority---the image of most worthy.

In certain embodiments, image grading device 190 is configured to based on advertisement that is adjacent with image or that show how often clicked and assign the priority for image thereon.Such as, if image be in often by the webpage watched but the advertisement be positioned on this image is clicked rarely, so this image can be given relatively high priority for label.In this example, image can be added label more than once.If not clicked with the frequency of expection based on the advertisement of initial labels, so this image can be tagged again.Again tag and performed by human reviewer alternatively, this human reviewer puts up logical one 65 via image and receives image and initial (unsuitable) label.Human reviewer can use this information to provide the label of improvement.

Fig. 2 illustrates image acquisition screen 210 according to various embodiments of the present invention.Image acquisition screen 210 as shown in the figure is such as generated by the application performed in smart phone, electronic glasses or other image source 120.Image acquisition screen 210 comprises and is configured to gather image, the interested specific region of label and receive the feature of image tag.Especially, image acquisition screen 210 comprises the shutter release button 220 being configured to take pictures.Once picture is taken, it is automatically sent to image processing system 110 via network 115 alternatively and is added for label.Image acquisition screen 210 comprises the rectangle being configured to outstanding interested point within image alternatively further.Rectangle 230 is controllable (such as, moveable) by using user input device to select on screen and/or pulling.On typical smart phone, this user input device can comprise the touch-screen in response to finger touch.As herein described by other place, interested point/region can be provided to the image processing system 110 be associated with the image that will be added label.

Image acquisition screen 210 comprises the field 240 of the previous image of collection of display and the image tag of generation further.In this example, illustrate that the image previously gathered comprises and do not have the identical white cup of rectangle 230 and comprise the image tag of " white Starbucks coffee cup (WhiteStarbucksCoffeeCup) ".What be also illustrated is the text of alleged " sliding for option (Slideforoptions) ".

Fig. 3 illustrates the Search Results based on graphical analysis according to various embodiments of the present invention.These results automatically or in response to " sliding for the option " input selected shown in Fig. 2 are shown alternatively.They can by automatically performing internet hunt and being generated on image tag.Fig. 3 is illustrated is sponsored advertisement 310, associated picture 320 and other Search Results 330.Search Results use alternatively ad system 180 and generate and image tag use image processing system 110 be generated.User can select to look back and previously add tagged image.This history can be stored on image source 120A or in storer 135.

Fig. 4 illustrates the method for process image according to various embodiments of the present invention.In these methods, image is received.This image is provided at least one destination in automatic recognition system 152 and destination 125.As a result, Practical computer teaching and the image audit that manually generates produced.Illustrated method uses the embodiment of illustrated system in Fig. 1 to be performed alternatively in the diagram.In Fig. 4 to Fig. 8, illustrated method step can be performed with plurality of replaceable order.

Reception image step 410 in, image receive by image processing system 110.This image receives from image source 120 via network 115 alternatively.This image can be the standard formats such as such as TIF, JPG, PNG, GIF.This image can be an image in a series of images of the image sequence forming video.This image can be used collected by camera by user.This image can be gathered from movie or television program by user.In certain embodiments, receive image step 410 to comprise user and use image acquisition should be used for gathering image being transmitted to image processing system 110 by this image.Within this application can be placed in camera, TV, video display apparatus, multimedia equipment and/or analog.Receiving image step 410 uses contents processing logical one 85 to be promoted alternatively.

In an illustrative example, image is received from image sequence (such as, video).This video is displayed on monitor, TV, eye guard, glasses or other display device.This video optional ground via such as youtube.com or and so on video streaming services received and/or be shown in browser.Logic (the image tagged logical one 47 such as, in image source 120A) in display system is configured such that user indicates the particular subset of the image in this video.Identical logic can be configured to receive and this advertisement is presented on video or with this video and show simultaneously from Computer image genration by the advertisement selected in response to image tag.Selection other place in this article based on the advertisement of image tag is discussed further.

Especially, use this system, user can select in video or film for label add object and as the label responding this object of receive featureization alternatively.User can also or alternately receive based on the advertisement selected by this label.This advertisement can be shown (such as, as the video sequence covered or add) in real time or be provided to user via other communication port (such as, Email) together with video.In an illustrative example, user see they in video the object liked.They select this object and this selection is received in reception image step 410.Responsively, they receive the advertisement relevant to this object.Along with video is watched over the display, advertisement is shown in real time as the covering on video, bar or captions.Interactively, because it comprises the link making purchase Advertisement Option.

In certain embodiments, the object in image can comprise the particular characteristics being configured to this object of aid identification.Such as, the AD HOC of data bit can be encoded in image or in the object of image.These data bit can be encoded for image tag.

Receive in subset identification step 415 optional, the data of one or more subsets of recognition image are received by image processing system 110.Usually, the project that one or more subset comprises special interests is positioned at one group of image pixel wherein.This one or more subset can be identified by the location of pixels on the image being received, screen coordinate, region and/or point.In certain embodiments, selected by this subset cursor of being used an image source in image source 120 by user or touch-screen.

Receive in source data step 420 optional, receive received in image step 410, received by image processing system 110 about the source data in the source of image.As other place herein discussed, source data can comprise geography information, Internet protocol address, URL(uniform resource locator), account name, the identifier of smart phone, information about the language used on the member of image source 120, searching request, user account information and/or similar.In certain embodiments, application/agency that source data is run image source 120 automatically sends.Such as, GPS coordinates can be generated automatically and be provided to image processing system 100 on smart phone.

Analyze in optional reception source in priority tasks 425, receive received in image step 410, received by image processing system 110 for the priority of the label of image.In certain embodiments, priority is manually inputted by the user of image source 120A.In certain embodiments, priority depends on the amount that the examination & verification for image is paid a price.In certain embodiments, priority depends on the type of image source 120A.Such as, compared with the image received from handheld mobile device, the image received from static website can automatically be given lower priority.Compared with the image being moved telephone number identification with its source, its source can be given lower priority by the image of URL(uniform resource locator) identification.Thus, priority is alternatively by receiving source data derivation received in source data step 420.

The received image of step 410-425 and data alternatively by together with receive and be stored in alternatively in storer 135.

In distribution image step 430, image and any data be associated received in step 415-425 are alternatively assigned to automatic recognition system 152 via automatic recognition interface 150.This distribution can within image processing system 110 or via network 115.

Be received from dynamic response step 435, the image audit of Practical computer teaching is received from automatic recognition system 152.The image audit of Practical computer teaching comprises one or more image tags that the system of being automatically recognized 152 is assigned to image.The image audit of Practical computer teaching also comprises the tolerance of degree of confidence.The tolerance of this degree of confidence is the tolerance of the image tag correctly degree of confidence of the content of characterized images being assigned to image.Such as, relative to the image of abstract shapes, the image mainly comprising the character that can be easy to identification can receive the tolerance of higher degree of confidence.

Optionally determining in degree of confidence step 440, the tolerance of the degree of confidence comprised at image audit is compared with one or more intended level.This intended level is the function of the priority of image audit, the price of image audit, the source of image and/or analog alternatively.Do you are optionally be sure oing? in step 445, if the degree of confidence of the image audit of Practical computer teaching is higher than this process of (multiple) predetermined threshold, proceeds to optional execution search step 450, if the degree of confidence of the image of Practical computer teaching is lower than (multiple) predetermined threshold, proceed to queuing image step 460.Determine that degree of confidence step 440 uses examination & verification logical one 70 to be performed alternatively.

In execution search step 450, the image tag being assigned to image is used to perform search.Such as, image tag " ford " can be used to use word " Ford " and " car " automatically to perform Google search.

Thering is provided in result step 455, be assigned to the image tag of image and be provided to the requestor of image audit alternatively in the result performing the search be performed in search step 450.Such as, if image is from image source 120A is received and image source 120A is smart phone, so image tag and Search Results are provided to smart phone usually.If image is received from the member the image source 120 of such as website and so on, image tag and optional Search Results can be provided to website main frame, to third party, to ad system 180 and/or analog.In certain embodiments, image tag is automatically added into website, and image tag can be searched for, such as, and can be searched to find audited image.

In queuing image step 460, image is arranged in image queue 145.This layout comprises the subset using image tagged logical one 47 marking image alternatively.Described by other place herein, mark is configured to the object of the special interests in recognition image usually.The lifting of the image in image queue 145 can depend on the priority of the examination & verification of image, the source of image, available mankind's image audit person, the tolerance of degree of confidence of examination & verification of image of Practical computer teaching and/or analog.

Determining in the step 465 of destination, the one or more members in destination 125 are by the manual examination & verification determined for image.The determination of destination is alternatively based on the image tag be included in the image audit of the Practical computer teaching received from automatic recognition system 152; Alternatively based on the specialty of the mankind's image audit person at different destination 120 place; Alternatively based on the examination & verification mark of these mankind's image audit person, and/or based on other standard discussed herein.In certain embodiments, the data of destination step 465 feature based image and the specialty of human reviewer is determined.The data of characterized images can be characteristics of image, image descriptor and/or the information from its derivation.As other place herein discussed, characteristics of image and/or image descriptor are received from the member image source 120 alternatively together with image.By its derive information can be created on image source 120 member, at image processing system 110 place and/or at automatic recognition system 152 place.

Putting up in image step 470, image is posted to determining in the step 465 of destination by least one destination in the destination 125 determined.In certain embodiments, put up image step 470 to comprise image is puted up concurrently to more than one destination 125.This image be posted via network 115 alternatively and alternatively with following listed together be posted: time before the mark of the subset of outstanding image, the source data for image, the examination & verification at image lost efficacy, for the image tag of image that receives from automatic recognition system 152 and/or analog.

In reception examination & verification step 475, the manual examination & verification of image is received from one or more (multiple) determined destination 125.This manual image audit can comprise the one or more image tags being assigned to image by mankind's image audit person.This one or more image tag represents the content of image.This instruction that manually examination & verification can also comprise upgrade request, the instruction of image that can not audit, the inappropriate instruction of image, examination & verification were lost efficacy and/or similar.

Tagged at image? step 480 in, the progress of the method depends on that whether image tag received in reception examination & verification step 475.If the image tag of the content of characterized images is received, so the method proceeds to alternatively and performs search step 450 and provide result step 455.In those steps, the image tag comprised at the image audit of manual image examination & verification and Practical computer teaching is alternatively used.The use of the image tag in the image audit of Practical computer teaching can depend on the confidence metric of this examination & verification.

If confidence metric is found to be higher than (multiple) predetermined threshold in step 445, step 460-475 is optional.

In optional upgrading? in step 485, the progress of the method depends on whether upgrade request is received.If such request is received, so method proceeds to and determines destination step 465, and wherein second/different member of destination 125 is determined.This determines to depend on image tag received in manual image examination & verification, and the examination & verification of this manual image is received in reception examination & verification step 475.Upgrade request can be received from mankind's image audit person or be received from the requestor (from image source 120A or 120B etc.) of image audit.Had an opportunity examination & verification after providing the image tag be provided in result step 455 requestor, and upgrade request can be received.Such as, first requestor can receive the image tag the upgrading of request examination & verification subsequently that comprise " white car ", because they expect to have further information.Examination & verification upgrading can cause image to be provided to having the mankind's image audit person in automotive field specialty.This mankind's image audit can be added into the image tag of existence to produce " white car, 1976 Ford Granadas ".In certain embodiments, when asking examination & verification upgrading, requestor can add the source data of the subset of indicating image.Such as, auditor can wish to indicate the special interests on the head lamp damaged.This, for this feature by mankind's image audit person notice guiding image, produces the label comprising " head lamp of damage ", and causes guiding for the search (performing search step 450) of the head lamp of the damage of 1976 Ford Granadas.

In certain embodiments, upgrade request is audited logical one 70 and is automatically generated.Such as, if image audit occurs too concise and to the point, such as, be only " car ", so auditing logical one 70 can automatically initialization examination & verification upgrading.In certain embodiments, the automatic generation of upgrade request is based on the existence of keyword in manual image examination & verification.Such as, some examination & verification specialty is associated with the list of keyword.In certain embodiments, when the keyword of in these keywords is received in manual image examination & verification and automatically examination & verification upgrading is initialised.Examination & verification upgrading preferably includes mankind's image audit person with the specialty be associated with the keyword received.In particular example, a specialty comprises " automobile " and is associated with keyword " car ", " truck ", " lorry ", " open car " and " Ford ".When the keyword of in these keywords is received in manual image examination & verification, examination & verification logical one 70 and examination & verification logical one 57 are consulted to determine whether the mankind's image audit person had in " automobile " specialty is active at present.If active, so auto-update is initialised and image is sent to the destination 125B of the auditor with " automobile " specialty.

If do not have upgrade request to make, so in termination step 490, this process completes.

Fig. 5 illustrates the alternative of process image according to various embodiments of the present invention.In these methods, at least some in step 430-445 and at least some in step 460-475 are performed concurrently.Manual image examination & verification in step 460-475 can start thus manual image examination & verification and start before the confidence metric of examination & verification knowing Practical computer teaching before the examination & verification of the Practical computer teaching of step 430-445 completes.If be sure of? step 445 confidence metric is found to be higher than (multiple) predetermined threshold, and so step 460-475 is terminated alternatively.

Fig. 6 illustrates the method in management auditor pond according to various embodiments of the present invention.In the method, the level that the state of auditor can audit image based on them is changed.Illustrated step can be the part of the illustrated method of Fig. 4 and Fig. 5 or as one man be performed with the illustrated method of Fig. 4 and Fig. 5.Such as, they can be audited between step 475 in reception image step 410 and reception and be partially executed one at a time.Illustrated method comprises to the more than one destination transmission image in destination 125.

In reception image step 410, image is received.As other local discussion in this article, image can be received at image processing system 110 place via network 115.Image can be generated by camera and/or obtain from webpage.In certain embodiments, image be often received together with the information of watching about this webpage is many.

In the step 610 of selection first destination, the first destination is by the analysis manually or automatically selected for image.Logical one 60 is performed and is the embodiment determining destination step 465 with selecting the first destination step 610 application target.As in this article described by other place, for the determination of the destination of image based on various factors, the state comprising human reviewer and the mark be associated with auditor.Such as, the member of the destination 125 be usually associated with active auditor will be selected, instead of does not have the destination of active auditor.Selected destination can be member and/or the automatic recognition system 152 of destination 125.

Putting up in image step 470, be posted to the member selected by destination 125 at the image receiving reception in image step 410.As other local is discussed in this article, the putting up to comprise of image uses the computer network with standard network protocol of such as TCP or UDP and so on to transmit image via network 115.

Monitor in step 620 optional, auditor's logical one 70 is used to the progress of the manual image examination & verification of the image at the member place monitoring the destination 125 selected in the step 610 of selection first destination.Supervision can comprise the input of human reviewer, image audit time used of detecting, the quantity of the word provided of characterized images and/or similar.Monitor to comprise alternatively and measure the label image time used.Wherein detect and comprise the detection of the input of human reviewer, monitor can based on keystroke one by one, based on word one by one and/or based on line by line.Thus, auditor's logical one 70 can be configured to the data of receive feature image character, word or a row.

Removing step 620, the member place of the destination 125 that image is selected in the step 610 of selection first destination is removed from process." remove " the selected member place that can be included in destination 125 notify human reviewer he or she no longer primary responsibility examination & verification image, the human reviewer of prime responsibility is removed (and need not notify human reviewer), the display of image from human reviewer is removed, and/or similar.In certain embodiments, remove step 630 to comprise and only human reviewer is placed in classification to have for the secondary of examination & verification image or the responsibility shared.Such as, if the human reviewer be associated with the member of the destination 125 selected in the step 610 of selection first destination has the prime responsibility for auditing image, this responsibility can be shared or be assigned to other auditor be associated with other member of destination 125 now.In this case, what " remove " is prime responsibility.

If the manual examination & verification of image is used too of a specified duration, to remove step 630 can occur.Such as, if found that auditor does not start input after the schedule time in supervision step 620, so removing step 630 can be performed.Other example for the trigger event removing step 630 comprises with the communication loss of the member selected by destination 125, to exceed the schedule time for the distribution of the examination & verification of image, receives incorrect or inappropriate image from human reviewer, receives the image tag of inaccurate (non-characterized images) from human reviewer, from the upgrade request of the recommendation of the first human reviewer to the second human reviewer, image audit and/or similar.

In the step 640 of selection second destination, the second member (or automatic recognition system 152) the application target ground logical one 60 of destination 125 is selected.Second member can based on above about selection first destination step 610 and determine that any standard that destination step 465 is discussed is selected.In addition, in certain embodiments, the selection of the second member can based on the specific recommendations by the human reviewer be associated with the first member of destination 125.Such as, the first human reviewer can the content of recognition image be the second human reviewer specialty and can by this image recommendation to the member of the destination 125 be associated with the second human reviewer.The automatic process of the image using automatic recognition system 152 is depended in the selection of the second member of the destination 125 in the step 640 of selection second destination alternatively.

Put up in image step 470 at another, image is posted the member to the destination 125 selected in the step 640 of selection second destination.In certain embodiments, more than one human reviewer can audit image concurrently.They can perform examination & verification independently or cooperative.Auditor can have the prime responsibility of the examination & verification of image or each auditor can have equal responsibility.Auditor can have the supervisory responsibility to other auditor one or more.In certain embodiments, in supervision step 620 and/or before removing step 630, the second destination step 640 is selected to be performed and image is posted two or more members to destination 125.

In reception examination & verification step 475, the examination & verification (such as, image tag) of image as other place in this article discussed as received.Examination & verification generally includes the image tag of the content of characterized images.Examination & verification can receive from the more than one destination destination 125.Such as, the label of characterized images can from selection first destination step 610 with select to be selected by the member of destination 125 that selects both the second destination steps 640.Thered is provided by (multiple) human reviewer with character or word, receive examination & verification step 475 and be performed in real time alternatively.

In optional correlation tag step 650, one or more image tag of characterized images and image are stored explicitly.Stored label comprises the label that provided by more than one mankind auditor alternatively and can be stored in storer 135.As in this article as described in other place, label can be provided to member's (such as, in the embodiment providing result step 455) of image source 120 further or be used for using ad system 180 to select advertisement.Label can also be provided to automatic recognition system 152 and provide the training of automated graphics identification process.

Fig. 7 illustrates the method receiving image tag in real time according to various embodiments of the present invention.These methods are performed by image processing system 110 alternatively and image tag can be the result that manual image is audited.These methods can be consistent with other method described herein, such as, as the part of the method shown in Fig. 4.The method starts to put up image step 470, and in this step, image is provided to one or more members of destination 125, as other place herein discussed.Method shown in Fig. 7 is first to receive image step 410 alternatively, and image is received from remote computing device in this step.

In reception input step 710, input and receive from one or more members of destination 125.This input generally includes the character provided by human reviewer.Such as, input can be the character inputted at destination 125A by human reviewer.Usually, be performed with other step shown in Fig. 7, receive image step 710 and continued.

In detection first word step 720, detected in the input that word is received in reception input step 710.This word can by the such as space character in ASCII space and so on or the existence of carriage return detect.Spell check is performed alternatively on the word detected.If word is not included in spell check dictionary, the trial so corrected can be made or human reviewer can notified identification word failure.

The detection of the word in detection first word step 720 causes the execution of sending the first word step 730, and in this step 730, word is transferred into the source of image.Such as, once word is detected, it can be provided to image source 120A in real time.At image source 120A, this word can show to user.Wait for until the whole set of image tag is received compared with before the set of display image tag, once showing a word can provide the analysis of image to occur in the impression of shorter time quantum.

In detection second word step 740, detected in the input that the second word is received in reception input step 710.Again, word can detect by the existence of blank character and can occur after the user the first word being provided to image source 120A place.Both first and second words are expected for the label of characterized images.Trigger in the detection of the second word of detection second word step 740 and send the second word step 750, in step 750, the second word is delivered to image source, such as, and image source 120A.Detect the second word step 740 and send the second word 750 and can be recycled and reused for the 3rd, the 4th and additional word, each word is as the part of image tag.

In detection completing steps 760, the data that the process of indicating image completes are received, and this processes precedent such as the word for detecting and comprises whole words (image tag) that will be provided by human reviewer.These data can comprise such as "/endtags ", ASCII carriage return and/or similar metadata tag.Usually, detect completing steps 760 one, two or more image tags received after occur.In optional correlation tag step 650, the image tag received is associated with image and/or is stored together with image, as other place herein discussed.

Although Fig. 7 illustrates word in detection once with send, in alternative embodiment, independent keystroke is detected and send.Receive input step 710 to continue concurrently with step 720-740 and/or 750.Step 710-760 can be included as the part receiving examination & verification step 475, and other place in this article comes into question.

Fig. 8 illustrates the method for upgrade image examination & verification according to various embodiments of the present invention.In these methods, the more than one stage of image-receptive image audit.Then the first examination & verification (stage 1), image audit is upgraded and is audited (stage 2) further.Request for upgrading can be generated automatically, and is initiated by the first human reviewer, and/or in response to the request in the source from image.Both first and second examination & verifications can be manual, namely have human reviewer to perform.Alternately, the first examination & verification can be automatic and one or more examination & verification subsequently can be manual, or the first examination & verification can be manual and one or more examination & verification be subsequently automatic.

In reception image step 410, image is received.First member of destination 125 is selected for the image in the step 610 of selection first destination.This image is posted putting up in image step 470 subsequently.These steps other place in this article comes into question.

Audit in step 810 in reception first, the first examination & verification of image is received.This first examination & verification can comprise one or more image tags of the content of characterized images.Such as, in response to the picture of image comprising black spider, image audit can comprise word " Black Aranea "; Or in response to comprising the image of red car, image can comprise word " screw oil expeller ".Receiving the first examination & verification step is the embodiment receiving examination & verification step 475 alternatively, and can comprise the real-time Communication for Power of image tag, as discussed with respect to FIG. 7.

In certain embodiments, the first examination & verification can comprise the instruction that process that provided by the human reviewer of execution first image audit, image should be upgraded.Such as, the first human reviewer can the speciality field of manually pointer (alternatively specialization) human reviewer to second.Such as, the first human reviewer can provide " screw oil expeller " image tag and advise being performed by the auditor with automobile speciality the examination & verification of upgrading.Alternately, the first examination & verification can comprise and is considered to valuable especially image tag.Such as, the possibility that indicating image comprises wedding clothes be 72% automatic examination & verification can trigger auto-update to manually examination & verification, because image tag " wedding clothes " has larger commercial value than other image tag potentially.In certain embodiments, this auto-update is audited logical one 70 and to be performed and whether based on relatively important or valuable keyword is stored in the list in storer 135.This list can comprise the tolerance associated of keyword and their value.As other place herein discussed, the auto-update performed by examination & verification logical one 70 is alternatively based on using automatic recognition system 152 and/or the many image tags often automatically will generated by the information of watching of predicted picture.These factors use alternatively to maximize and use the algorithm of the potential value of human reviewer's label image and provide advertisement based on these labels and be employed.The example of more valuable image tag can be relevant to footwear, car, jewelry, Reiseziel, book, game, clothes, vacation, food, beverage, real estate, bank, accident etc.

In certain embodiments, the upgrading of image audit is automatic.Such as, the label of " Black Aranea " can automatically cause comprising the upgrading sending the image audit of image to the human reviewer with specific speciality (such as, spider expert).The identification of specific plant or animal life generally includes (depending on) positional information, because the position of plant or animal can be overstate for appropriate identification to want.

In certain embodiments, as other place herein discussed, upgrade audit can be asked by the people audited by raw requests picture.Such as, the user of image source 120A can provide the image of dog and receive the image tag comprising " black dog ".User can subsequently by provide word " cultivate? " carry out the further details of requirement.In this case, image audit can be upgraded and be sent to the human reviewer of the specific instruction of the cultivation with dog.In certain embodiments, user charged or is required to have premium account for upgrading so that request upgrading.User can specify the specific part of image when requested image examination & verification upgrading.

The existence (automatically and/or manually generating) of upgrade request is detected in detection upgrade request step 820.Detection can based on the member from image source 120, the member from destination 125, automatically recognition interface 150 and/or from such as auditing logical one 70, data that the parts of image processing system 110 of contents processing logical one 85 or response logic 175 and so on receive or order.

In the step 640 of selection second destination, destination logical one 60 be used to select your destination 125 the second member and/or automatic recognition system 152 for the examination & verification of image.This selection can based on the first destination step 610 based on any one of standard, and image tag and/or the out of Memory that produces from the first examination & verification.Such as, the selection of the second member of destination 125 can be based on, is based, at least in part, on the image generated manually or automatically in the first image audit.Especially, the label of " Black Aranea " can be by, and destination logical one 60 uses the member selecting the destination 125 be associated with the human reviewer of the speciality of the identification with spider.In another example, the selection of the second member of destination 125 word that can provide based on the user audited by requested image.Especially, if first examination & verification produce image tag " white footwear " and user response with " brand? ", so destination logical one 60 can use this information to select the member of the destination 125 be associated with the human reviewer of the speciality with footwear brand.

In some embodiments of selection second destination step 640, destination logical one 60 is configured to select automatic recognition system 152 for the second examination & verification of image, instead of select your destination 125 member.Such as, when image had the name of performer by label and upgrade request requires " movie name? " time, this may occur.Under these circumstances, image can be searched in the storehouse of film image.Identical method can by other the reproducible object taked for such as currency, drawing, vehicle, trade mark, bar code, QR code, fatal personage etc.

In another example putting up image step 470, image be posted to destination 125 second selected by member or automatic recognition system 152 for image second examination & verification.Audit step 830 in reception second, the image tag of the content of characterized images is received usually.Receiving the second examination & verification step 830 is the embodiment receiving examination & verification step 475 alternatively.Alternately, indicating image is because some reason can be able to not be received by the additional reference of label or out of Memory.Image tag is received from the member of the destination 125 that image is posted to or automatic recognition system 152.If necessary, step 820,640,470 and 830 can be repeated.

In correlation tag step 650, the image tag received is associated with image and/or is provided to the source of image, as other place herein discussed.

In an illustrative examples of the method shown in Fig. 8, image is received from webpage.Image is sent to automatic recognition system 152 for automatic examination & verification.The result of automatic examination & verification comprises image tag " ring ".This label uses examination & verification logical one 70 to be processed and to be identified as potential valuable image and uses in advertisement.As other place herein discussed, this identification is alternatively also based on the many other factorses to be watched on webpage of being everlasting and so on of such as image.As the result identified, the examination & verification of image is automatically upgraded and is sent to the member of the destination 125 be associated with the human reviewer with jewelry speciality.This human reviewer revises image tag and comprises " gold wedding ring " and these labels are associated with image.These image tags can be used to use system and method for other local description herein to select advertisement subsequently.

In an illustrative examples of the method shown in Fig. 8, image is received from the application performed on the mobile apparatus.This image comprises streetscape and is sent to destination 125A to be audited for by human reviewer.Human reviewer makes response with image tag " streetscape " and these labels are provided to mobile device.The request of examination & verification upgrading is received from mobile device subsequently.This request comprise text " vehicle? " and the identification of the part of image comprises car.As the result of this request, image, text and identification are sent to the other member of destination 125A or destination 125 for further manually examination & verification.Further examination & verification causes image tag " 1909 model T ", and it is forwarded to mobile device subsequently.

In an illustrative examples of the method shown in Fig. 8, image is received from computing equipment.This image comprises multiple bank note of being placed on dish and is sent to destination 125A for manual examination & verification.The label produced comprises " American currency of Bai Panshang " and is sent to computing equipment.Request for image audit upgrading is received.This request comprises " how many? " as the result of this request, image is sent to automatic recognition system 152, is used to the amount of identification currency and provides sum alternatively at this place's automatic currency recognition logic.This information is sent back to computing equipment subsequently.

In an illustrative examples of the method shown in Fig. 8, image is received from mobile device.This image comprises plant leaves and is sent to destination 125A.Image tag " greenery " and upgrade image examination & verification is provided the human reviewer at 125A place, destination.As upgrading and the result of image tag, image is sent to destination 125B, and it is associated with second mankind's image audit person with botany speciality.The selection of destination 125B is based in part on image tag " greenery ".Concurrently, image tag " greenery " is sent to mobile device.At 125B place, destination, second mankind's image audit person adds word (label) " toxicodendron " to the image tag existed.These additional labels are also sent to mobile device subsequently.On the mobile apparatus, word " greenery " be first shown and subsequently once available word " toxicodendron " is added into display.

Method shown in Fig. 8 is used in consistent with other method described herein alternatively.Such as, image tag can be used in auction labelling step 1560 of other local description herein.Label can be used to select advertisement and advertisement to be provided to remote browser for showing on webpage together with image.

Fig. 9 illustrates the example comprising the image source 120A of electronic glasses according to various embodiments of the present invention.Electronic glasses comprises worn glasses.Example comprises Google " Google's glasses ", " M100 intelligent glasses ", Innovega iOptik ^tMcontact lenses and/or similar.These systems be configured to allow user at the same time between watch real world and electronic console.Electronic glasses can also comprise the OculusRift of such as OculusVR ^tMand so on virtual reality system.The system of these types uses electronic console to be shown to user by image, but direct viewing while not providing real world.Viewing and digitized viewing is not used to, such as, by the viewing of glasses or eyeglass during direct viewing.

Usually, electronic glasses provide when image in electronic glasses or when being watched by electronic glasses user can select the interactive interface of the subset of image in real time.As used in this article, " in real time " is selected to mean image and is watched with only inessential time delay when it is collected.Such as, the image watched in real time can by collected by camera and use image engine process and with only from electronic processing number of times produce time delay be shown.Real-time viewing allow user along with image by watched by mobile image acquisition equipment by interested objects location viewing image within.Thus, viewing eliminates the viewing of the image being stored the quite a while before viewing in real time.

As shown in Figure 9, image source 120A comprises the camera 910 being configured to gather image.The image gathered can comprise rest image or comprise the video of image sequence.Display 920 is configured to the user gathered image being presented to camera 910.In certain embodiments, such as image source 120A is those embodiments of smart phone, display 920 comprises the touch-screen of view finder being configured to be used as camera 910, to show gathered image and to show image acquisition screen 210 (herein described by other place).Display logic 925 is configured to other content on the display of managing image and display 920.Display logic 925 can comprise storage hardware on a computer-readable medium, firmware and/or software.

In the embodiment shown in fig. 9, image source 120A comprises the selection logic 930 that the user being arranged to image source 120 indicates the subset of the image gathered further.This instruction can just be shown along with image and/or be collected on display 920 and made in image acquisition screen 210 in real time.As other place herein discussed, such instruction is made selecting the interested special object in image.And then select, the subset of image uses image tagged logical one 47 to be labeled alternatively, as other place herein discussed.Image tagged logical one 47 can be used the image of display in such as display 920 to add mark.Thus, user can see the position be labeled.Logic 930 is selected to be arranged to the subset of user's marking image again alternatively, until user is satisfied to selection.

In certain embodiments, select logic 930 to comprise and follow the trail of logic 935 with the movement of track user eyes.Follow the trail of logic 935 to be included in alternatively in electronic glasses.Eye tracks can comprise detect the focus point of eyes, the direction (eyeball direction) of one or more eyes, the focusing of one or more eyes, nictation, eyeball move and/or similar.Follow the trail of logic 935 to be configured to the state of eyes of user to be associated with the position in gathered image alternatively.Represent that camera 910 and the geodata of the geographical relationship selected between the physical factor of logic 930 are used to the state of eyes of user to be associated with the position in the image using camera 910 to gather.

Follow the trail of the second camera that logic 935 comprises the glasses pointing to user alternatively.This camera can be installed in the part of other embodiment on electronic glasses or as image source 120.Such as, the tracking logic 935 being arranged to the eyes of track user can be included in network cameras, in smart phone, in computer monitor, in TV, in flat computer and/or in analog.

In certain embodiments, follow the trail of logic 935 and be configured to the nictation detecting one or more eyes.Such as, follow the trail of logic 935 and can be configured to the pattern detecting simple eye nictation or nictation.When such time is detected, logic 930 is selected based on the position patrolled from tracking in the eye position data selection image of 935 receptions, or can alternately to select the position at the center at the image watched at present.

Once image usage flag logical one 47 and select logic 930 to be labeled, the position of this mark and/or region the image of mark in display 920 can be displayed to user.Such as, add that the image of red " X " can shield in 210 in image acquisition in marked locations and be displayed to user.In certain embodiments, user can use subsequently and confirm that logic 940 confirms to select.Confirm logic 940 alternatively in response to tracking logic 935.Such as, confirmation can use nictation or other action, voice command, verbal order or touch order to be provided.In certain embodiments, follow the trail of logic 935 be configured to detect and be interpreted as that order eyes enter the movement of not physical slot (such as, esotropia).Such movement can be used to provide and confirm order.Confirm to be required prior to sending image to network 115 alternatively.

In certain embodiments, logic 930 is selected to comprise the tracking logic 935 being configured to some thing followed the trail of except eyes.Such as, follow the trail of logic 335 can be configured to detect user indication finger (pointingfinger), be worn on finger or wrist on electronic equipment and/or analog.In these embodiments, logic 930 is selected to be configured to the position of inferring based on detected object in image.In one embodiment, follow the trail of logic 935 be configured to detect the position of the finger of giving directions in image and infer and will be located at the tip of finger by the position selected.User can give directions the object in its visual field, to image source 120A provide audio frequency, based on eyes and/or based on the order touched, and the position of finger given directions will be used to the selection of the position made in image.In one embodiment, follow the trail of logic 935 and be configured to detect radio-based electronic devices relative to the position of image source 120A and infer will by the position selected along the line between radio-based electronic devices and the part of image source 120A.

Image source 120A comprises further and is arranged to image source 120A is sent to image processing system 110 I/O945 via network 115.I/O945 can comprise wired and/or wireless connection.Such as, in certain embodiments, I/O is configured to use bluetooth ^tMconnect and wirelessly to communicate from electronic glasses to cell phone and subsequently for using Wifi or cellular service that communication is forwarded to network 115 from cell phone.

Image source 120 comprises the embodiment of storer 135 of image being configured to store and using camera 910, geodata, account data and/or similar collection further.Storer 135 comprises the non-transitory storer of such as random access memory (RAM) or ROM (read-only memory) (ROM) and so on.Storer 135 generally includes the data structure being configured to store image and the mark position in these images gathered.

Image source 120 comprises processor 950 further.Processor 950 is the digital processing units being configured to perform computations.Such as, in certain embodiments, processor 950 is encoded with computations to perform display logic 925, selection logic 930, image tagged logical one 47 and/or to follow the trail of logic 935.Processor 950 comprises application-specific IC (ASIC) or programmable logic array (PLA) alternatively.

Image source 120 comprises object tracing logic 955 alternatively further.Object tracing logic 955 is configured to the movement of following the trail of interested object in a series of images.Such as, in certain embodiments, user can use and select logic 930 to select to be required to obtain the subset of the image of its information or the aspect of image.This subset can comprise one or more pixel.Object tracing logic 955 is configured to use (computer based) image interpretation logic automatically to take the special object of selected subset with identification.This object can be people, motor vehicle, animal or other object any.The border of selected object or other pixel are outstanding in display 920 by object tracing logic 955 alternatively.This is outstanding, and with object, in a series of images, movement can tracing object and can comprise the characteristic changing pixel.This is given prominence to and moves together with the object on display alternatively.An aspect of image can be objects within images brand, obtain the position etc. of the film of image, the content of image.In certain embodiments, image aspect can use such as " footwear brand? ", " film? ", " performer? ", " position ", " cultivate? " be designated as interested in text.This appointment can be provided adding image in tagged raw requests and/or in upgrade request.

In certain embodiments, from image source 120 to image processing system 100 by the image transmitted be the part of a series of images comprising short video sequences.These video sequences can use system and method for other local description herein to add label.Adding a tagged advantage to video sequence is that (multiple) label can characterization generation specific action in video.Such as, the label of figure skater can specifically jump (hooking hand two weeks) in characterization, and it identifies better than in rest image in video.Various embodiment comprises the specific restriction in the length of image sequence, such as, and necessary no more than 3,5,7 or 10 seconds of video.

Although the embodiment shown in Fig. 9 comprises electronic glasses, these embodiments can be adapted to be any equipment with ocular pursuit technology, comprise mobile phone, video display monitor (such as computing machine or TV screen), tablet computer, advertisement display etc.Such as, the embodiment of the image source 120A shown in Fig. 9 comprises having and is configured to determine that user is watching the TV of the ocular pursuit camera of which part of TV screen.

Image source 120A comprises further alternatively and is configured to perform for the image processing logic 960 of one or more steps of label image object.Image processing logic 960 is configured to the load reduced on image processing system 110 in one or more step of image source 120A this locality by performing these alternatively.Such as, image processing logic 960 can be arranged to the initial step of the label performing image and subsequently the result of these initial step is sent to image processing system 110 for synthetic image label.In certain embodiments, image processing logic 960 can complete some but need not to be the label process of all images.Image processing logic 960 comprises storage hardware on a computer-readable medium, firmware and/or software.Such as, some embodiments comprise the example of the processor 950 of the function be specially configured as performing image processing logic 960 discussed in this article.

In certain embodiments, image processing system 110 is configured to provide image processing logic 960 to image source 120.This is alternatively via the application shop " AppStore " of such as apple.Be suitable for, the member to image source 120 provides image processing logic 960 to be optional steps in various method shown in this article.Processing logic 960 can be provided as the computer instruction or " application (app) " that comprise other logic discussed in this article (such as, about the logic that Fig. 9 discusses) further.

In certain embodiments, image processing logic 960 is configured to the special characteristic in recognition image.Feature identification comprises the part of the feature of the given type of specified point yes or no determined whether in image.The type of feature includes but not limited to edge, corner, spot (blob) and ridge (ridge).Usual feature is " interesting " or " useful " part of the object of the content for recognition image of image.It is one or more that image processing logic 960 can be configured to perform in some different characteristic detection algorithms.In certain embodiments, image processing logic 960 is configured to select from some different algorithms based on the content of available processing power and/or image.The example of known feature detection algorithm comprises " Canny ", " Sobel ", " Harris & Stephens/Plessy ", " SUSAN ", " Shi & Tomasi ", " level curve curvature ", " FAST ", " Gauss-Laplace ", " Hai Sai (Hessian) determinant ", " MSER ", " PCBR " and " gray scale spot (grey-levelblobs) ".The algorithm of these types is performed on the computing device and other such algorithm will be apparent to those of ordinary skill in the art.The result of feature identification is included in the identification of the special characteristic type of the ad-hoc location in image.This can be encoded in " feature descriptor " or " proper vector " etc.The structure of feature detection can also comprise the value representing the level of confidence be identified in its place's feature.

In certain embodiments, image processing logic 960 is configured to further based on identified box counting algorithm image descriptor.Image descriptor is the visual signature of the content of image and comprises the characteristic of such as shape, color, texture and action (in video).Image descriptor can be the part in particular descriptor territory, such as relevant to face recognition or currency identification descriptor.The derivation of image descriptor is usually based on characteristics of image.Such as, the derivation of 3D shape description symbols can based on detected edge feature.Image descriptor can one or more identified object in characterized images.

Be used in the specific image identification algorithm that specific image features in specific embodiment and image descriptor depend on use.Great amount of images identification algorithm is known in the art.In certain embodiments, image processing logic 960 and/or image processing system 110 are configured to first attempt the identification of characteristics of image and the derivation of various types of image descriptor, and select from multiple alternative image processing algorithm based on the level of confidence be exported at its place's image descriptor subsequently.Such as, if the image descriptor in face recognition territory is exported have high-caliber degree of confidence, the image processing algorithm so specific to face recognition can be selected with synthetic image label from these image descriptors.

In those embodiments comprising image processing logic 960, the task of label image can be distributed between image source 120 and image processing system 110.How task is assigned with can be fixing can be maybe dynamic.In the embodiment that distribution is fixing, specific step and specific equipment are consistently performed.Be in dynamic embodiment in distribution, the distribution of step such as can in response to the processing power on communication bandwidth, image type (static or video), image source 120A, the present load on image processing system 110, the availability of the image audit person at destination 125 place, degree of confidence that step is done in image source 120 and/or the image descriptor data be presented on image source 120A.The combination in any of these factors is used to the distribution of dynamically allocation process step.Such as, if the derivation of image descriptor occurs with the degree of confidence of the low degree on image source 120A (relative to predetermined demand), so characteristics of image and/or image may be transferred into image processing system 110 and carry out deduced image descriptor for using more powerful or alternative image processing algorithm.Compare, if the derivation of iamge description degree occurs on the image source 120A of degree of confidence with enough degree, so this step does not need to be performed on image processing system 110 usually.

If image processing step is successfully performed on image source 120A by image processing logic 960, the result of image and/or these steps can use I/O945 to be transferred into image processing system 110.Such as, in certain embodiments, image and image descriptor are all transferred into image processing system 110.Image descriptor can be used in and automatically add tagged trial to image and maybe can be provided to mankind's image audit person at one or more destination 125 place.Image descriptor can be used to identify that descriptor field and this territory are used to the member of the destination 125 selecting image to be sent to subsequently.Such as, the descriptor field of " motor vehicle " can be used to the image audit selecting to have motor vehicle speciality.Based on image descriptor, Images Classification is entered in territory to occur on image processing system 110 or image processing logic 960.

In certain embodiments, the automated tag interpolation of image is attempted based on derived image descriptor.In various embodiments, this uses image processing logic 960 and/or automatic recognition system 152 to occur.Classification is alternatively by comparing from having and the image descriptor of deriving the image in the storehouse of the different classes of image descriptor be associated and occurring.Such as, cognitron motor-car shape image descriptor can be relevant to " motor vehicle " classification before the image descriptor that stores match.If the identification of applicable (classification, the scope etc.) classification of this classification enough automatically can select the label for image.Such as, those the image descriptor mating classification " children's face " can enough generating labels " faces of children ".

Usually, image processing system 110 comprises the larger storehouse with the different classes of image descriptor be associated relative to image source 120A.These storehouses are stored in the storer 135 of image processing system 110 or image source 120A or automatic recognition system 152 alternatively.Be stored in the storehouse of the image descriptor in image source 120A alternatively based on the image using the first pre-treatment of image source 120A.Such as, if multiple image from image source 120A is identified as having the label about currency and descriptor, can be stored in the storer 135 of image source 120A in the storehouse of the descriptor of currency territory/classification.These descriptors can be associated with the label of such as " 5 dollar bill " and so on.When the new images of the identity set with descriptor is received, image processing logic 960 is configured to use the label of association automatically to add label to image alternatively.Although descriptor storehouse can be received from image processing system 110, or the image tag received from image processing system 110 can be used to be developed, the label in above example does not depend on the real-time Communication for Power with image processing system 110.

In various embodiments, the data of the relation between characterized images descriptor and classification and/or label can be developed in image processing system 110, image source 120A, destination 125A and/or automatic recognition system 152.Once be developed, data can be transmitted to improve and/or be replenished the storehouse at any miscellaneous equipment place.

Although illustrated system shows master-slave architecture, image source 120 and destination 125 are connected with end-to-end framework in alternative embodiments.In these embodiments, the combination in any of the key element shown in image processing system 110 can be included in image source 120 and/or destination 125.One in image source 120 can perform image tag discussed in this article and add and Processing tasks at the image received from another in image source 120.

Figure 10 illustrates the method for process according to various embodiments of the present invention at least partially in the image on image source 120A.Method shown in Figure 10 can comprise a series of different disposal steps performed on image source 120A.Such as, those steps relating to image descriptor are performed alternatively on image processing system 110.

Reception image step 1010 in, image receive by image source 120A.Image can from the camera be included in image source 120A, from image source 120B, from network 115, from image processing system 110, from wireless device, be received from memory devices and/or analog.The image received is form an image in the image sequence of video alternatively.

In recognition feature step 1020, image processing logic 960 is used to identify the characteristics of image in the image that accepts.As other place herein discussed, the method for recognition image feature is well known in the art.Recognition feature step 1020 can apply one of these methods, two or more.The identification of feature comprises the estimated accuracy of reflection feature identification and/or the level of confidence of integrality alternatively.

Send in characterization step 1030 optional, the characteristics of image be identified in recognition feature step 1020 is sent to image processing system 110.Feature can with or be not sent out together with the image associated and can be sent out via network 115.If sending characterization step 1030 is included in the method, next the method proceeds to generate/receive labelling step 1070 alternatively, is received in this step 1070 for the label of image from image processing system 110.Image processing logic 960 is configured to perform based on the level of confidence of feature calculated in recognition feature step 1020 send characterization step 1030 alternatively.Such as, if degree of confidence is lower than threshold value, step can be performed and image and feature are all sent out.

Derive in descriptor step 1040 optional, image processing logic 960 is used to deduced image descriptor from the characteristics of image be identified recognition feature step 1020.As discussed in this article, method miscellaneous is all known to deduced image descriptor in the art.In certain embodiments, derive descriptor step 1040 and comprise use more than a kind of method.Derivation can comprise the estimated accuracy of reflection descriptor derivation and/or the level of confidence of integrality.The type of the descriptor derived and content depend on used (multiple) image edge sharpening arithmetic usually.

Send in descriptor step 1050 optional, the image descriptor be exported in derivation descriptor step 1040 is sent to image processing system 110.Image descriptor can with or be not sent out together with the image associated and can be sent out via network 115.If sending descriptor step 1050 is included in the method, next the method proceeds to generate/receive labelling step 1070 alternatively, is received in this step 1070 for the label of image from image processing system 110.Image processing logic 960 is configured to send descriptor step 1050 based on performing in the level of confidence deriving the characteristics of image be exported in descriptor step 1040 alternatively.Such as, if degree of confidence is lower than threshold value, step can be performed and image and feature are all sent out.

Optionally comparing in descriptor step 1060, compared with one or more image descriptors that the one or more image descriptor be exported in derivation descriptor step 1040 and this locality store.As other place herein discussed, these local image descriptors stored are associated with image category and/or image tag.Relatively can comprise the calculating of the characteristic of the quality of reflection coupling.

In certain embodiments, send descriptor step 1050 and compare both descriptor steps 160 and be performed.In this case, the process of image descriptor can occur on both image source 125A and image processing system 110.Similarly, in certain embodiments, send characterization step 1030 and derive both descriptor steps 1040 and to be performed and characteristics of image is processed in system/device.

In appointment/reception labelling step 1070, the image tag of characterized images is generated and/or receives.Such as, if image, characteristics of image or image descriptor have been sent to image processing system 110, so corresponding label can be received from the image processing system 100 of assigning/receiving labelling step 1070.If mating between the descriptor that the descriptor of deriving in relatively descriptor step 1060 and this locality store is found, the label that the image descriptor of the storage of so mating with this locality is associated is acquired from local storage and is assigned to image.Can both be assigned also by local reception by this locality for identical image tag.Image tag uses characteristics of image and/or descriptor to be generated alternatively, such as, does not have image processing system 110 to receive actual image.

In certain embodiments, assign/receive labelling step 1070 to comprise and classification is assigned to image, send image and classification to image processing system 110, and receive back corresponding label from image processing system 110.Label can use the method shown in Fig. 4 to be identified.Classification can be used automatic recognition system 152 and/or the human reviewer's generating labels at 125A place, destination by image processing system 100.

That assigned and/or receive label (and/or other result) and be provided providing in result step 455, as other place herein discussed.

Figure 11 illustrates the method based on image descriptor process image according to various embodiments of the present invention.The method is performed with in image source 120 one of city.In the exemplary embodiment, step 1010,1020,1040 and 1060 is performed, as herein described by other place.In classified image step 1110, just processed image is based on deriving the one or more image descriptor and mating between the image descriptor on that was previously stored in image source 120 that are exported in descriptor step 1040.The classification or the multiple classification that are assigned to image are the classification or multiple classification that are associated with the image descriptor of previous stored coupling.

In forwarding step 1120, image is sent to image processing system 110 with the classification or multiple classification being assigned to this image.This image is that processed to produce the image tag being assigned to image as herein described by other place.This process comprises use classes or multiple classification alternatively to select mankind's image audit or auxiliary automatically label image.

In reception labelling step 1130, the label being assigned to image by perform at this place receive image step 1010 image source 120 in one receive.Label is present in subsequently to be provided in result step 455.

Figure 12 illustrates the method for use feedback processing image according to various embodiments of the present invention.The method be performed on image source 120A alternatively and the multiple communication comprised between image source 120A and image processing system 110 so that improve the label of image.Thering is provided in image step 1210, image is provided to image processing system 110 from image source 120A.

In reception first response of step 1220, the first response is received from image processing system 110.This response can comprise one or more label.Thering is provided in feedback step 1230, the feedback about received image tag is provided to image processing system 110 from image source 120A.This feedback is manually inputted by the human user of image source 120A alternatively and can comprise upgrade request, as other place herein discussed.Feedback can comprise the correction of the one or more labels to received label.Such as, the instruction that a label that can comprise in label does not represent image is fed back.Feedback can comprise the classification of image.

Receive in the second response of step 1240 optional, the second response is received from image processing system 110.Second response is generally used in and provides the feedback provided in feedback step 1230 to be generated.In one example, consider the image of toy car, the first response comprises label " car ", and feedback comprises term " toy " and the second response comprises label " the expense super lorry of snow (Fisher-PriceSuperwagon) ".Method shown in Figure 12 is used to the accuracy improving image tag alternatively.

Figure 13 and Figure 14 illustrates the method providing image tag based on image descriptor according to various embodiments of the present invention.In fig. 13, image descriptor is used to generate the image tag in the source being transferred into image descriptor subsequently on computational grid.In fig. 14, image descriptor is used to determine the destination 125 for image.Method is in figs. 13 and 14 performed alternatively with herein other local methods combining illustrated.Such as, these methods step can with in the diagram shown in those combine.

Especially, with reference to Figure 13, in reception descriptor step 1310, one or more image descriptors of characterized images are received at image processing system 110 place.These image descriptors are received alternatively but do not have related image.Only receive descriptor usually than receiving the less bandwidth of image request.Image descriptor to be received from image source 120A via network 115 alternatively and to use the method shown in Figure 10 or 11 to be generated.

In relatively descriptor step 1320, the image descriptor received is compared with the one or more image descriptors previously storing (such as, being stored in storer 135) at image processing system 110 place.This compare made determining any in the descriptor that receives whether mate the descriptor that stores.The descriptor stored and one or more image tag and/or classify is stored explicitly.Such as, a set of the descriptor stored can be associated with image tag " Oak Tree ".

In acquisition labelling step 1330, be received in response to the one or more image tag that mates between accepted descriptor and stored descriptor.The image tag obtained is those that be associated with mated set.

Thering is provided in labelling step 1340, the image tag obtained is provided go back to the source of the descriptor received, such as, to image source 120A.They can be presented to user or being processed otherwise as herein described by other place at that.

The method that the data that Figure 14 illustrates image and this image of characterization are processed at image processing server 110 place.In reception image and data step 1410, the data of image and this image of characterization are received at image processing server 110 place.The data of characterized images such as can comprise the image descriptor of characterized images or the classification of image.Image and characterization data are received from image source 125A alternatively.Reception image and data step 1410 are the embodiment receiving image step 410 and receive source data step 420 alternatively.

Determining in the step 1420 of destination, the data for the destination feature based image of image are determined.Destination can be one in destination 125 and/or automatic recognition system 152.Such as, if the data of characterized images comprise specific classification and determined destination can be and one that has in the destination 125 that is associated at mankind's image audit of the speciality of this classification.Determine that destination step 1420 is determine the embodiment of destination step 465 alternatively.

Putting up in image step 1430, image and alternatively classification are transferred into determined destination.In reception labelling step 1440, one or more image tag is received.Image tag is based on image and selected with characterized images.Thering is provided in labelling step 1340, image tag is provided to the source of image, such as, and image source 125A.Putting up image step 1430 is the embodiment of putting up image step 470 alternatively.

Figure 15 illustrates the method sorting by priority image tag interpolation according to various embodiments of the present invention.In these methods, image grading device 190 is used to image assigned priority and this priority is used to determine how image is added label, if it will be added label.In reception image step 410, image is received at image processing system 110, as other place herein discussed.Image can come from one in image source 120, and can receive by crawling (crawling) webpage for image.In certain embodiments, one or more in image source 120 comprise and are configured to crawl website and obtain the logic of image from these websites.The information be received together with image can comprise about from the data of webpage wherein obtaining image.Such as, image can with can be received together with other data any of determining according to it from the many data of (viewing), the URL of webpage and/or the picture priority of being often loaded of metadata and the text of webpage, instruction webpage, as other position herein discussed.

In assigned priority step 1520, image grading device 190 is used to automatically priority is assigned to received image.Priority alternatively by from 1 to 100 numerical value, by captions grade or similar expression.Priority has implied (order) classification of image alternatively.As herein described by other place, priority can be determined based on various factors.

Determining in treatment step 1530, the method for image being added to label (process) is determined.Determine the priority of assigning based on image.In certain embodiments, there is the image not processed (interpolation label) of lowest priority.The method that label adds comprises automatic powder adding and tags and/or manually add label by human reviewer, as other place herein discussed.

Optionally automatically tagging in step 1540, image uses automatic recognition system 152 to be added label.Automatically the method that the step 1540 that tags is added at the label determining to determine in treatment step 1530 alternatively does not comprise in the embodiment of the use of automatic recognition system 152.Automatically the step 1540 that tags is performed prior to assigned priority step 1520 alternatively.Such as, image can use automatic recognition system 152 by label, and dares to the level of confidence of the label automatically generated and can be used in subsequently in assigned priority step 1520 and add tagged priority to determine manually (mankind).If the degree of confidence of the label automatically generated is high, so can be set lower for manually adding tagged priority, and if degree of confidence is relatively low, so can be set to relatively high for manually adding tagged priority.

Optionally manually tagging in step 1550, image one of being sent in destination 125 adds label for by human reviewer.Image can be sent out, as herein described by other place together with the label using automatic recognition system 152 and/or various out of Memory to generate.The step 1550 that manually tags can comprise any step in the step shown in Fig. 6 to Fig. 8.

In optional auction labelling step 1560, advertisement is assigned to image and shows on webpage.This webpage is that image is receiving in image step 410 from its obtained webpage alternatively.Along with the request for webpage is received, auction labelling step 1560 is perfomed substantially in real time with said storing the sensor signals alternatively.In this moment, (multiple) label being assigned to image can by auction to being ready to be provided for advertisement to be placed in above this image or the side of maximum remuneration on side.Auction labelling step 1560 uses ad system 180 to be performed alternatively and auction process can by such as Google and so on third party managed.

Optionally again tagging in step 1570, image is added label again.Again the step 1570 that tags can comprise (multiple) advertisement many often clicked analyses compared with the clicking rate of expectation being assigned to image.Such as, if the advertisement being assigned to image based on the first label is not clicked with the clicking rate of expection, so label may not be that the optimum of this image represents.This image can be added label again to attempt improving the clicking rate being added the advertisement of label.Again the step 1570 that tags can comprise any one of label addition method disclosed herein, such as, about those methods that Fig. 6-8 and 15 discusses.Again the step 1570 that tags can use the label produced from the first label not to be optimum knowledge.

Method as shown in Figure 15 can also be applied to image sequence, such as, and video.Image sequence can be presented in a browser or use various alternative application.Such as, video can from the website of such as youtube.com and so on or the member being provided to image source 120 from the stream service of such as Netflix, Comcast CATV (cable television), live telecast, Ruku or Hulu and so on.Be used to determine whether the image in video should be comprised by manually adding tagged factor: video is many often watched and/or the estimated value of label expected.The label expected can use the dialogue in automatic recognition system 152, video, with the text (such as, describe, captions or title) of video and/or analog indicated by the automatic examination & verification of image.Advertisement can be image sequence beginning or terminate to be added or video engaged in image sequence.Thus, advertisement and video can explicitly by together with present.Advertisement can comprise the covering in the part being placed in image sequence, the part normally comprising added tagged image.

Multiple embodiment specifically illustrates in this article and/or describes.But, be that modifications and variations are covered and the scope not departing from its spirit and be intended within the scope of claims by above training centre by what understand.Such as, image disclosed herein is the part of the video sequence of video alternatively.Mankind's image audit can use audio frequency input to provide image tag at destination 125 place.Audio frequency input can use the audio frequency be placed on destination 125 to be converted into text in real time to text-converted logic and/or image processing system 110.Image tag is alternatively handled by spell check logic.As used herein, term " in real time " means does not have non-essential time delay, user can easily be waited to be done.System and method described herein is used to add label, such as music or dialogue to audio content alternatively.This audio content can be the part of video or otherwise be associated with image.In certain embodiments, audio content be automatically converted to text and the text be used to auxiliary hand-operating ground or automatically to image add label.The text generated from audio content can be added label to assist to image to use to those the similar modes herein described by the text that finds the webpage comprising image.

Embodiment discussed in this article describes the present invention.Because these embodiments of the present invention are described with reference to illustrating, various amendment or the adaptation of described method and/or specific structure can become apparent to those skilled in the art.Rely on instruction of the present invention and advance these amendments all of this area by its these instruction, within adaptive or change considered to be in the spirit and scope of the present invention.Thus, these descriptions and accompanying drawing should not be considered to the meaning limited, because should be understood that the present invention is never only limitted to illustrated embodiment.

The computing system related to herein (such as, image processing system 110, image source 120 and destination 125) integrated circuit, microprocessor, personal computer, server, distributed computing system, communication facilities, the network equipment or similar devices can be comprised, and above various combinations.Computing system can also comprise volatile row and/or non-volatile line storage, such as random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), magnetic medium, light medium, nanometer medium, hard disk, compact disk, digital versatile disc (DVD) and/or be arranged to the miscellaneous equipment such as storing analog or digital information in a database.The various examples of the logic of above indication can comprise hardware, firmware or the software stored on a computer-readable medium or its combination.Computer-readable medium as used herein eliminates paper clearly.The step that the computing machine of the method pointed out herein performs can comprise storage one group of instruction on a computer-readable medium, makes computing system perform this step upon being performed.Be programmed with the specific use computing system according to the computing system of the instruction execution specific function from program software being for performing those specific functions.At least electrically being kept in the impact damper of computing system when performing those specific functions by the data that specific use computing system operates, utilizing and from a state, next state physically being changed to specific use computing system to the change of stored data at every turn.The logic discussed herein can comprise storage hardware on a computer-readable medium, firmware and/or software.This logic can be performed to produce specific use computing system in the electronic device.

Claims

1. process a method for image, described method comprises:

Receive one or more first descriptors of image from Terminal Server Client via communication network at image processing server place;

The first received descriptor is compared with the second descriptor be stored in the addressable position of described image processing server, to determine whether described first descriptor mates the set of described second descriptor;

In response to the set of the second descriptor described in described first descriptors match, obtain the one or more image tags stored explicitly with the set of described second descriptor; And

Described one or more image tag is provided to described client.

2. method according to claim 1, wherein said client comprises mobile device, and described first descriptor and described image are generated on described mobile device.

3. method according to claim 1, is included in the described image of described image processing server place reception from described Terminal Server Client further.

4. method according to claim 3, comprises further and described image is sent to long-range destination from described image processing server, and described long-range destination is associated with mankind's image audit person.

5. method according to claim 4, comprises further and receives one or more image tag from described mankind's image audit person.

6. method according to claim 1, comprises further and computer instruction is provided to described Terminal Server Client, and described computations is configured to generate described one or more first descriptor.

7. manage a method for image everywhere at image processing server, described method comprises:

Receive the data of image described in image and characterization from Terminal Server Client, wherein the described data of image described in characterization comprise characteristics of image or image descriptor;

Determine the destination for described image, described destination is associated with mankind's image audit person, and the described of described destination determines the described data of image described in feature based and the speciality of described human reviewer;

Described image is puted up to determined destination; And

From one or more image tags of image described in the receive feature of described destination.

8. method according to claim 7, comprises further and described one or more image tag is provided to described client.

9. method according to claim 7, comprises further and uses described one or more image tag to select advertisement.

10. method according to claim 7, wherein the described data of image described in characterization comprise characteristics of image.

11. methods according to claim 7, wherein the described data of image described in characterization comprise the image descriptor of deriving from described image.

12. 1 kinds of methods processing image, described method comprises:

In the data of image processing server place receive feature image, the described data of image described in characterization are from deriving in the process of mobile device to image and comprising the feature identified of described image or the descriptor of described image;

The described data genaration image tag of image described in feature based;

Described image tag is provided to described mobile device.

13. methods according to claim 12, wherein the described data of image described in characterization comprise the characteristics of image of described image and the described step of synthetic image label comprises the descriptor generating described image.

14. methods according to claim 12, wherein the described data of image described in characterization comprise one or more descriptors of described image.

15. 1 kinds of image processing systems, comprising:

I/O, is configured to the data transmitting characterized images on a communication network;

Automatic recognition interface, be configured to the described data of characterized images are sent to automatic recognition system and receive the examination & verification of the Practical computer teaching of described image from described automatic recognition system, the examination & verification of described Practical computer teaching comprises one or more image tags of the content of image described in characterization;

Storer, is configured to store described image; And

Microprocessor, is configured at least perform described automatic recognition interface.

16. systems according to claim 15, wherein the described data of image described in characterization comprise characteristics of image, and described automatic recognition system is configured to use described characteristics of image to generate described one or more image tag.

17. systems according to claim 15, wherein the described data of image described in characterization comprise image descriptor, and described automatic recognition system is configured to use described image descriptor to generate described one or more image tag.

18. systems according to claim 15, wherein said automatic recognition system is configured to generate described one or more image tag and do not receive described image.

19. systems according to claim 15, wherein said I/O is further configured to and receives described image on described communication network; And comprise destination logic further, described destination logic is configured to the destination determining to send the image to, described destination is a destination in multiple destination, and each destination is associated from different mankind's image audit persons; And comprise further and put up logic, described in put up logic and be configured to described image to put up to described destination.

20. systems according to claim 19, wherein determine described destination at least in part based on described one or more image tag; And the examination & verification of the described Practical computer teaching of wherein said image comprises the tolerance of the degree of confidence of examination & verification in accuracy of described Practical computer teaching.

21. systems according to claim 15, comprise ad system further, and described ad system is configured to based on the advertisement of described one or more image tag selections for showing together with described image.