CN105022773B - Image processing system including picture priority - Google Patents

Abstract

Image identification method uses the image tag that characterization image is generated with manually both image audits that computer generates.What the computer generated can sequentially or in parallel be performed with manually image audit.Image tag generated can be provided in real time requestor, be used to selection advertisement, and/or be used as the basis of internet hunt.In some embodiments, image tag generated is used as the basis of upgraded image audit.The confidence level for the image audit that computer generates may be used to determine whether to execute manual image audit.

Description

Image processing system including picture priority

Cross reference to related applications

The application is the beauty for entitled " image procossing " with application number 14/267,840 submitted on May 1st, 2014 The part of state's non-provisional application continues, and then requires the preferential of the provisional application 61/956,927 submitted on May 1st, 2013 Power；The priority and right of the following U.S. Provisional Patent Application of the application further requirement:

It is submitted on April 4th, 2014 and there is " visual search " of application number 61/975,691；

It is submitted on April 7th, 2014 and there is " the visual search advertisement " of application number 61/976,494；

It is submitted on May 1st, 2014 and there is " image procossing " of application number 61/987,156；

It is submitted on July 31st, 2014 and there is " the real-time target choosing in image procossing of application number 62/031,397 It selects "；

It is submitted on October 27th, 2014 and there is " the distributed image processing " of application number 62/069,160；And

It is submitted on November 25th, 2014 and there is " the selective image processing " of application number 62/084,509.

All above patent applications are incorporated herein by reference.

Technical field

The application is in the field of image procossing, more specifically, the field in the content for characterizing image.

Background technique

Information is usually extracted from text data from extraction information ratio in image is more difficult.However, most information is being schemed It is found as in.The reliability height of automatic image distinguishing system depends on the content of image.For example, optical character recognition OCR ratio Facial recognition is more reliable.The target of image identification is to add label to image.Label refers to characterizing the mark of the content of image Sign the identification of (word).For example, the image of automobile can be added word " automobile ", " Ford Granada (Ford ) " or the label of the white Ford Granada in 1976 of headlight " have damage " Granada.These labels include different number Information, and can thus change in purposes.

Summary of the invention

Embodiments herein includes adding the method for image tag worked along both lines.First method is to hold on the image The automatic image identification of row.The automatic image identification leads to the audit (review) of image.The image audit includes identification figure One or more labels of the content of picture and optionally further comprising indicate automated graphics identification reliability confidence level degree Amount.This adds the manual addition label that second method in tagged method includes image to image.Addition label packet manually People is included to watch each image, consider the content of image and the label for indicating picture material is manually provided.Automatic image is distinguished Know on the time analyze with each image or pecuniary cost can be relatively low.The manual label of image adds tool There is the advantages of higher accuracy and reliability.

The embodiment of the present invention combines both automated graphics identification and manual image identification.In some embodiments, certainly Motion video identification is performed first.The image audit of generation usually both include characterize image one or more labels and also including The measurement of confidence level of these labels in accuracy.If confidence level is higher than predetermined threshold, these labels and the image It is associated with and is provided as the output of addition label process.If confidence level is lower than predetermined threshold, the manual of the image is examined Core is performed.Audit leads to the additional and/or different label for the content for characterizing image manually.In some embodiments, automatically Image identification and image it is manual audit be executed in parallel.If automated graphics identification causes one or more labels to have Confidence level higher than predetermined threshold then audits manually and is optionally cancelled or terminated.

In some embodiments, the identification of image can be upgraded.The upgrading of image identification process includes for expression figure The requirement of the further or improved label of the content of picture.For example, if automated graphics identification leads to label " white vehicle ", the identification Upgrading can lead to label " white Ford Granada ".In some embodiments, the audit of upgrading is examined using the expert mankind Core person.For example, above example may include the human reviewer using the professional knowledge with automobile.The profession of human reviewer The other examples of knowledge are discussed in elsewhere herein.

Various embodiments of the present invention include that guiding improves the accuracy of image identification while also minimizing the feature of cost. By way of example, these features include the delivered in real-time, and/or image of the effective use of population auditor, image tag The seamless upgrade of identification.The method of image identification disclosed herein be optionally used to generate be adapted for carrying out internet hunt and/ Or the image tag of selection advertisement.For example, in some embodiments, image tag be automatically used to execute Google search and/ Or AdWords based on Google and sell advertisement.

Various embodiments of the present invention include image processing system, which includes being configured as in communication network The I/O of image and image tag is transmitted on network；Automatic identification interface, be configured as by image to automatic recognition system transmission and The audit that the computer of the image from automatic recognition system generates is received, the audit which generates includes identification image One or more labels of content；Destination logic is configured to determine that the first destination sent images to, with for by First mankind auditor carries out first manual audit to the image；Image puts up logic, is configured as image being posted to purpose Ground；Logic is audited, is configured as receiving the audit of the image from destination audited manually and receive computer generation, it should Audit includes one or more image tags of the content of identification image manually；Response logic is configured as mentioning to communication network The image tag of the audit generated for computer and the manually image tag audited；Memory is configured as storage image；With And microprocessor, it is configured as at least executing destination logic.

Various embodiments of the present invention include the method for handling image, and this method includes receiving the image from image source； Image is distributed to automated graphics identifying system；Receive the audit that the computer from automated graphics identifying system generates, the calculating The audit that machine generates includes the one or more image tags and confidence level that the image is assigned to by automated graphics identifying system Measurement, the image tag that the measurement of the confidence level is assigned to image correctly characterize the degree of the confidence level of the content of image Amount；Image is placed in image queue；Determine destination；The image for being used to audit manually is posted to the first destination, this One destination includes the display equipment of mankind's image censor；And the manual image audit of the image from destination is received, The image audit includes one or more image tags that image is assigned to by mankind's image censor, the one or more image The content of label characteristics image.

The various embodiments of the application include image source comprising are configured as the camera of acquisition image；Display is matched It is set to user and image is presented；Eye trace logic, is configured as the movement of one or more eyes of detection user；Optional figure It as mark logic, is configured as mark and is placed on image, which is configured as indicating specific subset and the response of image In movement detected；Display logic is configured as in real time showing label on the image；I/O is configured to supply image To computer network；And processor, it is configured as at least executing display logic.

The various embodiments of the application include image source comprising are configured as the camera of acquisition image；Display is matched It is set to user and image is presented；Eye trace logic, is configured as the movement of one or more eyes of detection user；Image tagged Logic is configured such that user indicates the object in the specific subset of image and the prominent subset, and the instruction is in response to being detected Movement；Display logic, be configured as in real time by institute it is outstanding display on the image；I/O is configured as to computer network The instruction of image and specific subset is provided；And processor, it is configured as at least executing display logic.

Various embodiments of the present invention include image source comprising are configured as the camera of acquisition image；Display is matched It is set to user and image is presented；Logic is selected, is configured for selecting；Image tagged logic is configured such that user indicates image Specific subset and the prominent subset in object, the instruction is in response to finger detected；I/O is configured as to calculating The instruction of machine network offer image and specific subset；Display logic is configured as displaying in real time image and in response to image Display characterizes the content of image from the received image tag of computer network, the image tag；And processor, it is configured as Execute at least display logic.

Various embodiments of the present invention include image processing system, which includes being configured as in communication network The I/O of image sequence and image tag is transmitted on network；Optional automatic identification interface is configured as knowing image sequence to automatic Other system transmits and receives the audit that the computer of the image from automatic recognition system generates, the audit which generates One or more labels including identifying the content of image；Destination logic is configured to determine that and is sent to image sequence First destination, for carrying out first manual audit to the image sequence by the first human reviewer；Image puts up logic, quilt It is configured to image sequence being posted to destination；Logic is audited, is configured as receiving the manual of the image sequence from destination Audit and optionally receive the audit that computer generates, the manual audit include one of movement in identification image sequence or Multiple images label；Response logic is configured as providing the image tag and hand of the audit that computer generates to communication network The image tag of dynamic audit；Memory is configured as storage image sequence；And microprocessor, it is configured as at least executing mesh Ground logic.

Various embodiments of the present invention include handle image method, this method comprises: image processing server via The first descriptor of one or more of image of the communication network reception from Terminal Server Client；By received first descriptor of institute with The second descriptor for being locally stored in image processing server compares to determine whether first the second descriptor of descriptors match Set；Match in response to the set of the first descriptor and the second descriptor, obtain it is associated with the set of the second descriptor and One or more image tags of storage；And one or more image tags are provided to client.

Various embodiments of the present invention include the method in image processing server processing image, this method comprises: receiving Image from Terminal Server Client and the data for characterizing the image；Determine the destination for being directed to image, the destination and the mankind Image audit person is associated, and the determination of the destination is based between the speciality of the data and human reviewer that characterize image Match；Image is posted to identified destination；Receive one or more image tags of the characterization image from destination； And one or more image tags are provided to client.

Each embodiment of the invention includes the method for handling image, this method comprises: receiving the spy from mobile device The data of signization image, the data characterization include the image of the feature of identified image or image descriptor；It is based on The data for characterizing image generate image tag；Image tag is provided to mobile device.

Various embodiments of the present invention include the method for handling image, this method comprises: receiving image using portable equipment； Use the feature of the processor identification image of portable equipment；Feature is provided to remote image processing server via communication network； Receive the image tag based on feature from image processing server；And image mark is shown on the display of a portable device Label.

Various embodiments of the present invention include the method for handling image, this method comprises: receiving image using portable equipment； Use the feature of the processor identification image of portable equipment；Based on known another characteristic deduced image descriptor；Via communication network Network provides descriptor to remote image processing server；Receive the image mark based on descriptor from image processing server Label；And image tag is shown on the display of a portable device.

Various embodiments of the present invention include the method for handling image, this method comprises: receiving image using portable equipment； Use the feature of the processor identification image of portable equipment；Based on known another characteristic deduced image descriptor；By iamge description Accord with the set of image descriptor being previously stored on portable equipment compare with determine whether image descriptor with stored Image descriptor set between exist matching；If between image descriptor and the set of the image descriptor stored One or more image tags associated with the set of image descriptor then are obtained from the memory of portable equipment in the presence of matching； Acquired one or more graphical labels are shown on the display of a portable device.

Various embodiments of the present invention include the method for handling image, this method comprises: receiving image using portable equipment； Use the feature of the processor identification image of portable equipment；Based on known another characteristic deduced image descriptor；By iamge description Accord with the set of image descriptor being previously stored on portable equipment compare with determine whether image descriptor with stored Image descriptor set between exist matching；Based between image descriptor and the set of the image descriptor stored Matching classify image；The classification of image and image is sent to remote image processing server；Receive one based on image or Multiple images label；And one or more image tags are shown on the display of a portable device.

Various embodiments of the present invention include image processing system comprising are configured as transmitting image on a communication network With the I/O of image tag；Image grading device is configured to determine that for adding tagged priority to image；Destination logic, It is configured to determine that the first destination sent images to for being examined by first manual of first human reviewer to image Core；Image puts up logic, is configured as image being posted to destination；Logic is audited, is configured as receiving from destination The manual audit of image, the manual audit include one or more image tags of the content of identification image；Memory is configured For one or more image tags are stored in data structure；And microprocessor, it is configured as at least executing image grading Device.

Various embodiments of the present invention include image processing system comprising are configured as receiving image on a communication network I/O；Image grading device is configured to determine that the priority of image and determines whether to add image based on the priority and marked It signs and/or how label is added to image；For adding label to image to generate the one or more images for characterizing image The manually or automatically device of label；Memory is configured as one that image and characterization image are stored in data structure Or multiple images label；And microprocessor, it is configured as at least executing image grading device.

Various embodiments of the present invention include image processing system comprising are configured as receiving image on a communication network I/O；Image grading device is configured to determine that the priority of image and is added based on priority selection to image tagged Process；For adding label to image to generate the device for the one or more image tags for characterizing image；Memory is matched It is set to one or more image tags that image and characterization image are stored in data structure；And it is configured as at least holding The microprocessor of row image grading device.

Various embodiments of the present invention include image processing system comprising are configured as transmitting image on a communication network With the I/O of image tag；Image grading device, be configured to determine that for based on include image video by viewing how many times come pair Image adds tagged priority；Destination logic is configured to determine that the destination sent images to for by the mankind Manual audit of the auditor to image；Image puts up logic, is configured as image being posted to destination；Logic is audited, is matched It is set to the manual audit for receiving the image from destination, which includes one or more figures of the content of identification image As label；Memory is configured as one or more image tags being stored in data structure；And microprocessor, matched It is set at least execution image grading device.

Various embodiments of the present invention include the method for handling image, and this method includes receiving the image from image source； Image is distributed to automated graphics identifying system；Receive the audit that the computer from automated graphics identifying system generates, the calculating The audit that machine generates includes the one or more image tags and confidence level that the image is assigned to by automated graphics identifying system Measurement, the image tag that the measurement of the confidence level is assigned to image correctly characterize the degree of the confidence level of the content of image Amount；Priority is assigned to image by the measurement based on confidence level；Determine that image should be by manual label based on priority；It will be used for The image audited manually is posted to the first destination, which includes the display equipment of mankind's image censor；And The manual image audit of the image from destination is received, which includes being assigned to image by mankind image audit person One or more image tags, the one or more image tag assigned by mankind image audit person characterize in image Hold.

Various embodiments of the present invention include the method for handling image, and this method includes receiving the image from image source； The priority of image is automatically determined using microprocessor；Determine how image should be added label based on priority；To figure Picture addition label is to generate one or more labels, the content of the one or more label characteristics image；And by the image It is stored in data structure with the one or more label.

Detailed description of the invention

Fig. 1 illustrates image processing systems according to various embodiments of the present invention.

Fig. 2 illustrates Image Acquisition screen according to various embodiments of the present invention.

Fig. 3 illustrates the search result based on image analysis according to various embodiments of the present invention.

The method that Fig. 4 illustrates processing image according to various embodiments of the present invention.

Fig. 5 illustrates the alternative of processing image according to various embodiments of the present invention.

The method that Fig. 6 illustrates management auditor pond according to various embodiments of the present invention.

Fig. 7 illustrates the method for receiving image tag in real time according to various embodiments of the present invention.

The method that Fig. 8 illustrates the audit of upgrade image according to various embodiments of the present invention.

Fig. 9 illustrates the example of the image source 120A including electronic glasses according to various embodiments of the present invention.

The method that Figure 10 illustrates the image in processing image source according to various embodiments of the present invention.

Figure 11 illustrates the method based on image descriptor processing image according to various embodiments of the present invention.

Figure 12 illustrates the method using feedback processing image according to various embodiments of the present invention.

Figure 13 and 14 illustrates the side based on image descriptor offer image tag according to various embodiments of the present invention Method.

Figure 15 illustrates the method for sorting by priority image tag according to various embodiments of the present invention.

Specific embodiment

Fig. 1 illustrates image processing systems 110 according to various embodiments of the present invention.Image processing system 110 is matched It sets for adding label to image and may include one or more distributive computing facilities.For example, image processing system 110 It may include one or more servers positioned at geographically different location.Image processing system 110 is configured as via network 115 are communicated.Network 115 may include a variety of communication networks, such as internet and/or cell phone system.Network 115 It is typically configured as transmitting data using standard agreements such as IP/TCP, FTP.The image handled by image processing system 110 It is received from image source 120 (separate marking 120A, 120B etc.).Image source 120 may include be connected to internet and/or The calculating source of Personal mobile computing equipment.For example, image source 120A, which can be, is configured to supply social network sites or picture sharing The network server of service.Image source 120B can be smart phone, camera, wearable camera, electronic glasses or other portable Image capture device.Image source can be by uniform resource locator, Internet protocol address, MAC Address, cell phone identifier And/or analog is identified.In some embodiments, image processing system 110 is configured as receiving from great amount of images source 120 Image.

The part of the image tag executed from image processing system 110 include to destination 125 (be respectively labeled as 125A, 125B etc.) send image.Destination 125 is at the calculating equipment of mankind image audit person and usual geographically-distant image Reason system 110.Destination 125 includes at least display and data input device, such as touch screen, keyboard and/or microphone.Example Such as, destination 125 can be the building different from image processing system 110, city, state and/or country.Destination 125 can To include personal computer, tablet computer, smart phone etc..In some embodiments, destination 125 includes being specifically configured to Promote (calculating) application of the audit of image.The application is optionally provided to destination 125 from image processing system 110.? In some embodiments, image processing system 110 is configured for mankind's image audit person from 125 login user account of destination. Destination 125 is usually associated with individual auditor and can be interconnected fidonetFido address, MAC Address, login sessions mark Symbol, cell phone identifier and/or analog are identified.In some embodiments, destination 125 includes audio to text conversion Device.Image processing system 110 is sent in the image tag data that several destinations 125 provide by mankind's image audit person. Image tag data may include text image label, the audio data including language label, and/or such as upgrade request or The non-label information of inappropriate (specific) material mark etc.

Image processing system 110 includes being configured for and the I/O of External system communication (input/output) 130.I/O 130 include router, interchanger, modem, firewall, and/or analog.I/O 130 is configured as receiving from figure The image of image source 120 sends image to destination 125, receives the label data from destination 125, and optionally to figure Image source 120 sends image tag.I/O 130 includes communication hardware and optionally includes application programming interfaces (API).

Image processing system 110 further comprises memory 135.Memory 135 includes being configured for such as image, figure The hardware stored as the non-transitory of the data of label, computations and other data discussed in this article etc.For example, depositing Reservoir 135 may include random access memory (RAM), hard disk drive, light-memory medium and/or analog.Memory 135 It is configured as by using specific data structure, index, file structure, data access routines, security protocol and/or analog And store specific data (as described herein).

Image processing system 110 further comprises at least one processor 140.Processor 140 is such as electronics micro process The hardware device of device etc.Processor 140 is configured as being loaded into posting for processor 140 by hardware, firmware or by software instruction Storage and execute specific function.Image processing system 110 optionally includes multiple processors 140.Processor 140 is configured as holding Various types of logics that row is discussed herein.

It is initially stored in image queue 145 by the received image of image processing system 110.Image queue 145 is to deposit Store up the ordered list of the pending image in storage list.The image being stored in image queue 145 is usually and for drawing It is stored in association with the image identifier of image and can have different priority.For example, from picture sharing website Received image can have than from the lower priority of the received image of smart phone.In general, relative to some other Those of the image tag that purpose uses image, for requestor's image tag etc. to be received for indicating image in real time Those images are given a higher priority.Image queue 145 is optionally stored in memory 135.

In image queue 145, image by optionally with image identifier or index and other be associated with each image Data store in association.For example, image can be associated with about one source data in image source 120.Source data It may include the geography information of such as GPS coordinates, street and/or city name, postcode, and/or analog.Source number According to may include Internet protocol address, uniform resource locator, account name, smart phone identifier, and/or analog. Source data may further include the priority about the language, request that use on the member of image source 120, searching request The information of (for example, request that internet hunt is carried out based on the image tag generated from image), and/or analog.

In some embodiments, the image within image queue 145 is associated by the instruction of the specific subset with image Ground storage, the subset generally include the project of special interests.For example, the requestor of image tag can be to acquisition about image The image tag of the content of specific subset is interested.When image includes several objects, this can occur.In order to illustrate consideration exists There is the image of the hand of ring on one finger, user may want to identify the ring to be interested specific region.Of the invention Some embodiments include being configured such that the object on display of the user by clicking object or touch image source 120B is specified The application of interested specific project.This, which is specified, usually occurs before sending image to image processing system 110.

If image with the specific subset of image there is the instruction of particular importance to store in association, image mark Note logic 147, which is optionally used to mark, to be placed on image.Label is arranged to protrude specific subset.The label can lead to It crosses modification image to correspond to the pixel of subset and make, and the label allows mankind's image audit person to be absorbed in the son marked On collection.For example, being posted in image to before one or more destinations 125, image can be labeled with rectangle or circle.Example Such as, the subset of prominent image or the object within image may include to object or subset application filter (filter, filter), And/or change the color of object or subset.In alternative embodiments, image tagged logic 147, which is included in, is configured to one Within the application executed on a or multiple images source 120 or destination 125.Image tagged logic 147 includes being stored in nonvolatile Hardware, firmware, and/or software on property computer-readable medium.As being discussed elsewhere herein, mark logic 147 are optionally configured to be generated with image and be arranged in label on image in real time.

In some embodiments, mark logic 147 is configured with detected characteristics of image within image to know The special object that can be labeled.The detection of characteristics of image is discussed in elsewhere herein and is occurred in visitor A part of image procossing in the side of family (for example, on image source 120A).For example, the feature at such as edge etc can be used The processor of image source 120A is detected.These features can be used in the object of prominent detection and first then also from figure Image source 120A is sent to image processing system 110, they are subsequently used to generate the figure of the part as processing image at this As descriptor.In the method, automatically processing for image is dispensed on image source 120A, image processing system 110 and/or automatic Between identifying system 152.

Under the control of processor 140, the image in image queue 145 is provided to automatic identification interface 150.Image Thus it is provided as the function of their priority and position in image queue 145.Automatic identification interface 150 includes being matched It is set to image, and optionally by the logic of any data transmission associated with the image to automatic recognition system 152.It should Logic is stored in hardware, firmware, and/or software on computer-readable medium.Automatic identification interface 150 may further be by It is configured to receive the audit that the computer of the image from automatic recognition system 152 generates, the audit which generates includes Identify one or more image tags of the content of image.In some embodiments, automatic identification interface 150 be configured as via Network 115 transmits image and data with the format for being suitable for the application programming interfaces (API) of automatic recognition system 152.Some In embodiment, automatic recognition system 152 is included in image processing system 110, and automatic identification interface 150 includes for example System in operating system or on local area network is called.

Automatic recognition system 152 is configured as audit image without the calculating being manually entered based on each picture Machine automatic system.The output of automatic recognition system 152 is the image audit of computer generation (for example, being not needed upon each figure The audit that generation is manually entered of piece).The preliminary example of this system is well known in the art.For example, with reference to Kooaba, Clarifai, AlchemyAPI and Catchoom.Automatic recognition system 152 is typically configured as based in image The shape, character and/or the pattern that detect automatically identify the object in two dimensional image.Automatic recognition system 152 optionally by It is configured to execute optical character identification and/or bar code paraphrase.In some embodiments, automatic recognition system 152 and the prior art System the difference is that automatic recognition system 152 be configured to supply computer generation audit, the audit be based on it is (more It is a) image subset indicates and/or image source data, and other places herein are discussed.

Whether automatic recognition system 152 is optionally configured to the determining copy from the different received images of image source Through being added label.For example, identical image can be included in multiple webpages.If image is from first in these webpages A webpage is extracted and is added label, and automatic recognition system 152 can recognize the image and be added label and to figure The example that each of picture is found automatically assigns these labels.Recognisable image has been added label and has optionally included and will scheme The data of picture, the part of image or expression image are compared with the database for the image for adding label before.Image can it It is preceding either automatically or manually to be added label.

In addition to one or more image tags, the audit generated by the computer that automatic recognition system 152 generates is optionally Including indicating that one or more image tags correctly identify the measurement of the confidence level of the confidence level of the content of image.For example, main If the audit that the computer of character or the image for the shape that may be readily recognized generates can have than by being abstracted or not knowing The bigger confidence metric of the audit that the computer of the image of shape composition generates.Different automated graphics identification systems can produce The raw different confidence levels for different types of image.Automatic identification interface 150 and automatic recognition system 152 are optionally In embodiment in the automatic identification executed by third party.

Image processing system 110 further comprises auditor pond 155 and the auditor for being configured as management auditor pond 155 Logic 157.Auditor pond 155 includes the pond (for example, group or set) of mankind's image audit person.Each mankind's image audit person It is usually associated with the different members of destination 125.For example, each member in the different members of destination 125 can be known It is logged in by different mankind image audit person operations or by the account of different mankind's image audit persons.Memory 135 is by optionally It is configured to storage auditor pond 155.In some embodiments, the mankind's image audit person's quilt being included in auditor pond 155 It is classified as " enlivening " and " inactive ".For purposes of this disclosure, active mankind image audit person is considered as currently Image tag is provided or has indicated that they are ready to provide the mankind image audit person of image tag with minimal time delay.Both it is wrapping It includes in the active also embodiment including sluggish mankind's image audit person, active auditor is provided image to audit. The quantity of active auditor can become appropriateness in real time in response to the requirement to image audit.For example, mankind's image audit The classification of person can be changed into actively based on the quantity for not auditing image in image queue 145 from inactive.It is sluggish to examine Core person is not yet active auditor, and the audit of image has been allowed to fail and/or indicate that they can not audit image.It does not live The auditor of jump can request to become active auditor.When needs are additional enlivens mankind image audit person, this has been made The sluggish auditor of kind request can be reclassified as active mankind image audit person.Which sluggish audit Person, which is reclassified, to be alternatively depended on auditor's score as the determination of active auditor and (begs in other places herein By).

Auditor's logic 157 is configured as management auditor pond 155.The management optionally includes mankind image audit person work For active or sluggish classification.Start to audit for example, auditor's logic 157 can be configured as monitoring mankind's image audit person The time of image, and if predetermined maximum audit time (herein referred as picture failure time), by mankind image audit person's Classify inactive from actively changing into.In another example, auditor's logic 157 can be configured as calculating for mankind's image The audit score of auditor.In some embodiments, audit fractional marks are examined by the image that specific mankind's image audit person executes Integrality, speed and/or the accuracy of core.The audit score can based on audit number and accidental test image be calculated or Change.For example, these test images can be placed in the image audited before by different mankind image audit persons In queue 145.The audit score can also be the function of monetary cost associated with mankind image audit person.Auditor's logic 157 include hardware, firmware, and/or the software being stored in non-transitory computer-readable medium.In some embodiments, it examines Core person score is manually determined by mankind coordinator.These mankind coordinators audit image and are assigned to by mankind image audit person The label of these images.The statistic sampling of the image of audit is optionally sent to coordinator and coordinator will be to the mark of image Label assign score.The score is optionally used in determining auditor's score.

In some embodiments, auditor's logic 157 is configured as monitoring the state of mankind image audit person in real time.Example Such as, auditor's logic 157, which can be configured as, monitors the individual word or keystroke inputted by the auditor in destination 125A Input.The monitoring can be used to determine which auditor actively audits image, which auditor just completes to scheme The audit of picture and/or which auditor do not provide label input also after several seconds or minute.Use the label list of audio frequency apparatus The input of word can also the person's of being reviewed logic 157 monitored.

In some embodiments, the member in auditor pond 155 and mankind's image audit person have speciality or special knowledge Profession is associated.For example, auditor can be the expert of automotive field and associated with the profession.Other professions may include skill Art, plant, animal, electronics, music, food medical professionalism, clothes, dress accessory, fine work etc..Such as other local institutes herein It discusses, the profession of auditor can be used to select the audit between initial manual period under review and/or during auditing upgrading Person.

Audit score associated with mankind image audit person and/or profession are not optionally reviewed person's logic 157 and are used to really It is settled that need additional which sluggish auditor when enlivening auditor to be changed to active.Auditor's logic 157 includes depositing Store up hardware, firmware, and/or software in non-transitory computer-readable medium.

Image processing system 110 further comprises destination logic 160.Destination logic 160 is configured to determine that will be to It sends image with one or more destinations (for example, destination 125) for auditing manually.Each destination 125 and examine Corresponding mankind's image audit person in core person pond 155 is associated.It is optionally based on by the determination that destination logic 160 is made in institute The characteristic of the mankind image audit person of determining destination.Destination can be calculating equipment, the intelligence of mankind image audit person Phone, tablet computer, personal computer etc..In some embodiments, destination is browser, and auditor is from the browsing Device logs in image processing system 110.In some embodiments, determine that destination includes a purpose in determining destination 125 MAC Address, Session ID, Internet protocol and/or the uniform resource locator on ground.Destination logic 160 includes being stored in Hardware, firmware and/or software in non-transitory computer-readable medium.

In general, destination logic 160 be configured to determine that with such as by auditor's logic 157 determination it is active rather than not The active associated destination 125 of mankind image audit person.Destination logic 160 is usually additionally configured to based on auditor's Audit score determines destination 125.For example, there is higher fractional for compared with the auditor with lower auditor's score Those of auditor can be selected for the audit of higher priority.Thus, the determination of the member of destination 125 can be based on Both auditor's score and image audit priority.

In some embodiments, destination logic 160 is configured as the movable reality of input based on associated auditor When monitoring to determine one or more members in destination 125.As other places herein are discussed, which can be with It is executed by auditor's logic 157, and may include by the detection of the individual word or keystroke of mankind image audit person input.? In some embodiments, destination logic 160, which is configured as having compared, knocks in figure on mankind's image audit person currently keyboard As the destination 125B of label, favor selection has just completed the destination 125A of the audit of image in mankind's image audit person.

In some embodiments, destination logic 160 is configured with via the received image of automatic recognition system 152 Label is to determine one or more members of destination 125.For example, if the image tag of " automobile " is via automatic identification interface 150 are received, then the information can be used to select and have mankind's image in automotive field profession in destination logic 160 The member of the associated destination 125 of auditor.

The value of image audit can also be considered in the selection for the destination audited manually.For example, the figure of high level As audit can lead to with the relatively high determination for auditing the associated destination of the mankind image audit person of score, while compared with The image audit of low value can lead to destination associated with having the relatively low audit mankind image audit person of score It determines.In some embodiments, for some image audits, destination logic 160 is configured as selecting between destination 125 In order to minimize the time for needing to audit image, such as minimum time until the image tag audited manually is provided to net Network 115.

Destination logic 160 is optionally configured to determine multiple destinations for single image.For example, the first destination Can be selected, and followed by upgrade request, the second destination can be determined.Upgrade request can come from and the first mesh The associated mankind's image audit person in ground or come from image source 120A.In some embodiments, destination logic 160 is configured For the multiple destinations of determination, image will be concurrently posted to multiple destination.For example, it is each from different mankind's images The associated two, three or more destination of auditor can be determined and same image is concurrently posted to institute There is determining destination.As used in the present context, " parallel " means that image is receiving audit from the first destination The second destination is at least posted to before any part.

In various embodiments, there are a variety of causes to allow two or more destinations by 160 institute of destination logic It determines.For example, the mankind image audit person for having particular professional can be requested to the request of the audit of upgrading.Show with reference to automobile Example, can lead to the upgrade request to more information by the image that label has " white vehicle " label first.Destination logic 160 can be by It is configured to then select and have in mankind's image audit person of automotive field profession (for example, " 1976 Ford lattice drawing can be provided The auditor of Na Da " label) associated destination.Upgrade request instruction image, which is subject to, further to be audited, for example, image Demand is further audited or can be benefited from further audit.Upgrade request can be by such as flag, order or data value Deng computing object represented by.

When the manual audit of image expends the too many time, it may be desired to which another example of the second destination occurs.In general, The label addition of image should occur within the period of distribution, and otherwise the audit is considered failing.The period of distribution is optional Ground is the function of the priority of image audit.Those audits for being intended to occur in real time can have compared lower priority audit and Say the shorter period.If the audit of image is failed, image processing system 110 is optionally configured to patrol to by destination It collects the associated additional mankind's image audit persons in destination that 160 determine and images is provided.

When the first human reviewer makes upgrade request, it may be desired to which another example of the second destination occurs.For example, leading The request of the upgrade audit of the label of " automobile " is caused to may be from providing the mankind image audit person of label " automobile ".Although the example It is to simplify, other examples may include the image of more esoteric theme, the integrated circuit such as encapsulated.

Image processing system 110 further comprises that image puts up logic 165, is configured as the figure that will be used to audit manually As being posted to the destination 125 determined by destination logic 160.It puts up and generally includes that image is sent to one via network 115 A or multiple destinations 125.In various embodiments, image puts up logic 165 and is further configured to provide to destination 125 Information associated with image.For example, image put up logic 165 can put up image and image subset instruction (for example, Subset identification), the image that is marked by image tagged logic 147, identify image source information (for example, herein otherly The source data just discussed), the priority of the audit of image, picture failure phase, location information associated with image and/or similar Object.As other places herein are discussed, source data may include uniform resource locator, global positioning coordinates, longitude Share account and/or analog with latitude, account, Internet protocol address, social account, picture.

In some embodiments, image is puted up logic 165 and is configured as in the about same time to more than one purpose Ground 125 provides the image for auditing manually.For example, image can be concurrently provided to destination 125A and destination 125B.For example, " parallel delivering " means that before being received back label information from both destination 125A and 125B, image is mentioned It is supplied to these destinations 125.

In some embodiments, image puts up logic 165 and is configured as receiving image tag from automatic recognition system 152 Forward direction one or more destination 125 provide for the image audited manually.Alternatively, in some embodiments, image Logic 165 is puted up to be configured as waiting before putting up image to one or more of destination 125 until for image The audit that computer generates is received from automatic recognition system 152.In these embodiments, computer generate audit (including One or more of image tag) be optionally also posted to destination 125 associated with image.

Image puts up logic 165 and is optionally configured to put up the identifier of image and image together.Image, which is puted up, patrols Collecting 165 includes hardware, firmware and/or the softwares being stored in non-transitory computer-readable medium.

Image processing system 110 further comprises the audit logic manually and automatically audited for being configured as management image 170.The management includes the progress of monitoring audit, receives the audit from automatic recognition system 152 and/or destination 125.It is connect The audit of receipts includes the image tag discussed such as other places herein.In some embodiments, 170 quilt of logic is audited The image for being configured to a destination of the measurement control of confidence level into destination 125 is puted up.The meter of confidence level Show the correct confidence level of one or more image tags being received.The one or more image tag can be known from automatic A destination in other system 152 and/or destination 125 is received.For example, in some embodiments, if by knowing automatically The confidence level of the image audit of other system 152 is greater than predetermined threshold, then audit logic 170 can determine the manual audit of image It is unnecessary.The predetermined threshold can be the value of image audit, image audit priority, available destination 125 Quality and quantity, and/or analog function.Audit logic 170 includes being stored in non-transitory computer-readable medium On hardware, firmware, and/or software.

In some embodiments, if image is concurrently sent to automatic recognition system 152 and is sent to destination One or more of 125, then having connecing for the audit from automatic recognition system 152 of the confidence level higher than predetermined threshold It receives and can lead to the one or more manual audits cancelled in destination 125 by audit logic 170.Similarly, if figure As being concurrently sent to multiple destinations 125, and image audit is connect from first destination in these destinations 125 It receives, is requested then audit logic 170 is optionally configured to cancel the audit for image at other destinations 125.? In some embodiments, once audit logic 170 is configured as keystroke or word and is connect from first destination in destination 125 Receive the audit request then cancelled at other destinations 125.

In some embodiments, audit logic 170 is configured as monitoring the activity of mankind image audit person in real time.The prison Depending on may include receive from destination 125 based on word one by one or single keystroke audit input.It is such as other herein What place was discussed, it is received as word and/or keystroke are reviewed logic 170, they are optionally transferred into image source 120 One.The movable monitoring of manual auditor can be used to determine when the audit of image fails and/or complete to scheme manually As the progress of audit.The state of mankind image audit person can be provided from audit logic 170 to auditor's logic 157 in real time. Using the state, auditor's logic 157 can by the state of auditor from it is active change into it is sluggish, adjust auditor's The audit score that is stored, establishment change for the profession of auditor, and/or similar.

In some embodiments, audit logic 170 is configured as the measurement by receiving confidence level (for example, image audit Accuracy measurement) and to destination logic 160 and/or image put up logic 165 send response signal and control image To putting up for destination 125.Audit logic 170 can be configured as the measurement based on confidence level and control to destination 125 as a result, In the images of one or more destinations put up.The measurement representation one or more image tag of confidence level correctly identifies figure The confidence level of the content of picture.In some embodiments, audit logic 170 is configured as receiving the packet from manual image auditor Include the audit of the other information other than image tag.For example, audit logic 170 can receive from mankind image audit person Upgrade request and cause the image audit of upgrading requested.Audit logic 170 is optionally configured to processing and receives in hand Other non-label informations in audit that dynamic or computer generates.The information may include inappropriate (for example, obscene) image Identification, do not include the identification that can recognize the image of object, the identification of the image sent to the auditor of wrong profession, And/or similar identification.

In some embodiments, audit logic 170 is configured as the image by comparing the identical image from multiple sources It audits to adjust the confidence level of image audit.These image audits can be entirely that computer generates, be entirely manually to examine Core or including at least one computer generate audit and at least one manually audit.

In some embodiments, audit logic 170 is configured to supply as first (computer generate or manually) The received image tag in the part of audit and provide institute a received image to the mankind image audit person at the 125B of destination Label.The agency's (for example, application of browser or specific purposes) executed on the 125B of destination is optionally configured to mesh Ground 125B display provide first audit image tag.In this manner, mankind's image audit at the 125B of destination Person can edit and (increase, delete and/or replace) image tag of the first audit.For example, from the received image of destination 125A Label can be provided to destination 125B for modifying.

In some embodiments, audit logic 170 was configured as based on time used in these image audits, these images The accuracy of audit and audit score is calculated from the result of the received image audit in destination 125.

In some embodiments, audit logic 170 is configured with response logic 175 to the source of image (for example, image One in source 120) image audit is provided.When based on character one by one or word one by one image audit is completed, image is examined Core can be provided.When based on providing to word character by character or one by one, with character or word from mankind image audit person It is received, image tag is optionally provided to the source of image.Optionally, response logic 175 is configured as mentioning via network 115 For image audit.

Image audit need not be returned to one in image source 120.For example, if image source 120A is that picture shares clothes Business or social networking website, the image audit of the image from image source 120A can be with picture sharing service or social network network diagram Account on standing is stored in association.The storage can position in memory 135 or outside image processing system 110 It sets, such as at the network server for accommodating the website.Image audit be optionally both returned to one in image source 120 or It is stored elsewhere.

In some embodiments, response logic 175 be configured as based on computer generate and/or manually image examine Received image tag in core and execute search.The result of the search can be provided to the source of image, for example, image source 120A Or 120B.For example, in some embodiments, user creates image using the camera of image source 120A using smart phone.It should Image is provided to image processing system 110, is examined using the image that automatic recognition system 152 and destination 125A generate image Core.The image audit includes then being automatically used to the internet hunt of execution image tag (for example, Google or Yahoo are searched Rope) image tag.The result of the internet hunt is then provided to the smart phone of user.

In some embodiments, response logic 175 be configured as to ad system 180 provide computer generate and/or The image tag manually audited.Ad system 180 is configured as selecting advertisement based on image tag.Selected advertisement is optional Ground is provided to the source of the image for generating image tag.For example, response logic 175 can provide mark to ad system 180 Label " the 1976 Ford Granadas with the head lamp of damage ", in response, ad system 180 can be for replacement head lamp selections Advertisement.If being used to generate the source of the image of these labels is network address, advertisement can be shown in the network address.Particularly, If the source of image is that picture is shared or the account of social networking website, advertisement can be shown on the account.Advertisement System 180 is alternatively included in image processing system 110.Ad system 180 is optionally configured in response to specific Label is to provide quotation for advertisement.Ad system 180 optionally includes the Adwords of Google.

Image processing system 110 optionally further comprises being configured as extracting from the member of image source 120 for label The contents processing logic 185 of the image of addition.Contents processing logic 185 is configured as parsing and includes image and optionally include The webpage of text, and the image added for label is extracted from these webpages.The image tag of generation can be then provided To ad system 180 for selecting that the advertisement from the webpage for wherein extracting image can be placed in.In some embodiments, interior Hold processing logic 185 and is configured as emulation browser function in order to load the image that will be usually displayed on webpage.These figures As can be displayed on webpage associated with particular account, social network sites, picture sharing website, Blog Website, news website, Meet-a-mate site, sports site, and/or etc website.Contents processing logic 185 be optionally configured to parsing metadata tag with Convenient for identifying image.

Contents processing logic 185 is optionally configured to parse the text being placed in same web page as image.It should Text can be automatically recognized system 152 used in image label add, in conjunction with the content of image.For example, contents processing is patrolled Volumes 185 can be configured as identification image subtitle, make about image comment, be related to image text, Web page subject or Text (as determined by optical character recognition OCR (OCR)) in title, the people of image interior label or object, image is, and/or similar Object.The text or its subset parsed by contents processing logic 185 can be used to improve the speed and/or quality of label.It is solved The text of analysis is provided to automatic recognition system 152 and/or a destination being provided in destination 125 is for by the mankind Auditor adds label.In some embodiments, automatic recognition system 152 is configured as in the generation for the label of image Use provided text.For example, provided text can be used to text, identification dictionary, ontology, language and/ Or information, provide the accuracy of image tag automatically and/or manually generated, accuracy, computational efficiency, and/or its Its quality.Provided text is generally independent of as just the source of label generated, but is used as input to improve figure The processing of picture.The label generated as a result, may include the word other than those of discovery in provided text.

In some embodiments, image puts up logic 165 and is configured as providing to destination 125 in net identical with image Both image and text for being found on page.For example, can have subtitle " mountain bike pin in the girl in park and the image of bicycle Sell " or comment " happy birthday Zhu Li ".In destination 125, the text can be presented to human reviewer together with image.People The information can be used to more fully understand the emphasis and/or context of image in class auditor, and thus provides better image Label.Similarly, in some embodiments, automatic identification interface 150 be configured as to automatic recognition system 152 provide with figure As both the image found on identical webpage and text.At automatic recognition system 152, provided text is used to be based on The automated tag of the content improvement image of image.In the above examples, provided text can be built to automatic recognition system 152 View should place protrusion on bicycle or on Zhu Li.This can lead to totally different label, such as " Schwinn bicycle " or " birthday Girl ".

Image processing system 110 optionally further comprises image grading device 190.Image grading device 190 is configured as really Surely it is used for the grade (for example, priority) of label image.Priority can be used to determine (as if really if wanting) how label Image.For example, the determination of priority can source based on image, image be loaded on number on webpage, image on webpage Position, image watched on webpage number, on it including the number of webpage of image, one including image or more The classification of a webpage, the mark of webpage including image, the classification of the second image on the webpage for including image including image The owner of webpage, the domain name of webpage including image, the keyword on the webpage including image, in the webpage including image Number, the other images that the text of upper discovery, the metadata found on the webpage for including image, image are clicked on webpage The figure whether number, the image being clicked on webpage are a part of video, are automatically generated using automatic recognition system 152 As label, these exemplary any combination, and/or similar.Image grading device 190 includes may be stored on the computer-readable medium Hardware, firmware, and/or software be form logic.Image grading device 190 includes may be stored on the computer-readable medium Hardware, firmware, and/or software are the logic of form.In various embodiments, the priority packet determined by image grading device 190 Include two ranks (label or without label), three ranks (automated tag, manual label or without label), ten priority-levels, Or some other hierarchy plans.

Destination logic 160 is optionally configured to the mesh of the manual label addition of the selection image of the priority based on image Ground.

In those embodiments, the number that wherein image is loaded on webpage is used to determine that priority, the quantity can To be every fixed time period, such as daily or monthly.The quantity can be by including the Java or html script on webpage Row is determined, as known in the art.Position of the image on webpage may be considered that some images can be in image by viewing Before need viewer to scroll down through.The number that image is actually watched as a result, can be used to calculate the priority of image.It is logical Often, bigger priority is assigned to the image more often watched.Image grading device 190 is optionally configured to based on image The number that the number and/or other images being clicked on webpage or on other webpages are clicked on webpage is assigned excellent to image First grade.Image grading device 190 is optionally configured to the number watched on more than one webpage based on image and determined preferentially Grade.For example, if image is found on 25 different webpages, the sum watched on all webpages can by with To determine the priority for image.In some embodiments, image grading device 190 is configured as based on image in a browser The number being loaded determines priority.

Popular image can be included in several webpages.For example, the image shared extensively in social media website It can be included on several websites.Image grading device 190, which can be configured as, is included in webpage thereon according to image Quantity and/or the quantity of the webpage of the link including leading to the image calculate the priority of image.Image grading device 190 is optionally It is configured as (this possibility is incoherent) image that identification is included on multiple webpages.In some embodiments, image grading Device 190 is configured with third party's service, such as TinEye.com, to determine the number of webpage that image is placed thereon Amount.In general, the quantity of the webpage including image is bigger on it, the priority for being assigned to the image is bigger.

In some embodiments, image grading device 190 is configured as point based on the one or more webpages for including image Grade calculates the priority of image.For example, if webpage be classified in a search engine it is very high, by a large amount of other web page interlinkages or What person was well classified under some other standards, then the image on the webpage can be given and be classified as letter with the webpage Several priority.In general, classification is higher, the classification that webpage has is higher, and the priority for being assigned to the image on webpage is bigger. Webpage classification is optionally obtained from the third party source of such as search engine etc.

Image grading device 190 is optionally configured to the mark based on the webpage for including image to image assigned priority. For example, can be referred to for compared with the image at another webpage for identical website for the image on the homepage of URL Send bigger priority.In addition, image can based on it include image webpage specific type and be assigned priority. For example, the image in social networking website can be given more for compared with the image in company's site or personal blog High priority.In another example, for other types of webpage, in such as dictionary.com or Image on the reference webpage of Wikipedia.com etc can be given a higher priority.It is assigned to the preferential of image Grade is optionally the mark of web-based owner.

In some embodiments, image grading device 190 is configured as using the priority of the second image on webpage as letter Number determines the priority of the first image on the same web page.For example, if the second image has high priority, the first image Priority can correspondingly be increased.

Image grading device 190 is optionally configured to refer to based on the other contents for the webpage on it including image to image Send priority.For example, if webpage includes text and/or metadata, specific term or key in the text or metadata The presence of word can be used to assign the priority of image.Specifically, if webpage includes valuable keyword, at this Image on webpage can be assigned higher priority.The monetary value of the estimation of keyword with for advertisement or some other The value of the word of purpose is associated, for example, in GoogleUpper valuable word.Including meeting quilt Image of the Adwords higher on the webpage of the term of valuation can be assigned suitably high priority.These terms make Frequency and their quantity on webpage can also be considered to determine picture priority by image grading device 190.Considered Text and/or metadata can be included in the URL of webpage, in the map title, in the comment made to figure, by third Directional image assign label in, name, brand name, trade mark, enterprise name, be related to image text near and/or it is similar.

In some embodiments, image grading device 190 is configured as receiving using optical character recognition OCR derived from image Text and the priority that image is directed to based on text determination.For example, image grading device 190 can receive by using automatic Identifying system 152 handles image and the text that generates, and based on the text to image assigned priority.In some embodiments, Image grading device 190 is configured as the first image on webpage and provides higher priority, relative in the more lower of the webpage For the image just occurred.

How image grading device 190 is configured to determine (if truly doing) based on the priority assigned Label is added to image.For example, the image of lowest priority can be by label.Image with somewhat higher priority can To use automatic recognition system 152 to be added label, and the image with higher priority can be by human reviewer in mesh Ground 125 in a purpose be located in and be added label.Those with enough high priorities, will by human reviewer add mark The image of label is optionally further split into higher priority group and lower priority group, in higher priority group It is given more concerns and more thoroughly or is carefully done outgoing label addition by human reviewer.Image puts up logic 165 can Selection of land is configured as the instruction that the member into destination 125 provides the priority of image and image.

In some embodiments, image is used first handled by automatic recognition system 152.Then, which can be with base Both confidence levels in the priority for image and the automated tag executed by automatic recognition system 152 addition are sent to One or more members in destination 125.For example, if image has relatively low priority, for being examined to the mankind The confidence criteria that core person sends image is arranged to relatively low.(low confidence standard means that automated tag adds possible quilt It is considered that the enough and image is not used in mankind's analysis and is sent.If) image have relatively high priority, Confidence criteria for sending from image to human reviewer is relatively high.Thus, the higher confidence of high priority image request Degree is more likely to be sent to human reviewer only to rely only on automated tag addition.

Can by image grading device 190 for image selection processing path for example including a) not add label, b) only Label, c are added using only automatic recognition system 152) confidence level based on importance and/or generation label, using with optional The automatic recognition system 152 of mankind's follow-up add label, d) automated tag addition followed by adding to the automated tag The mankind audit, e) by human reviewer add label and/or f) based on by will by human reviewer consider suggestion rank by Human reviewer adds label.These processing paths are at least partially based on the priority assigned from image grading device 190 to image And it is selected.Any combination of these processing paths can be found in a variety of different embodiments.In some embodiments, The result that the type for the processing for being used to label image is controlled cause may those of more valuable image have it is bigger Possibility is by label.As a result, mankind's label resources are applied to highest priority --- the image of most worthy.

In some embodiments, image grading device 190 is configured as based on advertisement that is adjacent with image or showing on it Mostly often it is clicked and assigns the priority for image.For example, if image is on the webpage often watched but position Advertisement on the image is rarely clicked, then the image can be given relatively high priority for label. In this example, it is more than one can be added label for image.If the advertisement based on initial labels is not with expected frequency It is clicked, then the image can be tagged again.Again it tags and is optionally executed by human reviewer, mankind audit Person puts up logic 165 via image and receives image and initial (unsuitable) label.Human reviewer can be used the information and mention For improved label.

Fig. 2 illustrates Image Acquisition screen 210 according to various embodiments of the present invention.Image Acquisition screen as shown in the figure 210 for example by executed in smart phone, electronic glasses or other image sources 120 application and generate.Image Acquisition screen 210 wraps It includes and is configured as acquisition image, the interested specific region of label and the feature for receiving image tag.Particularly, Image Acquisition Screen 210 includes the shutter release button 220 for being configured as taking pictures.Once picture is taken, by optionally via network 115 automatically Image processing system 110 is sent to add for label.Image Acquisition screen 210 optionally further comprises being configured as scheming The rectangle of prominent interested point as within.Rectangle 230 be select and/or pull on screen by using user input equipment be Controllable (for example, moveable).On typical smart phone, which may include touching in response to finger The touch screen touched.As described elsewhere herein, interested point/region be may be provided to and label to be added The associated image processing system 110 of image.

Image Acquisition screen 210 further comprises the field 240 for showing the image tag of the image and generation that had previously acquired.At this In example, identical white cup that the image previously acquired includes no rectangle 230 is shown and including " white Startbuck's coffee The image tag of coffee cup (White Starbucks Coffee Cup) ".What is be also shown as is alleged " slide for option The text of (Slide for options) ".

Fig. 3 illustrates the search result based on image analysis according to various embodiments of the present invention.These results are optional Ground is automatically or in response to select " sliding is for option " input shown in Fig. 2 and is shown.They can be by image Internet hunt is automatically carried out on label and is generated.Fig. 3 diagram is sponsored advertisement 310, associated picture 320 and other searches Hitch fruit 330.Search result optionally uses ad system 180 and generates and image tag uses image processing system 110 It is generated.User can choose review and previously add tagged image.The history can be stored on image source 120A or In memory 135.

The method that Fig. 4 illustrates processing image according to various embodiments of the present invention.In these methods, image is connect It receives.The image is provided to both at least one destinations in automatic recognition system 152 and destination 125.As a result, meter Both image audits that the sum that calculation machine generates manually generates are generated.The method illustrated in Fig. 4, which optionally uses in Fig. 1, to be illustrated The embodiment of system be performed.The method and step that Fig. 4 is illustrated into Fig. 8 can be sequentially executed with plurality of replaceable.

In receiving image step 410, image is received by image processing system 110.The image is optionally via network 115 receive from one in image source 120.The image can be the reference formats such as TIF, JPG, PNG, GIF.The image can To be an image in a series of images for be formed the image sequence of video.The image can be used by a user camera and adopt Collection.The image can be acquired from movie or television program by user.In some embodiments, image step 410 is received to wrap It includes user and acquires image using Image Acquisition application and transmit the image to image processing system 110.The application can be by It is placed within camera, TV, video display apparatus, multimedia equipment, and/or analog.Receiving image step 410 optionally makes It is promoted with contents processing logic 185.

In an illustrative example, image is received from image sequence (for example, video).The video is displayed on In monitor, TV, eye guard, glasses or other display equipment.The video optionally via such as youtube.com orEtc video streaming services be received and/or in browser be shown.Logic in display system (for example, image tagged logic 147 in image source 120A) is configured such that user indicates the specific son of the image in the video Collection.Identical logic can be configured as receive generated in response to image tag from image and the advertisement that is selected and this is wide Announcement is shown on video or shows simultaneously with the video.The selection of advertisement based on image tag other local quilts herein It further discusses.

Particularly, using the system, user can choose the object and work for label addition in video or film The label for characterizing the object is optionally received to respond.User can with or alternatively receive based on the label it is selected Advertisement.The advertisement can be shown (for example, video sequence as covering or addition) together with video or via other in real time Communication channel (for example, Email) is provided to user.In an illustrative example, user sees he in video The object liked.They select the object and the selection is received in receiving image step 410.In response, they Receive advertisement relevant to the object.As video is watched over the display, advertisement is as covering, item or the subtitle on video It is displayed in real time.Advertisement be optionally it is interactive, since it includes the link for making purchase.

In some embodiments, the object in image may include the specific spy for being configured as assisting in identifying the object Property.For example, the AD HOC of data bit can be encoded in image or in the object of image.These data bit can be figure As label coding.

It is optionally receiving in subset identification step 415, is identifying the data of one or more subsets of image by image Reason system 110 receives.In general, one group of image pixel that the project that one or more subsets include special interests is located therein.It should One or more subsets can by location of pixels, screen coordinate, region and/or point on received image identify.Some In embodiment, which is used by a user selected by the cursor or touch screen of an image source in image source 120.

It is optionally receiving in source data step 420, source be received in receiving image step 410, about image Source data received by image processing system 110.As discussed elsewhere herein, source data may include geographical letter Breath, Internet protocol address, uniform resource locator, account name, smart phone identifier, about image source 120 at The information of the language used on member, searching request, user account information, and/or similar.In some embodiments, source data quilt Application/the agency in operation image source 120 automatically sends.For example, GPS coordinates can be on smart phone by certainly It generates dynamicly and is provided to image processing system 100.

Optionally receive source analysis priority tasks 425 in, receive image step 410 in be received, for scheming The priority of the label of picture is received by image processing system 110.In some embodiments, priority is by the user of image source 120A It manually inputs.In some embodiments, priority depends on the amount paid a price for the audit of image.In some embodiments In, priority depends on the type of image source 120A.For example, compared with from the received image of handheld mobile device, from static network Received image of standing can automatically be given lower priority.Compared with the image that its source is identified by Mobile Directory Number, The image that its source is identified by uniform resource locator can be given lower priority.Priority is optionally by connecing as a result, Receive the source data export being received in source data step 420.

It is optionally received together in the step 410-425 image being received and data and is optionally stored in storage In device 135.

In distribution image step 430, image and optionally it is received in step 415-425 any associated Data are assigned via automatic identification interface 150 to automatic recognition system 152.The distribution can image processing system 110 with It is interior or via network 115.

It is automated toed respond in step 435 in reception, the image audit that computer generates is received from automatic recognition system 152. The image audit that computer generates includes one or more image tags that the system of being automatically recognized 152 is assigned to image.It calculates The image audit that machine generates further includes the measurement of confidence level.The measurement of the confidence level be assigned to image image tag it is correct Ground characterizes the measurement of the confidence level of the content of image.For example, the image relative to abstract shapes, main includes that may be readily recognized The image of character can receive the measurement of higher confidence level.

It is optional determine confidence level step 440 in, the measurement for the confidence level for including in image audit with one or more A intended level compares.The intended level be optionally the priority of image audit, the price of image audit, image source, And/or the function of analog.Optionally firmly believing? in step 445, if the confidence level for the image audit that computer generates is higher than (multiple) predetermined threshold process then proceeds to optional execution search step 450, if the confidence for the image that computer generates Degree then proceeds to lower than (multiple) predetermined threshold and is lined up image step 460.Determine that confidence level step 440 is optionally patrolled using audit 170 are collected to be performed.

In executing search step 450, the image tag for being assigned to image is used to execute search.For example, image mark Label " ford " can be used to word " Ford " and " vehicle " to automatically carry out Google search.

In result step 455 is provided, it is assigned to the image tag of image and is optionally executing search step 450 In the result of search that is performed be provided to the requestor of image audit.For example, if image is received from image source 120A And image source 120A is smart phone, then image tag and search result are normally provided to smart phone.If image It is received from the member in the image source 120 of such as website etc, image tag and optional search result can be provided to The host of website, to third party, to ad system 180, and/or analog.In some embodiments, image tag is by automatically It is added to website, so that image tag can search for, for example, can be searched to find audited image.

In being lined up image step 460, image is arranged in image queue 145.The arrangement is optionally included using figure As the subset of 147 tag image of mark logic.As described in elsewhere herein, label is typically configured as identification image In special interests object.The priority for promoting the audit that can depend on image of image in image queue 145, image Source, available mankind's image audit person, computer generate image audit confidence level measurement, and/or analog.

In determining destination step 465, one or more members in destination 125 are determined for the hand of image Dynamic audit.The determination of destination is optionally based on the image audit generated in the computer being received from automatic recognition system 152 The middle image tag for being included；It is optionally based on the profession of the mankind image audit person at different destinations 120；Optional ground In the audit score of these mankind image audit persons, and/or based on the other standards being discussed herein.In some embodiments, really Determine profession of the destination step 465 based on the data and human reviewer for characterizing image.The data for characterizing image can be Characteristics of image, image descriptor, and/or the information being derived from.As discussed elsewhere herein, characteristics of image and/ Or image descriptor is optionally received together with image from the member in image source 120.Information as derived from it can be given birth to At at the member of image source 120, at image processing system 110 and/or at automatic recognition system 152.

In putting up image step 470, image is posted to destination determined in determining destination step 465 At least one destination in 125.In some embodiments, put up image step 470 include image is concurrently posted to it is more In a destination 125.The image is optionally posted via network 115 and is optionally opened together with set forth below Paste: the label of the subset of prominent image, the time audited before failing in image, is used for from certainly the source data for image The image tag, and/or analog of the dynamic received image of identifying system 152.

In receiving audit step 475, the manual audit of image one from destination 125 determined by (multiple) or It is received in multiple.Manually image audit may include one or more figures that image is assigned to by mankind image audit person for this As label.The content of one or more image tag expression image.The manual audit can also include upgrade request, can not examine The inappropriate instruction of the instruction of the image of core, image, the instruction of audit failure, and/or similar.

It is tagged in image? step 480 in, the progress of this method depends on whether image tag is receiving audit step It is received in rapid 475.If the image tag for characterizing the content of image is received, this method is optionally continued to execution Search step 450 and offer result step 455.In those steps, it is audited in manual image and optionally computer is raw At image audit in include image tag used.The use of image tag in the image audit that computer generates can To depend on the confidence metric of the audit.

If confidence metric, which is found to be, in step 445 is higher than (multiple) predetermined threshold, step 460-475 is optional 's.

Optionally upgrading? in step 485, the progress of this method depends on whether upgrade request has been received.If Such request has been received, then method proceeds to determining destination step 465, wherein the second of destination 125/difference Member be determined.The determination can depend on the image tag being received in manual image audit, manual image audit It is received in receiving audit step 475.Upgrade request can be received from mankind image audit person or asking from image audit The person of asking (from image source 120A or 120B etc.) is received.Have an opportunity to audit the quilt in result step 455 is provided in requestor After the image tag of offer, upgrade request can be received.For example, requestor can receive the image including " white vehicle " first Label and then request audit upgrading, because they expect to have further information.Audit upgrading can cause image to be provided To the mankind image audit person having in automotive field profession.Mankind's image audit can be added to existing image tag with It generates " white vehicle, 1976 Ford Granadas ".In some embodiments, when request audit upgrading, requestor, which can add, to be referred to The source data of the subset of diagram picture.For example, auditor may desire to indicate the special interests on the head lamp of damage.This is used for will Mankind's image audit person's attention is oriented to this feature of image, generates the label including " head lamp of damage ", and guiding is caused to be used In the search (executing search step 450) of the head lamp of the damage of 1976 Ford Granadas.

In some embodiments, upgrade request is reviewed logic 170 and automatically generates.For example, if image audit occurs Must be too brief, such as be only " vehicle ", then audit logic 170 can automatically initialize audit upgrading.In some embodiments In, upgrade request automatically generates the presence based on the keyword in manual image audit.For example, certain audit professions and key The list of word is associated.In some embodiments, when a keyword in these keywords is received in manual image audit In and automatically audit upgrading be initialised.Audit upgrading, which preferably includes, has profession associated with the keyword received Mankind image audit person.In particular example, profession include " automobile " and with keyword " vehicle ", " truck ", " goods Vehicle ", " open car " and " Ford " are associated.When a keyword in these keywords is received in manual image audit When, whether audit logic 170 negotiates with logic 157 is audited to determine and have mankind's image audit person in " automobile " profession current It is active.If it is active, then automatic upgrading is initialised and image is sent to examining with " automobile " profession The destination 125B of core person.

If made without upgrade request, step 490 is being terminated, which completes.

Fig. 5 illustrates the alternative of processing image according to various embodiments of the present invention.In these methods, it walks At least some of at least some of rapid 430-445 and step 460-475 is executed in parallel.In step 460-475 Manual image audit can start before the audit that the computer of step 430-445 generates is completed thus manual image audit exists Start before knowing the confidence metric for the audit that computer generates.If firmly believing? step 445 confidence metric is found to be Higher than (multiple) predetermined threshold, then step 460-475 is optionally terminated.

The method that Fig. 6 illustrates management auditor pond according to various embodiments of the present invention.In the method, auditor State can audit the level of image based on them and be changed.Illustrated step can be the method for Fig. 4 and Fig. 5 diagram Part is consistently performed with Fig. 4 and Fig. 5 method illustrated.For example, they can be examined receiving image step 410 and receiving It is partially executed one at a time between core step 475.The method of diagram includes that the more than one destination into destination 125 sends image.

In receiving image step 410, image is received.As other places discuss herein, image can be via Network 115 is received at image processing system 110.Image can be generated by camera and/or be obtained from webpage.In some implementations In example, image is received together with the information mostly often watched about the webpage.

In selecting the first destination step 610, the first destination is selected for the manually or automatically analysis of image. It selects the first destination step 610 to be performed using destination logic 160 and is the embodiment of determining destination step 465. As described in other places herein, the determination for the destination of image is based on various factors, including human reviewer State and score associated with auditor.For example, the member of destination 125 usually associated with active auditor will It is selected, rather than the destination of not active auditor.Selected destination can be destination 125 member and/ Or automatic recognition system 152.

In putting up image step 470, received image is posted to destination 125 in receiving image step 410 Selected member.As other places discuss herein, putting up for image may include via network 115 using such as The computer network with standard network protocol of TCP or UDP etc transmits image.

In optionally monitoring step 620, auditor's logic 170 is used to monitoring and is selecting the first destination step 610 The progress of the manual image audit of image at the member of the destination 125 of middle selection.Monitoring may include detection mankind's audit Time used in the input of person, image audit, the quantity of provided word for characterizing image, and/or similar.It monitors optional Ground includes the time used in measurement label image.Wherein detection includes the detection of the input of human reviewer, and monitoring can be base In keystroke one by one, based on word one by one and/or on line by line basis.Auditor's logic 170 can be configured as once as a result, Receive the data for characterizing image character, word or row.

In removing step 620, at the member for the destination 125 that image selects in selecting the first destination step 610 from It is removed in processing." removal " may include notified at the selected member of destination 125 human reviewer he or she not Additionally and briefly be responsible for audit image, the human reviewer of prime responsibility released into (without notifying human reviewer), by image from The display of human reviewer removes and/or similar.In some embodiments, removing step 630 includes only auditing the mankind Person is placed in classification the secondary or shared responsibility having for audit image.For example, if with the first purpose is being selected The associated human reviewer of the member of the destination 125 selected in ground step 610 has the prime responsibility for audit image, The responsibility can be shared or be assigned to now other auditors associated with other members of destination 125.In the situation Under, " removal " is prime responsibility.

If the manual audit of image has been used too long, removing step 630 can occur.For example, if being walked in monitoring Have found that auditor did not start to input after the predetermined time in rapid 620, then removing step 630 can be performed.For The other examples of the trigger event of removing step 630 include with the communication loss of the selected member of destination 125, for figure The distribution of the audit of picture receives inappropriate or inappropriate image beyond the predetermined time, from human reviewer, from human reviewer Receive the image tag of inaccurate (not characterizing image), from the first human reviewer to the recommendation of the second human reviewer, figure As the upgrade request, and/or similar of audit.

In selecting the second destination step 640, the second member (or automatic recognition system 152) of destination 125 is used Destination logic 160 is selected.Second member can be based on above with respect to the first destination step 610 of selection and determining destination Any standard that step 465 is discussed is selected.In addition, in some embodiments, the selection of the second member can based on by with The specific recommendations of the associated human reviewer of the first member of destination 125.For example, the first human reviewer can identify figure The content of picture is the profession of the second human reviewer and the image recommendation can be given to the second human reviewer associated The member of destination 125.It is optionally taken in the selection for the second member for selecting the destination 125 in the second destination step 640 Certainly automatically processing in the image using automatic recognition system 152.

It is puted up in image step 470 another, image is posted to selected in the second destination step 640 of selection The member of destination 125.In some embodiments, more than one human reviewer can concurrently audit image.They can be with Independently or cooperative execute audit.One auditor can have the prime responsibility of the audit of image or each auditor can With same responsibility.One auditor can have the supervisory responsibility to one or more of the other auditor.In some realities It applies in example, before monitoring step 620 and/or removing step 630, the second destination step 640 of selection is performed and image It is posted to two or more members of destination 125.

In receiving audit step 475, audit (for example, image tag) such as other places herein of image are discussed As be received.Audit generally includes to characterize the image tag of the content of image.Audit can be from being more than in destination 125 One destination receives.For example, the label for characterizing image can be from the second mesh of the first destination step 610 of selection and selection Ground step 640 the two in selected in the member of destination 125 that is selected.It is audited with character or word by (multiple) mankind Person provides, and receives audit step 475 and is optionally performed in real time.

In optional correlation tag step 650, the one or more image tags for characterizing image are associated with image Ground is stored.Stored label is optionally included by the label of more than one mankind auditor's offer and can be stored In memory 135.As described in other places herein, label can be further supplied to the member of image source 120 (for example, in embodiment that result step 455 is provided) or it is used to select advertisement using ad system 180.Label can also quilt It is provided to automatic recognition system 152 and the training of automated graphics identification process is provided.

Fig. 7 illustrates the method for receiving image tag in real time according to various embodiments of the present invention.These methods can Selection of land by image processing system 110 execute and image tag can be manual image audit result.These methods can be with Other methods described herein are consistent, such as the part of method shown in Fig. 4.This method is to put up image step 470 Start, in this step, image is provided to one or more members of destination 125, as discussed elsewhere herein 's.Method shown in fig. 7 is optionally to receive image step 410 as first, image is from remote computing device in this step It is received.

In receiving input step 710, input is received from one or more members of destination 125.The input is usually wrapped The character provided by human reviewer is provided.For example, input can be the character inputted by human reviewer in destination 125A. In general, being performed with other steps shown in fig. 7, receives image step 710 and continue.

In detecting the first word step 720, it is detected in the input that word is received in receiving input step 710. The word can be detected by such as space character in space ASCII etc or the presence of carriage return.Spell check is optionally detecting Word on be performed.If word is not included in spell check dictionary, the trial corrected can be made or Human reviewer can be informed that identification word failure.

Lead to the execution for delivering the first word step 730 in the detection for detecting the word in the first word step 720, at this Word is transferred into the source of image in step 730.Once for example, word is detected image source can be provided in real time 120A.In image source 120A, which can show to user.It waits compared with before the set of display image tag until figure For being received as the entire set of label, once show that a word can provide the analysis of image and occur when shorter The impression of the area of a room.

In detecting the second word step 740, it is detected in the input that the second word is received in receiving input step 710 It surveys.Again, word can be detected by the presence of blank character and can be provided at image source 120A by the first word User after occur.Both first and second words are expected for characterizing the label of image.Detecting the second word step The detection of 740 the second word triggers the second word step 750 of delivering, and second word is delivered to image in step 750 Source, for example, image source 120A.Third, the can be recycled and reused for by detecting the second word step 740 and the second word 750 of delivering Four and additional word, part of each word as image tag.

It is completed in step 760 in detection, the data for indicating that the processing of image is completed are received, which completes to be, for example, to examine The word measured includes the whole words (image tag) to be provided by human reviewer.The data may include such as "/ Endtags ", ASCII carriage return, and/or similar metadata tag.In general, detection completion step 760 is at one, two or more More image tags occur after being received.In optional correlation tag step 650, the received image tag of institute and image It is associated and/or is stored together with image, as discussed elsewhere herein.

Although Fig. 7 illustrates word in primary detection and delivering, in alternative embodiment, individual keystroke is tested It surveys and delivers.Receiving input step 710 can concurrently continue with step 720-740 and/or 750.Step 710-760 can make It is included that other places herein are discussed to receive the part of audit step 475.

The method that Fig. 8 illustrates the audit of upgrade image according to various embodiments of the present invention.In these methods, image Receive the more than one stage of image audit.Then the first audit (stage 1), image audit are upgraded and are further audited (stage 2).Request for upgrading can be generated automatically, and be initiated by the first human reviewer, and/or in response to from figure The request in the source of picture.Both first and second audits can be manually, that is, have human reviewer to execute.Alternatively, One audit can be automatic and one or more subsequent audits can be manually or first audit can be manually And one or more subsequent audit be automatic.

In receiving image step 410, image is received.First member of destination 125 is selected in selection the Image in one destination step 610.The image is then posted in putting up image step 470.These steps are herein Other places be discussed.

It is audited in step 810 in reception first, the first audit of image is received.First audit may include characterizing One or more image tags of the content of image.For example, image audit can in response to the picture of the image including black spider To include word " Black Aranea "；Or in response to the image including red car, image may include word " screw oil expeller ".Receive the One audit step is optionally the embodiment of reception audit step 475, and may include the real time communication of image tag, is such as closed It is discussed in Fig. 7.

In some embodiments, the first audit may include being provided by the human reviewer for executing the first image audit , the instruction that the processing of image should be upgraded.For example, the first human reviewer can manually indicate for second (optional Ground specialization) the speciality field of human reviewer.For example, the first human reviewer can provide " screw oil expeller " image tag and build View is executed the audit of upgrading by the auditor with automobile speciality.Alternatively, the first audit may include being considered especially having The image tag of value.For example, a possibility that instruction image includes wedding clothes can trigger automatically for 72% automatic audit Manual audit is upgraded to, because image tag " wedding clothes " potentially has bigger commercial value than other image tags. In some embodiments, which is reviewed the execution of logic 170 and based on whether relatively important or valuable key Word is stored in the list in memory 135.The list may include the associated measurement of keyword and their value.Such as this Other places are discussed in text, are optionally based on by the automatic upgrading that audit logic 170 executes using automatic recognition system 152 And/or the image tag that forecast image mostly often automatically generates the information watched.These factors optionally use maximum Change the algorithm using the potential value of human reviewer's label image and provides advertisement based on these labels and applied.More have The example of the image tag of value can with shoes, vehicle, jewelry, Reiseziel, book, game, clothes, vacation, food, beverage, The correlations such as real estate, bank, accident.

In some embodiments, the upgrading of image audit is automatic.For example, the label of " Black Aranea " can automatically be led Cause includes that the upgrading of the image audit of image is sent to the human reviewer with specific speciality (for example, spider expert).It is specific Plant or the identification of animal life generally include and (depend on) location information because the position of plant or animal can be for It is important for appropriate identification.

In some embodiments, as discussed elsewhere herein, upgrade audit can be by raw requests picture quilt The people of audit requests.For example, the user of image source 120A can provide the image of dog and receive the image mark including " black dog " Label.User can then by provide word " cultivate? " to require further details.In this case, image audit can be with It is upgraded and is sent to the human reviewer of the specific instruction of the cultivation with dog.In some embodiments, user is for rising Premium account is charged or be requested to have to grade in order to request to upgrade.User can specify when requesting image audit upgrading The specific part of image.

The presence (automatically and/or manually generating) of upgrade request is detected in detection upgrade request step 820.Detection Can based on the member from image source 120, the member from destination 125, automatic identification interface 150, and/or from such as audit patrol Collect the received data of component or life of the image processing system 110 of 170, contents processing logic 185 or response logic 175 etc It enables.

In selecting the second destination step 640, destination logic 160 is used to the second member of selection destination 125 And/or automatic recognition system 152 is with the audit for image.The selection can based on the first destination step 610 based on mark Any one quasi- and image tag and/or the other information generated from the first audit.For example, the second one-tenth of destination 125 The selection of member can be based on, and be based at least partially on the image being generated manually or automatically in the first image audit.Particularly, The label of " Black Aranea " can be by destination logic 160 is used to select to audit with the mankind of the speciality of the identification with spider The member of the associated destination 125 of person.In another example, the selection of the second member of destination 125 can be based on by asking The word for asking the user of image audit to provide.Particularly, if the first audit generates image tag " white shoes " and user response With " brand? ", then human reviewer's phase of information selection and the speciality with shoes brand can be used in destination logic 160 The member of associated destination 125.

In some embodiments for selecting the second destination step 640, destination logic 160 is configured as to select certainly Dynamic identifying system 152 selects the member of destination 125 with the second audit for image.For example, when image by Label has that the name of performer and upgrade request require " movie name? " when, this may occur.In this case, image can To be searched in the library of film image.Identical method can be taken to be used for such as currency, drawing, vehicle, trade mark, item Other reproducible objects of shape code, QR code, fatal personage etc..

In another example for putting up image step 470, image is posted to the second selected member of destination 125 Or automatic recognition system 152 is with the second audit for image.Step 830 is audited in reception second, characterizes the interior of image The image tag of appearance is usually received.Receiving the second audit step 830 is optionally the embodiment for receiving audit step 475.It can Alternatively, instruction image because certain reasons cannot additional reference by label or other information can be received.Image tag The member for the destination 125 being posted to from image or automatic recognition system 152 are received.If necessary, step 820, 640, it 470 and 830 can be repeated.

In correlation tag step 650, received image tag it is associated with image and/or be provided to image Source, as discussed elsewhere herein.

In one illustrative examples of method shown in Fig. 8, image is received from webpage.Image is sent to automatic knowledge Other system 152 for auditing automatically.Automatically the result audited includes image tag " ring ".The label uses audit logic 170 is processed and be identified as potential valuable image with for using in advertisement.As discussed elsewhere herein , the other factors which is optionally also mostly often watched on webpage etc based on such as image.Knot as identification The audit of fruit, image is automatically upgraded and is sent to destination associated with having the human reviewer of jewelry speciality 125 member.Human reviewer modification image tag has included " golden wedding ring " and these labels are related to image Connection.These image tags can be subsequently used to select advertisement using the system and method described elsewhere herein.

In one illustrative examples of method shown in Fig. 8, image is received from the application executed on the mobile apparatus. The image includes streetscape and is sent to destination 125A for being audited by human reviewer.Human reviewer is with image tag " streetscape " responds and these labels are provided to mobile device.The subsequent slave mobile device of request of audit upgrading is connect It receives.The request include text " vehicle? " and the identification of the part of image includes vehicle.As the request as a result, image, text The other member of destination 125A or destination 125 is sent to for further audit manually with identification.Into one The audit of step leads to image tag " 1909 model T ", is then forwarded to mobile device.

In one illustrative examples of method shown in Fig. 8, image is received from equipment is calculated.The image includes being placed on Multiple bank note on disk are simultaneously sent to destination 125A for auditing manually.The label of generation includes the " U.S. of Bai Panshang Currency " is simultaneously sent to calculating equipment.The request of image audit upgrading is received.The request includes " how many? " as this Request as a result, image is sent to automatic recognition system 152, automatic currency recognition logic is used to identify currency at this It measures and optionally provides sum.The information is then sent back to calculating equipment.

In one illustrative examples of method shown in Fig. 8, image slave mobile device is received.The image includes plant Leaf is simultaneously sent to destination 125A.Human reviewer at the 125A of destination provides image tag " greenery " and also rises Grade image audit.As upgrading and image tag as a result, image is sent to destination 125B, and with botany speciality The second mankind image audit person it is associated.It is based on image tag " greenery " to the selected section of destination 125B.Concurrently, scheme As label " greenery " are sent to mobile device.At the 125B of destination, second mankind's image audit person is to already existing figure As label addition word (label) " toxicodendron ".These additional labels are then also routed to mobile device.On the mobile apparatus, Word " greenery " once it is shown first and then available word " toxicodendron " is added into display.

Method shown in Fig. 8 is optionally used in consistent with other methods described herein.For example, image tag can be with It is used in the auction labelling step 1560 described elsewhere herein.Label can be used to selection advertisement and advertisement quilt It is provided to the remote browser for showing on webpage together with image.

Fig. 9 illustrates the example of the image source 120A including electronic glasses according to various embodiments of the present invention.Electronics Glasses include the glasses being worn.Example includes Google" Google glass "," M100 The iOptik of intelligent glasses ", Innovega^TMContact lenses, and/or similar.These systems are configured as allowing user at the same time Between watch both real world and electronic console.Electronic glasses can also include the Oculus Rift of such as Oculus VR^TM Etc virtual reality system.The system of these types is shown image to user using electronic console, but is not provided true Direct viewing while the real world.It is not used for watching and digitized viewing when direct viewing, for example, passing through glasses or eyeglass Viewing.

In general, user can be real when electronic glasses are provided when image in electronic glasses or watched by electronic glasses When select image subset interactive interface.As used in this article, it is " real-time " selection mean image when it is collected with Only inessential time delay is watched.For example, the image watched in real time can be acquired by camera and use image engine It handles and shown with the time delay only generated from electron process number.The user of viewing permission in real time is watched with image to be passed through Interested object is located within the image of viewing by mobile image acquisition equipment.Thus, viewing, which eliminates, in real time is watching Before by the viewing of the image of storage quite a while.

As shown in figure 9, image source 120A includes the camera 910 for being configured as acquisition image.Acquired image can wrap Include static image or the video including image sequence.Display 920 is configured as acquired image being presented to camera 910 User.In some embodiments, such as image source 120A is those of smart phone embodiment, and display 920 includes being configured For the touch screen of the view finder as camera 910, to show acquired image and show Image Acquisition screen 210 (herein Described in other places).The display and other contents on display 920 that display logic 925 is configured as management image. Display logic 925 may include the hardware, firmware, and/or software that may be stored on the computer-readable medium.

In the embodiment shown in fig. 9, image source 120A further comprises the user's instruction for being configured for image source 120 The selection logic 930 of the subset of acquired image.The instruction can be just shown with image and/or be collected in display It is made in Image Acquisition screen 210 in real time on 920.As discussed elsewhere herein, such instruction is done Out to select the interested special object in image.And then it selects, the subset of image optionally uses image tagged logic 147 is labeled, as discussed elsewhere herein.Image tagged logic 147 can be used to be shown into such as display 920 Image add label.User can see that the position being labeled as a result,.Selection logic 930 is optionally configured for User re-flags the subset of image, until user is satisfied with selection.

In some embodiments, selection logic 930 includes tracking logic 935 to track the movement of eyes of user.Tracking is patrolled 935 are collected to be alternatively included in electronic glasses.Eye tracks may include the focus point for detecting eyes, one or more eyes Direction (eyeball direction), the focusing of one or more eyes, blink, eyeball it is mobile, and/or similar.It is optional to track logic 935 Ground is configured as the state of eyes of user is associated with the position in acquired image.Indicate camera 910 and selection logic The geodata of geographical relationship between 930 physical factor is used to be adopted the state of eyes of user with using camera 910 Position in the image of collection is associated.

Tracking logic 935 optionally includes the second camera for being directed toward the glasses of user.The camera can be installed in electronics The part of other embodiments on glasses or as image source 120.For example, the tracking for being configured for the eyes of tracking user is patrolled Volumes 935 can be included in network cameras, in smart phone, in computer monitor, in TV, in tablet computer and/ Or the like in.

In some embodiments, tracking logic 935 is configured as detecting the blink of one or more eyes.For example, tracking Logic 935 can be configured as the mode for detecting simple eye blink or blink.When being detected such time, logic 930 is selected It can be based on patrolling the position in 935 received position datas selection image from tracking, or alternatively select to see at present The position at the center for the image seen.

Once image has used mark logic 147 and selection logic 930 labeled, the position and/or region of the label User can be displayed on the image of the label in display 920.For example, in marked locations plus red " X " Image can be displayed to user in Image Acquisition screen 210.In some embodiments, user can be then using confirmation logic 940 confirmation selections.Confirmation logic 940 is optionally in response to tracking logic 935.For example, blink or other eyes can be used in confirmation Movement, voice command, verbal order or touch order are provided.In some embodiments, tracking logic 935 is configured as examining Survey and be interpreted as the movement that order eyes enter unnatural position (for example, esotropia).Such movement can be used to Confirmation order.Confirmation is optionally required prior to sending image to network 115.

In some embodiments, selection logic 930 is including being configured as tracking chasing after for certain things other than eyes Track logic 935.For example, tracking logic 335 can be configured as detection user indication finger (pointing finger), The electronic equipment, and/or analog being worn in finger or wrist.In these embodiments, selection logic 930 is configured as base The position in image is inferred in object detected.In one embodiment, tracking logic 935 is configured as detection in image The position that the position of the finger of indication and deduction will be selected is at the tip of finger.User can give directions in its visual field Object, order to image source 120A offer audio, based on eyes and/or based on touch, and the position for the finger given directions The selection of the position in image will be used to make.In one embodiment, tracking logic 935 is configured as detection wireless electron Equipment relative to image source 120A position and the position to be selected of deduction along radio-based electronic devices and image source 120A Part between line.

Image source 120A further comprises being configured for image source 120A to be sent to image processing system via network 115 110 I/O 945.I/O 945 may include wired and/or wireless connection.For example, in some embodiments, I/O is matched It is set to and uses bluetooth^TMConnection is wirelessly communicated to cellular phone from electronic glasses and is used subsequently to using Wifi or cellular service Communication is forwarded to network 115 from cellular phone.

Image source 120 further comprises being configured as storage to use camera 910, geodata, account data, and/or class Like the embodiment of the memory 135 of the image of acquisition.Memory 135 includes such as random access memory (RAM) or read-only depositing The non-transitory memory of reservoir (ROM) etc.Memory 135 generally includes to be configured as storage acquired image and these The data structure of mark position in image.

Image source 120 further comprises processor 950.Processor 950 is configured as executing at the number of computations Manage device.For example, in some embodiments, processor 950 is encoded with computations to execute display logic 925, selection logic 930, image tagged logic 147 and/or tracking logic 935.Processor 950 optionally includes application-specific IC (ASIC) Or programmable logic array (PLA).

Image source 120 optionally further comprises object tracing logic 955.Object tracing logic 955 is configured as one The movement of the interested object of tracking in image series.For example, in some embodiments, selection logic 930 can be used in user To select to be required to obtain the subset of the image of its information or the aspect of image.The subset may include one or more pixels. Object tracing logic 955 is configured with automatic (computer based) image interpretation logic and occupies selected son to identify The special object of collection.The object can be people, motor vehicle, animal or any other object.The boundary of selected object or its Its pixel is optionally prominent in display 920 by object tracing logic 955.The protrusion moves in a series of images with object Object can be tracked and may include the characteristic for changing pixel.The protrusion is optionally moved together with the object on display. The one aspect of image can be the brand of objects within images, the film, the position of content of image that obtain image etc..Some Can be used in embodiment, in terms of image such as " shoes brand? ", " film? ", " performer? ", " position ", " cultivate? " equal texts It is designated as interested.This specify can add image in tagged raw requests and/or mentioned in upgrade request For.

In some embodiments, being from image source 120 to the image that image processing system 100 is transmitted includes short-sighted frequency sequence The part of a series of images of column.The system and method that describe elsewhere herein can be used to add in these video sequences Label.Adding a tagged advantage to video sequence is that (multiple) label can characterize in video specific dynamic occurs Make.For example, the label of figure skater can characterize specific jump (hooked hand two weeks etc.), in video than quiet Only preferably identified in image.Various embodiments include the specific limitation in the length of image sequence, for example, video must be few In 3,5,7 or 10 seconds.

Although embodiment shown in Fig. 9 includes electronic glasses, these embodiments can be adapted to be with ocular pursuit technology Any equipment, including mobile phone, video display monitor (such as computer or video screen), tablet computer, advertisement be aobvious Show device etc..For example, the embodiment of image source 120A shown in Fig. 9 includes having to be configured to determine that user is watching TV screen The TV of the ocular pursuit camera of which part of curtain.

Image source 120A optionally further comprises being configured as executing one or more steps for label image purpose Rapid image processing logic 960.Image processing logic 960 is optionally configured to local in image source 120A by executing these One or more steps and reduce the load on image processing system 110.For example, image processing logic 960 can be configured For execute the label of image initial step and then by the result of these initial steps be sent to image processing system 110 with For generating image tag.In some embodiments, image processing logic 960 can be completed some but need not be all images Label process.Image processing logic 960 includes hardware, firmware and/or the software that may be stored on the computer-readable medium.For example, Some embodiments include being specially configured to execute the processor 950 of the function of image processing logic 960 discussed in this article Example.

In some embodiments, image processing system 110 is configured as providing image processing logic 960 to image source 120. This is optionally via the application shop of such as apple " App Store ".If being applicable in, image is provided to the member of image source 120 Processing logic 960 is the optional step in various methods shown in this article.Processing logic 960 can be provided as further wrapping Include computer instruction or " applying (app) " of other logics discussed in this article (for example, the logic discussed about Fig. 9).

In some embodiments, image processing logic 960 is configured as the special characteristic in identification image.Feature identification packet Include the part for the feature for determining whether that the yes or no of the specified point in image gives type.The type of feature includes but is not limited to side Edge, corner, spot (blob) and ridge (ridge).It is generally characterized by the " interesting of the purpose of the content of the image for identification of image " or " useful " part.Image processing logic 960 can be configured as one executed in several different characteristic detection algorithms Or it is multiple.In some embodiments, image processing logic 960 is configured as based in available processing capacity and/or image Appearance is selected from several different algorithms.The example of known feature detection algorithm includes " Canny ", " Sobel ", " Harris& Stephens/Plessy ", " SUSAN ", " Shi&Tomasi ", " level curve curvature ", " FAST ", " Gauss-Laplace ", " Hai Sai (Hessian) determinant ", " MSER ", " PCBR " and " gray scale spot (grey-level blobs) ".These types Algorithm be performed on the computing device and other such algorithms will pair it would be apparent to one of skill in the art that.Feature The result of identification includes the identification of the special characteristic type of the specific position in image.This can at " feature descriptor " or It is encoded in " feature vector " etc..The structure of feature detection can also include indicating the identified level of confidence of feature at which Value.

In some embodiments, image processing logic 960 is configured to based on the box counting algorithm identified Image descriptor.Image descriptor is the visual signature of the content of image and including such as shape, color, texture and movement The characteristic of (in video).Image descriptor can be the part in particular descriptor domain, such as recognize with face recognition or currency Relevant descriptor.The export of image descriptor is typically based on characteristics of image.For example, the export of 3D shape description symbols can be based on Edge feature detected.Image descriptor can characterize the object that the one or more in image is identified.

The specific image features and image descriptor being used in specific embodiment depend on the specific image used identification Algorithm.Great amount of images identification algorithm is known in the art.In some embodiments, image processing logic 960 and/or image Processing system 110 is configured as first attempting to the identification of characteristics of image and the export of various types of image descriptors, and with It is selected from multiple alternative image processing algorithms based on the level of confidence that image descriptor at which is exported afterwards.For example, If the image descriptor in face recognition domain is exported with high-caliber confidence level, specific to the image of face recognition Processing Algorithm can be chosen so as to generate image tag from these image descriptors.

It is including in those of image processing logic 960 embodiment, the task of label image can be distributed in image source 120 Between image processing system 110.How task, which is assigned, can be fixed or can be dynamically.It is fixed in distribution Specific step is performed consistently with specific equipment in embodiment.It is point of step in dynamic embodiment in distribution With for example can be in response to the processing capacity on communication bandwidth, image type (static or video), image source 120A, image procossing Present load in system 110, the availability of the image audit person at destination 125, step are completed in image source 120 Confidence level, and/or the image descriptor data that are presented on image source 120A.Any combination of these factors is used to dynamic The distribution of ground allocation processing step.For example, if the export of image descriptor is with the confidence level of the low degree on image source 120A (relative to predetermined demand) occurs, then characteristics of image and/or image may be transferred into image processing system 110 for making With more strength or alternative image processing algorithm come deduced image descriptor.It compares, if the export of iamge description degree Occur on the image source 120A of the confidence level with sufficient degree, then the step is not usually required in image processing system It is performed on 110.

If image processing step is successfully executed on image source 120A by image processing logic 960, image and/or this The result of a little steps can be used I/O 945 and be transferred into image processing system 110.For example, in some embodiments, image and Image descriptor is both transferred into image processing system 110.Image descriptor can be used in automatically to add image and mark The trial of label may be provided to mankind image audit person at one or more destinations 125.Image descriptor can be by For identifying descriptor field and the domain is subsequently used to the member of destination 125 that selection image is sent to.For example, " machine The descriptor field of motor-car " can be used to the image audit that selection has motor vehicle speciality.Image is divided based on image descriptor Class is into can occur on image processing system 110 or image processing logic 960 in domain.

In some embodiments, the automated tag addition of image is attempted based on image descriptor derived from institute.Various In embodiment, this can be occurred using image processing logic 960 and/or automatic recognition system 152.Classification is alternately through comparing From have with derived image descriptor in the image in the library of different classes of associated image descriptor and occur.For example, knowing The image descriptor of other motor vehicle shape can be relevant to the image descriptor phase stored before " motor vehicle " classification Match.If the identification of suitable (the classification, range etc.) classification of the category can be automatically selected for the label of image enough.Example Such as, the image descriptor for matching those of classification " children's face " can generate label " faces of children " enough.

In general, image processing system 110 includes and the different classes of associated iamge description relative to image source 120A The bigger library of symbol.These libraries are optionally stored on the memory 135 or image source 120A or automatic of image processing system 110 In identifying system 152.The library for the image descriptor being stored in image source 120A is optionally based on previous using image source 120A The image of processing.For example, if multiple images from image source 120A are identified as having the label about currency and description Symbol, in the memory 135 that the library of currency domain/classification descriptor can be stored in image source 120A.These descriptors can With associated with the label of such as " 5 dollar bill " etc.When the new images of the identical set with descriptor are received, figure As processing logic 960 is optionally configured to automatically add label to image using associated label.Although descriptor library can To be received from image processing system 110, or it can be used and developed from the received image tag of image processing system 110, Label in above example is not dependent on the real time communication with image processing system 110.

In various embodiments, the data for characterizing the relationship between image descriptor and classification and/or label can be It is developed in image processing system 110, image source 120A, destination 125A and/or automatic recognition system 152.Once by developing, Data can be transmitted for improving and/or replenishing the library at any other equipment.

Although illustrated system shows master-slave architecture, image source 120 and destination 125 in alternative embodiments It is connected with end-to-end framework.In these embodiments, any combination of element shown in image processing system 110 can be by It is included in image source 120 and/or destination 125.One in image source 120 can be from another in image source 120 Image tag addition discussed in this article and processing task are executed on received image.

Figure 10 illustrates processing according to various embodiments of the present invention at least partially in the image on image source 120A Method.Method shown in Fig. 10 may include a series of different disposal steps executed on image source 120A.For example, those are related to And the step of image descriptor, is optionally performed on image processing system 110.

In receiving image step 1010, image is received by image source 120A.Image can be from being included in image source Camera in 120A is set from image source 120B, from network 115, from image processing system 110, from wireless device, from memory Standby, and/or analog is received.Received image be optionally an image in the image sequence for be formed video.

In identification feature step 1020, image processing logic 960 is used to identify the spy of the image in received image Sign.As discussed elsewhere herein, the method for identifying characteristics of image is well known in the art.Identification feature step 1020 can apply the one, two or more of these methods.The identification of feature optionally includes the institute of reflection feature identification The accuracy of estimation and/or the level of confidence of integrality.

It is optionally sending in characterization step 1030, identified characteristics of image is sent in identification feature step 1020 To image processing system 110.Feature can be sent together with or without associated image and can be sent out via network 115 It send.If sending characterization step 1030 to be included in this method, this method optionally next proceeds to carry out generating/receiving mark Step 1070 is signed, the label in the step 1070 for image is received from image processing system 110.Image processing logic 960, which are optionally configured to the level of confidence based on feature calculated in identification feature step 1020, executes transmission feature Step 1030.For example, step can be performed and image and feature are both sent if confidence level is lower than threshold value.

In optionally export descriptor step 1040, image processing logic 960 is used to from identification feature step Deduced image descriptor in identified characteristics of image in 1020.As discussed in this article, miscellaneous method is in ability It is all known in domain for deduced image descriptor.In some embodiments, export descriptor step 1040 includes using More than one method.Export may include the confidence level water for reflecting accuracy and/or integrality estimated derived from descriptor It is flat.Derived from descriptor type and content generally depend on used in (multiple) image edge sharpening arithmetic.

It is optionally sending in descriptor step 1050, the image descriptor being exported in export descriptor step 1040 It is sent to image processing system 110.Image descriptor can be sent together with or without associated image and can be through It is sent by network 115.If sending descriptor step 1050 to be included in this method, this method is optionally next proceeded to It carries out generating/receiving labelling step 1070, the label in the step 1070 for image is connect from image processing system 110 It receives.Image processing logic 960 is optionally configured to based on the characteristics of image being exported in export descriptor step 1040 Level of confidence, which executes, sends descriptor step 1050.For example, step can be performed and scheme if confidence level is lower than threshold value Picture and feature are both sent.

Optionally comparing in descriptor step 1060, the one or more being exported in export descriptor step 1040 Image descriptor is compared with the one or more image descriptors being locally stored.As elsewhere herein is discussed, this The image descriptor being locally stored a bit is associated with image category and/or image tag.Compare may include reflecting matched matter The calculating of the characteristic of amount.

In some embodiments, it sends descriptor step 1050 and compares descriptor step 160 the two and be performed.In the feelings Under condition, the processing of image descriptor can occur in 110 the two of image source 125A and image processing system.Similarly, one In a little embodiments, send both characterization step 1030 and export descriptor step 1040 be performed and characteristics of image system/ It is processed in equipment the two.

In appointment/reception labelling step 1070, the image tag for characterizing image is generated and/or receives.For example, such as Fruit image, characteristics of image or image descriptor have been sent to image processing system 110, then corresponding label can be from finger Image processing system 100 in group/reception labelling step 1070 is received.If exported in relatively descriptor step 1060 Descriptor and the descriptor that is locally stored between matching be found, then the image descriptor with local matched storage Associated label is acquired from local storage and is assigned to image.Identical image tag both can locally be referred to Group is also by local reception.Image tag is optionally generated using characteristics of image and/or descriptor, for example, without image procossing System 110 receives actual image.

In some embodiments, appointment/reception labelling step 1070 includes that will classify to be assigned to image, to image procossing system System 110 sends image and classification, and is received back corresponding label from image processing system 110.Label can be used shown in Fig. 4 Method it is identified.Classification can be used by image processing system 100 to use automatic recognition system 152 and/or in destination Human reviewer at 125A generates label.

Assigned and/or reception label (and/or other results) is provided in providing result step 455, such as herein In other places discussed.

Figure 11 illustrates the method based on image descriptor processing image according to various embodiments of the present invention.This method It is performed on one in image source 120 with city.In the exemplary embodiment, step 1010,1020,1040 and 1060 are held Row, it is such as described elsewhere herein.In classification image step 1110, image being processed is based on describing in export Image in symbol step 1040 on the one or more image descriptors being exported and one be previously stored in image source 120 Matching between descriptor.The classification or multiple classifications for being assigned to image be and the matched image descriptor that was previously stored Associated classification or multiple classifications.

In sending step 1120, image and it is assigned to the classification of the image or multiple classifications are sent to image procossing System 110.The image is at that as described be treated to generate elsewhere herein is assigned to the image tag of image. The processing optionally includes use classes or multiple classifications to select mankind's image audit or auxiliary automatically label image.

In receiving labelling step 1130, the label for being assigned to image is executed at this receives image step 1010 One in image source 120 is received.Label, which is subsequently rendered in, to be provided in result step 455.

Figure 12 illustrates the method using feedback processing image according to various embodiments of the present invention.This method is optionally It is performed on image source 120A and changes including multiple communications between image source 120A and image processing system 110 Into the label of image.In providing image step 1210, image is provided to image processing system 110 from image source 120A.

In receiving the first response of step 1220, the first response is received from image processing system 110.The response can wrap Include one or more labels.In feedback step 1230 is provided, about received image tag feedback from image source 120A It is provided to image processing system 110.The feedback is optionally manually entered and can wrap by the human user of image source 120A Upgrade request is included, as discussed elsewhere herein.Feedback may include include to one or more of received label The correction of a label.For example, feedback may include the instruction that a label in label does not indicate image.Feedback may include figure The classification of picture.

It is optionally receiving in the second response of step 1240, the second response is received from image processing system 110.Second sound The feedback for providing and providing in feedback step 1230 should be generally used in be generated.In one example, it is contemplated that the figure of toy car Picture, the first response include label " vehicle ", and feedback includes term " toy " and the second response includes that label " takes the super lorry of snow (Fisher-Price Superwagon)".Method shown in Figure 12 is optionally used to improve the accuracy of image tag.

Figure 13 and Figure 14 illustrate according to various embodiments of the present invention provide image tag based on image descriptor Method.In Figure 13, image descriptor is used to generate the figure that the source of image descriptor is then transferred on calculating network As label.In Figure 14, image descriptor is used to determine the destination 125 for image.Shown in figs. 13 and 14 Method is optionally performed with the method illustrated elsewhere herein in combination.For example, the step of these methods can with Those are combined shown in Fig. 4.

Particularly, referring to Fig.1 3, in receiving descriptor step 1310, characterize one or more iamge descriptions of image Symbol is received at image processing system 110.These image descriptors are optionally received but without associated images.Only receive Descriptor bandwidth usually more smaller than reception image request.Image descriptor is optionally via network 115 from image source 120A quilt It receives and is generated using method shown in Figure 10 or 11.

In relatively descriptor step 1320, the received image descriptor of institute with previously deposited at image processing system 110 One or more image descriptors of storage (for example, being stored in memory 135) compare.This is relatively made is connect with determination Whether any in the descriptor of receipts matches stored descriptor.The descriptor and one or more image tags stored And/or classification is stored in association.For example, a set of the descriptor stored can be related to image tag " Oak Tree " Connection.

In obtaining labelling step 1330, in response to the matching one between the descriptor received and the descriptor stored A or multiple images label is received.Acquired image tag be it is associated with the matched set of institute those.

Provide labelling step 1340 in, acquired image tag be provided back received descriptor source, for example, To image source 120A.They can be presented to user at that or be located in other ways as described elsewhere herein Reason.

Figure 14 illustrate image and characterize the image data at image processing server 110 processed method. In receiving image and data step 1410, image and the data for characterizing the image are connect at image processing server 110 It receives.The data for characterizing image for example may include the classification for characterizing the image descriptor or image of image.Image and spy Signization data are optionally received from image source 125A.Receiving image and data step 1410 optionally is to receive image step 410 and receive source data step 420 embodiment.

In determining destination step 1420, the destination for image is determined based on the data for characterizing image.Mesh Ground can be one in destination 125 and/or automatic recognition system 152.For example, if the data for characterizing image include Specific classification and identified destination can be and have the associated mesh of mankind's image audit of speciality in the classification Ground 125 in one.Determining destination step 1420 optionally is the embodiment of determining destination step 465.

In putting up image step 1430, image and optionally classification are transferred into identified destination.It is receiving In labelling step 1440, one or more image tags are received.Image tag is based on image and is chosen so as to characterize image. In providing labelling step 1340, image tag is provided to the source of image, for example, image source 125A.Put up image step 1430 be optionally the embodiment for putting up image step 470.

Figure 15 illustrates the method for sorting by priority image tag addition according to various embodiments of the present invention.At this In a little methods, image grading device 190 is used to image assigned priority and the priority be used to determine image how by Label is added, if its label to be added.In receiving image step 410, image is connect in image processing system 110 It receives, as discussed elsewhere herein.Image may come from one in image source 120, and can be by being directed to Image crawls (crawling) webpage and is received.In some embodiments, one or more of image source 120 includes being configured For crawl website and from these websites obtain image logic.The information being received together with image may include about from wherein Obtain the data of the webpage of image.For example, image can with from webpage metadata and text, instruction webpage mostly often added Carry the data of (viewing), the URL of webpage, and/or picture priority can according to its determined any other data together by It receives, as other positions herein are discussed.

In assigned priority step 1520, image grading device 190 is used to automatically assign a priority to and be received Image.Priority is optionally by from 1 to 100 numerical value, by subtitle grade or similar expression.Priority optionally implys that figure (sequence) classification of picture.As described elsewhere herein, priority can be determined based on various factors.

In determining processing step 1530, the method for image addition label (processing) is determined.Determination is based on image The priority assigned.In some embodiments, the image with lowest priority is not (addition label) processed. The method of label addition includes automatic addition label and/or adds label manually by human reviewer, such as elsewhere herein It is discussed.

In the step 1540 that optionally tags automatically, image is added label using automatic recognition system 152.It is automatic to add The method for the label addition that labelling step 1540 optionally determines in determining processing step 1530 does not include automatic identification system In the embodiment of system 152 used.Automatically the step 1540 that tags optionally is performed prior to assigned priority step 1520.Example Such as, automatic recognition system 152 can be used by label in image, and the level of confidence for daring to the label automatically generated can be with After be used in assigned priority step 1520 to determine and add tagged priority for manual (mankind).If automatically generated The confidence level of label be high, then can be set low for adding tagged priority manually, and if set Reliability is relatively low, then for add manually tagged priority can be set to it is relatively high.

In the step 1550 that optionally tags manually, image is sent to one in destination 125 for by people Class auditor adds label.Image can be together with the label for using automatic recognition system 152 and/or various other information to generate It is sent, it is such as described elsewhere herein.The step 1550 that tags manually may include Fig. 6 to step shown in Fig. 8 In any step.

In optional auction labelling step 1560, advertisement is assigned to image for showing on webpage.The webpage It is optionally the webpage that image is obtained at it in receiving image step 410.It is competing as the request for webpage is received Labelling step 1560 is clapped optionally to be perfomed substantially in real time with said storing the sensor signals.At the moment, (multiple) label for being assigned to image can be by auction To being ready to provide a side of the maximum remuneration for advertisement to be placed in above the image or side.Auction labelling step 1560 is optional Ground is performed using ad system 180 and auction process can be by such as GoogleEtc third party institute Management.

In the step 1570 that optionally tags again, image is added label again.Again the step 1570 that tags can To include the analysis for being assigned to (multiple) advertisement of image and being mostly often clicked compared with the clicking rate of expectation.For example, if The advertisement for being assigned to image based on the first label is not clicked with expected clicking rate, then label may not be the image Optimal expression.The image can be attempted to improve the clicking rate for the advertisement for being added label by addition label again.Again mark-on Label step 1570 may include any one of label addition method disclosed herein, such as be discussed about Fig. 6-8 and 15 Those of method.Again it is not optimal knowledge that the step 1570 that tags, which can be used from the label that the first label generates,.

Method as shown in Figure 15 can also be applied to image sequence, for example, video.Image sequence can be present in In browser or use various alternative applications.For example, video can be from the website of such as youtube.com etc or from such as The stream service of Netflix, Comcast cable television, live telecast, Ruku or Hulu etc be provided to image source 120 at Member.The image being used to determine in video whether should be added manually tagged factor include: video mostly often by viewing and/ Or the estimated value for the label expected.The automatic recognition system 152, dialogue in video, adjoint can be used in the label expected Indicated by automatic audit of the text (for example, description, subtitle or title), and/or analog of video by image.Advertisement can be In the video that the beginning or end of image sequence are attached or are engaged in image sequence.Thus, advertisement and video can phases Associatedly presented together.Advertisement may include the part for being placed in image sequence, be usually including adding tagged image Covering on part.

Multiple embodiments specifically illustrate and/or describe herein.It will be appreciated, however, that be that modifications and variations are above Training centre covering and within scope of the appended claims without departing from its spirit and the range that is intended to.For example, herein Disclosed image is optionally the part of the video sequence of video.Audio input can be used in destination in mankind's image audit Image tag is provided at 125.The audio that is placed on destination 125 can be used to text conversion logic and/or figure in audio input It is converted into text in real time as processing system 110.Image tag is optionally handled by spell check logic.As made herein , term " real-time " means no non-essential time delay, user easily waited for.System described herein System and method are optionally used to add label, such as music or dialogue to audio content.The audio content can be video Part is associated with image in other ways.In some embodiments, audio content is automatically converted to text and this article Originally it is used to auxiliary and label manually or automatically is added to image.From audio content generate text can with herein for Those similar modes described in the text found in webpage including image are used to auxiliary and add label to image.

Embodiments described herein illustrates the present invention.Since embodiments of the invention are described referring to explanation, The various modifications or adaptation of described method and/or specific structure can become aobvious and easy to those skilled in the art See.It relies on the teachings of the present invention and has advanced into all these modifications, adaptation or change of this field by its these introduction Change is considered to be within the spirit and scope of the present invention.Thus, these descriptions and attached drawing are not construed as the meaning limited Justice, because it should be understood that the present invention is not restricted to embodiment described.

Present document relates to computing system (for example, image processing system 110, image source 120 and destination 125) may include Integrated circuit, microprocessor, personal computer, server, distributed computing system, communication equipment, the network equipment similar are set It is standby, and above various combinations.Computing system can also include volatile row and/or non-volatile line storage, such as deposit at random Access to memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), magnetic medium, light are situated between It matter, nanometer medium, hard disk, compact disk, digital versatile disc (DVD), and/or is configured for such as storing in the database Other equipment of analog or digital information.The various examples of logic indicated above may include hardware, firmware or be stored in meter Software on calculation machine readable medium, or combinations thereof.Computer-readable medium as used herein clearly eliminates paper.It herein refers to The step of computer of method out executes may include the one group of instruction of storage on a computer-readable medium, upon being performed So that computing system executes the step.Be programmed to be according to the computing system of the instruction execution specific function from program software For executing the specific use computing system of those specific functions.When executing those specific functions by specific use computing system The data of operation are at least electrically stored in the buffer of computing system, will using the change every time to stored data Specific use computing system physically changes from a state to next state.The logic being discussed herein may include being stored in Hardware, firmware, and/or software on computer-readable medium.It is special to generate that the logic can be performed in the electronic device Purposes computing system.

Claims (18)

1. a kind of image processing system, comprising:

I/O is configured as transmitting image and image tag on a communication network；

Image grading device is configured to determine that for adding tagged priority to described image；

Destination module is configured to determine that the destination sent the image to to be used for by human reviewer to the figure As being audited manually；

Image puts up module, is configured as described image being posted to the destination；

Auditing module is configured as receiving the manual audit of described image from the destination, and the manual audit includes Characterize one or more image tags of the content of described image；

Memory is configured as one or more of image tags being stored in data structure；

Automatic identification interface is configured as described image being sent to automatic recognition system and receive from the automatic recognition system The audit that the computer of described image generates, the audit that the computer generates include one of the content of characterization described image Or multiple images label, the audit that the computer generates further include indicating audit that the computer generates, correctly spy The confidence metric of the confidence level of one or more of image tags of the content of signization described image；And

Microprocessor is configured as at least executing described image clasfficiator, and wherein described image clasfficiator is additionally configured to be based on The confidence metric determines the priority.

2. system according to claim 1, wherein described image clasfficiator is configured to based on the computer One or more of image tags of the audit of generation determine the priority.

3. system according to claim 1, wherein described image clasfficiator is configured to send out by described image It send to the automatic recognition system or be sent between the destination and makes a choice, the selection is based on the priority.

4. system according to claim 1, wherein the destination module is configured to based on the priority Determine the destination.

5. system according to claim 1, wherein described image clasfficiator is configured to based on the priority Label is being added to described image and is not being made a choice between described image addition label.

6. a kind of image processing system, comprising:

I/O is configured as transmitting image and image tag on a communication network；

Image grading device is configured to determine that for adding tagged priority to described image；

Destination module is configured to determine that the destination sent the image to to be used for by human reviewer to the figure As being audited manually；

Image puts up module, is configured as described image being posted to the destination；

Auditing module is configured as receiving the manual audit of described image from the destination, and the manual audit includes Characterize one or more image tags of the content of described image；

Memory is configured as one or more of image tags being stored in data structure；And

Microprocessor is configured as at least executing described image clasfficiator, and wherein described image clasfficiator is additionally configured to be based on Webpage including described image mostly often determines the priority by viewing.

7. system according to claim 1, wherein described image clasfficiator is configured to based on including the figure The type of the webpage of picture and determine the priority.

8. system according to claim 1, wherein described image clasfficiator is configured to be based on being assigned to institute The advertisement for stating image is mostly often clicked and determines the priority.

9. a kind of image processing system, comprising:

I/O is configured as transmitting image and image tag on a communication network；

Image grading device is configured to determine that for adding tagged priority to described image；

Destination module is configured to determine that the destination sent the image to to be used for by human reviewer to the figure As being audited manually；

Image puts up module, is configured as described image being posted to the destination；

Auditing module is configured as receiving the manual audit of described image from the destination, and the manual audit includes Characterize one or more image tags of the content of described image；

Memory is configured as one or more of image tags being stored in data structure；And

Microprocessor is configured as at least executing described image clasfficiator, and wherein described image clasfficiator is additionally configured to be based on The mark in the source of described image determines the priority.

10. system according to claim 1, wherein described image clasfficiator be configured to based on be included in Text and the determining priority on the identical webpage of described image.

11. system according to claim 1, wherein described image clasfficiator be configured to based on be included in Metadata and the determining priority on the identical webpage of described image.

12. system according to claim 1, wherein described image clasfficiator is configured to based on including the figure The classification of the webpage of picture and determine the priority.

13. system according to claim 1, wherein described image clasfficiator is configured to based on including the figure The quantity of the webpage of picture and determine the priority.

14. system according to claim 1 further comprises ad system, the ad system is configured as based on institute It states one or more labels and assigns advertisement to described image.

15. system according to claim 1, wherein described image puts up module and is configured to mention described image It is supplied to the destination, while described image is automatically recognized system audit.

16. system according to claim 1 further comprises memory, the memory is configured as in data structure Middle storage image and the label for indicating described image.

17. system according to claim 1 further comprises contents processing logic, the contents processing logic is configured To extract described image from image source, wherein described image source is picture sharing website or social networking website.

18. system according to claim 1, wherein the destination is a destination in multiple destinations, each Destination is associated from different mankind image audit persons.