The application is the beauty for entitled " image procossing " with application number 14/267,840 submitted on May 1st, 2014
The part of state's non-provisional application continues, and then requires the preferential of the provisional application 61/956,927 submitted on May 1st, 2013
Power;The priority and right of the following U.S. Provisional Patent Application of the application further requirement:
It is submitted on April 4th, 2014 and there is " visual search " of application number 61/975,691;
It is submitted on April 7th, 2014 and there is " the visual search advertisement " of application number 61/976,494;
It is submitted on May 1st, 2014 and there is " image procossing " of application number 61/987,156;
It is submitted on July 31st, 2014 and there is " the real-time target choosing in image procossing of application number 62/031,397
It selects ";
It is submitted on October 27th, 2014 and there is " the distributed image processing " of application number 62/069,160;And
It is submitted on November 25th, 2014 and there is " the selective image processing " of application number 62/084,509.
All above patent applications are incorporated herein by reference.
Specific embodiment
Fig. 1 illustrates image processing systems 110 according to various embodiments of the present invention.Image processing system 110 is matched
It sets for adding label to image and may include one or more distributive computing facilities.For example, image processing system 110
It may include one or more servers positioned at geographically different location.Image processing system 110 is configured as via network
115 are communicated.Network 115 may include a variety of communication networks, such as internet and/or cell phone system.Network 115
It is typically configured as transmitting data using standard agreements such as IP/TCP, FTP.The image handled by image processing system 110
It is received from image source 120 (separate marking 120A, 120B etc.).Image source 120 may include be connected to internet and/or
The calculating source of Personal mobile computing equipment.For example, image source 120A, which can be, is configured to supply social network sites or picture sharing
The network server of service.Image source 120B can be smart phone, camera, wearable camera, electronic glasses or other portable
Image capture device.Image source can be by uniform resource locator, Internet protocol address, MAC Address, cell phone identifier
And/or analog is identified.In some embodiments, image processing system 110 is configured as receiving from great amount of images source 120
Image.
The part of the image tag executed from image processing system 110 include to destination 125 (be respectively labeled as 125A,
125B etc.) send image.Destination 125 is at the calculating equipment of mankind image audit person and usual geographically-distant image
Reason system 110.Destination 125 includes at least display and data input device, such as touch screen, keyboard and/or microphone.Example
Such as, destination 125 can be the building different from image processing system 110, city, state and/or country.Destination 125 can
To include personal computer, tablet computer, smart phone etc..In some embodiments, destination 125 includes being specifically configured to
Promote (calculating) application of the audit of image.The application is optionally provided to destination 125 from image processing system 110.?
In some embodiments, image processing system 110 is configured for mankind's image audit person from 125 login user account of destination.
Destination 125 is usually associated with individual auditor and can be interconnected fidonetFido address, MAC Address, login sessions mark
Symbol, cell phone identifier and/or analog are identified.In some embodiments, destination 125 includes audio to text conversion
Device.Image processing system 110 is sent in the image tag data that several destinations 125 provide by mankind's image audit person.
Image tag data may include text image label, the audio data including language label, and/or such as upgrade request or
The non-label information of inappropriate (specific) material mark etc.
Image processing system 110 includes being configured for and the I/O of External system communication (input/output) 130.I/O
130 include router, interchanger, modem, firewall, and/or analog.I/O 130 is configured as receiving from figure
The image of image source 120 sends image to destination 125, receives the label data from destination 125, and optionally to figure
Image source 120 sends image tag.I/O 130 includes communication hardware and optionally includes application programming interfaces (API).
Image processing system 110 further comprises memory 135.Memory 135 includes being configured for such as image, figure
The hardware stored as the non-transitory of the data of label, computations and other data discussed in this article etc.For example, depositing
Reservoir 135 may include random access memory (RAM), hard disk drive, light-memory medium and/or analog.Memory 135
It is configured as by using specific data structure, index, file structure, data access routines, security protocol and/or analog
And store specific data (as described herein).
Image processing system 110 further comprises at least one processor 140.Processor 140 is such as electronics micro process
The hardware device of device etc.Processor 140 is configured as being loaded into posting for processor 140 by hardware, firmware or by software instruction
Storage and execute specific function.Image processing system 110 optionally includes multiple processors 140.Processor 140 is configured as holding
Various types of logics that row is discussed herein.
It is initially stored in image queue 145 by the received image of image processing system 110.Image queue 145 is to deposit
Store up the ordered list of the pending image in storage list.The image being stored in image queue 145 is usually and for drawing
It is stored in association with the image identifier of image and can have different priority.For example, from picture sharing website
Received image can have than from the lower priority of the received image of smart phone.In general, relative to some other
Those of the image tag that purpose uses image, for requestor's image tag etc. to be received for indicating image in real time
Those images are given a higher priority.Image queue 145 is optionally stored in memory 135.
In image queue 145, image by optionally with image identifier or index and other be associated with each image
Data store in association.For example, image can be associated with about one source data in image source 120.Source data
It may include the geography information of such as GPS coordinates, street and/or city name, postcode, and/or analog.Source number
According to may include Internet protocol address, uniform resource locator, account name, smart phone identifier, and/or analog.
Source data may further include the priority about the language, request that use on the member of image source 120, searching request
The information of (for example, request that internet hunt is carried out based on the image tag generated from image), and/or analog.
In some embodiments, the image within image queue 145 is associated by the instruction of the specific subset with image
Ground storage, the subset generally include the project of special interests.For example, the requestor of image tag can be to acquisition about image
The image tag of the content of specific subset is interested.When image includes several objects, this can occur.In order to illustrate consideration exists
There is the image of the hand of ring on one finger, user may want to identify the ring to be interested specific region.Of the invention
Some embodiments include being configured such that the object on display of the user by clicking object or touch image source 120B is specified
The application of interested specific project.This, which is specified, usually occurs before sending image to image processing system 110.
If image with the specific subset of image there is the instruction of particular importance to store in association, image mark
Note logic 147, which is optionally used to mark, to be placed on image.Label is arranged to protrude specific subset.The label can lead to
It crosses modification image to correspond to the pixel of subset and make, and the label allows mankind's image audit person to be absorbed in the son marked
On collection.For example, being posted in image to before one or more destinations 125, image can be labeled with rectangle or circle.Example
Such as, the subset of prominent image or the object within image may include to object or subset application filter (filter, filter),
And/or change the color of object or subset.In alternative embodiments, image tagged logic 147, which is included in, is configured to one
Within the application executed on a or multiple images source 120 or destination 125.Image tagged logic 147 includes being stored in nonvolatile
Hardware, firmware, and/or software on property computer-readable medium.As being discussed elsewhere herein, mark logic
147 are optionally configured to be generated with image and be arranged in label on image in real time.
In some embodiments, mark logic 147 is configured with detected characteristics of image within image to know
The special object that can be labeled.The detection of characteristics of image is discussed in elsewhere herein and is occurred in visitor
A part of image procossing in the side of family (for example, on image source 120A).For example, the feature at such as edge etc can be used
The processor of image source 120A is detected.These features can be used in the object of prominent detection and first then also from figure
Image source 120A is sent to image processing system 110, they are subsequently used to generate the figure of the part as processing image at this
As descriptor.In the method, automatically processing for image is dispensed on image source 120A, image processing system 110 and/or automatic
Between identifying system 152.
Under the control of processor 140, the image in image queue 145 is provided to automatic identification interface 150.Image
Thus it is provided as the function of their priority and position in image queue 145.Automatic identification interface 150 includes being matched
It is set to image, and optionally by the logic of any data transmission associated with the image to automatic recognition system 152.It should
Logic is stored in hardware, firmware, and/or software on computer-readable medium.Automatic identification interface 150 may further be by
It is configured to receive the audit that the computer of the image from automatic recognition system 152 generates, the audit which generates includes
Identify one or more image tags of the content of image.In some embodiments, automatic identification interface 150 be configured as via
Network 115 transmits image and data with the format for being suitable for the application programming interfaces (API) of automatic recognition system 152.Some
In embodiment, automatic recognition system 152 is included in image processing system 110, and automatic identification interface 150 includes for example
System in operating system or on local area network is called.
Automatic recognition system 152 is configured as audit image without the calculating being manually entered based on each picture
Machine automatic system.The output of automatic recognition system 152 is the image audit of computer generation (for example, being not needed upon each figure
The audit that generation is manually entered of piece).The preliminary example of this system is well known in the art.For example, with reference to
Kooaba, Clarifai, AlchemyAPI and Catchoom.Automatic recognition system 152 is typically configured as based in image
The shape, character and/or the pattern that detect automatically identify the object in two dimensional image.Automatic recognition system 152 optionally by
It is configured to execute optical character identification and/or bar code paraphrase.In some embodiments, automatic recognition system 152 and the prior art
System the difference is that automatic recognition system 152 be configured to supply computer generation audit, the audit be based on it is (more
It is a) image subset indicates and/or image source data, and other places herein are discussed.
Whether automatic recognition system 152 is optionally configured to the determining copy from the different received images of image source
Through being added label.For example, identical image can be included in multiple webpages.If image is from first in these webpages
A webpage is extracted and is added label, and automatic recognition system 152 can recognize the image and be added label and to figure
The example that each of picture is found automatically assigns these labels.Recognisable image has been added label and has optionally included and will scheme
The data of picture, the part of image or expression image are compared with the database for the image for adding label before.Image can it
It is preceding either automatically or manually to be added label.
In addition to one or more image tags, the audit generated by the computer that automatic recognition system 152 generates is optionally
Including indicating that one or more image tags correctly identify the measurement of the confidence level of the confidence level of the content of image.For example, main
If the audit that the computer of character or the image for the shape that may be readily recognized generates can have than by being abstracted or not knowing
The bigger confidence metric of the audit that the computer of the image of shape composition generates.Different automated graphics identification systems can produce
The raw different confidence levels for different types of image.Automatic identification interface 150 and automatic recognition system 152 are optionally
In embodiment in the automatic identification executed by third party.
Image processing system 110 further comprises auditor pond 155 and the auditor for being configured as management auditor pond 155
Logic 157.Auditor pond 155 includes the pond (for example, group or set) of mankind's image audit person.Each mankind's image audit person
It is usually associated with the different members of destination 125.For example, each member in the different members of destination 125 can be known
It is logged in by different mankind image audit person operations or by the account of different mankind's image audit persons.Memory 135 is by optionally
It is configured to storage auditor pond 155.In some embodiments, the mankind's image audit person's quilt being included in auditor pond 155
It is classified as " enlivening " and " inactive ".For purposes of this disclosure, active mankind image audit person is considered as currently
Image tag is provided or has indicated that they are ready to provide the mankind image audit person of image tag with minimal time delay.Both it is wrapping
It includes in the active also embodiment including sluggish mankind's image audit person, active auditor is provided image to audit.
The quantity of active auditor can become appropriateness in real time in response to the requirement to image audit.For example, mankind's image audit
The classification of person can be changed into actively based on the quantity for not auditing image in image queue 145 from inactive.It is sluggish to examine
Core person is not yet active auditor, and the audit of image has been allowed to fail and/or indicate that they can not audit image.It does not live
The auditor of jump can request to become active auditor.When needs are additional enlivens mankind image audit person, this has been made
The sluggish auditor of kind request can be reclassified as active mankind image audit person.Which sluggish audit
Person, which is reclassified, to be alternatively depended on auditor's score as the determination of active auditor and (begs in other places herein
By).
Auditor's logic 157 is configured as management auditor pond 155.The management optionally includes mankind image audit person work
For active or sluggish classification.Start to audit for example, auditor's logic 157 can be configured as monitoring mankind's image audit person
The time of image, and if predetermined maximum audit time (herein referred as picture failure time), by mankind image audit person's
Classify inactive from actively changing into.In another example, auditor's logic 157 can be configured as calculating for mankind's image
The audit score of auditor.In some embodiments, audit fractional marks are examined by the image that specific mankind's image audit person executes
Integrality, speed and/or the accuracy of core.The audit score can based on audit number and accidental test image be calculated or
Change.For example, these test images can be placed in the image audited before by different mankind image audit persons
In queue 145.The audit score can also be the function of monetary cost associated with mankind image audit person.Auditor's logic
157 include hardware, firmware, and/or the software being stored in non-transitory computer-readable medium.In some embodiments, it examines
Core person score is manually determined by mankind coordinator.These mankind coordinators audit image and are assigned to by mankind image audit person
The label of these images.The statistic sampling of the image of audit is optionally sent to coordinator and coordinator will be to the mark of image
Label assign score.The score is optionally used in determining auditor's score.
In some embodiments, auditor's logic 157 is configured as monitoring the state of mankind image audit person in real time.Example
Such as, auditor's logic 157, which can be configured as, monitors the individual word or keystroke inputted by the auditor in destination 125A
Input.The monitoring can be used to determine which auditor actively audits image, which auditor just completes to scheme
The audit of picture and/or which auditor do not provide label input also after several seconds or minute.Use the label list of audio frequency apparatus
The input of word can also the person's of being reviewed logic 157 monitored.
In some embodiments, the member in auditor pond 155 and mankind's image audit person have speciality or special knowledge
Profession is associated.For example, auditor can be the expert of automotive field and associated with the profession.Other professions may include skill
Art, plant, animal, electronics, music, food medical professionalism, clothes, dress accessory, fine work etc..Such as other local institutes herein
It discusses, the profession of auditor can be used to select the audit between initial manual period under review and/or during auditing upgrading
Person.
Audit score associated with mankind image audit person and/or profession are not optionally reviewed person's logic 157 and are used to really
It is settled that need additional which sluggish auditor when enlivening auditor to be changed to active.Auditor's logic 157 includes depositing
Store up hardware, firmware, and/or software in non-transitory computer-readable medium.
Image processing system 110 further comprises destination logic 160.Destination logic 160 is configured to determine that will be to
It sends image with one or more destinations (for example, destination 125) for auditing manually.Each destination 125 and examine
Corresponding mankind's image audit person in core person pond 155 is associated.It is optionally based on by the determination that destination logic 160 is made in institute
The characteristic of the mankind image audit person of determining destination.Destination can be calculating equipment, the intelligence of mankind image audit person
Phone, tablet computer, personal computer etc..In some embodiments, destination is browser, and auditor is from the browsing
Device logs in image processing system 110.In some embodiments, determine that destination includes a purpose in determining destination 125
MAC Address, Session ID, Internet protocol and/or the uniform resource locator on ground.Destination logic 160 includes being stored in
Hardware, firmware and/or software in non-transitory computer-readable medium.
In general, destination logic 160 be configured to determine that with such as by auditor's logic 157 determination it is active rather than not
The active associated destination 125 of mankind image audit person.Destination logic 160 is usually additionally configured to based on auditor's
Audit score determines destination 125.For example, there is higher fractional for compared with the auditor with lower auditor's score
Those of auditor can be selected for the audit of higher priority.Thus, the determination of the member of destination 125 can be based on
Both auditor's score and image audit priority.
In some embodiments, destination logic 160 is configured as the movable reality of input based on associated auditor
When monitoring to determine one or more members in destination 125.As other places herein are discussed, which can be with
It is executed by auditor's logic 157, and may include by the detection of the individual word or keystroke of mankind image audit person input.?
In some embodiments, destination logic 160, which is configured as having compared, knocks in figure on mankind's image audit person currently keyboard
As the destination 125B of label, favor selection has just completed the destination 125A of the audit of image in mankind's image audit person.
In some embodiments, destination logic 160 is configured with via the received image of automatic recognition system 152
Label is to determine one or more members of destination 125.For example, if the image tag of " automobile " is via automatic identification interface
150 are received, then the information can be used to select and have mankind's image in automotive field profession in destination logic 160
The member of the associated destination 125 of auditor.
The value of image audit can also be considered in the selection for the destination audited manually.For example, the figure of high level
As audit can lead to with the relatively high determination for auditing the associated destination of the mankind image audit person of score, while compared with
The image audit of low value can lead to destination associated with having the relatively low audit mankind image audit person of score
It determines.In some embodiments, for some image audits, destination logic 160 is configured as selecting between destination 125
In order to minimize the time for needing to audit image, such as minimum time until the image tag audited manually is provided to net
Network 115.
Destination logic 160 is optionally configured to determine multiple destinations for single image.For example, the first destination
Can be selected, and followed by upgrade request, the second destination can be determined.Upgrade request can come from and the first mesh
The associated mankind's image audit person in ground or come from image source 120A.In some embodiments, destination logic 160 is configured
For the multiple destinations of determination, image will be concurrently posted to multiple destination.For example, it is each from different mankind's images
The associated two, three or more destination of auditor can be determined and same image is concurrently posted to institute
There is determining destination.As used in the present context, " parallel " means that image is receiving audit from the first destination
The second destination is at least posted to before any part.
In various embodiments, there are a variety of causes to allow two or more destinations by 160 institute of destination logic
It determines.For example, the mankind image audit person for having particular professional can be requested to the request of the audit of upgrading.Show with reference to automobile
Example, can lead to the upgrade request to more information by the image that label has " white vehicle " label first.Destination logic 160 can be by
It is configured to then select and have in mankind's image audit person of automotive field profession (for example, " 1976 Ford lattice drawing can be provided
The auditor of Na Da " label) associated destination.Upgrade request instruction image, which is subject to, further to be audited, for example, image
Demand is further audited or can be benefited from further audit.Upgrade request can be by such as flag, order or data value
Deng computing object represented by.
When the manual audit of image expends the too many time, it may be desired to which another example of the second destination occurs.In general,
The label addition of image should occur within the period of distribution, and otherwise the audit is considered failing.The period of distribution is optional
Ground is the function of the priority of image audit.Those audits for being intended to occur in real time can have compared lower priority audit and
Say the shorter period.If the audit of image is failed, image processing system 110 is optionally configured to patrol to by destination
It collects the associated additional mankind's image audit persons in destination that 160 determine and images is provided.
When the first human reviewer makes upgrade request, it may be desired to which another example of the second destination occurs.For example, leading
The request of the upgrade audit of the label of " automobile " is caused to may be from providing the mankind image audit person of label " automobile ".Although the example
It is to simplify, other examples may include the image of more esoteric theme, the integrated circuit such as encapsulated.
Image processing system 110 further comprises that image puts up logic 165, is configured as the figure that will be used to audit manually
As being posted to the destination 125 determined by destination logic 160.It puts up and generally includes that image is sent to one via network 115
A or multiple destinations 125.In various embodiments, image puts up logic 165 and is further configured to provide to destination 125
Information associated with image.For example, image put up logic 165 can put up image and image subset instruction (for example,
Subset identification), the image that is marked by image tagged logic 147, identify image source information (for example, herein otherly
The source data just discussed), the priority of the audit of image, picture failure phase, location information associated with image and/or similar
Object.As other places herein are discussed, source data may include uniform resource locator, global positioning coordinates, longitude
Share account and/or analog with latitude, account, Internet protocol address, social account, picture.
In some embodiments, image is puted up logic 165 and is configured as in the about same time to more than one purpose
Ground 125 provides the image for auditing manually.For example, image can be concurrently provided to destination 125A and destination
125B.For example, " parallel delivering " means that before being received back label information from both destination 125A and 125B, image is mentioned
It is supplied to these destinations 125.
In some embodiments, image puts up logic 165 and is configured as receiving image tag from automatic recognition system 152
Forward direction one or more destination 125 provide for the image audited manually.Alternatively, in some embodiments, image
Logic 165 is puted up to be configured as waiting before putting up image to one or more of destination 125 until for image
The audit that computer generates is received from automatic recognition system 152.In these embodiments, computer generate audit (including
One or more of image tag) be optionally also posted to destination 125 associated with image.
Image puts up logic 165 and is optionally configured to put up the identifier of image and image together.Image, which is puted up, patrols
Collecting 165 includes hardware, firmware and/or the softwares being stored in non-transitory computer-readable medium.
Image processing system 110 further comprises the audit logic manually and automatically audited for being configured as management image
170.The management includes the progress of monitoring audit, receives the audit from automatic recognition system 152 and/or destination 125.It is connect
The audit of receipts includes the image tag discussed such as other places herein.In some embodiments, 170 quilt of logic is audited
The image for being configured to a destination of the measurement control of confidence level into destination 125 is puted up.The meter of confidence level
Show the correct confidence level of one or more image tags being received.The one or more image tag can be known from automatic
A destination in other system 152 and/or destination 125 is received.For example, in some embodiments, if by knowing automatically
The confidence level of the image audit of other system 152 is greater than predetermined threshold, then audit logic 170 can determine the manual audit of image
It is unnecessary.The predetermined threshold can be the value of image audit, image audit priority, available destination 125
Quality and quantity, and/or analog function.Audit logic 170 includes being stored in non-transitory computer-readable medium
On hardware, firmware, and/or software.
In some embodiments, if image is concurrently sent to automatic recognition system 152 and is sent to destination
One or more of 125, then having connecing for the audit from automatic recognition system 152 of the confidence level higher than predetermined threshold
It receives and can lead to the one or more manual audits cancelled in destination 125 by audit logic 170.Similarly, if figure
As being concurrently sent to multiple destinations 125, and image audit is connect from first destination in these destinations 125
It receives, is requested then audit logic 170 is optionally configured to cancel the audit for image at other destinations 125.?
In some embodiments, once audit logic 170 is configured as keystroke or word and is connect from first destination in destination 125
Receive the audit request then cancelled at other destinations 125.
In some embodiments, audit logic 170 is configured as monitoring the activity of mankind image audit person in real time.The prison
Depending on may include receive from destination 125 based on word one by one or single keystroke audit input.It is such as other herein
What place was discussed, it is received as word and/or keystroke are reviewed logic 170, they are optionally transferred into image source 120
One.The movable monitoring of manual auditor can be used to determine when the audit of image fails and/or complete to scheme manually
As the progress of audit.The state of mankind image audit person can be provided from audit logic 170 to auditor's logic 157 in real time.
Using the state, auditor's logic 157 can by the state of auditor from it is active change into it is sluggish, adjust auditor's
The audit score that is stored, establishment change for the profession of auditor, and/or similar.
In some embodiments, audit logic 170 is configured as the measurement by receiving confidence level (for example, image audit
Accuracy measurement) and to destination logic 160 and/or image put up logic 165 send response signal and control image
To putting up for destination 125.Audit logic 170 can be configured as the measurement based on confidence level and control to destination 125 as a result,
In the images of one or more destinations put up.The measurement representation one or more image tag of confidence level correctly identifies figure
The confidence level of the content of picture.In some embodiments, audit logic 170 is configured as receiving the packet from manual image auditor
Include the audit of the other information other than image tag.For example, audit logic 170 can receive from mankind image audit person
Upgrade request and cause the image audit of upgrading requested.Audit logic 170 is optionally configured to processing and receives in hand
Other non-label informations in audit that dynamic or computer generates.The information may include inappropriate (for example, obscene) image
Identification, do not include the identification that can recognize the image of object, the identification of the image sent to the auditor of wrong profession,
And/or similar identification.
In some embodiments, audit logic 170 is configured as the image by comparing the identical image from multiple sources
It audits to adjust the confidence level of image audit.These image audits can be entirely that computer generates, be entirely manually to examine
Core or including at least one computer generate audit and at least one manually audit.
In some embodiments, audit logic 170 is configured to supply as first (computer generate or manually)
The received image tag in the part of audit and provide institute a received image to the mankind image audit person at the 125B of destination
Label.The agency's (for example, application of browser or specific purposes) executed on the 125B of destination is optionally configured to mesh
Ground 125B display provide first audit image tag.In this manner, mankind's image audit at the 125B of destination
Person can edit and (increase, delete and/or replace) image tag of the first audit.For example, from the received image of destination 125A
Label can be provided to destination 125B for modifying.
In some embodiments, audit logic 170 was configured as based on time used in these image audits, these images
The accuracy of audit and audit score is calculated from the result of the received image audit in destination 125.
In some embodiments, audit logic 170 is configured with response logic 175 to the source of image (for example, image
One in source 120) image audit is provided.When based on character one by one or word one by one image audit is completed, image is examined
Core can be provided.When based on providing to word character by character or one by one, with character or word from mankind image audit person
It is received, image tag is optionally provided to the source of image.Optionally, response logic 175 is configured as mentioning via network 115
For image audit.
Image audit need not be returned to one in image source 120.For example, if image source 120A is that picture shares clothes
Business or social networking website, the image audit of the image from image source 120A can be with picture sharing service or social network network diagram
Account on standing is stored in association.The storage can position in memory 135 or outside image processing system 110
It sets, such as at the network server for accommodating the website.Image audit be optionally both returned to one in image source 120 or
It is stored elsewhere.
In some embodiments, response logic 175 be configured as based on computer generate and/or manually image examine
Received image tag in core and execute search.The result of the search can be provided to the source of image, for example, image source 120A
Or 120B.For example, in some embodiments, user creates image using the camera of image source 120A using smart phone.It should
Image is provided to image processing system 110, is examined using the image that automatic recognition system 152 and destination 125A generate image
Core.The image audit includes then being automatically used to the internet hunt of execution image tag (for example, Google or Yahoo are searched
Rope) image tag.The result of the internet hunt is then provided to the smart phone of user.
In some embodiments, response logic 175 be configured as to ad system 180 provide computer generate and/or
The image tag manually audited.Ad system 180 is configured as selecting advertisement based on image tag.Selected advertisement is optional
Ground is provided to the source of the image for generating image tag.For example, response logic 175 can provide mark to ad system 180
Label " the 1976 Ford Granadas with the head lamp of damage ", in response, ad system 180 can be for replacement head lamp selections
Advertisement.If being used to generate the source of the image of these labels is network address, advertisement can be shown in the network address.Particularly,
If the source of image is that picture is shared or the account of social networking website, advertisement can be shown on the account.Advertisement
System 180 is alternatively included in image processing system 110.Ad system 180 is optionally configured in response to specific
Label is to provide quotation for advertisement.Ad system 180 optionally includes the Adwords of Google.
Image processing system 110 optionally further comprises being configured as extracting from the member of image source 120 for label
The contents processing logic 185 of the image of addition.Contents processing logic 185 is configured as parsing and includes image and optionally include
The webpage of text, and the image added for label is extracted from these webpages.The image tag of generation can be then provided
To ad system 180 for selecting that the advertisement from the webpage for wherein extracting image can be placed in.In some embodiments, interior
Hold processing logic 185 and is configured as emulation browser function in order to load the image that will be usually displayed on webpage.These figures
As can be displayed on webpage associated with particular account, social network sites, picture sharing website, Blog Website, news website,
Meet-a-mate site, sports site, and/or etc website.Contents processing logic 185 be optionally configured to parsing metadata tag with
Convenient for identifying image.
Contents processing logic 185 is optionally configured to parse the text being placed in same web page as image.It should
Text can be automatically recognized system 152 used in image label add, in conjunction with the content of image.For example, contents processing is patrolled
Volumes 185 can be configured as identification image subtitle, make about image comment, be related to image text, Web page subject or
Text (as determined by optical character recognition OCR (OCR)) in title, the people of image interior label or object, image is, and/or similar
Object.The text or its subset parsed by contents processing logic 185 can be used to improve the speed and/or quality of label.It is solved
The text of analysis is provided to automatic recognition system 152 and/or a destination being provided in destination 125 is for by the mankind
Auditor adds label.In some embodiments, automatic recognition system 152 is configured as in the generation for the label of image
Use provided text.For example, provided text can be used to text, identification dictionary, ontology, language and/
Or information, provide the accuracy of image tag automatically and/or manually generated, accuracy, computational efficiency, and/or its
Its quality.Provided text is generally independent of as just the source of label generated, but is used as input to improve figure
The processing of picture.The label generated as a result, may include the word other than those of discovery in provided text.
In some embodiments, image puts up logic 165 and is configured as providing to destination 125 in net identical with image
Both image and text for being found on page.For example, can have subtitle " mountain bike pin in the girl in park and the image of bicycle
Sell " or comment " happy birthday Zhu Li ".In destination 125, the text can be presented to human reviewer together with image.People
The information can be used to more fully understand the emphasis and/or context of image in class auditor, and thus provides better image
Label.Similarly, in some embodiments, automatic identification interface 150 be configured as to automatic recognition system 152 provide with figure
As both the image found on identical webpage and text.At automatic recognition system 152, provided text is used to be based on
The automated tag of the content improvement image of image.In the above examples, provided text can be built to automatic recognition system 152
View should place protrusion on bicycle or on Zhu Li.This can lead to totally different label, such as " Schwinn bicycle " or " birthday
Girl ".
Image processing system 110 optionally further comprises image grading device 190.Image grading device 190 is configured as really
Surely it is used for the grade (for example, priority) of label image.Priority can be used to determine (as if really if wanting) how label
Image.For example, the determination of priority can source based on image, image be loaded on number on webpage, image on webpage
Position, image watched on webpage number, on it including the number of webpage of image, one including image or more
The classification of a webpage, the mark of webpage including image, the classification of the second image on the webpage for including image including image
The owner of webpage, the domain name of webpage including image, the keyword on the webpage including image, in the webpage including image
Number, the other images that the text of upper discovery, the metadata found on the webpage for including image, image are clicked on webpage
The figure whether number, the image being clicked on webpage are a part of video, are automatically generated using automatic recognition system 152
As label, these exemplary any combination, and/or similar.Image grading device 190 includes may be stored on the computer-readable medium
Hardware, firmware, and/or software be form logic.Image grading device 190 includes may be stored on the computer-readable medium
Hardware, firmware, and/or software are the logic of form.In various embodiments, the priority packet determined by image grading device 190
Include two ranks (label or without label), three ranks (automated tag, manual label or without label), ten priority-levels,
Or some other hierarchy plans.
Destination logic 160 is optionally configured to the mesh of the manual label addition of the selection image of the priority based on image
Ground.
In those embodiments, the number that wherein image is loaded on webpage is used to determine that priority, the quantity can
To be every fixed time period, such as daily or monthly.The quantity can be by including the Java or html script on webpage
Row is determined, as known in the art.Position of the image on webpage may be considered that some images can be in image by viewing
Before need viewer to scroll down through.The number that image is actually watched as a result, can be used to calculate the priority of image.It is logical
Often, bigger priority is assigned to the image more often watched.Image grading device 190 is optionally configured to based on image
The number that the number and/or other images being clicked on webpage or on other webpages are clicked on webpage is assigned excellent to image
First grade.Image grading device 190 is optionally configured to the number watched on more than one webpage based on image and determined preferentially
Grade.For example, if image is found on 25 different webpages, the sum watched on all webpages can by with
To determine the priority for image.In some embodiments, image grading device 190 is configured as based on image in a browser
The number being loaded determines priority.
Popular image can be included in several webpages.For example, the image shared extensively in social media website
It can be included on several websites.Image grading device 190, which can be configured as, is included in webpage thereon according to image
Quantity and/or the quantity of the webpage of the link including leading to the image calculate the priority of image.Image grading device 190 is optionally
It is configured as (this possibility is incoherent) image that identification is included on multiple webpages.In some embodiments, image grading
Device 190 is configured with third party's service, such as TinEye.com, to determine the number of webpage that image is placed thereon
Amount.In general, the quantity of the webpage including image is bigger on it, the priority for being assigned to the image is bigger.
In some embodiments, image grading device 190 is configured as point based on the one or more webpages for including image
Grade calculates the priority of image.For example, if webpage be classified in a search engine it is very high, by a large amount of other web page interlinkages or
What person was well classified under some other standards, then the image on the webpage can be given and be classified as letter with the webpage
Several priority.In general, classification is higher, the classification that webpage has is higher, and the priority for being assigned to the image on webpage is bigger.
Webpage classification is optionally obtained from the third party source of such as search engine etc.
Image grading device 190 is optionally configured to the mark based on the webpage for including image to image assigned priority.
For example, can be referred to for compared with the image at another webpage for identical website for the image on the homepage of URL
Send bigger priority.In addition, image can based on it include image webpage specific type and be assigned priority.
For example, the image in social networking website can be given more for compared with the image in company's site or personal blog
High priority.In another example, for other types of webpage, in such as dictionary.com or
Image on the reference webpage of Wikipedia.com etc can be given a higher priority.It is assigned to the preferential of image
Grade is optionally the mark of web-based owner.
In some embodiments, image grading device 190 is configured as using the priority of the second image on webpage as letter
Number determines the priority of the first image on the same web page.For example, if the second image has high priority, the first image
Priority can correspondingly be increased.
Image grading device 190 is optionally configured to refer to based on the other contents for the webpage on it including image to image
Send priority.For example, if webpage includes text and/or metadata, specific term or key in the text or metadata
The presence of word can be used to assign the priority of image.Specifically, if webpage includes valuable keyword, at this
Image on webpage can be assigned higher priority.The monetary value of the estimation of keyword with for advertisement or some other
The value of the word of purpose is associated, for example, in GoogleUpper valuable word.Including meeting quilt
Image of the Adwords higher on the webpage of the term of valuation can be assigned suitably high priority.These terms make
Frequency and their quantity on webpage can also be considered to determine picture priority by image grading device 190.Considered
Text and/or metadata can be included in the URL of webpage, in the map title, in the comment made to figure, by third
Directional image assign label in, name, brand name, trade mark, enterprise name, be related to image text near and/or it is similar.
In some embodiments, image grading device 190 is configured as receiving using optical character recognition OCR derived from image
Text and the priority that image is directed to based on text determination.For example, image grading device 190 can receive by using automatic
Identifying system 152 handles image and the text that generates, and based on the text to image assigned priority.In some embodiments,
Image grading device 190 is configured as the first image on webpage and provides higher priority, relative in the more lower of the webpage
For the image just occurred.
How image grading device 190 is configured to determine (if truly doing) based on the priority assigned
Label is added to image.For example, the image of lowest priority can be by label.Image with somewhat higher priority can
To use automatic recognition system 152 to be added label, and the image with higher priority can be by human reviewer in mesh
Ground 125 in a purpose be located in and be added label.Those with enough high priorities, will by human reviewer add mark
The image of label is optionally further split into higher priority group and lower priority group, in higher priority group
It is given more concerns and more thoroughly or is carefully done outgoing label addition by human reviewer.Image puts up logic 165 can
Selection of land is configured as the instruction that the member into destination 125 provides the priority of image and image.
In some embodiments, image is used first handled by automatic recognition system 152.Then, which can be with base
Both confidence levels in the priority for image and the automated tag executed by automatic recognition system 152 addition are sent to
One or more members in destination 125.For example, if image has relatively low priority, for being examined to the mankind
The confidence criteria that core person sends image is arranged to relatively low.(low confidence standard means that automated tag adds possible quilt
It is considered that the enough and image is not used in mankind's analysis and is sent.If) image have relatively high priority,
Confidence criteria for sending from image to human reviewer is relatively high.Thus, the higher confidence of high priority image request
Degree is more likely to be sent to human reviewer only to rely only on automated tag addition.
Can by image grading device 190 for image selection processing path for example including a) not add label, b) only
Label, c are added using only automatic recognition system 152) confidence level based on importance and/or generation label, using with optional
The automatic recognition system 152 of mankind's follow-up add label, d) automated tag addition followed by adding to the automated tag
The mankind audit, e) by human reviewer add label and/or f) based on by will by human reviewer consider suggestion rank by
Human reviewer adds label.These processing paths are at least partially based on the priority assigned from image grading device 190 to image
And it is selected.Any combination of these processing paths can be found in a variety of different embodiments.In some embodiments,
The result that the type for the processing for being used to label image is controlled cause may those of more valuable image have it is bigger
Possibility is by label.As a result, mankind's label resources are applied to highest priority --- the image of most worthy.
In some embodiments, image grading device 190 is configured as based on advertisement that is adjacent with image or showing on it
Mostly often it is clicked and assigns the priority for image.For example, if image is on the webpage often watched but position
Advertisement on the image is rarely clicked, then the image can be given relatively high priority for label.
In this example, it is more than one can be added label for image.If the advertisement based on initial labels is not with expected frequency
It is clicked, then the image can be tagged again.Again it tags and is optionally executed by human reviewer, mankind audit
Person puts up logic 165 via image and receives image and initial (unsuitable) label.Human reviewer can be used the information and mention
For improved label.
Fig. 2 illustrates Image Acquisition screen 210 according to various embodiments of the present invention.Image Acquisition screen as shown in the figure
210 for example by executed in smart phone, electronic glasses or other image sources 120 application and generate.Image Acquisition screen 210 wraps
It includes and is configured as acquisition image, the interested specific region of label and the feature for receiving image tag.Particularly, Image Acquisition
Screen 210 includes the shutter release button 220 for being configured as taking pictures.Once picture is taken, by optionally via network 115 automatically
Image processing system 110 is sent to add for label.Image Acquisition screen 210 optionally further comprises being configured as scheming
The rectangle of prominent interested point as within.Rectangle 230 be select and/or pull on screen by using user input equipment be
Controllable (for example, moveable).On typical smart phone, which may include touching in response to finger
The touch screen touched.As described elsewhere herein, interested point/region be may be provided to and label to be added
The associated image processing system 110 of image.
Image Acquisition screen 210 further comprises the field 240 for showing the image tag of the image and generation that had previously acquired.At this
In example, identical white cup that the image previously acquired includes no rectangle 230 is shown and including " white Startbuck's coffee
The image tag of coffee cup (White Starbucks Coffee Cup) ".What is be also shown as is alleged " slide for option
The text of (Slide for options) ".
Fig. 3 illustrates the search result based on image analysis according to various embodiments of the present invention.These results are optional
Ground is automatically or in response to select " sliding is for option " input shown in Fig. 2 and is shown.They can be by image
Internet hunt is automatically carried out on label and is generated.Fig. 3 diagram is sponsored advertisement 310, associated picture 320 and other searches
Hitch fruit 330.Search result optionally uses ad system 180 and generates and image tag uses image processing system 110
It is generated.User can choose review and previously add tagged image.The history can be stored on image source 120A or
In memory 135.
The method that Fig. 4 illustrates processing image according to various embodiments of the present invention.In these methods, image is connect
It receives.The image is provided to both at least one destinations in automatic recognition system 152 and destination 125.As a result, meter
Both image audits that the sum that calculation machine generates manually generates are generated.The method illustrated in Fig. 4, which optionally uses in Fig. 1, to be illustrated
The embodiment of system be performed.The method and step that Fig. 4 is illustrated into Fig. 8 can be sequentially executed with plurality of replaceable.
In receiving image step 410, image is received by image processing system 110.The image is optionally via network
115 receive from one in image source 120.The image can be the reference formats such as TIF, JPG, PNG, GIF.The image can
To be an image in a series of images for be formed the image sequence of video.The image can be used by a user camera and adopt
Collection.The image can be acquired from movie or television program by user.In some embodiments, image step 410 is received to wrap
It includes user and acquires image using Image Acquisition application and transmit the image to image processing system 110.The application can be by
It is placed within camera, TV, video display apparatus, multimedia equipment, and/or analog.Receiving image step 410 optionally makes
It is promoted with contents processing logic 185.
In an illustrative example, image is received from image sequence (for example, video).The video is displayed on
In monitor, TV, eye guard, glasses or other display equipment.The video optionally via such as youtube.com or
Netflix.Etc video streaming services be received and/or in browser be shown.Logic in display system
(for example, image tagged logic 147 in image source 120A) is configured such that user indicates the specific son of the image in the video
Collection.Identical logic can be configured as receive generated in response to image tag from image and the advertisement that is selected and this is wide
Announcement is shown on video or shows simultaneously with the video.The selection of advertisement based on image tag other local quilts herein
It further discusses.
Particularly, using the system, user can choose the object and work for label addition in video or film
The label for characterizing the object is optionally received to respond.User can with or alternatively receive based on the label it is selected
Advertisement.The advertisement can be shown (for example, video sequence as covering or addition) together with video or via other in real time
Communication channel (for example, Email) is provided to user.In an illustrative example, user sees he in video
The object liked.They select the object and the selection is received in receiving image step 410.In response, they
Receive advertisement relevant to the object.As video is watched over the display, advertisement is as covering, item or the subtitle on video
It is displayed in real time.Advertisement be optionally it is interactive, since it includes the link for making purchase.
In some embodiments, the object in image may include the specific spy for being configured as assisting in identifying the object
Property.For example, the AD HOC of data bit can be encoded in image or in the object of image.These data bit can be figure
As label coding.
It is optionally receiving in subset identification step 415, is identifying the data of one or more subsets of image by image
Reason system 110 receives.In general, one group of image pixel that the project that one or more subsets include special interests is located therein.It should
One or more subsets can by location of pixels, screen coordinate, region and/or point on received image identify.Some
In embodiment, which is used by a user selected by the cursor or touch screen of an image source in image source 120.
It is optionally receiving in source data step 420, source be received in receiving image step 410, about image
Source data received by image processing system 110.As discussed elsewhere herein, source data may include geographical letter
Breath, Internet protocol address, uniform resource locator, account name, smart phone identifier, about image source 120 at
The information of the language used on member, searching request, user account information, and/or similar.In some embodiments, source data quilt
Application/the agency in operation image source 120 automatically sends.For example, GPS coordinates can be on smart phone by certainly
It generates dynamicly and is provided to image processing system 100.
Optionally receive source analysis priority tasks 425 in, receive image step 410 in be received, for scheming
The priority of the label of picture is received by image processing system 110.In some embodiments, priority is by the user of image source 120A
It manually inputs.In some embodiments, priority depends on the amount paid a price for the audit of image.In some embodiments
In, priority depends on the type of image source 120A.For example, compared with from the received image of handheld mobile device, from static network
Received image of standing can automatically be given lower priority.Compared with the image that its source is identified by Mobile Directory Number,
The image that its source is identified by uniform resource locator can be given lower priority.Priority is optionally by connecing as a result,
Receive the source data export being received in source data step 420.
It is optionally received together in the step 410-425 image being received and data and is optionally stored in storage
In device 135.
In distribution image step 430, image and optionally it is received in step 415-425 any associated
Data are assigned via automatic identification interface 150 to automatic recognition system 152.The distribution can image processing system 110 with
It is interior or via network 115.
It is automated toed respond in step 435 in reception, the image audit that computer generates is received from automatic recognition system 152.
The image audit that computer generates includes one or more image tags that the system of being automatically recognized 152 is assigned to image.It calculates
The image audit that machine generates further includes the measurement of confidence level.The measurement of the confidence level be assigned to image image tag it is correct
Ground characterizes the measurement of the confidence level of the content of image.For example, the image relative to abstract shapes, main includes that may be readily recognized
The image of character can receive the measurement of higher confidence level.
It is optional determine confidence level step 440 in, the measurement for the confidence level for including in image audit with one or more
A intended level compares.The intended level be optionally the priority of image audit, the price of image audit, image source,
And/or the function of analog.Optionally firmly believing? in step 445, if the confidence level for the image audit that computer generates is higher than
(multiple) predetermined threshold process then proceeds to optional execution search step 450, if the confidence for the image that computer generates
Degree then proceeds to lower than (multiple) predetermined threshold and is lined up image step 460.Determine that confidence level step 440 is optionally patrolled using audit
170 are collected to be performed.
In executing search step 450, the image tag for being assigned to image is used to execute search.For example, image mark
Label " ford " can be used to word " Ford " and " vehicle " to automatically carry out Google search.
In result step 455 is provided, it is assigned to the image tag of image and is optionally executing search step 450
In the result of search that is performed be provided to the requestor of image audit.For example, if image is received from image source 120A
And image source 120A is smart phone, then image tag and search result are normally provided to smart phone.If image
It is received from the member in the image source 120 of such as website etc, image tag and optional search result can be provided to
The host of website, to third party, to ad system 180, and/or analog.In some embodiments, image tag is by automatically
It is added to website, so that image tag can search for, for example, can be searched to find audited image.
In being lined up image step 460, image is arranged in image queue 145.The arrangement is optionally included using figure
As the subset of 147 tag image of mark logic.As described in elsewhere herein, label is typically configured as identification image
In special interests object.The priority for promoting the audit that can depend on image of image in image queue 145, image
Source, available mankind's image audit person, computer generate image audit confidence level measurement, and/or analog.
In determining destination step 465, one or more members in destination 125 are determined for the hand of image
Dynamic audit.The determination of destination is optionally based on the image audit generated in the computer being received from automatic recognition system 152
The middle image tag for being included;It is optionally based on the profession of the mankind image audit person at different destinations 120;Optional ground
In the audit score of these mankind image audit persons, and/or based on the other standards being discussed herein.In some embodiments, really
Determine profession of the destination step 465 based on the data and human reviewer for characterizing image.The data for characterizing image can be
Characteristics of image, image descriptor, and/or the information being derived from.As discussed elsewhere herein, characteristics of image and/
Or image descriptor is optionally received together with image from the member in image source 120.Information as derived from it can be given birth to
At at the member of image source 120, at image processing system 110 and/or at automatic recognition system 152.
In putting up image step 470, image is posted to destination determined in determining destination step 465
At least one destination in 125.In some embodiments, put up image step 470 include image is concurrently posted to it is more
In a destination 125.The image is optionally posted via network 115 and is optionally opened together with set forth below
Paste: the label of the subset of prominent image, the time audited before failing in image, is used for from certainly the source data for image
The image tag, and/or analog of the dynamic received image of identifying system 152.
In receiving audit step 475, the manual audit of image one from destination 125 determined by (multiple) or
It is received in multiple.Manually image audit may include one or more figures that image is assigned to by mankind image audit person for this
As label.The content of one or more image tag expression image.The manual audit can also include upgrade request, can not examine
The inappropriate instruction of the instruction of the image of core, image, the instruction of audit failure, and/or similar.
It is tagged in image? step 480 in, the progress of this method depends on whether image tag is receiving audit step
It is received in rapid 475.If the image tag for characterizing the content of image is received, this method is optionally continued to execution
Search step 450 and offer result step 455.In those steps, it is audited in manual image and optionally computer is raw
At image audit in include image tag used.The use of image tag in the image audit that computer generates can
To depend on the confidence metric of the audit.
If confidence metric, which is found to be, in step 445 is higher than (multiple) predetermined threshold, step 460-475 is optional
's.
Optionally upgrading? in step 485, the progress of this method depends on whether upgrade request has been received.If
Such request has been received, then method proceeds to determining destination step 465, wherein the second of destination 125/difference
Member be determined.The determination can depend on the image tag being received in manual image audit, manual image audit
It is received in receiving audit step 475.Upgrade request can be received from mankind image audit person or asking from image audit
The person of asking (from image source 120A or 120B etc.) is received.Have an opportunity to audit the quilt in result step 455 is provided in requestor
After the image tag of offer, upgrade request can be received.For example, requestor can receive the image including " white vehicle " first
Label and then request audit upgrading, because they expect to have further information.Audit upgrading can cause image to be provided
To the mankind image audit person having in automotive field profession.Mankind's image audit can be added to existing image tag with
It generates " white vehicle, 1976 Ford Granadas ".In some embodiments, when request audit upgrading, requestor, which can add, to be referred to
The source data of the subset of diagram picture.For example, auditor may desire to indicate the special interests on the head lamp of damage.This is used for will
Mankind's image audit person's attention is oriented to this feature of image, generates the label including " head lamp of damage ", and guiding is caused to be used
In the search (executing search step 450) of the head lamp of the damage of 1976 Ford Granadas.
In some embodiments, upgrade request is reviewed logic 170 and automatically generates.For example, if image audit occurs
Must be too brief, such as be only " vehicle ", then audit logic 170 can automatically initialize audit upgrading.In some embodiments
In, upgrade request automatically generates the presence based on the keyword in manual image audit.For example, certain audit professions and key
The list of word is associated.In some embodiments, when a keyword in these keywords is received in manual image audit
In and automatically audit upgrading be initialised.Audit upgrading, which preferably includes, has profession associated with the keyword received
Mankind image audit person.In particular example, profession include " automobile " and with keyword " vehicle ", " truck ", " goods
Vehicle ", " open car " and " Ford " are associated.When a keyword in these keywords is received in manual image audit
When, whether audit logic 170 negotiates with logic 157 is audited to determine and have mankind's image audit person in " automobile " profession current
It is active.If it is active, then automatic upgrading is initialised and image is sent to examining with " automobile " profession
The destination 125B of core person.
If made without upgrade request, step 490 is being terminated, which completes.
Fig. 5 illustrates the alternative of processing image according to various embodiments of the present invention.In these methods, it walks
At least some of at least some of rapid 430-445 and step 460-475 is executed in parallel.In step 460-475
Manual image audit can start before the audit that the computer of step 430-445 generates is completed thus manual image audit exists
Start before knowing the confidence metric for the audit that computer generates.If firmly believing? step 445 confidence metric is found to be
Higher than (multiple) predetermined threshold, then step 460-475 is optionally terminated.
The method that Fig. 6 illustrates management auditor pond according to various embodiments of the present invention.In the method, auditor
State can audit the level of image based on them and be changed.Illustrated step can be the method for Fig. 4 and Fig. 5 diagram
Part is consistently performed with Fig. 4 and Fig. 5 method illustrated.For example, they can be examined receiving image step 410 and receiving
It is partially executed one at a time between core step 475.The method of diagram includes that the more than one destination into destination 125 sends image.
In receiving image step 410, image is received.As other places discuss herein, image can be via
Network 115 is received at image processing system 110.Image can be generated by camera and/or be obtained from webpage.In some implementations
In example, image is received together with the information mostly often watched about the webpage.
In selecting the first destination step 610, the first destination is selected for the manually or automatically analysis of image.
It selects the first destination step 610 to be performed using destination logic 160 and is the embodiment of determining destination step 465.
As described in other places herein, the determination for the destination of image is based on various factors, including human reviewer
State and score associated with auditor.For example, the member of destination 125 usually associated with active auditor will
It is selected, rather than the destination of not active auditor.Selected destination can be destination 125 member and/
Or automatic recognition system 152.
In putting up image step 470, received image is posted to destination 125 in receiving image step 410
Selected member.As other places discuss herein, putting up for image may include via network 115 using such as
The computer network with standard network protocol of TCP or UDP etc transmits image.
In optionally monitoring step 620, auditor's logic 170 is used to monitoring and is selecting the first destination step 610
The progress of the manual image audit of image at the member of the destination 125 of middle selection.Monitoring may include detection mankind's audit
Time used in the input of person, image audit, the quantity of provided word for characterizing image, and/or similar.It monitors optional
Ground includes the time used in measurement label image.Wherein detection includes the detection of the input of human reviewer, and monitoring can be base
In keystroke one by one, based on word one by one and/or on line by line basis.Auditor's logic 170 can be configured as once as a result,
Receive the data for characterizing image character, word or row.
In removing step 620, at the member for the destination 125 that image selects in selecting the first destination step 610 from
It is removed in processing." removal " may include notified at the selected member of destination 125 human reviewer he or she not
Additionally and briefly be responsible for audit image, the human reviewer of prime responsibility released into (without notifying human reviewer), by image from
The display of human reviewer removes and/or similar.In some embodiments, removing step 630 includes only auditing the mankind
Person is placed in classification the secondary or shared responsibility having for audit image.For example, if with the first purpose is being selected
The associated human reviewer of the member of the destination 125 selected in ground step 610 has the prime responsibility for audit image,
The responsibility can be shared or be assigned to now other auditors associated with other members of destination 125.In the situation
Under, " removal " is prime responsibility.
If the manual audit of image has been used too long, removing step 630 can occur.For example, if being walked in monitoring
Have found that auditor did not start to input after the predetermined time in rapid 620, then removing step 630 can be performed.For
The other examples of the trigger event of removing step 630 include with the communication loss of the selected member of destination 125, for figure
The distribution of the audit of picture receives inappropriate or inappropriate image beyond the predetermined time, from human reviewer, from human reviewer
Receive the image tag of inaccurate (not characterizing image), from the first human reviewer to the recommendation of the second human reviewer, figure
As the upgrade request, and/or similar of audit.
In selecting the second destination step 640, the second member (or automatic recognition system 152) of destination 125 is used
Destination logic 160 is selected.Second member can be based on above with respect to the first destination step 610 of selection and determining destination
Any standard that step 465 is discussed is selected.In addition, in some embodiments, the selection of the second member can based on by with
The specific recommendations of the associated human reviewer of the first member of destination 125.For example, the first human reviewer can identify figure
The content of picture is the profession of the second human reviewer and the image recommendation can be given to the second human reviewer associated
The member of destination 125.It is optionally taken in the selection for the second member for selecting the destination 125 in the second destination step 640
Certainly automatically processing in the image using automatic recognition system 152.
It is puted up in image step 470 another, image is posted to selected in the second destination step 640 of selection
The member of destination 125.In some embodiments, more than one human reviewer can concurrently audit image.They can be with
Independently or cooperative execute audit.One auditor can have the prime responsibility of the audit of image or each auditor can
With same responsibility.One auditor can have the supervisory responsibility to one or more of the other auditor.In some realities
It applies in example, before monitoring step 620 and/or removing step 630, the second destination step 640 of selection is performed and image
It is posted to two or more members of destination 125.
In receiving audit step 475, audit (for example, image tag) such as other places herein of image are discussed
As be received.Audit generally includes to characterize the image tag of the content of image.Audit can be from being more than in destination 125
One destination receives.For example, the label for characterizing image can be from the second mesh of the first destination step 610 of selection and selection
Ground step 640 the two in selected in the member of destination 125 that is selected.It is audited with character or word by (multiple) mankind
Person provides, and receives audit step 475 and is optionally performed in real time.
In optional correlation tag step 650, the one or more image tags for characterizing image are associated with image
Ground is stored.Stored label is optionally included by the label of more than one mankind auditor's offer and can be stored
In memory 135.As described in other places herein, label can be further supplied to the member of image source 120
(for example, in embodiment that result step 455 is provided) or it is used to select advertisement using ad system 180.Label can also quilt
It is provided to automatic recognition system 152 and the training of automated graphics identification process is provided.
Fig. 7 illustrates the method for receiving image tag in real time according to various embodiments of the present invention.These methods can
Selection of land by image processing system 110 execute and image tag can be manual image audit result.These methods can be with
Other methods described herein are consistent, such as the part of method shown in Fig. 4.This method is to put up image step 470
Start, in this step, image is provided to one or more members of destination 125, as discussed elsewhere herein
's.Method shown in fig. 7 is optionally to receive image step 410 as first, image is from remote computing device in this step
It is received.
In receiving input step 710, input is received from one or more members of destination 125.The input is usually wrapped
The character provided by human reviewer is provided.For example, input can be the character inputted by human reviewer in destination 125A.
In general, being performed with other steps shown in fig. 7, receives image step 710 and continue.
In detecting the first word step 720, it is detected in the input that word is received in receiving input step 710.
The word can be detected by such as space character in space ASCII etc or the presence of carriage return.Spell check is optionally detecting
Word on be performed.If word is not included in spell check dictionary, the trial corrected can be made or
Human reviewer can be informed that identification word failure.
Lead to the execution for delivering the first word step 730 in the detection for detecting the word in the first word step 720, at this
Word is transferred into the source of image in step 730.Once for example, word is detected image source can be provided in real time
120A.In image source 120A, which can show to user.It waits compared with before the set of display image tag until figure
For being received as the entire set of label, once show that a word can provide the analysis of image and occur when shorter
The impression of the area of a room.
In detecting the second word step 740, it is detected in the input that the second word is received in receiving input step 710
It surveys.Again, word can be detected by the presence of blank character and can be provided at image source 120A by the first word
User after occur.Both first and second words are expected for characterizing the label of image.Detecting the second word step
The detection of 740 the second word triggers the second word step 750 of delivering, and second word is delivered to image in step 750
Source, for example, image source 120A.Third, the can be recycled and reused for by detecting the second word step 740 and the second word 750 of delivering
Four and additional word, part of each word as image tag.
It is completed in step 760 in detection, the data for indicating that the processing of image is completed are received, which completes to be, for example, to examine
The word measured includes the whole words (image tag) to be provided by human reviewer.The data may include such as "/
Endtags ", ASCII carriage return, and/or similar metadata tag.In general, detection completion step 760 is at one, two or more
More image tags occur after being received.In optional correlation tag step 650, the received image tag of institute and image
It is associated and/or is stored together with image, as discussed elsewhere herein.
Although Fig. 7 illustrates word in primary detection and delivering, in alternative embodiment, individual keystroke is tested
It surveys and delivers.Receiving input step 710 can concurrently continue with step 720-740 and/or 750.Step 710-760 can make
It is included that other places herein are discussed to receive the part of audit step 475.
The method that Fig. 8 illustrates the audit of upgrade image according to various embodiments of the present invention.In these methods, image
Receive the more than one stage of image audit.Then the first audit (stage 1), image audit are upgraded and are further audited
(stage 2).Request for upgrading can be generated automatically, and be initiated by the first human reviewer, and/or in response to from figure
The request in the source of picture.Both first and second audits can be manually, that is, have human reviewer to execute.Alternatively,
One audit can be automatic and one or more subsequent audits can be manually or first audit can be manually
And one or more subsequent audit be automatic.
In receiving image step 410, image is received.First member of destination 125 is selected in selection the
Image in one destination step 610.The image is then posted in putting up image step 470.These steps are herein
Other places be discussed.
It is audited in step 810 in reception first, the first audit of image is received.First audit may include characterizing
One or more image tags of the content of image.For example, image audit can in response to the picture of the image including black spider
To include word " Black Aranea ";Or in response to the image including red car, image may include word " screw oil expeller ".Receive the
One audit step is optionally the embodiment of reception audit step 475, and may include the real time communication of image tag, is such as closed
It is discussed in Fig. 7.
In some embodiments, the first audit may include being provided by the human reviewer for executing the first image audit
, the instruction that the processing of image should be upgraded.For example, the first human reviewer can manually indicate for second (optional
Ground specialization) the speciality field of human reviewer.For example, the first human reviewer can provide " screw oil expeller " image tag and build
View is executed the audit of upgrading by the auditor with automobile speciality.Alternatively, the first audit may include being considered especially having
The image tag of value.For example, a possibility that instruction image includes wedding clothes can trigger automatically for 72% automatic audit
Manual audit is upgraded to, because image tag " wedding clothes " potentially has bigger commercial value than other image tags.
In some embodiments, which is reviewed the execution of logic 170 and based on whether relatively important or valuable key
Word is stored in the list in memory 135.The list may include the associated measurement of keyword and their value.Such as this
Other places are discussed in text, are optionally based on by the automatic upgrading that audit logic 170 executes using automatic recognition system 152
And/or the image tag that forecast image mostly often automatically generates the information watched.These factors optionally use maximum
Change the algorithm using the potential value of human reviewer's label image and provides advertisement based on these labels and applied.More have
The example of the image tag of value can with shoes, vehicle, jewelry, Reiseziel, book, game, clothes, vacation, food, beverage,
The correlations such as real estate, bank, accident.
In some embodiments, the upgrading of image audit is automatic.For example, the label of " Black Aranea " can automatically be led
Cause includes that the upgrading of the image audit of image is sent to the human reviewer with specific speciality (for example, spider expert).It is specific
Plant or the identification of animal life generally include and (depend on) location information because the position of plant or animal can be for
It is important for appropriate identification.
In some embodiments, as discussed elsewhere herein, upgrade audit can be by raw requests picture quilt
The people of audit requests.For example, the user of image source 120A can provide the image of dog and receive the image mark including " black dog "
Label.User can then by provide word " cultivate? " to require further details.In this case, image audit can be with
It is upgraded and is sent to the human reviewer of the specific instruction of the cultivation with dog.In some embodiments, user is for rising
Premium account is charged or be requested to have to grade in order to request to upgrade.User can specify when requesting image audit upgrading
The specific part of image.
The presence (automatically and/or manually generating) of upgrade request is detected in detection upgrade request step 820.Detection
Can based on the member from image source 120, the member from destination 125, automatic identification interface 150, and/or from such as audit patrol
Collect the received data of component or life of the image processing system 110 of 170, contents processing logic 185 or response logic 175 etc
It enables.
In selecting the second destination step 640, destination logic 160 is used to the second member of selection destination 125
And/or automatic recognition system 152 is with the audit for image.The selection can based on the first destination step 610 based on mark
Any one quasi- and image tag and/or the other information generated from the first audit.For example, the second one-tenth of destination 125
The selection of member can be based on, and be based at least partially on the image being generated manually or automatically in the first image audit.Particularly,
The label of " Black Aranea " can be by destination logic 160 is used to select to audit with the mankind of the speciality of the identification with spider
The member of the associated destination 125 of person.In another example, the selection of the second member of destination 125 can be based on by asking
The word for asking the user of image audit to provide.Particularly, if the first audit generates image tag " white shoes " and user response
With " brand? ", then human reviewer's phase of information selection and the speciality with shoes brand can be used in destination logic 160
The member of associated destination 125.
In some embodiments for selecting the second destination step 640, destination logic 160 is configured as to select certainly
Dynamic identifying system 152 selects the member of destination 125 with the second audit for image.For example, when image by
Label has that the name of performer and upgrade request require " movie name? " when, this may occur.In this case, image can
To be searched in the library of film image.Identical method can be taken to be used for such as currency, drawing, vehicle, trade mark, item
Other reproducible objects of shape code, QR code, fatal personage etc..
In another example for putting up image step 470, image is posted to the second selected member of destination 125
Or automatic recognition system 152 is with the second audit for image.Step 830 is audited in reception second, characterizes the interior of image
The image tag of appearance is usually received.Receiving the second audit step 830 is optionally the embodiment for receiving audit step 475.It can
Alternatively, instruction image because certain reasons cannot additional reference by label or other information can be received.Image tag
The member for the destination 125 being posted to from image or automatic recognition system 152 are received.If necessary, step 820,
640, it 470 and 830 can be repeated.
In correlation tag step 650, received image tag it is associated with image and/or be provided to image
Source, as discussed elsewhere herein.
In one illustrative examples of method shown in Fig. 8, image is received from webpage.Image is sent to automatic knowledge
Other system 152 for auditing automatically.Automatically the result audited includes image tag " ring ".The label uses audit logic
170 is processed and be identified as potential valuable image with for using in advertisement.As discussed elsewhere herein
, the other factors which is optionally also mostly often watched on webpage etc based on such as image.Knot as identification
The audit of fruit, image is automatically upgraded and is sent to destination associated with having the human reviewer of jewelry speciality
125 member.Human reviewer modification image tag has included " golden wedding ring " and these labels are related to image
Connection.These image tags can be subsequently used to select advertisement using the system and method described elsewhere herein.
In one illustrative examples of method shown in Fig. 8, image is received from the application executed on the mobile apparatus.
The image includes streetscape and is sent to destination 125A for being audited by human reviewer.Human reviewer is with image tag
" streetscape " responds and these labels are provided to mobile device.The subsequent slave mobile device of request of audit upgrading is connect
It receives.The request include text " vehicle? " and the identification of the part of image includes vehicle.As the request as a result, image, text
The other member of destination 125A or destination 125 is sent to for further audit manually with identification.Into one
The audit of step leads to image tag " 1909 model T ", is then forwarded to mobile device.
In one illustrative examples of method shown in Fig. 8, image is received from equipment is calculated.The image includes being placed on
Multiple bank note on disk are simultaneously sent to destination 125A for auditing manually.The label of generation includes the " U.S. of Bai Panshang
Currency " is simultaneously sent to calculating equipment.The request of image audit upgrading is received.The request includes " how many? " as this
Request as a result, image is sent to automatic recognition system 152, automatic currency recognition logic is used to identify currency at this
It measures and optionally provides sum.The information is then sent back to calculating equipment.
In one illustrative examples of method shown in Fig. 8, image slave mobile device is received.The image includes plant
Leaf is simultaneously sent to destination 125A.Human reviewer at the 125A of destination provides image tag " greenery " and also rises
Grade image audit.As upgrading and image tag as a result, image is sent to destination 125B, and with botany speciality
The second mankind image audit person it is associated.It is based on image tag " greenery " to the selected section of destination 125B.Concurrently, scheme
As label " greenery " are sent to mobile device.At the 125B of destination, second mankind's image audit person is to already existing figure
As label addition word (label) " toxicodendron ".These additional labels are then also routed to mobile device.On the mobile apparatus,
Word " greenery " once it is shown first and then available word " toxicodendron " is added into display.
Method shown in Fig. 8 is optionally used in consistent with other methods described herein.For example, image tag can be with
It is used in the auction labelling step 1560 described elsewhere herein.Label can be used to selection advertisement and advertisement quilt
It is provided to the remote browser for showing on webpage together with image.
Fig. 9 illustrates the example of the image source 120A including electronic glasses according to various embodiments of the present invention.Electronics
Glasses include the glasses being worn.Example includes Google" Google glass "," M100
The iOptik of intelligent glasses ", InnovegaTMContact lenses, and/or similar.These systems are configured as allowing user at the same time
Between watch both real world and electronic console.Electronic glasses can also include the Oculus Rift of such as Oculus VRTM
Etc virtual reality system.The system of these types is shown image to user using electronic console, but is not provided true
Direct viewing while the real world.It is not used for watching and digitized viewing when direct viewing, for example, passing through glasses or eyeglass
Viewing.
In general, user can be real when electronic glasses are provided when image in electronic glasses or watched by electronic glasses
When select image subset interactive interface.As used in this article, it is " real-time " selection mean image when it is collected with
Only inessential time delay is watched.For example, the image watched in real time can be acquired by camera and use image engine
It handles and shown with the time delay only generated from electron process number.The user of viewing permission in real time is watched with image to be passed through
Interested object is located within the image of viewing by mobile image acquisition equipment.Thus, viewing, which eliminates, in real time is watching
Before by the viewing of the image of storage quite a while.
As shown in figure 9, image source 120A includes the camera 910 for being configured as acquisition image.Acquired image can wrap
Include static image or the video including image sequence.Display 920 is configured as acquired image being presented to camera 910
User.In some embodiments, such as image source 120A is those of smart phone embodiment, and display 920 includes being configured
For the touch screen of the view finder as camera 910, to show acquired image and show Image Acquisition screen 210 (herein
Described in other places).The display and other contents on display 920 that display logic 925 is configured as management image.
Display logic 925 may include the hardware, firmware, and/or software that may be stored on the computer-readable medium.
In the embodiment shown in fig. 9, image source 120A further comprises the user's instruction for being configured for image source 120
The selection logic 930 of the subset of acquired image.The instruction can be just shown with image and/or be collected in display
It is made in Image Acquisition screen 210 in real time on 920.As discussed elsewhere herein, such instruction is done
Out to select the interested special object in image.And then it selects, the subset of image optionally uses image tagged logic
147 is labeled, as discussed elsewhere herein.Image tagged logic 147 can be used to be shown into such as display 920
Image add label.User can see that the position being labeled as a result,.Selection logic 930 is optionally configured for
User re-flags the subset of image, until user is satisfied with selection.
In some embodiments, selection logic 930 includes tracking logic 935 to track the movement of eyes of user.Tracking is patrolled
935 are collected to be alternatively included in electronic glasses.Eye tracks may include the focus point for detecting eyes, one or more eyes
Direction (eyeball direction), the focusing of one or more eyes, blink, eyeball it is mobile, and/or similar.It is optional to track logic 935
Ground is configured as the state of eyes of user is associated with the position in acquired image.Indicate camera 910 and selection logic
The geodata of geographical relationship between 930 physical factor is used to be adopted the state of eyes of user with using camera 910
Position in the image of collection is associated.
Tracking logic 935 optionally includes the second camera for being directed toward the glasses of user.The camera can be installed in electronics
The part of other embodiments on glasses or as image source 120.For example, the tracking for being configured for the eyes of tracking user is patrolled
Volumes 935 can be included in network cameras, in smart phone, in computer monitor, in TV, in tablet computer and/
Or the like in.
In some embodiments, tracking logic 935 is configured as detecting the blink of one or more eyes.For example, tracking
Logic 935 can be configured as the mode for detecting simple eye blink or blink.When being detected such time, logic 930 is selected
It can be based on patrolling the position in 935 received position datas selection image from tracking, or alternatively select to see at present
The position at the center for the image seen.
Once image has used mark logic 147 and selection logic 930 labeled, the position and/or region of the label
User can be displayed on the image of the label in display 920.For example, in marked locations plus red " X "
Image can be displayed to user in Image Acquisition screen 210.In some embodiments, user can be then using confirmation logic
940 confirmation selections.Confirmation logic 940 is optionally in response to tracking logic 935.For example, blink or other eyes can be used in confirmation
Movement, voice command, verbal order or touch order are provided.In some embodiments, tracking logic 935 is configured as examining
Survey and be interpreted as the movement that order eyes enter unnatural position (for example, esotropia).Such movement can be used to
Confirmation order.Confirmation is optionally required prior to sending image to network 115.
In some embodiments, selection logic 930 is including being configured as tracking chasing after for certain things other than eyes
Track logic 935.For example, tracking logic 335 can be configured as detection user indication finger (pointing finger),
The electronic equipment, and/or analog being worn in finger or wrist.In these embodiments, selection logic 930 is configured as base
The position in image is inferred in object detected.In one embodiment, tracking logic 935 is configured as detection in image
The position that the position of the finger of indication and deduction will be selected is at the tip of finger.User can give directions in its visual field
Object, order to image source 120A offer audio, based on eyes and/or based on touch, and the position for the finger given directions
The selection of the position in image will be used to make.In one embodiment, tracking logic 935 is configured as detection wireless electron
Equipment relative to image source 120A position and the position to be selected of deduction along radio-based electronic devices and image source 120A
Part between line.
Image source 120A further comprises being configured for image source 120A to be sent to image processing system via network 115
110 I/O 945.I/O 945 may include wired and/or wireless connection.For example, in some embodiments, I/O is matched
It is set to and uses bluetoothTMConnection is wirelessly communicated to cellular phone from electronic glasses and is used subsequently to using Wifi or cellular service
Communication is forwarded to network 115 from cellular phone.
Image source 120 further comprises being configured as storage to use camera 910, geodata, account data, and/or class
Like the embodiment of the memory 135 of the image of acquisition.Memory 135 includes such as random access memory (RAM) or read-only depositing
The non-transitory memory of reservoir (ROM) etc.Memory 135 generally includes to be configured as storage acquired image and these
The data structure of mark position in image.
Image source 120 further comprises processor 950.Processor 950 is configured as executing at the number of computations
Manage device.For example, in some embodiments, processor 950 is encoded with computations to execute display logic 925, selection logic
930, image tagged logic 147 and/or tracking logic 935.Processor 950 optionally includes application-specific IC (ASIC)
Or programmable logic array (PLA).
Image source 120 optionally further comprises object tracing logic 955.Object tracing logic 955 is configured as one
The movement of the interested object of tracking in image series.For example, in some embodiments, selection logic 930 can be used in user
To select to be required to obtain the subset of the image of its information or the aspect of image.The subset may include one or more pixels.
Object tracing logic 955 is configured with automatic (computer based) image interpretation logic and occupies selected son to identify
The special object of collection.The object can be people, motor vehicle, animal or any other object.The boundary of selected object or its
Its pixel is optionally prominent in display 920 by object tracing logic 955.The protrusion moves in a series of images with object
Object can be tracked and may include the characteristic for changing pixel.The protrusion is optionally moved together with the object on display.
The one aspect of image can be the brand of objects within images, the film, the position of content of image that obtain image etc..Some
Can be used in embodiment, in terms of image such as " shoes brand? ", " film? ", " performer? ", " position ", " cultivate? " equal texts
It is designated as interested.This specify can add image in tagged raw requests and/or mentioned in upgrade request
For.
In some embodiments, being from image source 120 to the image that image processing system 100 is transmitted includes short-sighted frequency sequence
The part of a series of images of column.The system and method that describe elsewhere herein can be used to add in these video sequences
Label.Adding a tagged advantage to video sequence is that (multiple) label can characterize in video specific dynamic occurs
Make.For example, the label of figure skater can characterize specific jump (hooked hand two weeks etc.), in video than quiet
Only preferably identified in image.Various embodiments include the specific limitation in the length of image sequence, for example, video must be few
In 3,5,7 or 10 seconds.
Although embodiment shown in Fig. 9 includes electronic glasses, these embodiments can be adapted to be with ocular pursuit technology
Any equipment, including mobile phone, video display monitor (such as computer or video screen), tablet computer, advertisement be aobvious
Show device etc..For example, the embodiment of image source 120A shown in Fig. 9 includes having to be configured to determine that user is watching TV screen
The TV of the ocular pursuit camera of which part of curtain.
Image source 120A optionally further comprises being configured as executing one or more steps for label image purpose
Rapid image processing logic 960.Image processing logic 960 is optionally configured to local in image source 120A by executing these
One or more steps and reduce the load on image processing system 110.For example, image processing logic 960 can be configured
For execute the label of image initial step and then by the result of these initial steps be sent to image processing system 110 with
For generating image tag.In some embodiments, image processing logic 960 can be completed some but need not be all images
Label process.Image processing logic 960 includes hardware, firmware and/or the software that may be stored on the computer-readable medium.For example,
Some embodiments include being specially configured to execute the processor 950 of the function of image processing logic 960 discussed in this article
Example.
In some embodiments, image processing system 110 is configured as providing image processing logic 960 to image source 120.
This is optionally via the application shop of such as apple " App Store ".If being applicable in, image is provided to the member of image source 120
Processing logic 960 is the optional step in various methods shown in this article.Processing logic 960 can be provided as further wrapping
Include computer instruction or " applying (app) " of other logics discussed in this article (for example, the logic discussed about Fig. 9).
In some embodiments, image processing logic 960 is configured as the special characteristic in identification image.Feature identification packet
Include the part for the feature for determining whether that the yes or no of the specified point in image gives type.The type of feature includes but is not limited to side
Edge, corner, spot (blob) and ridge (ridge).It is generally characterized by the " interesting of the purpose of the content of the image for identification of image
" or " useful " part.Image processing logic 960 can be configured as one executed in several different characteristic detection algorithms
Or it is multiple.In some embodiments, image processing logic 960 is configured as based in available processing capacity and/or image
Appearance is selected from several different algorithms.The example of known feature detection algorithm includes " Canny ", " Sobel ", " Harris&
Stephens/Plessy ", " SUSAN ", " Shi&Tomasi ", " level curve curvature ", " FAST ", " Gauss-Laplace ",
" Hai Sai (Hessian) determinant ", " MSER ", " PCBR " and " gray scale spot (grey-level blobs) ".These types
Algorithm be performed on the computing device and other such algorithms will pair it would be apparent to one of skill in the art that.Feature
The result of identification includes the identification of the special characteristic type of the specific position in image.This can at " feature descriptor " or
It is encoded in " feature vector " etc..The structure of feature detection can also include indicating the identified level of confidence of feature at which
Value.
In some embodiments, image processing logic 960 is configured to based on the box counting algorithm identified
Image descriptor.Image descriptor is the visual signature of the content of image and including such as shape, color, texture and movement
The characteristic of (in video).Image descriptor can be the part in particular descriptor domain, such as recognize with face recognition or currency
Relevant descriptor.The export of image descriptor is typically based on characteristics of image.For example, the export of 3D shape description symbols can be based on
Edge feature detected.Image descriptor can characterize the object that the one or more in image is identified.
The specific image features and image descriptor being used in specific embodiment depend on the specific image used identification
Algorithm.Great amount of images identification algorithm is known in the art.In some embodiments, image processing logic 960 and/or image
Processing system 110 is configured as first attempting to the identification of characteristics of image and the export of various types of image descriptors, and with
It is selected from multiple alternative image processing algorithms based on the level of confidence that image descriptor at which is exported afterwards.For example,
If the image descriptor in face recognition domain is exported with high-caliber confidence level, specific to the image of face recognition
Processing Algorithm can be chosen so as to generate image tag from these image descriptors.
It is including in those of image processing logic 960 embodiment, the task of label image can be distributed in image source 120
Between image processing system 110.How task, which is assigned, can be fixed or can be dynamically.It is fixed in distribution
Specific step is performed consistently with specific equipment in embodiment.It is point of step in dynamic embodiment in distribution
With for example can be in response to the processing capacity on communication bandwidth, image type (static or video), image source 120A, image procossing
Present load in system 110, the availability of the image audit person at destination 125, step are completed in image source 120
Confidence level, and/or the image descriptor data that are presented on image source 120A.Any combination of these factors is used to dynamic
The distribution of ground allocation processing step.For example, if the export of image descriptor is with the confidence level of the low degree on image source 120A
(relative to predetermined demand) occurs, then characteristics of image and/or image may be transferred into image processing system 110 for making
With more strength or alternative image processing algorithm come deduced image descriptor.It compares, if the export of iamge description degree
Occur on the image source 120A of the confidence level with sufficient degree, then the step is not usually required in image processing system
It is performed on 110.
If image processing step is successfully executed on image source 120A by image processing logic 960, image and/or this
The result of a little steps can be used I/O 945 and be transferred into image processing system 110.For example, in some embodiments, image and
Image descriptor is both transferred into image processing system 110.Image descriptor can be used in automatically to add image and mark
The trial of label may be provided to mankind image audit person at one or more destinations 125.Image descriptor can be by
For identifying descriptor field and the domain is subsequently used to the member of destination 125 that selection image is sent to.For example, " machine
The descriptor field of motor-car " can be used to the image audit that selection has motor vehicle speciality.Image is divided based on image descriptor
Class is into can occur on image processing system 110 or image processing logic 960 in domain.
In some embodiments, the automated tag addition of image is attempted based on image descriptor derived from institute.Various
In embodiment, this can be occurred using image processing logic 960 and/or automatic recognition system 152.Classification is alternately through comparing
From have with derived image descriptor in the image in the library of different classes of associated image descriptor and occur.For example, knowing
The image descriptor of other motor vehicle shape can be relevant to the image descriptor phase stored before " motor vehicle " classification
Match.If the identification of suitable (the classification, range etc.) classification of the category can be automatically selected for the label of image enough.Example
Such as, the image descriptor for matching those of classification " children's face " can generate label " faces of children " enough.
In general, image processing system 110 includes and the different classes of associated iamge description relative to image source 120A
The bigger library of symbol.These libraries are optionally stored on the memory 135 or image source 120A or automatic of image processing system 110
In identifying system 152.The library for the image descriptor being stored in image source 120A is optionally based on previous using image source 120A
The image of processing.For example, if multiple images from image source 120A are identified as having the label about currency and description
Symbol, in the memory 135 that the library of currency domain/classification descriptor can be stored in image source 120A.These descriptors can
With associated with the label of such as " 5 dollar bill " etc.When the new images of the identical set with descriptor are received, figure
As processing logic 960 is optionally configured to automatically add label to image using associated label.Although descriptor library can
To be received from image processing system 110, or it can be used and developed from the received image tag of image processing system 110,
Label in above example is not dependent on the real time communication with image processing system 110.
In various embodiments, the data for characterizing the relationship between image descriptor and classification and/or label can be
It is developed in image processing system 110, image source 120A, destination 125A and/or automatic recognition system 152.Once by developing,
Data can be transmitted for improving and/or replenishing the library at any other equipment.
Although illustrated system shows master-slave architecture, image source 120 and destination 125 in alternative embodiments
It is connected with end-to-end framework.In these embodiments, any combination of element shown in image processing system 110 can be by
It is included in image source 120 and/or destination 125.One in image source 120 can be from another in image source 120
Image tag addition discussed in this article and processing task are executed on received image.
Figure 10 illustrates processing according to various embodiments of the present invention at least partially in the image on image source 120A
Method.Method shown in Fig. 10 may include a series of different disposal steps executed on image source 120A.For example, those are related to
And the step of image descriptor, is optionally performed on image processing system 110.
In receiving image step 1010, image is received by image source 120A.Image can be from being included in image source
Camera in 120A is set from image source 120B, from network 115, from image processing system 110, from wireless device, from memory
Standby, and/or analog is received.Received image be optionally an image in the image sequence for be formed video.
In identification feature step 1020, image processing logic 960 is used to identify the spy of the image in received image
Sign.As discussed elsewhere herein, the method for identifying characteristics of image is well known in the art.Identification feature step
1020 can apply the one, two or more of these methods.The identification of feature optionally includes the institute of reflection feature identification
The accuracy of estimation and/or the level of confidence of integrality.
It is optionally sending in characterization step 1030, identified characteristics of image is sent in identification feature step 1020
To image processing system 110.Feature can be sent together with or without associated image and can be sent out via network 115
It send.If sending characterization step 1030 to be included in this method, this method optionally next proceeds to carry out generating/receiving mark
Step 1070 is signed, the label in the step 1070 for image is received from image processing system 110.Image processing logic
960, which are optionally configured to the level of confidence based on feature calculated in identification feature step 1020, executes transmission feature
Step 1030.For example, step can be performed and image and feature are both sent if confidence level is lower than threshold value.
In optionally export descriptor step 1040, image processing logic 960 is used to from identification feature step
Deduced image descriptor in identified characteristics of image in 1020.As discussed in this article, miscellaneous method is in ability
It is all known in domain for deduced image descriptor.In some embodiments, export descriptor step 1040 includes using
More than one method.Export may include the confidence level water for reflecting accuracy and/or integrality estimated derived from descriptor
It is flat.Derived from descriptor type and content generally depend on used in (multiple) image edge sharpening arithmetic.
It is optionally sending in descriptor step 1050, the image descriptor being exported in export descriptor step 1040
It is sent to image processing system 110.Image descriptor can be sent together with or without associated image and can be through
It is sent by network 115.If sending descriptor step 1050 to be included in this method, this method is optionally next proceeded to
It carries out generating/receiving labelling step 1070, the label in the step 1070 for image is connect from image processing system 110
It receives.Image processing logic 960 is optionally configured to based on the characteristics of image being exported in export descriptor step 1040
Level of confidence, which executes, sends descriptor step 1050.For example, step can be performed and scheme if confidence level is lower than threshold value
Picture and feature are both sent.
Optionally comparing in descriptor step 1060, the one or more being exported in export descriptor step 1040
Image descriptor is compared with the one or more image descriptors being locally stored.As elsewhere herein is discussed, this
The image descriptor being locally stored a bit is associated with image category and/or image tag.Compare may include reflecting matched matter
The calculating of the characteristic of amount.
In some embodiments, it sends descriptor step 1050 and compares descriptor step 160 the two and be performed.In the feelings
Under condition, the processing of image descriptor can occur in 110 the two of image source 125A and image processing system.Similarly, one
In a little embodiments, send both characterization step 1030 and export descriptor step 1040 be performed and characteristics of image system/
It is processed in equipment the two.
In appointment/reception labelling step 1070, the image tag for characterizing image is generated and/or receives.For example, such as
Fruit image, characteristics of image or image descriptor have been sent to image processing system 110, then corresponding label can be from finger
Image processing system 100 in group/reception labelling step 1070 is received.If exported in relatively descriptor step 1060
Descriptor and the descriptor that is locally stored between matching be found, then the image descriptor with local matched storage
Associated label is acquired from local storage and is assigned to image.Identical image tag both can locally be referred to
Group is also by local reception.Image tag is optionally generated using characteristics of image and/or descriptor, for example, without image procossing
System 110 receives actual image.
In some embodiments, appointment/reception labelling step 1070 includes that will classify to be assigned to image, to image procossing system
System 110 sends image and classification, and is received back corresponding label from image processing system 110.Label can be used shown in Fig. 4
Method it is identified.Classification can be used by image processing system 100 to use automatic recognition system 152 and/or in destination
Human reviewer at 125A generates label.
Assigned and/or reception label (and/or other results) is provided in providing result step 455, such as herein
In other places discussed.
Figure 11 illustrates the method based on image descriptor processing image according to various embodiments of the present invention.This method
It is performed on one in image source 120 with city.In the exemplary embodiment, step 1010,1020,1040 and 1060 are held
Row, it is such as described elsewhere herein.In classification image step 1110, image being processed is based on describing in export
Image in symbol step 1040 on the one or more image descriptors being exported and one be previously stored in image source 120
Matching between descriptor.The classification or multiple classifications for being assigned to image be and the matched image descriptor that was previously stored
Associated classification or multiple classifications.
In sending step 1120, image and it is assigned to the classification of the image or multiple classifications are sent to image procossing
System 110.The image is at that as described be treated to generate elsewhere herein is assigned to the image tag of image.
The processing optionally includes use classes or multiple classifications to select mankind's image audit or auxiliary automatically label image.
In receiving labelling step 1130, the label for being assigned to image is executed at this receives image step 1010
One in image source 120 is received.Label, which is subsequently rendered in, to be provided in result step 455.
Figure 12 illustrates the method using feedback processing image according to various embodiments of the present invention.This method is optionally
It is performed on image source 120A and changes including multiple communications between image source 120A and image processing system 110
Into the label of image.In providing image step 1210, image is provided to image processing system 110 from image source 120A.
In receiving the first response of step 1220, the first response is received from image processing system 110.The response can wrap
Include one or more labels.In feedback step 1230 is provided, about received image tag feedback from image source 120A
It is provided to image processing system 110.The feedback is optionally manually entered and can wrap by the human user of image source 120A
Upgrade request is included, as discussed elsewhere herein.Feedback may include include to one or more of received label
The correction of a label.For example, feedback may include the instruction that a label in label does not indicate image.Feedback may include figure
The classification of picture.
It is optionally receiving in the second response of step 1240, the second response is received from image processing system 110.Second sound
The feedback for providing and providing in feedback step 1230 should be generally used in be generated.In one example, it is contemplated that the figure of toy car
Picture, the first response include label " vehicle ", and feedback includes term " toy " and the second response includes that label " takes the super lorry of snow
(Fisher-Price Superwagon)".Method shown in Figure 12 is optionally used to improve the accuracy of image tag.
Figure 13 and Figure 14 illustrate according to various embodiments of the present invention provide image tag based on image descriptor
Method.In Figure 13, image descriptor is used to generate the figure that the source of image descriptor is then transferred on calculating network
As label.In Figure 14, image descriptor is used to determine the destination 125 for image.Shown in figs. 13 and 14
Method is optionally performed with the method illustrated elsewhere herein in combination.For example, the step of these methods can with
Those are combined shown in Fig. 4.
Particularly, referring to Fig.1 3, in receiving descriptor step 1310, characterize one or more iamge descriptions of image
Symbol is received at image processing system 110.These image descriptors are optionally received but without associated images.Only receive
Descriptor bandwidth usually more smaller than reception image request.Image descriptor is optionally via network 115 from image source 120A quilt
It receives and is generated using method shown in Figure 10 or 11.
In relatively descriptor step 1320, the received image descriptor of institute with previously deposited at image processing system 110
One or more image descriptors of storage (for example, being stored in memory 135) compare.This is relatively made is connect with determination
Whether any in the descriptor of receipts matches stored descriptor.The descriptor and one or more image tags stored
And/or classification is stored in association.For example, a set of the descriptor stored can be related to image tag " Oak Tree "
Connection.
In obtaining labelling step 1330, in response to the matching one between the descriptor received and the descriptor stored
A or multiple images label is received.Acquired image tag be it is associated with the matched set of institute those.
Provide labelling step 1340 in, acquired image tag be provided back received descriptor source, for example,
To image source 120A.They can be presented to user at that or be located in other ways as described elsewhere herein
Reason.
Figure 14 illustrate image and characterize the image data at image processing server 110 processed method.
In receiving image and data step 1410, image and the data for characterizing the image are connect at image processing server 110
It receives.The data for characterizing image for example may include the classification for characterizing the image descriptor or image of image.Image and spy
Signization data are optionally received from image source 125A.Receiving image and data step 1410 optionally is to receive image step
410 and receive source data step 420 embodiment.
In determining destination step 1420, the destination for image is determined based on the data for characterizing image.Mesh
Ground can be one in destination 125 and/or automatic recognition system 152.For example, if the data for characterizing image include
Specific classification and identified destination can be and have the associated mesh of mankind's image audit of speciality in the classification
Ground 125 in one.Determining destination step 1420 optionally is the embodiment of determining destination step 465.
In putting up image step 1430, image and optionally classification are transferred into identified destination.It is receiving
In labelling step 1440, one or more image tags are received.Image tag is based on image and is chosen so as to characterize image.
In providing labelling step 1340, image tag is provided to the source of image, for example, image source 125A.Put up image step
1430 be optionally the embodiment for putting up image step 470.
Figure 15 illustrates the method for sorting by priority image tag addition according to various embodiments of the present invention.At this
In a little methods, image grading device 190 is used to image assigned priority and the priority be used to determine image how by
Label is added, if its label to be added.In receiving image step 410, image is connect in image processing system 110
It receives, as discussed elsewhere herein.Image may come from one in image source 120, and can be by being directed to
Image crawls (crawling) webpage and is received.In some embodiments, one or more of image source 120 includes being configured
For crawl website and from these websites obtain image logic.The information being received together with image may include about from wherein
Obtain the data of the webpage of image.For example, image can with from webpage metadata and text, instruction webpage mostly often added
Carry the data of (viewing), the URL of webpage, and/or picture priority can according to its determined any other data together by
It receives, as other positions herein are discussed.
In assigned priority step 1520, image grading device 190 is used to automatically assign a priority to and be received
Image.Priority is optionally by from 1 to 100 numerical value, by subtitle grade or similar expression.Priority optionally implys that figure
(sequence) classification of picture.As described elsewhere herein, priority can be determined based on various factors.
In determining processing step 1530, the method for image addition label (processing) is determined.Determination is based on image
The priority assigned.In some embodiments, the image with lowest priority is not (addition label) processed.
The method of label addition includes automatic addition label and/or adds label manually by human reviewer, such as elsewhere herein
It is discussed.
In the step 1540 that optionally tags automatically, image is added label using automatic recognition system 152.It is automatic to add
The method for the label addition that labelling step 1540 optionally determines in determining processing step 1530 does not include automatic identification system
In the embodiment of system 152 used.Automatically the step 1540 that tags optionally is performed prior to assigned priority step 1520.Example
Such as, automatic recognition system 152 can be used by label in image, and the level of confidence for daring to the label automatically generated can be with
After be used in assigned priority step 1520 to determine and add tagged priority for manual (mankind).If automatically generated
The confidence level of label be high, then can be set low for adding tagged priority manually, and if set
Reliability is relatively low, then for add manually tagged priority can be set to it is relatively high.
In the step 1550 that optionally tags manually, image is sent to one in destination 125 for by people
Class auditor adds label.Image can be together with the label for using automatic recognition system 152 and/or various other information to generate
It is sent, it is such as described elsewhere herein.The step 1550 that tags manually may include Fig. 6 to step shown in Fig. 8
In any step.
In optional auction labelling step 1560, advertisement is assigned to image for showing on webpage.The webpage
It is optionally the webpage that image is obtained at it in receiving image step 410.It is competing as the request for webpage is received
Labelling step 1560 is clapped optionally to be perfomed substantially in real time with said storing the sensor signals.At the moment, (multiple) label for being assigned to image can be by auction
To being ready to provide a side of the maximum remuneration for advertisement to be placed in above the image or side.Auction labelling step 1560 is optional
Ground is performed using ad system 180 and auction process can be by such as GoogleEtc third party institute
Management.
In the step 1570 that optionally tags again, image is added label again.Again the step 1570 that tags can
To include the analysis for being assigned to (multiple) advertisement of image and being mostly often clicked compared with the clicking rate of expectation.For example, if
The advertisement for being assigned to image based on the first label is not clicked with expected clicking rate, then label may not be the image
Optimal expression.The image can be attempted to improve the clicking rate for the advertisement for being added label by addition label again.Again mark-on
Label step 1570 may include any one of label addition method disclosed herein, such as be discussed about Fig. 6-8 and 15
Those of method.Again it is not optimal knowledge that the step 1570 that tags, which can be used from the label that the first label generates,.
Method as shown in Figure 15 can also be applied to image sequence, for example, video.Image sequence can be present in
In browser or use various alternative applications.For example, video can be from the website of such as youtube.com etc or from such as
The stream service of Netflix, Comcast cable television, live telecast, Ruku or Hulu etc be provided to image source 120 at
Member.The image being used to determine in video whether should be added manually tagged factor include: video mostly often by viewing and/
Or the estimated value for the label expected.The automatic recognition system 152, dialogue in video, adjoint can be used in the label expected
Indicated by automatic audit of the text (for example, description, subtitle or title), and/or analog of video by image.Advertisement can be
In the video that the beginning or end of image sequence are attached or are engaged in image sequence.Thus, advertisement and video can phases
Associatedly presented together.Advertisement may include the part for being placed in image sequence, be usually including adding tagged image
Covering on part.
Multiple embodiments specifically illustrate and/or describe herein.It will be appreciated, however, that be that modifications and variations are above
Training centre covering and within scope of the appended claims without departing from its spirit and the range that is intended to.For example, herein
Disclosed image is optionally the part of the video sequence of video.Audio input can be used in destination in mankind's image audit
Image tag is provided at 125.The audio that is placed on destination 125 can be used to text conversion logic and/or figure in audio input
It is converted into text in real time as processing system 110.Image tag is optionally handled by spell check logic.As made herein
, term " real-time " means no non-essential time delay, user easily waited for.System described herein
System and method are optionally used to add label, such as music or dialogue to audio content.The audio content can be video
Part is associated with image in other ways.In some embodiments, audio content is automatically converted to text and this article
Originally it is used to auxiliary and label manually or automatically is added to image.From audio content generate text can with herein for
Those similar modes described in the text found in webpage including image are used to auxiliary and add label to image.
Embodiments described herein illustrates the present invention.Since embodiments of the invention are described referring to explanation,
The various modifications or adaptation of described method and/or specific structure can become aobvious and easy to those skilled in the art
See.It relies on the teachings of the present invention and has advanced into all these modifications, adaptation or change of this field by its these introduction
Change is considered to be within the spirit and scope of the present invention.Thus, these descriptions and attached drawing are not construed as the meaning limited
Justice, because it should be understood that the present invention is not restricted to embodiment described.
Present document relates to computing system (for example, image processing system 110, image source 120 and destination 125) may include
Integrated circuit, microprocessor, personal computer, server, distributed computing system, communication equipment, the network equipment similar are set
It is standby, and above various combinations.Computing system can also include volatile row and/or non-volatile line storage, such as deposit at random
Access to memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), magnetic medium, light are situated between
It matter, nanometer medium, hard disk, compact disk, digital versatile disc (DVD), and/or is configured for such as storing in the database
Other equipment of analog or digital information.The various examples of logic indicated above may include hardware, firmware or be stored in meter
Software on calculation machine readable medium, or combinations thereof.Computer-readable medium as used herein clearly eliminates paper.It herein refers to
The step of computer of method out executes may include the one group of instruction of storage on a computer-readable medium, upon being performed
So that computing system executes the step.Be programmed to be according to the computing system of the instruction execution specific function from program software
For executing the specific use computing system of those specific functions.When executing those specific functions by specific use computing system
The data of operation are at least electrically stored in the buffer of computing system, will using the change every time to stored data
Specific use computing system physically changes from a state to next state.The logic being discussed herein may include being stored in
Hardware, firmware, and/or software on computer-readable medium.It is special to generate that the logic can be performed in the electronic device
Purposes computing system.