CN108596070A - Character recognition method, device, storage medium, program product and electronic equipment - Google Patents

Character recognition method, device, storage medium, program product and electronic equipment Download PDF

Info

Publication number
CN108596070A
CN108596070A CN201810350902.6A CN201810350902A CN108596070A CN 108596070 A CN108596070 A CN 108596070A CN 201810350902 A CN201810350902 A CN 201810350902A CN 108596070 A CN108596070 A CN 108596070A
Authority
CN
China
Prior art keywords
personage
image
event
area
characteristic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810350902.6A
Other languages
Chinese (zh)
Inventor
汤晓鸥
黄青虬
熊宇
林达华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to CN201810350902.6A priority Critical patent/CN108596070A/en
Publication of CN108596070A publication Critical patent/CN108596070A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Abstract

A kind of character recognition method of offer of the embodiment of the present invention, device, storage medium, program product and electronic equipment, are related to artificial intelligence field.Wherein, the method includes：Feature extraction processing is carried out to image, obtains multiple first area characteristics of first personage's example in described image；Multiple first area characteristics based on the first personage example determine the weighted value of each first area characteristic in the multiple first area characteristic；Based on the weighted value of each first area characteristic in the multiple first area characteristic and the multiple first area characteristic, the recognition result of the first personage example is determined.Through the embodiment of the present invention, provincial characteristics data based on personage's example determine the weighted value of the provincial characteristics data of personage's example, and provincial characteristics data and its weighted value based on personage's example, it determines the recognition result of personage's example, the accuracy of person recognition can be improved.

Description

Character recognition method, device, storage medium, program product and electronic equipment
Technical field
The present embodiments relate to field of artificial intelligence more particularly to a kind of character recognition method, device, storage Jie Matter, program product and electronic equipment.
Background technology
As public safety problem is by social more and more concerns, the research of face recognition technology receives science The great attention on boundary, business circles and government.In China, the research of face recognition technology is started late, but in recent years due to spy Social influence caused by big public safety accident, face recognition technology have successively obtained government bodies, business circles and academia Pay attention to, the research of face recognition technology has obtained unprecedented development.In the 2008 Beijing Olympic Games and Shenzhen University's life in 2011 Fortune work(can go up face recognition technology with good public security effect is achieved, form good social effect.
Although face recognition technology achieves immense success, under abandoned environment, recognition of face still has It is challenging.In adverse conditions, recognition of face produces a large amount of difficulty, for example, relative to camera lens face be in it is non-just The position in face, face meet with unfavorable light, and face is blocked or face is too far etc. from camera lens.In practical applications, these Situation generally existing.Therefore, in complicated scene (such as side face, the figure viewed from behind, illumination variation), a people is identified by face It is inaccurate.
Invention content
The purpose of the embodiment of the present invention is, provides a kind of technical solution of person recognition.
According to a first aspect of the embodiments of the present invention, a kind of character recognition method is provided.The method includes：To image Feature extraction processing is carried out, multiple first area characteristics of first personage's example in described image are obtained；Based on described Multiple first area characteristics of first personage's example, determine each first area in the multiple first area characteristic The weighted value of characteristic；Based on every in the multiple first area characteristic and the multiple first area characteristic The weighted value of a first area characteristic determines the recognition result of the first personage example.
Optionally, described that feature extraction processing is carried out to image, obtain the multiple of first personage's example in described image First area characteristic, including：The multiple regions image of the first personage example is obtained from described image；To described more Each area image carries out feature extraction processing in a area image, and it is special to obtain the corresponding first area of each area image Levy data.
Optionally, the multiple area image corresponds to the different body regions of the first personage example respectively.
Optionally, the multiple area image includes face image, head image, the upper part of the body of the first personage example Image and whole body images.
Optionally, multiple first area characteristics based on the first personage example determine the multiple The weighted value of each first area characteristic in one provincial characteristics data, including：To the multiple first area characteristic Concatenation is carried out, the splicing feature of the first personage example is obtained；Based on the splicing feature, the multiple first is determined The weighted value of each first area characteristic in provincial characteristics data.
Optionally, described to be based on the splicing feature, determine each firstth area in the multiple first area characteristic The weighted value of characteristic of field evidence, including：The splicing feature is handled using visual attention network, is obtained the multiple The weighted value of each first area characteristic in the characteristic of first area.
Optionally, described that the splicing feature is handled using visual attention network, obtain the multiple first The weighted value of each first area characteristic in provincial characteristics data, including：Using visual attention network to the splicing Feature carries out convolution operation, obtains the characteristic pattern of the first personage example；Map operation is carried out to the characteristic pattern, obtains institute State the weight feature vector of the multiple regions feature of first personage's example；Operation is normalized to the weight feature vector, Obtain the weighted value of each first area characteristic of the first personage example.
Optionally, the weighted value of the first area characteristic depends on the corresponding area of the first area characteristic Visibility and the first area characteristic corresponding region contribution to the recognition result of the domain in described image Ratio.
Optionally, described based in the multiple first area characteristic and the multiple first area characteristic The weighted value of each first area characteristic, determines the recognition result of the first personage example, including：Based on the multiple The weighted value of each first area characteristic in the characteristic of first area carries out the multiple first area characteristic Fusion treatment obtains the barment tag data of the first personage example；Based on the barment tag data, described first is determined The recognition result of personage's example.
Optionally, the weight of each first area characteristic in described based on the multiple first area characteristic Value, before carrying out fusion treatment to the multiple first area characteristic, the method further includes：To the multiple firstth area Characteristic of field obtains multiple second area characteristics of the first personage example according to dimension-reduction treatment is carried out；Correspondingly, described Based on the weighted value of each first area characteristic in the multiple first area characteristic, to the multiple first area Characteristic carries out fusion treatment, obtains the barment tag data of the first personage example, including：Based on the multiple first The weighted value of each first area characteristic, merges the multiple second area characteristic in provincial characteristics data Processing obtains the barment tag data of the first personage example.
Optionally, described to be based on the barment tag data, determine the recognition result of the first personage example, including： It determines the barment tag data of the first personage example and each to preset appearance at least one default barment tag data special Levy the similarity of data；According to the barment tag data of the first personage example and at least one default barment tag number The similarity that barment tag data are each preset in, determines the recognition result of the first personage example.
Optionally, described based in the multiple first area characteristic and the multiple first area characteristic The weighted value of each first area characteristic, determines the recognition result of the first personage example, including：Determine described first Each first area characteristic and known piece identity's is at least one in multiple first area characteristics of personage's example Similarity in second personage's example between the corresponding region characteristic of each second personage example；According to first personage The weighted value of each of example first area characteristic and the corresponding similarity determine that first personage is real The matching value of example and the second personage example；According to the matching value, the recognition result of the first personage example is determined.
Optionally, the weighted value according to each of the first personage example first area characteristic and The corresponding similarity determines the matching value of the first personage example and the second personage example, including：According to described The weighted value of each of first personage's example first area characteristic, the corresponding region feature of the second personage example The weighted value of data and the corresponding similarity, determine the matching of the first personage example and the second personage example Value.
Optionally, before the progress feature extraction processing to image, the method further includes：Described image is carried out Size adjusting processing, the described image after being adjusted；It is described that feature extraction processing is carried out to image, it obtains in described image Multiple first area characteristics of first personage's example, including：Using convolutional neural networks to the described image after adjustment into Row feature extraction is handled, multiple first area characteristics of first personage's example in the described image after being adjusted.
Optionally, the method further includes：Determine the social context information of the affiliated image set of described image, described image collection Including multiple images, there is at least one personage's example, the social context information of described image collection includes described in each image The thing of character relation information and/or the corresponding multiple events of described image collection between the multiple personage's examples occurred in image set Part information；It is described to be based on each first in the multiple first area characteristic and the multiple first area characteristic The weighted value of provincial characteristics data determines the recognition result of the first personage example, including：Based on the multiple first area The weighted value and described image collection of each first area characteristic in characteristic, the multiple first area characteristic Social context information, determine the recognition result of the first personage example.
Optionally, the social context information of the affiliated image set of determining described image, including：Determine that described image is concentrated The scene characteristic data of at least one image and at least two each personage's examples pair of personage's example centering of described image collection Match information；Match information based on described image at least two each personage's examples pair of personage's example centering of concentration and institute The scene characteristic data for stating at least one image determine the social context information of described image collection.
Optionally, the determining described image concentrates the scene characteristic data of at least one image, including：Utilize convolution god Feature extraction processing is carried out to each image at least one image through network, is obtained each at least one image The scene characteristic data of image.
Optionally, the matching letter that at least two each personage's examples pair of personage's example centering are concentrated based on described image The scene characteristic data of breath and at least one image, determine the social context information of described image collection, including：Initialization The piece identity at least one third personage example to be identified that described image collection includes, and initialize described image collection Social context information；The corresponding personage's body of at least one 4th personage's example of known piece identity is concentrated based on described image Part, the scene of the match information of each personage's example pair of at least two personages example centering and at least one image The piece identity of characteristic, the social context information and at least one third personage example to initialization changes Generation update, until meeting stopping criterion for iteration, wherein at least two personages example is to comprising by the third personage Personage's example pair that example and the 4th personage's example are constituted.
Optionally, the personage's body at least one third personage example to be identified that the initialization described image collection includes Part, including：Based at least one corresponding piece identity of 4th personage's example and at least one third personage example In the matching of at least one personage's example pair that is constituted at least one 4th personage's example of each third personage example believe Breath, determines the initial identity of each third personage example.
Optionally, the social context information of the initialization described image collection, including：Include by described image collection at random Multiple images are divided at least one event packets so that each image in described multiple images only with an event correlation, And it is in preset range with the quantity of the image of same event correlation in described multiple images.
Optionally, the corresponding people of at least one 4th personage's example that known piece identity is concentrated based on described image Object identity, the match information of each personage's example pair of at least two personages example centering and at least one image Scene characteristic data, the piece identity of the social context information and at least one third personage example to initialization into Row iteration updates, including：Match information based on each personage's example pair of at least two personages example centering and current The social context information, update the current piece identity of at least one third personage example；Based on it is described at least The corresponding piece identity of one the 4th personage's example, the corresponding updated piece identity of at least one third personage example And the scene characteristic data of at least one image, update the current social context information.
Optionally, the match information based on each personage's example pair of at least two personages example centering and work as The preceding social context information updates the current piece identity of at least one third personage example, including：Based on institute State the match informations of at least two each personage's examples pair of personage's example centering, current the multiple event event information with And the current character relation information, update the current piece identity of at least one third personage example.
Optionally, the match information based on each personage's example pair of at least two personages example centering, current The multiple event event information and the current character relation information, update at least one third personage it is real The current piece identity of example, including：Match information based on each personage's example pair of at least two personages example centering, It is currently closed at least one image of each event correlation, the current personage in the multiple event in described multiple images It is the current probability that each personage's example participates in each event in the multiple event in information and the multiple personage's example Data update the current piece identity of at least one third personage example.
Optionally, it is described based on the corresponding piece identity of at least one 4th personage's example, described at least one the The scene characteristic data of the corresponding updated piece identity of three personage's examples and at least one image, update current The social context information, including：Based on the corresponding piece identity of at least one 4th personage's example, described at least one The scene characteristic data of the updated piece identity of third personage's example and at least one image update current institute State the event information of multiple events；Based on the corresponding piece identity of at least one 4th personage's example and described at least one The updated piece identity of third personage's example updates the current character relation information.
Optionally, it is described based on the corresponding piece identity of at least one 4th personage's example, described at least one the The scene characteristic data of the updated piece identity of three personage's examples and at least one image update current described The event information of multiple events, including：Based on the corresponding piece identity of at least one 4th personage's example, described at least one The updated piece identity of a third personage example, the scene characteristic data of at least one image, the multiple event In each event current scene characteristic data and the multiple personage's example in each personage's example participate in it is the multiple The current probability data of each event in event, update described multiple images in currently with each event in the multiple event Associated at least one image；In scene characteristic data and updated described multiple images based at least one image With at least one image of each event correlation in the multiple event, the current of each event in the multiple event is updated Scene characteristic data；Based in updated described multiple images in the multiple event each event correlation it is at least one The update of image, at least one corresponding piece identity of 4th personage's example and at least one third personage example Piece identity afterwards updates each personage's example in the multiple personage's example and participates in working as each event in the multiple event Preceding probability data.
Optionally, in the scene characteristic data and updated described multiple images based at least one image With at least one image of each event correlation in the multiple event, the current of each event in the multiple event is updated Scene characteristic data, including：Based on every at least one image of each event correlation in updated the multiple event The scene characteristic data of a image update the current scene characteristic data of each event in the multiple event.
Optionally, described based on every at least one image of each event correlation in updated the multiple event The scene characteristic data of a image update the current scene characteristic data of each event in the multiple event, including：Pass through To the scene characteristic data of each image at least one image of each event correlation in updated the multiple event It is averaging processing, obtains the updated scene characteristic data of each event in the multiple event.
Optionally, the match information based on each personage's example pair of at least two personages example centering and work as The preceding social context information updates the current piece identity of at least one third personage example, including：Based on institute The match information of at least two each personage's examples pair of personage's example centering and the current social context information are stated, is passed through So that object function maximizes, the current piece identity of at least one third personage example is updated.
Optionally, the stopping criterion for iteration, including：The updated social context information and the society before update Hand over context information identical, and the updated piece identity of at least one third personage example and described at least one the Piece identity before the update of three personage's examples is identical.
Optionally, the event information of the multiple event includes at least one of following：In described multiple images with institute State in multiple events the scene characteristic data of each event at least one image of each event correlation, the multiple event, Each personage's example participates in the probability data of each event in the multiple event in the multiple personage's example.
Optionally, the character relation information include different personage's examples in the multiple personage's example appear in it is same Probability data in image.
According to a second aspect of the embodiments of the present invention, a kind of character recognition method is provided.The method includes：Obtain figure The match information of at least two each personage's examples pair of personage's example centering in image set, wherein described image collection includes multiple Image, has at least one personage's example in each image, and described multiple images include at least one example to be identified and At least one reference example of known piece identity；Based on the corresponding piece identity of at least one reference example and it is described extremely The match information of few two each personage's examples pair of personage's example centering, determines the social context information of described image collection, described The social context information of image set include described image concentrate occur multiple personage's examples between character relation information and/or The event information of the corresponding multiple events of described image collection；Social context information based on described image collection, determine described at least The piece identity of each example to be identified in one example to be identified.
Optionally, the matching letter for obtaining at least two each personage's examples pair of personage's example centering in image set Breath, including：Barment tag data by determination the first personage of personage's example centering example and personage's example centering The similarity of the barment tag data of second personage's example obtains the match information of personage's example pair.
Optionally, the matching letter for obtaining at least two each personage's examples pair of personage's example centering in image set Breath, including：Determine each provincial characteristics in multiple first area characteristics of personage's example centering the first personage example Similarity between data and the corresponding region characteristic of personage's example centering the second personage example；According to described first The weighted value of the weighted value of each provincial characteristics data of personage's example, each provincial characteristics data of the second personage example And the similarity, determine the match information of personage's example pair.
Optionally, at least one corresponding piece identity of reference example and at least two personage are based on described The match information of each personage's example pair of example centering, before the social context information for determining described image collection, the method is also Including：Determine the scene characteristic data of at least one image in described multiple images；It is described real based at least one reference The match information of example corresponding piece identity and each personage's example pair of at least two personages example centering, determine the figure The social context information of image set, including：Based on the corresponding piece identity of at least one reference example, at least two people The scene characteristic number of at least one image in the match information and described multiple images of each personage's example pair of object example centering According to determining the social context information of described image collection.
Optionally, in the determining described multiple images at least one image scene characteristic data, including：Utilize convolution Neural network carries out feature extraction processing to each image at least one image, and the scene for obtaining each image is special Levy data.
Optionally, described to be based on the corresponding piece identity of at least one reference example, at least two personage reality The scene characteristic data of at least one image in the match information and described multiple images of the example each personage's example pair of centering, really Determine the social context information of described image collection, including：The people at least one example to be identified that initialization described image collection includes Object identity, and initialize the social context information of described image collection；Based on the corresponding personage of at least one reference example At least one in identity, the match information and described multiple images of each personage's example pair of at least two personages example centering The scene characteristic data of a image, the personage of the social context information and at least one example to be identified to initialization Identity is iterated update, until meeting stopping criterion for iteration.
Optionally, the piece identity at least one example to be identified that the initialization described image collection includes, including：Base Each example to be identified at least one corresponding piece identity of reference example and at least one example to be identified The match information of at least one personage's example pair constituted at least one reference example determines each reality to be identified The initial identity of example.
Optionally, the social context information of the initialization described image collection, including：Include by described image collection at random Multiple images are divided at least one event packets so that each image in described multiple images only with an event correlation, And it is in preset range with the quantity of the image of same event correlation in described multiple images.
Optionally, described to be based on the corresponding piece identity of at least one reference example, at least two personage reality The scene characteristic data of at least one image, right in the match information and described multiple images of the example each personage's example pair of centering The social context information of initialization and the piece identity of at least one example to be identified are iterated update, including： Match information based on each personage's example pair of at least two personages example centering and current social context letter Breath updates the current piece identity of each example to be identified at least one example to be identified；Based on described at least one The corresponding piece identity of a reference example, the updated piece identity of at least one example to be identified and the multiple The scene characteristic data of at least one image in image update the current social context information.
Optionally, the match information based on each personage's example pair of at least two personages example centering and work as The preceding social context information updates current personage's body of each example to be identified at least one example to be identified Part, including：Match information based on each personage's example pair of at least two personages example centering, current the multiple thing The event information of part and the current character relation information update each to be identified at least one example to be identified The current piece identity of example.
Optionally, the match information based on each personage's example pair of at least two personages example centering, current The multiple event event information and the current character relation information, update at least one example to be identified In each example to be identified current piece identity, including：Based on at least two personages example centering, each personage is real The match information of example pair, in described multiple images currently at least one image of each event correlation in the multiple event, Each personage's example participates in every in the multiple event in the current character relation information and the multiple personage's example The current probability data of a event update current personage's body of each example to be identified at least one example to be identified Part.
Optionally, it is described based on the corresponding piece identity of at least one reference example, it is described at least one to be identified The scene characteristic data of at least one image, update current in the updated piece identity of example and described multiple images The social context information, including：Based on the corresponding piece identity of at least one reference example, described at least one wait knowing The scene characteristic data of at least one image in the updated piece identity of other example and described multiple images, update are current The multiple event event information；Based on the corresponding piece identity of at least one reference example and described at least one The updated piece identity of example to be identified updates the current character relation information.
Optionally, it is described based on the corresponding piece identity of at least one reference example, it is described at least one to be identified The scene characteristic data of at least one image, update current in the updated piece identity of example and described multiple images The event information of the multiple event, including：Based on the corresponding piece identity of at least one reference example, described at least one It is scene characteristic data of at least one image in the updated piece identity of a example to be identified, described multiple images, described Each personage's example participates in the current scene characteristic data of each event and the multiple personage's example in multiple events The current probability data of each event in the multiple event, update described multiple images in currently in the multiple event At least one image of each event correlation；Scene characteristic data based at least one image in described multiple images and update With at least one image of each event correlation in the multiple event in described multiple images afterwards, the multiple event is updated In each event current scene characteristic data；Based in updated described multiple images with it is each in the multiple event It at least one image of event correlation, the corresponding piece identity of at least one reference example and described at least one waits knowing The updated piece identity of other example updates each personage's example in the multiple personage's example and participates in the multiple event The current probability data of each event.
Optionally, scene characteristic data based at least one image in described multiple images and updated described With at least one image of each event correlation in the multiple event in multiple images, each thing in the multiple event is updated The current scene characteristic data of part, including：Based in updated the multiple event each event correlation it is at least one The scene characteristic data of each image in image update the current scene characteristic number of each event in the multiple event According to.
Optionally, described based on every at least one image of each event correlation in updated the multiple event The scene characteristic data of a image update the current scene characteristic data of each event in the multiple event, including：Pass through To the scene characteristic data of each image at least one image of each event correlation in updated the multiple event It is averaging processing, obtains the updated scene characteristic data of each event in the multiple event.
Optionally, the match information based on each personage's example pair of at least two personages example centering and work as The preceding social context information updates current personage's body of each example to be identified at least one example to be identified Part, including：Match information based on each personage's example pair of at least two personages example centering and the current society Context information is handed over, by making object function maximize, updates each example to be identified at least one example to be identified Current piece identity.
Optionally, the stopping criterion for iteration, including：The updated social context information and the society before update Hand over context information identical, and the updated piece identity of at least one example to be identified at least one waits knowing with described Piece identity before the update of other example is identical.
Optionally, the event information of the multiple event includes at least one of following：In described multiple images with institute State in multiple events the scene characteristic data of each event at least one image of each event correlation, the multiple event, Each personage's example participates in the probability data of each event in the multiple event in the multiple personage's example.
Optionally, the character relation information include different personage's examples in the multiple personage's example appear in it is same Probability data in image.
According to a third aspect of the embodiments of the present invention, a kind of person recognition device is provided.Described device includes：At first Module is managed, for carrying out feature extraction processing to image, obtains multiple first areas of first personage's example in described image Characteristic；First determining module, be used for multiple first area characteristics based on the first personage example, determine described in The weighted value of each first area characteristic in multiple first area characteristics；Second determining module, for based on described The weight of each first area characteristic in multiple first area characteristics and the multiple first area characteristic Value, determines the recognition result of the first personage example.
Optionally, the first processing module, is specifically used for：The more of the first personage example are obtained from described image A area image；Feature extraction processing is carried out to each area image in the multiple area image, obtains each region The corresponding first area characteristic of image.
Optionally, the multiple area image corresponds to the different body regions of the first personage example respectively.
Optionally, the multiple area image includes face image, head image, the upper part of the body of the first personage example Image and whole body images.
Optionally, first determining module, including：Splice submodule, for the multiple first area characteristic According to concatenation is carried out, the splicing feature of the first personage example is obtained；First determination sub-module, for being based on the splicing Feature determines the weighted value of each first area characteristic in the multiple first area characteristic.
Optionally, first determination sub-module, is specifically used for：Using visual attention network to the splicing feature into Row processing, obtains the weighted value of each first area characteristic in the multiple first area characteristic.
Optionally, first determination sub-module, is specifically used for：Using visual attention network to the splicing feature into Row convolution operation obtains the characteristic pattern of the first personage example；Map operation is carried out to the characteristic pattern, obtains described first The weight feature vector of the multiple regions feature of personage's example；Operation is normalized to the weight feature vector, obtains institute State the weighted value of each first area characteristic of first personage's example.
Optionally, the weighted value of the first area characteristic depends on the corresponding area of the first area characteristic Visibility and the first area characteristic corresponding region contribution to the recognition result of the domain in described image Ratio.
Optionally, second determining module, including：First processing submodule, for being based on the multiple first area The weighted value of each first area characteristic, carries out at fusion the multiple first area characteristic in characteristic Reason obtains the barment tag data of the first personage example；Second determination sub-module, for being based on the barment tag number According to determining the recognition result of the first personage example.
Optionally, before the first processing submodule, second determining module further includes：Second processing submodule Block obtains multiple the second of the first personage example for carrying out dimension-reduction treatment to the multiple first area characteristic Provincial characteristics data；Correspondingly, the first processing submodule, is specifically used for：Based on the multiple first area characteristic In each first area characteristic weighted value, fusion treatment is carried out to the multiple second area characteristic, obtains institute State the barment tag data of first personage's example.
Optionally, second determination sub-module, is specifically used for：Determine the barment tag data of the first personage example With the similarity for each presetting barment tag data at least one default barment tag data；According to the first personage example Barment tag data and at least one default barment tag data in each preset the similarities of barment tag data, really The recognition result of the fixed first personage example.
Optionally, second determining module, including：Third determination sub-module, for determining the first personage example Multiple first area characteristics in each first area characteristic and known piece identity at least one second personage Similarity in example between the corresponding region characteristic of each second personage example；4th determination sub-module is used for basis The weighted value of each of the first personage example first area characteristic and the corresponding similarity, determine institute State the matching value of first personage's example and the second personage example；5th determination sub-module is used for according to the matching value, really The recognition result of the fixed first personage example.
Optionally, the 4th determination sub-module, is specifically used for：According to each of described first personage example described first The weighted value of the corresponding region characteristic of the weighted value of provincial characteristics data, the second personage example and corresponding described Similarity determines the matching value of the first personage example and the second personage example.
Optionally, before the first processing module, described device further includes：Second processing module, for described Image carries out size adjusting processing, the described image after being adjusted；The first processing module, is specifically used for：Utilize convolution Neural network carries out feature extraction processing to the described image after adjustment, and the first personage in the described image after being adjusted is real Multiple first area characteristics of example.
Optionally, described device further includes：Third determining module, the social feelings for determining the affiliated image set of described image Border information, described image collection include multiple images, have at least one personage's example, the social activity of described image collection in each image Contextual information includes the character relation information and/or described image set pair between multiple personage's examples that described image concentration occurs The event information for the multiple events answered；Second determining module, including：6th determination sub-module, for based on the multiple The weighted value of each first area characteristic and institute in first area characteristic, the multiple first area characteristic The social context information for stating image set determines the recognition result of the first personage example.
Optionally, the third determining module, including：7th determination sub-module, for determining that described image is concentrated at least Of the scene characteristic data of one image and at least two each personage's examples pair of personage's example centering of described image collection With information；8th determination sub-module, for concentrating at least two each personage's examples pair of personage's example centering based on described image Match information and at least one image scene characteristic data, determine the social context information of described image collection.
Optionally, the 7th determination sub-module, is specifically used for：Using convolutional neural networks at least one image In each image carry out feature extraction processing, obtain the scene characteristic data of each image at least one image.
Optionally, the 8th determination sub-module, including：Initialization unit includes for initializing described image collection The piece identity of at least one third personage example to be identified, and initialize the social context information of described image collection；Repeatedly For updating unit, the corresponding personage's body of at least one 4th personage's example for concentrating known piece identity based on described image Part, the scene of the match information of each personage's example pair of at least two personages example centering and at least one image The piece identity of characteristic, the social context information and at least one third personage example to initialization changes Generation update, until meeting stopping criterion for iteration, wherein at least two personages example is to comprising by the third personage Personage's example pair that example and the 4th personage's example are constituted.
Optionally, the initialization unit, is specifically used for：Based on the corresponding personage of at least one 4th personage's example Each third personage example and at least one 4th personage example structure in identity and at least one third personage example At at least one personage's example pair match information, determine the initial identity of each third personage example.
Optionally, the initialization unit, is specifically used for：At random by the multiple images that described image collection includes be divided into Few event packets so that each image in described multiple images only with an event correlation, and described multiple images In be in preset range with the quantity of the image of same event correlation.
Optionally, the iteration updating unit, including：First update subelement, for being based at least two personage The match information of each personage's example pair of example centering and the current social context information, update described at least one the The current piece identity of three personage's examples；Second update subelement, for being based at least one 4th personage's example pair Piece identity, at least one corresponding updated piece identity of third personage example and at least one figure answered The scene characteristic data of picture update the current social context information.
Optionally, the first update subelement, is specifically used for：Based on at least two personages example centering everyone The match information of object example pair, the event information of current the multiple event and the current character relation information, more The current piece identity of new at least one third personage example.
Optionally, the first update subelement, is specifically used for：Based on at least two personages example centering everyone Current at least one figure with each event correlation in the multiple event in the match information of object example pair, described multiple images Each personage's example participates in the multiple event in picture, the current character relation information and the multiple personage's example The current probability data of each event, update the current piece identity of at least one third personage example.
Optionally, the second update subelement, is specifically used for：It is corresponding based at least one 4th personage's example The scene of piece identity, the updated piece identity of at least one third personage example and at least one image Characteristic updates the event information of current the multiple event；It is corresponding based at least one 4th personage's example The updated piece identity of piece identity and at least one third personage example update current character relation letter Breath.
Optionally, the second update subelement, is specifically used for：It is corresponding based at least one 4th personage's example Piece identity, the updated piece identity of at least one third personage example, the scene of at least one image are special Levy data, in the multiple event in the current scene characteristic data of each event and the multiple personage's example everyone Object example participates in the current probability data of each event in the multiple event, update in described multiple images currently with it is described At least one image of each event correlation in multiple events；Scene characteristic data based at least one image and update With at least one image of each event correlation in the multiple event in described multiple images afterwards, the multiple event is updated In each event current scene characteristic data；Based in updated described multiple images with it is each in the multiple event At least one image of event correlation, the corresponding piece identity of at least one 4th personage's example and described at least one It is the multiple to update each personage's example participation in the multiple personage's example by the updated piece identity of third personage's example The current probability data of each event in event.
Optionally, the second update subelement, is specifically used for：Based on each event in updated the multiple event The scene characteristic data of each image in associated at least one image update the current of each event in the multiple event Scene characteristic data.
Optionally, the second update subelement, is specifically used for：By to each thing in updated the multiple event The scene characteristic data of each image in the associated at least one image of part are averaging processing, and obtain in the multiple event The updated scene characteristic data of each event.
Optionally, the first update subelement, is specifically used for：Based on at least two personages example centering everyone The match information of object example pair and the current social context information, by making object function maximize, described in update The current piece identity of at least one third personage example.
Optionally, the stopping criterion for iteration, including：The updated social context information and the society before update Hand over context information identical, and the updated piece identity of at least one third personage example and described at least one the Piece identity before the update of three personage's examples is identical.
Optionally, the event information of the multiple event includes at least one of following：In described multiple images with institute State in multiple events the scene characteristic data of each event at least one image of each event correlation, the multiple event, Each personage's example participates in the probability data of each event in the multiple event in the multiple personage's example.
Optionally, the character relation information include different personage's examples in the multiple personage's example appear in it is same Probability data in image.
According to a fourth aspect of the embodiments of the present invention, a kind of person recognition device is provided, described device includes：First obtains Modulus block, the match information for obtaining at least two each personage's examples pair of personage's example centering in image set, wherein institute It includes multiple images to state image set, has at least one personage's example in each image, and described multiple images include at least At least one reference example of one example to be identified and known piece identity；4th determining module is used for based on described at least The match information of the corresponding piece identity of one reference example and each personage's example pair of at least two personages example centering, Determine the social context information of described image collection, the social context information of described image collection includes that described image concentrates the more of appearance The event information of the corresponding multiple events of character relation information and/or described image collection between a personage's example；5th determines Module is used for the social context information based on described image collection, determines each to be identified at least one example to be identified The piece identity of example.
Optionally, first acquisition module, is specifically used for：Pass through determination the first personage of personage's example centering example Barment tag data and personage's example centering the second personage example barment tag data similarity, obtain the people The match information of object example pair.
Optionally, first acquisition module, is specifically used for：Determine the more of personage's example centering the first personage example The corresponding region of each provincial characteristics data and personage's example centering the second personage example in a first area characteristic Similarity between characteristic；According to the weighted value of each provincial characteristics data of the first personage example, described second The weighted value of each provincial characteristics data of personage's example and the similarity determine the matching letter of personage's example pair Breath.
Optionally, before the 4th determining module, described device further includes：6th determining module, for determining State the scene characteristic data of at least one image in multiple images；4th determining module, including：9th determination sub-module, For being based on the corresponding piece identity of at least one reference example, each personage's reality of at least two personages example centering The scene characteristic data of at least one image, determine the society of described image collection in the match information and described multiple images of example pair Hand over contextual information.
Optionally, the 6th determining module, is specifically used for：Using convolutional neural networks at least one image Each image carries out feature extraction processing, obtains the scene characteristic data of each image.
Optionally, the 9th determination sub-module, including：Initialization unit includes for initializing described image collection The piece identity of at least one example to be identified, and initialize the social context information of described image collection；Iteration updating unit, For being based on the corresponding piece identity of at least one reference example, each personage's reality of at least two personages example centering The scene characteristic data of at least one image, the social activity to initialization in the match information and described multiple images of example pair The piece identity of context information and at least one example to be identified is iterated update, is until meeting stopping criterion for iteration Only.
Optionally, the initialization unit, is specifically used for：Based on the corresponding piece identity of at least one reference example It is constituted at least one reference example with each example to be identified at least one example to be identified at least one The match information of personage's example pair determines the initial identity of each example to be identified.
Optionally, the initialization unit, is specifically used for：At random by the multiple images that described image collection includes be divided into Few event packets so that each image in described multiple images only with an event correlation, and described multiple images In be in preset range with the quantity of the image of same event correlation.
Optionally, the iteration updating unit, including：First update subelement, for being based at least two personage The match information of each personage's example pair of example centering and the current social context information, update is described at least one to be waited for Identify the current piece identity of each example to be identified in example；Second update subelement, for based on described at least one The corresponding piece identity of reference example, the updated piece identity of at least one example to be identified and the multiple figure The scene characteristic data of at least one image, update the current social context information as in.
Optionally, the first update subelement, is specifically used for：Based on at least two personages example centering everyone The match information of object example pair, the event information of current the multiple event and the current character relation information, more The current piece identity of each example to be identified in new at least one example to be identified.
Optionally, the first update subelement, is specifically used for：Based on at least two personages example centering everyone Current at least one figure with each event correlation in the multiple event in the match information of object example pair, described multiple images Each personage's example participates in the multiple event in picture, the current character relation information and the multiple personage's example The current probability data of each event update the current personage of each example to be identified at least one example to be identified Identity.
Optionally, the second update subelement, is specifically used for：Based on the corresponding personage of at least one reference example At least one image in identity, the updated piece identity of at least one example to be identified and described multiple images Scene characteristic data update the event information of current the multiple event；It is corresponding based at least one reference example The updated piece identity of piece identity and at least one example to be identified update current character relation letter Breath.
Optionally, the second update subelement, is specifically used for：Based on the corresponding personage of at least one reference example The field of at least one image in identity, the updated piece identity of at least one example to be identified, described multiple images It is every in the current scene characteristic data of each event and the multiple personage's example in scape characteristic, the multiple event A personage's example participates in the current probability data of each event in the multiple event, update in described multiple images currently with At least one image of each event correlation in the multiple event；Scene based at least one image in described multiple images In characteristic and updated described multiple images at least one image of each event correlation in the multiple event, more The current scene characteristic data of each event in new the multiple event；Based in updated described multiple images with it is described At least one image of each event correlation in multiple events, the corresponding piece identity of at least one reference example and institute The updated piece identity for stating at least one example to be identified updates each personage's example in the multiple personage's example and participates in The current probability data of each event in the multiple event.
Optionally, the second update subelement, is specifically used for：Based on each event in updated the multiple event The scene characteristic data of each image in associated at least one image update the current of each event in the multiple event Scene characteristic data.
Optionally, the second update subelement, is specifically used for：By to each thing in updated the multiple event The scene characteristic data of each image in the associated at least one image of part are averaging processing, and obtain in the multiple event The updated scene characteristic data of each event.
Optionally, the first update subelement, is specifically used for：Based on at least two personages example centering everyone The match information of object example pair and the current social context information, by making object function maximize, described in update The current piece identity of each example to be identified at least one example to be identified.
Optionally, the stopping criterion for iteration, including：The updated social context information and the society before update Hand over context information identical, and the updated piece identity of at least one example to be identified at least one waits knowing with described Piece identity before the update of other example is identical.
Optionally, the event information of the multiple event includes at least one of following：In described multiple images with institute State in multiple events the scene characteristic data of each event at least one image of each event correlation, the multiple event, Each personage's example participates in the probability data of each event in the multiple event in the multiple personage's example.
Optionally, the character relation information include different personage's examples in the multiple personage's example appear in it is same Probability data in image.
According to a fifth aspect of the embodiments of the present invention, a kind of computer readable storage medium is provided, meter is stored thereon with Calculation machine program instruction, wherein the people described in first aspect of the embodiment of the present invention is realized in described program instruction when being executed by processor The step of object recognition methods.
According to a sixth aspect of the embodiments of the present invention, a kind of computer readable storage medium is provided, meter is stored thereon with Calculation machine program instruction, wherein the people described in second aspect of the embodiment of the present invention is realized in described program instruction when being executed by processor The step of object recognition methods.
According to a seventh aspect of the embodiments of the present invention, a kind of computer program product is provided comprising have computer journey Sequence instructs, wherein the person recognition described in first aspect of the embodiment of the present invention is realized in described program instruction when being executed by processor The step of method.
According to a eighth aspect of the embodiments of the present invention, a kind of computer program product is provided comprising have computer journey Sequence instructs, wherein the person recognition described in second aspect of the embodiment of the present invention is realized in described program instruction when being executed by processor The step of method.
According to a ninth aspect of the embodiments of the present invention, a kind of electronic equipment is provided, including：First processor and first is deposited Reservoir, the first memory make the first processor hold for storing an at least executable instruction, the executable instruction Character recognition method of the row as described in first aspect of the embodiment of the present invention.
According to a tenth aspect of the embodiments of the present invention, a kind of electronic equipment is provided, including：Second processor and second is deposited Reservoir, the second memory make the second processor hold for storing an at least executable instruction, the executable instruction Character recognition method of the row as described in second aspect of the embodiment of the present invention.
The technical solution provided according to embodiments of the present invention carries out feature extraction processing to image, obtains the in image Multiple first area characteristics of one personage's example, multiple first area characteristics based on first personage's example determine The weighted value of each first area characteristic in multiple first area characteristics, and it is based on multiple first area characteristics And in multiple first area characteristics each first area characteristic weighted value, determine the identification of first personage's example As a result, compared with other modes, the provincial characteristics data based on personage's example determine the power of the provincial characteristics data of personage's example Weight values, and provincial characteristics data and its weighted value based on personage's example, determine the recognition result of personage's example, can improve people The accuracy of object identification.
Description of the drawings
Fig. 1 is a kind of flow chart of character recognition method according to some embodiments of the invention；
Fig. 2 is the schematic diagram of the person recognition for the embodiment of the method for implementing Fig. 1；
Fig. 3 is the schematic diagram of the person recognition for the embodiment of the method for implementing Fig. 1；
Fig. 4 is the schematic diagram of the network frame for the embodiment of the method for implementing Fig. 1；
Fig. 5 is the flow chart according to a kind of character recognition method of other embodiments of the invention；
Fig. 6 is the flow chart according to a kind of character recognition method of other embodiments of the invention；
Fig. 7 is the schematic diagram of the personage for the embodiment of the method for implementing Fig. 6 and the relationship of event；
Fig. 8 is the schematic diagram of the personage for the embodiment of the method for implementing Fig. 6 and the relationship of personage；
Fig. 9 is the schematic diagram of the social context model for the embodiment of the method for implementing Fig. 6；
Figure 10 is a kind of structure diagram of person recognition device according to some embodiments of the invention；
Figure 11 is the structure diagram according to a kind of person recognition device of other embodiments of the invention；
Figure 12 is the structure diagram according to a kind of person recognition device of other embodiments of the invention；
Figure 13 is the structure diagram according to a kind of person recognition device of other embodiments of the invention；
Figure 14 is the structure diagram according to a kind of person recognition device of other embodiments of the invention；
Figure 15 is the structure diagram of a kind of electronic equipment according to some embodiments of the invention；
Figure 16 is the structure diagram according to a kind of electronic equipment of other embodiments of the invention.
Specific implementation mode
(identical label indicates identical element in several attached drawings) and embodiment below in conjunction with the accompanying drawings, implement the present invention The specific implementation mode of example is described in further detail.Following embodiment is not limited to the present invention for illustrating the present invention Range.
It will be understood by those skilled in the art that the terms such as " first ", " second " in the embodiment of the present invention are only used for distinguishing Different step, equipment or module etc. neither represent any particular technology meaning, also do not indicate that the inevitable logic between them is suitable Sequence.
Fig. 1 is a kind of flow chart of character recognition method according to some embodiments of the invention.
In step S101, feature extraction processing is carried out to image, obtains the more of first personage's example in described image A first area characteristic.
In the embodiments of the present disclosure, for the content for including from image, described image may include the one of first personage's example The image of partial image or entire first personage example, for example, the face image of first personage's example, first personage's example Head image, the back image of first personage's example, the upper part of the body image of first personage's example and first personage's example Whole body images etc..For the classification of image, described image can be still image, or be the video frame figure in video sequence Picture can also be composograph etc..The embodiment of the present disclosure is not construed as limiting the specific implementation of image.
In the embodiments of the present disclosure, the first area characteristic may include face characteristic data, head feature data, Upper part of the body characteristic or systemic features data etc..In some optional embodiments, the first area characteristic can have Body is provincial characteristics vector, for example, face feature vector, head feature are vectorial, upper part of the body feature vector or systemic features are vectorial Deng.Optionally, the dimension of provincial characteristics vector can be 1024 dimensions or 2048 dimensions or other numerical value.The embodiment of the present disclosure to this Any restriction is not done in the specific implementation of one provincial characteristics data.
In the embodiments of the present disclosure, can feature extraction processing directly be carried out to image, alternatively, can also be carried out to image One or more processing, and image carries out feature extraction processing to treated.Optionally, image progress feature is carried described Before taking processing, size adjusting processing, the described image after being adjusted can be carried out to described image；Recycle convolutional Neural net Network carries out feature extraction processing to the described image after adjustment, first personage's example in the described image after being adjusted it is more A first area characteristic.It in some optional embodiments, can be according to pre- when carrying out size adjusting processing to described image If size carries out size adjusting processing, the described image after being adjusted to described image.Wherein, the pre-set dimension can be by this Field technology personnel set according to actual needs, and the embodiment of the present disclosure is not intended to be limited in any this.Take this, it can be accurately Obtain the first area characteristic of first personage's example in image.
In some optional embodiments, when carrying out feature extraction processing to image, described the is obtained from described image The multiple regions image of one personage's example；Feature extraction processing is carried out to each area image in the multiple area image, is obtained Obtain the corresponding first area characteristic of each area image.It is understood that the present embodiment is without being limited thereto, it is any right The embodiment that image carries out feature extraction processing may be applicable to this.Wherein, the multiple area image can correspond to institute respectively State the different body regions of first personage's example.In some optional embodiments, the multiple area image includes described first Face image, head image, upper part of the body image and the whole body images of personage's example.It is understood that the embodiment of the present disclosure is not It is limited to this, for example, the multiple area image may also include the face image, head image, upper half of the first personage example Body image and lower part of the body image, the multiple area image may also include the face image of the first personage example, head figure Picture and upper part of the body image.The embodiment of the present disclosure is not limited in any way the specific implementation of area image.
In some optional embodiments, in the multiple regions image for obtaining the first personage example from described image When, the multiple of the first personage example can be obtained from described image by the neural network for obtaining region limitting casing Area image.It is understood that the present embodiment is without being limited thereto, any multiple regions that first personage's example is obtained from image The embodiment of image may be applicable to this, and the present embodiment is not intended to be limited in any this.Further, it is also possible to obtain by other means The multiple regions image of the first personage example is taken, for example, can be obtained by the data set or detector of mark the first The multiple regions image of object example.The embodiment of the present disclosure does not do this any restrictions.
It, can be first by the neural network for obtaining region limitting casing, from image in an alternate embodiment of the present invention The limitting casing of the middle multiple regions for obtaining first personage's example, then according to the limitting casing pair of the multiple regions of first personage's example Image is cut, and the multiple regions image of first personage's example is obtained.In another alternative embodiment of the present invention, it can lead to first Cross mark data set or detector obtained from image first personage's example multiple regions limitting casing, then according to first The limitting casing of the multiple regions of personage's example cuts image, obtains the multiple regions image of first personage's example.Certainly, The embodiment of the present disclosure is without being limited thereto, the embodiment of any multiple regions image that first personage's example is obtained from image Suitable for this, the embodiment of the present disclosure is not intended to be limited in any this.In addition, if some area image does not detect or first The some regions of personage's example are largely invisible, simply can replace the area image not detected using black image Or most of sightless area image.
In some optional embodiments, each area image carries out feature extraction processing in the multiple area image When, feature extraction can be carried out to each area image in the multiple area image by the neural network for feature extraction, Obtain the corresponding first area characteristic of each area image in the multiple area image.It is understood that the disclosure Embodiment is without being limited thereto, and any embodiment for carrying out feature extraction respectively to the multiple area image may be applicable to this, The embodiment of the present disclosure does not do this any restriction.
In a specific example, when the multiple area image includes face image, the head of first personage's example When image, upper part of the body image and whole body images, to the face image of the first personage example, head image, scheme above the waist Picture and whole body images zoom in and out respectively, so that the face image of the first personage example, head image, above the waist figure The size of picture and whole body images adjusts separately as normal size.Then, the face figure after size being adjusted Picture, head image, upper part of the body image and whole body images are separately input into the first convolutional neural networks, the second convolution nerve net Network, third convolutional neural networks and Volume Four accumulate neural network, and each convolutional neural networks distinguish corresponding area image Feature extraction processing is carried out, obtains the face feature data, head feature data, upper part of the body characteristic of first personage's example respectively According to this and systemic features data.It is understood that above description is merely illustrative ground, optionally, its other party can also be passed through Formula carries out feature extraction processing respectively to the multiple area image, and the embodiment of the present disclosure does not limit this.
In step s 102, multiple first area characteristics based on the first personage example, determine the multiple The weighted value of each first area characteristic in the characteristic of first area.
In some optional embodiments, it in multiple first area characteristics based on the first personage example, determines In the multiple first area characteristic when the weighted value of each first area characteristic, to the multiple first area spy It levies data and carries out concatenation, obtain the splicing feature of the first personage example；Based on the splicing feature, determine described more The weighted value of each first area characteristic in a first area characteristic.It is understood that the embodiment of the present disclosure is not It is limited to this, any provincial characteristics data based on personage's example determine the implementation of the weighted value of the provincial characteristics data of personage's example Mode may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, it based on the splicing feature, determines in the multiple first area characteristic When the weighted value of each first area characteristic, the splicing feature is handled using visual attention network, is obtained To the weighted value of each first area characteristic in the multiple first area characteristic.It is understood that the disclosure Embodiment is without being limited thereto, and any splicing feature based on personage's example determines the weighted value of the provincial characteristics data of personage's example Embodiment may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, it when being handled the splicing feature using visual attention network, utilizes Visual attention network carries out convolution operation to the splicing feature, obtains the characteristic pattern of the first personage example；To described Characteristic pattern carries out map operation, obtains the weight feature vector of the multiple regions feature of the first personage example；To the power Operation is normalized in weight feature vector, obtains the weighted value of each first area characteristic of the first personage example. It is any using visual attention network splicing feature to be handled it is understood that the embodiment of the present disclosure is without being limited thereto Embodiment may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In an alternate embodiment of the present invention, the visual attention network may include convolutional layer, be connected to the convolution Layer output end full articulamentum and be connected to the full articulamentum output end computation layer.Optionally, the computation layer Can be sigmoid function layers.It is understood that above description is merely illustrative ground, optionally, the knot of visual attention network Structure can also be other structures, and the embodiment of the present disclosure does not limit this.
In a specific example, when being handled the splicing feature using visual attention network, pass through The convolutional layer carries out convolution operation to the splicing feature, obtains the characteristic pattern of the first personage example；By described complete Articulamentum carries out map operation to the characteristic pattern, obtains the weight feature vector of the provincial characteristics of the first personage example； Finally by the computation layer, operation is normalized to the weight feature vector, obtains the every of the first personage example The weighted value of a first area characteristic.It is understood that above description is merely illustrative ground, optionally, vision is utilized The other embodiment that attention network handles splicing feature may be applicable to this, and the embodiment of the present disclosure does not limit this It is fixed.
In some optional embodiments, the weighted value of the first area characteristic depends on the first area feature Visibility and the first area characteristic corresponding region of the corresponding region of data in described image are to the knowledge The contribution proportion of other result.Take this, visibility that can be based on each region of first personage's example and to first personage's example Recognition result percentage contribution adaptively determine first personage's example the corresponding weight of each first area feature Value, so as to improve the accuracy of person recognition.
In a specific example, since the coverage of camera is limited or is blocked, the one of first personage's example A little regions may be sightless.As shown in Fig. 2, the body of personage's example of left figure is sightless, and the personage of left figure is real The face of example is visible；The body of personage's example of right figure is visible, and the face of personage's example of right figure is invisible 's.Therefore, the face feature weight and physical trait weight of personage's example of left figure and right figure are different.Specifically, right For personage's example of left figure, face feature weight is more than physical trait weight；For personage's example of right figure, face Feature weight is less than physical trait weight.In addition, identification result of the different zones of different personage's examples to personage's example Percentage contribution be different.As shown in figure 3, identification knot of the face area of personage's example of left figure to personage's example The percentage contribution of fruit will be significantly greater than the contribution of the face area of personage's example of right figure to the identification result of personage's example Degree.
In step s 103, the multiple first area characteristic and the multiple first area characteristic are based on In each first area characteristic weighted value, determine the recognition result of the first personage example.
In some optional embodiments, based on the multiple first area characteristic and the multiple first area The weighted value of each first area characteristic can base when determining the recognition result of the first personage example in characteristic The weighted value of each first area characteristic in the multiple first area characteristic, to the multiple first area spy It levies data and carries out fusion treatment, obtain the barment tag data of the first personage example；Based on the barment tag data, really The recognition result of the fixed first personage example.It is understood that the embodiment of the present disclosure is without being limited thereto, it is any real based on personage The provincial characteristics data and its weighted value of example determine that the embodiment of the recognition result of personage's example may be applicable to this, the disclosure Embodiment does not do this any restriction.
In the embodiments of the present disclosure, can fusion treatment directly be carried out to the multiple first area characteristic, alternatively, One or more processing, and the multiple firstth area to treated can also be carried out to the multiple first area characteristic Characteristic of field is according to progress fusion treatment.Optionally, right before carrying out fusion treatment to the multiple first area characteristic The multiple first area characteristic carries out dimension-reduction treatment respectively, and the multiple second areas for obtaining the first personage example are special Data, then the weighted value based on each first area characteristic in the multiple first area characteristic are levied, to described more A second area characteristic carries out fusion treatment, obtains the barment tag data of the first personage example.Take this, Neng Goujie Calculating time and the computing resource for saving equipment, to improve the speed of person recognition.
In some optional embodiments, when carrying out dimension-reduction treatment respectively to the multiple first area characteristic, can lead to The full articulamentum for reducing data dimension is crossed, dimension-reduction treatment is carried out respectively to each first area characteristic, is obtained The corresponding second area characteristic of the first personage example.It is understood that the embodiment of the present disclosure is without being limited thereto, appoint The embodiment what carries out the multiple first area characteristic dimension-reduction treatment respectively may be applicable to this, and the disclosure is implemented Example does not do this any restriction.
In a specific example, when the face that the multiple first area characteristic includes first personage's example is special When levying data, head feature data, upper part of the body characteristic and systemic features data, by the face of the first personage example Characteristic, head feature data, upper part of the body characteristic and systemic features data are separately input into the first full articulamentum, Two full articulamentums, the full articulamentum of third and the 4th full articulamentum, each full articulamentum distinguish corresponding provincial characteristics data Dimension-reduction treatment is carried out, obtains face feature data, head feature data, upper half after the dimension-reduction treatment of first personage's example respectively Body characteristic and systemic features data.Specifically, by each full articulamentum to it is corresponding 2048 dimension provincial characteristics to After amount carries out dimension-reduction treatment, the provincial characteristics vector of 256 dimensions after dimension-reduction treatment can get.It is understood that retouching above It states and is merely illustrative ground, optionally, each first area characteristic can also be dropped respectively by other means Dimension processing, the embodiment of the present disclosure do not limit this.
In some optional embodiments, the input terminal of the described first full articulamentum can be with above-mentioned first convolutional neural networks Output end connects, and the input terminal of the second full articulamentum can be connect with the output end of above-mentioned second convolutional neural networks, described The input terminal of the full articulamentum of third can be connect with the output end of above-mentioned third convolutional neural networks, the 4th full articulamentum it is defeated Entering end can connect with the output end of above-mentioned Volume Four product neural network.Specifically, each convolutional neural networks and it is connected to convolution The full articulamentum of neural network output end may make up regiospecificity convolutional neural networks.For example, the first convolution nerve net Network and the first full articulamentum constitute the regiospecificity convolutional neural networks for face image feature extraction, the second convolutional Neural Network and the second full articulamentum constitute the regiospecificity convolutional neural networks for head image feature extraction, third convolution god The regiospecificity convolutional neural networks for upper part of the body image characteristics extraction, Volume Four are constituted through network and the full articulamentum of third Product neural network and the 4th full articulamentum constitute the regiospecificity convolutional neural networks for whole body images feature extraction.It can be with Understanding, above description is merely illustrative ground, and optionally, regiospecificity convolutional neural networks also have other structures, The embodiment of the present disclosure does not limit this.
In some optional embodiments, it when carrying out fusion treatment to each second area characteristic, can be based on The weighted value of each first area characteristic in the multiple first area characteristic, to each first area feature The corresponding second area characteristic of data is weighted processing, obtains the second area characteristic after multiple weightings；To institute It states the second area characteristic after multiple weightings and carries out concatenation, obtain the barment tag number of the first personage example According to.Wherein, multiple second area characteristics of the first personage example are mutual complementary, are taken this, acquisition it is the first The barment tag data of object example can more accurately describe the appearance of first personage's example.It is understood that the disclosure Embodiment is without being limited thereto, and any embodiment for carrying out fusion treatment to each second area characteristic may be applicable to This, the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, based on the barment tag data, the identification of the first personage example is determined When as a result, determine that the barment tag data of the first personage example are each preset at least one default barment tag data The similarity of barment tag data；According to the barment tag data of the first personage example and at least one default appearance The similarity that barment tag data are each preset in characteristic, determines the recognition result of the first personage example.It can manage Solution, the embodiment of the present disclosure is without being limited thereto, any recognition result that first personage's example is determined based on barment tag data Embodiment may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, it is preset at least one in the barment tag data for determining the first personage example When each presetting the similarity of barment tag data in barment tag data, calculate the barment tag of the first personage example to Amount respectively with COS distance that barment tag vector is each preset at least one default barment tag vector.It is appreciated that It is that the embodiment of the present disclosure is without being limited thereto, barment tag data and the default barment tag data of any determination the first personage example The embodiment of similarity may be applicable to this, the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, according to the barment tag data of the first personage example with it is described at least one The similarity that barment tag data are each preset in default barment tag data, determines the recognition result of the first personage example When, by the barment tag vector of the first personage example respectively with each preset at least one default barment tag vector it is outer The COS distance of table feature vector is compared with default COS distance threshold value respectively, is determining the outer of the first personage example When table feature vector and the COS distance of default barment tag vector are more than default COS distance threshold value, first personage is determined The recognition result of example is the corresponding identity information of the default barment tag vector.Wherein, the default COS distance threshold value It can according to actual needs be set by those skilled in the art, the embodiment of the present disclosure does not do this any restriction.Optionally, institute It states identity information and may include name or identification card number of personage's example etc..
In some optional embodiments, based on the multiple first area characteristic and the multiple first area The weighted value of each first area characteristic in characteristic when determining the recognition result of the first personage example, determines Each first area characteristic is with known piece identity's in multiple first area characteristics of the first personage example Similarity at least one second personage example between the corresponding region characteristic of each second personage example；According to described The weighted value of each of first personage's example first area characteristic and the corresponding similarity determine described The matching value of one personage's example and the second personage example；According to the matching value, the knowledge of the first personage example is determined Other result.It is understood that the embodiment of the present disclosure is without being limited thereto, any provincial characteristics data and its power based on personage's example Weight values determine that the embodiment of the recognition result of personage's example may be applicable to this, and the embodiment of the present disclosure does not do this any limit It is fixed.
In some optional embodiments, each in the multiple first area characteristics for determining the first personage example In first area characteristic and at least one second personage example the corresponding region characteristic of each second personage example it Between similarity when, calculate each first area feature vector in multiple first area feature vectors of the first personage example With the COS distance between the corresponding region feature vector of each second personage example at least one second personage example.It can be with Understand, the embodiment of the present disclosure is without being limited thereto, first area characteristic and the second people of any determination the first personage example The embodiment of similarity between the corresponding region characteristic of object example may be applicable to this, the embodiment of the present disclosure to this not Do any restriction.
In some optional embodiments, according to each of the first personage example first area characteristic Weighted value and the corresponding similarity, when determining matching value of the first personage example with the second personage example, It can be according to the weighted value of each of the first personage example first area characteristic, pair of the second personage example The weighted value of provincial characteristics data and the corresponding similarity are answered, determines the first personage example and second personage The matching value of example.It is understood that the embodiment of the present disclosure is without being limited thereto, any determination the first personage example and the second personage The embodiment of the matching value of example may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In a specific example, according to each of the first personage example first area characteristic The weighted value of the corresponding region characteristic of weighted value, the second personage example and the corresponding similarity, determine institute When stating matching value of the first personage's example with the second personage example, it can be calculated by following formula one：
Wherein, s (i, j) indicates that the matching value of the first personage example i and the second personage example j, R indicate personage The quantity of the different zones of example,Indicate the weighted value of the provincial characteristics vector of the region r of the first personage example i,Indicate the weighted value of the provincial characteristics vector of the region r of the second personage example j, sr(i, j) indicates first personage The COS distance of the provincial characteristics vector of the region r of example i and the provincial characteristics vector of the region r of the second personage example j.
In some optional embodiments, according to the matching value, when determining the recognition result of the first personage example, The first personage example is compared with preset matching value threshold value respectively with the matching value of each second personage example, When determining that the matching value of the first personage example and the second personage example is more than preset matching value threshold value, described in determination The recognition result of first personage's example is the identity information of the second personage example.That is to say, the first personage example with The second personage example is identical personage's example.Wherein, the preset matching value threshold value can be by those skilled in the art's root It is set according to actual needs, the embodiment of the present disclosure does not do this any restriction.Optionally, the identity information may include personage Name or identification card number of example etc..
In some optional embodiments, based on the multiple first area characteristic and the multiple first area The weighted value of each first area characteristic in characteristic, it is right before the recognition result for determining the first personage example Each first area characteristic carries out dimension-reduction treatment respectively in the multiple first area characteristic, obtains described the The corresponding second area characteristic of one personage's example, then based on the multiple second area characteristic and the multiple The weighted value of each second area characteristic, determines the identification knot of the first personage example in second area characteristic Fruit.Wherein, dimensionality reduction is carried out respectively to each first area characteristic in the multiple first area characteristic herein The embodiment of processing is divided with above with respect to each first area characteristic in the multiple first area characteristic Not carry out dimension-reduction treatment embodiment it is similar, details are not described herein.Each second in the multiple second area characteristic The weighted value of provincial characteristics data is specially the power of the corresponding first area characteristic of each second area characteristic Weight values.Take this, calculating time and the computing resource of equipment can be saved, to improve the speed of person recognition.
In some optional embodiments, based on the multiple second area characteristic and the multiple second area The weighted value of each second area characteristic in characteristic when determining the recognition result of the first personage example, determines Each second area characteristic is with known piece identity's in multiple second area characteristics of the first personage example Similarity at least one second personage example between the corresponding region characteristic of each second personage example；According to described The weighted value of each of first personage's example second area characteristic and the corresponding similarity determine described The matching value of one personage's example and the second personage example；According to the matching value, the knowledge of the first personage example is determined Other result.Wherein, it is determined here that the second area characteristic of first personage's example and the corresponding region of second personage's example are special Levy the embodiment of the similarity between data and the first area characteristic of the first personage example identified above and the second people The embodiment of similarity between the corresponding region characteristic of object example is similar, and details are not described herein.It is determined here that first The embodiment of the matching value of personage's example and second personage's example and the first personage example identified above and second personage's example Matching value embodiment it is similar, details are not described herein.The first personage example is determined herein according to the matching value The embodiment party of the embodiment of recognition result and the recognition result that the first personage example is determined above according to the matching value Formula is similar, and details are not described herein.It is understood that the embodiment of the present disclosure is without being limited thereto, any region based on personage's example Characteristic and its weighted value determine that the embodiment of the recognition result of personage's example may be applicable to this, the embodiment of the present disclosure pair This does not do any restriction.
In a specific example, as shown in figure 4, including four personage's instance Xs in images to be recognized1、X2、X3And X4.It is assumed that personage's instance X1For target person example, personage's instance X is obtained first1Face image, head image, above the waist scheme Picture and whole body images, then by personage's instance X1Face image, head image, upper part of the body image and whole body images difference The regiospecificity convolutional neural networks being made of accordingly convolutional neural networks and full articulamentum are inputted, personage's instance X is obtained1 Dimension-reduction treatment after face feature data, the head feature data after dimension-reduction treatment, the upper part of the body characteristic after dimension-reduction treatment Systemic features data according to this and after dimension-reduction treatment.Subsequently, to personage's instance X of corresponding convolutional neural networks output1's Face feature data, head feature data, upper part of the body characteristic and systemic features data carry out concatenation, are integrated Personage's instance X1Face area, head zone, upper part of the body region and the splicing feature in whole body region.Subsequently, will splice Feature inputs the visual attention network being made of convolutional layer, full articulamentum and sigmoid function layers, obtains personage's instance X1 Face feature weight, head feature weight, upper part of the body feature weight and systemic features weight.Finally, according to personage's example X1Face feature weight, head feature weight, upper part of the body feature weight and systemic features weight to personage's instance X1Dimensionality reduction at The head feature data after face feature data, dimension-reduction treatment after reason, the upper part of the body characteristic and dimensionality reduction after dimension-reduction treatment Systemic features data that treated carry out fusion treatment, final to obtain personage's instance X1Barment tag data, or according to people Object instance X1Face feature weight, head feature weight, upper part of the body feature weight, systemic features weight, after dimension-reduction treatment Upper part of the body characteristic after head feature data, dimension-reduction treatment after face feature data, dimension-reduction treatment and dimension-reduction treatment Systemic features data afterwards, determine personage's instance X1With the matching value of other personage's examples of known identities information.Wherein, optional Ground needs before using area specificity convolutional neural networks and visual attention network to regiospecificity convolutional Neural Network and visual attention network are trained.Specifically, all regiospecificity convolutional neural networks and visual attention Network is used together intersection entropy loss and carries out joint training in a manner of end to end.
The character recognition method provided according to embodiments of the present invention carries out feature extraction processing to image, obtains in image First personage's example multiple first area characteristics, multiple first area characteristics based on first personage's example, It determines the weighted value of each first area characteristic in multiple first area characteristics, and is based on multiple first area features The weighted value of each first area characteristic, determines first personage's example in data and multiple first area characteristics Recognition result, compared with other modes, the provincial characteristics data based on personage's example determine the provincial characteristics data of personage's example Weighted value, and provincial characteristics data and its weighted value based on personage's example determine the recognition result of personage's example, Neng Gouti The accuracy of high person recognition.
The character recognition method of the present embodiment can be set by any suitable terminal with image or data-handling capacity Standby or server executes, wherein the terminal device includes but not limited to：Camera, terminal, mobile terminal, PC machine, server, Mobile unit, amusement equipment, advertising equipment, personal digital assistant (PDA), tablet computer, laptop, handheld device, Intelligent glasses, smartwatch, wearable device, virtual display device or display enhancing equipment (such as Google Glass, Oculus Rift, Hololens, Gear VR) etc., it is not limited in the embodiment of the present invention.
Fig. 5 is the flow chart according to a kind of character recognition method of other embodiments of the invention.
In step s 201, feature extraction processing is carried out to image, obtains the more of first personage's example in described image A first area characteristic.
Since the embodiment of step S201 is similar with the embodiment of above-mentioned steps S101, details are not described herein.
In step S202, multiple first area characteristics based on the first personage example determine the multiple The weighted value of each first area characteristic in the characteristic of first area.
Since the embodiment of step S202 is similar with the embodiment of above-mentioned steps S102, details are not described herein.
In step S203, the social context information of the affiliated image set of described image is determined.
In the embodiments of the present disclosure, described image collection includes multiple images, is had in each image in described multiple images There is at least one personage's example, and described multiple images include at least one third personage example to be identified and known personage At least one 4th personage's example of identity.The social context information of described image collection includes that described image concentrates the multiple of appearance The event information of the corresponding multiple events of character relation information and/or described image collection between personage's example.Wherein, the people Object relation information can conceptually be interpreted as often waiting for personage's example together with target person example or often stay in Specific crowd together.The event information can conceptually be interpreted as the work that specific participant participates in specific place It is dynamic.One event not only shares similar scene, but also the set of also specific participant.In some optional embodiments, The character relation information includes the multiple personage's example (including at least one third personage example and described at least one A 4th personage's example) in different personage's examples appear in the probability data in same image.The event of the multiple event Information includes at least one of following：In described multiple images in the multiple event each event correlation it is at least one Each personage's example participates in the scene characteristic data of each event, the multiple personage's example in image, the multiple event The probability data of each event in the multiple event.It is understood that above description is merely illustrative ground, the disclosure is implemented Example does not limit this.
In some optional embodiments, when determining the social context information of the affiliated image set of described image, described in determination In image set at least two personage's example centerings of the scene characteristic data of at least one image and described image collection everyone The match information of object example pair；The matching letter of at least two each personage's examples pair of personage's example centering is concentrated based on described image The scene characteristic data of breath and at least one image, determine the social context information of described image collection.It is appreciated that It is that the embodiment of the present disclosure is without being limited thereto, the embodiment of the social context information of any determining affiliated image set of image can fit For this, the embodiment of the present disclosure does not do this any restriction.
In an alternate embodiment of the present invention, each personage in at least two personage's examples pair for determining described image collection When the match information of example pair, by determine each personage's example centering personage's example barment tag data with The similarity of the barment tag data of another personage's example obtains the match information of each personage's example pair.Wherein, this The embodiment and the first personage of acquisition above that place obtains the barment tag data of each each personage's example of personage's example centering The embodiment of the barment tag data of example is similar, and details are not described herein.It is determined here that each personage's example centering The embodiment of the similarity of the barment tag data of one personage's example and the barment tag data of another personage's example with The barment tag data of first personage example identified above are similar with the default embodiment of similarity of barment tag data, This is repeated no more.It is understood that the embodiment of the present disclosure is without being limited thereto, at least two personages in any determining image set are real The embodiment of the match information of the example each personage's example pair of centering may be applicable to this, and the embodiment of the present disclosure does not do this any It limits.
In another alternative embodiment of the present invention, in at least two personage's examples pair for determining described image collection everyone When the match information of object example pair, multiple first area characteristics of each described one personage's example of personage's example centering are determined Similarity between each first area characteristic and the corresponding region characteristic of another personage's example；According to institute It is described to state each of the weighted value of each of personage's example first area characteristic, another described personage's example The weighted value of first area characteristic and the similarity determine the match information of each personage's example pair.Wherein, It is determined here that the embodiment of the similarity between the provincial characteristics data of each personage's example two personage's examples of centering with it is upper Text determines the phase between the first area characteristic and the corresponding region characteristic of second personage's example of first personage's example Embodiment like degree is similar, and details are not described herein.It is determined here that the embodiment party of the match information of each personage's example pair Formula is similar with each embodiment of matching value of second personage example with the first personage example identified above, no longer superfluous herein It states.It is understood that the embodiment of the present disclosure is without being limited thereto, at least two personage's example centerings in any determining image set are every The embodiment of the match information of a personage's example pair may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, when determining that described image concentrates the scene characteristic data of at least one image, profit Feature extraction processing is carried out to each image at least one image with convolutional neural networks, obtains at least one figure The scene characteristic data of each image as in.It is understood that the embodiment of the present disclosure is without being limited thereto, any determining described image The embodiment of the scene characteristic data of at least one image is concentrated to may be applicable to this, further, it is also possible to by other means The scene characteristic data of at least one image are obtained, such as receive the scene characteristic number of at least one image from other equipment According to.In a specific example, server can receive the scene characteristic data of at least one image from terminal device, etc. Deng the embodiment of the present disclosure is not intended to be limited in any this.
In some optional embodiments, each personage's example in concentrating at least two personage's examples pair based on described image To match information and at least one image scene characteristic data, determine the social context information of described image collection When, the piece identity at least one third personage example to be identified that initialization described image collection includes, and initialize institute State the social context information of image set；At least one 4th personage's example of known piece identity is concentrated to correspond to based on described image Piece identity, each personage's example pair of at least two personages example centering match information and at least one figure The scene characteristic data of picture, personage's body of the social context information and at least one third personage example to initialization Part is iterated update, until meeting stopping criterion for iteration.Wherein, at least two personages example is to comprising by described Personage's example pair that third personage example and the 4th personage's example are constituted.It is understood that the embodiment of the present disclosure is unlimited In this, the embodiment of the social context information of any determining image set may be applicable to this, and the embodiment of the present disclosure does not do this Any restriction.
In some optional embodiments, the stopping criterion for iteration, including：The updated social context information with more The social context information before new is identical, and the updated piece identity of at least one third personage example and institute The piece identity stated before the update of at least one third personage example is identical.It is understood that the embodiment of the present disclosure is not limited to This, such as the stopping criterion for iteration can also include：Iterations reach predetermined threshold value, or may be the group of the two Close, etc., the embodiment of the present disclosure is not intended to be limited in any this.
In some optional embodiments, at least one third personage reality to be identified for including in initialization described image collection When the piece identity of example, it is based at least one corresponding piece identity of 4th personage's example and at least one third party At least one personage's example pair that each third personage example is constituted at least one 4th personage's example in object example Match information determines the initial identity of each third personage example.It is understood that the embodiment of the present disclosure is not limited to This, the embodiment of the piece identity at least one third personage example to be identified that any initialisation image collection includes Suitable for this, the embodiment of the present disclosure does not do this any restriction.
In a specific example, when determining the initial identity of each third personage example, described is determined Three personage's examples and the maximum value in the matching value of each 4th personage's example at least one 4th personage's example, are based on Matching value with the third personage example is that the piece identity of the 4th personage's example of maximum value determines that the third personage is real The initial identity of example.It is understood that above description is merely illustrative ground, the embodiment of the present disclosure does not limit this.
In some optional embodiments, when initializing the social context information of described image collection, at random by described image The multiple images that collection includes are divided at least one event packets so that each image in described multiple images only with a thing Part is associated with, and is in preset range with the quantity of the image of same event correlation in described multiple images.Wherein, described pre- If range can according to actual needs be set by those skilled in the art, the embodiment of the present disclosure does not do this any restriction.It can With understanding, the embodiment of the present disclosure is without being limited thereto, the embodiment of the social context information of any initialisation image collection Suitable for this, the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, concentrating at least one 4th personage of known piece identity real based on described image The corresponding piece identity of example, each personage's example pair of at least two personages example centering match information and it is described at least The scene characteristic data of one image, the social context information and at least one third personage example to initialization When piece identity is iterated update, the match information based on each personage's example pair of at least two personages example centering with And the current social context information, update the current piece identity of at least one third personage example；Based on institute State the corresponding piece identity of at least one 4th personage's example, the corresponding updated people of at least one third personage example The scene characteristic data of at least one image in object identity and described multiple images update current social context letter Breath.It is understood that the embodiment of the present disclosure is without being limited thereto, any social context information to initialization and at least one third The piece identity of personage's example is iterated newer embodiment and may be applicable to this, and the embodiment of the present disclosure does not do this any It limits.
In some optional embodiments, in based on at least two personages example pair each personage's example pair matching Information and the current social context information, update the current piece identity of at least one third personage example When, match information, current the multiple event based on each personage's example pair of at least two personages example centering Event information and the current character relation information, update current personage's body of at least one third personage example Part.It is understood that the embodiment of the present disclosure is without being limited thereto, the reality of the current piece identity of any update third personage's example The mode of applying may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, in based on at least two personages example pair each personage's example pair matching Information, the event information of current the multiple event and the current character relation information, update are described at least one When the current piece identity of third personage's example, based on each personage's example pair of at least two personages example centering With in information, described multiple images currently at least one image of each event correlation, current institute in the multiple event It states each personage's example in character relation information and the multiple personage's example and participates in each event in the multiple event Current probability data update the current piece identity of at least one third personage example.It is understood that the disclosure Embodiment is without being limited thereto, and the embodiment of the current piece identity of any update third personage's example may be applicable to this, this Open embodiment does not do this any restriction.
In some optional embodiments, when updating the current piece identity of at least one third personage example, In match information, described multiple images based on each personage's example pair of at least two personages example centering currently with it is described At least one image of each event correlation, the current character relation information and the multiple personage are real in multiple events Each personage's example participates in the probability data of each event in the multiple event in example, by making object function maximize, Update the current piece identity of at least one third personage example.It is understood that the embodiment of the present disclosure is not limited to This, the embodiment of the current piece identity of any at least one third personage example of update may be applicable to this, the disclosure Embodiment does not do this any restriction.
In a specific example, the matching of each personage's example pair in based on at least two personages example pair In information, described multiple images currently at least one image of each event correlation in the multiple event, current described Each personage's example participates in the general of each event in the multiple event in character relation information and the multiple personage's example Rate data, by making object function maximize, when updating the current piece identity of at least one third personage example, Especially by making following object function one maximize, current personage's body of at least one third personage example is updated Part：
Wherein,Indicate the match information institute structure of each personage's example pair of at least two personages example centering At the first matrix in jth row L dimensional vectors, IiIndicate the personage's example index for i-th of image that described image is concentrated Set, α is coefficient, may be, for example, 0.05, xjIndicate at least one 4th personage's example and at least one third The column vector of jth row in the 4th matrix that the piece identity of personage's example is constituted, that is, in the image of described image collection Personage's example j identity vector, be only arranged to 1 there are one element in the vector, other elements are arranged to 0, It indicates at least one 4th personage's example and at least one third personage example described in each personage's example participation In the third matrix that the probability data of each event is constituted in multiple eventsThe column vector of row, that is, it is described at least Each personage's example participates in event in one the 4th personage's example and at least one third personage exampleProbability data,Indicate the associated event of i-th of image that described image is concentrated, β is coefficient, and may be, for example, 0.01, α and β can pass through intersection Verification determines that Q indicates the difference in the 7th matrix that the character relation information is constituted, that is, the multiple personage's example Personage's example appears in the 7th matrix that the probability data in same image is constituted, and Q (l, l ') indicates the figure of described image collection The probability number that personage's example that personage's example that piece identity as in is l. is l ' with piece identity occurs in same image According to xj′Indicate the column vector of jth ' row in the 4th matrix, that is, personage's example j ' in the image of described image collection Identity vector.Specifically, the 4th matrix is the matrix of L × N, and N indicates personage's example in the image of described image collection The quantity of (third personage example and the 4th personage's example), L indicate the different people of personage's example in the image of described image collection The quantity of object identity, the third matrix are the matrix of L × K, and K indicates the associated different event of image that described image is concentrated Quantity, the 7th matrix are the matrix of L × L.
In a specific example, when described image concentrate comprising third personage's example image in only include the third party When object example, the current piece identity of the third personage example can be updated according to following formula two：
Wherein,Indicate the identity vector of the updated third personage example, that is, the updated described 4th The column vector of the jth row of matrix, l indicate the piece identity of the third personage example.Specifically, above-mentioned formula two can be by above-mentioned Object function one is converted to obtain.
In a specific example, when described image concentrate comprising third personage's example image in include two or two When a above personage's example, it can will be converted into the problem of maximizing above-mentioned object function one in the image based on described image collection The MRF (Markov Random Field, markov random file) of the piece identity of personage's example then uses max product Algorithm jointly updates the piece identity of the third personage example.
In some optional embodiments, in based on at least two personages example pair each personage's example pair matching Information and the current social context information, update the current piece identity of at least one third personage example When, match information and current the multiple event based on each personage's example pair of at least two personages example centering Event information updates the current piece identity of at least one third personage example.It is understood that the disclosure is implemented Example is without being limited thereto, and the embodiment of the current piece identity of any at least one third personage example of update may be applicable to This, the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, in based on at least two personages example pair each personage's example pair matching The event information of information and current the multiple event updates current personage's body of at least one third personage example It is current in match information, described multiple images based on each personage's example pair of at least two personages example centering when part With each personage's example at least one image of each event correlation in the multiple event and the multiple personage's example It participates in the current probability data of each event in the multiple event, updates the current of at least one third personage example Piece identity.It is understood that the embodiment of the present disclosure is without being limited thereto, it is any to update the current of at least one third personage example The embodiment of piece identity may be applicable to this, the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, when updating the current piece identity of at least one third personage example, In match information, described multiple images based on each personage's example pair of at least two personages example centering currently with it is described Each personage's example participates in institute at least one image of each event correlation and the multiple personage's example in multiple events The current probability data for stating each event in multiple events, by making object function maximize, update described at least one the The current piece identity of three personage's examples.It is understood that the embodiment of the present disclosure is without being limited thereto, any update third personage The embodiment of the current piece identity of example may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In a specific example, the matching of each personage's example pair in based on at least two personages example pair In information, described multiple images currently at least one image of each event correlation in the multiple event and the multiple Each personage's example participates in the probability data of each event in the multiple event in personage's example, by making object function most Bigization, when updating the current piece identity of at least one third personage example, especially by making following object function Two maximize, and update the current piece identity of at least one third personage example：
In a specific example, when described image concentrate comprising third personage's example image in only include the third party When object example, the current piece identity of the third personage example can be updated according to above formula two, details are not described herein.
In a specific example, when described image concentrate comprising third personage's example image in include two or two When a above personage's example, it can will be converted into the problem of maximizing above-mentioned object function two in the image based on described image collection Then the MRF of the piece identity of personage's example uses max product algorithm jointly to update the personage of the third personage example Identity.
In some optional embodiments, in based on at least two personages example pair each personage's example pair matching Information and the current social context information, update the current piece identity of at least one third personage example When, match information and the current character relation based on each personage's example pair of at least two personages example centering are believed Breath updates the current piece identity of at least one third personage example.It is understood that the embodiment of the present disclosure is unlimited In this, the embodiment of the current piece identity of any at least one third personage example of update may be applicable to this, this public affairs It opens embodiment and does not do any restriction to this.
In some optional embodiments, in based on at least two personages example pair each personage's example pair matching Information and the current character relation information, when updating the current piece identity of at least one third personage example, In match information and the multiple personage's example based on each personage's example pair of at least two personages example centering not The current probability data in same image are appeared in personage's example, update the current of at least one third personage example Piece identity.It is understood that the embodiment of the present disclosure is without being limited thereto, current personage's body of any update third personage's example The embodiment of part may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, when updating the current piece identity of at least one third personage example, In match information and the multiple personage's example based on each personage's example pair of at least two personages example centering not The current probability data in same image are appeared in personage's example, by making object function maximize, described in update at least The current piece identity of one third personage's example.It is understood that the embodiment of the present disclosure is without being limited thereto, any update the The embodiment of the current piece identity of three personage's examples may be applicable to this, and the embodiment of the present disclosure does not do this any limit It is fixed.
In a specific example, the matching of each personage's example pair in based on at least two personages example pair Different personage's examples in information and the multiple personage's example appear in the current probability data in same image, by making Object function maximizes, when updating the current piece identity of at least one third personage example, especially by make with Lower object function three maximizes, and updates the current piece identity of at least one third personage example：
In a specific example, when described image concentrate comprising third personage's example image in only include the third party When object example, the current piece identity of the third personage example can be updated according to following formula three：
Wherein, above-mentioned formula three can be converted to obtain by above-mentioned object function three.
In a specific example, when described image concentrate comprising third personage's example image in include two or two When a above personage's example, it can will be converted into the problem of maximizing above-mentioned object function three in the image based on described image collection Then the MRF of the piece identity of personage's example uses max product algorithm jointly to update the personage of the third personage example Identity.
In some optional embodiments, based on the corresponding piece identity of at least one 4th personage's example, described The field of at least one image in the corresponding updated piece identity of at least one third personage example and described multiple images Scape characteristic when updating the current social context information, is based on the corresponding people of at least one 4th personage's example At least one figure in object identity, the updated piece identity of at least one third personage example and described multiple images The scene characteristic data of picture update the event information of current the multiple event；It is real based at least one 4th personage The updated piece identity of example corresponding piece identity and at least one third personage example, update the current people Object relation information.It is understood that the embodiment of the present disclosure is without being limited thereto, any implementation for updating current social context information Mode may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, based on the corresponding piece identity of at least one 4th personage's example, described The scene of at least one image is special in the updated piece identity of at least one third personage example and described multiple images Data are levied, it is corresponding based at least one 4th personage's example when updating the event information of current the multiple event At least one figure in piece identity, the updated piece identity of at least one third personage example, described multiple images The current scene characteristic data of each event and the multiple personage are real in the scene characteristic data of picture, the multiple event Each personage's example participates in the current probability data of each event in the multiple event in example, updates in described multiple images Currently at least one image of each event correlation in the multiple event；Based at least one image in described multiple images Scene characteristic data and updated described multiple images in in the multiple event each event correlation it is at least one Image updates the current scene characteristic data of each event in the multiple event；Based on updated described multiple images In people corresponding at least one image of each event correlation in the multiple event, at least one 4th personage's example The updated piece identity of object identity and at least one third personage example updates every in the multiple personage's example A personage's example participates in the current probability data of each event in the multiple event.It is understood that the disclosure is implemented Example is without being limited thereto, and the embodiment of the event information of any current multiple events of update may be applicable to this, and the disclosure is implemented Example does not do this any restriction.
In some optional embodiments, it is currently closed with each event in the multiple event in updating described multiple images When at least one image of connection, by being based on the corresponding piece identity of at least one 4th personage's example, described at least one The scene characteristic data of at least one image, institute in the updated piece identity of a third personage example, described multiple images State in multiple events each personage's example ginseng in the current scene characteristic data of each event and the multiple personage's example Object function is maximized with the current probability data of each event in the multiple event, is updated current in described multiple images With at least one image of each event correlation in the multiple event.It is understood that the embodiment of the present disclosure is without being limited thereto, The current embodiment party at least one image of each event correlation in the multiple event in any update described multiple images Formula may be applicable to this, and the embodiment of the present disclosure does not do this any restriction.
In a specific example, by be based on the corresponding piece identity of at least one 4th personage's example, The scene of at least one image is special in the updated piece identity of at least one third personage example, described multiple images Levy data, in the multiple event in the current scene characteristic data of each event and the multiple personage's example everyone The current probability data that object example participates in each event in the multiple event maximizes object function, updates the multiple figure As in currently in the multiple event when at least one image of each event correlation, especially by the following target letter of maximization Number four updates current at least one image with each event correlation in the multiple event in described multiple images：
Wherein, the constraints one and the constraints to the object function four second is that carry out maximized two Constraints, M indicate that the quantity for the image that described image is concentrated, K indicate the associated different event of image that described image is concentrated Quantity,Indicate the vector for i-th of image, k-th of event of association that described image is concentrated, that is, in described multiple images With at least one image of each event correlation in the multiple event (or associated thing of each image in described multiple images Part) the i-th row in the second matrix for being constituted column vector, be only arranged to 1 there are one element in the vector, other elements It is arranged to 0, second matrix is the matrix of K × M.
Wherein,
Wherein, pkIndicate the column vector of the kth row of the third matrix, that is, at least one 4th personage's example The probability data of event k, f are participated in each personage's example at least one third personage exampleiIndicate the multiple figure The column vector of the i-th row in the 5th matrix that the scene characteristic data of at least one image are constituted as in, that is, the figure The scene characteristic vector of i-th of image in image set, the 5th matrix are DfThe matrix of × M, DfIndicate what described image was concentrated The dimension of the scene characteristic vector of image,Indicate the scene characteristic data of each event in the multiple event are constituted The column vector of kth row in six matrixes, that is, the scene characteristic of the associated event k of image of described image concentration are vectorial, institute It is D to state the 6th matrixfThe matrix of × K, xjIt indicates the column vector of the jth row in updated 4th matrix, that is, updates The identity vector of personage's example j in the image of described image collection afterwards, vminAnd vmaxThe granularity of control event is indicated respectively Parameter.The inventors of the present application found that the granularity of event is very significant considering that the performance of person recognition.If by image set In image be grouped into the event of coarseness, event each in this way may include many personage's examples or many scenes, So event and the relation information of personage's example can not provide the prediction to person recognition well.However, if by image The image of concentration is grouped into fine-grained event, it is difficult to reliably estimate relevant parameter.Therefore, it is necessary to compromise to find a kind of put down Weighing apparatus, so that the quantity with the image of an event correlation falls into rational range.In the present embodiment, present inventor Use parameter vminAnd vmaxThe granularity of control event, and the quantity with the image of an event correlation is required to fall into range [vmin, vmax]。
In a particular embodiment, above-mentioned constraints one executes a kind of constraint, which makes described image concentrate Each image at most with an event correlation.Above-mentioned constraints two executes the constraint of the granularity of another control event.This Sample, the maximization Solve problems based on above-mentioned object function four, constraints one and constraints two are a linear programmings Problem can be easy solution by LP (Linear Programming, linear programming) solver and obtain.Also, optimal solution is protected Card is complete.
In some optional embodiments, the scene characteristic data of at least one image and more in based on described multiple images At least one image in described multiple images after new with each event correlation in the multiple event, updates the multiple thing In part when the current scene characteristic data of each event, extremely based on each event correlation in updated the multiple event The scene characteristic data of each image in a few image, the current scene for updating each event in the multiple event are special Levy data.It is understood that the embodiment of the present disclosure is without being limited thereto, each event is current in any the multiple event of update The embodiments of scene characteristic data may be applicable to this, the embodiment of the present disclosure does not do this any restriction.
In some optional embodiments, in based on updated the multiple event each event correlation it is at least one The scene characteristic data of each image in image update the current scene characteristic data of each event in the multiple event When, it is special by the scene to each image at least one image of each event correlation in updated the multiple event Sign data are averaging processing, and obtain the updated scene characteristic data of each event in the multiple event.It is appreciated that , the embodiment of the present disclosure is without being limited thereto, the current scene characteristic data of each event in any the multiple event of update Embodiment may be applicable to this, the embodiment of the present disclosure does not do this any restriction.
In a specific example, at least the one of each event correlation in by updated the multiple event The scene characteristic data of each image in a image are averaging processing, and obtain the update of each event in the multiple event When rear scene characteristic data, it can be calculated especially by following formula four in the multiple event after the update of each event Scene characteristic data：
Wherein,Indicate the column vector of the kth row in updated 6th matrix, that is, the updated figure The scene characteristic vector of the associated event k of image in image set,Indicate that described image concentrates association thing The set of the image of part k, fiIndicate the column vector of the i-th row of the 5th matrix, that is, i-th of figure that described image is concentrated The scene characteristic vector of picture.
In a specific example, in based on updated described multiple images with each thing in the multiple event The associated at least one image of part, the corresponding piece identity of at least one 4th personage's example and described at least one The updated piece identity of three personage's examples updates each personage's example in the multiple personage's example and participates in the multiple thing In part when the current probability data of each event, it can be calculated especially by following formula seven in the multiple personage's example Each personage's example participates in the current probability data of each event in the multiple event：
Wherein, pkIndicate the column vector of the kth row of the updated third matrix, that is, it is updated it is described at least Each personage's example participates in the probability data of event k in one the 4th personage's example and at least one third personage example,Indicate that described image concentrates the set of the image of correlating event k, IiIndicate described image is concentrated i-th The set of personage's example index of a image, xjIndicate the column vector of the jth row of updated 4th matrix, that is, more The identity vector of personage's example j in the image of described image collection after new.
In some optional embodiments, based on the corresponding piece identity of at least one 4th personage's example and described The updated piece identity of at least one third personage example, when updating the current character relation information, based on described Updated personage's body of at least one 4th personage's example corresponding piece identity and at least one third personage example Part, the different personage's examples updated at least one 4th personage's example and at least one third personage example occur Current probability data in same image.
In a specific example, based on the corresponding piece identity of at least one 4th personage's example and described The updated piece identity of at least one third personage example, update at least one 4th personage's example and it is described at least It, can be especially by when different personage's examples in one third personage's example appear in the current probability data in same image The difference at least one 4th personage's example and at least one third personage example is calculated in following formula six Personage's example appears in the updated probability data in same image：
Q=Q '/| | Q ' | |FFormula six
Wherein,
Wherein, Q indicates that updated 7th matrix, M indicate the quantity for the image that described image is concentrated, xjIt indicates more The column vector of the jth row of the 4th matrix after new, that is, personage's example j in the image of updated described image collection Identity vector, xj′Indicate the column vector of jth ' row of updated 4th matrix, that is, updated described image The identity vector of personage's example j ' in the image of collection.
In some optional embodiments, based on the corresponding piece identity of at least one 4th personage's example, described The field of at least one image in the corresponding updated piece identity of at least one third personage example and described multiple images Scape characteristic when updating the current social context information, is based on the corresponding people of at least one 4th personage's example At least one figure in object identity, the updated piece identity of at least one third personage example and described multiple images The scene characteristic data of picture update the event information of current the multiple event.Wherein, it is based on described at least one the herein The corresponding piece identity of four personage's examples, the updated piece identity of at least one third personage example and described more The scene characteristic data of at least one image in a image, update the embodiment of the event information of current the multiple event With above based on the corresponding piece identity of at least one 4th personage's example, at least one third personage example more The scene characteristic data of at least one image in piece identity and described multiple images after new update current the multiple The embodiment of the event information of event is similar, and details are not described herein.It is understood that the embodiment of the present disclosure is without being limited thereto, Any embodiment for updating current social context information may be applicable to this, and the embodiment of the present disclosure does not do this any limit It is fixed.
In some optional embodiments, based on the corresponding piece identity of at least one 4th personage's example, described The field of at least one image in the corresponding updated piece identity of at least one third personage example and described multiple images Scape characteristic when updating the current social context information, is based on the corresponding people of at least one 4th personage's example The updated piece identity of object identity and at least one third personage example updates current character relation letter Breath.Wherein, it is based at least one corresponding piece identity of 4th personage's example and at least one third personage herein The updated piece identity of example, update the embodiment of the current character relation information with above based on it is described at least The updated piece identity of one the 4th personage's example corresponding piece identity and at least one third personage example, more The embodiment of the new current character relation information is similar, and details are not described herein.It is understood that the embodiment of the present disclosure It is without being limited thereto, it is any update current social context information embodiment may be applicable to this, the embodiment of the present disclosure to this not Do any restriction.
In a specific example, concentrating at least one 4th personage of known piece identity real based on described image The corresponding piece identity of example, each personage's example pair of at least two personages example centering match information and it is described at least The scene characteristic data of one image, the social context information and at least one third personage example to initialization When piece identity is iterated update, based on first matrix, second matrix, the third matrix and the described 7th Matrix updates current the 4th matrix；Based on the third matrix, the 4th matrix, the 5th matrix and institute The 6th matrix is stated, current second matrix is updated；Based on second matrix and the 5th matrix, current institute is updated State the 6th matrix；Based on the 4th matrix and second matrix, the current third matrix is updated；Based on the described 4th Matrix updates current the 7th matrix；It is iteratively performed above step, until updated second matrix, described Updated third matrix, updated 4th matrix, updated 6th matrix and described updated Until seven matrixes are restrained.Wherein, in iterative processing for the first time, second matrix, the third matrix, the 4th matrix, 6th matrix and the 7th matrix are identified as initial value.In non-iterative processing for the first time, previous iteration is obtained Updated second matrix, the updated third matrix, updated 4th matrix, after the update The 6th matrix and updated 7th matrix be identified as second matrix, the third square when time iteration Battle array, the 4th matrix, the 6th matrix and the 7th matrix.In the embodiments of the present disclosure, the social context letter Breath includes the event information of the character relation information and the multiple event.Take this, in the base of the barment tag of personage's example Combine the relationship between personage's example event participated in and personage's example that personage is identified on plinth, so as to further carry The accuracy rate of high person recognition.
In a specific example, concentrating at least one 4th personage of known piece identity real based on described image The corresponding piece identity of example, each personage's example pair of at least two personages example centering match information and it is described at least The scene characteristic data of one image, the social context information and at least one third personage example to initialization When piece identity is iterated update, it is based on first matrix, second matrix and the third matrix, update is current The 4th matrix；Based on the third matrix, the 4th matrix, the 5th matrix and the 6th matrix, more New current second matrix；Based on second matrix and the 5th matrix, current the 6th matrix is updated；Base In the 4th matrix and second matrix, the current third matrix is updated；It is iteratively performed above step, Zhi Daosuo State updated second matrix, the updated third matrix, updated 4th matrix and described updated Until 6th matrix is restrained.Wherein, in iterative processing for the first time, second matrix, the third matrix, the 4th matrix And the 6th matrix is identified as initial value.In non-iterative processing for the first time, by the update of previous iteration acquisition The second matrix, the updated third matrix, updated 4th matrix and updated 6th square afterwards Battle array is identified as second matrix, the third matrix, the 4th matrix and the 6th matrix when time iteration. In the embodiments of the present disclosure, the social context information includes the event information of the multiple event.Take this, in personage's example Personage is identified in the event for combining personage's example to participate on the basis of barment tag, knows so as to further increase personage Other accuracy rate.
In a specific example, concentrating at least one 4th personage of known piece identity real based on described image The corresponding piece identity of example, each personage's example pair of at least two personages example centering match information and it is described at least The scene characteristic data of one image, the social context information and at least one third personage example to initialization When piece identity is iterated update, it is based on first matrix and the 7th matrix, updates current the 4th matrix； Based on the 4th matrix, current the 7th matrix is updated；It is iteratively performed above step, until described updated Until four matrixes and the updated 7th matrix convergence.Wherein, in iterative processing for the first time, the 4th matrix and described 7th matrix is identified as initial value.In non-iterative processing for the first time, by the described updated 4th of previous iteration acquisition the Matrix and updated 7th matrix are identified as the 4th matrix when time iteration and the 7th matrix.At this In open embodiment, the social context information includes the character relation information.Take this, in the barment tag of personage example On the basis of combine personage's example between relationship personage is identified, so as to further increase the accurate of person recognition Rate.
In step S204, based in the multiple first area characteristic, the multiple first area characteristic The weighted value of each first area characteristic and the social context information of described image collection, determine the first personage example Recognition result.
In the embodiments of the present disclosure, with the determination of the social context information of described image collection, the first personage example Piece identity be also determined obtaining.
In some optional embodiments, as described image concentrates the character relation between the multiple personage's examples occurred to believe The determination of the event information of breath multiple events corresponding with described image collection, the piece identity of the first personage example are also true Surely it obtains.In a specific example, as different personage's examples in the multiple personage's example appear in same image In probability data, in described multiple images at least one image of each event correlation in the multiple event, described more In a event in the scene characteristic data of each event and the multiple personage's example each personage's example participate in it is the multiple The determination of the probability data of each event, the piece identity of the first personage example are also determined obtaining in event.Namely It says, it is probability data in different personage's examples in determining the multiple personage's example appear in same image, the multiple In image with the scene of each event at least one image of each event correlation, the multiple event in the multiple event Each personage's example participates in the probability number of each event in the multiple event in characteristic and the multiple personage's example According to while, the piece identity of the first personage example is also determined obtaining.This embodiment can be updated last by iteration Output convergent second matrix, the convergent third matrix, convergent 4th matrix, the convergent described 6th Matrix and convergent 7th matrix are confirmed.
In some optional embodiments, described with the determination of the event information of the corresponding multiple events of described image collection The piece identity of first personage's example is also determined obtaining.In a specific example, in described multiple images with institute The scene characteristic data of each event at least one image of each event correlation, the multiple event in multiple events are stated, And each personage's example participates in the determination of the probability data of each event in the multiple event in the multiple personage's example, The piece identity of the first personage example is also determined obtaining.That is, in determining described multiple images with it is described more In a event at least one image of each event correlation, the multiple event each event scene characteristic data, and It is described while each personage's example participates in the probability data of each event in the multiple event in the multiple personage's example The piece identity of first personage's example is also determined obtaining.This embodiment can update the convergent institute finally exported by iteration The second matrix, the convergent third matrix, convergent 4th matrix and convergent 6th matrix is stated to be demonstrate,proved It is real.
In some optional embodiments, as described image concentrates the character relation between the multiple personage's examples occurred to believe The determination of breath, the piece identity of the first personage example are also determined obtaining.In a specific example, with described more Different personage's examples in a personage's example appear in the determination of the probability data in same image, the first personage example Piece identity is also determined obtaining.That is, different personage's examples in determining the multiple personage's example appear in together While probability data in one image, the piece identity of the first personage example is also determined obtaining.This embodiment Convergent 4th matrix finally exported and convergent 7th matrix, which can be updated, by iteration is confirmed.
The character recognition method provided according to embodiments of the present invention determines the social context information of the affiliated image set of image, Weighted value again based on each first area characteristic in multiple first area characteristics, multiple first area characteristics And the social context information of image set, the recognition result of first personage's example is determined, compared with other modes, multiple first Increase again on the basis of the weighted value of each first area characteristic in provincial characteristics data and multiple first area characteristics The social context information of image set is added to determine the piece identity of personage's example, to further increase the accuracy of person recognition.
The character recognition method of the present embodiment can be set by any suitable terminal with image or data-handling capacity Standby or server executes, wherein the terminal device includes but not limited to：Camera, terminal, mobile terminal, PC machine, server, Mobile unit, amusement equipment, advertising equipment, personal digital assistant (PDA), tablet computer, laptop, handheld device, Intelligent glasses, smartwatch, wearable device, virtual display device or display enhancing equipment (such as Google Glass, Oculus Rift, Hololens, Gear VR) etc., it is not limited in the embodiment of the present invention.
Fig. 6 is the flow chart according to a kind of character recognition method of other embodiments of the invention.
In step S301, the matching letter of at least two each personage's examples pair of personage's example centering in image set is obtained Breath.
In the embodiments of the present disclosure, described image collection includes multiple images, is had in each image in described multiple images Have at least one personage's example, and described multiple images include at least one example to be identified and known piece identity at least One reference example.
Since the embodiment of step S301 and at least two personage's example centerings of described image collection identified above are every The embodiment of the match information of a personage's example pair is similar, and details are not described herein.
In step s 302, at least one corresponding piece identity of reference example and at least two personage are based on The match information of each personage's example pair of example centering, determines the social context information of described image collection.
In the embodiments of the present disclosure, the social context information of described image collection includes that described image concentrates the multiple people occurred The event information of the corresponding multiple events of character relation information and/or described image collection between object example.In some optional realities It applies in example, the character relation information includes the multiple personage's example (including at least one example to be identified and described At least one reference example) in different personage's examples appear in the probability data in same image.The thing of the multiple event Part information includes at least one of following：With at least one of each event correlation in the multiple event in described multiple images Each personage's example ginseng in the scene characteristic data of each event, the multiple personage's example in a image, the multiple event With the probability data of each event in the multiple event.It is understood that above description is merely illustrative ground, the disclosure is real Example is applied not limit this.
In some optional embodiments, based on the corresponding piece identity of at least one reference example and it is described at least The match information of two each personage's examples pair of personage's example centering, before the social context information for determining described image collection, really Determine the scene characteristic data of at least one image in described multiple images；It is based on the corresponding people of at least one reference example again In object identity, the match information and described multiple images of each personage's example pair of at least two personages example centering at least The scene characteristic data of one image determine the social context information of described image collection.It is understood that the embodiment of the present disclosure Without being limited thereto, the embodiment of the social context information of any determining image set may be applicable to this, and the embodiment of the present disclosure is to this Any restriction is not done.
Due to it is determined here that in described multiple images the embodiment of the scene characteristic data of at least one image with above Determine that described image concentrates the embodiment of the scene characteristic data of at least one image similar, details are not described herein.
Due to being based on the corresponding piece identity of at least one reference example, at least two personages example pair herein In each personage's example pair match information and described multiple images at least one image scene characteristic data, determine institute The embodiment for stating the social context information of image set concentrates at least two personage's example centerings every with above-mentioned based on described image The scene characteristic data of the match information of a personage's example pair and at least one image, determine the social activity of described image collection The embodiment of contextual information is similar, only need to by described in the embodiment of the social context information of image set identified above extremely Few third personage's example replaces at least one example to be identified, and by the social context of image set identified above The 4th personage's example of at least one of embodiment of information replaces at least one reference example, can obtain herein Determine the embodiment of the social context information of image set.Therefore, details are not described herein.
Based on a big image set, an event may merely relate to the personage of the sub-fraction in the image of image set Example.If it is possible to infer the associated event of the image of capture, which can carry out person recognition pre- well It surveys.Fig. 7 is the schematic diagram of the personage for the embodiment of the method for implementing Fig. 6 and the relationship of event.As shown in fig. 7, if it is known that the figure Event as belonging to " people loudly require assistance after Titanic sinks ", then personage's example in identification image is respectively " Lay The probability of Ang Naduo " and " Kate " will increase.
It, can also be to image while personage's example in the image to image set is identified in actual application The image of concentration is classified.Specifically, each image classification corresponds to an event.This technology can be applied to mobile phone photo album In automatic classification.Specifically, intelligence is carried out to the image stored in user mobile phone according to the event information of multiple events of image set It can divide.
In actual application, character relation information is generally interpreted as waiting for specific group together.For For certain group, the presence of personage's example can indicate the presence of other personage's examples in this crowd of people.Fig. 8 is real Apply the schematic diagram of the personage of the embodiment of the method for Fig. 6 and the relationship of personage.As shown in figure 8, if being readily available same image In the piece identity of personage's example 1 and personage's example 2 therefore can be according to personage's example 1 in same image and personage's example 3 Relationship be inferred to the piece identity of personage's example 3 in same image, or can be according to personage's example 2 and personage in same image The relationship of example 3 is inferred to the piece identity of personage's example 3 in same image.
In step S303, the social context information based on described image collection determines at least one example to be identified In each example to be identified piece identity.
In the embodiments of the present disclosure, described at least one to wait knowing with the determination of the social context information of described image collection The piece identity of each example to be identified is also determined obtaining in other example.
In a particular embodiment, due to the embodiment class of the embodiment and above-mentioned steps S204 of step S303 Seemingly, details are not described herein.
Fig. 9 is the schematic diagram of the social context model for the embodiment of the method for implementing Fig. 6.As shown in Figure 9, it is assumed that an image In contain 4 personage's examples, it is believed that containing an event Y in this image, and each of image object example (X1,X2, X3, X4) all with the event generate it is a kind of contact, as shown in phantom in FIG..Meanwhile also having between each personage's example pair The character relation answered, i.e., it is shown in solid in figure.Specifically, ψυIt indicates visual consistency, that is to say and promote piece identity and vision Consistency between matching value, first multinomial of the above-mentioned object function in embody the thought of visual consistency.φep It indicates event consistency, that is to say being associated between concern image and event, and the image for belonging to the same event is promoted to exist Personage's example collection that is consistent and promoting to belong in the different images of the same event is consistent in scene, above-mentioned target letter Second multinomial of the number in embodies the thought of event consistency.φppIndicate different personage's examples in same image Common appearance, the relationship considered between personage's example and personage's example is that is to say, for example, which different personage's example becomes To in occurring jointly in same image, third multinomial of the above-mentioned object function in considers personage's example and personage Relationship between example.Wherein, the social context model can be according to the associated event of image and image set in image set Image in different personage's examples between relationship build for describe personage's example social action model.
In addition, the character recognition method calculation amount smaller that the embodiment of the present disclosure provides, is embodied in regiospecificity convolution god It is simpler through network and visual attention network, compared with regiospecificity convolutional neural networks and visual attention network, figure As the computational processing of the character relation information in associated event and image can be ignored.
In actual application scenarios, know using personage provided in this embodiment in personage's intelligent recognition in monitoring Other method searches for all images containing some personage's example such as in Large image database, and also or safety-security area is to disliking It doubts using character recognition method provided in this embodiment in the automatic tracing of people, is locked in the image that different monitoring is passed back same One personage's example, as long as this personage's example appears in picture, no matter the various situations such as positive face, side face, half body, can identify or This personage's example is deduced according to the associated event of image and character relation information, to be tracked to it.In addition, more Using the identification of the performer in character recognition method provided in this embodiment, such as film, TV play etc. in media materials.
The character recognition method provided according to embodiments of the present invention obtains at least two personage's example centerings in image set The match information of each personage's example pair, wherein the multiple images in image set include at least one example to be identified and known At least one reference example of piece identity；It is based on the corresponding piece identity of at least one reference example and at least two personages again The match information of each personage's example pair of example centering, determines the social context information of image set；And the social activity based on image set Contextual information determines that the piece identity of each example to be identified at least one example to be identified can compared with other modes The piece identity that example to be identified is determined based on the social context information of image set, to further increase the accurate of person recognition Degree.
The character recognition method of the present embodiment can be set by any suitable terminal with image or data-handling capacity Standby or server executes, wherein the terminal device includes but not limited to：Camera, terminal, mobile terminal, PC machine, server, Mobile unit, amusement equipment, advertising equipment, personal digital assistant (PDA), tablet computer, laptop, handheld device, Intelligent glasses, smartwatch, wearable device, virtual display device or display enhancing equipment (such as Google Glass, Oculus Rift, Hololens, Gear VR) etc., it is not limited in the embodiment of the present invention.
The description of each embodiment is focused on herein and emphasizes its difference, same or similar place can mutually join It examines, for example, the description of Fig. 1 or Fig. 5 corresponding embodiments is readily applicable to the description of Fig. 6 corresponding embodiments, for sake of simplicity, here It repeats no more.
Based on the same technical idea, Figure 10 is a kind of structure of person recognition device according to some embodiments of the invention Block diagram.It can be used to execute the character recognition method flow described in above example.
Referring to Fig.1 0, which includes that first processing module 401, the first determining module 402 and second determine Module 403.
First processing module 401, for carrying out feature extraction processing to image, the first personage obtained in described image is real Multiple first area characteristics of example；
First determining module 402 is used for multiple first area characteristics based on the first personage example, determines institute State the weighted value of each first area characteristic in multiple first area characteristics；
Second determining module 403, for being based on the multiple first area characteristic and the multiple first area The weighted value of each first area characteristic, determines the recognition result of the first personage example in characteristic.
The person recognition device provided through this embodiment carries out feature extraction processing to image, obtains the in image Multiple first area characteristics of one personage's example, multiple first area characteristics based on first personage's example determine The weighted value of each first area characteristic in multiple first area characteristics, and it is based on multiple first area characteristics And in multiple first area characteristics each first area characteristic weighted value, determine the identification of first personage's example As a result, compared with other modes, the provincial characteristics data based on personage's example determine the power of the provincial characteristics data of personage's example Weight values, and provincial characteristics data and its weighted value based on personage's example, determine the recognition result of personage's example, can improve people The accuracy of object identification.
Optionally, the first processing module 401, is specifically used for：The first personage example is obtained from described image Multiple regions image；Feature extraction processing is carried out to each area image in the multiple area image, is obtained described each The corresponding first area characteristic of area image.
Optionally, the multiple area image corresponds to the different body regions of the first personage example respectively.
Optionally, the multiple area image includes face image, head image, the upper part of the body of the first personage example Image and whole body images.
Optionally, first determining module 402, including：Splice submodule 4021, for the multiple first area Characteristic carries out concatenation, obtains the splicing feature of the first personage example；First determination sub-module 4022 is used for base In the splicing feature, the weighted value of each first area characteristic in the multiple first area characteristic is determined.
Optionally, first determination sub-module 4022, is specifically used for：Using visual attention network to splicing spy Sign is handled, and the weighted value of each first area characteristic in the multiple first area characteristic is obtained.
Optionally, first determination sub-module 4022, is specifically used for：Using visual attention network to splicing spy Sign carries out convolution operation, obtains the characteristic pattern of the first personage example；Map operation is carried out to the characteristic pattern, described in acquisition The weight feature vector of the multiple regions feature of first personage's example；Operation is normalized to the weight feature vector, is obtained Obtain the weighted value of each first area characteristic of the first personage example.
Optionally, the weighted value of the first area characteristic depends on the corresponding area of the first area characteristic Visibility and the first area characteristic corresponding region contribution to the recognition result of the domain in described image Ratio.
Optionally, second determining module 403, including：First processing submodule 4032, for based on the multiple the The weighted value of each first area characteristic, melts the multiple first area characteristic in one provincial characteristics data Conjunction is handled, and obtains the barment tag data of the first personage example；Second determination sub-module 4033, for being based on the appearance Characteristic determines the recognition result of the first personage example.
Optionally, before the first processing submodule 4032, second determining module 403 further includes：At second Submodule 4031 is managed, for carrying out dimension-reduction treatment to the multiple first area characteristic, obtains the first personage example Multiple second area characteristics；Correspondingly, the first processing submodule 4032, is specifically used for：Based on the multiple The weighted value of each first area characteristic, melts the multiple second area characteristic in one provincial characteristics data Conjunction is handled, and obtains the barment tag data of the first personage example.
Optionally, second determination sub-module 4033, is specifically used for：Determine the barment tag of the first personage example Data and the similarity that barment tag data are each preset at least one default barment tag data；According to first personage The barment tag data of example to the similar of barment tag data is each preset at least one default barment tag data Degree, determines the recognition result of the first personage example.
Based on the same technical idea, Figure 11 is the knot according to a kind of person recognition device of other embodiments of the invention Structure block diagram.It can be used to execute the character recognition method flow described in above example.
Referring to Fig.1 1, which includes that first processing module 502, the first determining module 503 and second determine Module 504.Wherein, first processing module 502 obtain first in described image for carrying out feature extraction processing to image Multiple first area characteristics of personage's example；First determining module 503, for based on the multiple of the first personage example First area characteristic determines the weighted value of each first area characteristic in the multiple first area characteristic； Second determining module 504, for being based on the multiple first area characteristic and the multiple first area characteristic In each first area characteristic weighted value, determine the recognition result of the first personage example.
Optionally, second determining module 504, including：Third determination sub-module 5041, it is described the first for determining At least one the of each first area characteristic and known piece identity in multiple first area characteristics of object example Similarity in two personage's examples between the corresponding region characteristic of each second personage example；4th determination sub-module 5042, for according to the weighted value of each of the first personage example first area characteristic and corresponding described Similarity determines the matching value of the first personage example and the second personage example；5th determination sub-module 5043, is used for According to the matching value, the recognition result of the first personage example is determined.
Optionally, the 4th determination sub-module 5042, is specifically used for：According to described in each of described first personage example The weighted value of the corresponding region characteristic of the weighted value of first area characteristic, the second personage example and corresponding The similarity determines the matching value of the first personage example and the second personage example.
Optionally, before the first processing module 502, described device further includes：Second processing module 501, is used for Size adjusting processing, the described image after being adjusted are carried out to described image；The first processing module 502, is specifically used for： Feature extraction processing is carried out to the described image after adjustment using convolutional neural networks, the in the described image after being adjusted Multiple first area characteristics of one personage's example.
Based on the same technical idea, Figure 12 is the knot according to a kind of person recognition device of other embodiments of the invention Structure block diagram.It can be used to execute the character recognition method flow described in above example.
Referring to Fig.1 2, which includes that first processing module 601, the first determining module 602 and second determine Module 604.Wherein, first processing module 601 obtain first in described image for carrying out feature extraction processing to image Multiple first area characteristics of personage's example；First determining module 602, for based on the multiple of the first personage example First area characteristic determines the weighted value of each first area characteristic in the multiple first area characteristic； Second determining module 604, for being based on the multiple first area characteristic and the multiple first area characteristic In each first area characteristic weighted value, determine the recognition result of the first personage example.
Optionally, described device further includes：Third determining module 603, the society for determining the affiliated image set of described image Contextual information is handed over, described image collection includes multiple images, has at least one personage's example in each image, described image collection Social context information includes the character relation information and/or described image between multiple personage's examples that described image concentration occurs Collect the event information of corresponding multiple events；Second determining module 604, including：6th determination sub-module 6041 is used for base The power of each first area characteristic in the multiple first area characteristic, the multiple first area characteristic The social context information of weight values and described image collection determines the recognition result of the first personage example.
Optionally, the third determining module 603, including：7th determination sub-module 6031, for determining described image collection In the scene characteristic data of at least one image and at least two each personage's examples of personage's example centering of described image collection To match information；8th determination sub-module 6032, for concentrating at least two personage's example centerings each based on described image The scene characteristic data of the match information of personage's example pair and at least one image determine the social feelings of described image collection Border information.
Optionally, the 7th determination sub-module 6031, is specifically used for：Using convolutional neural networks to described at least one Each image carries out feature extraction processing in image, obtains the scene characteristic data of each image at least one image.
Optionally, the 8th determination sub-module 6032, including：Initialization unit 6033, for initializing described image The piece identity at least one third personage example to be identified that collection includes, and initialize the social context of described image collection Information；Iteration updating unit 6034, at least one 4th personage's example for concentrating known piece identity based on described image Corresponding piece identity, each personage's example pair of at least two personages example centering match information and described at least one The scene characteristic data of a image, the people of the social context information and at least one third personage example to initialization Object identity is iterated update, until meeting stopping criterion for iteration, wherein at least two personages example to comprising by Personage's example pair that the third personage example and the 4th personage's example are constituted.
Optionally, the initialization unit 6033, is specifically used for：It is corresponding based at least one 4th personage's example Each third personage example and at least one 4th personage are real in piece identity and at least one third personage example The match information at least one personage's example pair that example is constituted, determines the initial identity of each third personage example.
Optionally, the initialization unit 6033, is specifically used for：The multiple images for including by described image collection at random divide At at least one event packets, so that each image in described multiple images is only with an event correlation, and it is the multiple It is in preset range with the quantity of the image of same event correlation in image.
Optionally, the iteration updating unit 6034, including：First update subelement 6035, is used for based on described at least The match information of two each personage's examples pair of personage's example centering and the current social context information, described in update extremely The current piece identity of few third personage's example；Second update subelement 6036, for based on described at least one the The corresponding piece identity of four personage's examples, the corresponding updated piece identity of at least one third personage example and institute The scene characteristic data for stating at least one image update the current social context information.
Optionally, the first update subelement 6035, is specifically used for：It is every based on at least two personages example centering The match information of a personage's example pair, the event information of current the multiple event and current character relation letter Breath updates the current piece identity of at least one third personage example.
Optionally, the first update subelement 6035, is specifically used for：It is every based on at least two personages example centering In the match information of a personage's example pair, described multiple images currently with each event correlation in the multiple event at least one Each personage's example participates in the multiple thing in a image, the current character relation information and the multiple personage's example The current probability data of each event in part update the current piece identity of at least one third personage example.
Optionally, the second update subelement 6036, is specifically used for：Based at least one 4th personage's example pair The updated piece identity of the piece identity, at least one third personage example that answer and at least one image Scene characteristic data update the event information of current the multiple event；Based at least one 4th personage's example pair The updated piece identity of the piece identity and at least one third personage example that answer update the current personage and close It is information.
Optionally, the second update subelement 6036, is specifically used for：Based at least one 4th personage's example pair The updated piece identity of the piece identity, at least one third personage example that answer, the field of at least one image It is every in the current scene characteristic data of each event and the multiple personage's example in scape characteristic, the multiple event A personage's example participates in the current probability data of each event in the multiple event, update in described multiple images currently with At least one image of each event correlation in the multiple event；Scene characteristic data based at least one image and With at least one image of each event correlation in the multiple event in updated described multiple images, update the multiple The current scene characteristic data of each event in event；Based in updated described multiple images in the multiple event At least one image of each event correlation, the corresponding piece identity of at least one 4th personage's example and it is described at least The updated piece identity of one third personage's example updates in the multiple personage's example described in each personage's example participation The current probability data of each event in multiple events.
Optionally, the second update subelement 6036, is specifically used for：Based on each in updated the multiple event The scene characteristic data of each image at least one image of event correlation update each event in the multiple event Current scene characteristic data.
Optionally, the second update subelement 6036, is specifically used for：By to every in updated the multiple event The scene characteristic data of each image at least one image of a event correlation are averaging processing, and obtain the multiple thing The updated scene characteristic data of each event in part.
Optionally, the first update subelement 6035, is specifically used for：It is every based on at least two personages example centering The match information of a personage's example pair and the current social context information are updated by making object function maximize The current piece identity of at least one third personage example.
Optionally, the stopping criterion for iteration, including：The updated social context information and the society before update Hand over context information identical, and the updated piece identity of at least one third personage example and described at least one the Piece identity before the update of three personage's examples is identical.
Optionally, the event information of the multiple event includes at least one of following：In described multiple images with institute State in multiple events the scene characteristic data of each event at least one image of each event correlation, the multiple event, Each personage's example participates in the probability data of each event in the multiple event in the multiple personage's example.
Optionally, the character relation information include different personage's examples in the multiple personage's example appear in it is same Probability data in image.
Based on the same technical idea, Figure 13 is the knot according to a kind of person recognition device of other embodiments of the invention Structure block diagram.It can be used to execute the character recognition method flow described in above example.
Referring to Fig.1 3, which includes that the first acquisition module 701, the 4th determining module 703 and the 5th determine Module 704.
First acquisition module 701, for obtaining at least two each personage's examples pair of personage's example centering in image set Match information, wherein described image collection includes multiple images, has at least one personage's example, and institute in each image It includes at least one example to be identified and at least one reference example of known piece identity to state multiple images；
4th determining module 703, for based on the corresponding piece identity of at least one reference example and it is described at least The match information of two each personage's examples pair of personage's example centering, determines the social context information of described image collection, the figure The social context information of image set includes the character relation information and/or institute between multiple personage's examples that described image concentration occurs State the event information of the corresponding multiple events of image set；
5th determining module 704 is used for the social context information based on described image collection, determines and described at least one waits knowing The piece identity of each example to be identified in other example.
The person recognition device provided through this embodiment, at least two personage's example centerings obtained in image set are each The match information of personage's example pair, wherein the multiple images in image set include at least one example to be identified and known personage At least one reference example of identity；It is based on the corresponding piece identity of at least one reference example and at least two personage's examples again The match information of each personage's example pair of centering, determines the social context information of image set；And the social context based on image set Information determines that the piece identity of each example to be identified at least one example to be identified can be based on compared with other modes The social context information of image set determines the piece identity of example to be identified, to further increase the accuracy of person recognition.
Optionally, first acquisition module 701, is specifically used for：Pass through determination first personage of personage's example centering The similarity of the barment tag data of example and the barment tag data of personage's example centering the second personage example obtains institute State the match information of personage's example pair.
Optionally, first acquisition module 701, is specifically used for：Determine personage's example centering the first personage example Multiple first area characteristics in each provincial characteristics data it is corresponding with personage's example centering the second personage example Similarity between provincial characteristics data；According to the weighted value of each provincial characteristics data of the first personage example, described The weighted value of each provincial characteristics data of second personage's example and the similarity, determine the matching of personage's example pair Information.
Optionally, before the 4th determining module 703, described device further includes：6th determining module 702, is used for Determine the scene characteristic data of at least one image in described multiple images；4th determining module 703, including：9th really Stator modules 7031, for being based on the corresponding piece identity of at least one reference example, at least two personages example The scene characteristic data of at least one image in the match information and described multiple images of each personage's example pair of centering determine The social context information of described image collection.
Optionally, the 6th determining module 702, is specifically used for：Using convolutional neural networks at least one figure Each image carries out feature extraction processing as in, obtains the scene characteristic data of each image.
Based on the same technical idea, Figure 14 is the knot according to a kind of person recognition device of other embodiments of the invention Structure block diagram.It can be used to execute the character recognition method flow described in above example.
Referring to Fig.1 4, which includes that the first acquisition module 801, the 4th determining module 802 and the 5th determine Module 803.Wherein, the first acquisition module 801, it is real for obtaining each personage of at least two personage's example centerings in image set The match information of example pair, wherein described image collection includes multiple images, has at least one personage's example in each image, and And described multiple images include at least one example to be identified and at least one reference example of known piece identity；4th determines Module 802, for being based at least one corresponding piece identity of reference example and at least two personages example centering The match information of each personage's example pair determines the social context information of described image collection, the social context letter of described image collection Breath include described image concentrate appearance multiple personage's examples between character relation information and/or described image collection it is corresponding more The event information of a event；5th determining module 803, be used for the social context information based on described image collection, determine described in extremely The piece identity of each example to be identified in a few example to be identified.
Optionally, the 9th determination sub-module 8021, including：Initialization unit 8023, for initializing described image The piece identity at least one example to be identified that collection includes, and initialize the social context information of described image collection；Iteration Updating unit 8024, for being based on the corresponding piece identity of at least one reference example, at least two personages example The scene characteristic data of at least one image in the match information and described multiple images of each personage's example pair of centering, to first The piece identity of the social context information of beginningization and at least one example to be identified is iterated update, until meeting Until stopping criterion for iteration.
Optionally, the initialization unit 8023, is specifically used for：Based on the corresponding personage of at least one reference example Each example to be identified is constituted at least at least one reference example in identity and at least one example to be identified The match information of one personage's example pair determines the initial identity of each example to be identified.
Optionally, the initialization unit 8023, is specifically used for：The multiple images for including by described image collection at random divide At at least one event packets, so that each image in described multiple images is only with an event correlation, and it is the multiple It is in preset range with the quantity of the image of same event correlation in image.
Optionally, the iteration updating unit 8024, including：First update subelement 8025, is used for based on described at least The match information of two each personage's examples pair of personage's example centering and the current social context information, described in update extremely The current piece identity of each example to be identified in a few example to be identified；Second update subelement 8026, for being based on The corresponding piece identity of at least one reference example, at least one example to be identified updated piece identity with And in described multiple images at least one image scene characteristic data, update the current social context information.
Optionally, the first update subelement 8025, is specifically used for：It is every based on at least two personages example centering The match information of a personage's example pair, the event information of current the multiple event and current character relation letter Breath updates the current piece identity of each example to be identified at least one example to be identified.
Optionally, the first update subelement 8025, is specifically used for：It is every based on at least two personages example centering In the match information of a personage's example pair, described multiple images currently with each event correlation in the multiple event at least one Each personage's example participates in the multiple thing in a image, the current character relation information and the multiple personage's example The current probability data of each event in part update the current of each example to be identified at least one example to be identified Piece identity.
Optionally, the second update subelement 8026, is specifically used for：It is corresponding based at least one reference example At least one figure in piece identity, the updated piece identity of at least one example to be identified and described multiple images The scene characteristic data of picture update the event information of current the multiple event；Based at least one reference example pair The updated piece identity of the piece identity and at least one example to be identified that answer, update the current character relation Information.
Optionally, the second update subelement 8026, is specifically used for：It is corresponding based at least one reference example At least one image in piece identity, the updated piece identity of at least one example to be identified, described multiple images Scene characteristic data, the current scene characteristic data of each event and the multiple personage's example in the multiple event In each personage's example participate in the current probability data of each event in the multiple event, in update described multiple images when Preceding at least one image with each event correlation in the multiple event；Based at least one image in described multiple images In scene characteristic data and updated described multiple images at least one figure of each event correlation in the multiple event Picture updates the current scene characteristic data of each event in the multiple event；Based in updated described multiple images Piece identity corresponding at least one image of each event correlation in the multiple event, at least one reference example And the updated piece identity of at least one example to be identified, it is real to update each personage in the multiple personage's example Example participates in the current probability data of each event in the multiple event.
Optionally, the second update subelement 8026, is specifically used for：Based on each in updated the multiple event The scene characteristic data of each image at least one image of event correlation update each event in the multiple event Current scene characteristic data.
Optionally, the second update subelement 8026, is specifically used for：By to every in updated the multiple event The scene characteristic data of each image at least one image of a event correlation are averaging processing, and obtain the multiple thing The updated scene characteristic data of each event in part.
Optionally, the first update subelement 8025, is specifically used for：It is every based on at least two personages example centering The match information of a personage's example pair and the current social context information are updated by making object function maximize The current piece identity of each example to be identified at least one example to be identified.
Optionally, the stopping criterion for iteration, including：The updated social context information and the society before update Hand over context information identical, and the updated piece identity of at least one example to be identified at least one waits knowing with described Piece identity before the update of other example is identical.
Optionally, the event information of the multiple event includes at least one of following：In described multiple images with institute State in multiple events the scene characteristic data of each event at least one image of each event correlation, the multiple event, Each personage's example participates in the probability data of each event in the multiple event in the multiple personage's example.
Optionally, the character relation information include different personage's examples in the multiple personage's example appear in it is same Probability data in image.
The embodiment of the present invention additionally provides a kind of electronic equipment, including：First processor and first memory, described first Memory makes the first processor execute such as of the invention implement for storing an at least executable instruction, the executable instruction Character recognition method described in example first aspect.For example, electronic equipment can be mobile terminal, personal computer (PC), tablet Computer, server etc..Below with reference to Figure 15, it illustrates suitable for for realizing the terminal device or server of the embodiment of the present invention Electronic equipment 900 structural schematic diagram.As shown in figure 15, electronic equipment 900 includes one or more first processors, first Communication device etc., one or more of first processors are for example：One or more central processing unit (CPU) 901, and/or One or more image processors (GPU) 913 etc., first processor can be according to being stored in read-only memory (ROM) 902 Executable instruction executes respectively from the executable instruction that storage section 908 is loaded into random access storage device (RAM) 903 Kind action appropriate and processing.In the present embodiment, the first read-only memory 902 and random access storage device 903 are referred to as first Memory.First communication device includes communication component 912 and/or communication interface 909.Wherein, communication component 912 may include but not It is limited to network interface card, the network interface card may include but be not limited to IB (Infiniband) network interface card, and communication interface 909 includes such as LAN card, adjusts The communication interface of the network interface card of modulator-demodulator etc., communication interface 909 execute communication process via the network of such as internet.
First processor can be communicated with read-only memory 902 and/or random access storage device 903 to execute executable finger It enables, is connected with communication component 912 by the first communication bus 904 and is communicated with other target devices through communication component 912, from And the corresponding operation of character recognition method any one of provided in an embodiment of the present invention is completed, for example, carrying out feature extraction to image Processing obtains multiple first area characteristics of first personage's example in described image；Based on the first personage example Multiple first area characteristics, determine the power of each first area characteristic in the multiple first area characteristic Weight values；It is special based on each first area in the multiple first area characteristic and the multiple first area characteristic The weighted value for levying data, determines the recognition result of the first personage example.
In addition, in RAM 903, it can also be stored with various programs and data needed for device operation.CPU901 or GPU913, ROM902 and RAM903 are connected with each other by the first communication bus 904.In the case where there is RAM903, ROM902 For optional module.RAM903 stores executable instruction, or executable instruction is written into ROM902 at runtime, executable instruction First processor is set to execute the corresponding operation of above-mentioned communication means.Input/output (I/O) interface 905 is also connected to the first communication Bus 904.Communication component 912 can be integrally disposed, may be set to be with multiple submodule (such as multiple IB network interface cards), and It is chained in communication bus.
It is connected to I/O interfaces 905 with lower component：Importation 906 including keyboard, mouse etc.；It is penetrated including such as cathode The output par, c 907 of spool (CRT), liquid crystal display (LCD) etc. and loud speaker etc.；Storage section 908 including hard disk etc.； And the communication interface 909 of the network interface card including LAN card, modem etc..Driver 910 is also according to needing to connect It is connected to I/O interfaces 905.Detachable media 911, such as disk, CD, magneto-optic disk, semiconductor memory etc. are pacified as needed On driver 910, in order to be mounted into storage section 908 as needed from the computer program read thereon.
It should be noted that framework as shown in figure 15 is only a kind of optional realization method, it, can root during concrete practice The component count amount and type of above-mentioned Figure 15 are selected, are deleted, increased or replaced according to actual needs；It is set in different function component It sets, separately positioned or integrally disposed and other implementations, such as separable settings of GPU and CPU or can be by GPU collection can also be used At on CPU, the separable setting of communication device, can also be integrally disposed on CPU or GPU, etc..These interchangeable embodiment party Formula each falls within protection scope of the present invention.
Particularly, according to embodiments of the present invention, it is soft to may be implemented as computer for the process above with reference to flow chart description Part program.For example, the embodiment of the present invention includes a kind of computer program products comprising be tangibly embodied in machine readable media On computer program, computer program includes the program code for method shown in execution flow chart, and program code can wrap The corresponding instruction of corresponding execution method and step provided in an embodiment of the present invention is included to obtain for example, carrying out feature extraction processing to image Obtain multiple first area characteristics of first personage's example in described image；Multiple based on the first personage example One provincial characteristics data determine the weighted value of each first area characteristic in the multiple first area characteristic；Base Each first area characteristic in the multiple first area characteristic and the multiple first area characteristic Weighted value, determine the recognition result of the first personage example.In such embodiments, which can pass through Communication device is downloaded and installed from network, and/or is mounted from detachable media 911.In the computer program by first When processor executes, the above-mentioned function of being limited in the method for the embodiment of the present invention is executed.
The embodiment of the present invention additionally provides a kind of electronic equipment, including：Second processor and second memory, described second Memory makes the second processor execute such as of the invention implement for storing an at least executable instruction, the executable instruction Character recognition method described in example second aspect.For example, electronic equipment can be mobile terminal, personal computer (PC), tablet Computer, server etc..Below with reference to Figure 16, it illustrates suitable for for realizing the terminal device or server of the embodiment of the present invention Electronic equipment 1000 structural schematic diagram.As shown in figure 16, electronic equipment 1000 includes one or more second processors, the Two communication devices etc., one or more of second processors are for example：One or more central processing unit (CPU) 1001, and/ Or one or more image processor (GPU) 1013 etc., second processor can be according to being stored in read-only memory (ROM) 1002 In executable instruction or be loaded into the executable instruction in random access storage device (RAM) 1003 from storage section 1008 and Execute various actions appropriate and processing.In the present embodiment, the second read-only memory 1002 and random access storage device 1003 are united Referred to as second memory.Second communication device includes communication component 1012 and/or communication interface 1009.Wherein, communication component 1012 may include but be not limited to network interface card, and the network interface card may include but be not limited to IB (Infiniband) network interface card, communication interface 1009 The communication interface of network interface card including LAN card, modem etc., communication interface 1009 is via such as internet Network executes communication process.
Second processor can communicate executable to execute with read-only memory 1002 and/or random access storage device 1003 Instruction is connected by the second communication bus 1004 with communication component 1012 and logical through communication component 1012 and other target devices Letter, the corresponding operation of any one character recognition method that embodiment provides thereby completing the present invention, for example, obtaining in image set The match information of at least two each personage's examples pair of personage's example centering, wherein described image collection includes multiple images, each There is at least one personage's example, and described multiple images include at least one example to be identified and known personage's body in image At least one reference example of part；Based at least one corresponding piece identity of reference example and at least two personage The match information of each personage's example pair of example centering, determines the social context information of described image collection, the society of described image collection Friendship border information includes the character relation information and/or described image collection between multiple personage's examples that described image concentration occurs The event information of corresponding multiple events；Social context information based on described image collection determines described at least one to be identified The piece identity of each example to be identified in example.
In addition, in RAM 1003, it can also be stored with various programs and data needed for device operation.CPU1001 or GPU1013, ROM1002 and RAM1003 are connected with each other by the second communication bus 1004.In the case where there is RAM1003, ROM1002 is optional module.RAM1003 stores executable instruction, or executable instruction is written into ROM1002 at runtime, Executable instruction makes second processor execute the corresponding operation of above-mentioned communication means.Input/output (I/O) interface 1005 also connects To the second communication bus 1004.Communication component 1012 can be integrally disposed, may be set to be (such as more with multiple submodule A IB network interface cards), and chained in communication bus.
It is connected to I/O interfaces 1005 with lower component：Importation 1006 including keyboard, mouse etc.；Including such as cathode The output par, c 1007 of ray tube (CRT), liquid crystal display (LCD) etc. and loud speaker etc.；Storage section including hard disk etc. 1008；And the communication interface 1009 of the network interface card including LAN card, modem etc..The also root of driver 1010 According to needing to be connected to I/O interfaces 1005.Detachable media 1011, such as disk, CD, magneto-optic disk, semiconductor memory etc., It is mounted on driver 1010 as needed, in order to be mounted into storage part as needed from the computer program read thereon Divide 1008.
It should be noted that framework as shown in figure 16 is only a kind of optional realization method, it, can root during concrete practice The component count amount and type of above-mentioned Figure 16 are selected, are deleted, increased or replaced according to actual needs；It is set in different function component It sets, separately positioned or integrally disposed and other implementations, such as separable settings of GPU and CPU or can be by GPU collection can also be used At on CPU, the separable setting of communication device, can also be integrally disposed on CPU or GPU, etc..These interchangeable embodiment party Formula each falls within protection scope of the present invention.
Particularly, according to embodiments of the present invention, it is soft to may be implemented as computer for the process above with reference to flow chart description Part program.For example, the embodiment of the present invention includes a kind of computer program products comprising be tangibly embodied in machine readable media On computer program, computer program includes the program code for method shown in execution flow chart, and program code can wrap The corresponding instruction of corresponding execution method and step provided in an embodiment of the present invention is included, for example, obtaining at least two people in image set The match information of each personage's example pair of object example centering, wherein described image collection includes multiple images, is had in each image At least one personage's example, and described multiple images include at least the one of at least one example to be identified and known piece identity A reference example；It is every based at least one corresponding piece identity of reference example and at least two personages example centering The match information of a personage's example pair determines the social context information of described image collection, the social context information of described image collection Character relation information and/or described image collection between the multiple personage's examples occurred including described image concentration is corresponding multiple The event information of event；Social context information based on described image collection determines each at least one example to be identified The piece identity of example to be identified.In such embodiments, the computer program can by communication device from network quilt It downloads and installs, and/or be mounted from detachable media 1011.When the computer program is executed by second processor, execute The above-mentioned function of being limited in the method for the embodiment of the present invention.
It may be noted that according to the needs of implementation, all parts/step described in this application can be split as more multi-section The part operation of two or more components/steps or components/steps can be also combined into new components/steps by part/step, To realize the purpose of the embodiment of the present invention.
Methods and apparatus of the present invention, equipment may be achieved in many ways.For example, software, hardware, firmware can be passed through Or any combinations of software, hardware, firmware realize method and apparatus, the equipment of the embodiment of the present invention.Step for method Merely to illustrate, the step of method of the embodiment of the present invention, is not limited to described in detail above suitable for rapid said sequence Sequence, unless specifically stated otherwise.In addition, in some embodiments, also the present invention can be embodied as to be recorded in record Jie Program in matter, these programs include for realizing machine readable instructions according to the method for the embodiment of the present invention.Thus, this hair Recording medium of the bright also covering storage for executing program according to the method for the embodiment of the present invention.
The description of the embodiment of the present invention provides for the sake of example and description, and is not exhaustively or to incite somebody to action The present invention is limited to disclosed form, and many modifications and variations are obvious for the ordinary skill in the art.Choosing It is and to make those skilled in the art to more preferably illustrate the principle of the present invention and practical application to select and describe embodiment It will be appreciated that various embodiments with various modifications of the present invention to design suitable for special-purpose.

Claims (10)

1. a kind of character recognition method, which is characterized in that the method includes：
Feature extraction processing is carried out to image, obtains multiple first area characteristics of first personage's example in described image According to；
Multiple first area characteristics based on the first personage example, determine in the multiple first area characteristic The weighted value of each first area characteristic；
It is special based on each first area in the multiple first area characteristic and the multiple first area characteristic The weighted value for levying data, determines the recognition result of the first personage example.
2. a kind of character recognition method, which is characterized in that the method includes：
Obtain the match information of at least two each personage's examples pair of personage's example centering in image set, wherein described image Collection includes multiple images, has at least one personage's example in each image, and described multiple images include at least one wait for Identify at least one reference example of example and known piece identity；
Based on the corresponding piece identity of at least one reference example and each personage of at least two personages example centering The match information of example pair determines the social context information of described image collection, and the social context information of described image collection includes institute State character relation information between the multiple personage's examples occurred in image set and/or the corresponding multiple events of described image collection Event information；
Social context information based on described image collection determines each example to be identified at least one example to be identified Piece identity.
3. a kind of person recognition device, which is characterized in that described device includes：
First processing module obtains the more of first personage's example in described image for carrying out feature extraction processing to image A first area characteristic；
First determining module is used for multiple first area characteristics based on the first personage example, determines the multiple The weighted value of each first area characteristic in the characteristic of first area；
Second determining module, for being based on the multiple first area characteristic and the multiple first area characteristic In each first area characteristic weighted value, determine the recognition result of the first personage example.
4. a kind of person recognition device, which is characterized in that described device includes：
First acquisition module, the matching for obtaining at least two each personage's examples pair of personage's example centering in image set are believed Breath, wherein described image collection includes multiple images, has at least one personage's example, and the multiple figure in each image At least one reference example as including at least one example to be identified and known piece identity；
4th determining module, for being based at least one corresponding piece identity of reference example and at least two personage The match information of each personage's example pair of example centering, determines the social context information of described image collection, the society of described image collection Friendship border information includes the character relation information and/or described image collection between multiple personage's examples that described image concentration occurs The event information of corresponding multiple events；
5th determining module is used for the social context information based on described image collection, determines at least one example to be identified In each example to be identified piece identity.
5. a kind of computer readable storage medium, is stored thereon with computer program instructions, wherein described program instruction is handled The step of device realizes character recognition method described in claim 1 when executing.
6. a kind of computer readable storage medium, is stored thereon with computer program instructions, wherein described program instruction is handled The step of character recognition method described in claim 2 is realized when device executes.
7. a kind of computer program product comprising there is computer program instructions, wherein described program instruction is executed by processor The step of Shi Shixian character recognition methods described in claim 1.
8. a kind of computer program product comprising there is computer program instructions, wherein described program instruction is executed by processor The step of character recognition method described in Shi Shixian claims 2.
9. a kind of electronic equipment, including：First processor and first memory, the first memory can for storing at least one It executes instruction, the executable instruction makes the first processor execute character recognition method as described in claim 1.
10. a kind of electronic equipment, including：Second processor and second memory, the second memory is for storing at least one Executable instruction, the executable instruction make the second processor execute character recognition method as claimed in claim 2.
CN201810350902.6A 2018-04-18 2018-04-18 Character recognition method, device, storage medium, program product and electronic equipment Pending CN108596070A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810350902.6A CN108596070A (en) 2018-04-18 2018-04-18 Character recognition method, device, storage medium, program product and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810350902.6A CN108596070A (en) 2018-04-18 2018-04-18 Character recognition method, device, storage medium, program product and electronic equipment

Publications (1)

Publication Number Publication Date
CN108596070A true CN108596070A (en) 2018-09-28

Family

ID=63611261

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810350902.6A Pending CN108596070A (en) 2018-04-18 2018-04-18 Character recognition method, device, storage medium, program product and electronic equipment

Country Status (1)

Country Link
CN (1) CN108596070A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109784261A (en) * 2019-01-09 2019-05-21 深圳市烨嘉为技术有限公司 Pedestrian's segmentation and recognition methods based on machine vision
CN110210572A (en) * 2019-06-10 2019-09-06 腾讯科技（深圳）有限公司 Image classification method, device, storage medium and equipment
CN110942036A (en) * 2019-11-29 2020-03-31 深圳市商汤科技有限公司 Person identification method and device, electronic equipment and storage medium
CN111476070A (en) * 2019-01-24 2020-07-31 深圳市商汤科技有限公司 Image processing method, image processing device, electronic equipment and computer readable storage medium
WO2021008023A1 (en) * 2019-07-18 2021-01-21 北京市商汤科技开发有限公司 Image processing method and apparatus, and electronic device and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104850828A (en) * 2015-04-29 2015-08-19 小米科技有限责任公司 Person identification method and person identification device

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104850828A (en) * 2015-04-29 2015-08-19 小米科技有限责任公司 Person identification method and person identification device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DAHUA LIN, ASHISH KAPOOR, GANG HUA, SIMON BAKER: "Joint People, Event, and Location Recognition in Personal Photo Collections using Cross-Domain Context", 《IN PROCEEDINGS OF THE EUROPE CONFERENCE ON COMPUTER VISION》 *
HAOXIANG LI, JONATHAN BRANDT, ZHE LIN, XIAOHUI SHEN, GANG HUA: "A Multi-Level Contextual Model For Person Recognition in Photo Albums", 《IN PROCEEDINGS OF THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 *
XIAO LIU, TIAN XIA, JIANG WANG, YI YANG, FENG ZHOU AND YUANQING: "Fully Convolutional Attention Networks for Fine-Grained Recognition", 《IN PROCEEDINGS OF THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109784261A (en) * 2019-01-09 2019-05-21 深圳市烨嘉为技术有限公司 Pedestrian's segmentation and recognition methods based on machine vision
CN109784261B (en) * 2019-01-09 2020-08-21 深圳市烨嘉为技术有限公司 Pedestrian segmentation and identification method based on machine vision
CN111476070A (en) * 2019-01-24 2020-07-31 深圳市商汤科技有限公司 Image processing method, image processing device, electronic equipment and computer readable storage medium
CN110210572A (en) * 2019-06-10 2019-09-06 腾讯科技（深圳）有限公司 Image classification method, device, storage medium and equipment
CN110210572B (en) * 2019-06-10 2023-02-07 腾讯科技（深圳）有限公司 Image classification method, device, storage medium and equipment
WO2021008023A1 (en) * 2019-07-18 2021-01-21 北京市商汤科技开发有限公司 Image processing method and apparatus, and electronic device and storage medium
US11481574B2 (en) 2019-07-18 2022-10-25 Beijing Sensetime Technology Development Co., Ltd. Image processing method and device, and storage medium
CN110942036A (en) * 2019-11-29 2020-03-31 深圳市商汤科技有限公司 Person identification method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN108596070A (en) Character recognition method, device, storage medium, program product and electronic equipment
Zhang et al. Semantic image inpainting with progressive generative networks
CN108205655B (en) Key point prediction method and device, electronic equipment and storage medium
CN111783647B (en) Training method of face fusion model, face fusion method, device and equipment
Yang et al. Single image haze removal via region detection network
US20210383199A1 (en) Object-Centric Learning with Slot Attention
CN109003297A (en) A kind of monocular depth estimation method, device, terminal and storage medium
CN108021923A (en) A kind of image characteristic extracting method for deep neural network
CN115917608A (en) Machine learning in augmented reality content items
US20230094206A1 (en) Image processing method and apparatus, device, and storage medium
CN113379627B (en) Training method of image enhancement model and method for enhancing image
CN112163717A (en) Population data prediction method and device, computer equipment and medium
CN112990053B (en) Image processing method, device, equipment and storage medium
CN115187706B (en) Lightweight method and system for face style migration, storage medium and electronic equipment
Ferrari et al. Computer Vision–ECCV 2018: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part V
US10970331B2 (en) Determining contextual confidence of images using associative deep learning
KR102091643B1 (en) Apparatus for processing image using artificial neural network, method thereof and computer recordable medium storing program to perform the method
CN109191388A (en) A kind of dark image processing method and system
CN117083641A (en) Real-time experience real-size eye wear device
CN115967823A (en) Video cover generation method and device, electronic equipment and readable medium
CN107864333A (en) Image processing method, device, terminal and storage medium
Peng et al. Mpib: An mpi-based bokeh rendering framework for realistic partial occlusion effects
CN115100334B (en) Image edge tracing and image animation method, device and storage medium
Lee et al. Bokeh-loss gan: multi-stage adversarial training for realistic edge-aware bokeh
CN113223128B (en) Method and apparatus for generating image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination