Embodiment
Use description to realize embodiments of the present invention below.
Fig. 1 shows the conceptual illustration according to the functional structure of the facial image tracking apparatus 1 of the embodiment of the invention.
As shown in Figure 1, facial image tracking apparatus 1 identification and follow the tracks of that in predetermined period, catch continuously and be stored in the face of the same people in a plurality of original images in the image storage apparatus 3 by camera 2.Here, " a plurality of original images of in predetermined period, catching continuously " moving image not necessarily.For example, " a plurality of original images of catching continuously in predetermined period " comprise the rest image of catching continuously with the interval of several seconds.
Facial image tracking apparatus 1 comprises that face image extracts parts 11, new identifier and authorizes parts 11a, position comparing unit 12, identifier and authorize parts 12a, similarity comparing unit 13, identifier and authorize parts 13a, and face image memory unit 14.
It is such functional modules that face image extracts parts 11, be used for from reading to extract the parts of images (face image) of the face that catches the people from the original image of image storage apparatus 3, when position comparing unit 12 and similarity comparing unit 13 can not be authorized any identifier of face image, the indication new identifier is authorized parts 11a and is authorized new identifier to the face image that extracts, and it is registered in the face image memory unit 14.Be used for for example describing in detail, therefore, will no longer describe in detail here at Japanese patent application No.3307354 from the concrete device of original image extraction face image.
Position comparing unit 12 is such functional modules, be used for reading in the face image (face image of being registered) that extracts and be registered in the original image that the next-door neighbour catches in the cycle before the original image that image storage apparatus 3 is read (next-door neighbour at preceding original image) in the face image memory unit 14, calculate the distance of extracting the face image that parts 11 extract from original image by face image, be the position appears in face image in original image difference, suppose that institute's face image that extracts and the face image of registering are same people's images when distance is not more than predetermined reference range, then indicator identifiers is authorized parts 12a and will be authorized the face image that extracts with the face image identical identifier of being registered, and it is stored in the face image memory unit 14.
Similarity comparing unit 13 is such functional modules, be used for when for all face images of registering that extract at preceding original image from the next-door neighbour, to the distance of the face image that extracts during greater than predetermined reference range, the face image that calculating extracts and register similarity between the face image, suppose when similarity surpasses pre-determined reference value, the face image that is extracted is same people's a image with the face image of registering, then will authorize the face image that extracts, and it will be stored in the face image memory unit 14 with registration face image identical identifier.Be used for calculating the concrete device of similarity and be used to utilize the device of similarity coupling face image for example to describe in detail, therefore, will no longer describe in detail here at Japanese patent gazette No.3976058 and No.4099981.
Face image memory unit 14 is the storage unit that are used to store face image data piece 4, and face image data piece 4 will be lumped together by face image extracts the position of the face image in the face image that extracts (its image information) that parts 11 extract and this original image from the original image of reading from image storage apparatus 3 coordinate and group identifier.
As shown in Figure 2, face image data piece 4 comprises identifier 41, imaging date and time 42, position coordinates 43 and face image data 44.
Identifier 41 is by combination camera identification code 411, record date and time 412 and number 413 formation.Camera identification code 411 is the exclusive codes of camera 2 of having caught the face image that extracts.Record date and time 412 be illustrate date when the face image that is awarded identifier 41 first is hunted down, hour, minute and second data.Numbering 413 is the sequence numbers that give a plurality of new face image that extracts from an original image by extraction order (order that identifier 41 is awarded).
Imaging date and time 42 be the date of the expression face image that extracts when being hunted down, hour, minute and second data.Position coordinates 43 is in the rectangular coordinate system that is fixed on the original image, expression the seat target value of the position of the face image that extracts in original image.
Face image data 44 is the numerical datas that are used to show the face image that extracts.Concrete restricted format, but can select any known format such as GIF and JPEG.Can at random select binary code, GTG and color according to purpose and environment.
Fig. 3 is the conceptual illustration that the physical arrangement of facial image tracking apparatus 1 is shown.Facial image tracking apparatus 1 is realized on computers by specific program is installed.In other words, Fig. 3 shows the physical configuration of the computing machine of realizing facial image tracking apparatus 1.
As shown in Figure 3, facial image tracking apparatus 1 comprises communication interface (I/F) 15, I/O unit 16, arithmetical unit 17 and storage unit 18.
Communication interface 15 is the interfaces that are used for transmission/reception data and signal between facial image tracking apparatus 1 and image storage apparatus 3.Communication interface 15 can also be used for to unshowned external unit transmission data and signal/from unshowned outer equipment receiving data and signal, unshowned external unit for example is used for to the result of computing machine transmission/reception from facial image tracking apparatus 1.
I/O unit 16 is the unit that are used for the result of instruction input facial image tracking apparatus 1 or demonstration facial image tracking apparatus 1, such as keyboard unit, mouse and the display unit such as liquid crystal panel.
Arithmetical unit 17 is the unit that are made of for example CPU (CPU (central processing unit)), and it reads the program in the storage unit 18 that is stored in such as ROM (ROM (read-only memory)) and RAM (random access memory), and working procedure is to carry out predetermined arithmetical operation.The function that aforementioned face image extracts parts 11, position comparing unit 12 and similarity comparing unit 13 realizes by moving aforementioned program.
Storage unit 18 for example is made of RAM or ROM, and is stored on the arithmetical unit 17 program of operation and from the result of arithmetical unit 17.Storage unit 18 is designated as the function of face image memory unit 14 at least in part.
Fig. 4 is used to illustrate that the distance based between the position of the position of extraction face image and the face image of registering follows the tracks of the key diagram of the primitive rule of face image.In Fig. 4, the original image that label 51 to 53 expressions are caught continuously by camera 2.More specifically, original image 51 is caught in first imaging cycle, and original image 52 is caught in next imaging cycle, and original image 53 is caught in follow-up imaging cycle.Original image 51 comprises face image 61 to 63.Original image 52 comprises face image 64 to 66.Original image 53 comprises face image 67 to 69.
In original image, moved limited distance when as shown in Figure 4, the face image that extracts from original image is to next imaging cycle.In other words, if the face image (last face image) that extracts from original image and the face image (back one face image) of catching next imaging cycle are same people's images, then they are very likely closer to each other.Therefore, if the alternate position spike between back face image in the original image and the last face image (i.e. distance) is not more than predetermined reference range, then can infer this two face images that face image is same people.
For example, if the distance between two face images of catching is not more than R in original image 51 to 53, think that then two face images are face images of same people in the tightly adjacent cycle.Then, same people's face image is integrated in the image, thus the image 54 to 56 after having obtained to merge.Image 54 to 56 after the merging comprises face image 61,64 and 67 respectively, face image 62,65 and 68, and face image 63,66 and 69.Circle on the image 54,55 and 56 after the merging comprises that having is the circle of radius centered R with face image 64,65 and 66 respectively.
Fig. 5 is the process flow diagram that is illustrated in the summary of the face image extraction procedure of operation on the arithmetical unit 17.The face image extraction procedure that utilization moves on arithmetical unit 17, arithmetical unit 17 extracts parts 11, new identifier as face image and authorizes parts 11a, position comparing unit 12, identifier and authorize parts 12a, similarity comparing unit 13 and identifier and authorize parts 13a.Below with reference to Fig. 5 the face image extraction procedure is described.
At first, read a frame (step S1) of original image from image storage apparatus 3.From the original image of reading, extract face image (face image that extracts) (step S2).Start position described later comparison program (step S3).
The position comparison program is authorized the face image that extracts with identifier.If the face image that extracts is stored in (step S4 in the face image memory unit 14; Be), then program advances to step S8.If the face image that extracts is not stored in (step S4 in the face image memory unit 14; ), then do not start similarity comparison program described later (step S5).
At step S5, the similarity comparison program is authorized the face image that extracts with identifier.If the face image that extracts is stored in (step S6 in the face image memory unit 14; Be), then program advances to step S8.If the face image that extracts is not stored in (step S6 in the face image memory unit 14; ), think that then the people who is extracted in the face image is different from the people who is followed the tracks of, and the face image that extracts is not awarded new identifier and is stored in (step S7) in the face image memory unit 14.Then, program advances to step S8.
If all included in the original image that reads in step S1 face images all are extracted out (step S8; Be), then process finishes.If there is any face image (the step S8 that is not extracted as yet; ), then program is not returned step S2 and is repeated said process.
Fig. 6 is the process flow diagram that the summary of position comparison program is shown.The position comparison program that reads in arithmetical unit 17 is in the operation of the step S3 place of face image extraction procedure.By the position comparison program of operation on arithmetical unit 17, arithmetical unit 17 is as position comparing unit 12.Below with reference to Fig. 6 the position comparison program is described.
At first, in the face image that face image memory unit 14 is stored, read following face image data piece 4 from face image memory unit 14, it is the face image data piece 4 (step S 11) of the face image (face image of registering) that extracts the original image with the original image next-door neighbour who reads from image storage apparatus 3 in the step S1 of face image extraction procedure, and calculates the distance (step S12) of the face image that extracts.
(step S 13 if the distance that calculates in step S 12 is not more than predetermined reference range; Be), then think the face image that extracts and the face image of registering be same people's image.Therefore, the face image that extracts be awarded with the face image identical identifier of registering and be stored in (step S 14) in the face image memory unit 14.Then, program is returned process shown in Figure 5.
On the other hand, if the distance that in step S12, calculates greater than predetermined reference range (step S13; Not), then program advances to step S15.If all be read from next-door neighbour's the face image of registering to some extent that extracts of original image, in other words, if anyly register all discoveries the face image and extract people in the face image (step S 15 what the original image from the next-door neighbour extracted; Be), then program is returned process shown in Figure 5.If exist any do not read the image of registering (step S15; ), then program is not returned step S11 and is repeated top process.
Fig. 7 be used for illustrating people's the homogeny of the image that extracts and the face image of registering and the key diagram of the relation between the reference range.In Fig. 7, the label 71 expression face images of registering.In the present embodiment, the position of face image is represented with the position of right eye.More specifically, with the corresponding locations of pixels of the right eye of registration face image 71 be considered to the position of the face image of registering 71 in original image.Therefore, use the rectangular coordinate system X-Y of true origin at the right eye place of registration face image 71.
As mentioned above, reference range is the ultimate range that face image can move in imaging cycle.Ultimate range is long-pending corresponding with maximum translational speed and imaging cycle.For example, suppose that the people that wants imaging moves with the maximal rate of 1000mm/s and imaging cycle is 1/10s, then people's transportable ultimate range before following one-period is 100mm.Then, will compare with its size (pixel count) that manifests on original image by the actual size of the object of camera 2 imagings, and calculating and the corresponding number of pixels R of 100mm linear scale.Pixel count between the right eye of the face image of registering and the right eye of the face image that extracts is not more than R, if these face images are images of same people.
In Fig. 7, the right eye of face image 72 is the circle inside of radius centered R in the true origin (right eye of the face image of being registered 71) that has with rectangular coordinate system X-Y.Therefore, think the face image of registering 71 and face image 72 are face images of same people.On the other hand, the right eye of face image 73 is in this circle outside.Therefore, think the face image of registering 71 and face image 73 be not same people's face image.
Pixel count R selects according to the purpose of facial image tracking apparatus 1.For the more important facial image tracking of processing speed, increase pixel count R, so that determine same people's ratio increase and determine that based on similarity same people's ratio reduces based on distance.Otherwise, for the more important facial image tracking of accuracy, then reduce pixel count R, so that reduce and determine same people's ratio increase based on the definite same people's of distance ratio based on similarity.
Can regulate pixel count R according to the number of face image included in the original image.When original image comprises a small amount of face image,, can not reduce the accuracy of determining same people even pixel count R increases yet.Therefore, can increase pixel count R.On the other hand, when original image comprises a large amount of face image,, determine that then same people's accuracy reduces if pixel count R increases.Therefore, should reduce pixel count R.
Fig. 8 is the process flow diagram that the summary of similarity comparison program is shown.The similarity comparison program that reads in arithmetical unit 17 is moved in the step S5 of face image extraction procedure.More specifically, when the position comparison program be can not determine the identifier of the face image that extracts, then start the similarity comparison program.In addition, by the similarity comparison program of operation on arithmetical unit 17, arithmetical unit 17 is as similarity comparing unit 13.Below with reference to Fig. 8 the similarity comparison program is described.
At first, read following face image from face image memory unit 14, its be from among the step S1 of face image extraction procedure, read and write the face image (face image of registering) (step S21) that extracts in the original image of the original image next-door neighbour the face image memory unit 14, and the similarity (step S22) of the calculating and the face image that extracts from image storage apparatus 3.
If the similarity that calculates in step S22 is not less than pre-determined reference value (step S23; Be), then think the face image that extracts and the face image of registering be same people's image.Therefore, the face image that extracts is awarded and the face image identical identifier of registering, and is stored in (step S24) in the face image memory unit 14.Then, program is returned process shown in Figure 5.
On the other hand, if the similarity that in step S22, calculates less than pre-determined reference value (step S23; Not), then program advances to step S25.If all be read from next-door neighbour's the face image of registering to some extent that extracts of original image, in other words, if anyly register all discoveries in the face image and extract people (step S25 in the face image what from next-door neighbour's original image, extract; Be), then program is returned process shown in Figure 5.If exist any without read the face image of registering (step S25; Not), then return step S21 and repeat top process.
As mentioned above, since facial image tracking apparatus 1 at first based on extraction face image and the distance between the face image registered judge the face image that extracts with the face image of registering whether be same people's face image, therefore, compare with judging based on the similarity of face image, facial image tracking apparatus 1 allows high speed processing also to reduce the computer operation load.Then,, then carry out judgement based on the similarity of face image if the above-mentioned distance between the position-based is not found same people's the face image of registering, so accuracy and the reliability of facial image tracking apparatus 1 when having guaranteed definite homogeny.
In addition, same people's face image is awarded same identifier and face image is stored.Therefore, can obtain and follow the tracks of the positional information that specific people moves in chronological order.
14 storages of face image memory unit have the face image data piece 4 at the same identifier 41 of same people's face image.Therefore, can retrieve same people's face image data piece 4.The same people's that can read out by analysis face image data piece 4 calculates this people's track.
Technical scope of the present invention is not limited thereto embodiment.Under the situation that does not break away from the technological thought of setting forth in the patent claims, can carry out various application, modification and change.For example, in the present embodiment, specific program is installed on the arithmetical unit 17 is embodied as software face image is extracted parts 11, position comparing unit 12 and similarity comparing unit 13.Yet, realize that by specialized hardware these parts drop within the technical scope of the present invention.
The application is submit to and Japanese patent application No.2008-326380 that comprise instructions, claims, accompanying drawing and summary of the invention based on Dec 22nd, 2008.The open integral body by reference of above-mentioned Japanese patent application is incorporated into this.