JP4208450B2 - Face image monitoring system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus, and more particularly to a face image monitoring system using image information from a camera or the like.
[0002]
[Prior art]
Recently, for example, in a store such as a convenience store, a monitoring system based on a monitoring image from an ITV camera or the like installed in the store is very popular. There are various forms of this monitoring system. For example, the following three types are representatively mentioned.
One is a monitoring system that transmits a monitoring image taken by an ITV camera installed in a store to a monitoring center so that a monitoring person can monitor it.
[0003]
Similarly, there is a monitoring system that records a monitoring video taken by an ITV camera in a time-lapse VTR at regular intervals. In this system, it is not necessary for the supervisor to constantly monitor, and the recorded images can be referred to later as needed.
Similarly, there is a monitoring system that detects a person based on a change in an image from a monitoring video imaged by an ITV camera and records it on a VTR or saves it on a disk. In this system, since an image is stored only when there is a change in an image to be monitored, more efficient monitoring image collection can be performed with a small storage capacity.
[0004]
[Problems to be solved by the invention]
  However, each of the monitoring systems described above has problems as follows. In other words, in the first monitoring system, there is a problem that the monitoring staff must always monitor the monitoring image, and the labor cost becomes very high as the system cost. In the second monitoring system, since only video is recorded at regular intervals, many meaningless images without a person are recorded, so there are problems such as insufficient storage capacity and difficulty in searching for necessary images. Furthermore, in the third monitoring system, only an image with a person is obtained, but there is a problem that it is not always an image suitable for confirmation of the monitoring image later.
  In the present invention, only images necessary for monitoring are taken from images taken from an ITV camera or the like.outputThus, an object is to provide a face image monitoring system that realizes more efficient monitoring processing.
[0005]
[Means for Solving the Problems]
  The present invention relates to image receiving means for continuously receiving a plurality of pieces of image information to be monitored from the outside, and image information received by the image receiving means.If a face area is detected fromA person image determining unit that determines that the image is a person image, and the person image determining unit determines that the image is a person image;The direction of the person's face is detected for the face area, and the image is close to the front face based on the face direction.Face to output evaluation valuedirectionAn evaluation means;High evaluation value output from the plurality of image information by the face direction evaluation meansA predetermined number of the image informationPreferentiallyA face image monitoring system comprising image information selecting means for selecting and outputting.
  The present invention has the above-described configuration.Instead of selecting and outputting all the face images, the face direction is detected, and for example, the face image that is very effective for face image monitoring is automatically output by selecting and outputting mainly the front face image that is effective by person determination. It becomes possible to detect.
[0006]
  The present invention also provides image receiving means for continuously receiving a plurality of pieces of image information to be monitored from the outside, and image information received by the image receiving means.If a face area is detected fromA person image determining unit that determines that the image is a person image, and the person image determining unit determines that the image is a person image;The face area is detected for the face area, and the face area is detected.Based onHigh for images with large face areaFace to output evaluation valueareaAn evaluation means;The evaluation value output by the face area evaluation means from the plurality of image information is highA predetermined number of the image informationPreferentiallyAn image information selection means for selecting and outputting the face image monitoring system.
  The present invention does not selectively output all face images by the above-described configuration,For example, more efficient face monitoring processing can be realized by selectively outputting only images having a relatively large face area and effective for face image monitoring.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below in detail with reference to the drawings.
(First embodiment)
First, a first embodiment will be described with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of the first to fourth embodiments.
[0017]
In FIG. 1, the entire face image monitoring system includes an image input unit 1, an A / D conversion unit 2, an image buffer (image memory) 3, and an image processing unit 4 (processing processor).
The image captured by the image input unit 1 is converted into a digital signal by the A / D conversion unit 2 and stored in the image memory 3. The processor 4 reads the image data and performs digital image processing. The image input unit 1 may be a normal CCD camera, for example.
[0018]
Hereinafter, the processing contents in the processing processor 4 will be described in detail. FIG. 2 is a flowchart showing the flow of processing of the processing processor unit of the first embodiment.
FIG. 2 shows the flow of processing performed by the processing processor 4. The processing procedure includes a process of cutting out N consecutive image sequences in which the same person exists (S5), a process of determining the number of selected images P (S6), and a process of selecting P images from the N image sequences (S7). ).
[0019]
FIG. 3 is an explanatory diagram showing scene extraction. In the above-described cut-out processing (S5) of the N consecutive image series, the image read from the image memory 3 is compared with the reference background image. Then, a person candidate area that has entered from the difference is detected, and a continuous image series in which a person is detected is extracted as a scene.
[0020]
For example, human area detection is performed by performing difference processing on successive images, binarizing with a certain threshold value after performing smoothing processing to suppress the influence of noise, and in the sense of 4-connected or 8-connected. Integration and labeling may be performed as spatially connected regions, and a region having a high likelihood as a person may be selected from the size and shape of the regions. At this time, the threshold value for binarization may be a value given in advance, or may be determined by the technique [1] that sets the value that maximizes the variance ratio of the frequency distribution.
[0021]
As another method, the method [2] based on the posterior probability may be used. With the event θ0 as the background and the event θ1 as the person, the prior probabilities of θ0 and θ1 before observing the luminance of the pixels in an image are w0 and w1 (where w0 + w1 = 1), respectively, and the event θk Assuming that the conditional probability of the image luminance I is P (I | θk), according to Bayes' theorem, the posterior probability w1 ′ after I is observed is
w1 ′ = w1 * P (I | θ1) / {w0 * P (I | θ0) + w1 * P (I | θ0)}. A pixel whose w1 ′ is larger than a certain threshold value is set as a pixel belonging to a person, and a certain pixel having this area may be detected as a person.
[0022]
Next, an example of a procedure for cutting out a continuous image series in which the same person is detected as one scene will be described. The first image (first frame) is determined when, for example, a person candidate area is detected for the first time at one end of the monitoring screen (when a person enters). The last image (final frame) is determined when, for example, the person candidate area is not detected after the person candidate area is detected at one end of the monitoring screen (when the person leaves). As a result, the number of cut out images is N.
[0023]
Cutting out continuous images including the same person is not limited to the above. For example, the images may be cut out by dividing each zone in advance according to the detection position in the screen. In addition, if the person's movement is not considered by performing tracking processing while associating people between images based on their position, size, moving speed, similarity of color information, etc., the person candidate area is selected as a candidate. It is also possible to increase the reliability of person determination by performing processing such as more removal.
[0024]
In addition, the number of persons is not limited to one person, and even when a plurality of persons are included, it is possible to cut out continuous images in which each person exists by tracking them (in this case, the images are duplicated). Cut out). Or you may cut out the continuous image in which several persons exist as a unit.
[0025]
Finally, a method for determining the number of selected images P will be described.
Here, the selected image number P is set to P = [maximum integer not exceeding α * N] based on the selection rate α. The selection rate α may be given in advance by the user, or may be dynamically changed according to the degree of congestion of the image transmission unit 5 :. Then, it is assumed that P sheets are selected by thinning evenly out of N sheets.
[0026]
According to the procedure shown as the first embodiment above, instead of selecting and outputting all (N) images showing moving objects as in the prior art, a representative method is used by selecting them by a certain method. P sheets are selected. By doing so, it is possible to save a great amount of memory even when saving an image, and it is possible to perform efficient processing when performing a collation search or the like later.
[0027]
(Second Embodiment)
Next, a second embodiment of the present invention will be described with reference to the drawings. The overall configuration of the embodiment of the present invention is the same as that of the first embodiment shown in FIG.
Hereinafter, the processing contents in the processing processor 4 will be described in detail.
[0028]
FIG. 4 is a flowchart showing a processing flow of the processing processor unit of the second embodiment, and shows processing performed by the processing processor 4. In FIG. 4, the processing procedure includes a process of buffering N consecutive image series (S8), a process of performing face recognition on each image of the continuous image series and calculating a selection priority (S9), And processing for selecting an image based on the selection priority (S10).
[0029]
In the process S8, N sheets may be simply buffered at regular time intervals, or motion detection may be performed in the same manner as in the first embodiment, and N consecutive images in which a person exists may be cut out. The value of the parameter P may be given as a fixed value in advance (the selection is not performed when P is greater than or equal to N), or P = [α based on the selectivity α as in the first embodiment. * Maximum integer not exceeding N]. Next, a method for performing face recognition on each image of a given continuous image series and calculating a selection priority will be specifically described with reference to FIG. FIG. 5 is an explanatory diagram illustrating a specific example of processing in the image selection unit.
[0030]
2-1. Human face direction evaluation value
For example, an image in which a person's face is seen in front is considered to be an image more suitable for person verification. Therefore, a high evaluation value is given to an image close to the face direction desired by the user (for example, the front face), and the image is preferentially selected.
[0031]
The face direction of a person can be obtained by canonical discriminant analysis [3], for example. This can be realized by preparing learning face images classified into classes such as front face, left face, right face, etc., and performing identification by canonical discriminant analysis as follows.
(A) The pixel value of the face area of A × B dots is considered as an A × B dimensional vector, and intra-class variance Sb and inter-class variance Sw are calculated.
(B) Solve the eigenvalue problem inv (Sw) Sb v = λv (inv (Sw) is the inverse matrix of Sw).
(C) A projection matrix having V = eigenvector vi as a column vector is obtained.
(D) The average vector of each class image is projected onto V, and the average projection vector of each class is calculated.
(E) At the time of recognition, the input image area is projected onto V, and the result is compared with the average projection vector in (d). By performing the process on the person candidate area detected by the image selection process, it is possible to detect the face and identify the face direction.
[0032]
FIG. 6 is an explanatory diagram showing the concept of face direction identification processing, and supplements the above description.
In addition to this method, a standard pattern such as eyes / nose / mouth may be prepared, the pattern may be matched with a person region, and the face orientation may be recognized from the pattern positional relationship.
[0033]
Evaluation value given by the user for the face direction obtained in this way
{Front face = 100, Left diagonal face = 70, Right diagonal face = 70,
Right face = 40, left face = 40, back side = 10, ...}
By applying, the face direction evaluation value E1 of the person can be determined.
[0034]
2-2. Human face area evaluation value
An image with a large person's face is considered to be an image more suitable for human verification. Therefore, a high evaluation value is given to an image having a large face area of a person, and the image is preferentially selected. The area S of the face region can be obtained, for example, by calculating the area of the connected skin color region. Therefore, this value may be used as the human face area evaluation value.
E2 = S
2-3. Human face contrast evaluation value
An image with high face contrast is considered to be an image more suitable for human verification. Therefore, a high evaluation value is given to the image with high contrast, and the image is preferentially selected. The face contrast can be calculated based on the variance value μ of the brightness of the face area. Since it can be said that the greater the variance, the greater the difference in luminance and the higher the contrast, this may be used as the evaluation value. Therefore,
E3 = μ
It can be.
[0035]
2-4. Human skin color clarity evaluation value
An image having a high flesh color clarity is considered to be an image more suitable for human verification. Therefore, an evaluation value is given to the image with high definition and is preferentially selected. The skin color sharpness can be evaluated by looking at the average saturation (saturation) of the face area based on the equation [3] for converting an RGB image into an HSV (Hue, Saturation, Value) space.
[0036]
For example, saturation S is
S = (I−i) / I
However, I = max (R, G, B), i = min (R, G, B)
Or
S = {(BR)²+ (RG)²+ (GB)²} / 3
Can be obtained. Therefore, the skin color clarity evaluation value is
E4 = S
It can be.
[0037]
2-5. Human facial expression evaluation value
An image with a neutral facial expression (without blinking) is considered to be an image more suitable for human verification. Therefore, the evaluation value is calculated by giving priority to the facial expression. For facial expression evaluation values, for example, learning face images classified into classes such as normal facial expressions, blinking faces, and open mouth faces are prepared, and identification by canonical discriminant analysis similar to the above-described face direction identification is performed. Evaluation value given by the user for each class
{Normal expression = 100, blinking face = 70, open mouth face = 60, ...}
E5 may be determined based on the above.
Another method is to prepare standard patterns such as open eyes, closed eyes, open mouths, closed mouths, etc., match the pattern with the face area, and identify facial expressions from the combination of patterns. You may do it.
[0038]
The embodiment is not limited to the above. When there are a plurality of persons in the scene, for example, an evaluation value for each person may be added. Moreover, although the said was set as the separate independent evaluation value, it is good also as one evaluation value combining these. For example, weighting coefficients w1, w2,..., Wx may be defined in advance for a plurality of evaluation values E1, E2,.
[0039]
E6 = w1 * E1 + w2 * E2 + ... + wx * Ex
In addition, a priority order is defined for the evaluation scale itself, a priority order is assigned at E1, and if it is a peer, a priority order is assigned at E2, and if E2 is also a peer, a priority order is assigned at E3. You may do it.
[0040]
Next, processing for selecting an image based on the selection priority will be described. Here, P images are selected from the N image series in descending order of selection priority (P is an integer equal to or less than N and is appropriately given by the system). The selected image can be used for the purpose of network image transmission or saving to an appropriate image storage device such as a magnetic disk or VTR, and only the appropriate selected image needs to be transmitted and stored. Less capacity and storage capacity.
[0041]
As for the processing form, it is assumed here that P sheets are selected from N sheets. However, each time one image is input, the priority Ei is calculated, and the selected image is selected when the priority is equal to or higher than a certain threshold value. Such sequential processing may be used.
As described above, by selecting and outputting the moving image by the method shown in the second embodiment, it is possible to realize more efficient face image monitoring processing according to various conditions of the face image.
[0042]
(Third embodiment)
Next, a third embodiment of the present invention will be described with reference to the drawings. The overall configuration of the embodiment of the present invention is the same as that of the first embodiment shown in FIG.
Hereinafter, the processing contents in the processing processor 4 will be described in detail.
FIG. 7 is a flowchart showing a processing flow of the processing processor unit of the third embodiment, and shows a processing flow performed by the processing processor 4. The processing procedure includes a process for buffering N consecutive image sequences (S11), a process for performing recognition of a person's whole body for each image in the continuous image sequence and calculating a selection priority (S12), And processing for selecting an image based on the selection priority (S13). The process (S11) may be the same as in the second embodiment.
[0043]
Next, a method (processing S12) for calculating the selection priority by performing overall recognition of a person for each image in a given continuous image series will be described in detail. FIG. 8 is an explanatory diagram showing the process of counting, moving and recognizing a person, and explains the third embodiment.
[0044]
3-1. Number of people evaluated
It is conceivable that images having a large number of persons capture the state of the monitoring area better and select them as appropriate images. On the other hand, scenes with a small number of persons are likely to be fraudulent in the monitoring area, and it may be possible to select these as appropriate images. Therefore, the priority evaluation value can be obtained based on the number of persons in the image. First, person areas are extracted by the same method as in the first embodiment, and the number of persons Nh included in the screen is obtained from the number of areas. Therefore, the number of people evaluated is
E7 = Nh (when priority is given to a large number of people)
Or
E7 = 1 / Nh (when priority is given to those with a small number of people)
It can be.
[0045]
3-2. Human systemic evaluation value
An image that captures the entire image of a person is useful for later human verification because the height, color pattern of clothes, etc. are known. Therefore, a high evaluation value is given to such an image, and the image is preferentially selected. First, the area of the person area for each image is calculated by the same method as in the first embodiment, and then the face area is calculated by the same method as in the second embodiment. That is, if the area of the person area is S0 and the area of the face area is S1,
E8 = S0 / S1
Is an evaluation value of how much the whole image is captured. For example, the fact that the face area S1 is smaller than the person area S0 means that the whole image up to the body part is well captured, and then S0 / S1 has a large value.
[0046]
The embodiment is not limited to the above. When there are a plurality of persons in the scene, for example, an evaluation value for each person may be added. Moreover, although the said was set as the separate independent evaluation value, it is good also as one evaluation value combining these. For example, weighting coefficients w7 and w8 may be defined in advance for a plurality of evaluation values E7 and E8, and weighted addition may be performed.
[0047]
E9 = w7 * E7 + w8 * E8
Further, a hierarchical process may be performed in which a priority order is defined for the evaluation scale itself, and a priority order is assigned at E7 and a priority order is assigned at E8 in the case of peers.
[0048]
Next, the process (S13) for selecting an image based on the selection priority will be described. Here, as in the second embodiment, P images are selected from N image sequences in descending order of selection priority (P is an integer equal to or less than N, and is appropriately given by the system). ). The selected image can be used for the purpose of network image transmission or saving to an appropriate image storage device such as a magnetic disk or VTR, and only the appropriate selected image needs to be transmitted and stored. Less capacity and storage capacity.
[0049]
As for the processing form, it is assumed here that P sheets are selected from N sheets. However, each time one image is input, the priority Ei is calculated, and the selected image is selected when the priority is equal to or higher than a certain threshold value. Such sequential processing may be used.
Also in the third embodiment, more efficient person monitoring processing can be realized by providing conditions such as the number of persons and the generality of the person.
[0050]
(Fourth embodiment)
The overall configuration of the embodiment of the present invention is the same as that of the first embodiment shown in FIG.
Hereinafter, the processing contents in the processing processor 4 will be described in detail. FIG. 9 is a flowchart showing a processing flow of the processing processor unit of the fourth embodiment, and shows a processing flow performed by the processing processor 4. The processing procedure includes a process of buffering N consecutive image sequences (S14) and an image I having the given N consecutive image sequences._iIn contrast, I_iA function E (f) for obtaining a selection priority based on information of itself and the past k images_i) = F (I_i-k, I_{i-k + 1}, ..., I_i) And define (I_i-kRepresents “i-kth image I”), and processing for calculating a selection priority based on this function (S15) and processing for selecting an image based on the selection priority (S16). . About a process (S14), it may be the same as that of 2nd Embodiment.
Next, the process (S15) for calculating the selection priority based on the function will be described. FIG. 8 is an explanatory view showing the process of counting, moving and recognizing a person.
[0051]
4-1. Human moving speed evaluation value
A scene with a low moving speed is considered to be an appropriate image in which the person is stable and stationary and is easier to verify the person. Therefore, a high evaluation value is given to a scene having a lower moving speed, and the scene is preferentially selected.
[0052]
Image I_i-1And I_iIs obtained from the person area information contained in the image based on the similarity of position, size, color, etc. (person tracking), and the movement vector (movement speed) V is determined from the change in the position. Is obtained. Therefore, the moving speed evaluation value of the person is
E10 = 1 / (c + V) (c is a constant for avoiding division by zero when V = 0)
It can be.
[0053]
4-2. Person non-stationarity evaluation value
A scene in which a person performs an unusual operation is likely to have some special event, and is important in verification. Therefore, a high evaluation value is given to these non-stationarities and selected preferentially.
[0054]
In advance, a standard pattern of a change sequence of a feature amount (for example, a movement vector, a position, a size, and a shape) of a person region with respect to a steady motion / unsteady motion is learned in advance. Input image series I_i-k, I_{i-k + 1}, ..., I_iSimilarly, a change sequence of the feature amount of the person region is obtained, and these are matched with a standard pattern by performing DP matching or matching with an HMM model.
[0055]
As a steady operation at this time, for example, in the case of a face image monitoring system in a convenience store, an intrusion into a cashier counter that a general customer cannot enter can be considered. For example, by setting a register counter area in an image, it is determined that a moving object enters the image area as a special event.
[0056]
Here, with respect to the input image series, the matching score Score (Am) is the highest in Am in the feature amount change pattern group of the steady motion, and the matching score Score (Bn) is highest in Bn in the feature amount change pattern group of the non-steady motion. Is high, the evaluation value indicating non-stationarity is
EE = (similarity with non-stationary state) − (similarity with stationary state) = Score (Bn) −Score (Am)
Can be obtained. When the evaluation values EE1, EE2,..., EEx for a plurality of feature quantity series are used, weighted addition may be performed using weighting coefficients ww1, ww2,.
[0057]
E11 = ww1 * EE1 + ww2 * EE2 + ... + wwx * EEx
The above is a separate independent evaluation value, but these may be combined into a single evaluation value. For example, weighting coefficients w10 and w11 may be defined in advance for a plurality of evaluation values E10 and E11, and weighted addition may be performed.
[0058]
E12 = w10 * E10 + w11 * E11
Further, a hierarchical process may be performed in which a priority order is defined for the evaluation scale itself, and a priority order is assigned at E10 and a priority order is assigned at E11 if the priority order is the same.
Next, the process (S11) for selecting an image based on the selection priority will be described. Here, as in the second embodiment, P images are selected from N image sequences in descending order of selection priority (P is an integer equal to or less than N, and is appropriately given by the system). ). The selected image can be used for the purpose of network image transmission or saving to an appropriate image storage device such as a magnetic disk or VTR, and only the appropriate selected image needs to be transmitted and stored. Less capacity and storage capacity.
[0059]
As for the processing form, it is assumed here that P sheets are selected from N sheets. However, each time one image is input, the priority Ei is calculated, and the selected image is selected when the priority is equal to or higher than a certain threshold value. Such sequential processing may be used.
Also in the fourth embodiment, more efficient image monitoring processing can be realized by selecting images with reference to the movement speed of a person, detection of intrusion into a predetermined area, and the like.
[0060]
(Fifth embodiment)
The overall configuration of the embodiment of the present invention is the same as that of the first embodiment shown in FIG.
Hereinafter, the processing contents in the processing processor 4 will be described in detail.
FIG. 10 is a flowchart showing a processing flow of the processing processor unit of the fifth embodiment, and shows a processing flow performed by the processing processor 4. The processing procedure includes a process of buffering a sequence of N consecutive images (S17), and all N images (I₀To I_N) To select P sheets from “combinations” nCp, define an evaluation function g (), calculate the selection priority based on this function (S18), and select an image based on the selection priority Processing (S19). About a process (S17), it may be the same as that of 2nd Embodiment.
Next, the process 14 for calculating the selection priority based on the function will be described with reference to FIG. FIG. 11 is an explanatory diagram illustrating a specific example of processing in the image selection unit based on a combination of P sheets.
[0061]
5-1. Diversity of person positions
If an image is selected that samples the trajectory that has moved from the intrusion to the exit of the person as equally as possible, it is thought that it better summarizes the entire behavior of the person. Therefore, P images are selected. (Simple equidistant sampling based on the number of images may select only the stationary state.)
Image I_iThe coordinates of the person (X (I_i), Y (I_i)) Is obtained as a person candidate region in the selection unit 8: N image series. I_iTo I_jThe movement vector to is I_iCoordinates and I_jIs obtained as a difference in coordinates. Therefore, P images are selected from N images {I ′_i, ..., I '_P} For a set of images (nCp ways)_aveAnd movement vector variance V_dispFor V_aveIs the maximum, V_dispIt is sufficient to select a set of P sheets that minimizes. That is, V_aveA large value indicates that the movement between images is appropriately selected, and V_dispA small value indicates that sampling at an equal interval can be performed. FIG. 12 is a diagram illustrating an example of selection based on diversity of person positions.
[0062]
5-2. Diversity of human faces
If a person selects an image that is directed in a variety of directions, it captures more of the person's characteristics and is considered useful for verification. Therefore, P images are selected. First, the face direction and its likelihood are obtained by face direction identification by the canonical discriminant analysis described above. At this time, the number of types of face direction classes included in the scene is C. In order to obtain images with more various face directions, it is only necessary to select a set having a large dispersion in the face direction from a combination of N to P. For example, class numbers 1, 2,..., C are assigned to the face direction classes, and selection is made so that the variance of the class number values of P images is maximized. (If all classes are selected evenly, the variance of the class number value increases.) At this time, the face likelihood height of each image may be weighted. Through the above processing, face images facing more various directions can be selected.
[0063]
The first to fifth embodiments described above may be performed in real time, or images may be accumulated once and processed later in a batch type.
As described above, also in the fifth embodiment, efficient monitoring processing can be realized by selecting and outputting an image in consideration of diversity of person positions and diversity of person faces.
[0064]
(Sixth embodiment)
Finally, the overall configuration of the embodiment of the present invention is shown in FIG.
In FIG. 13, the face image monitoring system includes an image input unit 20, an A / D conversion unit 21, an image buffer (image memory) 22, an image processing unit 23 (processing processor), and an image storage unit 24. Become.
[0065]
The image captured by the image input unit 20 is converted into a digital signal by the A / D conversion unit 21 and stored in the image memory 22. The processor 23 reads the image data and performs digital image processing. The image input unit 20 may be an ordinary CCD camera, for example, and the image storage unit 24 may be an ordinary magnetic disk.
[0066]
Hereinafter, the processing contents in the processing processor 23 will be described in detail.
FIG. 14 shows a flow of processing performed by the processing processor 23. The processing procedure includes a process of cutting out N consecutive image sequences in which the same person exists (S25), and a process of performing person and face image recognition on each image and selecting P images from the N image sequences (S26). ), Processing for registering P images in the image storage unit (S27), and processing for searching for and displaying images from the image storage unit (S28).
[0067]
The process of cutting out N consecutive image sequences in which the same person is present (S25) is the same as in the first embodiment. Further, the process of performing person and face image recognition on each image and selecting P images from the N image sequences (S26) is the same as in the second to fifth embodiments. Regarding the process of registering P images in the image storage unit, database registration is performed together with attached information related to the image such as the camera ID and time when the image was taken. It is assumed that the definition relating to the data structure of the database is performed in advance in, for example, a relational table format. Here, a method for searching and displaying an image will be specifically described. FIG. 15 is an explanatory diagram showing the concept of image search and collation according to the present invention.
[0068]
By displaying the selected image as a thumbnail (representative image display), the user can easily verify the monitoring image. Furthermore, it is possible to automatically collate the face photograph (for example, a wanted person offender) presented by the user with these images and display a face image or image series having a high degree of similarity. The user may specify an image series instead of an image.
[0069]
In the matching of a person, a pattern similar to the designated person is searched by principal component analysis or canonical discriminant analysis based on the designated person image / image series and other person face image groups [4. ] [5]. In addition, the similarity of color information may be used for the search, or the similarity of the feature amount series as shown in the fourth embodiment may be used. Thus, for example, it is possible to search a certain person from the monitoring image series, or to extract only a certain behavior (for example, entry into a cash register).
[0070]
As described above, according to the sixth embodiment, by providing the face image monitoring system with the image storage unit, it is possible to automatically and sequentially store images selected and output in accordance with a predetermined condition. Furthermore, for these stored images, representative images corresponding to certain conditions are displayed as thumbnails, thereby enabling a more intuitive grasp of the monitoring status.
[0071]
References are listed below.
[1] Otsu: “Automatic threshold selection method based on discriminant and least square criterion”, IEICE, Vol.J63-D, No.4, pp.349-356, 1980.
[2] Nakai: “A moving object detection method using posterior probabilities”, Information Processing Research Bulletin, 94-CV-90, pp.1-8, 1994.
[3] Takagi, Shimoda: “Image Analysis Handbook”, The University of Tokyo Press, 1991.
[4] W Zhao, R. Chellappa et al. “Discriminant Analysis of Principal Components for Face Recognition”, International Conference on Automatic Face & Gesture Recognition, Nara, Japan, April 1998.
[5] B. Moghaddam, A. Pentland et al. “Beyond Eigenfaces: Probabilistic Matching for Face Recognition”, International Conference on Automatic Face & Gesture Recognition, Nara, Japan, April 1998.
[0072]
【The invention's effect】
  As described above in detail, according to the present invention, the monitoring imageFace orientationBy selecting only the appropriate image as the monitoring imageoutputcan do.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an overall configuration of first to fourth embodiments.
FIG. 2 is a flowchart illustrating a processing flow of a processing processor unit according to the first embodiment;
FIG. 3 is an explanatory diagram showing scene extraction.
FIG. 4 is a flowchart illustrating a processing flow of a processing processor unit according to the second embodiment;
FIG. 5 is an explanatory diagram showing a specific example of processing in an image selection unit.
FIG. 6 is an explanatory diagram showing a concept of face direction identification processing;
FIG. 7 is a flowchart illustrating a processing flow of a processing processor unit according to the third embodiment;
FIG. 8 is an explanatory view showing a process of counting, moving, and systemic recognition of a person.
FIG. 9 is a flowchart illustrating a processing flow of a processing processor unit according to the fourth embodiment;
FIG. 10 is a flowchart illustrating a processing flow of a processing processor unit according to the fifth embodiment;
FIG. 11 is an explanatory diagram illustrating a specific example of processing in an image selection unit based on a combination of P sheets.
FIG. 12 is a diagram illustrating an example of selection based on diversity of person positions.
FIG. 13 is a block diagram showing the overall configuration of a sixth embodiment.
FIG. 14 is a flowchart illustrating a processing flow of a processing processor unit according to the sixth embodiment;
FIG. 15 is an explanatory diagram showing the concept of image search and collation according to the present invention.
[Explanation of symbols]
1 ... Image input section
2 ... A / D converter
3 ... Image memory
4 ... Processing processor
24 ... Image storage unit

Claims (2)

Image receiving means for continuously receiving a plurality of pieces of image information to be monitored from outside;
A person image judging means for judging that the face area is detected from the image information received by the image receiving means;
A face direction evaluation unit that detects a face direction of a person for the face region that the person image determination unit has determined to be a person image, and outputs a high evaluation value to an image close to a front face based on the face direction;
A face image monitoring system comprising: image information selection means for preferentially selecting and outputting a predetermined number of the image information having a high evaluation value output by the face direction evaluation means from the plurality of image information. . Image receiving means for continuously receiving a plurality of pieces of image information to be monitored from outside;
A person image judging means for judging that the face area is detected from the image information received by the image receiving means;
The person image determining unit detects the area of the face for the face area is determined that the portrait image, a face area evaluation means for outputting a high evaluation value in the face area is large image based on the face area,
A face image monitoring system comprising: image information selection means for preferentially selecting and outputting a predetermined number of the image information having a high evaluation value output by the face area evaluation means from the plurality of image information. .

Images