JP2011060024A - Apparatus for selecting feature information applied for image recognition processing, and image recognition processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for selecting feature information capable of selecting feature information effective in image recognition processing for achieving of chasing and identifying a person even when there are variations in environmental conditions. <P>SOLUTION: In an image forming apparatus 10 with the apparatus for selecting the feature information to extract a recognition target from image information photographed by cameras 11A-11C, there is provided a function for selecting the feature information effective in the image recognition processing from a plurality of feature information based on information in importance and reliability. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image recognition processing technique for extracting a recognition target from image information captured by a camera, and more particularly to a technique for selecting feature information of a recognition target.

  In recent years, monitoring systems using image (video) information captured by a camera have become widespread. In particular, there is an increasing demand for a wide area monitoring system that monitors an area where many people pass.

  In the monitoring system, an image recognition processing apparatus for processing image information captured by a camera and extracting characteristics of a person to be monitored is applied. In the wide area monitoring system, an image recognition processing device is required for identifying and tracking a moving person based on the characteristics of the person to be monitored using a plurality of cameras.

  Conventionally, prior art that performs personal authentication based on a plurality of feature information of a person has been proposed (see, for example, Patent Documents 1 and 2). These prior arts are configured to acquire a plurality of types of biometric information such as fingerprints, irises, faces, voiceprints, handwriting, etc., and perform personal authentication by collating with biometric information registered in advance. These prior arts are based on pre-registration of biometric information in a personal authentication device, and execute processing using biometric information collected in an ideal environment.

  Prior art relating to pattern recognition technology included in image processing technology has been proposed (see, for example, Patent Documents 3 and 4). The prior art described in Patent Document 3 relates to a mechanism for generating excessive features from an image and selecting necessary features from the image assuming character recognition and the like. This prior art is based on the fact that many features are input to the apparatus in advance for the recognition target, and the average pattern is registered as dictionary data. In addition, the prior art described in Patent Document 4 extracts features necessary for pattern identification from an image by incorporating a pattern learning function such as a neural network.

Japanese Patent No. 3827600 Japanese Patent No. 4196973 Japanese Patent No. 3060922 Japanese Patent No. 3480563

  As described above, for example, in a wide area monitoring system that targets an area where a large number of people pass, image recognition processing technology for tracking and identifying a person based on characteristics related to the person obtained from image information is available. Required. However, human characteristics vary greatly due to changes in environmental conditions such as the distance from the camera to the person and the orientation of the person with respect to the camera. The prior arts of Patent Documents 1 and 2 described above are difficult to adapt to situations in which environmental conditions vary. In addition, the prior arts of Patent Documents 3 and 4 require prior learning for a recognition target, and are difficult to apply to a wide-area monitoring system in which feature information cannot be learned in advance for each person.

  Accordingly, an object of the present invention is to provide a feature information selection device capable of selecting feature information effective for image recognition processing capable of reliably realizing tracking and identification of a person even when environmental conditions vary.

  An aspect of the present invention is effective for image recognition processing in which feature information used for tracking and identification of a person using image processing is defined by two attributes of importance and reliability, and the tracking and identification of a person can be realized with certainty. This is a feature information selection device that selects feature information.

  A feature information selection device according to an aspect of the present invention extracts a plurality of feature information of a recognition target in a feature information selection device applied to an image recognition process for extracting a recognition target from image information captured by a camera. Defining the importance indicating the feature deviating from the average feature as the attribute included in the feature information, and setting the importance level information indicating the importance level of the feature information, A reliability setting unit that defines reliability set based on a shooting environment of the camera as an attribute included in the feature information, and sets reliability information indicating the reliability of the feature information; and Selection means for selecting feature information effective for the image recognition processing from the plurality of feature information based on the importance information and the reliability information when extracting a recognition target based on It is configured to include.

  According to the present invention, it is possible to select feature information effective for image recognition processing that can reliably realize tracking and identification of a person even when environmental conditions vary.

The block diagram which shows the principal part of the image processing apparatus and monitoring system regarding embodiment of this invention. The figure for demonstrating an example of the mechanism which identifies a person in the image processing apparatus regarding this embodiment. 6 is a flowchart for explaining tracking and identification processing of the image processing apparatus according to the embodiment. The figure for demonstrating the definition of the importance regarding this embodiment. The figure for demonstrating the definition of the importance regarding this embodiment. The flowchart for demonstrating the setting process of the importance regarding this embodiment. The flowchart for demonstrating the setting process of the reliability regarding this embodiment. The figure for demonstrating the environmental fluctuation | variation regarding this embodiment. The figure for demonstrating the setting method of the reliability regarding this embodiment. The figure for demonstrating the identification process regarding this embodiment. The figure for demonstrating the method of the feature information selection regarding this embodiment. The flowchart for demonstrating the identification process regarding this embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

(Configuration of image processing apparatus)
FIG. 1 is a block diagram illustrating a main part of the image processing apparatus according to the present embodiment.

  The present embodiment is an image processing apparatus 10 applied to a monitoring system that uses a plurality of cameras 11A to 11C. The image processing apparatus 10 includes an image input unit 12, an image processing unit 13, a storage unit 14, and a display device 15, and specifically includes a computer system such as a personal computer.

  The image input unit 12 inputs images (videos) taken by the cameras 11 </ b> A to 11 </ b> C, executes various preprocessing, and sends them to the image processing unit 13. The image processing unit 13 includes a microprocessor (CPU) and image processing software, and executes image processing including image recognition processing described later. The storage unit 14 stores processing results of the image processing unit 13 and information necessary for image processing of the image processing unit 13. The display device 15 has a display screen that displays the processing result from the image processing unit 13. The display device 15 constitutes a monitoring monitor that displays images taken by the cameras 11A to 11C.

(Operation of monitoring system)
As shown in FIG. 1, the monitoring system according to the present embodiment includes cameras 11 </ b> A to 11 </ b> C and an image processing device 10 installed in a large-scale store, for example. The cameras 11A to 11C are arranged for each of the monitoring areas 110A to 110C assigned to the store floor 100. The image processing apparatus 10 extracts a person 120 from each image taken by different cameras 11A to 11C. Further, the image processing apparatus 10 performs image recognition processing for tracking and identifying the person 120 moving on the store floor 100 as indicated by a dotted line 130 based on the feature information of the person 120.

  FIG. 2 is a diagram illustrating an example of a mechanism in which the image processing apparatus 10 identifies a person 120 using image information from different cameras 11A to 11C. As shown in FIG. 2A, the image processing apparatus 10 extracts a person in the back from an image 200A taken by the camera 11A. Further, as shown in FIG. 2B, the image processing apparatus 10 extracts a person from the front from an image 200B photographed by the camera 11B.

  Furthermore, the image processing apparatus 10 detects, for example, orange clothes and gray pants as extracted person characteristic information from the image 200A captured by the camera 11A. Further, the image processing apparatus 10 detects, for example, orange clothes and gray pants as extracted person characteristic information from the image 200B taken by the camera 11B.

  The image processing apparatus 10 identifies the same person as the extracted person's orange clothes and gray pants as the feature information acquired from the image information of the different cameras 11A and 11B. Therefore, the image processing apparatus 10 can identify and track a person moving between the monitoring ranges (shooting ranges) of the different cameras 11A and 11B.

  Next, image processing including image recognition processing for tracking and identifying the image processing apparatus 10 will be described with reference to the flowchart of FIG.

  First, the image input unit 12 inputs images taken by the two cameras 11A and 11B here, executes various pre-processes, and sends them to the image processing unit 13. The image processing unit 13 executes a process of cutting out a human region (human region) from each image information from the image input unit 12 (steps S1A and S1B). The image processing unit 13 extracts the feature amount (feature information) of the cut-out human region (Steps S2A and S2B).

  Next, the image processing unit 13 determines the degree of similarity between the extracted feature quantities (feature information) (step S3). When the similarity is higher than the reference value, the image processing unit 13 outputs a processing result for identifying that the human regions photographed by the two cameras 11A and 11B are the same person. Accordingly, the monitoring system can execute a monitoring process for identifying and tracking a person who is moving between the monitoring ranges 110A and 110B of the cameras 11A and 11B based on the image processing result of the image processing unit 13. (Step S4).

(Select feature information)
As described above, in the wide area monitoring system, the image processing apparatus 10 performs image processing on images captured by the plurality of cameras 11A to 11C, and acquires a plurality of feature information about a person from the image processing result. The image processing apparatus 10 can identify and track a person to be monitored by selecting effective feature information from a plurality of acquired feature information.

  However, in reality, when tracking and identifying a person, the monitoring performance tends to deteriorate due to changes in environmental conditions (environmental parameters) such as the distance from the camera to the person and the orientation of the person with respect to the camera. . Specifically, the environmental condition is an environmental condition for photographing with the cameras 11A to 11C. This embodiment realizes an effective feature information selection mechanism for realizing high-performance tracking and identification even when there is a change in environmental conditions. Hereinafter, the feature information selection mechanism will be described in detail.

  In the present embodiment, two types of attributes of importance and reliability are defined as effective feature information (feature amount) when tracking and identifying a person using image processing. The importance is numerical information indicating the degree of peculiarity of feature information when the feature information is compared with the average. The reliability is information indicating a degree of reliability when certain feature information is compared with other feature information.

  In a monitoring operation for tracking and identifying a person, when there are many persons having characteristics similar to the monitoring range, it becomes difficult to identify the monitoring target, and thus the monitoring performance deteriorates. In such a case, it is possible to suppress a decrease in monitoring performance by selecting unique information from among a plurality of feature information about a person and placing importance on the information.

  As a method for determining the specificity of feature information, as shown in FIG. 4, a standard distribution of feature amounts is prepared in advance, and if the obtained feature amount is a value 410 that deviates from the average value 400, the specificity is high. Judge that there is, and set the importance high. As another method, as shown in FIG. 5, when a standard space 500 of feature quantities is prepared in advance and there is a deviated value 510 where the distance from the obtained feature quantities to the space is different, the specificity is unique. It is judged that there is, and the importance is set high. In FIG. 5, reference numeral 520 denotes a hyperplane.

  FIG. 6 is a flowchart for explaining processing for setting the importance of feature information in the image processing unit 13.

  First, the image processing unit 13 takes in images captured by the cameras 11A to 11C via the image input unit 12 (step S11). The image processing unit 13 cuts out a human area from each image information from the image input unit 12 (step S12). The image processing unit 13 extracts a plurality of feature information (feature amounts) of the cut-out human region (step S13).

  Next, the image processing unit 13 determines the specificity for each of the extracted pieces of feature information (step S14). Based on the determination result of the specificity, the image processing unit 13 sets an importance that is a numerical value for each feature information, and stores and registers the feature information (feature data) to which the importance is added in the storage device 14. (Step S16).

  Next, as an attribute of the feature information (feature amount), the reliability indicating the level of reliability of the feature information will be specifically described with reference to FIGS.

  When an environmental change of the monitoring system occurs, there is information that does not include information indicating a difference between people among a plurality of feature information. Moreover, feature information with high reliability and feature information with low reliability may change depending on the contents of environmental changes.

  FIG. 8 is a diagram illustrating rules of a variation detection method for detecting environmental parameters, types of variations thereof, and the occurrence of environmental variations. As shown in FIG. 8, the environmental parameters (camera shooting conditions) include, for example, illuminance, contrast, video (image) color, camera depression angle, camera lens focal length, camera and humanity when shooting with the camera. There are distances. In addition, types of fluctuation include reduction, fluctuation, increase, and change.

  As shown in FIG. 7, the image processing unit 13 cuts out a human region from each piece of image information from the image input unit 12, and extracts a plurality of feature information (feature amounts) of the human region (step S21). The image processing unit 13 sets the level of reliability for the extracted plurality of feature information (step S22). Here, the reliability is set by a method of setting the level of reliability for a plurality of feature information according to the rules shown in FIG. In this case, there are three types of feature information indicating the human feature quantity: luminance distribution information, luminance contour information, and color distribution information.

  Next, the image processing unit 13 detects the occurrence of a change in the environmental parameter by a change detection method as shown in FIG. 8 (step S23). When detecting a change in the environmental parameter, the image processing unit 13 executes a process of changing the reliability of the feature information in conjunction with the change (YES in step S24, S25). Specifically, among the environmental parameters, for example, when a decrease in illuminance during camera shooting occurs, among the three types of feature information, the reliability of the luminance contour information is relatively increased, and the luminance distribution The information and color distribution information are changed and set so that the reliability is relatively low.

  Specific method for tracking and identifying a person with reference to FIGS. 10 to 12 based on importance and reliability information defined and set as two types of attributes of feature information (feature amount) as described above Will be explained.

  When the image processing unit 13 tracks and identifies a person between different images taken by different cameras, as shown in FIG. 10, the image processing unit 13 compares images of people having different apparent sizes and orientations with respect to the camera. It is determined whether or not the same person is the same. FIG. 10A shows an image 600A of a large person when the shooting conditions of the camera are relatively good. On the other hand, FIG. 10B shows an image 600B of a small person when the shooting conditions of the camera are relatively poor. That is, as shown in FIGS. 10A and 10B, the image processing unit 13 compares images having different environmental conditions, for example, illuminance and contrast, and executes identification processing for determining whether or not they are the same person. To do.

  Here, when tracking and identifying a person between different images, it is highly possible that the feature information of a person assumed to be effective in one image and the feature information assumed to be effective in the other image are different. Since person identification and tracking are performed using the same feature information between two images, feature information having different importance and reliability settings is used, and monitoring performance is likely to deteriorate.

  Therefore, the image processing apparatus 10 according to the present embodiment selects a plurality of pieces of feature information based on the set feature information selection criteria, and executes a human tracking and identification process by using them in combination. The feature information selection criteria are when selecting feature information with high reliability and selecting feature information with high importance on the basis of an image with poorer shooting conditions among two images.

  FIGS. 11A and 11B are diagrams illustrating specific examples of feature information (feature amount) selection when tracking and identifying a person between different images. 11A and 11B, reference numeral 700 indicates selected feature information. Reference numeral 710 indicates that feature information having a high degree of importance has been selected. Reference numeral 720 indicates that the main feature information of an image with poor shooting conditions is selected as a reference.

  FIGS. 12A and 12B are flowcharts for explaining the process of tracking and identifying a person using feature information having different selection criteria in patterns A and B. FIG.

  As shown in FIG. 12A, as the pattern A, the image processing apparatus 10 executes a process of selecting an identification candidate person based on feature information having a high importance level, and a feature having high reliability from the identification candidate. A process of identifying a person based on the information is executed. On the other hand, as shown in FIG. 12B, as the pattern B, the image processing apparatus 10 executes a process of selecting an identification candidate person based on feature information having a high degree of importance. A process of identifying a person based on high feature information is executed.

  In short, according to the present embodiment, two types of attributes, importance and reliability, are defined as effective feature information (feature amount) when tracking and identifying a person using image recognition processing. The setting of the attribute is dynamically changed according to environmental changes such as shooting conditions. Here, the importance is, in other words, a feature that deviates from the average feature, and is a time series such as individual information such as color, outline (body shape), gender, and how to walk (gait or gait). It corresponds to information. For example, among people who go in and out of a large-scale store building such as a department store, the weight is given to the characteristics of a person such as a person wearing black clothes in summer or a man in a place where a woman goes in and out. In other words, the reliability is reliability information that is set based on environmental conditions in photographing by the camera.

  With such an image recognition process of the present embodiment, it is possible to reliably track and identify a person even when there is a change in environmental conditions. Therefore, the present invention can be effectively applied to a wide area monitoring system that uses a plurality of cameras.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 11A-11C ... Camera, 12 ... Image input part,
13 ... Image processing unit, 14 ... Storage device, 15 ... Display device,
100 ... Store floor, 110A to 110C ... Monitoring area (shooting range).

Claims (13)

In a feature information selection device applied to image recognition processing for extracting a recognition target from image information photographed by a camera,
Feature extraction means for extracting a plurality of feature information of the recognition target;
An importance setting means for defining importance indicating a feature deviating from an average feature as an attribute included in the feature information, and setting importance information indicating the importance of the feature information;
A reliability setting unit that defines reliability set based on a shooting environment of the camera as an attribute included in the feature information, and sets reliability information indicating the reliability of the feature information;
Selecting means for selecting feature information effective for the image recognition process from the plurality of feature information based on the importance information and the reliability information when extracting a recognition target based on the feature information; A feature information selection device characterized by that. The importance information is numerical information indicating the degree of peculiarity of the feature information when compared with the average,
The feature information selection device according to claim 1, wherein the reliability information is information indicating a degree of reliability of the feature information when compared with other feature information.   The importance setting means is configured to determine specificity for each feature information, and to calculate and set the importance information for each feature information based on the determination result. The feature information selection device according to claim 1 or 2.   The reliability setting means is configured to determine the level of reliability for each feature information and to set the reliability information for each feature information based on the determination result. The feature information selection device according to any one of claims 1 to 3. The reliability setting means includes:
5. The feature information selection apparatus according to claim 4, further comprising means for changing the reliability information set for each feature information in accordance with a change in the photographing environment. The feature extraction means includes
6. The feature information selection apparatus according to claim 1, wherein brightness distribution information, brightness contour information, and color distribution information are extracted as the plurality of feature information.   7. The photographing environment according to claim 1, wherein the photographing environment includes a plurality of parameters relating to illuminance, contrast, image color, camera depression angle, lens focal length, and camera-to-person distance. The feature information selection device according to item. It has the feature information selection device according to any one of claims 1 to 7,
Image input means for inputting image information photographed by the camera;
An image recognition processing apparatus comprising: an image recognition processing unit that executes an image recognition process for extracting a recognition target based on the feature information selected by the selection unit. A method of selecting feature information to be applied to image recognition processing for extracting a recognition target from image information captured by a camera,
A process of extracting a plurality of feature information of the recognition target;
A process of defining importance indicating a feature deviating from an average feature as an attribute included in the feature information, and setting importance information indicating the importance of the feature information;
Defining reliability set based on the shooting environment of the camera as an attribute included in the feature information, and setting reliability information indicating the reliability of the feature information;
When extracting a recognition target based on the feature information, executing a process of selecting feature information effective for the image recognition process from the plurality of feature information based on the importance information and the reliability information Feature information selection method characterized by It has the feature information selection method according to claim 9,
Processing for inputting image information taken by the camera;
An image recognition processing method that executes image processing for executing image recognition processing for extracting the recognition target based on the selected feature information. A program for selecting feature information to be applied to image recognition processing for extracting a recognition target from image information photographed by a camera,
A process of extracting a plurality of feature information of the recognition target;
A process of defining importance indicating a feature deviating from an average feature as an attribute included in the feature information, and setting importance information indicating the importance of the feature information;
Defining reliability set based on the shooting environment of the camera as an attribute included in the feature information, and setting reliability information indicating the reliability of the feature information;
A process of selecting feature information effective for the image recognition processing from the plurality of feature information based on the importance information and the reliability information when extracting a recognition target based on the feature information. A program to be executed by a computer.   The program according to claim 11, further causing the computer to execute a procedure of an image recognition process for executing an image recognition process for extracting the recognition target based on the selected feature information. Multiple cameras,
Means for inputting image information taken by each camera;
Means for extracting a human region from the image information;
Means for extracting a plurality of feature information of the human area;
Means for defining importance indicating a feature deviating from an average feature as an attribute included in the feature information, and setting importance information indicating the importance of the feature information;
Means for defining reliability set based on a shooting environment of the camera as an attribute included in the feature information, and setting reliability information indicating the reliability of the feature information;
Means for selecting feature information effective for image recognition processing from the plurality of feature information based on the importance information and the reliability information when performing tracking and identification of a person based on the feature information;
A monitoring system comprising: an image recognition process for executing tracking and identification of a person based on the selected feature information.

Images