WO2012098842A1 - 特徴抽出装置、特徴抽出方法、特徴抽出プログラム、および、画像処理装置 - Google Patents
特徴抽出装置、特徴抽出方法、特徴抽出プログラム、および、画像処理装置 Download PDFInfo
- Publication number
- WO2012098842A1 WO2012098842A1 PCT/JP2012/000175 JP2012000175W WO2012098842A1 WO 2012098842 A1 WO2012098842 A1 WO 2012098842A1 JP 2012000175 W JP2012000175 W JP 2012000175W WO 2012098842 A1 WO2012098842 A1 WO 2012098842A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- feature extraction
- local binary
- binary pattern
- image
- histogram
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/36—Applying a local operator, i.e. means to operate on image points situated in the vicinity of a given point; Non-linear local filtering operations, e.g. median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]
Definitions
- the present invention relates to a feature extraction device, a feature extraction method, a feature extraction program, and an image processing device using the feature extraction device that extract image features from image data.
- Non-Patent Document 1 Non-Patent Document 1, for example (hereinafter referred to as “conventional technology”).
- the local binary pattern is a binary pattern obtained by binarizing and arranging the luminance difference from each pixel in the vicinity of each pixel. According to the local binary pattern, the grayscale pattern included in the image can be extracted. .
- the conventional technique calculates a local binary pattern for all pixels or partial pixels in a certain area of an image to be identified (hereinafter referred to as “target image”).
- the conventional technique generates a histogram of local binary pattern values as image features.
- a discriminator is generated and stored in advance based on a histogram similarly generated from an image including a predetermined object and an image not including the predetermined object (hereinafter collectively referred to as “learning image”). Then, the conventional technique evaluates the histogram of the target image using a discriminator, and determines whether or not a predetermined object is included in the target image.
- the local binary pattern histogram can express texture differences and shading patterns with higher precision than image features such as Histograms of Oriented Gradients (HOG).
- the local binary pattern histogram can be calculated with a smaller processing load than image features such as a luminance gradient direction histogram. Therefore, the object detection using the local binary pattern as in the prior art is expected to be applied to various fields.
- the conventional technique has a problem that object detection cannot be performed with high accuracy unless the noise level of the captured image is substantially the same in all of the learning image and the target image. That is, the conventional technique has a problem that object detection cannot be performed with high accuracy unless the shooting environment is similar in both the learning image and the target image. This is because even when the same object is copied, the local binary pattern may differ due to different noise levels, and such a local binary pattern difference may occur in the entire image.
- An object of the present invention is to extract an image feature capable of detecting an object robust against variations in shooting environments while using a local binary pattern, a feature extraction device, a feature extraction method, a feature extraction program, and an image It is to provide a processing device.
- the feature extraction apparatus generates a local binary pattern that indicates, by a bit value, whether or not a difference in pixel value from surrounding neighboring pixels is greater than or equal to a predetermined threshold value for every pixel of all or part of the image.
- a histogram generation unit that generates a histogram indicating the distribution of the local binary pattern generated from the image.
- An image processing apparatus uses an identifier for identifying a predetermined object, and determines whether or not the image includes the predetermined object from the histogram generated by the feature extraction apparatus. Part.
- the feature extraction method of the present invention generates a local binary pattern for each pixel of all or a part of an image, which indicates whether or not the difference in pixel value from surrounding neighboring pixels is greater than or equal to a predetermined threshold value by each bit value. For each of the generated local binary patterns, determining a weight according to a difference between the original pixel values, applying the determined weight to the corresponding local binary pattern, and Generating a histogram showing the distribution of the local binary pattern generated from
- the feature extraction program of the present invention allows a computer to display a local binary that indicates whether or not a pixel value difference with surrounding neighboring pixels is greater than or equal to a predetermined threshold value for each pixel of all or part of an image.
- a process for generating a pattern a process for determining a weight according to a difference between the original pixel values for each generated local binary pattern, and applying the determined weight to the corresponding local binary pattern And generating a histogram indicating the distribution of the local binary pattern generated from the image.
- the present invention it is possible to extract, as an image feature of an image, a histogram that can detect an object robustly against variations in shooting environments while using a local binary pattern.
- FIG. 1 is a system configuration diagram showing the configuration of an object detection system including a feature extraction device according to an embodiment of the present invention.
- the block diagram which shows the detailed structure of the feature extraction part in this Embodiment Flowchart showing the operation of the object detection apparatus according to the present embodiment
- the figure which shows an example of the histogram of the target image in this Embodiment The figure which shows an example of the histogram produced
- FIG. 1 is a system configuration diagram showing a configuration of an object detection system including a feature extraction device according to an embodiment of the present invention.
- FIG. 1 also illustrates the configuration of each device.
- the object detection system 100 includes a classifier learning device 200, a classifier storage device 300, and an object detection device 400.
- the discriminator learning device 200 and the object detection device 400 can be connected to the discriminator storage device 300 via a communication network (not shown) such as the Internet, for example.
- Boosting is used as a machine learning method.
- the discriminator learning device 200 learns a discriminator for detecting an object to be detected (hereinafter referred to as “detection target object”) from a prepared learning image, and discriminates the discriminator as a learning result.
- the data is stored in the storage device 300.
- the discriminator learning device 200 includes a learning data storage unit 210, a feature extraction region acquisition unit 220, a feature extraction unit 230 including the feature extraction device according to the present invention, and a learning unit 240.
- the learning data storage unit 210 stores in advance a plurality of learning images (positive samples) including a detection target object and a plurality of learning images (negative samples) including no detection target object.
- the feature extraction region acquisition unit 220 acquires a feature extraction region for each learning image stored in the learning data storage unit 210, and outputs the acquired feature extraction region to the feature extraction unit 230.
- the feature extraction area is an image area from which image features are extracted.
- the feature extraction area can be a large number of randomly arranged image areas including an image area such as a human face part, for example, an eye or a nose.
- the feature extraction region can be a large number of randomly arranged image regions including image regions such as the head, arms, and feet.
- the feature extraction unit 230 extracts the image feature for each feature extraction region and outputs it to the learning unit 240. More specifically, the feature extraction unit 230 first generates a local binary pattern for every pixel of all or part of the feature extraction region. Then, the feature extraction unit 230 generates a histogram indicating the distribution of the generated local binary pattern (hereinafter simply referred to as “histogram”) as an image feature of the feature extraction region.
- histogram indicating the distribution of the generated local binary pattern
- the local binary pattern is information indicating by a bit value whether or not the difference in pixel value between the pixel of interest and surrounding pixels around it is equal to or greater than a predetermined threshold value.
- the feature extraction unit 230 weights the local binary pattern according to the difference between the base pixel values when generating the histogram.
- the learning unit 240 generates one or a plurality of classifiers.
- the one or more discriminators discriminate between an image including the detection target object and an image not including the image based on the histogram group obtained from the positive sample and the histogram group obtained from the negative sample. belongs to. That is, the learning unit 240 generates feature extraction region information and identification information corresponding to the feature extraction region information as a discriminator. Then, the learning unit 240 transmits the generated identification information together with the feature extraction region information to the classifier storage device 300, and stores the identification information in combination with the feature extraction region information.
- Feature extraction area information is information indicating the range of the feature extraction area, and includes, for example, the position and size of the feature extraction area.
- the identification information is information for evaluating the histogram of the feature extraction area of the target image and determining whether or not the target image includes a predetermined object.
- the object detection device 400 acquires the classifier stored in the classifier storage device 300 and performs object detection on the target image.
- the object detection device 400 includes a camera 410, an image input unit 420, a feature extraction region acquisition unit 430, a feature extraction unit 440 including the feature extraction device according to the present invention, and an identification unit 450.
- the camera 410 captures the target image and outputs the target image to the image input unit 420.
- the image input unit 420 scans the target image with a window having a predetermined size, and outputs each scanned image region (hereinafter referred to as “window region”) to the feature extraction region acquisition unit 430.
- the feature extraction area acquisition unit 430 acquires, for each window area, a range indicated by the feature extraction area information stored in the discriminator storage device as the feature extraction area. Then, the feature extraction region acquisition unit 430 outputs the acquired feature extraction region to the feature extraction unit 440.
- the process performed by the feature extraction area acquisition unit 430 on the window area is similar to the process performed by the feature extraction area acquisition unit 220 on the learning image. That is, the feature extraction area acquisition unit 220 acquires, for example, all of many randomly arranged areas as the feature extraction area.
- the feature extraction region acquisition unit 430 acquires only the region indicated by the feature extraction region information already selected by the learning unit 240 from the classifier storage device 300 as the feature extraction region.
- the description of one configuration and operation will be appropriately omitted below, and the description of the other configuration and description will be omitted.
- the feature extraction unit 440 extracts the image feature for each feature extraction region and outputs it to the identification unit 450. More specifically, the feature extraction unit 440 generates a local binary pattern for each pixel of the feature extraction region, and generates a histogram of the local binary pattern as an image feature of the feature extraction region.
- the feature extraction unit 440 weights the local binary pattern according to the difference between the base pixel values. In other words, the feature extraction unit 440 generates a histogram in consideration of the magnitude of each local binary pattern.
- the identification unit 450 acquires identification information stored in the classifier storage device 300.
- the identification unit 450 uses the acquired identification information to determine whether or not the detection target object is included in the target image from the histogram generated from the window area scanned by the image input unit 420. Then, the identification unit 450 notifies the user of the determination result via, for example, an image display device or a sound output device (not shown).
- FIG. 2 is a block diagram showing a detailed configuration of the feature extraction unit 440. As shown in FIG.
- the feature extraction unit 440 includes a pixel neighborhood acquisition unit 441, a neighborhood difference calculation unit 442, a binary pattern generation unit 443, a weight generation unit 444, and a histogram generation unit 445.
- the pixel neighborhood acquisition unit 441 shifts the target pixel with respect to the feature extraction region.
- the pixel neighborhood acquisition unit 441 acquires, for each target pixel, nine pixel values in a 3 ⁇ 3 region centered on the target pixel (hereinafter referred to as “neighboring region”). Then, the pixel neighborhood acquisition unit 441 outputs the obtained 9 pixel values for each neighborhood region to the neighborhood difference calculation unit 442.
- the pixel value for example, a value obtained by primary differentiation or secondary differentiation of a luminance value obtained by performing preprocessing such as edge enhancement on an image, or a color expressing luminance of red, blue, and green as one vector value Various values such as a value can be adopted. In the present embodiment, it is assumed that the pixel value is a luminance value.
- the neighborhood difference calculation unit 442 calculates, for each neighborhood region, a difference in luminance value between the pixel of interest and each surrounding neighborhood pixel, and obtains eight calculation results (hereinafter referred to as “neighbor differences”). Then, the neighborhood difference calculation unit 442 outputs the neighborhood difference of each neighborhood region to the binary pattern generation unit 443 and the weight generation unit 444.
- the binary pattern generation unit 443 generates a code in which a bit value indicating whether each neighborhood difference is equal to or greater than a predetermined threshold is arranged in a predetermined order for each neighborhood region. Then, the binary pattern generation unit 443 outputs the generated code to the histogram generation unit 445 as a local binary pattern.
- the weight generation unit 444 determines the sum of the absolute values of the eight neighborhood differences for each neighborhood region as a weight for the corresponding local binary pattern, and outputs it to the histogram generation unit 445.
- the histogram generation unit 445 generates a local binary pattern histogram for each feature extraction region and outputs the histogram to the identification unit 450 of FIG.
- the histogram generation unit 445 adds the weight input corresponding to the local binary pattern to the bin value corresponding to the local binary pattern.
- the discriminator learning device 200 and the object detection device 400 can take the configuration of a computer including a storage medium such as a CPU (central processing unit) and a RAM (random access memory), respectively.
- the discriminator learning device 200 and the object detection device 400 each operate when the CPU executes a control program to be stored.
- the object detection device 400 may be a dedicated chip that performs only the calculation.
- the classifier storage device 300 is a network server including a storage medium such as a semiconductor memory or a hard disk.
- the frequency of the local binary pattern in the vicinity region where the luminance difference (contrast) is large is relatively high. That is, the frequency of the local binary pattern in the neighboring area that clearly represents the gray pattern of the subject is relatively high. The frequency corresponding to an erroneous binary pattern caused by noise is relatively low.
- the object detection system 100 is an image feature using a local binary pattern when detecting an object with respect to an image, and an image feature that can detect an object robust to variations in shooting environments. Can be used.
- the operation of the object detection device 400 will be described. Note that the operation of the feature extraction unit 230 in the classifier learning device 200 is the same as the operation of the feature extraction unit 440 in the object detection device 400, and thus description thereof is omitted.
- FIG. 3 is a flowchart showing the operation of the object detection apparatus 400.
- step S1100 the image input unit 420 scans the target image with a window having a predetermined size.
- FIG. 4 is a schematic diagram showing an example of a state of image scanning.
- the image input unit 420 scans the entire target image 510 with a window 511 having a predetermined size, and acquires window regions 512 from various positions. Depending on the position of the window 511, the detection target object 513 is included in the window region 512 as shown in FIG.
- the size of the window 511 is, for example, 64 pixels ⁇ 128 pixels.
- the feature extraction region acquisition unit 430 acquires one piece of feature extraction region information (position, size, etc.) obtained by learning of the discriminator learning device 200 and stored in the discriminator storage device. .
- the feature extraction region acquisition unit 430 acquires a rectangular region including a human head as the feature extraction region 520.
- step S1300 the pixel neighborhood acquisition unit 441 of the feature extraction unit 440 selects one pixel in the feature extraction region as a pixel of interest, and determines a neighborhood region centered on the pixel of interest.
- the pixel neighborhood acquisition unit 441 selects an unselected pixel every time the process returns to step S1300 by a determination process in step S1800 described later, and as a result, shifts the neighborhood region.
- FIG. 5 is a schematic diagram showing an example of a state of shifting in the vicinity area.
- the pixel neighborhood acquisition unit 441 shifts the pixel position to be selected as the target pixel 521 in the entire feature extraction region 520.
- a 3 ⁇ 3 neighborhood region 523 including the target pixel 521 and the eight neighborhood pixels 522 surrounding the pixel of interest 521 is sequentially determined for the entire feature extraction region 520.
- the neighborhood difference calculation unit 442 of the feature extraction unit 440 calculates a difference (neighbor difference) between the luminance value of the target pixel and each of the luminance values of the eight neighboring pixels.
- step S1500 the binary pattern generation unit 443 of the feature extraction unit 440 binarizes the neighborhood difference to generate a local binary pattern.
- the luminance value of the pixel of interest is g c
- the number of neighboring pixels (8 in this embodiment) is P
- the order of neighboring pixels is p
- the luminance value of the p-th neighboring pixel is g p
- the radius of the neighboring region Let R be the number of pixels corresponding to (1 in this embodiment).
- the local binary patterns LBP P and R are represented by the following formula (1), for example. Note that the coordinates of the p-th neighboring pixel are expressed as [Rcos (2 ⁇ p / P), Rsin (2 ⁇ p / P)].
- step S1600 the weight generation unit 444 of the feature extraction unit 440 calculates the sum of the absolute values of the eight neighboring differences as a weight.
- the weights WP and R are expressed by, for example, the following formula (2).
- step S1700 the histogram generation unit 445 adds the weights WP and R to the bins of the local binary patterns LBP P and R among the bins constituting the histogram.
- step S1800 the pixel neighborhood acquisition unit 441 determines whether there is an unprocessed pixel in the feature extraction area.
- step S1800: YES If there is an unprocessed pixel (S1800: YES), the pixel neighborhood acquisition unit 441 returns to step S1300, selects the unprocessed pixel, and repeats the process. If the pixel neighborhood acquisition unit 441 completes the process for all the feature extraction regions (S1800: NO), the process proceeds to step S1900.
- FIG. 6 is a schematic diagram showing an outline of an example of processing until a local binary pattern in a certain neighboring area is reflected in the histogram.
- the luminance value g c of the target pixel is set to “128”, and the luminance value g p of the neighboring pixel is set to [244, 178, 48, 27, 253, 208, 238, 28] is assumed to be acquired.
- the luminance value g p shall be obtained from the upper left neighboring pixel clockwise.
- the numerical value group 533 of the neighborhood difference (g p ⁇ g c ) is [116, 50, ⁇ 80, ⁇ 101, 125, 80, 110, ⁇ 100].
- the binarization threshold is “0”
- the numerical value group 534 obtained by binarizing the neighborhood difference is [1, 1, 0, 0, 1, 1, 1, 0].
- the local binary pattern 535 is “11001110” (“206” in decimal).
- the calculation 536 of the weight W is a calculation for adding the absolute values
- the histogram H (k) is expressed by the following equation (3), where K is the maximum number of local binary patterns obtained from a feature extraction region having a size of I ⁇ J.
- the histogram generation unit 445 normalizes the histogram to feature amounts that do not depend on the size of the feature extraction region. Specifically, the histogram generation unit 445 normalizes the histogram with, for example, the sum of the frequencies of all bins of the histogram.
- FIG. 7 is a diagram illustrating an example of how the histogram is normalized.
- the frequencies H 1 , H 2 ,..., H 8 of the bins before normalization are “5, 6, 4, 7, 10 , 5 , 8 , 3” in order.
- the total sum SumH i is calculated as in the following equation (6).
- FIG. 8 is a diagram showing an example of a histogram generated from the target image.
- the histogram 540 has a peak (for example, indicated by a portion 541) in the bin of the local binary pattern extracted more from the image.
- the shape of the histogram 540 representing the local binary pattern distribution varies depending on the object included in the image. Therefore, the object detection system 100 uses where the peak of the histogram 540 appears and how large the peak is as an image feature for identifying the detection target object.
- the histogram 540 having a larger difference between the peak and the frequency of another part expresses the image feature more strongly. When such an image feature is used, object detection is performed with high accuracy. It can be performed.
- step S2000 the identification unit 450 calculates the noise level (hereinafter referred to as “region noise level”) of the feature extraction region of the target image based on the normalized histogram.
- the identification unit 450 has a high noise level of the target image. Is determined.
- the bin corresponding to such noise is specifically the bin corresponding to the local binary pattern when all the neighborhood differences are positive values, that is, the bin corresponding to “255” in decimal.
- the bin corresponding to noise is a bin corresponding to a local binary pattern when all neighboring differences are negative values, that is, a bin corresponding to “0” in decimal.
- FIG. 9 is a diagram showing an example of a histogram that is generated by the prior art and is not weighted, and corresponds to FIG.
- the difference between the peak indicated by the portion 541 and the other portion indicated by the portion 542 and the like is smaller than that in FIG. This is because the degree of contribution of information with low reliability to the histogram 540 is high.
- Such a histogram 540 can represent the presence or absence of an object with high accuracy.
- the conventional technique increases the frequency of bins of the local binary pattern of decimal number “0” or “255” based on pixels that are very likely to indicate noise.
- the bin value of the decimal number “0” or “255” has a relative value even though the actual noise level is the same. It will be high. If the object detection is performed based on the bin frequency of decimal number “0” or “255”, the determination accuracy is lowered.
- the object detection system 100 can determine the noise level with higher accuracy than the conventional technique.
- the identification unit 450 generates a histogram to which the conventional technique is applied, and determines the noise level of the target image based on the comparison of the “0” bin and the comparison of the “255” bin. May be. That is, in this case, for example, the identification unit 450 compares the part 543 in FIG. 8 with the part 543 in FIG. 9 and compares the part 544 in FIG. 8 with the part 544 in FIG. 9.
- step S 2100 the identification unit 450 acquires a classifier from the classifier storage device 300. Then, the identification unit 450 uses the acquired classifier, and based on the normalized histogram, a score indicating the likelihood for determining whether or not the detection target object is included in the current feature extraction region. calculate.
- step S2200 the object detection apparatus 400 cumulatively adds the calculated scores.
- step S2300 the feature extraction region acquisition unit 430 determines whether there is an unprocessed feature extraction region. That is, the feature extraction region acquisition unit 430 determines whether scores have been calculated for all feature extraction regions corresponding to the feature extraction region information stored in the classifier storage device 300.
- the feature extraction region acquisition unit 430 returns to step S1200, and proceeds to processing on the unprocessed feature extraction region. Also, the feature extraction region acquisition unit 430 proceeds to step S2400 when the processing is completed for all the feature extraction regions (S2300: NO).
- step S2400 the noise level of the scanned window is determined using the region noise levels of all feature extraction regions.
- step S2500 the identification unit 450 determines whether a detection target object is included for each scanned window based on the cumulatively added score value. That is, the identification unit 450 determines that the detection target object is included in the window when the score is equal to or greater than a predetermined threshold. Note that the identification unit 450 may make this determination based on a result of applying a predetermined function to the score.
- step S2600 the identification unit 450 outputs a determination result of object detection.
- the identification unit 450 may determine whether the designated object is included in the target image, or may determine what object is included in the target image. In the former case, the discriminating unit 450 may output information indicating only whether or not the object is included, using only the discriminator corresponding to the designated object, for example. In the latter case, the discriminator 450 may apply the plurality of discriminators sequentially and repeat the determination, and when an object is detected, output information indicating which object has been detected.
- step S2700 the image input unit 420 determines whether or not an instruction to end the process is given by an operation or the like.
- the image input unit 420 returns to step S1100 and proceeds to the process for the next scan or the next target image. In addition, when instructed to end the process (S2700: YES), the image input unit 420 ends the series of processes.
- the object detection device 400 can perform weighting according to the neighborhood difference, generate a histogram of a local binary pattern, and perform object detection using this as an image feature.
- the discriminator learning device 200 similarly performs weighting according to the neighborhood difference, generates a histogram of the local binary pattern, and uses this as an image feature for object detection. Generate a classifier.
- the histogram weighted according to the neighborhood difference is an image feature that enables object detection that is robust against variations in the shooting environment.
- FIG. 10 is a schematic diagram showing an outline of an example of a histogram generation process for a local binary pattern acquired from an image with low contrast in the object detection system 100, and corresponds to FIG.
- the luminance value g c of the pixel of interest in a certain neighboring region 551 is set to “128”, and the luminance value g p of the neighboring pixel is set to [133, 148, 115, 113, 159, 166, 202, 112 ] Is obtained.
- the numerical value group 553 of the neighborhood difference (g p -g c ) is [5, 20, -13, -15, 31, 38, 74, -16].
- the numerical value group 554 obtained by binarizing the neighborhood difference is [1, 1, 0, 0, 1, 1, 1, 0], and the local binary pattern 555 is “11001110” (“206” in decimal). ]).
- FIG. 11 is a schematic diagram showing an outline of a histogram generation process that does not perform weighting according to the prior art, and corresponds to FIG.
- the same parts as those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted.
- the conventional technique calculates a local binary pattern 535 of “11001110” (“206” in decimal) from the neighborhood region 531 shown in FIGS. 6 and 11.
- the prior art does not perform the above-described weighting, and generates a histogram 561 by adding a fixed value such as “1” to the bin “206”, for example. That is, the histogram H (k) generated by the conventional technique is expressed by the following equation (8).
- the histogram is reliable. The degree of contribution of low information is high. As a result, the accuracy of the discriminator generated from the histogram and the accuracy of object detection performed using the histogram are lowered.
- the feature extraction units 230 and 440 calculate the sum of the absolute values of the neighborhood differences having a high correlation with the contrast as a weight, and perform the weighting to generate a histogram. That is, the feature extraction units 230 and 440 digitize the reliability of each local binary pattern by the sum of the absolute values of the neighborhood differences for the local binary pattern that can change depending on the image quality even for the same shooting target. The numerical reliability is, for example, “762” and “212”. Then, the digitized reliability is reflected in the histogram as a weight for adding the frequency. As a result, the feature extraction units 230 and 440 can extract, as image features, a histogram that enables object detection that is robust against variations in the shooting environment while using a local binary pattern.
- each object pixel is weighted according to the difference from the luminance value in the neighboring region. Do. Accordingly, the object detection system 100 can extract image features that are robust against differences in image brightness and noise level. That is, the object detection system 100 can extract an image feature that can detect an object that is robust against variations in the shooting environment.
- the present invention is not limited to this. That is, two or all of these may be integrally configured as one device.
- the feature extraction region acquisition units 220 and 430 and the feature extraction units 230 and 440 are configured as common functional units, respectively. can do.
- Boosting is used as the machine learning method.
- other machine learning methods such as SVM (Support Vector Vector) and decision tree (Decision Tree) may be used.
- the feature extraction device, the feature extraction method, the feature extraction program, and the image processing device according to the present invention are capable of extracting an image feature that can detect an object that is robust against variations in a shooting environment while using a local binary pattern. This is useful as a feature extraction device, a feature extraction method, a feature extraction program, and an image processing device using such a feature extraction device.
- SYMBOLS 100 Object detection system 200 Classifier learning apparatus 210 Learning data storage part 220, 430 Feature extraction area acquisition part 230, 440 Feature extraction part 240 Learning part 300 Classifier storage apparatus 400 Object detection apparatus 410 Camera 420 Image input part 441 Pixel vicinity Acquisition unit 442 Neighborhood difference calculation unit 443 Binary pattern generation unit 444 Weight generation unit 445 Histogram generation unit 450 Identification unit
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Nonlinear Science (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
Description
200 識別器学習装置
210 学習用データ記憶部
220、430 特徴抽出領域取得部
230、440 特徴抽出部
240 学習部
300 識別器記憶装置
400 物体検出装置
410 カメラ
420 画像入力部
441 画素近傍取得部
442 近傍差分計算部
443 バイナリパターン生成部
444 重み生成部
445 ヒストグラム生成部
450 識別部
Claims (9)
- 画像の全部または一部の画素毎に、周囲の近傍画素との画素値の差分が所定の閾値以上であるか否かをビット値により示すローカルバイナリパターンを生成するバイナリパターン生成部と、
生成された前記ローカルバイナリパターン毎に、基の前記画素値の差分に応じた重みを決定する重み生成部と、
決定された前記重みを対応する前記ローカルバイナリパターンに適用して、前記画像から生成された前記ローカルバイナリパターンの分布を示すヒストグラムを生成するヒストグラム生成部と、を有する、
特徴抽出装置。 - 前記ローカルバイナリパターンは、前記画素に対して所定の相対位置にある複数の前記近傍画素との前記差分が前記所定の閾値以上であるか否かを示すデジタル値を所定の順序で並べた符号である、
請求項1記載の特徴抽出装置。 - 前記重みは、前記複数の近傍画素についての前記差分の絶対値の総和に応じた重みである、
請求項2記載の特徴抽出装置。 - 前記重み生成部は、
生成された前記ローカルバイナリパターンのそれぞれについて、前記絶対値の和を重みとして決定し、
前記ヒストグラム生成部は、
前記ローカルバイナリパターンが生成される毎に、当該ローカルバイナリパターンに対応するビンの値に当該ローカルバイナリパターンに対応する前記重みを加算して、前記ヒストグラムを生成する、
請求項3記載の特徴抽出装置。 - 前記ヒストグラム生成部は、
前記ヒストグラムに対して正規化処理を行う、
請求項4記載の特徴抽出装置。 - 所定の物体を識別するための識別器を用いて、請求項1から請求項5のいずれかに記載の特徴抽出装置により生成された前記ヒストグラムから、前記画像に前記所定の物体が含まれるか否か判断する識別部を有する、
画像処理装置。 - 前記識別器は、
前記近傍画素の全ての前記差分が正の値であるときの前記ローカルバイナリパターンに対応するビンの値、および、前記近傍画素の全ての前記差分が負の値であるときの前記ローカルバイナリパターンに対応するビンの値の少なくとも1つに基づいて、前記画像のノイズのレベルを判断する、
請求項6記載の画像処理装置。 - 画像の全部または一部の画素毎に、周囲の近傍画素との画素値の差分が所定の閾値以上であるか否かを各ビット値により示すローカルバイナリパターンを生成するステップと、
生成された前記ローカルバイナリパターン毎に、基の前記画素値の差分に応じた重みを決定するステップと、
決定された前記重みを対応する前記ローカルバイナリパターンに適用して、前記画像から生成された前記ローカルバイナリパターンの分布を示すヒストグラムを生成するステップと、を有する、
特徴抽出方法。 - コンピュータに、
画像の全部または一部の画素毎に、周囲の近傍画素との画素値の差分が所定の閾値以上であるか否かを各ビット値により示すローカルバイナリパターンを生成する処理と、
生成された前記ローカルバイナリパターン毎に、基の前記画素値の差分に応じた重みを決定する処理と、
決定された前記重みを対応する前記ローカルバイナリパターンに適用して、前記画像から生成された前記ローカルバイナリパターンの分布を示すヒストグラムを生成する処理と、を実行させる、
特徴抽出プログラム。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012553605A JP5861123B2 (ja) | 2011-01-20 | 2012-01-13 | 特徴抽出装置、特徴抽出方法、特徴抽出プログラム、および、画像処理装置 |
CN201280002161.3A CN103026384B (zh) | 2011-01-20 | 2012-01-13 | 特征提取装置、特征提取方法以及图像处理装置 |
EP12736848.8A EP2667353B1 (en) | 2011-01-20 | 2012-01-13 | Feature extraction unit, feature extraction method, feature extraction program, and image processing device |
US13/819,508 US9053384B2 (en) | 2011-01-20 | 2012-01-13 | Feature extraction unit, feature extraction method, feature extraction program, and image processing device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011010238 | 2011-01-20 | ||
JP2011-010238 | 2011-01-20 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012098842A1 true WO2012098842A1 (ja) | 2012-07-26 |
Family
ID=46515478
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2012/000175 WO2012098842A1 (ja) | 2011-01-20 | 2012-01-13 | 特徴抽出装置、特徴抽出方法、特徴抽出プログラム、および、画像処理装置 |
Country Status (5)
Country | Link |
---|---|
US (1) | US9053384B2 (ja) |
EP (1) | EP2667353B1 (ja) |
JP (1) | JP5861123B2 (ja) |
CN (1) | CN103026384B (ja) |
WO (1) | WO2012098842A1 (ja) |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014113392A1 (en) * | 2013-01-17 | 2014-07-24 | Google Inc. | Structure descriptors for image processing |
US8866928B2 (en) | 2012-12-18 | 2014-10-21 | Google Inc. | Determining exposure times using split paxels |
US8866927B2 (en) | 2012-12-13 | 2014-10-21 | Google Inc. | Determining an image capture payload burst structure based on a metering image capture sweep |
US9008423B2 (en) | 2013-01-10 | 2015-04-14 | Electronics And Telecommunications Research Institute | Method and apparatus for detecting and recognizing object using local binary patterns |
WO2015076433A1 (ko) * | 2013-11-20 | 2015-05-28 | (주)나노인사이드 | 지역 미세 패턴을 이용한 얼굴 영상 분석 방법 |
US9066017B2 (en) | 2013-03-25 | 2015-06-23 | Google Inc. | Viewfinder display based on metering images |
US9077913B2 (en) | 2013-05-24 | 2015-07-07 | Google Inc. | Simulating high dynamic range imaging with virtual long-exposure images |
US9087391B2 (en) | 2012-12-13 | 2015-07-21 | Google Inc. | Determining an image capture payload burst structure |
US9100589B1 (en) | 2012-09-11 | 2015-08-04 | Google Inc. | Interleaved capture for high dynamic range image acquisition and synthesis |
US9117134B1 (en) | 2013-03-19 | 2015-08-25 | Google Inc. | Image merging with blending |
US9131201B1 (en) | 2013-05-24 | 2015-09-08 | Google Inc. | Color correcting virtual long exposures with true long exposures |
US9247152B2 (en) | 2012-12-20 | 2016-01-26 | Google Inc. | Determining image alignment failure |
US9615012B2 (en) | 2013-09-30 | 2017-04-04 | Google Inc. | Using a second camera to adjust settings of first camera |
US9686537B2 (en) | 2013-02-05 | 2017-06-20 | Google Inc. | Noise models for image processing |
CN110008920A (zh) * | 2019-04-09 | 2019-07-12 | 南京工业大学 | 一种人脸表情识别方法研究 |
CN111047579A (zh) * | 2019-12-13 | 2020-04-21 | 中南大学 | 一种特征质量评估方法及图像特征均匀提取方法 |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9053384B2 (en) * | 2011-01-20 | 2015-06-09 | Panasonic Intellectual Property Management Co., Ltd. | Feature extraction unit, feature extraction method, feature extraction program, and image processing device |
TWI493476B (zh) * | 2014-01-02 | 2015-07-21 | Mstar Semiconductor Inc | 影像處理電路與方法 |
US9576221B2 (en) * | 2014-07-09 | 2017-02-21 | Ditto Labs, Inc. | Systems, methods, and devices for image matching and object recognition in images using template image classifiers |
EP3324183B1 (en) * | 2015-07-15 | 2019-03-13 | Osaka University | Image analysis device, image analysis method, image analysis system, image analysis program, and recording medium |
US9858498B2 (en) * | 2015-09-23 | 2018-01-02 | Qualcomm Incorporated | Systems and methods for incremental object detection using dual-threshold local binary pattern operators |
CN106485238B (zh) * | 2016-11-01 | 2019-10-15 | 深圳大学 | 一种高光谱遥感图像特征提取和分类方法及其系统 |
US10262187B1 (en) * | 2016-11-10 | 2019-04-16 | Synaptics Incorporated | Systems and methods for spoof detection based on local binary patterns |
FR3069087B1 (fr) * | 2017-07-11 | 2019-08-23 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Procede de traitement d'une image |
CN108460343B (zh) * | 2018-02-06 | 2019-06-07 | 北京达佳互联信息技术有限公司 | 图像处理方法、系统及服务器 |
CN108897856A (zh) * | 2018-06-29 | 2018-11-27 | 联想(北京)有限公司 | 一种信息处理方法及电子设备 |
CN109241905B (zh) * | 2018-08-31 | 2021-07-02 | 北方工业大学 | 一种图像处理方法和装置 |
CN111768410B (zh) * | 2019-05-22 | 2024-04-05 | 北京沃东天骏信息技术有限公司 | 图像处理方法和装置 |
FR3098623B1 (fr) * | 2019-07-09 | 2021-07-23 | Commissariat Energie Atomique | Système et procédé de génération de descripteurs d'une scène |
CN111709312B (zh) * | 2020-05-26 | 2023-09-22 | 上海海事大学 | 一种基于联合主模式的局部特征人脸识别方法 |
CN112507847B (zh) * | 2020-12-03 | 2022-11-08 | 江苏科技大学 | 一种基于邻域像素差加权模式的人脸反欺诈方法 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007188504A (ja) * | 2006-01-11 | 2007-07-26 | Mitsubishi Electric Research Laboratories Inc | 画像中の画素強度をフィルタリングする方法 |
JP2009211179A (ja) | 2008-02-29 | 2009-09-17 | Canon Inc | 画像処理方法、パターン検出方法、パターン認識方法及び画像処理装置 |
JP2011010238A (ja) | 2009-06-29 | 2011-01-13 | Nippon Telegr & Teleph Corp <Ntt> | パケット転送品質制御システムおよび方法 |
JP2011186595A (ja) * | 2010-03-05 | 2011-09-22 | Sony Corp | 情報生成装置、物体検出装置、情報生成方法、物体検出方法、およびプログラム |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6985628B2 (en) * | 2002-01-07 | 2006-01-10 | Xerox Corporation | Image type classification using edge features |
JP2006048322A (ja) * | 2004-08-04 | 2006-02-16 | Seiko Epson Corp | オブジェクト画像検出装置、顔画像検出プログラムおよび顔画像検出方法 |
US7519220B2 (en) * | 2004-11-15 | 2009-04-14 | Siemens Medical Solutions Usa, Inc. | GPU accelerated isoperimetric algorithm for image segmentation, digital photo and video editing |
JP5121506B2 (ja) * | 2008-02-29 | 2013-01-16 | キヤノン株式会社 | 画像処理装置、画像処理方法、プログラム及び記憶媒体 |
CN101923640B (zh) * | 2010-08-04 | 2013-03-20 | 北京中科虹霸科技有限公司 | 基于鲁棒纹理特征和机器学习对伪造虹膜图像判别的方法 |
US9053384B2 (en) * | 2011-01-20 | 2015-06-09 | Panasonic Intellectual Property Management Co., Ltd. | Feature extraction unit, feature extraction method, feature extraction program, and image processing device |
US8903198B2 (en) * | 2011-06-03 | 2014-12-02 | International Business Machines Corporation | Image ranking based on attribute correlation |
US8879800B2 (en) * | 2011-06-15 | 2014-11-04 | Honeywell International Inc. | Quality driven image processing for ocular recognition system |
US8718385B2 (en) * | 2012-07-23 | 2014-05-06 | State Farm Mutual Automobile Insurance Company | Siding identification systems and methods |
-
2012
- 2012-01-13 US US13/819,508 patent/US9053384B2/en active Active
- 2012-01-13 EP EP12736848.8A patent/EP2667353B1/en active Active
- 2012-01-13 CN CN201280002161.3A patent/CN103026384B/zh active Active
- 2012-01-13 JP JP2012553605A patent/JP5861123B2/ja active Active
- 2012-01-13 WO PCT/JP2012/000175 patent/WO2012098842A1/ja active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007188504A (ja) * | 2006-01-11 | 2007-07-26 | Mitsubishi Electric Research Laboratories Inc | 画像中の画素強度をフィルタリングする方法 |
JP2009211179A (ja) | 2008-02-29 | 2009-09-17 | Canon Inc | 画像処理方法、パターン検出方法、パターン認識方法及び画像処理装置 |
JP2011010238A (ja) | 2009-06-29 | 2011-01-13 | Nippon Telegr & Teleph Corp <Ntt> | パケット転送品質制御システムおよび方法 |
JP2011186595A (ja) * | 2010-03-05 | 2011-09-22 | Sony Corp | 情報生成装置、物体検出装置、情報生成方法、物体検出方法、およびプログラム |
Non-Patent Citations (3)
Title |
---|
NAVNEET DALAL; BILL TRIGGS: "Histograms of oriented gradients for human detection", CVPR, 2005 |
See also references of EP2667353A4 |
TIMO AHONEN; ABDENOUR HADID; MATTI PIETIKÄINEN: "Face Description with Local Binary Patterns: Application to Face Recognition", IEEE, PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 28, no. 12, December 2006 (2006-12-01), pages 2037 - 2041 |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9100589B1 (en) | 2012-09-11 | 2015-08-04 | Google Inc. | Interleaved capture for high dynamic range image acquisition and synthesis |
US9087391B2 (en) | 2012-12-13 | 2015-07-21 | Google Inc. | Determining an image capture payload burst structure |
US8866927B2 (en) | 2012-12-13 | 2014-10-21 | Google Inc. | Determining an image capture payload burst structure based on a metering image capture sweep |
US8964060B2 (en) | 2012-12-13 | 2015-02-24 | Google Inc. | Determining an image capture payload burst structure based on a metering image capture sweep |
US9118841B2 (en) | 2012-12-13 | 2015-08-25 | Google Inc. | Determining an image capture payload burst structure based on a metering image capture sweep |
US8866928B2 (en) | 2012-12-18 | 2014-10-21 | Google Inc. | Determining exposure times using split paxels |
US9172888B2 (en) | 2012-12-18 | 2015-10-27 | Google Inc. | Determining exposure times using split paxels |
US9247152B2 (en) | 2012-12-20 | 2016-01-26 | Google Inc. | Determining image alignment failure |
US9008423B2 (en) | 2013-01-10 | 2015-04-14 | Electronics And Telecommunications Research Institute | Method and apparatus for detecting and recognizing object using local binary patterns |
US8995784B2 (en) | 2013-01-17 | 2015-03-31 | Google Inc. | Structure descriptors for image processing |
WO2014113392A1 (en) * | 2013-01-17 | 2014-07-24 | Google Inc. | Structure descriptors for image processing |
US9686537B2 (en) | 2013-02-05 | 2017-06-20 | Google Inc. | Noise models for image processing |
US9749551B2 (en) | 2013-02-05 | 2017-08-29 | Google Inc. | Noise models for image processing |
US9117134B1 (en) | 2013-03-19 | 2015-08-25 | Google Inc. | Image merging with blending |
US9066017B2 (en) | 2013-03-25 | 2015-06-23 | Google Inc. | Viewfinder display based on metering images |
US9131201B1 (en) | 2013-05-24 | 2015-09-08 | Google Inc. | Color correcting virtual long exposures with true long exposures |
US9077913B2 (en) | 2013-05-24 | 2015-07-07 | Google Inc. | Simulating high dynamic range imaging with virtual long-exposure images |
US9615012B2 (en) | 2013-09-30 | 2017-04-04 | Google Inc. | Using a second camera to adjust settings of first camera |
WO2015076433A1 (ko) * | 2013-11-20 | 2015-05-28 | (주)나노인사이드 | 지역 미세 패턴을 이용한 얼굴 영상 분석 방법 |
CN110008920A (zh) * | 2019-04-09 | 2019-07-12 | 南京工业大学 | 一种人脸表情识别方法研究 |
CN111047579A (zh) * | 2019-12-13 | 2020-04-21 | 中南大学 | 一种特征质量评估方法及图像特征均匀提取方法 |
CN111047579B (zh) * | 2019-12-13 | 2023-09-05 | 中南大学 | 一种特征质量评估方法及图像特征均匀提取方法 |
Also Published As
Publication number | Publication date |
---|---|
EP2667353B1 (en) | 2019-07-10 |
JPWO2012098842A1 (ja) | 2014-06-09 |
EP2667353A4 (en) | 2017-10-11 |
EP2667353A1 (en) | 2013-11-27 |
CN103026384A (zh) | 2013-04-03 |
US9053384B2 (en) | 2015-06-09 |
US20130163870A1 (en) | 2013-06-27 |
JP5861123B2 (ja) | 2016-02-16 |
CN103026384B (zh) | 2016-08-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5861123B2 (ja) | 特徴抽出装置、特徴抽出方法、特徴抽出プログラム、および、画像処理装置 | |
JP5891409B2 (ja) | 特徴抽出装置、特徴抽出方法、および特徴抽出プログラム | |
JP5789751B2 (ja) | 特徴抽出装置、特徴抽出方法、特徴抽出プログラム、および画像処理装置 | |
US9740965B2 (en) | Information processing apparatus and control method thereof | |
JP4588575B2 (ja) | デジタル画像の複数対象物検出方法および装置並びにプログラム | |
US8103058B2 (en) | Detecting and tracking objects in digital images | |
JP2008117391A (ja) | デジタル画像における顔を検出する方法、およびデジタル画像における顔を検出する装置 | |
US9633284B2 (en) | Image processing apparatus and image processing method of identifying object in image | |
JP5671928B2 (ja) | 学習装置、学習方法、識別装置、識別方法、およびプログラム | |
US9171224B2 (en) | Method of improving contrast for text extraction and recognition applications | |
Russell et al. | Feature-based image patch classification for moving shadow detection | |
JP4795864B2 (ja) | 特徴点検出装置および方法並びにプログラム | |
JP2012150730A (ja) | 特徴抽出装置、特徴抽出方法、特徴抽出プログラム、および画像処理装置 | |
US8300927B2 (en) | Mouth removal method for red-eye detection and correction | |
JP2006323779A (ja) | 画像処理方法、画像処理装置 | |
KR101539058B1 (ko) | 이미지 분류 및 검색 장치 및 그 방법 | |
JP2013011950A (ja) | 画像処理装置、画像処理方法及びプログラム | |
Brehar et al. | Local information statistics of lbp and hog for pedestrian detection | |
KR101672946B1 (ko) | 눈 개폐 분류 장치 및 방법 | |
Ojo et al. | Illumination invariant face detection using hybrid skin segmentation method | |
JP2015191590A (ja) | 画像処理装置、画像処理方法及びプログラム | |
JP2012230642A (ja) | 画像処理装置、画像処理方法及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201280002161.3 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12736848 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012553605 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13819508 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012736848 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |