JP2010211468A - Device for creating learning model, object detection system, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To create a learning model which is reduced in the number of characteristics while ensuring detection performance substantially equal to that of the related art. <P>SOLUTION: For each of a plurality of ranks 1-4 according to the easiness and the difficulty in detection of a detection object, a plurality of learning images is selected, based on a plurality of learning images preliminarily classified to the easiest rank 1 to the most difficult rank 4 in detection of the detection object, so that the ratio of the number N1 of learning images classified to the ranks 2 and 3 between the easiest rank 1 and the most difficult rank 4 to the number N1 of learning images classified to the ranks 1 and 4 is a predetermined value or less, and a learning model including a group of characteristics of the number according to each of the plurality of selected learning images is created based on the selected learning images. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a learning model generation device, an object detection system, and a program.

  Conventionally, from two types of learning images, an image of an object (for example, a person, also referred to as a detection object) and an image of a non-object (non-detection object), for example, a technique of Viola & Jones (for example, “Paul Viola” and Michael Jones: Using the method described in "Rapid Object Detection using a Boosted Cascade of Simple Features", IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, cascade boosting algorithm), Haar-like features A learning model generation device that generates a learning model composed of a set, and an object detection device that includes detection means for detecting an object from an input image based on the learning model and the input image generated by the learning model generation device Is known (for example, refer to Patent Document 1).

JP 2008-16596A

  However, the number of Haar-like features (number of features) constituting the generated learning model varies greatly depending on the quality of the learning image. Here, the quality of the learning image refers to, for example, the size of the resolution of the learning image, and the portion where the detection target object (detection target) in the learning image is hidden by an object other than the detection target object. , The degree of blur in the learning image, the background noise in the learning image, and the like. For example, the quality of the learning image becomes better as the resolution of the learning image increases, and the detection target object (detection target) in the learning image is hidden by an object other than the detection target object. The smaller the size of the portion, the better. The smaller the degree of blur in the learning image, the better. The smaller the background noise in the learning image, the better. That is, the number of features increases as the number of images with low resolution increases in the learning image. In addition, the number of features increases as the number of images in which the detection target in the learning image is hidden by a target other than the detection target increases. In addition, the number of features increases as the number of images having a large degree of blur in the learning image increases. The number of features increases as the number of images with large background noise in the learning image increases.

  The learning model generation device described in Patent Literature 1 does not consider the quality of the learning image, and generates the learning model using the learning image at random. Therefore, learning is performed by increasing the number of features. There is a case where the size of the model becomes large, and in such a case, there is a problem that the capacity of the storage means for storing the learning model becomes large. Further, when the number of features increases, there is a problem that it takes a long time for detection processing (identification processing) of a detection target in the target detection device.

  Therefore, in order to reduce the number of features, the quality learning image (for example, the resolution is large, the portion where the detection target is hidden by the target other than the detection target is small, the degree of blur is small, and It is conceivable to generate a learning model using only learning images with low background noise, but learning models generated based only on such learning images are based on objects of various quality variations. Since it is not a learning model, it cannot cope with the detection of various detection objects, and there is a problem that the detection performance of the object detection device is lowered.

  The present invention has been made to solve the above-described problems, and learning capable of generating a learning model having the same detection performance and reduced number of features as compared with the conventional technique. It is an object to provide a model generation device, an object detection system, and a program.

  In order to achieve the above object, the learning model generation device according to the first aspect of the present invention detects the detection object most frequently for each of a plurality of ranks according to the ease of detection of the detection object or the degree of difficulty of detection. The number of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect based on a plurality of learning images that are classified in advance from the rank that is most difficult to detect to the rank that is most difficult to detect Selecting means for selecting a plurality of learning images such that a ratio of the number of the plurality of learning images classified into the rank that is most easily detected and the rank that is most difficult to detect is equal to or less than a predetermined value; Based on a plurality of learning images selected by the selection unit, a generation unit that generates a learning model including a set of features corresponding to each of the plurality of learning images is configured.

  According to the learning model generation device according to the first invention, the most easily detected rank and the most easily detected number of learning images classified into a rank between the most easily detected rank and the most difficult to detect rank. A plurality of learning images are selected such that the ratio of the number of learning images classified into ranks that are difficult to detect is equal to or less than a predetermined value, and a plurality of learning images are selected based on the selected plurality of learning images. A learning model including a set of features corresponding to each of the learning images is generated. The learning model generated in this way is a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  Therefore, according to the learning model generation device according to the first aspect of the present invention, it is possible to generate a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  In order to achieve the above object, the learning model generation device according to the second aspect of the present invention is configured to detect the detection target object for each of a plurality of ranks according to the ease of detection of the detection target object or the degree of detection difficulty. Classification means for classifying each of the plurality of learning images into any one of the plurality of ranks based on a plurality of reference images previously classified from a rank that is most easily detected to a rank that is most difficult to detect; Based on the plurality of learning images classified by the classification means, the number of the plurality of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect is the most easily detected. A selection unit that selects a plurality of learning images so that a ratio of the number of the plurality of learning images classified into a rank and a rank that is most difficult to detect is equal to or less than a predetermined value; Based on the plurality of learning images, and is configured to include a generation means for generating a learning model that includes a set of number of features corresponding to each of the plurality of learning images.

  According to the learning model generation device according to the second aspect of the present invention, each of the plurality of learning images is based on a plurality of reference images that are classified in advance from the rank that is most easily detected to the rank that is most difficult to detect. The most easily detected rank and the most probable rank for the number of learning images classified into one of a plurality of ranks and classified into a rank between the most easily detected rank and the most difficult to detect rank A plurality of learning images are selected such that the ratio of the number of learning images classified into ranks that are difficult to detect is equal to or less than a predetermined value, and a plurality of learning images are selected based on the selected plurality of learning images. A learning model including a set of features corresponding to each of the learning images is generated. The learning model generated in this way is a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  Therefore, according to the learning model generation device according to the second invention, it is possible to generate a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  Further, the degree of ease of detection of the detection object increases as the resolution of the learning image increases, and as the part of the learning image hidden by the object other than the detection object increases. It may be determined to be smaller and smaller as the degree of blur in the learning image increases or smaller as the background noise in the learning image increases.

  In addition, the degree of difficulty in detecting the detection object decreases as the resolution of the learning image increases, and as the part of the learning image hidden by the object other than the detection object increases. It may be determined so that it increases as the degree of blur in the learning image increases, or as the background noise in the learning image increases.

  In order to achieve the above object, an object detection system according to a third aspect of the present invention is based on the learning model generation device, the learning model generated by the learning model generation device, and the input image. And an object detection device including detection means for detecting the object from the input image.

  In order to achieve the above object, a program according to a fourth aspect of the present invention provides a computer that detects a detection object for each of a plurality of ranks according to the ease of detection of the detection object or the degree of difficulty of detection. A plurality of learning images classified into a rank between a rank that is most easily detected and a rank that is most difficult to detect based on a plurality of learning images classified in advance from a rank that is most easily detected to a rank that is most difficult to detect Selection means for selecting a plurality of learning images such that a ratio of the number of the plurality of learning images classified into the rank that is most easily detected and the rank that is difficult to detect with respect to the number of And a function for generating a learning model including a number of features corresponding to each of the plurality of learning images based on the plurality of learning images selected by the selection unit. A.

  According to the program of the fourth invention, the most easily detected rank and the most easily detected number of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect. A plurality of learning images are selected such that the ratio of the number of learning images classified into ranks that are difficult to detect is equal to or less than a predetermined value, and a plurality of learning images are selected based on the selected plurality of learning images. A learning model including a set of features corresponding to each of the learning images is generated. The learning model generated in this way is a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  Therefore, according to the program according to the fourth aspect of the present invention, it is possible to generate a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  In order to achieve the above object, a program according to a fifth aspect of the present invention provides a computer that detects a detection object for each of a plurality of ranks according to the ease of detection of the detection object or the degree of difficulty of detection. Classification means for classifying each of a plurality of learning images into any one of the plurality of ranks based on a plurality of reference images previously classified from a rank that is most easily detected to a rank that is most difficult to detect, Based on the plurality of learning images classified by the classification means, the most easily detected rank with respect to the number of the plurality of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect. And a selection unit that selects a plurality of learning images so that a ratio of the number of the plurality of learning images classified into the rank that is most difficult to detect is equal to or less than a predetermined value, and the selection unit Based on the-option by a plurality of learning images, it is intended to function as a generating means for generating a learning model including the features of a number corresponding to each of the plurality of learning images.

  According to the program according to the fifth aspect of the present invention, each of the plurality of learning images is based on a plurality of reference images that are pre-classified by a computer from a rank that is most easily detected to a rank that is most difficult to detect. The most easily detected rank and the most probable rank for the number of learning images classified into one of a plurality of ranks and classified into a rank between the most easily detected rank and the most difficult to detect rank A plurality of learning images are selected such that the ratio of the number of learning images classified into ranks that are difficult to detect is equal to or less than a predetermined value, and a plurality of learning images are selected based on the selected plurality of learning images. A learning model including a set of features corresponding to each of the learning images is generated. The learning model generated in this way is a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  Therefore, according to the program according to the fifth aspect of the present invention, it is possible to generate a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  As described above, according to the learning model generation device, the object detection system, and the program according to the present invention, the learning model has the same detection performance and the reduced number of features compared to the conventional technology. Can be produced.

It is a figure which shows the target object detection system which concerns on this Embodiment. It is a figure which shows an example of the reference | standard image which concerns on this Embodiment. It is a figure which shows the flowchart of the process routine of the learning model production | generation process which the learning model production | generation apparatus 20 which concerns on this Embodiment performs. It is a figure which shows the flowchart of the process routine of the target object detection process which the target object detection apparatus 30 which concerns on this Embodiment performs. Pedestrian detection performance (ie, conventional technology) when a learning model is generated by randomly selecting a learning image and used for pedestrian detection, and a pedestrian in the object detection system of the present embodiment It is a graph showing the comparative example of the experimental result of detection performance. A graph showing a comparative example of the number of features when a learning model is generated by randomly selecting a learning image and generating a learning model used for pedestrian detection and the result of experiments on the number of features of the object detection system of this embodiment is there.

  Hereinafter, with reference to the drawings, a learning model generation device that generates a learning model for detecting a person (more specifically, a pedestrian) as a detection target object (detection target), and a learning model generation device Implementation when the present invention is applied to an object detection system including an object detection device including a detection unit that detects a person as a detection object from an input image based on a generated learning model and an input image Will be described in detail.

  FIG. 1 is a diagram showing a configuration of an object detection system 10 according to an embodiment of the present invention. The object detection system 10 according to the present embodiment includes an object detection device 30 that detects a detection object and a learning model generation device 20 that generates a learning model used by the object detection device 30. Has been.

  The object detection device 30 includes a computer having a CPU (Central Processing Unit) (not shown), a ROM (Read Only Memory) (not shown), a RAM (Random Access Memory) (not shown), and a bus (not shown) for connecting them. It is configured to include. The CPU controls the entire object detection device 30 by reading the program from the ROM and executing it. The ROM stores a basic program such as an OS and a program for executing a processing routine of an object detection process described later. Data is temporarily stored in the RAM as a work area.

  The target object detection device 30 will be described in terms of functional blocks divided for each function realization means determined based on hardware and software. As shown in FIG. 1, a window image extraction unit 32, an identification unit 34, and a result And an output unit 36.

  The window image extraction unit 32 extracts a window image (an image of an area based on the window) from an input image input from the outside. In this embodiment, when a detection target is detected from an input image, a window having a predetermined size (referred to as a search window) is moved from the input image by a predetermined movement amount (referred to as a search pitch). Then, the image is cut out (the image is extracted), and the detection target object is searched from the cut out image (the extracted image). Here, the cut image is referred to as a window image, and the size of the window image (that is, the size of the search window) is referred to as a window size. A plurality of window sizes are set, and the window image extraction unit 32 extracts window images using all the set search windows. In addition, the window image extraction unit 32 converts the extracted window image into an image having a preset number of pixels (for example, an image of 16 × 32 pixels).

  The identification unit (evaluation unit) 34 uses the window image and a learning model described later to calculate an evaluation value indicating the likelihood of the detection target, and detects the target. That is, the identification unit 34 determines whether or not the window image is a detection target (a person in the present embodiment) by comparing the window image and the learning model, and determines that the window image is a person. In this case, it is determined that the window image is a person. Thereby, a detection target is detected from the input image.

  The result output unit 36 uses the presence / absence information indicating the presence / absence of the detection target, the image data indicating the image of the detection target, the position information indicating the position of the detection target, and the like as the detection result of the detection target. For example, by outputting to the display device 38, for example, the display of the display device 38 is controlled so that the detection result is displayed superimposed on the input image.

  The object detection device 30 extracts window images while moving a search window of each size set in advance with respect to the input image by the search pitch, and the detection object of all the extracted window images is detected. In order to execute the detection process, the processes in the window image extraction unit 32 and the evaluation unit 34 are repeated for the number of window images to be extracted.

  The learning model generation device 20 includes a CPU (Central Processing Unit) (not shown), a ROM (Read Only Memory) (not shown), a RAM (Random Access Memory) (not shown), an HDD (Hard Disk Drive) (not shown), and a bus ( (Not shown). The CPU reads the program from the ROM and HDD and executes it. The ROM stores basic programs such as an OS. Data is temporarily stored in the RAM as a work area. The HDD stores a program for executing a processing routine of learning model generation processing described later.

  Also, the HDD has a plurality of ranks corresponding to the degree of ease of detection or difficulty of detection of the detection objects shown in Table 1 below (in this embodiment, a person, more specifically a pedestrian). For each of 1-4, a plurality of reference images classified in advance from the rank that is most easily detected to the rank that is most difficult to detect are stored.

  Here, this rank will be described. In the present embodiment, it is assumed that an image having a higher degree of good quality belongs to a lower rank. Note that the image quality refers to, for example, the size of the resolution of the image, the size of the portion of the image where the detection target object (detection target) is hidden by a target other than the detection target, and the blur in the image. And the size of background noise in the image. For example, the quality of the image becomes better as the resolution of the image increases, and the size of the portion of the image where the detection target object (detection target) is hidden by an object other than the detection target is small. The smaller the degree of blur in the image, the better. The smaller the background noise in the image, the better. That is, the degree of ease of detection of the detection target increases as the resolution of the learning image increases, and decreases as the portion of the learning image hidden by the target other than the detection target increases. Thus, the degree of blur is set to be smaller as the degree of blur in the learning image is increased, or to be decreased as background noise in the learning image is increased. Similarly, the degree of difficulty of detection of the detection object decreases as the resolution of the learning image increases, and as the part of the learning image hidden by the object other than the detection object increases. It is set so that it increases as the degree of blur in the learning image increases, or as the background noise in the learning image increases.

  In the present embodiment, a plurality of reference images corresponding to each of the ranks 1 to 4 are classified and stored in the HDD in advance by the designer of the detection target system 10 or the like. For example, as shown in FIG. 2, the reference image 60a of “a pedestrian with low blur and hiding, low background noise and good image quality” is classified into rank 1 by a manual operation of a designer or the like, and “some blur or The reference image 60b of “pedestrian with hiding and background noise but relatively good image quality” is classified into rank 2, and “pedestrian with hiding, hiding, background noise, poor image quality, and difficult to detect” ”Is classified into rank 3, and‘ pedestrians that seem to be undetectable because of blurring, hiding, and background noise are large, ”categorized as rank 4, and the reference images 60 a to d are classified as rank images 60 a to d. Stored in HDD.

  The learning model generation device 20 will be described using functional blocks divided for each function realization means determined based on hardware and software. As shown in FIG. 1, a learning image classification unit 42 and a learning image selection unit 44, a learning unit 46, and a learning model DB (identification model DB) 48.

  A plurality of learning images (for example, about 30,000 images) having a preset window size are input to the learning image classification unit 42. As the learning image, an image including a detection target (positive data) and a non-target image including no detection target (negative data) can be considered. When one learning image including an object to be detected is input, the learning image classifying unit 42 is based on, for example, Nearest Neighbor (nearest neighbor determination rule) based on the reference images 60a to 60d stored in the HDD. The learning image including the input detection target is classified into one of a plurality of ranks 1 to 4. Then, the learning image classification unit 42 performs this classification for the number of images including the input detection target. Note that when a learning image of a non-object is input, the learning image classification unit 42 does not perform the above classification on the learning image.

The learning image selection unit 44 is the most easily detected rank 1 and the most difficult to detect based on the plurality of learning images classified into the corresponding ranks by the learning image classification unit 42 according to the following equation (1). The ratio of the number N ₂ of the plurality of learning images classified into rank 1 and rank 4 to the number N ₁ of the plurality of learning images classified into rank 2 and rank 3 between rank 4 is a predetermined value. A learning image of a plurality of M ₁ (N ₁ + N ₂ ) detection objects is selected so as to be T or less.

For example, when the value of T is set to 1.0, for example, the learning image selection unit 44 has, for example, 5000 learning images classified into rank 1 and learning images classified into rank 2. 5000 images, 0 learning images classified into rank 3 and 0 learning images classified into rank 4 are selected. At this time, are classified into No. 2 and No. 3, and the number N ₁ of the learning images to be selected is 5000 (5000 + 0), the learning images are classified into the rank 1 and rank 4, and is selected The number N ₂ is 5000 (5000 + 0), and the value of “N ₂ / N ₁ ” is 1 (5000/5000).

Similarly, when the value of T is set to 0.2, for example, the learning image selection unit 44, for example, 1000 learning images classified into rank 1 and learning images classified into rank 2 5000 images, 1000 learning images classified into rank 3, and 0 learning images classified into rank 4 are selected. At this time, are classified into No. 2 and No. 3, and the number _{N 1} of the learning images to be selected is 6000 (1000 + 5000), the learning images are classified into the rank 1 and rank 4, and is selected The number N ₂ is 1000 (1000 + 0), and the value of “N ₂ / N ₁ ” is 0.16... (1000/6000).

Further, the learning image selection unit 44 selects a plurality of M ₂ learning images of non-objects that have not been classified by the learning image classification unit 42.

The learning unit 46 uses the learning image of the M _one detection object selected by the learning image selection unit 44 and the learning image of the M ₂ non-objects selected by the learning image selection unit 44. A learning model indicating the characteristics of the detection target is generated.

  The learning model generated by the learning unit 46 is stored in the learning model DB 48. In the present embodiment, an example in which the learning model DB 48 is realized using an HDD as a storage unit will be described, but the present invention is not limited to this. For example, the learning model DB 48 is not particularly limited as long as the learning model DB 48 is an external storage unit such as a CD-ROM and includes a medium that can store the learning model. The learning model stored in the learning model DB 48 is used for detection of an object by the identification unit 34 of the object detection device 30.

  In the present embodiment, an example in which the learning model generation device 20 and the object detection device 30 are realized by independent computers will be described, but these may be realized on the same computer.

  In the object detection system 10 of the present embodiment configured as described above, a detection object such as a pedestrian is detected from an input image using the Viola & Jones method (hereinafter referred to as V & J) as described above. To do.

  The V & J method is a two-class identification method, which realizes high-speed identification processing by using Haar-like features, a high-speed calculation method based on its integral image, and cascaded Adaboost classifiers (Paul Viola and (See Michael Jones: "Rapid Object Detection using a Boosted Cascade of Simple Features", IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001.)

  According to the V & J method, the learning model generation apparatus 20 according to the present embodiment uses a learning model (a target object) that includes a set of Haar-like features from two types of learning images, ie, a detection target image and a non-target image. That serves as a dictionary used to identify whether or not).

  Further, as described above, the object detection device 30 extracts a window image while gradually shifting a plurality of windows called search windows vertically and horizontally with respect to the input image, and the window image and the above-described generated image are generated. An object is extracted by comparing with a learning model.

  Next, a processing routine of learning model generation processing executed by the CPU of the learning model generation device 20 will be described with reference to FIG. In this embodiment, for example, instruction learning means (not shown) such as a touch panel or a keyboard is provided in the learning model generation apparatus 20 in the present embodiment, and the user operates this instruction reception means. This is executed when the CPU of the learning model generation device 20 receives an instruction to generate a learning model. Here, the V & J method described above is used as a learning method. In addition, a predetermined number (for example, 15000 each) of two types of learning images, that is, a detection target image and a non-target image are prepared in advance. The size of the input learning image is set to a preset size (for example, 16 × 32 pixels).

  First, at step 100, one learning image is read.

  In the next step 102, it is determined whether or not the input learning image is an image including a detection target, and when it is determined that the input learning image is an image including a detection target. Based on the reference images 60a to 60d stored in the HDD, the learning image including the input detection target is selected from any one of the ranks 1 to 4 by, for example, Nearest Neighbor (nearest neighbor determination rule). It is classified into the rank. Thereby, the learning image including the input detection target can be classified into the same rank as the most similar reference image in the reference images 60a to 60d. The similarity between images may be a normalized correlation between feature values such as pixel values and four-way plane features, and an Euclidean distance, a city distance, etc. may be used as an index indicating similarity. Also good. Further, as a method for determining whether or not the input learning image is an image including a detection target, for example, a label (whether or not it is an image including a detection target) included in the learning image. A method of determining based on the label) is conceivable.

  In the next step 104, it is determined whether or not the processing in step 102 has been performed for all prepared learning images. If it is determined in step 104 that the processing in step 102 has not been performed for all of the prepared learning images, the process returns to step 100, and the next learning image is read in step 100. On the other hand, if it is determined in step 104 that the processing in step 102 has been performed on all prepared learning images, the process proceeds to the next step 106.

In step 106, rank 1 that is most easily detected based on the plurality of learning images (learning images including the detection target) classified into the corresponding ranks in step 102 according to the following equation (2): The number N ₂ of the plurality of learning images classified into rank 1 and rank 4 with respect to the number N ₁ of the plurality of learning images classified into rank 2 and rank 3 between rank 4 and the most difficult to detect A learning image of a plurality of M ₁ (N ₁ + N ₂ ) detection objects is selected so that the ratio is equal to or less than a predetermined value T.

As described above, for example, when the value of T is set to 1.0, for example, in step 106, for example, 5000 learning images classified into rank 1 and learning images classified into rank 2 are used. It is possible to select 5000 images, 0 learning images classified into rank 3, and 0 learning images classified into rank 4. At this time, the number N ₁ of learning images classified into rank 2 and rank 3 and selected is 5000 (5000 + 0), and is classified into rank 1 and rank 4 and selected learning images. The number N ₂ is 5000 (5000 + 0), and the value of “N ₂ / N ₁ ” is 1 (5000/5000).

Further, as described above, when the value of T is set to 0.2, for example, in step 106, for example, 1000 learning images classified into rank 1 and learning images classified into rank 2 are used. It is possible to select 5000 images, 1000 learning images classified into rank 3, and 0 learning images classified into rank 4. At this time, are classified into No. 2 and No. 3, and the number _{N 1} of the learning images to be selected is 6000 (1000 + 5000), the learning images are classified into the rank 1 and rank 4, and is selected The number N ₂ is 1000 (1000 + 0), and the value of “N ₂ / N ₁ ” is 0.16 ·· (1000/6000).

In step 106, a plurality of M ₂ learning images of non-objects not classified in step 102 are selected.

In the next step 108, from the learning image of the M ₁ detection object selected in step 106 and the learning image of the M ₂ non-object selected in step 106, according to the V & J method, A learning model composed of a set of Haar-like features (a learning model indicating features of the detection target) is generated, and the generated learning model is stored in the learning model DB 48. Then, the learning model generation process ends.

  Steps 100, 102, and 104 are executed by the learning image classification unit 42, step 106 is executed by the learning image selection unit 44, and step 108 is executed by the learning unit 46.

  In the above description, the example in which each of the plurality of learning images is classified into any one of the plurality of ranks by the processing of the CPU has been described. However, each of the plurality of learning images is determined by a designer or the like. You may classify | categorize beforehand in any rank of several ranks.

  Next, the processing routine of the target object detection process which CPU of the target object detection apparatus 30 performs is demonstrated using FIG. In this embodiment, for example, in this embodiment, an instruction receiving unit (not shown) (for example, a touch panel or a keyboard) is provided in the target object detection device 30, and the user operates the instruction receiving unit. As a result, when the CPU of the object detection device 30 receives an instruction to detect the detection object, the instruction is started. After the instruction is received, the instruction is detected at a predetermined time interval (for example, at an interval of 100 ms). .

  First, in step 200, one image to be detected is detected (in the present embodiment, this image is referred to as an input image). Note that this input image is, for example, an image photographed by photographing means (not shown) such as a camera. In step 200, the input image is read from this photographing means.

  In the next step 202, a search window having a predetermined initial size, for example, 16 × 32 pixels, is set to the initial position (for example, the left corner area of the input image) for the input image.

  In the next step 204, a window image of a predetermined size, for example, 16 × 32 pixels is extracted from the input image using the set search window. If the set search window is a window having a size exceeding 16 × 32 pixels, the extracted window image is converted to a predetermined size, for example, 16 × 32 pixels.

  In the next step 206, the window image extracted in step 204 is compared with the learning model read from the learning model DB 48, and the probability of the detection target is calculated as an evaluation value.

  In the next step 208, the detection target is detected by determining whether or not the window image is a detection target (for example, a person in the present embodiment) based on the calculated evaluation value.

  There are various methods for detecting the detection object using the window image and the learning model, but the detection can be performed using the AdaBoost algorithm or the like employed in the above-described V & J method.

  If it is determined in step 208 that the window image is a detection target, it is determined that the detection target has been detected, and the process proceeds to the next step 210. On the other hand, if it is determined in step 208 that the window image is not a detection object, the process proceeds to the next step 212.

  In step 210, information such as the position and size of the search window when the window image is determined to be a detection target in step 208 is stored in the RAM as a detection result of the detection target.

  In the next step 212, it is determined whether or not the search has been completed by scanning the search window for the entire input image. When the determination is affirmative (when it is determined that the search has been completed by scanning the search window for the entire input image), the process proceeds to step 216. When the determination is negative (the search window is scanned for the entire input image, the search ends). If it is determined that it is not, the process proceeds to step 214.

  In step 214, the position of the search window is moved by a predetermined search pitch, and the process returns to step 204. Then, the processing from step 204 to step 212 is repeated. When the search window searches the entire image, the process proceeds to step 216.

  In step 216, it is determined whether or not the search in all size search windows has been completed. Here, the search window is used as a frame for extracting a window image for detecting the detection object. However, if the size of the search window is different, detection objects of various sizes can be detected. . In the present embodiment, search windows of various sizes are prepared in advance, and it is necessary to search the entire image in each search window. Therefore, if a negative determination is made in step 216 (when it is determined that the search in all size search windows has not been completed), the process proceeds to step 218, and in step 218, a search window having a different window size is determined. Set. For example, in step 218, a search window (for example, a search window whose size is 1.2 times larger) than the previous search window is set as an initial position (for example, a left corner area of the input image). To do. Then, returning to step 204, the processing after step 204 is repeatedly executed in the same manner as described above.

  On the other hand, when an affirmative determination is made at step 216 (when it is determined that the search in the search windows of all sizes has been completed), the routine proceeds to the next step 220. In step 220, based on the detection result of the detection object stored in the RAM, the display device 38 is controlled so that the detected detection object is displayed surrounded by a window with respect to the input image. Then, the object detection process ends.

  Steps 200, 202, 204, 212, 214, 216, and 218 are executed by the window image extraction unit 32, steps 206, 208, and 210 are executed by the identification unit 34, and step 220 is executed by the result output unit 36. The

  Next, the effect of the present embodiment will be described with reference to FIGS.

  FIG. 5 shows a pedestrian detection performance (that is, a conventional technique) when a learning model is generated by randomly selecting a learning image and performing pedestrian detection, and object detection according to the present embodiment. The graph showing the comparative example of the experimental result of the pedestrian detection performance of the system 10 is shown. This graph is a ROC (receiver operating characteristic) curve in which the detection rate and the number of false detections per frame are plotted for each stage of the discriminator. The vertical axis is the detection rate, and the horizontal axis is the number of false detections per frame. is there. The detection rate is expressed as “detected pedestrian / pedestrian to be detected”, and the number of erroneous detections per frame is expressed as “number of detections other than detection target / number of evaluation frames”. In the ROC curve, the higher the graph is in the upper left, the higher the performance. That is, when the value on the horizontal axis is the same (the number of false detections is equal), the larger the value on the vertical axis, the higher the detection rate. When the value on the vertical axis is the same (the detection rate is equal), the value on the horizontal axis is small. It means that there are few false detections.

In the experiment, three data sets were created using learning images classified by rank. The first data set (data set 1) has 0 learning images classified into rank 1, 5000 learning images classified into rank 2, 0 learning images classified into rank 3, rank 4 It was created using 0 learning images classified into the above. The second data set (data set 2) has 5000 learning images classified into rank 1, 5000 learning images classified into rank 2, 0 learning images classified into rank 3, and rank 4 It was created using 0 learning images classified into the above. The third data set (data set 3) has 1000 learning images classified into rank 1, 5000 learning images classified into rank 2, 1000 learning images classified into rank 3, rank 4 It was created using 0 learning images classified into the above. Each of these data sets is rank 1 with respect to the number N ₁ of a plurality of learning images classified into rank 2 and rank 3 between rank 1 that is most easily detected and rank 4 that is most difficult to detect. And a plurality of learning images of M ₁ (N ₁ + N ₂ ) detection objects in which the ratio of the number N ₂ of the plurality of learning images classified into rank 4 is equal to or less than a predetermined value T is used. . FIG. 5 shows ROC curves 70, 72, and 74 corresponding to the data sets 1, 2, and 3, respectively.

  Also, in FIG. 5, for comparison, a case where 5000 images are randomly selected from the same set of learning images without considering quality, learned, and a learning model used for pedestrian detection is generated (random) The ROC curve 76 at the highest performance in (sampling) is shown.

  As shown by the ROC curves 70, 72, 74, 76 in FIG. 5, it can be seen that the detection performance is almost the same in any case.

  FIG. 6 shows experimental results of the number of features when a learning model is generated by randomly selecting an image for learning and used for pedestrian detection, and the number of features of the object detection system 10 of the present embodiment. A graph representing a comparative example is shown. In this graph, the vertical axis represents the number of features, and the horizontal axis represents the number of erroneous detections per frame.

  FIG. 6 shows graphs 80, 82, and 84 corresponding to the data sets 1, 2, and 3, respectively. FIG. 6 shows a graph 86 in the case of random sampling.

  As shown in FIG. 6, it can be seen that the number of features is reduced (reduced) when using data sets 1 to 3 as compared to the case of random sampling.

  As described above, the learning model generated by the learning model generation apparatus 20 according to the present embodiment has the same detection performance as that of the conventional technique, as shown in the experimental results of FIGS. This learning model has a reduced number of features.

As described above, according to the learning model generation device 20 of the present embodiment, for each of a plurality of ranks 1 to 4 according to the ease of detection of the detection target or the degree of difficulty of detection, Rank 2 between rank 1 that is most easily detected and rank 4 that is most difficult to detect based on a plurality of learning images previously classified from rank 1 that is most easily detected to rank 4 that is most difficult to detect. The ratio of the number N ₂ of the plurality of learning images classified into rank 1 and rank 4 to the number N ₁ of the plurality of learning images classified into 3 is equal to or less than a predetermined value T. A learning model including a set of features (Haar-like features in this embodiment) corresponding to each of the plurality of learning images is selected based on the plurality of learning images selected. Is generated.

Moreover, according to the learning model generation apparatus 20 of the present embodiment, the highest detection target is obtained for each of the plurality of ranks 1 to 4 according to the degree of ease of detection or detection difficulty of the detection target. Based on a plurality of reference images 60a to 60d previously classified from rank 1 that is easy to detect to rank 4 that is most difficult to detect, each of the plurality of learning images is in any one of ranks 1 to 4. It is classified, based on the classified plurality of learning images, to the number N ₁ of detected most easily No. 1 and detected most difficult plurality of learning images classified into ranks 2, 3 between the No. 4 as the proportion of the number N ₂ of a plurality of learning images classified into the rank 1 and rank 4 is equal to or less than a predetermined value T, a plurality of learning images is selected, a plurality of learning images selected Each of the plurality of learning images based on (In this embodiment Haar-like feature) The number of features in accordance with the learning model is generated that includes a set of.

  The learning model generated by the learning model generation apparatus 20 according to the present embodiment has the same detection performance and the number of features as the experimental results shown in FIGS. This is a reduced learning model.

  Therefore, according to the learning model generation apparatus 20 according to the present embodiment, it is possible to generate a learning model in which the detection performance is comparable and the number of features is reduced as compared with the conventional technique.

  In the present embodiment, the number of ranks is four (1 to 4), but the present invention is not limited to this. For example, the number of ranks may be a plurality of numbers other than four.

Further, in step 106, according to the above equation (2), to the number N ₁ of a plurality of learning images classified into No. 2 and No. 3 between the detected most difficult No. 4 as easily No. 1 was detected most The learning images of a plurality of M ₁ (N ₁ + N ₂ ) detection objects so that the ratio of the number N ₂ of the plurality of learning images classified into rank 1 and rank 4 is equal to or less than a predetermined value T. Although the example which selects is demonstrated, this invention is not limited to this. For example, in step 106, according to equation (3) below, to the number N ₁ of a plurality of learning images classified into No. 2 and No. 3 between the detected most difficult No. 4 as easily No. 1 was detected most The learning images of a plurality of M ₁ (N ₁ + N ₂ ) detection objects so that the ratio of the number N ₂ of the plurality of learning images classified into rank 1 and rank 4 becomes a predetermined value T ′. May be selected.

Further, in step 106, according to equation (4) below, to the number N ₁ of a plurality of learning images classified into No. 2 and No. 3 between the detected most difficult No. 4 as easily No. 1 was detected most , A plurality of M ₁ (N ₁ + N) so that the ratio of the number N ₂ of the plurality of learning images classified into rank 1 and rank 4 falls within the range from the predetermined value T ″ to the predetermined value T ″ ′. ₂ ) You may make it select the image for learning of the detection target object of 1 sheet.

  It should be noted that each of the predetermined value T, the predetermined value T ′, the predetermined value T ″, and the predetermined value T ′ ″ described above has the same detection performance as compared with the conventional technique, for example, through experiments. A predetermined value T, or a predetermined value T ′, or a predetermined value T ″ and a predetermined value T ′ ″ when a learning model with a reduced number of features is generated in advance, What is necessary is just to set the value calculated | required previously.

DESCRIPTION OF SYMBOLS 10 Object detection system 20 Learning model production | generation apparatus 30 Target object detection apparatus 42 Image classification part 44 for learning Image selection part 46 for learning 46 Learning part 48 Learning model DB

Claims (7)

For each of a plurality of ranks according to the degree of ease of detection or detection difficulty of the detection object, a plurality of learning images pre-classified from the rank that is most easily detected to the rank that is most difficult to detect Based on the number of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect, a plurality of classifications that are classified into the rank that is most easily detected and the rank that is most difficult to detect Selection means for selecting a plurality of learning images so that the ratio of the number of learning images is equal to or less than a predetermined value;
Generating means for generating a learning model including a set of features corresponding to each of the plurality of learning images based on the plurality of learning images selected by the selection means;
A learning model generation apparatus including: Based on a plurality of reference images that are pre-classified from a rank that is most easily detected to a rank that is most difficult to detect, for each of a plurality of ranks according to the degree of ease or difficulty of detection of the detection object. Classifying means for classifying each of the plurality of learning images into any one of the ranks;
Based on the plurality of learning images classified by the classification means, the most easily detected with respect to the number of the plurality of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect. Selection means for selecting a plurality of learning images so that the ratio of the number of the plurality of learning images classified into the rank and the most difficult to detect is equal to or less than a predetermined value;
Generating means for generating a learning model including a set of features corresponding to each of the plurality of learning images based on the plurality of learning images selected by the selection means;
A learning model generation apparatus including:   The degree of ease of detection of the detection object increases as the resolution of the learning image increases, and decreases as the portion of the learning image hidden by the object other than the detection object increases. The learning model generation device according to claim 1, wherein the learning model generation device is determined so as to decrease as the degree of blur in the learning image increases or to decrease as the background noise in the learning image increases.   The degree of difficulty in detecting the detection object decreases as the resolution of the learning image increases, and increases as the portion of the learning image hidden by the object other than the detection object increases. The learning model generation apparatus according to claim 1 or 2, wherein the learning model generation device is determined so as to increase as a degree of blur in the learning image increases or as a background noise increases in the learning image. The learning model generation device according to any one of claims 1 to 4,
An object detection device comprising detection means for detecting an object from the input image based on the learning model and the input image generated by the learning model generation device;
Object detection system including Computer
For each of a plurality of ranks according to the degree of ease of detection or detection difficulty of the detection object, a plurality of learning images pre-classified from the rank that is most easily detected to the rank that is most difficult to detect Based on the number of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect, a plurality of classifications that are classified into the rank that is most easily detected and the rank that is most difficult to detect Selection means for selecting a plurality of learning images so that the ratio of the number of learning images is equal to or less than a predetermined value, and the plurality of learning images based on the plurality of learning images selected by the selection means A program for functioning as a generation means for generating a learning model including a number of features corresponding to each image. Computer
Based on a plurality of reference images that are pre-classified from a rank that is most easily detected to a rank that is most difficult to detect, for each of a plurality of ranks according to the degree of ease or difficulty of detection of the detection object. Classifying means for classifying each of the plurality of learning images into any one of the ranks;
Based on the plurality of learning images classified by the classification means, the most easily detected with respect to the number of the plurality of learning images classified into a rank between the rank that is most easily detected and the rank that is most difficult to detect. A selection unit that selects a plurality of learning images so that a ratio of the number of the plurality of learning images classified into a rank and a rank that is most difficult to detect is a predetermined value or less; and a plurality of selections selected by the selection unit A program for functioning as generation means for generating a learning model including a number of features corresponding to each of the plurality of learning images based on the learning images.