JP6448036B2 - Object region identification method, apparatus, and program - Google Patents

Description

  The present invention relates to an object region specifying method, apparatus, and program.

  With the spread of image sharing sites and the advancement of search engines, it has become possible to easily access large-scale image data uploaded on the web. For example, by performing a keyword image search for an arbitrary object (for example, “airplane”, “dog”, etc.) using a web search engine, it is possible to obtain a plurality of image data showing the object. In recent years, a technique for automatically constructing a “dictionary” related to the common object (hereinafter referred to as a target object) using the “image group in which objects in many images are commonly captured” as an input has been proposed in recent years. .

  For example, by extracting multiple rectangular candidate areas from each image included in the image set, and simultaneously evaluating and optimizing the target object likelihood and the similarity between candidate areas, the target objects that are included in the image group A technique for selecting a candidate region that best captures the image from each image is disclosed (for example, Non-Patent Document 1). In the technique of Non-Patent Document 1, an automatically constructed dictionary is a set of selected candidate areas (rectangular areas in an image). By using this as a dictionary, it is possible to construct a more accurate dictionary related to the target object than using the input image group as it is.

  As a similar technique, for example, a target object region can be accurately identified from an image set represented by a web image search result in which images not including the target object are mixed in a part of the image set. Which candidate area best identifies the target object based on a term that evaluates whether or not the target object is included in the image based on gender / identity and the similarity / identity between multiple candidate areas extracted from each image By solving a quadratic programming problem that includes a term that evaluates whether or not the image is captured, it is possible to find and remove an image that does not include the target object, and to identify an area that includes the target object (the rectangular candidate area in the image (For example, Non-Patent Document 2) is disclosed.

  Here, considering the conditions that an image area set (hereinafter referred to as an object area set) obtained as an output of the above-described dictionary automatic construction technique should have as a target object dictionary, it can be said that there are the following two points.

(I) Each region captures the “individual” of the target object. (Ii) The object region set comprehensively captures the target object included in the image set.

(I) indicates that, for example, when a plurality of target objects are included in an image, each obtained image area captures the area of each object. For example, considering the case where the dictionary is used as learning data for realizing an application such as object detection, it is obvious that the condition (i) is indispensable. In addition, it is assumed that a general object usually appears in various images (direction, size, lighting conditions, etc.) in the image as well as the appearance of the individual object. In order to build a dictionary that is robust to these individual differences / changes in appearance, it is necessary to include various appearance examples for various individuals in the dictionary. (Ii) is an indispensable condition for realizing this automatically.

T. Deselaers et al., “Weakly Supervised Localization and Learning with Generic Knowledge”, in IJCV, 2012. K. Tang et al., “Co-localization in Real-World Images”, in Proc. CVPR, 2014.

  However, in light of the above conditions, it can be seen that the techniques described in Non-Patent Document 1 and Non-Patent Document 2 do not necessarily satisfy these conditions. The techniques described in Non-Patent Document 1 and Non-Patent Document 2 are both formulated as a problem of selecting at most one region that is considered to capture the target object region most correctly from each image in the set. ing.

  If it is guaranteed that two or more target objects are not included in each image in the set, the above technique satisfies the conditions (i) and (ii), but for example the input image shown in FIG. When a plurality of target objects (here, dogs) are included in the image, the known technique cannot capture all the target objects in the image, and therefore the condition (ii) is not satisfied. There's a problem. Furthermore, as shown in the input image (c) in FIG. 9, when occlusion occurs between target objects, the well-known technique correctly captures how many individual objects each candidate area obtained from each image captures. Since it has not been evaluated at all, candidate areas in which a plurality of objects are connected may be output as a result, as shown on the right side of FIG. Therefore, there is a problem that the condition (i) is not always satisfied.

  As is clear from the above, the known technique for automatically constructing a “dictionary” relating to a target object using “an image group in which objects in many images are shown in common” as an input includes: In the case where a plurality of target objects are included, it is impossible to comprehensively grasp them, and (2) there are two problems that the selected object region does not necessarily capture the target object individual.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an object region specifying method, apparatus, and program capable of accurately specifying a target object included in an image.

  In order to achieve the above object, the object region specifying method of the present invention includes a candidate region extraction unit, a feature amount extraction unit, a duplication degree calculation unit, a model estimation unit, and an optimization unit, and for each of a plurality of images, An object area specifying method in an object area specifying device for specifying an area of a target object included in the image, wherein the candidate area extracting unit extracts a candidate area of the target object from each of the plurality of images; The feature amount extraction unit extracting a feature amount from the candidate region for each of the candidate regions extracted from each of the plurality of images by the candidate region extraction unit; Calculating a degree of overlap between the candidate regions for each pair of candidate regions extracted from the same image by the candidate region extracting unit; and Estimating a model for identifying whether to represent a target object based on the feature amount based on an object region set that is a subset extracted from a set of regions, and the optimization unit, The feature amount extracted for each of the candidate regions by the feature amount extraction unit, the redundancy degree calculated for each of the pairs by the redundancy degree calculation portion, and the model estimation portion estimated Based on the model, between the candidate areas included in the object area set using the function and a function representing the target object likelihood of the candidate area included in the object area set using the model and the feature amount Of the candidate region so as to optimize an objective function including a function that represents a degree of similarity and a function that represents a degree of overlap between candidate regions in the same image included in the object region set. Wherein extracting the object region set from engagement, wherein the model estimator is based on the object region set extracted by the optimization, estimating the model.

  The object region specifying device of the present invention is an object region specifying device that specifies a region of a target object included in the image for each of a plurality of images, and the candidate region of the target object is determined from each of the plurality of images. A candidate area extracting unit for extracting, a feature quantity extracting unit for extracting a feature quantity from the candidate area for each of the candidate areas extracted from each of the plurality of images by the candidate area extracting unit, and the candidate area For each pair of candidate regions extracted from the same image by the extraction unit, an overlap calculation unit that calculates the overlap between the candidate regions, and an object that is a subset extracted from the set of candidate regions A model estimation unit for estimating a model for identifying whether or not the target object is represented based on the feature amount based on the region set, and each of the candidate regions by the feature amount extraction unit. The model is used based on the feature amount extracted in the above, the redundancy calculated for each of the pairs by the redundancy calculation unit, and the model estimated by the model estimation unit. A function that represents the target object likelihood of the candidate area included in the object area set, a function that represents the degree of similarity between the candidate areas included in the object area set using the feature amount, and the object area set An optimization unit that extracts the object region set from the set of candidate regions so as to optimize an objective function including a function that represents a degree of overlap between candidate regions in the same image included in the image, The model estimation unit estimates the model based on the object region set extracted by the optimization.

  Further, the object region initialization unit extracts the feature amount extracted for each of the candidate regions by the feature amount extraction unit, and the degree of duplication calculated for each of the pairs by the duplication degree calculation unit. And a function that represents the target object likelihood of the candidate area included in the object area set, and a function that represents the degree of similarity between the candidate areas included in the object area set using the feature amount, Extracting the object region set from the set of candidate regions to optimize a function including a function that includes a function representing a degree of overlap between candidate regions in the same image included in the object region set, and The model estimation unit may estimate the model based on the object region set extracted by the object region initialization unit.

  The convergence determination unit further includes a step of repeating estimation by the model estimation unit and extraction by the optimization unit until a convergence condition is satisfied, and the model estimation unit is extracted by the object region initialization unit The model may be estimated based on the object region set or the object region set extracted by the optimization unit.

  The program of the present invention is a program for causing a computer to execute each step of the object region specifying method.

  As described above, according to the object region specifying method, apparatus, and program of the present invention, for each target object candidate region extracted from each of a plurality of images, a feature amount is extracted from the candidate region, For each pair of candidate regions extracted from the same image, calculate the degree of overlap between candidate regions and represent the target object based on the object region set, which is a subset extracted from the set of candidate regions Estimating a model for identifying, based on the feature amount extracted for each of the candidate regions, the redundancy calculated for each of the candidate region pairs, and the estimated model, Included in the object region set is a function that represents the target object likeness of the candidate region included in the object region set using the model, a function that represents the similarity between candidate regions included in the object region set using the feature amount To the same image By extracting an object area set from a set of candidate areas so as to optimize an objective function including a function representing the degree of overlap between candidate areas, it is possible to accurately identify target objects included in the image The effect of is obtained.

It is explanatory drawing for demonstrating the outline | summary of embodiment of this invention. It is explanatory drawing for demonstrating the outline | summary of embodiment of this invention. It is a block diagram which shows the functional structural example of the object area | region identification apparatus which concerns on this Embodiment. It is the figure which showed notation of an image notionally. It is explanatory drawing for demonstrating the candidate area | region extraction part of the object area | region identification apparatus which concerns on this Embodiment. It is explanatory drawing for demonstrating the object area initialization part of the object area specification apparatus which concerns on this Embodiment. It is explanatory drawing for demonstrating the model estimation part of the object area | region identification apparatus which concerns on this Embodiment. It is explanatory drawing for demonstrating the optimization part of the object area | region identification apparatus which concerns on this Embodiment. It is a flowchart which shows the object area | region specific process routine in this Embodiment. It is a figure which shows an example at the time of extracting an object area | region from an image using the conventional image segmentation technique.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline of Embodiment of the Present Invention>

  First, an outline of an embodiment of the present invention will be described. Embodiments of the present invention relate to a technique for automatically specifying an object region in a huge image.

  In the embodiment of the present invention, an effective image area (for example, an area of each target object) is automatically specified from “an easily obtainable but miscellaneous image group” represented by a web image group.

  FIG. 1 is a diagram for explaining an outline of an embodiment of the present invention. As shown in FIG. 1, the above image group (image group obtained by the keyword “airplane”) is miscellaneous in the following points (1) to (3).

(1) Images that do not include airplanes are mixed.
(2) Various visible objects (planes) are included.
(3) A plurality of objects are included per image (occlusion occurs).

  According to the embodiment of the present invention, more accurate recognition can be realized by specifying an area surrounded by a white frame as an object area and using it as teacher data, rather than using the web search result as it is. Can do.

  FIG. 2 is a diagram for explaining the outline of the embodiment of the present invention. In the embodiment of the present invention, by considering the following (1) to (3) at the same time, it is possible to specify a region capturing an individual target object as shown in FIG.

(1) Target object likeness of candidate area (2) Similarity between candidate areas (3) Degree of non-overlap between selected areas

  In the present embodiment, the target object area in the set of candidate areas can be comprehensively specified by alternately repeating the model estimation of the target object and the optimization described above.

  Specifically, the first idea of the present embodiment that solves the above problem (2) “the selected object region is not capturing the object individual” is that the object individual among the candidate regions in each image In order to select a target that is well captured, the target object likeness of the candidate area, the apparent similarity between the candidate areas, and the candidate areas selected as the target object areas are “not spatially overlapping” The candidate area is selected in consideration of the three factors of degree at the same time.

  Although the non-patent document 1 simultaneously evaluates two elements, ie, the target object likelihood of the candidate area and the similarity between the candidate areas, generally, the target object likelihood that can be calculated by a known technique is the candidate area. There is a problem that the number of objects actually included is almost irrelevant. Therefore, not only the candidate area capturing a single object but also a candidate area capturing a plurality of objects is simultaneously selected as the object area.

In the embodiment of the present invention, based on the fact that no spatial overlap occurs between ideal candidate regions capturing a single target object, in addition to the elements considered in the known technology, the selected candidate At the same time, an index for evaluating the degree of spatial non-overlap between regions is considered. As a result, it is possible to select a set of candidate areas that specifically capture a single object among candidate areas that are likely to be target objects.

  Estimate the model of the target object based on the set of candidate areas selected as described above, and select the object area set that captures the target object from the set of candidate areas based on the estimated model To do is considered complementary. The second idea of the present embodiment that solves the above problem (1) “when a plurality of target objects are included in an image, cannot be comprehensively specified” is the embodiment of the present invention. The selection of candidate areas based on the first idea and the construction of a model based on the set of selected candidate areas are performed alternately. Accordingly, it is possible to improve both the completeness of the target object correctly specified in the image set and the robustness of the target object model.

  According to the above two ideas, in the embodiment of the present invention, even if a plurality of target objects are included in the image of the input image set, the plurality of target objects are comprehensively captured while capturing the target object individual. An area can be specified.

<Configuration of Object Area Identification Device According to Embodiment of the Present Invention>
An object region specifying apparatus 100 according to an embodiment of the present invention includes a CPU (Central Processing Unit), a ROM (Read Only Memory) storing a program for the CPU to execute each processing routine described later, a RAM ( Random Access Memory). Functionally, the object region specifying apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 40 as shown in FIG. The object area specifying device 100 specifies the area of the target object included in each of the plurality of images.

  The input unit 10 receives a plurality of images.

  The calculation unit 20 includes an image set database 22, a candidate region extraction unit 24, a feature amount extraction unit 26, a redundancy calculation unit 28, an object region initialization unit 30, a model estimation unit 32, and an optimization unit 34. And a convergence determination unit 36.

  The image set database 22 stores a plurality of images received by the input unit 10.

  The candidate area extraction unit 24 extracts a candidate area of the target object from each of the plurality of images stored in the image set database 22.

Specifically, the candidate area extraction unit 24 sets a candidate area set x _ij from each image I _i included in the ^N image sets I = {I _i } _{i = 1} ^N stored in the image set database 22. ΕX _i is extracted. The candidate areas extracted here may overlap each other. J represents the index of the candidate area.

  As a method for extracting candidate areas, for example, the object candidate area extraction method disclosed in Reference Document 1 or Reference Document 2 below is applied to each image, and among the obtained candidate areas, targets that are calculated simultaneously What is necessary is just to extract the top M areas with high object likeness.

[Reference 1] P. Arbelaez et al., “Multiscale Combinatorial Grouping”, in Proc. CVPR, 2014.
[Reference 2] M.-M. Cheng et al., “BING: Binarized Normed Gradients for Objectness Estimation at 300fps”, in Proc. CVPR, 2014.

Alternatively, the candidate area extraction unit 24 evaluates the saliency of each candidate area in the image and defines the likelihood of the target object using a value representing the saliency. May be. Specifically, the candidate area extraction unit 24 first applies, for example, the object saliency calculation process disclosed in Reference 3 to each image, and the saliency map (the saliency score is calculated for each pixel in the image). Get the calculated map). Then, the candidate area extraction unit 24 may obtain the average value of the saliency scores of the pixels included in the candidate area as the target object likelihood s _ij of each candidate area.

[Reference 3] F. Perazzi et al., Saliency Filters: Contrast Based Filtering for Salient Region Detection, in Proc. CVPR, 2012.

In the present embodiment, as shown in FIG. 4, the candidate area extraction unit 24 calculates the target object likelihood s _ij of each candidate area x _ij as shown in FIG. Then, the candidate area extraction unit 24 extracts the top M areas having a high target object likelihood s _ij and extracts a set of candidate areas.

  The feature amount extraction unit 26 extracts a feature amount from the candidate region for each of the candidate regions extracted from each of the plurality of images by the candidate region extraction unit 24.

Specifically, the feature amount extraction unit 26 calculates an image feature amount f _ij from each candidate region x _ij of each image obtained by the candidate region extraction processing by the candidate region extraction unit 24. As an arbitrary image feature amount, for example, SPM, VLAD, GIST, and RCNN as shown in Reference Documents 4 to 7 below can be used.

[Reference 4] S. Lazebnik et al., “Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories”, in CVPR, 2006.
[Reference 5] H. Jegou et al., “Aggregating local descriptors into a compact image representation”, in CVPR, 2010.
[Reference 6] A. Oliva et al., “Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope”, in IJCV, 2001.
[Reference 7] R. Girshick et al., “Rich feature hierarchies for accurate object detection and semantic segmentation”, in Proc. CVPR, 2014.

  The overlapping degree calculating unit 28 calculates the overlapping degree between candidate areas for each candidate area pair extracted from the same image by the candidate area extracting unit 24.

Specifically, the overlap degree calculation unit 28 calculates the overlap degree of the candidate areas for each pair of sets of candidate areas in the image obtained by the candidate area extraction process by the candidate area extraction unit 24. The degree of overlap J (x _ij , x _{ij ′} ) between the candidate regions x _ij and x _{ij ′} can be calculated, for example, by the Jaccard similarity of the following equation (1).

Here, b _ij is a binary mask corresponding to the candidate region x _ij . As the binary mask, for example, a binary mask in which the candidate area is 1 and the area different from the candidate area is 0 can be used.

  The object region initialization unit 30 calculates the feature amount extracted for each candidate region by the feature amount extraction unit 26 and the overlap degree calculated for each pair of candidate regions by the overlap degree calculation unit 28. Based on the candidate area set extracted by the candidate area extraction unit 24, a subset of candidate areas that are considered to capture the target object is extracted as an object area set, and the object area set is initialized. The object area set is a subset extracted from the set of candidate areas.

  Specifically, as shown in FIG. 5, the object region initialization unit 30 selects a subset of candidate regions that are considered to capture the target object from the set of candidate regions extracted by the candidate region extraction unit 24. Extract as a set of object regions.

  For example, the object region initialization unit 30 uses the feature amount extracted for each candidate region by the feature amount extraction unit 26 and the degree of overlap calculated for each pair of candidate regions by the overlap degree calculation unit 28. Based on the above, an object region set is extracted from the set of candidate regions extracted by the candidate region extraction unit 24 so as to optimize a predetermined function.

  In this embodiment, the predetermined function represents the degree of similarity between the candidate area included in the object area set using the function indicating the target object likelihood of the candidate area included in the object area set and the feature amount. And a function representing the degree of overlap between candidate areas in the same image included in the object area set.

  Specifically, the object region initialization unit 30 considers the spatial redundancy between the candidate regions in the image at the time of initialization, and sets a set of candidate regions.

For the purpose of selecting a subset of a set of candidate regions that are “likely target objects, similar to each other and minimally overlap in space”, and defined by the following equation (2): A set of labels that minimizes the function

You can ask for. Note that the label y _ij means that it is selected as a region representing the target object when it is 1, and that it is not selected as a region representing the target object when it is 0.

The first term on the right side of the above equation (2) is a function representing the target object likelihood of the candidate areas included in the object area set. The function uses, for example, the target object likelihood value s _ij of each candidate region x _ij obtained by the candidate region extraction unit 24, for example,

Can be defined as β is a parameter.

The second term on the right side is a function that represents the degree of similarity between candidate regions included in the object region set using the feature amount, and is a term that evaluates the apparent similarity between candidate regions. A function representing the degree of similarity between candidate areas included in the object area set is obtained by using the feature quantity f _ij of each candidate area x _ij calculated by the feature quantity extraction unit 26, for example.

Can be defined as follows. λ _Ψ and σ _Ψ are parameters, and μ _Ψ is a term for evaluating the label compatibility of the candidate regions x _ij and x _{i′j ′.}

Can be defined as follows. μ _Ψ is a variable that takes 1 when the label of the candidate area x _ij and the candidate area x _{i′j ′} is different, and takes 0 when the label of the candidate area x _ij and the candidate area x _{i′j ′} is the same. .

The third term on the right side is a function representing the degree of overlap between candidate areas in the same image included in the object area set, and is a term for evaluating the degree of non-overlap between candidate areas. A function representing the degree of overlap between candidate areas in the same image included in the object area set is obtained by using, for example, the degree of overlap J (x _ij , x _{ij ′} ) between candidate areas obtained by the degree-of-redundancy calculation unit 28.

Can be defined as follows. λ _Ω and σ _Ω are parameters, and μ _Ω is a term for evaluating label compatibility of candidate regions x _ij and x _{i′j ′} , for example,

Can be defined as follows. The mu _Omega 1 when the label are both 1 and candidate region x _ij and the candidate region x _{i'j ',} a variable which takes 0 otherwise. Also, in the second term on the right side and the third term on the right side of the above formula (2)

  Represents summation for combinations of i, i ', j, and j' different from i = i 'and j = j'.

  The above equation (2) can be solved using a known algorithm such as Belief Propagation or TRW-S.

  The model estimation unit 32 estimates a model for identifying whether or not the target object is represented based on the feature amount based on the object region set that is a subset extracted from the set of candidate regions. Further, the model estimation unit 32 estimates a model based on the object region set extracted by the optimization process of the optimization unit 34 described later.

  Specifically, as shown in FIG. 6, the model estimation unit 32 uses the object region set initialized by the object region initialization unit 30 or the object region set extracted by the optimization unit 34 described later. Thus, the model z for identifying whether or not the target object is represented is estimated. An arbitrary model can be used as the model, and for example, a GMM, a latent SVM, a linear SVM, or the like can be used. For example, when using a linear SVM, specifically, the model parameter w of the linear SVM is learned using a feature amount set corresponding to a selected subset of candidate regions as a positive example.

  The feature amount used here may be the one extracted by the feature amount extraction unit 26, or may be newly extracted. Moreover, the selection of the negative example is arbitrary, and for example, it may be given as an average of feature amounts extracted from candidate regions other than the selected subset, or feature amounts concentrated from an external image may be used. .

  The optimization unit 34 includes the feature amount extracted for each of the candidate regions by the feature amount extraction unit 26, the degree of overlap calculated for each pair of candidate regions by the redundancy degree calculation unit 28, and model estimation. Based on the model estimated by the unit 32, an object region set is extracted from the set of candidate regions so as to optimize a predetermined objective function.

  In the present embodiment, the predetermined objective function is the candidate included in the object region set using the function that represents the target object likeness of the candidate region included in the object region set using the model and the feature amount. A function representing the degree of similarity between regions and a function representing the degree of overlap between candidate regions in the same image included in the object region set are included.

  Specifically, the optimization unit 34 sets a set of candidate areas obtained by the candidate area extraction process.

An object region set, which is a subset of candidate regions that closely captures the region of the target object individual, is extracted from. Here, the object region set is based on the results of the candidate region extraction unit 24, the feature amount extraction unit 26, the duplication degree calculation unit 28, and the model estimation unit 32. It is obtained by solving an optimization problem that selects a subset of a set of candidate regions such that they overlap only minimally.

  Optimization satisfying the above is, for example, a set of labels that minimizes the objective function defined by the following equation (6):

Can be formulated as a problem to find the following equation (6). FIG. 7 shows an objective function of the following equation (6).

The first term on the right side of the above equation is a function that represents the target object likelihood of the candidate area included in the object area set using the model, and is a term for evaluating the target object likelihood of each candidate area, for example. For example, a function representing the target object likelihood of a candidate area included in the object area set is obtained by the target object likelihood value s _ij obtained for each candidate area x _ij calculated by the candidate area extraction unit 24 and obtained by the model estimation unit 32. Using the model z and the feature value f _ij used for model estimation

Can be defined as w is a model parameter when the model z is estimated as a linear SVM, and β ′ is a parameter.

  The second term and the third term on the right side of the formula (6) may be defined in the same manner as the formula (4) and the formula (5), for example. The second term on the right side of Equation (6) is a function that represents the degree of similarity between the candidate regions included in the object region set using the feature amount. In addition, the third term on the right side of the above equation (6) is a function representing the degree of overlap between candidate areas in the same image included in the object area set. Moreover, said Formula (6) can be solved with well-known algorithms, such as believe propagation and TRW-S, similarly to said Formula (2).

  The convergence determination unit 36 repeats the estimation by the model estimation unit 32 and the extraction by the optimization unit 34 until a predetermined convergence condition is satisfied.

  Specifically, the convergence determination unit 36 performs convergence determination based on the object region set obtained as a result of processing by the optimization unit 34. When it determines with having converged, the obtained object area | region set is output as a result in the output part 40 mentioned later. If it is determined that it has not converged, the processes after the model estimation unit 32 are repeated based on the obtained object region set.

  The convergence determination condition is arbitrary. For example, the convergence condition is that there is no change between the object region set obtained by the immediately preceding optimization unit 34 and the object region set obtained one step before, or model estimation is performed. The convergence condition may be that the number of repetitions of the unit 32 and the optimization unit 34 exceeds the parameter t. The parameter t is arbitrary and may be set to t = 5, for example.

  The output unit 40 outputs the object region set obtained by the optimization unit 34 as a result.

<Operation of Object Region Identification Device According to Embodiment of Present Invention>
Next, the operation of the object region specifying apparatus 100 according to the embodiment of the present invention will be described. When a plurality of images are input from the input unit 10, the object region specifying apparatus 100 executes an object region specifying process routine shown in FIG.

  In step S <b> 100, the input unit 10 receives a plurality of images and stores them in the image set database 22.

  In step S102, the candidate area extraction unit 24 extracts a candidate area of the target object from each of the plurality of images stored in the image set database 22 in step S100.

  In step S104, the feature quantity extraction unit 26 extracts a feature quantity from the candidate area for each candidate area extracted from each of the plurality of images in step S102.

  In step S106, the degree-of-duplication calculation unit 28 calculates the degree of overlap between candidate areas for each pair of candidate areas extracted from the same image in step S102, according to the above equation (1).

  In step S108, the object region initialization unit 30 determines the feature amount extracted for each candidate region in step S104 and the overlap calculated for each pair of candidate regions in step S106. Based on the set of candidate areas extracted in step S102, a subset of candidate areas that are considered to be capturing the target object so as to minimize the function defined by the above formula (2) Extract as a region set and initialize the object region set.

  In step S110, the model estimation unit 32 determines whether to represent the target object based on the feature amount based on the object region set initialized in step S108 or the object region set extracted last time in step S112 described later. Estimate a model to identify

  In step S112, the optimization unit 34 determines the feature amount extracted for each candidate area in step S104, the degree of overlap calculated for each candidate area pair in step S106, and the step Based on the model estimated in S110, an object region set is extracted from the set of candidate regions so as to optimize the objective function shown in the above equation (6).

  In step S114, the convergence determination unit 36 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is satisfied, the process proceeds to step S116. On the other hand, if the convergence condition is not satisfied, the process proceeds to step S110.

  In step S116, the output unit 40 outputs the object region set obtained in step S112 as a result, and ends the object region specifying process routine.

  As described above, according to the object region specifying device according to the embodiment of the present invention, for each of the candidate regions of the target object extracted from each of the plurality of images, the feature amount is extracted from the candidate region, For each pair of candidate regions extracted from the same image, calculate the degree of overlap between candidate regions and represent the target object based on the object region set, which is a subset extracted from the set of candidate regions Estimating a model for identifying, based on the feature amount extracted for each of the candidate regions, the redundancy calculated for each of the candidate region pairs, and the estimated model, Included in the object region set is a function that represents the target object likeness of the candidate region included in the object region set using the model, a function that represents the degree of similarity between the candidate regions included in the object region set using the feature amount, and the like. In the same image So as to optimize an objective function that includes a function representing the degree of overlapping between the regions, by extracting the object region set from a set of candidate regions, the target object included in the image can be specified accurately.

  In addition, even when a plurality of target objects are included in the image, it is possible to specify a region of the target object while comprehensively capturing the target object and capturing the target object.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, in the above embodiment, as a method for initializing the object region set, the object region set is extracted from the candidate region set extracted by the candidate region extraction unit 24 so as to optimize a predetermined function. Although the case has been described as an example, the present invention is not limited to this. For example, based on the target object likelihood of each candidate area calculated by the candidate area extraction unit 24, K elements may be selected from candidate areas having a particularly large target object likelihood value from the set of all candidate areas. K is a parameter and can be set to K = 10, for example. Alternatively, the existing object region specifying method disclosed in Non-Patent Document 1 or Non-Patent Document 2 may be applied for the purpose of extracting at most one object region from each input image.

  Moreover, although the above-mentioned object area | region identification apparatus 100 demonstrated the case where the image collection database 22 was provided, the image collection database 22 is provided in the external device of the object area | region identification apparatus 100, for example, The image set database 22 may be referred to by communicating with an external device using a communication unit.

  Further, the object area specifying apparatus 100 described above has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. Shall be.

  Further, in the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium such as a CD-ROM or a memory card. It is.

DESCRIPTION OF SYMBOLS 10 Input part 20 Computation part 22 Image set database 24 Candidate area extraction part 26 Feature-value extraction part 28 Overlap degree calculation part 30 Object area initialization part 32 Model estimation part 34 Optimization part 36 Convergence determination part 40 Output part 100 Object area specification apparatus

Claims (5)

In an object region specifying apparatus that includes a candidate region extraction unit, a feature amount extraction unit, a duplication degree calculation unit, a model estimation unit, and an optimization unit, and specifies a region of a target object included in the image for each of a plurality of images An object region identification method comprising:
The candidate area extracting unit extracting a candidate area of the target object from each of the plurality of images;
The feature amount extraction unit extracting a feature amount from the candidate region for each of the candidate regions extracted from each of the plurality of images by the candidate region extraction unit;
The step of calculating the degree of overlap between the candidate areas for each of the pair of candidate areas extracted from the same image by the candidate area extraction unit;
A step of estimating a model for identifying whether or not the model estimation unit represents a target object based on the feature quantity based on an object area set that is a subset extracted from the set of candidate areas; ,
The optimization unit extracts the feature amount extracted for each of the candidate regions by the feature amount extraction unit, the degree of redundancy calculated for each of the pairs by the redundancy degree calculation unit, and Based on the model estimated by the model estimation unit, the object region set using the feature and a function representing the target object likelihood of the candidate region included in the object region set using the model and the feature amount The candidate area so as to optimize an objective function including a function representing a similarity degree between the candidate areas included in the object area and a function representing a degree of overlap between candidate areas in the same image included in the object area set. Extracting the object region set from a set of:
Including
The model estimation unit estimates the model based on the object region set extracted by the optimization. The object region initialization unit converts the feature amount extracted for each of the candidate regions by the feature amount extraction unit, and the overlap degree calculated for each of the pairs by the overlap degree calculation unit. A function that represents the target object likelihood of the candidate area included in the object area set, a function that represents the degree of similarity between the candidate areas included in the object area set using the feature amount, and the object Extracting the object region set from the set of candidate regions to optimize a function including a function that includes a function representing a degree of overlap between candidate regions in the same image included in the region set;
The object region specifying method according to claim 1, wherein the model estimation unit estimates the model based on the object region set extracted by the object region initialization unit. The convergence determination unit further includes a step of repeating estimation by the model estimation unit and extraction by the optimization unit until a convergence condition is satisfied,
The object according to claim 2, wherein the model estimation unit estimates the model based on the object region set extracted by the object region initialization unit or the object region set extracted by the optimization unit. Area identification method. An object region specifying device that specifies a region of a target object included in the image for each of a plurality of images,
A candidate area extraction unit that extracts a candidate area of the target object from each of the plurality of images;
A feature quantity extraction unit that extracts a feature quantity from the candidate area for each of the candidate areas extracted from each of the plurality of images by the candidate area extraction unit;
For each of the candidate area pairs extracted from the same image by the candidate area extraction section, a redundancy calculation section that calculates the overlap between the candidate areas;
A model estimation unit that estimates a model for identifying whether to represent a target object based on the feature amount based on an object region set that is a subset extracted from the set of candidate regions;
The feature amount extracted for each of the candidate regions by the feature amount extraction unit, the redundancy degree calculated for each of the pairs by the redundancy degree calculation unit, and estimated by the model estimation unit. Based on the model, the candidate region included in the object region set using the model and a function expressing the target object likelihood of the candidate region included in the object region set and the feature amount. The object region from the set of candidate regions so as to optimize an objective function including a function representing a similarity between them and a function representing a degree of overlap between candidate regions in the same image included in the object region set An optimization unit for extracting the set;
Including
The model estimation unit estimates the model based on the object region set extracted by the optimization.   The program for making a computer perform each step of the object area | region identification method of any one of Claims 1-3.