JP2018180646A - Object candidate area estimation device, object candidate area estimation method and object candidate area estimation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object candidate area estimation device, an object candidate area estimation method and an object candidate area estimation program which can estimate a candidate area which is a candidate of an area which suitably contains a single object in an image at high speed and with smaller number of the candidates.SOLUTION: A method includes steps of: accepting a visible light image and a depth image as inputs; estimating candidate area groups which consists of a plurality of candidate areas which are candidates of areas capturing objects, from the visible light image; deleting the candidate area determined that a substance projected in the candidate area is not the object, from the candidate area groups, based on the depth image; estimating object likelihood indicating the degree of a capture state in which the candidate area has caught a single object, about each of the candidate area which was not deleted, based on the depth change between pixels in the depth image; and correcting the shape of the candidate area so as to increase the object likelihood, about each of the candidate area which was not deleted.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an object candidate area estimation apparatus, an object candidate area estimation method, and an object candidate area estimation program for estimating a candidate area which is a candidate of an area including only a single object in an image.

  The object candidate area estimation is a technology for estimating an area likely to contain a single object in an image as a candidate area, and is widely used as a pre-processing for object detection to estimate the type and position of an object shown in an image. It is done.

  In object detection using object candidate area estimation (see Non-Patent Document 1 and Non-patent Document 2), a framework for performing object recognition (object type estimation) for each candidate area estimated by object candidate area estimation Object detection is performed. If it is possible to detect an object captured in an image captured by a smartphone or the like by object detection, such an application as intuitive information retrieval and presentation by superimposing and displaying detailed information on the captured object, related information, etc. Realization can be expected.

  In particular, it is difficult for a user who does not use the written language as the native language to obtain information on objects for which detailed information is presented in language, such as allergy information on product packages, route guidance information on signs, etc. The improvement of UX (user experience) by realizing intuitive information acquisition as an interface is considered to be great.

  As the simplest method of object candidate area estimation, a sliding window can be considered in which an area is cut out while scanning a rectangular frame of a predetermined size on the image. However, in this method, it is necessary to perform object recognition for a large number of candidate areas to be estimated, which increases the calculation cost.

  Also, in the case where the estimated candidate region includes only a part of the object, in the case where the region other than the object (a background unrelated to the object, other objects, etc.) is included in a large number, etc. This leads to recognition failure (see Non-Patent Document 3).

  Furthermore, in view of the fact that real-time processing is also important for seamless information presentation by object detection, object candidate area estimation needs to operate at high speed.

  From the above, it is required for object candidate area estimation to rapidly estimate candidate areas including a small number of candidate areas and including just enough individual objects in the image.

R. Girshick, et al., Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, in CVPR, 2014. K. He, et al., Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition, in ECCV, 2014. J. Hosang, et al., What Makes for Effective Detection Proposals ?, IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 38, no. 4, 2016. J. R. R. Uijings, et al., Selective Search for Object Recognition, Int. Journal on Computer Vision, vol. 104, no. 2, pp. 154-171, 2013. P. Arbelaez, et al., Multiscale Combinatorial Grouping. In CVPR, 2014. R. Achanta, et al., SLIC Superpixels Compared to State-of-the-art superpixel Methods, IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274-2282, 2012. C. Lawrence Zitnick et al., Edge Boxes: Locating Object Proposals from Edges, in ECCV, 2014. B. Alexe, et al., Measuring the Objectness of Image Windows, IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 24, no. 11, pp. 2189-2202, 2012. A. Kanezaki, et al., 3D Selective Search for Obtaining Object Candidates, in IROS, 2015. J. Liu, et al., Depth-aware Layered Edge for Object Proposal, in ICME, 2016.

  The known object candidate area estimation is a method of integrating small areas (see non-patent document 4 and non-patent document 5) and a method of calculating object likeliness of area (see non-patent documents 6 and 7) It is divided roughly.

  In the method of integrating small areas, as shown in FIG. 6 as an example, first, image features such as color and size are extracted for small areas generated by Superpixel (see Non-Patent Document 8). Then, small regions having similar image features are sequentially integrated, and regions generated in the process are estimated as candidate regions. In order to estimate a candidate area with high accuracy by such a method, it is necessary to perform feature extraction and similarity calculation from a small area for a huge number of integration procedures, and a lot of calculation time is required.

  On the other hand, in the method of calculating the object likeness of a region, as shown in FIG. 7 as an example, for each region cut out by the sliding window, it is calculated how much the region includes just enough objects (object likeness) Then, the regions are ranked in descending order of object likeness, and several regions with higher object likeness are output as candidate regions. Many object resemblance calculation methods aim to increase the object resemblance to the area circumscribed to the object border, and calculate the object resemblance from image border information such as edges and contrast extracted from the area edge. There is.

  However, in this method, the region including a plurality of objects is also a region circumscribing the boundary of the objects, so the object-likeness of the region including the plurality of objects is also enhanced. Therefore, there is a problem that many candidate regions have to be output in order to estimate a region that includes just a single object.

  That is, although the center of the region including a plurality of objects includes the boundary of the object, the boundary of the image includes not only the boundary of the object but also the boundary extracted from the pattern of the object. As a result, even when a single object is included without excess or deficiency, the central portion of the region includes the image boundary extracted from the pattern of the object. Therefore, it is difficult to estimate whether the area includes a plurality of objects from the boundary of the image included in the center of the area.

  As described above, in the case of estimating a region including just a single object using the known object candidate region estimation method, the method of integrating small regions requires a large amount of calculation, and the object-likeness of the region is calculated. There is a problem in that there are many areas where a single object is not accurately captured.

  In addition to the image to be estimated, a method using a depth image has also been proposed in order to realize more accurate object candidate region estimation. A depth image is an image in which a distance (depth) from a camera to a certain pixel of a visible light image is embedded, and can be acquired by a depth sensor, a stereo camera or the like.

  The technology disclosed in Non Patent Literature 9 utilizes depth images in a method of integrating small regions. In this method, high accuracy is achieved by calculating not only the feature obtained from the image as the feature to be extracted from the small region but also the volume of the small region. However, as in the case of using only an image, it is necessary to calculate the degree of similarity for a combination of a large number of small areas for high accuracy estimation, and it is considered that it still requires a large amount of calculation time.

  The technique disclosed in Non-Patent Document 10 divides an image into a plurality of layers according to depth, and integrates results obtained by calculating the object likeness in each layer, thereby including a plurality of objects having widely different distances. It suppresses the generation of the area. However, since only image boundaries are used to calculate the object likeness, many regions including multiple objects are still estimated in situations where multiple objects are included in the same layer, so there is no shortage of single objects. There is a problem that many candidate regions have to be output in order to estimate the region included.

  As described above, in the object candidate area estimation using the depth image, the same problem as in the case of using only the image to be estimated remains.

  The present invention has been made in view of the above circumstances, and it is possible to estimate candidate areas that are candidates for areas including a single object in an image without excess or short with high speed and a small number of candidates. An object candidate area estimation apparatus, an object candidate area estimation method, and an object candidate area estimation program are provided.

  In order to achieve the above object, an object candidate area estimation apparatus according to the present invention receives a visible light image and a depth image corresponding to the visible light image as input, and a plurality of candidates for an area in which an object appears from the visible light image. And an initial candidate area estimation unit for estimating a candidate area group consisting of candidate areas, and deleting from the candidate area group the candidate areas that are determined not to be the object based on the depth image. The degree to which the candidate area captures a single object for each of the candidate areas not deleted by the candidate area reduction unit based on the candidate area reduction unit and the change in depth between pixels in the depth image The object likelihood estimating unit estimates each of an object likelihood estimating unit that estimates an object likelihood to be shown, and the candidate regions that are not deleted by the candidate region reducing unit As serial object likelihood is high, including a candidate area correcting unit for correcting the shape of the candidate region.

  Note that the object likeness estimation unit is configured to calculate the edge portion of the candidate region based on the depth boundary extracted by the edge detection on the depth image for each of the candidate regions not deleted by the candidate region reduction unit. The object likeliness may be estimated using an index which is higher as the density of the depth boundary is higher and the density of the depth boundary inside the candidate area is lower.

In the object likeness S (r), the area at the edge of the candidate area is r ₁ , the area inside the candidate area is r _2, and φ (r) is the density of the depth boundary in the area r.

If is a standard sigmoid function, it may be expressed by the following equation.

  Further, the candidate area reduction unit determines whether or not the object in the candidate area is the object based on the size in real space of the object in the candidate area obtained using the depth image. The candidate area determined to be not the object in the candidate area may be deleted from the candidate area group.

  Further, the candidate area correction unit is configured to, for each of the candidate areas not deleted by the candidate area reduction unit, the object of the candidate area of each shape obtained by deforming the shape of the candidate area by a plurality of deformation methods. The likelihood may be estimated, and the shape of the candidate area may be corrected so that the estimated likelihood of the object has the highest shape.

  In order to achieve the above object, an object candidate area estimation method according to the present invention includes an initial candidate area estimation unit, a candidate area reduction unit, an object likelihood estimation unit, a candidate area correction unit, and an object candidate area An object candidate area estimation method in an estimation apparatus, wherein the initial candidate area estimation unit receives a visible light image and a depth image corresponding to the visible light image, and is a candidate for an area in which an object appears from the visible light image. Estimating the candidate area group including a plurality of candidate areas, and the candidate area determined by the candidate area reduction unit not to be the object according to the candidate area based on the depth image, Removing from the region group, and each of the candidate regions not deleted by the candidate region reduction unit based on a change in depth between pixels in the depth image, and the object likeness estimation unit Estimating the likelihood of the object indicating the degree to which the candidate area captures a single object, and the candidate area correction unit does not delete the candidate area for each of the candidate areas not deleted by the candidate area reduction unit. Correcting the shape of the candidate area such that the object likelihood estimated by the object likeness estimation unit is high.

  In order to achieve the above object, an object candidate area estimation program of the present invention is a program for causing a computer to function as each part of the object candidate area estimation apparatus.

  According to the present invention, it is possible to estimate candidate areas which are candidates for an area including just a single object in an image with high speed and a small number of candidates.

It is a block diagram which shows the functional structure of the object candidate area | region estimation apparatus which concerns on embodiment. It is a schematic diagram for demonstrating the estimation method of object likeness which concerns on embodiment. It is a schematic diagram which shows the area | region of the edge of the candidate area | region which concerns on embodiment, and the area | region inside. It is a flow chart which shows a flow of object candidate field presumed processing concerning an embodiment. It is a flowchart which shows the flow of the subroutine of the correction process which concerns on embodiment. It is a schematic diagram for demonstrating the method to integrate the conventional small area | region. It is a schematic diagram for demonstrating the method to calculate the object likelihood of the conventional area | region.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a functional configuration of the object candidate area estimation apparatus 10 according to the first embodiment. As shown in FIG. 1, the object candidate area estimation device 10 according to the first embodiment includes an initial candidate area setting unit 12, a candidate area reduction unit 14, an object likelihood estimation unit 16, a candidate area correction unit 18, and a candidate area A storage unit 20 is provided.

  The initial candidate area setting unit 12 receives a visible light image and a depth image corresponding to the visible light image, and estimates, from the visible light image, a candidate area group consisting of a plurality of candidate areas that are candidates for the area in which the object appears. . In this embodiment, a plurality of areas of a predetermined size at a plurality of predetermined positions in the visible light image are set as candidate areas. For example, on the visible light image, a plurality of candidate areas have rectangular frames (for example, one side of 300 × 400 [pixel]) of a predetermined size and a predetermined interval (for example, rectangular width). It is set by scanning at intervals of 50%.

  The candidate area reduction unit 14 deletes, from the candidate area group, the candidate area determined to be not an object in the candidate area based on the depth image.

  In the present embodiment, the candidate area reduction unit 14 determines whether or not the object in the candidate area is an object based on the size in real space of the object in the candidate area obtained using the depth image. The candidate area determined to be not an object that is reflected in the area is deleted from the candidate area group. As described above, by deleting a candidate area in which the object is not captured from the candidate area group, it is possible to further increase the ratio of the area in which the object is captured in the candidate area output as the estimation result.

Specifically, the candidate area reduction unit 14 calculates the size of the candidate area in the real space from the depth image for each of the candidate areas, and sets the range of the size set in advance as the size of the object. The candidate area which deviates is deleted from the candidate area group. For example, in the case where restriction is provided by the height of the region in the real space, the height r _{h ′} of the region r in the real space can be calculated by the following equation (1).

... (1)

Here, r _d is a representative value of the depth in the region r, and for example, the median value of the depth in the region or the like is used. Further, r _h is the height in the image region r, f _y is the internal parameters of the camera, the focal length of the vertical pixels, acquired by camera calibration.

  The object resemblance estimation unit 16 determines the extent to which the candidate area captures a single object without excess for each of the candidate areas not deleted by the candidate area reduction unit 14 based on the change in depth between pixels in the depth image. Estimate the object likeness.

  In the present embodiment, as shown in FIG. 2 as an example, edge detection (see, for example, reference 1 below) is performed on the depth image 32 based on the depth image 32 corresponding to the input visible light image 30. Thus, the depth boundary image 34 representing the depth boundary in the visible light image 30 is generated. In the edge detection with respect to the depth image 32, a point at which a change in depth between pixels is large (for example, a point at which a change in depth between pixels is equal to or more than a predetermined threshold) is detected as a depth boundary.

Then, the object likelihood estimation unit 16, based on the depth border image 34, for each of the candidate regions, as shown in FIG. 3 as an example, depth boundaries many regions r ₁ of edge candidate areas 35, and more area depth boundaries within the region r ₂ is less candidate areas, in order to calculate the index of the object likelihood that seems object, the interior of the density of the depth boundaries in region r ₁ of edge candidate areas 35 and candidate regions 35 The object likeness is estimated using the density of the depth boundary in the region r ₂ of

  Here, the object likeness is expressed using an index which increases as the density of the depth boundary of the edge of the candidate area is larger and the density of the depth boundary inside the candidate area is smaller. In the present embodiment, the object likeness estimation unit 16 estimates the object likeness S (r) defined by the following equation (2).

... (2)

In the above equation (2), φ (·) is the density of the depth boundary in a certain area, and the density φ (r) is always 0 or more, so the standard sigmoid function

Is normalized to the range of [0.5, 1].

  The number of calculations of the object likeness increases in proportion to the number of candidate areas. However, by calculating the integral image of the depth image in advance (see, for example, reference 2 below), the calculation of the density of the depth boundary can be calculated at high speed for any region, so the object likeness calculation is performed. Can be done at high speed.

[Reference 1] J. Canny, A Computational Approach to Edge Detection, IEEE Trans. On Pattern Analysis and Meachine Inteligence, vol. 8, no. 6, pp. 679-698, 1986.

[Reference 2] F. C. Crow, Summed-Area Tables for Texture Mapping, in SIGGRAPH, 1984.

  The candidate area correction unit 18 corrects the shape of the candidate area so that the likelihood of an object estimated by the object likeness estimation unit 16 is high for each of the candidate areas not deleted by the candidate area reduction unit 14.

  In the present embodiment, the candidate area correction unit 18 determines, for each candidate area not deleted by the candidate area reduction unit 14, the object likelihood of the candidate area of each shape obtained by deforming the shape of the candidate area by a plurality of deformation methods. Is estimated, and the shape of the candidate area is corrected so that the estimated likelihood of the object has the highest shape. Further, the candidate area correction unit 18 stores data indicating the candidate area whose shape has been corrected in the candidate area storage unit 20. In this manner, the positions, shapes, and the like of the candidate areas included in the candidate area group are corrected so as to increase the object-likeness so that each candidate area catches an individual object more accurately, and is ranked according to the object-likeness. The candidate area is output as the final estimation result.

  At this time, it is also possible to select and correct the shape that gives the highest object likelihood among all the shapes that can be considered as the shape of the candidate area, but in that case, the object likeness of a large number of times is calculated It is necessary to do so and the computational cost will be high.

  Therefore, for example, it is conceivable to greedily search for the shape of the candidate area. Specifically, the correction width of the candidate area is a predetermined value (for example, 50% of the area width, etc.), and each side of the candidate area is increased or decreased by the correction width, vertically or horizontally for the most object likelihood. The candidate area is corrected to have a high shape. After that, the correction width is previously determined by reducing the correction width by a predetermined amount (for example, 50% of the currently set correction width) and further correcting the candidate area into a shape with high object-likeness. By repeating the process until it becomes equal to or less than the threshold value, it is possible to correct the candidate area while suppressing the number of times of calculating the object likeness.

  The candidate area to be output may be an area of an arbitrary shape (polygon, ellipse or the like), but in the present embodiment, it is output as a rectangular area for the sake of simplicity.

  The object candidate area estimation apparatus 10 according to the present embodiment is, for example, a computer apparatus provided with a central processing unit (CPU), a random access memory (RAM), and a read only memory (ROM) for storing various programs. Moreover, the computer which comprises the object candidate area | region estimation apparatus 10 may be equipped with memory | storage parts, such as a hard disk drive and non-volatile memory. In the present embodiment, when the CPU reads and executes the program stored in the storage unit such as the ROM and the hard disk, the above-described hardware resource and the program cooperate with each other to realize the above-described function.

  The flow of object candidate area estimation processing by the object candidate area estimation apparatus 10 according to the present embodiment will be described using the flowchart shown in FIG. In the present embodiment, the object candidate area estimation process is started at a timing when predetermined information for starting the execution of the object candidate area estimation process is input to the object candidate area estimation device 10. The timing at which the processing is started is not limited to this. For example, the object candidate area estimation processing may be started at the timing when the visible light image and the depth image are input.

  In step S101, the initial candidate area setting unit 12 receives a visible light image and an input of a depth image corresponding to the visible light image, and estimates a candidate area group including a plurality of candidate areas from the visible light image.

  In step S103, based on the depth image, the candidate area reduction unit 14 deletes, from the candidate area group, candidate areas that are determined not to be objects in the candidate area.

  In step S105, the object likelihood estimation unit 16 estimates the object likeness for each of the candidate regions not deleted by the candidate region reduction unit 14 based on the change in depth between the pixels in the depth image.

  In step S107, the candidate area correction unit 18 corrects the shape of the candidate area so that the likelihood of an object estimated by the object likeness estimation unit 16 is high for each of the candidate areas not deleted by the candidate area reduction unit 14 Perform correction processing.

  In step S109, the candidate area correction unit 18 outputs data indicating the candidate area whose shape has been corrected, and ends the execution of the program of the object candidate area estimation process. In the present embodiment, data indicating a candidate area whose shape is corrected is displayed on a display unit such as a display, data indicating a candidate area whose shape is corrected is transmitted to an external device, or the shape is corrected. By storing data indicating the candidate area in the storage unit, data indicating the candidate area whose shape has been corrected is output.

  Next, the flow of a subroutine of the above-described correction processing by the object candidate region estimation apparatus 10 according to the present embodiment will be described using the flowchart shown in FIG. The flowchart shown in FIG. 5 is executed for each of the candidate areas.

  In step S201, the candidate area correction unit 18 deforms the shape of the candidate area into a plurality of shapes by a plurality of deformation methods. In the present embodiment, although the case where each side of the candidate area is increased or decreased with a predetermined correction width is described, the present invention is not limited thereto. As a method of deforming the shape of the candidate area, for example, a method of increasing or decreasing the candidate area at a predetermined ratio may be mentioned.

  In step S203, the candidate area correction unit 18 estimates the object likeness of the candidate area of each of the deformed shapes.

  In step S205, the candidate area correction unit 18 corrects the shape of the candidate area so that the estimated shape of the object has the highest shape, and stores data indicating the candidate area whose shape is corrected in the candidate area storage unit 20.

  In step S207, the candidate area correction unit 18 determines whether the correction width of each side of the candidate area is equal to or less than a predetermined threshold. Here, the predetermined threshold is a value indicating how finely the correction is to be performed, and is, for example, a value previously input by the user.

  If it is determined in step S207 that the correction width of each side of the candidate area is not equal to or less than the threshold (S207, N), the process proceeds to step S209. If it is determined in step S207 that the correction width of each side of the candidate area is equal to or less than the threshold value, the execution of the program of the present correction processing is ended.

  In step S209, the candidate area correction unit 18 reduces the correction width of each side of the candidate area, and proceeds to step S201.

  As described above, in the present embodiment, a visible light image and a depth image corresponding to the visible light image are input, and a candidate area group including a plurality of candidate areas is estimated from the visible light image. Moreover, the candidate area | region determined that what is reflected to a candidate area | region is not an object based on a depth image is deleted from a candidate area group. Also, based on the change in depth between pixels in the depth image, an object likeness indicating the degree to which the candidate area captures a single object is estimated for each of the candidate areas not deleted. Then, for each of the candidate areas not deleted, the shape of the candidate area is corrected such that the likelihood of an object is high.

  At this time, the number of depth boundaries in the candidate area is calculated by performing edge detection on the candidate area, which is used when calculating the object-likeness of objects appearing in the candidate area, and By calculating the sum at high speed using the integral image, the object likeness can be calculated at high speed.

  As described above, according to the present embodiment, a region circumscribed to the boundary of each object shown in the visible light image and not including the boundary of the object, that is, a candidate region including just a single object Can be estimated quickly and with a small number of candidates.

  In this embodiment, the operation of the component of the function shown in FIG. 1 is constructed as a program and installed in a computer used as the object candidate area estimation apparatus 10 and executed, but the present invention is not limited thereto. You may distribute it.

  Further, the constructed program may be stored in a portable storage medium such as a hard disk or a CD-ROM, and may be installed in a computer or distributed.

10 Object candidate area estimation device 12 Initial candidate area setting unit 14 Candidate area reduction unit 16 Object likelihood estimation unit 18 Candidate area correction unit 20 Candidate area storage unit

Claims (7)

An initial candidate area estimation unit configured to receive a visible light image and a depth image corresponding to the visible light image and estimate a candidate area group consisting of a plurality of candidate areas that are candidates for an area in which an object appears from the visible light image;
A candidate area reduction unit which deletes, from the candidate area group, the candidate area determined to be not the object based on the depth image and determined to be not the object in the candidate area;
An object likeness indicating the degree to which the candidate area captures a single object is estimated for each of the candidate areas not deleted by the candidate area reduction unit based on a change in depth between pixels in the depth image. An object likeness estimation unit,
A candidate area correction unit that corrects the shape of the candidate area such that the likelihood of the object estimated by the object likeness estimation unit becomes high for each of the candidate regions not deleted by the candidate area reduction unit;
An object candidate area estimation apparatus including: The object likeness estimation unit is configured to determine the depth boundary of the edge of the candidate region based on the depth boundary extracted by the edge detection on the depth image for each of the candidate regions not deleted by the candidate region reduction unit. The object candidate area estimation apparatus according to claim 1, wherein the likelihood of the object is estimated using an index that increases as the density of the target area increases and the density of the depth boundary inside the candidate area decreases. In the object likeness S (r), an area at the edge of the candidate area is r ₁ , an area inside the candidate area is r _2, and φ (r) is a density of the depth boundary in the area r.

The object candidate area estimation apparatus according to claim 2, which is expressed by the following equation, where is a standard sigmoid function.
The candidate area reduction unit determines whether or not the object in the candidate area is the object based on the size in real space of the object in the candidate area obtained using the depth image. The object candidate area | region estimation apparatus in any one of Claims 1-3 which deletes the said candidate area | region determined that what is reflected to an area | region is not the said object from the said candidate area group. The candidate area correction unit determines, for each of the candidate areas not deleted by the candidate area reduction unit, the object likeness of the candidate area of each shape obtained by deforming the shape of the candidate area by a plurality of deformation methods. The object candidate area estimation device according to any one of claims 1 to 4, wherein the shape of the candidate area is corrected so as to be estimated and the shape of the estimated object likelihood is the highest. An object candidate area estimation method in an object candidate area estimation apparatus having an initial candidate area estimation unit, a candidate area reduction unit, an object likeness estimation unit, a candidate area correction unit, and a unit,
The initial candidate area estimation unit receives a visible light image and a depth image corresponding to the visible light image, and estimates a candidate area group consisting of a plurality of candidate areas that are candidates for an area in which an object appears from the visible light image Step to
Deleting, from the candidate area group, the candidate area determined by the candidate area reduction unit not to be the object based on the depth image and determined that the object in the candidate area is not the object;
The degree to which the candidate area captures a single object for each of the candidate areas not deleted by the candidate area reduction unit based on a change in depth between pixels in the depth image Estimating an object likeness indicating
The candidate area correction unit corrects the shape of the candidate area so that the likelihood of the object estimated by the object likeness estimation unit is high for each of the candidate areas not deleted by the candidate area reduction unit. Step and
An object candidate area estimation method including:   The object candidate area | region estimation program for functioning a computer as each part of the object candidate area | region estimation apparatus in any one of Claims 1-5.