JP6641313B2 - Region extraction device and program - Google Patents

Description

  The present invention is suitable for speeding up and increasing accuracy of visual hull generation and free viewpoint video generation based thereon, even in a case where occlusion of an object occurs in a multi-view camera image in which a plurality of objects are captured. The present invention relates to an area extracting device and a program capable of extracting a spatial area of an individual object.

  In the generation of a model-based free viewpoint video, a three-dimensional shape model (Visual Hull, Visual Hull, hereinafter abbreviated as “VH” as appropriate) in a voxel space of an object captured from a multi-view camera video. Is used. In order to generate a free viewpoint video at high speed and with high accuracy, it is desired to generate VH at high speed and with high accuracy.

  Non-Patent Literatures 1 and 2 relate to speeding up and increasing accuracy of VH generation.

  In Non-Patent Document 1, aiming at real-time VH generation, as a first approach, an initial rectangular parallelepiped region (bounding box) in a voxel space for VH generation is calculated in advance and stored in a lookup table. As a second approach, the image is downsampled so that the center point of the voxel is approximately projected to one pixel of the image. FIG. 8 is a schematic diagram of the initial area (bounding box).

  Non-Patent Document 2 proposes a robust and accurate calculation of VH, which includes two steps of estimating an initial rectangular parallelepiped region (bounding box) and detecting an intersection in the visual volume intersection method.

Alexander Ladikos, Selim Benhimane, and Nassir Navab. "Efficient Visual Hull Computation for Real-Time 3D Reconstruction using CUDA", Proceedings of 2008 Conference on Computer Vision and Pattern Recognition Workshops, pp. 1-8, 2008. Peng Song, Xiaojun Wu, and Michael Yu Wang. "A robust and Accurate Method for Visual Hull Computation", International Conference on Information and Automation, pp. 784-789, 2009.

  As described above, appropriate setting of the initial region with respect to voxel density and the like is indispensable for increasing the speed and accuracy of VH generation.

  However, in each of the conventional techniques, as shown schematically in FIG. 9, when a plurality of objects are captured in a multi-view camera image, the plurality of objects are used as an appropriate initial region (bounding box) for VH generation. Could not get an initial area. That is, FIG. 9 shows an example in which there are two objects, and in the conventional method, only one initial area can be obtained even if there are two objects as in [2]. This is because occlusion of a plurality of objects occurs in a camera image due to the presence of a plurality of regions, and the conventional method cannot cope with such a case. As shown in [1], it is desired to extract an initial region corresponding to each of two objects.

  The present invention has been made in view of the above-described problems of the related art, and provides an area extraction device and a program that can respectively extract a space area of an individual object even when occlusion of the object occurs in a multi-view camera image obtained by capturing a plurality of objects. The purpose is to do.

  In order to achieve the above object, the present invention relates to an area extracting apparatus, which extracts one or more image areas including an object from each camera image of a plurality of captured multi-view camera images. An extraction unit, and a correspondence acquisition unit that determines which of the plurality of objects the one or more extracted image regions correspond to in each camera image, along with a correspondence relationship between the camera images. A second extraction unit configured to extract, based on the extracted image area in each camera image linked in correspondence, a spatial area including the target in the correspondence. Further, it is a program for causing a computer to function as the region extraction device.

  ADVANTAGE OF THE INVENTION According to this invention, even when several objects are image | photographed in a multi-view camera image, the space area of an individual object can be each extracted appropriately.

It is a functional block diagram of an area extraction device concerning one embodiment. FIG. 4 is a diagram illustrating a schematic example for describing processing of each unit of the region extraction device. FIG. 4 is a diagram illustrating a schematic example for describing processing of each unit of the region extraction device. It is a flowchart of operation | movement of the correspondence acquisition part which concerns on one Embodiment. It is a figure for explaining processing of the 2nd extraction part. It is a figure for explaining processing of the 2nd extraction part. 5 is a table showing the effect of the present invention in comparison with a conventional method. It is a schematic diagram of an initial area (bounding box) in VH generation. FIG. 9 is a schematic diagram for explaining a problem of the conventional method.

  FIG. 1 is a functional block diagram of an area extraction device 10 according to one embodiment. The region extraction device 10 includes a first extraction unit 1, a correspondence acquisition unit 2, a second extraction unit 3, and an additional processing unit 4. The first extracting unit 1 further includes a contour extracting unit 11, a representative point determining unit 12, a target number determining unit 13, and an image area determining unit 14. The processing outline of each unit is as follows.

  In the first extraction unit 1, images P (1), P (2), ..., P () of each camera viewpoint k (k = 1, 2, ..., N) configured as a multi-view camera image N), and at least one region including the target in each image P (k) is set as an image region, and its associated information (information on the representative point position of each target, information on the number of targets, etc.) and a corresponding acquisition unit 2 and output to the second extraction unit 3. A plurality of objects are photographed in the multi-view camera image. The multi-view camera image can be configured as an image at each time t in the video. However, in the following description, reference to time is omitted as an image P (k) at an arbitrary time t.

  The processing outline of each unit of the first extraction unit 1 is as follows. The contour extracting unit 11 extracts one or more target contours from each camera image P (k) of the multi-view camera image, and outputs the extracted contours to the representative point determining unit 12, the target number determining unit 13, and the image area determining unit 14. I do. The representative point determination unit 12 determines a representative point from the extracted outline of each object and outputs the representative point to the correspondence acquisition unit 2. The number-of-objects determination unit 13 determines the number of a plurality of objects captured in the multi-view camera image by examining the number of the extracted contours in each image P (k), and sends the number to the second extraction unit 3. Output. The image area determination unit 14 determines an image area of each target as an area including the extracted outline, and outputs the image area to the correspondence acquisition unit 2 and the second extraction unit 3.

  In the correspondence acquisition unit 2, one or more image regions including the object of each image P (k) obtained by the first extraction unit 1 correspond to any of a plurality of objects captured in the entire multi-view camera image. Is determined together with the determination of the target correspondence between the different camera images P (i), P (j) (i ≠ j), and the obtained correspondence is output to the second extraction unit 3. I do.

  In the second extraction unit 3, the correspondence between the objects among the images P (k) obtained by the correspondence acquisition unit 2, and each object in each image P (k) obtained by the image region determination unit 14. Based on the included image area, a spatial area including each object is determined and output in the voxel space where the multi-view camera image is captured. The determined spatial region can be used as an initial region when generating a target VH by the visual volume intersection method. According to the present invention, in particular, a case where a plurality of targets are photographed in a multi-view camera image, occlusion of the target occurs in some camera images P (k), and the regions overlap each other, etc. However, it is possible to obtain a spatial region having a size optimized for each object.

  The additional processing unit 4 can perform any additional processing using the spatial region of each target extracted by the second extraction unit 3. As an additional process, a target VH may be generated from the spatial region of each target, and the generated VH may be used to freely correspond to a multi-view camera image (video) input to the region extraction device 10. A viewpoint video may be generated. For the processing of VH generation and free viewpoint video generation in the additional processing unit 4, any existing method may be used.

  Hereinafter, the processing of each unit in FIG. 1 will be described in detail. FIG. 2 and FIG. 3 are diagrams showing schematic examples of the data obtained by the processing of the respective parts separately in [1] to [3] in FIG. 2 and [4] and [5] in FIG. , Will be referred to as appropriate in the following description. In FIGS. 2 and 3, the multi-view camera image captures two soccer players as an example of a plurality of targets, and the multi-view camera image input to the region extraction device 10 is configured by eight camera images. In this case, a schematic example of data obtained by each unit in FIG. 1 is shown. In FIGS. 2 and 3, each data obtained in each image of the camera 1 to the camera 8 is enumerated, but only [4] of FIG. Note that the arrangement is changed from the other [1] to [3], [5].

  The contour extraction unit 11 extracts one or more target contours from each camera image P (k) of the multi-view camera image, and outputs the extracted contours to the units 12, 13, and 14. For the contour extraction, any existing method may be used, for example, foreground / background region division using a mixed normal distribution may be used, or a time change of a multi-view camera image as an image (for example, optical flow, etc.) ) May be used, or such existing methods may be combined. Tracking may be performed on a contour extracted at a certain time with respect to a subsequent time. After the region is extracted, the target contour may be obtained by performing noise removal such as merging an excessively small region with a nearby large region.

  FIG. 2 [1] shows an example of the obtained contour image (silhouette image) as a mask image, and two players from cameras 1, 3, 4, 5, 7, and 8 are shown. The two contours are extracted in a state where the area is separated on the image, and only one contour is extracted from the cameras 2 and 6 due to the occlusion due to the overlap of two players. I have.

  The representative point determination unit 12 determines a representative point for each contour of each image extracted by the contour extraction unit 11, and outputs information on the position of the representative point to the correspondence acquisition unit 2. As the determined representative point, any type of representative point defined based on the contour area may be used. For example, the representative point may be obtained as a center of gravity (centroid).

The center of gravity can be specifically calculated, for example, as follows. The contour image (silhouette image) in the image P (k) of the camera k is given by I _k (x, y) (“1” is assigned to the position (x, y) belonging to the contour, and the position (x, When y) in providing a "0"), the object o _k ^l (l image P (k) is first in the following equation (1) is an index that identifies the object determined as a contour region) about zero and first The following moment features ((p, q) = (0,0), (1, 0), (0,1)) are obtained.

Here, vertical in image coordinates (x, y) is assumed to take lateral and y-axis in the vertical direction in the x-axis, W, for L surrounds the object o _k ^l rectangular (camera image P (k) A rectangle having sides parallel to each other). Since the rectangle is determined in the image area determination unit 14 described below, it is possible to acquire information on W and L. (Flow of the information acquisition is omitted in FIG. 1.) (X _l, y _l) is a pixel positions belonging to the target o _k ^l, x = 4m for formula (1) in computing speed, Although the calculation is performed by thinning out only every four pixels as in y = 4n, the number of pixels for thinning out may be set arbitrarily. Further, it is not necessary to perform the thinning.

Then, by using each moment characteristic obtained by the above formula (1), the center of gravity (centroid) position of the object o _k ^l image P (k) (x (o k l), y (o k l) ) Can be obtained as in the following equation (2).

  In FIG. 2, [3] shows an example of the position of the center of gravity (centroid position) of each object determined by the representative point determination unit 12 as a representative point by indicating the position with a “+” sign along with its coordinate value. Have been.

In the object number determination unit 13, the contour extraction unit 11 as the maximum number of contour regions extracted in each camera image P (k), to determine the target number theta _s has been taken in the entire multi-viewpoint camera image, The determined number of targets is output to the correspondence acquisition unit 13. If the number of objects extracted in the image P (k) (the number of contour areas, which is reduced from the actual number if there is occlusion, etc.) is written as θ _k , the number of objects is given by the following equation (3). . For example, in the example of FIGS. 2 and 3, the number of targets θ _s = 2 because two players are photographed.

  In automatically calculating the number of objects θs, the present invention makes the following assumptions. That is, in a scene where a plurality of objects are photographed, occurrence of occlusion between the objects cannot be avoided. However, it is assumed that, by setting each camera at a different position in obtaining a multi-viewpoint camera image, at least one camera does not cause occlusion and all objects are separately photographed.

The image area determination unit 14, the inclusion region of interest o _k ^l where the outline is extracted by the contour extraction unit 11 for each image P (k), the target o _k ^l of the image region ^{_{R (o k l) (Region}} of (Interest) to the correspondence acquisition unit 2 and the second extraction unit 3. The encompassing image region R (o _k ^l) may be prepared by the outline is expanded and deformed by a predetermined method may be made as a rectangle enclosing the contour of the object o _k ^l. (The information of the rectangle is referred to when calculating the above-described formula (1).) When the rectangle is formed as a surrounding rectangle, the height and width of the rectangle are parallel to the height and width of the image P (k), and the outline is Should be made the smallest rectangle that covers. In FIG. 2, [2] shows an example of a rectangle obtained by the image area determination unit 14 as a rectangle surrounding the outline.

In correspondence acquisition section 2, as an input a set of information obtained by the first extraction unit 1 described above, the target o _k ^l where the image region R (o _k ^l) is obtained in each image P (k) In the other image P (k + 1), the correspondence of which object o _{k + 1} ^{l ′} is applicable is performed among all camera images, and the obtained correspondence is sent to the second extraction unit 3. Is output. That is, the target o _kl of the image P (k) obtained at the time of the first extraction unit 1 is in an unknown state in which the target o ^kl corresponds to another target in another image P (k + 1). (The target index l is not consistent between different images), but the correspondence relationship is obtained by the correspondence acquisition unit 2. Incidentally, the obtained correspondence relationship is also a corresponding relationship between the object o _k ^l brute string in the image region R (o _k ^l).

  FIG. 4 is a flowchart of the operation of the correspondence acquisition unit 2 according to one embodiment.

In step S1, the reference position is initialized, and the process proceeds to step S2. Specifically, the image P (k) in which the number of objects θ _s is given as the maximum value in the above-described equation (3) (that is, an image in which the contours of all objects are separated without causing occlusion) is select, representative points of each target o _k ^l where the representative point determining unit 12 is obtained for the image P (k) (x (o k l), y (o k l)) (l = 1,2,. .., as it theta _s), (see the position of each object o ^l in _{k) (S x (o k} l) the image P, and set as Sy (o _k ^l)).

  Note that the “reference position” is used to determine the correspondence between targets with another image P (k + 1) using the image P (k) as a reference (reference target). Is attached. In step S1, since the reference position is obtained for the first time, it is called "initialization".

In step S1, when a plurality images give the object number θ _s P (k) is the maximum value, in one embodiment, may be used for setting the reference position randomly any one.

Further, as another embodiment, when there are a plurality of images P (k) that give the number of targets θ _s as the maximum value, a case where spatial separation on the images of the targets is large (the image regions of the targets are (Which are farther apart) may be determined for setting the reference position. Here, the spatial separation of the objects in the images P (k) on the images may be calculated, for example, as the variance of the representative point positions determined by the representative point determining unit 12. The obtained variance may be further normalized by dividing by the total area of the target image area (or contour area) in each image P (k). Alternatively, the spatial separation may be calculated not as the variance of the representative point positions but as the sum of the distances between all image region (or contour region) pairs. The one with the largest spatial separation calculated by any of the above methods may be used for setting the reference position, or the one where the spatial separation is equal to or more than a predetermined value may be used to randomly set the reference position. What to use may be determined.

  Hereinafter, in the description of the flow of FIG. 4, for convenience of explanation, it is assumed that P (1) of k = 1 is used for setting the reference position in step S1 (that is, k = 1 at the time of step S1). First, the correspondence between the object and the image P (2) is determined by referring to the image P (1), and then the correspondence between the object and the image P (3) is determined by referring to the image P (2). , And similarly, continuously referring to the image P (k) to determine the correspondence with the image P (k + 1).

  Step S2 is a dummy step (dummy step for indicating a repetitive structure of the flowchart) to indicate that the above-described repetitive processing is repeated in subsequent steps S3 to S7, and the P (k ) Is set, and the process proceeds to step S3.

In step S3, the corresponding position in the reference position of each object ^{_{o k l (S x (o}} k l), Sy (o k l)) image P of the (k + 1) in the image P (k) as the reference source (s ' _x (o _{k + 1} ^l ), s' _y (o _{k + 1} ^l )) and coordinate transformation (perspective transfomation) between camera images P (k) and P (k + 1) The conversion is performed using the homography matrix H _{(k, k + 1)} to be obtained as in the following equation (4), and then the process proceeds to step S4.

In step S4, the target o in the image P (k) as the reference source _k for the reference position of the ^q corresponding positions in at determined image P (k + 1) the formula _{(4) (s' x (} o k + ₁ ^q ), s' _y (ok _{+ 1} ^q )) and the representative position (x (ok _k ) determined by the representative point determination unit 12 for each object ok _{+ 1} ^l of the image P (k + 1). ₊₁ ^l ), y (o _{k + 1} ^l )) and the Euler distance (that is, the number of combinations of the number of possible values of q and the number of possible values of l is the same) Then, the process proceeds to step S5. In the distance calculation, any distance (Ln norm distance, L∞ norm distance, etc.) defined in Euclidean space may be used as a distance other than the Euler distance (L1 norm distance).

In step S5, the corresponding o _{k + 1} ^q in each target o _k ^q in an image P in the image P (k) as a reference source (k + 1), as to minimize the distance obtained in step S4 By determining as in the following equation, the correspondence between the objects between the images P (k) and P (k + 1) is obtained, and then the process proceeds to step S6.

In step S6, after setting the reference position of each object o _{k + 1} ^q in the image P (k + 1), the process proceeds to step S7. Here, when the corresponding object o _{k + 1} ^q is 1 to 1, i.e., subject only only one in the image P (k) is determined corresponding to the target o _{k + 1} ^q In step S6 In this case, the representative position (x (ok ₊ ^1q ), y (ok ₊ ^1q )) obtained by the representative point determination unit 12 for the object ok ₊ ^1q is set as a reference position. On the other hand, if it is not one-to-one, i.e., if more than one object o _k ^q in the image P (k) is determined corresponding to the target o _{k + 1} ^q In step S6, the corresponding two or more object o _k ^q positions of the aforementioned formula (4) by the image P (k + 1) 2 or more positions was converted to upper (s of ^{_{'x (o k + 1 q}} ), s' y (o k + ₁ ) Set ^q )) as the reference position of each of the two or more objects.

  In step S7, it is determined whether or not the repetition of the above steps S2 to S6 has been performed for the camera images P (k) of all viewpoints. If the repetition has been performed, the flow ends. When the entire flow is completed, the correspondence acquisition unit 2 acquires a correspondence between the objects between all the camera images P (k), and the acquired correspondence is sent to the second extraction unit 3. Is output. On the other hand, if the processing has not been completed for all camera images P (k) in step S7, the process returns to step S2, and after setting the next image P (k + 1) as a processing target, the processing in steps S2 to Repeat S7. As is clear from the above description, the reference position set for the image P (k + 1) in step S6 is used for obtaining the correspondence with the next image P (k + 2) in the next repetition processing. Will be done.

  In FIG. 3, [4] shows an example in which the conversion position is obtained in step S3 for the object corresponding to “player 1” between camera images 1, 2, 2, 3,... ing. Here, the “+” mark indicates the original reference position to be converted, and the “x” mark indicates the position obtained by converting the reference position. In FIG. 3, [5] shows an example in which the area of “player 1” and the area of “player 2” are obtained over the entire camera images 1 to 8 by the flow of FIG. I have. Note that in the camera images 2 and 6, one area where occlusion has occurred is “player 1” and is also determined as the area of “player 2” at the same time. (However, as described above in step S6 in FIG. 4, the representative points of "player 1" and the representative points of "player 2" are obtained as different points in camera images 2 and 6 as well.)

Second extraction unit 3, or more image regions R (o _k ^q) of each target o _k ^q corresponding is acquired at the corresponding acquisition unit 2 between the image P (k) (the image region image region determination (Obtained from the unit 14) as an input, a spatial region Vol (o ^q ) that includes each object o ^q in the voxel space is obtained, and is output from the region extracting device 10. Obtaining spatial domain Vol (o _q) utilize any existing technique that satisfies the relationship of spatial domain Vol (o _q) falls within each image P (k) in the image region R (o _k ^q) be able to.

Further, the second extraction unit 3 may obtain the space area Vol (o _q ) of each object o ^{q in} the following manner. FIG. 5 and FIG. 6 are schematic diagrams for explaining the method. In FIG. 5, the direction from the camera center (c _k (x), c _k (y), c _k (z)) of the camera k to four vertices of a rectangle (image area) surrounding a certain object in the image P (k) is shown. The vectors (light rays) are shown as l _k ¹ , l _k ² , l _k ³ , and l _k ⁴ , respectively, in a clockwise order starting from l _k ¹ corresponding to the upper right vertex of the rectangle. In the image P (k + 1), the same thing is shown for the rectangle surrounding the target corresponding to the image P (k). As shown in FIG. 5, a plane P _{(k, k + 1)} is extended by the rays l _{k + 1} ¹ and l _{k + 1} ² of the camera k + 1 (the normal vector is represented by (n ₁ , n ₂ , n ₃ )), and the point p ³ _{(k, k + 1)} (x, y, as the intersection of the plane P _{(k, k + 1)} and the rays l _k ³ , l _k ⁴ of the camera k. z) and the point p ⁴ _{(k, k + 1)} (x, y, z) are obtained. With the above notation, the intersection point p ³ _{(k, k + 1)} (x, y, z) and the point p ⁴ _{(k, k + 1)} (x, y, z) are respectively expressed by the following equations (5), It can be obtained as in (6).

Furthermore, the following formula (7) is used to ^convert the spatial region Vol (o ^q ) of each object o ^q into a rectangular region Vol (o ^q ) = {(x, y, z) | x _min ≦ x ≦ x _max in the voxel space. , y _min ≦ y ≦ y _max , z _min ≦ z ≦ z _max }. FIG. 6 schematically shows the rectangular area to be obtained by cutting out the (x, y) plane in the voxel space, and projections from eight camera stops are performed.

  As described above, according to the present invention, even when a plurality of objects are captured in a multi-view camera image, it is possible to separately determine the volume regions of the plurality of objects. FIG. 7 shows the effect of the present invention in the form of the output of equation (7). The example of FIG. 7 shows a result of applying the present invention when two players 1 and 2 of FIGS. 3 and 4 are included as a target and a result of the conventional method without applying the present invention (the players 1 and 2 are distinguished). In the case where the volume area encompassing the whole is obtained without using the above method), the areas of the players 1 and 2 are extracted by 1/7 and 1/8, respectively, compared to the volume of the conventional method. You can see that there is.

  Hereinafter, supplementary items of the present invention will be described.

(1) The homography matrix H _{(k, k + 1)} for obtaining the conversion position in step S3 in FIG. 4 is obtained in advance as a predetermined matrix by performing calibration between the cameras k and k + 1. You can use what you have set. Information necessary for the calculations described in FIGS. 5 and 6 may be acquired at the time of the calibration. Further, the point correspondence may be obtained between the images P (k) and P (k + 1) without performing the calibration, and the point correspondence may be obtained on the spot by the existing method.

  (2) The order of the images P (1), P (2),..., P (N) for performing the flow of FIG. 4 depends on the image set in step S1, and A predetermined order may be determined in advance according to the positional relationship. In FIG. 4, for convenience of explanation, P (1) is set for initializing the reference position, and P (1) → P (2) → P (3) → P (4) → P (5) → P (6) ) → P (7) → P (8). Similarly, for example, if P (4) is set for initializing the reference position, P (4) → P (5) → P (6) → P (7) → P (8) → P (1) → P (2) → P (3) in order, or P (4) → P (3) → P (2) → P (1) → P (8) → The correspondence may be obtained in the order of P (7) → P (6) → P (5).

  (3) The present invention can also be provided as a program that causes a computer to function as the region extraction device 10. The computer may have a well-known hardware configuration such as a CPU (Central Processing Unit), a memory, and various I / Fs, and the CPU executes an instruction corresponding to a function of each unit of the area extraction device 10. It will be.

  10… Area extraction device, 1… First extraction unit, 2… Correspondence acquisition unit, 3… Second extraction unit

Claims (9)

A first extraction unit that extracts one or more image regions including the target from each camera image in the captured multi-viewpoint camera images of the plurality of targets;
A correspondence acquisition unit for determining which of the plurality of objects the one or more extracted image regions correspond to in each camera image, together with the correspondence of the objects between camera images;
Based on the extracted image region in each camera image linked by the determined correspondence, based on the second extraction unit that extracts a spatial region that includes the target in the corresponding relationship ,
In the correspondence acquisition unit, by calculating the correspondence between the positions of the representative points of the objects in the image area, between different camera images, the correspondence of the objects between the camera images is obtained,
The correspondence obtaining unit obtains a correspondence between positions of the representative points of the object in the image area between different camera images in a predetermined order determined between the camera images, and obtains the obtained correspondence. When it is determined that the overlap of the target occurs in the image area in the first camera image, the representative position of the target in the second camera image for which the correspondence relationship with the first camera image has been obtained is set to the first camera image. by converting the coordinate, area extraction apparatus according to claim Rukoto set representative points of the target in the first camera image. A first extraction unit that extracts one or more image regions including the target from each camera image in the captured multi-viewpoint camera images of the plurality of targets;
A correspondence acquisition unit for determining which of the plurality of objects the one or more extracted image regions correspond to in each camera image, together with the correspondence of the objects between camera images;
Based on the extracted image region in each camera image linked by the determined correspondence, based on the second extraction unit that extracts a spatial region that includes the target in the corresponding relationship ,
In the correspondence acquisition unit, by calculating the correspondence between the positions of the representative points of the objects in the image area, between different camera images, the correspondence of the objects between the camera images is obtained,
And in the correspondence acquisition section, as a representative point of the object in the image region, region extraction apparatus which is characterized that you adopt the centroid of the target contour area. A first extraction unit that extracts one or more image regions including the target from each camera image in the captured multi-viewpoint camera images of the plurality of targets;
A correspondence acquisition unit for determining which of the plurality of objects the one or more extracted image regions correspond to in each camera image, together with the correspondence of the objects between camera images;
Based on the extracted image region in each camera image linked in the determined correspondence, based on the second extraction unit that extracts a spatial region that includes the target in the corresponding relationship ,
In the correspondence acquisition unit, by calculating the correspondence between the positions of the representative points of the objects in the image area, between different camera images, the correspondence of the objects between the camera images is obtained,
In the correspondence acquisition unit, a maximum number of the image areas extracted by the first extraction unit in each camera image is determined as the number of the plurality of objects, and one of the camera images from which the maximum number is extracted is set. By setting the position of the representative point of the object in the reference position as a reference position and obtaining the corresponding position of the reference position in the other camera image, the correspondence of the position of the representative point of the object in the image area between the different camera images is obtained. area extraction apparatus according to claim Rukoto obtained relation. In the correspondence acquisition unit, when there are a plurality of camera images that give the maximum number, from among the camera images that are determined to have a large spatial separation between the image regions extracted by the first extraction unit, 4. The region extracting apparatus according to claim 3 , wherein one camera image for setting the reference position is determined. A first extraction unit that extracts one or more image regions including the target from each camera image in the captured multi-viewpoint camera images of the plurality of targets;
A correspondence acquisition unit for determining which of the plurality of objects the one or more extracted image regions correspond to in each camera image, together with the correspondence of the objects between camera images;
Based on the extracted image region in each camera image linked by the determined correspondence, based on the second extraction unit that extracts a spatial region that includes the target in the corresponding relationship ,
In the first extraction unit, the image area is obtained as a rectangular area,
In the second extraction unit, two vertices of the rectangular area of the first camera image from the camera center of the first camera image based on the image area of the extracted rectangular area in each camera image connected in the determined correspondence relationship The two intersections of the two straight lines extending to and the plane on which the camera center of the second camera image and the two vertices of the rectangular area of the second camera image ride are found, and a series of the first camera image and the second camera image are obtained. based on a set of two-intersection to which the determined between, area extraction apparatus characterized that you extracted spatial region encompassing subjects at the corresponding relationship. The first extraction unit extracts a contour of the object from each camera image, the area extracting device according to any one of claims 1 and extracts the image area as an area encompassing the contour 5 . The correspondence obtaining unit obtains the correspondence between the representative points of the objects in the image area and the position between different camera images, thereby obtaining the correspondence between the camera images. Item 7. The region extraction device according to any one of Items 1 to 6 . And in the correspondence acquiring section, the region extraction device according to correspondence relationship between the position between the different camera images, to claim 1, 2, 3, 4 or 7, wherein the determination by the conversion by the homography matrix. A program for causing a computer to function as the region extraction device according to any one of claims 1 to 8 .