JP5197469B2 - Front subject image extraction apparatus and operation control method thereof - Google Patents

Description

  The present invention relates to a front subject image extraction device and an operation control method thereof.

  Various use of two images having different viewpoints is considered. For example, there is an apparatus that generates edge images of left and right images and extracts a specific subject image by matching both edge images (Patent Document 1). Also, there is an apparatus that generates an intermediate image corresponding to an arbitrary viewpoint of two images from two images having different viewpoints (Patent Document 2). In addition, there is a device that measures the distance to the background of the subject (Patent Document 3).

  However, it is not considered to extract a desired subject image relatively easily.

JP-A-8-210847 JP 9-27969 A JP 2000-125196 A

  An object of the present invention is to extract a desired subject image relatively easily.

  The front subject image extraction device according to the present invention inputs first image data representing a first image captured from a first viewpoint and a second viewpoint, and second image data representing a second image. The input means generates a first occlusion image in which an image portion that is visible in the first image but not in the second image is represented as an occlusion partial region in the first image. Of the second image, the first occlusion image generation means is a second image in which an image portion that is visible in the second image but not in the first image is represented as an occlusion portion area. A second occlusion image generating means for generating an occlusion image; a first occlusion image generated by the first occlusion image generation means so that a part or one point (for example, the center of the face) of the target image portion to be extracted matches. Image and second shielding image generation means Superimposed image generating means for generating a superimposed image by superimposing the second occluded image generated in this way, and a shielded image partial region including the part or one point from the superimposed image generated by the superimposed image generating means The image processing apparatus includes extraction means for extracting an image portion existing in the image (for example, the entire person image including the center of the face) or an image portion existing in the occlusion image portion area designated by the user.

  The present invention also provides an operation control method suitable for the front subject image extraction apparatus. That is, in this method, the image input means inputs the first image data representing the first image captured from the first viewpoint and the second viewpoint, and the second image data representing the second image. The first occlusion image generation means is configured to display an image portion of the first image that is visible in the first image but not in the second image as a occlusion portion area. The first occlusion image is generated, and the second occlusion image generation means occludes an image portion of the second image that is visible in the second image but not in the first image. A second occlusion image represented as a partial area is generated, and the first image generated by the first occlusion image generation unit is generated by the superimposed image generation unit so that a part or one point of the target image portion to be extracted matches. Produced by the second occlusion image generation means and the second occlusion image generation means. A superimposition image is generated by superimposing the second occlusion image, and an extraction unit exists in the occlusion image partial region including the part or one point from the superimposition image generated by the superimposition image generation unit. The image part to be extracted or the image part existing in the occlusion image part region designated by the user is extracted.

  The present invention also stores a program suitable for carrying out the operation control method for the front subject image extraction apparatus. Also, a recording medium storing such a program may be provided.

  According to the present invention, at least the first image captured from the first viewpoint and the second image captured from the second viewpoint are used. Of the first image, an image portion that is visible (present) in the first image but not visible (not present) in the second image is represented as a shielded partial region (occlusion region). A first occlusion image is generated. Similarly, a second occlusion image is generated in which the image portion that is visible in the second image but not in the first image is represented as the occlusion portion area. The first occlusion image and the second occlusion image are superimposed so that a part or one point of the target image portion to be extracted matches, and a superimposed image is generated. From the superimposed image, an image portion existing in the occlusion partial region including the part or one point or an image portion existing in the occlusion image partial region designated by the user is extracted.

  Two images with different viewpoints are shaded by the subject, resulting in a shielded portion that is visible in one image but not in the other image. Since the occluded portion becomes larger as the subject ahead, the superimposed image is generated by making the first occluded image and the second occluded image coincide with a part or one point of the desired target image portion. By extracting the image part existing in the shielding part region including the part or one point included in the extracted part, the extracted image part becomes a desired object image in front. A desired subject image portion can be extracted relatively easily.

  The extraction means is, for example, two straight lines parallel to a straight line connecting the first viewpoint and the second viewpoint, the two straight lines in contact with the largest shielding portion area, and the largest The image part surrounded by the shielding part area is extracted.

  The input means includes, for example, first image data representing first images of three frames taken from three first viewpoints, second viewpoints, and third viewpoints that do not exist on the same straight line; The second image data representing the image and the third image data representing the third image are input. The first occlusion image generating means is visible in the first image, but The second occlusion image generation means generates the first occlusion image in which the image portion not visible in the second image or the third image is represented as an occlusion image region. A second occlusion image is generated in which an image portion that is visible in the first image but not in the first image or the third image is represented as a occlusion image region.

  In this case, a third occlusion image is generated in which an image portion that is visible in the third image but not in the first image and the second image is represented as an occlusion image region. And a third occlusion image generating means. Further, the superimposed image generation means includes the first occlusion image generated by the first occlusion image generation means and the second occlusion image generation means so that a part or one point of the target image portion to be extracted matches. The second occlusion image generated by the above and the third occlusion image generated by the third occlusion image generation unit will generate a superimposed image.

  When the target image portion to be extracted is a human image, the superimposed image generating means generates a superimposed image so that, for example, at least one point in the face image portion included in the human image matches. The extraction means includes an image part existing in the largest or widest occlusion image partial area from the generated superimposed image (including the part or one point), and the occluded partial area closest to the part or one point. An image portion existing in the image may be extracted.

It is a block diagram which shows the electric constitution of a digital still camera. It is a flowchart which shows the process sequence of a digital still camera. (A) and (B) are examples of two frames taken from different viewpoints. (A) and (B) are examples of the occlusion image. It is an example of a superimposed image. In the superimposed image, a straight line used for subject image extraction is shown. The extracted subject image is shown. It is a flowchart which shows the shielding image generation process procedure. The structure of an imaging device is shown. The positional relationship of an imaging device is shown. It is a flowchart which shows the process sequence of a digital still camera. It is an example of the image obtained by imaging. It is an example of a shielding image. It is an example of a superimposed image. It is a flowchart which shows the shielding image generation process procedure.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing an electrical configuration of a digital still camera.

  The overall operation of the digital still camera is controlled by the CPU 11.

  The digital still camera includes a memory 16 in which an operation program described later and other data are stored. The operation program may be written in a recording medium such as the memory card 19 and read from the memory card 19 and installed in the digital still camera or may be pre-installed. .

  The digital still camera includes a shutter release button 12 and a mode switch 13. A signal indicating depression of the shutter release button 12 is input to the CPU 11. The mode switch 13 is used to select an imaging mode or a reproduction mode, and the switch S1 or S2 can be selectively turned on. The imaging mode is set when the switch S1 is turned on, and the reproduction mode is set when the switch S2 is turned on.

  The digital still camera according to this embodiment can capture two frames of images with parallax on the left and right (upper and lower, or diagonal directions) simultaneously (not necessarily simultaneously). For this purpose, a digital still camera is provided with a first imaging device 1 and a second imaging device 2 that are separated from each other by a predetermined distance.

  When the imaging mode is set by the mode switch 13, the subject is imaged by one of the first imaging device 1 and the second imaging device 2 at a constant cycle (for example, 1/30 second cycle). A video signal representing the subject image is output from the first image pickup device 1 or the second image pickup device 2 at a constant period and is input to the analog / digital conversion circuit 9. The video signal is converted into digital image data by the analog / digital conversion circuit 9.

  The digital image data is input to the image signal processing circuit 6 by the image input controller 5 and subjected to predetermined signal processing such as gamma correction. The digital image data is written in a VRAM (Video Random Access Memory) 17 and then read out and given to the image display device 10 so that it is displayed as a moving image on the display screen of the image display device 27 (fixed cycle). Displayed).

  The digital still camera 1 according to this embodiment can detect a face image portion included in a subject image obtained by imaging. For this purpose, the digital still camera includes a face detection circuit 8.

  Digital image data obtained by imaging and data indicating the position of the detected face image portion are input to an AF (autofocus) detection circuit 14. The zoom position of the zoom lens (not shown, included in the first imaging device 1 and the second imaging device 2) so that the detected face image portion is focused in the AF detection circuit 14. Is controlled. The digital image data obtained by imaging and the data indicating the position of the detected face image portion are also input to the AE (automatic exposure) / AWB (automatic white balance) detection circuit 15. In the AE / AWB detection circuit 15, a stop (not shown, included in the first imaging device 1 and the second imaging device 2) so that the brightness of the detected face image portion becomes an appropriate brightness. ) Is determined. Further, white balance adjustment is also performed in the AE / AWB detection circuit 15. Of course, if the aperture and zoom lens are unnecessary, the aperture and zoom lens may not be included in the first imaging device 1 and the second imaging device 2.

  When the shutter release button 12 is pressed, the subject is imaged by the imaging devices 1 and 2 of both the first imaging device 1 and the second imaging device 2. Both the first image data obtained by imaging by the first imaging device 1 and the second image data obtained by the second imaging device are input to the compression processing circuit 7. The first image data and the second image data subjected to predetermined compression processing in the compression processing circuit 7 are input to the video encoder 20 and encoded. The encoded first image data and second image data are recorded on the memory card 19 under the control of the memory card controller 18.

  When the playback mode is set by the mode switch 4, the first image data and the second image data recorded on the memory card 28 are read. Since the first image data and the second image data are picked up from different viewpoints where parallax occurs, the first image (right image) represented by the first image data includes the first image data. The occluded image portion that is visible but not visible in the second image is included. Similarly, the second image (left image) represented by the second image data includes a shielded image portion that is visible in the second image but not in the first image. . In this embodiment, a desired subject image is extracted from the first image (second image) using these shielding image portions. The extracted subject image is displayed on the display screen of the image display device 10.

  FIG. 2 is a flowchart showing the processing procedure of the digital still camera. 3A and 3B to FIG. 7 are examples of images.

  First image data and second image data captured at two different first viewpoints and second viewpoints are read from the memory card 19, and the right image and the second image represented by the first image data are displayed. A left image represented by the image data is obtained (step 31). A right occlusion image and a left occlusion image are generated from the obtained right image and left image (step 32).

  FIG. 3A is an example of a left image, and FIG. 3B is an example of a right image. The left image 40 in FIG. 3A is taken by the second imaging device 2 as described above, and the right image 50 in FIG. 3B is taken by the first imaging device 1. is there. The left image 40 and the right image 50 are captured at a timing that is considered to be substantially the same.

  As shown in FIG. 3A, the left image 40 includes a subject image of a human image 41 ahead. Tree images 42, 43 and 44 are included behind the human figure 40. As shown in FIG. 3B, the right image 50 also includes a subject image of the person image 51 in front. Tree images 52, 53, and 54 are included behind the human figure 51. The person of the person image 41 included in the left image 40 and the person of the person image 51 included in the right image 50 are the same person. Further, the trees 42, 43 and 44 included in the left image 40 and the trees 52, 53 and 54 included in the right image 50 are the same.

  Since the left image 40 and the right image 50 are captured from different viewpoints, as described above, there are image portions that are visible in the left image 40 but not in the right image 50. Similarly, there is an image portion that is visible in the right image 50 but not in the left image 40 (the image portion that does not look like this is called a shielded partial region, and an image that represents the shielded partial region is blocked. I will call it an image).

  4A is an example of a left occlusion image, and FIG. 4B is an example of a right occlusion image.

  Referring to FIG. 4A, in the left occlusion image 60, an occlusion partial area 45 is displayed on the left side of the person image 41. The shielded partial region 45 is a portion that is visible in the left image 40 shown in FIG. 3A but is not visible in the right image 50 shown in FIG. Similarly, with reference to FIG. 4B, a shielded partial area 55 is displayed on the right side of the person image 51. This shielded partial region 55 is a portion that is visible in the right image 50 shown in FIG. 3B but is not visible in the left image 40 shown in FIG. In this way, the left occlusion image 60 and the right occlusion image 70 are generated. Details of the method for generating the left occlusion image 60 and the right occlusion image 70 will be described later.

  In FIG. 4A, in addition to the occlusion image portion 45 formed on the left portion of the person image 41, an occlusion image portion is also generated on the left side of the tree images 42, 43 and 44. Since the occlusion image portion generated on the left side of 43 and 44 is small, the illustration is omitted. Similarly, in FIG. 4B, in addition to the occluded image portion 55 formed on the right side of the person image 51, an occluded image portion is also generated on the right side of the tree images 52, 53 and 54. Since the occlusion image portion generated on the right side of 53 and 54 is small, the illustration is omitted.

  Returning to FIG. 2, when the left occlusion image 60 and the right occlusion image 70 are generated, the face images of the human images 41 and 51 included in the respective images 60 and 70 are detected, and the detected face images ( Are superimposed so that the centers of the two coincide (step 34).

  FIG. 5 is an example of the superimposed image 80 generated in this way.

  When the left occlusion image 60 and the right occlusion image 70 are superimposed so that the face image of the left occlusion image 60 and the face image of the right occlusion image 70 coincide with each other, a person image 81 is not blurred. . The background of the person image 81 includes tree images 42, 43 and 44 included in the left occlusion image 60 and tree images 52, 53 and 54 included in the right occlusion image 70. On the left side of the person image 81, a shielding partial area 45 included in the left shielding image 60 is displayed. Similarly, the occlusion partial area 55 included in the right occlusion image 70 is displayed on the right side of the person image.

  The center of the face used for the generation of the superimposed image 80 is included in the image part surrounded by the shield part regions 45 and 55. Therefore, it is considered that the image part surrounded by the shielding part regions 45 and 55 is the image part to be extracted.

  Returning to FIG. 2 again, a straight line connecting the upper ends of the shielding partial regions 45 and 55 including the center of the face and a straight line connecting the lower ends are calculated (step 35).

  FIG. 6 shows a straight line L1 connecting the upper ends of the shielded partial areas 45 and 55 including the center of the face and a straight line L2 connecting the lower ends in the superimposed image 80.

  A straight line L1 is obtained by obtaining a straight line connecting the upper ends of the shielded partial regions 45 and 55 including the center of the face. Similarly, a straight line L2 is obtained by obtaining a straight line connecting the lower ends of the shielded partial areas 45 and 55 including the center of the face.

  Referring to FIG. 2 again, in the superimposed image 80, an image part surrounded by the shielding part regions 45 and 55 including the center of the face, a straight line L1 connecting the upper ends, and a straight line L2 connecting the lower ends is extracted (step). 36).

  FIG. 7 is an example of an image obtained by extracting from the superimposed image 80 an image portion surrounded by shielded partial areas 45 and 55 including the center of the face, a straight line L1 connecting the upper ends, and a straight line L2 connecting the lower ends. An image 90 including the extracted person image 81 is obtained. The entire person image 81 is a desired subject image to be extracted. The entire person image 81 including the face image can be extracted from the face image.

  In this embodiment, the left image 40 and the right image 50 in which left and right parallax are generated are used. Therefore, in order to define the vertical range of the image to be extracted, the shielded partial regions 45 and 55 including the center of the face are used. A straight line connecting the top and bottom edges of each other is used, but when two images with top and bottom parallax are used, the straight lines connecting the left and right edges of the shielded partial area including the center of the face Needless to say, a desired subject image is extracted using. If the parallax is in an oblique direction, a subject image in front will be extracted using two straight lines oblique to the oblique direction.

  In the above-described embodiment, straight lines connecting the upper ends and the lower ends of the largest shielding partial regions 45 and 55 are used to define the upper and lower ranges of the image to be extracted. When the part is surrounded by a shielding part including the center of the face, an image part surrounded by the largest shielding part region may be extracted without using these straight lines.

  FIG. 8 is a flowchart showing the occlusion image generation processing procedure (the processing procedure of step 32 in FIG. 2). This process generates a left occlusion image (left occlusion image).

  The pixel p of the image of the left image 40 is detected (step 101). In the right image 50, a pixel corresponding to the pixel p is detected (step 102).

  When a pixel corresponding to the pixel p is found in the right image (YES in step 103), the pixel p of the left image is pasted to the corresponding position of the occlusion image of the left image to be generated (step 104). If a pixel corresponding to the pixel p is not found in the right image (NO in step 103), a flag indicating that it is a shielded image portion is set at a position corresponding to the pixel p in the shielded image (step 105). . The portion specified by the flagged pixel is the above-described occlusion image portion.

  Until the above processing is completed for all the pixels (step 106), the pixel p is repeated while being incremented one by one (step 107). The above-described pixel p is incremented so that the above-described processing is performed for all the pixels of the left image.

  When the occlusion image of the left image is generated, the occlusion image of the right image is similarly generated.

  9 to 16 show another embodiment. The above-described embodiment uses two frames imaged with two different parallaxes, whereas this embodiment uses three frame images imaged with three different parallaxes.

  FIG. 9 is a block diagram showing an electrical configuration of an image pickup apparatus included in the digital still camera. FIG. 10 shows the positional relationship of the imaging devices included in the digital still camera.

  In this embodiment, three imaging devices 1, 2, and 3 are used. As described above, when the camera is held horizontally, the first imaging device 1 and the second imaging device 2 are configured to capture images from different left and right viewpoints so that left and right parallax do not occur. Is positioned. The third imaging device 3 is positioned so as to generate vertical parallax with the first imaging device and the second imaging device when the camera is held horizontally. The third imaging device 3 is positioned at an intermediate position in the horizontal direction between the first imaging device 1 and the second imaging device 2.

  When the subject is imaged by the first imaging device 1, the second imaging device 2, and the third imaging device 3, video signals representing the subject image are output from the imaging devices 1, 2, and 3. The output video signal is input to the analog / digital conversion circuit 4 and converted into digital image data. The converted digital image data is recorded on the memory card 19 as described above.

  As described above, an image captured using the first imaging device 1 is a right image (FIG. 3B), and an image captured using the second imaging device 2 is a left image ( FIG. 3 (A)). Further, an image captured using the third imaging device 3 is an upper image described later (FIG. 12).

  FIG. 11 is a flowchart showing the processing procedure of the digital still camera, and corresponds to the processing procedure of FIG.

  Two frames of images with parallax on the left and right, and two frames of images and image data representing an image of one frame with parallax on the top and bottom are recorded on the memory card 19, and these three frames of image data are recorded from the memory card 19. It is read (step 111). The two frames having left and right parallax are the left image 40 and the right image 50 shown in FIGS. 3A and 3B, and the remaining one frame is the upper image 120 shown in FIG. The left image 40 is an image taken from the left side of the subject, the right image 50 is an image taken from the right side of the subject, and the upper image 120 is an image taken from above the subject. It is.

  Referring to FIG. 12, in upper image 120, person image 121 is displayed forward. Tree images 122, 123 and 124 are displayed behind the person image 121. The tree image 122 included in the upper image 120 is the same tree as the tree image 42 included in the left image 40 and the tree image 52 included in the right image 50. The tree image 123 included in the upper image 120 is the same tree as the tree image 43 included in the left image 40 and the tree image 53 included in the right image 50. Similarly, the tree image 124 included in the upper image 120 is the same tree as the tree image 44 included in the left image 40 and the tree image 54 included in the right image 50.

  Returning to FIG. 11, the occlusion image is generated for each of the three images having different viewpoints, and three occlusion images are generated (step 112). For the left image 40 and the right image 50, the left occlusion image 60 (FIG. 4A) and the right occlusion image 70 (FIG. 4B) are generated as described above. An upper occlusion image is also generated for the upper image 120.

  FIG. 13 is an example of the upper occlusion image 130.

  Similar to the left occlusion image 60 and the right occlusion image 70, the upper occlusion image 130 has a portion that is visible in the upper image 120 but not in the left image 40 or the right image 50. That part is displayed as a shield part region 125 in the top shield image 130. Since the upper image 120 is obtained by capturing the subject from above as compared with the left image 40 and the right image 50, the upper image 120 shows the upper part of the subject image 121, but the left image 40 and the right image 50 are visible. It can be understood that the portion corresponding to the upper portion of the subject image 121 is not visible. In addition, since the upper image 120 and the left and right images 40 and 50 have not only upper and lower parallaxes but also left and right parallaxes, portions that cannot be seen on the left and right sides of the human image 121 are generated. Therefore, the shielding partial area 125 is formed on the upper side, the left side, and the right side of the subject image 121. However, since the third imaging device 3 that captures the upper image 120 is in the middle of the first imaging device 1 that captures the right image 50 and the second imaging device 2 that captures the left image 40, the left and right sides. The width of the shielding part region 125 is half the width of the shielding part regions 45 and 55.

  Returning to FIG. 2, a face image is detected from each of the images 60, 70 and 130 of the three frames of occlusion images (step 113), and the three frames of occlusion images 60, so that the detected face images (centers) coincide with each other. 70 and 130 are superimposed (step 114).

  FIG. 14 is an example of a superimposed image 150 in which the three occluded images 60, 70, and 130 are superimposed.

  In the superimposed image 150, a person image 131 is displayed. The person image 131 represents the person image 41 of the left image 40, the person image 51 of the right image 50, and the person image 121 of the upper image 120. On the person image 131, a shielded partial area 125 is displayed. In addition, a shielded partial area 45 is displayed on the left of the person image 131. Further, a shielded partial area 55 is displayed on the right side of the person image 131. Behind the person image 131, tree images 42, 43 and 44, tree images 52, 53 and 54 and tree images 122, 123 and 124 are displayed.

  Returning again to FIG. 11, the image portion surrounded by the occluded portion regions 45, 55 and 125 including the face in the superimposed image 150 is extracted (step 115). As a result, an image from which the person image 81 is extracted is obtained as shown in FIG. When the image part is not surrounded by the shielding part area, a part surrounded by any one of the four sides of the image and the shielding part area is extracted.

  FIG. 15 is a flowchart showing the occlusion image generation processing procedure, which corresponds to the processing procedure of FIG.

  Among the three frames, the pixel p of one image is detected (step 161), and in the other two frames, the pixel corresponding to the pixel p is detected (step 162).

  If a corresponding pixel is found in any of the other two frames (YES in step 163), the pixel p of one image is pasted to the corresponding position of the occlusion image to be generated (step 164). If the corresponding pixel is not found in any of the other two frames (NO in step 163), the occlusion image partial flag is set at the position corresponding to the pixel p in the occlusion image (step 165).

  The above process is repeated for all pixels (steps 166 and 167).

  In the above-described embodiment, two or three imaging devices obtain images with two or three frames of different viewpoints. However, a single imaging device is used to image a subject from two or three different viewpoints. Two or three subject images may be obtained. Further, imaging may be performed using one camera, or imaging may be performed using two or three cameras.

  In the embodiment described above, a person image in the occluded image partial region including the face image (a part or one point of the target image portion to be extracted) used to generate the superimposed image is extracted from the superimposed image. However, an image portion existing in the largest or widest occlusion image partial region (in the occlusion image partial region including the face image) may be extracted. In addition, when there are a plurality of occlusion image partial areas, the user designates a desired occlusion image partial area from the plurality of occlusion image partial areas, and extracts an image part in the designated occlusion image partial area. It may be. For example, a desired shielding image partial area can be designated by forming a touch panel on the display screen and touching the desired shielding image partial area.

1, 2, 3 Imaging device (image input means)
11 CPU (first occlusion image generation means, second occlusion image generation means, superimposed image generation means, extraction means)

Claims (8)

Image input means for inputting first image data representing a first image captured from a first viewpoint and a second viewpoint, and second image data representing a second image;
Among the first images, a first image that generates a first occlusion image in which an image portion that is visible in the first image but not in the second image is represented as an occluded portion area. Occlusion image generation means,
Of the second image, a second image that generates a second occluded image in which an image portion that is visible in the second image but not in the first image is represented as a occluded portion area. Occlusion image generation means,
The first occlusion image generated by the first occlusion image generation means and the second occlusion image generated by the second occlusion image generation means so that a part or one point of the target image portion to be extracted matches. And a superimposed image generating means for generating a superimposed image, and 2 parallel to a straight line connecting the first viewpoint and the second viewpoint from among the superimposed images generated by the superimposed image generating means. Extraction means for extracting an image portion surrounded by two straight lines that are in contact with the largest shielding part area and the largest shielding part area ;
A forward subject image extraction device. The input means is
First image data representing a first image of three frames captured from three first viewpoints, a second viewpoint, and a third viewpoint that do not exist on the same straight line, and a second image representing a second image Inputting data and third image data representing a third image;
The first occlusion image generation means includes:
Generating a first occlusion image in which an image portion that is visible in the first image but not in the second image or the third image is represented as an occlusion image region;
The second occlusion image generation means includes:
Generating a second occlusion image in which an image portion that is visible in the second image but not in the first image or the third image is represented as an occlusion image region;
A third occlusion that generates a third occlusion image in which an image portion that is visible in the third image but not in the first image and the second image is represented as an occlusion image region. An image generating means;
The superimposed image generating means includes
The first occlusion image generated by the first occlusion image generation means and the second occlusion image generated by the second occlusion image generation means so that a part or one point of the target image portion to be extracted matches. And a superimposed image obtained by superimposing the third occlusion image generated by the third occlusion image generation means,
The front subject image extraction device according to claim 1 . The target image part to be extracted is a human figure,
The superimposed image generating means includes
A superimposed image is generated so that at least one point in the face image portion included in the human image matches.
The front subject image extraction device according to claim 1 or 2 . The image input means inputs first image data representing the first image captured from the first viewpoint and the second viewpoint, and second image data representing the second image,
The first occlusion image generating means is a first image in which an image portion that is visible in the first image but not in the second image is represented as an occlusion portion area. Generate occlusion images for
The second occluded image generation means is a second image in which an image portion that is visible in the second image but not in the first image is represented as an occluded portion area. Generate occlusion images for
The superimposed image generation means is generated by the first occlusion image generated by the first occlusion image generation means and the second occlusion image generation means so that a part or one point of the target image portion to be extracted matches. A superimposed image is generated by superimposing the second occlusion image,
The extracting means is two straight lines parallel to a straight line connecting the first viewpoint and the second viewpoint out of the superimposed images generated by the superimposed image generating means, and the largest shielding portion Extract the image part surrounded by the two straight lines that touch the area and the largest occluded part area .
Operation control method of front subject image extraction device. A program for controlling a computer of a front subject image extraction device,
First image data representing a first image captured from a first viewpoint and a second viewpoint, and second image data representing a second image are input,
Generating a first occlusion image in which an image portion of the first image that is visible in the first image but not in the second image is represented as a occlusion portion region;
Generating a second occlusion image in which an image portion of the second image that is visible in the second image but not in the first image is represented as a occlusion portion region;
A superimposed image is generated by superimposing the generated first occlusion image and the generated second occlusion image so that a part or one point of the target image portion to be extracted matches.
Among the superimposed images, two straight lines parallel to a straight line connecting the first viewpoint and the second viewpoint, which are in contact with the largest shielding partial area, and the largest shielding. A computer-readable program for controlling the computer of the front subject image extraction device so as to extract the image portion surrounded by the partial region . Three first viewpoint does not exist on the same straight line, a second viewpoint and a third of the first image data representing a three Komano first image captured from the viewpoint, the second image representing the second image Image input means for inputting data and third image data representing the third image ;
A first occlusion that generates a first occlusion image in which an image portion that is visible in the first image but not in the second image or the third image is represented as an occlusion image region. Image generation means,
A second occlusion that generates a second occlusion image in which an image portion that is visible in the second image but not in the first image or the third image is represented as an occlusion image region. Image generation means,
A third occlusion that generates a third occlusion image in which an image portion that is visible in the third image but not in the first image and the second image is represented as an occlusion image region. Image generation means,
The first occlusion image generated by the first occlusion image generation means and the second occlusion image generated by the second occlusion image generation means so that a part or one point of the target image portion to be extracted matches. and the third shield third shielding image and superimposed image generating means for generating a superimposed image in which superposed generated by the image generating means, and
From the superimposed image generated by the superimposed image generation means , extract an image portion existing in the occlusion image partial region including the part or one point or an image portion existing in the occluded image partial region designated by the user Extraction means to perform,
A forward subject image extraction device. The image input means outputs first image data and second image representing three first images captured from three first viewpoints, second viewpoints, and third viewpoints that do not exist on the same straight line. Inputting second image data representing and third image data representing a third image;
The first occlusion image generation means is a first image in which an image portion that is visible in the first image but not in the second image or the third image is represented as an occlusion image region. Generate occlusion images,
The second occlusion image generation means is a second image in which an image portion that is visible in the second image but not in the first image or the third image is represented as an occlusion image region. Generate occlusion images,
The third occlusion image generation means is a third occluded image area that is visible in the third image but not in the first image and the second image. Generate occlusion images,
The superimposed image generation means is generated by the first occlusion image generated by the first occlusion image generation means and the second occlusion image generation means so that a part or one point of the target image portion to be extracted matches. Generating a superimposed image obtained by superimposing the second occlusion image and the third occlusion image generated by the third occlusion image generation unit,
The extraction means exists in the image portion existing in the occlusion image partial area including the part or one point, or in the occlusion image partial area designated by the user, from among the superimposed images generated by the superimposed image generation means. Extract the image part ,
Operation control method of front subject image extraction device . A program for controlling a computer of a front subject image extraction device,
First image data representing a first image of three frames captured from three first viewpoints, a second viewpoint, and a third viewpoint that do not exist on the same straight line, and a second image representing a second image Input data and third image data representing a third image;
Generating a first occlusion image in which an image portion that is visible in the first image but not in the second image or the third image is represented as an occlusion image region;
Generating a second occlusion image in which an image portion that is visible in the second image but not in the first image or the third image is represented as an occlusion image region;
Generating a third occlusion image in which an image portion that is visible in the third image but not in the first image and the second image is represented as an occlusion image region;
Generating a superimposed image in which the first occlusion image, the second occlusion image, and the third occlusion image are superimposed so that a part or one point of the target image portion to be extracted matches,
From the generated superimposed image, the front subject image is extracted so as to extract the image part existing in the occlusion image partial area including the part or one point, or the image part existing in the occlusion image partial area specified by the user. A computer readable program for controlling the computer of the extraction device.

Images