JP2014158118A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device and a method of controlling the same, capable of generating a still picture obtained by removing a moving object from a photographed image.SOLUTION: An imaging device generating a background image obtained by removing a moving object from a photographed image, comprises: image holding means holding a standard image, and a reference image photographed at a different timing from the standard image; distance information holding means holding distance information on the standard image and the reference image; background image generation means generating the background image from the standard image and the reference image by using the distance information on the standard image and the distance information on the reference image; difference information generation means generating difference information from the reference image and the background image; adjacent region detection means detecting whether or not a region obtained from the difference information and a region updated by the background image generation means are adjacent to each other; and background image update means updating a region detected by the adjacent region detection means, of the background image by the reference image.

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus that generates an image obtained by removing a moving object from a plurality of images and a control method for the imaging apparatus.

  When taking a still image of a landscape or a building using a digital camera, an unnecessary moving object such as a person or a vehicle may be reflected. In particular, in a shooting environment where there is a large amount of traffic such as a person or a vehicle, it is very difficult to capture a still image so that a moving object does not appear.

  In order to solve this problem, methods such as Patent Document 1 and Patent Document 2 are disclosed.

JP 2002-288664 A JP 2011-182341 A

  As a technique disclosed in Patent Document 1, at least three or more still images continuously shot at the same angle are divided into a plurality of divided regions, and the correlation of the divided regions between the still images is determined in divided region units. . Then, one still image is created by synthesizing divided images having a high degree of correlation for each divided region.

  As a technique disclosed in Patent Document 2, distance information to a subject in a plurality of regions of two still images continuously shot at the same angle is acquired, and distance information between the still images is compared in units of a plurality of regions. . Then, an area with a long distance is detected, and the detected area is updated to generate a single still image.

  However, in Patent Document 1, three or more still images are required, and more memory is required to hold the images. In Patent Document 2, two still images are sufficient. However, when the captured still image has a shadow of a subject, the distance from the camera to the shadow is the same as the distance from the camera to the background. There was a problem that it could not be detected.

  For example, as shown in FIG. 2, when a moving object is removed using only distance information from the two still images shown in FIGS. 2 (a) and 2 (b), as shown in FIG. There was a problem that shadows remained.

  Therefore, an object of the present invention is to provide an imaging apparatus and a control method for the imaging apparatus that can generate a still image in which a moving object and a shadow of the moving object are removed from two captured images.

In order to achieve the above object, the present invention provides:
In an imaging device that generates a background image in which a moving object is removed from a captured image,
Imaging means for imaging a subject and outputting captured image data;
Reference image holding means for holding the captured image data as a reference image;
Reference image holding means for holding the captured image data captured at a time different from the reference image as a reference image;
Reference image distance information holding means for holding the distance to the subject in the reference image as reference image subject distance information;
Reference image distance information holding means for holding the distance to the subject in the reference image as reference image subject-specific distance information;
Background image generating means for generating a background image from the reference image and the reference image using the reference image subject distance information and the reference image subject distance information;
Difference information generating means for generating difference information from the reference image and the background image;
An adjacent region detection unit that detects whether the region obtained from the difference information and the region updated by the background image generation unit are adjacent to each other;
Background image update means for updating the area detected by the adjacent area detection means of the background image with the reference image;
It is characterized by having.

  According to the present invention, it is possible to provide an imaging apparatus capable of generating an image obtained by removing a moving object from a plurality of images and a control method for the imaging apparatus.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus 101 according to an embodiment of the present invention. It is a figure for demonstrating the standard image in the embodiment of this invention, a reference image, standard image distance information, and reference image distance information. It is a figure for demonstrating the background image generation process in embodiment of this invention. It is a figure for demonstrating the background image update process in embodiment of this invention. It is a flowchart which shows a part of background image generation process and background image update process in the embodiment of the present invention. It is a flowchart which shows a part of background image generation process and background image update process in the embodiment of the present invention. It is a flowchart which shows the determination process for determining whether the background image generation process and background image update process in embodiment of this invention were complete | finished.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[Example 1]
Hereinafter, an imaging apparatus according to an embodiment of the present invention will be described with reference to FIG.

  Reference numeral 101 denotes an imaging apparatus. Reference numeral 102 denotes an input image. 103 is distance information. Reference numeral 104 denotes an imaging unit. Reference numeral 105 denotes a reference image holding unit. Reference numeral 106 denotes a reference image holding unit. Reference numeral 107 denotes a reference image distance information holding unit. Reference numeral 108 denotes a reference image distance information holding unit. Reference numeral 109 denotes a background image generation unit. Reference numeral 110 denotes a difference information generation unit. Reference numeral 111 denotes an adjacent region detection unit. Reference numeral 112 denotes a background image update unit. Reference numeral 113 denotes an output image.

  Next, the operation of the block diagram shown in FIG. 1 will be described.

  The imaging unit 104 inputs the input image 102 and outputs captured image data.

  The reference image holding unit 105 is a buffer memory that receives captured image data from the imaging unit 104 and holds the input captured image data as reference image data.

  Similarly, the reference image storage unit 106 is a buffer memory that receives captured image data from the imaging unit 104 and stores the input captured image data as reference image data.

  The reference image distance information holding unit 107 is a buffer memory that receives the distance information 103 and holds distance information corresponding to the reference image data input from the reference image holding unit 105 as reference image distance information.

  The reference image distance information holding unit 108 is a buffer memory that receives the distance information 103 and holds distance information corresponding to the reference image data input from the reference image holding unit 106 as reference image distance information.

  The background image generation unit 109 detects a region where the reference image data is farther by using the standard image data, the reference image data, the standard image distance information, and the reference image distance information.

  For the detected area, the base image data is updated with the reference image data, and a background image is generated by removing the moving object from the captured image data.

  The difference information generation unit 110 generates difference information between the reference image data and the background image generated by the background image generation unit 109.

  The adjacent image detection unit 111 detects whether the region obtained by the difference information generation unit 110 and the region updated by the background image generation unit 109 are adjacent to each other.

  The background image update unit 112 updates the area detected by the adjacent area detection unit 111 of the background image with reference image data.

  As a result, a background image obtained by removing the moving object from the image data generated by the background image generation unit 109 is generated and output as the output image 113.

  Next, with reference to FIG. 2, the standard image, the reference image, the standard image distance information, and the reference image distance information in the embodiment of the present invention will be described.

  FIG. 2A shows a reference image 201 in the present embodiment. This reference image 201 is an image of a road 202 and a box 203, but a person 204 and a person's shadow 205 are captured.

  FIG. 2B shows a reference image 207 in the present embodiment, and the reference image 207 is taken at the same angle and at a different time as the standard image 201.

  The reference image 207 is the same as the reference image 201 in that the person 204 and the person's shadow 205 are reflected in a photograph of the road 202 and the box 203, but the positions of the person 204 and the person's shadow 205 are different. ing.

  FIG. 2C is an enlarged view of the area surrounded by the dashed rectangular frame 206 in FIG.

  FIG. 2D is an enlarged view of a region surrounded by a broken-line rectangular frame 208 in FIG.

  FIG. 2 (e) shows the distance information in FIG. 2 (c). Each cell in FIG. 2E corresponds to one pixel, and the numerical value in the cell is a parameter indicating the distance to the subject.

  The distance information is such that the numerical value decreases as the distance to the subject decreases, and the numerical value increases as the distance to the subject increases.

  For example, the cell “80” in FIG. 2E indicates the distance to the road 202, and the cell “40” indicates the person 204 located closer to the imaging apparatus 101 than the road 202. Indicates distance.

  The distance to the person's shadow 205 is the same as the distance to the road 202.

  FIG. 2 (f) shows the distance information in FIG. 2 (d).

  Similarly to FIG. 2E, each cell in FIG. 2F corresponds to one pixel, and the numerical value in the cell is a parameter indicating the distance to the subject.

  3 and 4 will be described in the background image generation process and the background image update process that will be described later.

  Next, a background image generation process and a background image update process for generating a background image in which a moving object and a shadow of the moving object are removed from a captured image will be described with reference to FIGS.

  In step 501 in FIG. 5, it is determined whether or not the operation mode of the imaging apparatus 101 is a background image generation mode for generating a background image.

  If it is determined that the operation mode of the imaging apparatus 1 is the background image generation mode, the process proceeds to step 503. If it is determined that the operation mode is not the background image generation mode, the process proceeds to step 502.

  In step 502, normal photographing processing is executed and the processing is terminated.

  In step 503, it is determined whether or not the shutter button of the imaging apparatus 101 has been pressed.

  If it is determined that the shutter button has been pressed, the process proceeds to step 504. If it is determined that the shutter button has not been pressed, the process returns to step 501.

  In step 504, the imaging unit 104 is controlled so that the subject is imaged and the captured image data is output.

  Then, the captured image data output from the imaging unit 104 is held in the reference image holding unit 105 as reference image data.

  Note that FIG. 2A described above is an example of a reference image.

  In step 505, the distance information corresponding to the reference image data imaged in step 504 is held in the reference image distance information holding unit 107 as reference image distance information.

  Note that FIG. 2E is an example of the reference image distance information.

  In step 601 in FIG. 6, a background image generation end determination process for determining whether or not the background image generation process has ended is executed.

  The contents of this processing will be described later using the flowchart shown in FIG.

  In step 602, if it is determined that the background image generation process has ended as a result of the background image generation end determination process in step 601, the process ends.

  On the other hand, if it is determined that the background image generation process has not ended as a result of the background image generation end determination process in step 602, the process proceeds to step 603.

  In step 603, the imaging unit 104 is controlled so that a new subject is imaged and the captured image data is output.

  The captured image data output from the imaging unit 104 is stored in the reference image storage unit 106 as reference image data.

  Note that FIG. 2B described above is an example of a reference image.

In step 604, the distance information corresponding to the reference image data imaged in step 603 is held in the reference image distance information holding unit 108 as reference image distance information.
Note that FIG. 2F described above is an example of reference image distance information.

  In step 605, the reference image distance information is compared with the reference image distance information, and an area that is farther from the reference image data is detected from the areas included in the reference image data.

  Specifically, first, the base image data and the reference image data are divided into a plurality of areas.

  Then, for the region having the same position among the divided regions, the numerical value of the standard image distance information and the numerical value of the reference image distance information are compared, and when the numerical value of the reference image distance information is larger than the numerical value of the standard image distance information, The area is detected as an area where the distance of the reference image data is longer.

  In step 606, if a region far from the reference image data is detected as a result of detection in step 605, the process proceeds to step 607. If a region far from the reference image data is not detected, the process proceeds to step 601. Return.

  In step 607, the background image generation unit 109 is controlled so that the reference image data is updated with the reference image data for the region of the reference image data farther away from the reference image data detected in step 605.

  On the other hand, the image other than the region detected in step 605 is not updated.

  As a result, an image from which the person 204 as a moving object is removed is obtained as shown in FIG.

  In step 608, the difference information generation unit 109 is controlled so as to generate difference information between the image data updated in step 607 and the reference image data.

  As a result, image data as shown in FIG. 4A is generated. FIG. 4B is an enlarged view of the area surrounded by the dashed rectangular frame 402 in FIG.

  In step 609, it is determined whether the area obtained in step 608 and the area updated in step 607 are adjacent to each other.

  As a result, if it is adjacent to the updated area, the process proceeds to step 610. If it is not adjacent to the updated area, the process proceeds to step 611.

  FIG. 4C shows the area 405 updated in step 607 added to FIG. The area 403 obtained in step 608 is adjacent to the area 405 updated in step 607.

  In step 610, the image data updated in step 607 is held in the reference image holding unit 105 as reference image data.

In step 611, for the region obtained in step 608, the background image update unit 112 is controlled so that the image data updated in step 607 is updated with reference image data.
As a result, image data as shown in FIG. 4D is generated.

  In step 612, the image data updated in step 611 is held in the reference image holding unit 105 as reference image data.

In step 701 in FIG. 7, it is determined whether or not the end determination timer has reached a predetermined value.
If it is determined that the predetermined value has been reached, the process proceeds to step 705. If it is determined that the predetermined value has not been reached, the process proceeds to step 702.

  In step 702, it is determined whether or not the determination button of the imaging apparatus 101 has been pressed by the user.

  If it is determined that the enter button has been pressed, the process proceeds to step 705. If it is determined that the enter button has not been pressed, the process proceeds to step 703.

  In step 703, it is determined whether or not the user has pressed the shutter button of the imaging apparatus 101.

  If it is determined that the shutter button has been pressed, the process proceeds to step 706. If it is determined that the enter button has not been pressed, the process proceeds to step 704.

  In step 704, it is determined whether the self-timer has timed out after a predetermined time.

  If the predetermined time has elapsed and timed out, the process proceeds to step 706. If the predetermined time has not elapsed and timed out, the process returns to step 701.

  In step 705, it is determined that the background image generation process has ended, and the process ends.

  In step 706, it is determined that the background image generation process has not ended, and the process ends.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  The present invention relates to an imaging apparatus, and in particular, can be used for an imaging apparatus that generates an image obtained by removing a moving object from a plurality of images and a control method for the imaging apparatus.

DESCRIPTION OF SYMBOLS 101 Image pick-up apparatus 102 Input image 103 Distance information 104 Imaging part 105 Standard image holding part 106 Reference image holding part 107 Reference image distance information holding part 108 Reference image distance information holding part 109 Background image generation part 110 Difference information generation part 111 Adjacent area detection Unit 112 background image update unit 113 output image

Claims (9)

In an imaging device that generates a background image in which a moving object is removed from a captured image,
Imaging means for imaging a subject and outputting captured image data;
Reference image holding means for holding the captured image data as a reference image;
Reference image holding means for holding the captured image data captured at a time different from the reference image as a reference image;
Reference image distance information holding means for holding the distance to the subject in the reference image as reference image subject distance information;
Reference image distance information holding means for holding the distance to the subject in the reference image as reference image subject-specific distance information;
Background image generating means for generating a background image from the reference image and the reference image using the reference image subject distance information and the reference image subject distance information;
Difference information generating means for generating difference information from the reference image and the background image;
An adjacent region detection unit that detects whether the region obtained from the difference information and the region updated by the background image generation unit are adjacent to each other;
Background image update means for updating the area detected by the adjacent area detection means of the background image with the reference image;
An imaging device comprising: The imaging apparatus according to claim 1, wherein after the processing by the background image updating unit is completed, the reference image holding unit and the reference image are used to execute the next processing using the processed image and distance information as reference image information. An image pickup apparatus characterized by being held in a distance information holding means. The imaging apparatus according to claim 1, wherein after the processing by the background image update unit is completed, the imaging unit is controlled to capture a new image, and the new image and distance information are used as reference image information. An image pickup apparatus, wherein the image pickup apparatus is held in the reference image holding unit and the reference image distance information holding unit to execute processing. The background image generation means according to claim 1, wherein the reference image distance information is compared with the reference image distance information, an area farther from the reference image is detected, and the reference image is detected for the detected area. An imaging apparatus, wherein a background image is generated by updating with the reference image. The image processing apparatus according to claim 1, wherein the processing is repeatedly executed in the background image generation unit, the generation unit, and the background image update unit. The imaging apparatus according to claim 2 or 3, wherein the processing is repeatedly executed. The imaging apparatus according to claim 6 or 7, wherein the repeated execution of the process is terminated by an instruction from a user. The imaging apparatus according to claim 6 or 7, wherein the repetition of the process is terminated when a predetermined time has elapsed. In a control method of an imaging apparatus that generates a background image in which a moving object is removed from a captured image,
An imaging step of imaging a subject and outputting captured image data;
A reference image holding step for holding the captured image data in a memory as a reference image;
A reference image holding step of holding the captured image data captured at a time different from the reference image in a memory as a reference image;
A reference image distance information holding step of holding a distance to a subject in the reference image as a reference image subject distance information in a memory;
A reference image distance information holding step of holding a distance to a subject in the reference image as reference image subject distance information in a memory;
A background image generation step of generating a background image from the reference image and the reference image using the reference image subject distance information and the reference image subject distance information;
A difference information generating step for generating difference information from the reference image and the background image;
An adjacent region detection step for detecting whether the region obtained from the difference information and the region updated in the background image generation step are adjacent;
A background image update step of updating the region detected in the adjacent region detection step of the background image with the reference image;
A method for controlling an imaging apparatus, comprising: