JP2007074620A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus or method for imaging a human figure without shake. <P>SOLUTION: An imaging apparatus comprises an imaging means for performing imaging a plurality of times as pre-imaging before imaging is instructed by a user, and performing imaging for recording on a recording medium when imaging is instructed by the user. A human figure is detected for each image captured as pre-imaging and based on a plurality of detection results, the motion of the human figure in the image is estimated. In accordance with the motion, the human figure is suppressed from being shaken in the image captured by the imaging means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique that is effective when applied to an imaging device that captures still images.

If imaging is performed while the subject is moving, the subject may be blurred and imaged. As a technique for suppressing such blurring, a moving object is detected from the difference between a plurality of images acquired while the release button is half-pressed, and the shutter speed is adjusted according to the speed of the movement. A technique has been proposed (see Patent Document 1).
JP 2004-120576 A

  However, even when using the above-described conventional technology, if there is an object that moves at a speed different from that of a person or an object that moves in a different movement direction, the shutter speed is adjusted according to the movement of such an object, There was a case where a person was blurred and imaged.

  Accordingly, an object of the present invention is to solve these problems and to provide an apparatus and a method for realizing imaging without blurring a person.

  In order to solve the above problems, the present invention performs imaging so that a human image is not blurred. In order to perform such imaging, the present invention has the following configuration. Note that the image of a person here may be an entire image of a human body or an image of a part of a human body. The part of the human body refers to, for example, the head, face, upper body, and organs (forehead, eyebrows, eyes, ears, nose, lips, etc.) included in the face.

  A first aspect of the present invention is an imaging apparatus, which includes an imaging unit, a detection unit, and a shake suppression unit. The imaging unit performs imaging a plurality of times as prior imaging before the user gives an imaging instruction, and performs imaging for recording on a recording medium when the imaging instruction is given by the user. A detection means detects a person for every image imaged as prior imaging. The blur suppression unit estimates the movement of the person in the image based on the plurality of detection results, and suppresses the person from shaking in the image captured by the imaging unit in accordance with the movement.

  According to the first aspect of the present invention, the motion of a person is estimated from a plurality of images captured as pre-imaging. Then, the blur suppression unit suppresses that the image of the person is blurred and captured in accordance with the movement. For this reason, the blurring of the person is effectively suppressed.

  The blur suppression unit in the first aspect of the present invention may be configured to adjust the shutter speed in the imaging unit according to the movement. In the present invention configured as described above, the shutter speed of the imaging unit is controlled to be faster as the person moves faster. For this reason, even if the person in the image moves quickly, the image of the person is effectively prevented from being blurred.

Further, in the configuration in which the shutter speed is determined as the control content as described above, the detection unit may be configured to acquire the size of the person image. In this case, the blur suppression unit may be configured to adjust the shutter speed faster as the size is smaller. In general, when the size of the subject is small, the influence of the blurring on the person who sees the image when the same level of blurring occurs is greater than when the subject size is large. For this reason, according to the present invention configured as described above, when the size of the subject is small, the shutter speed is controlled to be faster, so that the influence of blurring can be suppressed.

  The image pickup unit according to the first aspect of the present invention may be configured to be able to suppress blurring of the subject projected on the image pickup device by relatively moving the image pickup device and / or the optical system when performing image pickup. In this case, the blur suppression unit may be configured to adjust the relative movement of the image sensor and / or the optical system in accordance with the speed and direction of movement.

  In the present invention configured as described above, when image capturing is performed, relative movement of the image sensor and / or the optical system is adjusted according to the movement of the person so as not to cause blurring. For this reason, it is suppressed that a person's image blurs.

  A second aspect of the present invention is an imaging apparatus, which includes an imaging unit, a tracking unit, and a shake suppression unit. The imaging unit repeatedly performs imaging as prior imaging before the user gives an imaging instruction, and when the user gives an imaging instruction, performs imaging for recording on a recording medium. The tracking means tracks a person in an image that is repeatedly captured as pre-imaging. The blur suppression unit estimates the movement of the person in the image based on the tracking result of the tracking unit, and suppresses the person from shaking in the image captured by the imaging unit in accordance with the movement.

  According to the second aspect of the present invention, the shake suppression unit estimates the movement of the person according to the tracking result of the tracking unit, and suppresses the image of the person from being blurred based on the movement. For this reason, the blurring of the person is effectively suppressed.

  In addition, as in the case of the first aspect of the present invention described above, the blur suppression means and the image pickup means in the second aspect of the present invention can be controlled by adjusting the shutter speed and the image sensor and / or the optical system. You may be comprised so that it may become.

  The first aspect and the second aspect may be realized by executing the program by the information processing apparatus. That is, the present invention specifies the processing executed by each unit in the first aspect and the second aspect described above as a program for causing the information processing apparatus to execute or a recording medium on which the program is recorded. Can do. Further, the present invention may be specified by a method in which the information processing apparatus executes the processing executed by each of the above-described means.

  According to the present invention, it is possible to capture an image without blurring a person.

  [System Configuration] The configuration of the imaging apparatus 1 will be described. In terms of hardware, the imaging device 1 includes a CPU (Central Processing Unit), a main storage device (RAM), an auxiliary storage device, an imaging unit, and the like connected via a bus.

FIG. 1 is a diagram illustrating functional blocks of the imaging apparatus 1. The imaging device 1 is loaded with various programs (OS, applications, etc.) stored in the auxiliary storage device and executed by the CPU, so that the imaging unit 2, the temporary storage unit 3, and the head detection unit 4 are loaded. , And functions as a device including the head tracking unit 5 and the shake suppression unit 6. The head detection unit 4, the head tracking unit 5, and the shake suppression unit 6 are realized by executing a program by the CPU. Further, the head detection unit 4, the head tracking unit 5, and the shake suppression unit 6 may be configured as a dedicated chip. Next, each functional unit included in the imaging device 1 will be described. Note that the imaging apparatus 1 is configured such that the head of a person is tracked and the imaging unit 2 is controlled by the shake suppression unit 6 according to the speed of movement of the head in the image. Such a configuration of the imaging apparatus 1 is merely an example, and may be configured to be controlled based on other portions of the person who is the subject.

<Imaging Unit> The imaging unit 2 is configured as an apparatus using an optical system, a mechanical mechanism, an imaging element (for example, a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor), a motor, and the like. The optical system includes a correction optical system for adjusting the imaging position on the image sensor. The optical system may also include, for example, a zoom lens for realizing a zoom function, a focus lens for focusing on an arbitrary subject, and the like. The mechanical mechanism includes a mechanical shutter, a diaphragm mechanism, a filter, and the like. The motor includes a shutter motor. The motor may include a zoom lens motor, a focus motor, and the like. The imaging unit 2 may also include a mechanism for adjusting the relative movement of the optical system and the imaging element as necessary.

  When the shutter speed and the aperture are controlled by the blur suppression unit 6, the imaging unit 2 performs imaging with the shutter speed and the aperture according to the control. In addition, when the shake suppression unit 6 controls the optical system and the imaging device, the imaging unit 2 performs imaging by adjusting the relative movement of the optical system and the imaging device according to the control.

  The imaging unit 2 performs imaging by converting an image of a subject imaged through the optical system in the state controlled as described above into an electrical signal by an imaging device.

  The imaging unit 2 performs user imaging processing and pre-imaging processing. The user imaging process is an imaging process performed when the user presses the release button, and the captured image is recorded in a nonvolatile storage device (not shown). On the other hand, the prior imaging process is an imaging process that is repeatedly executed when the power of the imaging apparatus 1 is turned on, when the imaging is possible, or when the release button is pressed halfway. The captured image is temporarily recorded in the temporary storage unit 3. The image captured by the pre-imaging process is used for the shake suppression process in the imaging apparatus 1.

  <Temporary Storage Unit> The temporary storage unit 3 is configured using a storage device. Any specific technique such as a volatile storage device or a nonvolatile storage device may be applied to the storage device used for the temporary storage unit 3.

  The temporary storage unit 3 temporarily stores data of an image captured by the imaging unit 2 performing pre-imaging processing. The image data stored in the temporary storage unit 3 is read by the head detection unit 4 and the head tracking unit 5. The temporary storage unit 3 holds data of an image that is a target of the processing at least until the processing by the head detection unit 4 and the head tracking unit 5 is completed.

  <Head Detection Unit> The head detection unit 4 reads out image data from the temporary storage unit 3, detects a human head from the image, and indicates the detected position and size of the head. Identify the information. The head detection unit 4 may be configured to detect the head by detecting the face by template matching using a reference template corresponding to the outline of the entire face, for example. The head detection unit 4 may be configured to detect a vertex such as a head by chroma key processing and detect the head based on the vertex. The head detection unit 4 may be configured to detect a head by detecting a region close to the skin color and detecting the region as a face. Further, the head detection unit 4 may be configured to perform learning based on a teacher signal using a neural network and detect a face-like area or a head-like area as a head. In addition, the detection processing by the head detection unit 4 may be realized by applying any existing technique.

  <Head Tracking Unit> The head tracking unit 5 tracks (tracks) the head detected by the head detection unit 4 in an image that is updated and recorded in the temporary storage unit 3. Moreover, the head tracking part 5 determines the head used as the object of a tracking process according to a specific reference | standard, when the head of a several person is detected from an image. The predetermined reference is, for example, a face size, a face orientation, a face position in an image, or the like. The tracking process in the head tracking unit 5 is performed by searching for and tracking feature points included in the head (for example, feature points on the forehead, eyebrows, eyes, ears, nose, lips, etc.) around the feature points of the previous image. It can be realized by doing. Further, this tracking process may be realized by other existing technology.

  The head tracking unit 5 holds the tracking result, that is, the movement history of the head to be tracked (such as the position of the head in the image) for a certain number of frames. This fixed number is a number that is at least equal to or greater than the number required in the processing of the shake suppression unit 6 described later.

  <Blur Suppression Unit> The blur suppression unit 6 controls the imaging unit 2 based on the movement history of the head tracked by the head tracking unit 5 so that the head does not blur in the captured image. Do. The blur suppression unit 6 can be configured to suppress blur by controlling the shutter speed, or can be configured to suppress blur by controlling the optical system and / or the image sensor. Hereinafter, each configuration will be described.

  << Configuration for Controlling Shutter Speed >> In this case, first, the blur suppression unit 6 acquires the past movement amount of the head tracked by the head tracking unit 5. FIG. 2 is a diagram schematically showing the movement amount (D) of the head. For example, the blur suppression unit 6 moves the head between the latest image (image captured last in the pre-imaging process: m-th image) and the previous image (m−1th image). To get. The shake suppression unit 6 may acquire a movement amount of a rectangle surrounding the head or face as the movement amount of the head (FIG. 2A). Further, the shake suppression unit 6 may acquire the movement amount of the feature point used for the tracking process as the movement amount of the head (FIG. 2B). Then, the speed of movement of the head in the image is obtained from the imaging interval (time) between the latest image and the previous image and the above movement amount.

  Next, the blur suppression unit 6 determines the control content of the shutter speed based on this speed. For example, the blur suppression unit 6 determines the shutter speed by acquiring the shutter speed corresponding to this speed from the shutter speed table. FIG. 3 is a diagram illustrating an example of a shutter speed table. The shutter speed table is a table that defines correspondence between speed and shutter speed. The shutter speed table may be configured by associating only the speed and the shutter speed (for example, FIG. 3A), or configured by associating the speed and brightness in the image with the shutter speed and the aperture. It is also possible (for example, FIG. 3B). Here, the brightness in the image may be the brightness of the entire image, or the brightness of the head such as the head to be tracked and the face in the head. Then, the blur suppression unit 6 controls the imaging unit 2 according to the determined shutter speed and aperture.

  << Configuration for Controlling Optical System / Imaging Device >> In this case, first, the blur suppression unit 6 acquires the past speed and direction of movement of the head tracked by the head tracking unit 5. The speed and direction may be acquired as, for example, the speed and direction of head movement between the latest image and the previous image. Specifically, the shake suppression unit 6 may acquire the speed and direction of movement of a rectangle surrounding the head or face as the speed and direction of movement of the head. Further, the shake suppression unit 6 may acquire the speed and direction of movement of the feature points used for the tracking process as the speed and direction of movement of the head.

Next, the blur suppression unit 6 controls the optical system and the imaging element so that the imaging range moves following the moving direction of the head. FIG. 4 is a diagram illustrating an example of control of the optical system and the image sensor by the blur suppression unit 6. In each of (a) and (b), the left side shows a state immediately before imaging, and the right side shows a state immediately after imaging. In any case, the head of the subject person moves from the upper right of the imaging frame to the lower left from immediately before to immediately after imaging. This movement causes blurring of the subject person's image. Such blurring occurs when the head of the subject person moves in the real world (subject blurring) and when the imaging device 1 moves (camera blurring).

  FIG. 4A is a diagram illustrating a case where the control by the shake suppression unit 6 is not performed. In this case, movement of the head of the subject person causes blurring in the image of the subject person.

  FIG. 4B is a diagram illustrating a case where control by the shake suppression unit 6 is performed. The broken-line frame on the right side indicates the position of the imaging range when control is not performed. In this case, the head of the subject person has not moved within the imaging range. That is, the blur suppression unit 6 controls the relative movement of the optical system and / or the image sensor based on the speed and direction of movement of the head, so that the head of the subject person falls within the imaging range during imaging. Do not move relative to it. In other words, the blur suppression unit 6 controls the optical system or the image sensor so that the head of the subject person is projected onto the same part of the image sensor while the image is captured. Such control suppresses the shake of the head of the subject person.

  Note that the blur suppression unit 6 and the imaging unit 2 may be configured such that both the shutter speed control and the optical system / imaging device control are performed. In addition, when the control of the shutter speed and the control of the optical system / image sensor are performed in this way, the shutter speed is controlled to be slower than when only the shutter speed is controlled. good. By controlling in this way, blurring is suppressed by the control of the optical system / image sensor, and further, the effect of panning can be obtained by lowering the shutter speed. By obtaining the effect of panning, there is an advantage that, for example, the face appears clearly.

  [Operation Example] FIG. 5 is a flowchart illustrating an operation example of the imaging apparatus 1. First, the imaging unit 2 executes a pre-imaging process (S01). An image captured by the pre-imaging process is stored in the temporary storage unit 3. Next, the head detection unit 4 detects the head of the subject person in the image captured by the pre-imaging process (S02). Then, the head tracking unit 5 continuously tracks the detected head in the image captured by the subsequent pre-imaging process (S03, S05).

  When the user presses the release button of the imaging device 1 while tracking by the head tracking unit 5 is performed (S04-YES), the shake suppression unit 6 performs the shake suppression process, and the imaging unit 2 User imaging processing is performed according to the control of the suppression unit 6 (S06). Then, the image captured by the user imaging process is recorded in an image storage unit (not shown) (S07).

  [Operation / Effect] When the imaging apparatus 1 performs user imaging processing, the imaging unit 2 is controlled according to the movement of the head of the subject person. For this reason, according to the imaging device 1, particularly in the head of the subject person, the influence of subject blur and camera shake is suppressed, and a clearer image is acquired.

[Modification] The blur suppression unit 6 may be configured to cut out a predetermined portion from the image captured by the image sensor and record the image as a result of the user imaging process. FIG. 6 is a schematic diagram showing the control content of the shake suppression unit 6 configured as described above. A thick frame on the outside indicates a maximum range of an image captured by the image sensor (hereinafter referred to as “maximum range”). A thin line frame on the inner side indicates an image range (hereinafter referred to as “output range”) that is actually recorded (output) in the user imaging process. Generally, since the maximum range imaged by the image sensor is different from the actually recorded range, the thin line frame is smaller than the thick line frame as shown in FIG.

  FIG. 6A is a diagram illustrating a case where the control by the shake suppression unit 6 is not performed. In this case, the control of the output range by the shake suppression unit 6 is not performed. For this reason, the movement of the head of the subject person shifts the position of the subject head within the output range.

  FIG. 6B is a diagram illustrating a case where the control by the shake suppression unit 6 is performed, and a thin broken line frame on the right side in the thick frame indicates an output range when the control is not performed. A thin solid frame on the left side of the thick frame indicates an output range when control by the shake suppression unit 6 is performed. In this case, the head of the subject person has not moved within the output range. That is, the shake suppression unit 6 shifts the output range within the maximum range based on the speed and direction of movement of the head so that the head position of the subject person does not move relative to the output range. To do. By such control, the position of the head of the subject person is suppressed from being shifted within the output range.

  Such control by the blur suppression unit 6 may be performed in combination with control of the shutter speed. Further, the blur suppression unit 6 may be configured to perform only the control of the output range described above without performing control of the shutter speed or control of the optical system / image sensor.

  In addition, the shake suppression unit 6 may be configured to control the shutter speed based on the size of the head to be tracked and the speed of movement thereof. In this case, the faster the movement speed, the faster the shutter speed is controlled, and the smaller the head size, the faster the shutter speed is controlled. For example, the blur suppression unit 6 may be configured to calculate “speed / head size” and control the shutter speed faster as the value increases.

  Further, the imaging apparatus 1 may be configured not to include the head tracking unit 5. In this case, the blur suppression unit 6 determines the control content of the imaging unit 2 based on the detection results of the head detection unit 4 in a plurality of images. Specifically, for example, the imaging device 1 can be configured as follows.

  First, the temporary storage unit 3 holds at least two images taken by the pre-imaging process from new ones. When the release button is pressed, the head detection unit 4 detects the head from the latest image and the previous image. Then, based on this detection result, the blur suppression unit 6 determines the control content of the imaging unit 2.

  If the head tracking unit 5 is not provided, the imaging device 1 may be configured as follows. First, the head detection unit 4 detects the head for each image captured by the pre-imaging process and holds the position. When the release button is pressed, the blur suppression unit 6 determines the control content of the imaging unit 2 from the detection position of the latest image and the previous image.

  When the head tracking unit 5 is not provided as described above, the pre-imaging process is executed so as to capture a predetermined number of images (for example, two images) when the release button is pressed halfway. Also good.

It is a figure showing an example of a functional block of an imaging device. It is the figure which showed the movement amount of the head typically. It is a figure which shows the example of a shutter speed table. It is a figure which shows the example of control of the optical system and imaging device by a blurring suppression part. It is a flowchart which shows the operation example of an imaging device. It is a figure which shows typically the process of the blur suppression part by a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Imaging part 3 Temporary storage part 4 Head detection part 5 Head tracking part 6 Shake suppression part

Claims (9)

Imaging means for performing imaging a plurality of times as pre-imaging before an imaging instruction is given by the user, and for taking an image for recording on a recording medium when the imaging instruction is given by the user;
Detecting means for detecting a person for each image captured as pre-imaging;
An imaging apparatus comprising: a blur suppression unit that estimates a motion of a person in an image based on a plurality of detection results, and suppresses the blur of the person in an image captured by the imaging unit according to the motion.   The imaging apparatus according to claim 1, wherein the blur suppression unit adjusts a shutter speed in the imaging unit according to the speed of the movement. The imaging unit is configured to be able to suppress blurring of a subject projected on the imaging element by relatively moving the imaging element and / or the optical system when performing imaging.
The imaging apparatus according to claim 1, wherein the blur suppression unit adjusts a relative movement of the imaging element and / or the optical system in accordance with the speed and direction of the movement. The detection means acquires a size in a person image,
The imaging apparatus according to claim 2, wherein the blur suppression unit adjusts the shutter speed faster as the size is smaller. Imaging means for repeatedly performing imaging as prior imaging before the user gives an imaging instruction, and for taking an image for recording on a recording medium when the imaging instruction is given by the user
Tracking means for tracking a person in an image repeatedly captured as pre-imaging;
Based on the result of tracking by the tracking unit, the motion of the person in the image is estimated, and according to this motion, the blur suppression unit that suppresses blurring of the person in the image captured by the imaging unit;
An imaging apparatus comprising: An information processing apparatus including an imaging unit that performs imaging a plurality of times as prior imaging before the user gives an imaging instruction and performs imaging for recording on a recording medium when the imaging instruction is given by the user Detecting a person for each image captured as:
The information processing apparatus estimating a movement of a person in an image based on a plurality of detection results, and suppressing the movement of the person in an image captured by the imaging unit according to the movement;
An imaging method including: An information processing apparatus including an imaging unit that repeatedly performs imaging as prior imaging before the user gives an imaging instruction and performs imaging for recording on a recording medium when the user instructs imaging is repeated as prior imaging. Tracking a person in a captured image;
The information processing apparatus estimating a movement of a person in an image based on a tracking result, and suppressing the movement of the person in an image captured by the imaging unit according to the movement;
An imaging method including: For an information processing apparatus including an imaging unit that performs imaging a plurality of times as pre-imaging before a user gives an imaging instruction, and performs imaging for recording on a recording medium when the user gives an imaging instruction.
Detecting a person for each image captured as pre-imaging;
Estimating a movement of a person in an image based on a plurality of detection results, and suppressing the movement of the person in an image captured by the imaging unit according to the movement;
A program for running For an information processing apparatus including an imaging unit that repeatedly performs imaging as pre-imaging before a user issues an imaging instruction and performs imaging for recording on a recording medium when the user instructs to take an image,
Tracking a person in an image repeatedly imaged as pre-imaging;
Estimating the movement of the person in the image based on the tracking result, and suppressing the movement of the person in the image captured by the imaging unit according to the movement;
A program for running

Images