JP2010273183A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of easily performing panning. <P>SOLUTION: The imaging apparatus includes: an imaging unit which images a subject and generates an image signal; a first detection unit which detects a moving body based on the image signal; a second detection unit which detects information specifying the moving body detected by the first detection unit in a picked-up image; and an image processing unit which performs image processing on an image other than the image of the image area of the moving body in the picked-up image based on the information detected by the second detection unit and specifying the moving body. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus.

  Conventionally, a technique for automatically calculating a shutter speed when a photographer takes a panning shot is known (see Patent Document 1).

JP-A-5-232562

  However, in the conventional technique, when the photographer takes a panning shot, the panning is not performed normally by following the subject to be shot in order to shoot with a long exposure to the extent that the panning can be taken. There was a problem that the panning could not be done well. Therefore, for beginners, there is a problem that panning is difficult.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus that can easily perform panning.

  The present invention has been made to solve the above-described problems, and includes an imaging unit that captures an image of a subject and generates an image signal, a first detection unit that detects a moving body based on the image signal, A second detection unit that detects information that identifies the moving body detected by the first detection unit in the image to be captured, and information that identifies the moving body detected by the second detection unit after shooting. An image processing apparatus comprising: an image processing unit that performs image processing on an image other than the image area of the moving body based on the captured image.

  According to the present invention, the image processing unit of the imaging device can easily perform image processing on images other than the image area of the moving body among the captured images based on the information specifying the moving body after shooting. You can take a panning shot.

1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. It is explanatory drawing which shows the operation | movement outline | summary of an imaging device in case there exists panning. It is explanatory drawing which shows the operation | movement outline | summary of an imaging device in case there is no panning. FIG. 3 is an explanatory diagram for explaining detection of a panning target subject in the case of FIG. 2. FIG. 4 is an explanatory diagram for explaining detection of a panning target subject in the case of FIG. 3. It is a flowchart explaining operation | movement of the imaging device by this embodiment. It is explanatory drawing explaining the image processing of the imaging device by this embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of the imaging apparatus 100 according to the first embodiment of the present invention. The imaging apparatus 100 includes an imaging unit 10, a nonvolatile memory 11, a buffer memory 12, an operation detection circuit 13, a monitor control circuit 14, a monitor (display unit) 15, a memory control circuit 16, a memory 17, and a control unit 20. .

  The imaging unit 10 includes an optical system 1, an angular velocity sensor 2, an optical system control circuit 3, an image sensor control circuit 7, an image sensor 8, and a video circuit 9, and takes an optical image (subject image) by the optical system 1. To generate and output an image signal. The optical system 1 includes, for example, an optical element that corrects exposure to the image sensor 8, a focus lens that has a function of adjusting the focus, an anti-shake lens that has a function of correcting image shake due to camera shake, an aperture blade, and A shutter is provided.

  The angular velocity sensor 2 detects an angle change (including a movement speed and a movement direction) of the imaging apparatus 100. The angular velocity sensor 2 detects the amount of rotation about the three axes of the imaging apparatus 100. For example, in the case of panning, the orientation of the imaging device 100 is changed by the photographer following the subject (moving body) to be panned. In this case, it is possible to detect whether or not the panning is being performed by detecting the angular change of the imaging device 100 by the angular velocity sensor 2.

  The optical system control circuit 3 controls the focus lens to automatically adjust the focus and automatically adjusts the focus (hereinafter referred to as AF) control function, the shutter control function to control the opening / closing of the shutter, and automatically controls the aperture of the aperture blade. AE control function for adjusting the exposure, etc., the optical system 1 is driven, and the exposure to the image sensor 8 is controlled.

  The image pickup device 8 is a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and has a light receiving surface on which an optical image exposed from the optical system 1 is formed. It converts it into an electrical signal and outputs an analog image signal. Further, the image sensor 8 is controlled by the image sensor control circuit 7, and before the shutter is released, the through image signal is continuously output for each frame based on the incident optical image, and the shutter is released. In response, a still image signal for a still image is output based on the incident optical image.

  The video circuit 9 amplifies the image signal output from the image sensor 8 and converts it into a digital signal. The image sensor control circuit 7 drives the image sensor 8 and controls operations such as conversion of an optical image formed on the image sensor 8 into an image signal and output of the converted image signal. For example, the image sensor control circuit 7 performs exposure time control for forming an optical image on the image sensor 8 with an exposure time controlled by the control unit 20, and outputs a plurality of through image signals to the image sensor 8 at intervals of a predetermined time t. Perform through image generation control to be generated. The interval of the fixed time t is, for example, 1/60 seconds.

The nonvolatile memory 11 stores a program for operating the control unit 20 and information such as various settings and shooting conditions input by the user. The non-volatile memory 11 stores information such as the presence / absence of automatic release setting and the setting conditions for automatic release setting based on information input from the operation detection circuit 13, for example.
The buffer memory 12 is a temporary information storage area used for the control process of the control unit 20. For example, the buffer memory 12 is a through image signal output from the image sensor 8 or a captured image signal image-processed by the control unit 20. Etc. are temporarily stored by the control unit 20.

  The operation detection circuit 13 includes input units such as a power switch 13a, a release switch 13b,... 13n, and a control signal corresponding to the operation is sent to the control unit 20 in response to the input unit being operated by the user. Output. For example, the operation detection circuit 13 outputs a release signal to the control unit 20 in response to the release switch 13b being fully pressed.

  For example, the monitor control circuit 14 performs display control such as turning on and off the monitor 15 and adjusting brightness, and processing for causing the monitor 15 to display the image signal image-processed by the control unit 20. The monitor 15 is a display that displays an image according to an image signal, and is a liquid crystal display such as an LCD (Liquid Crystal Display).

  The memory control circuit 16 controls input / output of information between the control unit 20 and the memory 17. For example, the memory control circuit 16 stores the image signal image-processed by the control unit 20 in the memory 17 or the image stored in the memory 17. A process of reading information such as a signal and outputting the information to the control unit 20 is performed. The memory 17 is a storage medium that can be inserted into and removed from the imaging apparatus 100, such as a memory card, and stores an image signal that has undergone image processing by the control unit 20.

  The control unit 20 is a CPU (Central Processing Unit) that controls the operation of each unit of the imaging apparatus 100 based on a program stored in the nonvolatile memory 11. For example, the control unit 20 turns on the power to the imaging apparatus 100, controls the drive of the optical system 1 via the optical system control circuit 3, and captures an image in accordance with a control signal input from the operation detection circuit 13 by a user operation. Drive control of the image sensor 8 via the element control circuit 7 or display control of the monitor 15 via the monitor control circuit 14 is performed.

  The control unit 20 includes an image processing unit 21, a display control unit 22, a first detection unit 23, a second detection unit 24, a first image processing unit 251, and a second image processing unit 252. A post-image processing unit 25 (image processing unit), a panning detection unit 26, an imaging condition setting unit 27, and an imaging control unit 28 are provided.

  The image processing unit 21 performs image processing on the image signals (still image signal and through image signal) output from the image sensor 8 to the video circuit 9. The image processing includes color adjustment processing, white balance adjustment processing, noise reduction processing, and the like. For example, the image processing unit 21 performs image processing on the through image signal so as to be displayed on the monitor 15 as a moving image, and stores it in the buffer memory 12.

The display control unit 22 reads the through image signal that has been image-processed by the image processing unit 21 and is stored in the buffer memory 12 and outputs it to the monitor 15 in real time.
The panning detection unit 26 detects panning of the imaging device 100 based on the rotation amounts around the three axes output from the angular velocity sensor 2.

  The first detection unit 23 detects the motion vector of the subject from the image based on the image signal output from the imaging unit 10, the detected motion vector, the panning detected by the panning detection unit 26, and the detected motion Based on the vector, the subject for panning is detected. The image based on the image signal output from the imaging unit 10 is, for example, an image subjected to image processing by the image processing unit 21.

  For example, when the panning is detected by the panning detection unit 26, the first detection unit 23 detects the subject in the image based on the panning detected by the panning detection unit 26 and the image signal output from the imaging unit 10. The subject corresponding to the motion vector is detected as a panning target subject (main subject). Further, the first detection unit 23 determines that the motion vector of the subject in the image based on the image signal output from the imaging unit 10 is a predetermined value when panning is not detected by the panning detection unit 26. Is detected as a subject for panning.

  The second detection unit 24 detects information specifying the panning target subject detected by the first detection unit 23 in the captured image when the image is captured. The information for specifying the panning target subject includes, for example, information indicating the moving direction of the panning target subject in the captured image, information indicating the position of the panning target subject in the captured image, and imaging. Information indicating the shape of the subject to be shot in the shot image is included. Further, each piece of information included in the information for specifying the panning target subject may be information predicted when the panning target subject is shot (exposure).

  The post-shooting image processing unit 25 captures images other than the image area of the panning target subject among the picked up images based on the information specifying the panning target subject detected by the second detection unit 24 after the shooting. To process.

  The first image processing unit 251 included in the post-shooting image processing unit 25 moves and blurs an image other than the panning target subject based on the information specifying the panning target subject detected by the second detection unit 24. Process. In addition, the first image processing unit 251 performs a movement blurring process by determining a blur direction based on information indicating the moving direction of the panning target subject included in the information for specifying the panning target subject.

  The second image processing unit 252 included in the post-shooting image processing unit 25 synthesizes the image blurred by the first image processing unit 251 and the image of the subject to be shot. In addition, the second image processing unit 252 synthesizes so that the image of the subject to be shot is over the blurred image.

  The imaging condition setting unit 27 sets shooting conditions based on conditions such as a set shooting mode. This shooting condition includes a high-speed shutter. Here, the high-speed shutter is a shutter speed that is faster than that in the case of panning, for example, the shutter speed at which a subject to be panned is imaged in a stationary manner.

  In response to the input of the release signal, the imaging control unit 28 controls the imaging element 8 based on the imaging conditions set by the imaging condition setting unit 27 to capture an image. On the other hand, the image processing unit 21 is controlled to execute the above-described image processing. In addition, the shooting control unit 28 controls the post-shooting image processing unit 25 with respect to the image captured in this manner, and based on the information specifying the panning target subject detected by the second detection unit 24. Then, image processing is performed on images other than the image area of the subject to be panned out of the captured images. Note that the image processing unit 21 and the post-shooting image processing unit 25 may use the buffer memory 12 as a work area when performing image processing.

  Further, the shooting control unit 28 stores the image processed by the post-shooting image processing unit 25 in the memory 17 via the memory control circuit 16. The shooting control unit 28 may associate the image processed by the image processing unit 21 with the image processed by the post-shooting image processing unit 25 and store the image in the memory 17 via the memory control circuit 16. .

  Next, an outline of the operation as an example of the imaging apparatus 100 described above will be described with reference to FIGS. 2 and 3. FIG. 2 shows a case where the photographer pans the imaging apparatus 100 by following a traveling car (see reference A in FIG. 2) and takes a panning shot. FIG. 3 shows a case where the photographer has fixed the imaging device 100 and photographed a traveling car (see reference A in FIG. 3). In any case, a case where there is a house in the background (see reference sign B in FIGS. 2 and 3) will be described. Each of FIGS. 2 and 3 corresponds to an image on the screen captured by the image capturing apparatus 100.

  First, the case of FIG. 2 will be described. In FIG. 2, the photographer pans the image capturing apparatus 100 while following a car traveling from left to right and captures an image. In this case, the panning detection unit 26 detects this panning. The first detection unit 23 detects a motion vector of each subject between frames (see FIGS. 2A and 2B) (see FIG. 2D).

  In the case of FIG. 2, since the imaging apparatus 100 is panned, the house as the background is moving from right to left in the screen (see symbols B1 and B2 in FIG. 2D). On the other hand, since the imaging device 100 is panned by following the vehicle, the vehicle is not moving in the screen (see reference A in FIG. 2D). Therefore, in this case, the first detection unit 23 detects a car that is a subject that has not moved in the screen as a subject to be shot.

  Thereafter, in response to the photographer pressing the release switch 13b, the imaging device 100 captures a still image without blurring with a high-speed shutter (see FIG. 2C). After the imaging, the first image processing unit 251 of the post-imaging image processing unit 25 performs image processing on the background or the like other than the panning target subject (here, only the house is illustrated) and hangs it in the panning direction ( (See symbol B3 in FIG. 2 (e)). Thereafter, the second image processing unit 252 of the post-shooting image processing unit 25 synthesizes the hanging background and the non-hanging panning target subject (see FIG. 2F).

  In this way, even if the panning target subject is not followed and panning is not successful, shooting with the high-speed shutter captured as a still image captures the panning subject subject without blurring. can do. Then, by performing image processing on images other than the panning target subject by image processing after shooting, it is possible to capture the same image as when panning is normally performed following the panning target subject.

  Next, the case of FIG. 3 will be described. In FIG. 3, the photographer fixes the imaging device 100 and photographs a car moving from left to right on the paper. The panning detection unit 26 detects panning that the imaging apparatus 100 is fixed. The first detection unit 23 detects the movement and motion vector of each subject between frames (see FIGS. 3A and 3B) (see FIG. 3D). In the case of FIG. 3, since the imaging device 100 is fixed, the house that is the background does not move within the screen (see the symbol B in FIG. 3D). Conversely, since the car is moving, the car is moving within the screen (see symbols A1 and A2 in FIG. 3D). Therefore, in this case, the first detection unit 23 sets a car, which is a subject moving within the screen, as a subject to be shot.

  Thereafter, in response to the photographer pressing the release switch 13b, the imaging apparatus 100 captures a still image without blurring with a high-speed shutter (see FIG. 3C). Thereafter, image processing similar to that described with reference to FIGS. 2E and 2F is performed.

  In this way, even when the imaging device 100 is fixed and is not panned by following the subject to be shot, it is the same as when panning is normally performed by following the subject to be shot. Imaging can be performed.

  Next, with reference to FIG. 4 and FIG. 5, a description will be given of a method in which the first detection unit 23 detects the panning target subject in each of the cases of FIG. 2 and FIG. In FIGS. 4 and 5, a car is assumed to be a subject 2 and a background such as a house is assumed to be a subject 1.

  FIG. 4 is a diagram showing the panning (camera pan speed) detected by the panning detection unit 26 in the case of FIG. 2 and the movement vector for each subject detected by the first detection unit 23 for each frame. . In this case, since the value of the camera pan speed is not 0, it can be determined that the subject to be shot is being followed. In addition, it can be seen that the subject 2 that is the subject of the panning shot does not move within the screen, and the subject 1 that is not the subject of the panning shot moves within the screen. Accordingly, when the first detection unit 23 follows the panning target subject, that is, when panning is detected by the panning detection unit 26, the first subject 2 is a subject that has not moved in the screen. (Car) is detected as a subject for panning.

  FIG. 5 is a diagram showing the panning (camera pan speed) detected by the panning detection unit 26 in the case of FIG. 3 and the movement vector for each subject detected by the first detection unit 23 for each frame. . In this case, since the value of the camera pan speed is almost 0, it can be seen that the subject to be shot is not being followed. In addition, it can be seen that the subject 2 that is the subject of the panning shot moves within the screen, and the subject 1 that is not the subject of the panning shot does not move within the screen. Accordingly, when the first detection unit 23 does not follow the subject to be shot, that is, when panning is not detected by the panning detection unit 26, the first detection unit 23 is the subject 2 that is moving within the screen. (Car) is detected as a subject for panning.

  In FIGS. 4 and 5, a release operation is started between frames n + 3 and n + 4, and exposure is performed between frames n + 4 and n + 5 to capture an image. In general, a predetermined time interval occurs between the start of the release operation and the exposure. Therefore, as shown in FIG. 6 or FIG. 7, the position of the panning target subject in the screen changes at a predetermined time interval that occurs between the start of the release operation and the exposure. Therefore, the post-shooting image processing unit 25 cannot perform image processing based on the position of the subject to be shot in the frame immediately before the release operation is started (frame n + 3).

  Accordingly, the second detection unit 24 estimates the position of the panning target subject predicted during shooting (exposure) in the screen (see the subject 2 predicted position in FIGS. 4 and 5). Then, the post-photographing image processing unit 25 performs image processing based on the position of the panning target subject in the screen predicted by the second detection unit 24. As a result, even if the position of the panning target subject in the screen changes at a predetermined time interval that occurs between the start of the release operation and the exposure, the post-shooting image processing unit 25 executes image processing. be able to. Note that the second detection unit 24 may predict not only the position of the panning target subject in the screen but also the direction and shape of the panning target subject in the screen.

  Next, an operation as an example of the imaging apparatus 100 will be described with reference to FIG. Here, an operation of the imaging apparatus 100 after the user presses the power switch 13a and the power is turned on will be described.

  First, in response to the power supply of the imaging apparatus 100 being turned on, in step S001, the circuit system is reset to initialize the system. In the next step S003, the image sensor 8 is activated to make it possible to continuously shoot through images. In the next step S004, the monitor 15 is turned on, and a through image from the image sensor 8 is displayed on the monitor 15.

  In the next steps S021, S022, and S023, the imaging condition setting unit 27 performs AE control, AWB control, and AF control, and sets imaging conditions. In this case, the exposure, color, and focus of the image displayed on the monitor 15 are controlled to be optimal.

  The AE control is to measure the luminance of the subject and adjust the imaging speed of the imaging device 8 and the aperture of the optical system 1 so that the exposure is appropriate. AWB control is a process of measuring the subject color and adjusting the amplification factor for each color so as to obtain an optimum color. The AF control is control for focusing on a subject.

  In the next step S024, the control unit 20 determines whether or not the release switch 13b is half pressed. If the release switch 13b is not half-pressed in step S024, the control unit 20 repeats the processing from step S021. On the other hand, when the release switch 13b is half-pressed in step S024, the control unit 20 advances the process to step S041.

  Next, in steps S041, S042, and S043, the imaging condition setting unit 27 performs AE control, AWB control, and AF control again, and sets imaging conditions. In this case, the exposure, color, and focus of the image to be captured are controlled to be optimal.

  Next, in step S044, the first detection unit 23 detects a panning target subject. In step S044, the first detection unit 23 determines whether or not the imaging device 100 is panned by the panning detection unit 26. If the imaging device 100 is panned, the position in the screen does not change. When a subject exists, it is assumed that this subject is detected as a panning target subject and the panning is performed. On the other hand, when the imaging device 100 is not panned and the imaging device 100 is fixed, the first detection unit 23 takes a shot of the subject when there is a subject whose position in the screen changes. It is detected as a target subject and is determined to be a panning shot.

  Next, in step S045, the imaging condition setting unit 27 proceeds to step S046 if the determination result in step S044 described above is panning, and proceeds to step S048 if it is not panning.

  In step S046, the imaging condition setting unit 27 sets imaging conditions for the high-speed shutter. Next, in step S047, the second detection unit 24 determines the position in the screen of the panning target subject (subject position), the shape of the panning target subject (subject shape), and the moving direction of the panning target subject or the imaging device. Information indicating the panning direction of 100 (subject moving direction) is stored in the storage unit. This storage unit is, for example, the nonvolatile memory 11 or the buffer memory 12.

  In step S047 described above, the second detection unit 24 causes the storage unit to store the predicted position (subject predicted position) in the screen of the panning target subject when the release signal is input for imaging. For example, the second detection unit 24 calculates the predicted position from the moving speed of the past subject to be shot in the screen. Also, the subject shape and the subject moving direction are predicted in the same manner as the subject predicted position.

  Next, in step S048, the control unit 20 determines whether or not the release switch 13b that has been half-pressed is further pressed and a release signal is input from the release switch 13b. If no release signal is input in step S048, the process proceeds to step S049.

  In step S049, it is determined whether the release switch 13b is half-pressed. If the release switch 13b is half-pressed in this step S049 (when waiting for release), the process proceeds to step S050, and if it is not half-pressed, the processes from step S021 are repeated.

  In step S050, it is determined whether the set shooting mode is the panning mode. If it is not the panning mode, the process from step S044 is repeated, and if it is the panning mode, the process from step S041 is repeated. That is, when not in the panning mode, AE, AE and AWB are locked, and when in the panning mode, AE, AE and AWB are controlled again.

  On the other hand, when the release signal is input in step S048 described above, one frame shooting process is executed under the shooting conditions set in steps S041 to S043 described above. Next, in step S052, if the determination result in step S044 described above is panning, the process proceeds to step S053. If not, the process proceeds to step S054.

  Next, in step S053, the post-shooting image processing unit 25 reads the subject position, the subject shape, and the subject moving direction stored in step S047 from the storage unit, and based on the read information, Image processing is performed on an image area other than the subject, blurring is generated in the panning direction of the imaging apparatus 100 or the moving direction of the target subject, and thereafter, the image is synthesized with the subject to be shot. This blur is generated by, for example, a moving average. Image processing for generating such blurring is known as blurring (moving) filter processing in image processing software, for example.

  Next, in step S054, the white balance of the image captured based on the AWB set in step S042 is adjusted and recorded in the memory 17, and then the processing from step S021 is repeated.

  According to the above-described imaging device 100 according to the present embodiment, in the case of panning, the subject is shot with a high-speed shutter, and after this shooting, based on information for specifying the subject to be shot, the subject of the panning shot Image processing is performed on an image other than the image area of the subject. As a result, the panning target subject has not been properly panned, or even if the panning subject subject has not been panned, the panning target subject has been successfully panned. A panning shot similar to the case can be simplified.

  As shown in FIG. 7, the post-shooting image processing unit 25 may generate blur in both the same direction and the opposite direction with respect to the panning direction or the moving direction of the target subject. However, the blur may be generated only in the panning direction or the direction opposite to the moving direction of the target subject (see FIG. 7B). Further, the magnitude of the blur in the movement direction may be determined based on the movement amount of panning or the movement amount of the target subject.

  Note that the angular velocity sensor 2 and the panning detection unit 26 described above can be used for a camera shake prevention mechanism. Therefore, when the imaging apparatus having the camera shake prevention mechanism is used as the imaging apparatus according to the present embodiment, the internal configuration of the control unit 20 is changed, but there is no addition of a new configuration. The increase in the cost of the configuration to be performed can be suppressed. In addition, since a newly added configuration is unnecessary, the size of the entire imaging apparatus is not increased.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within a scope not departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Imaging part, 21 ... Image processing part, 23 ... 1st detection part, 24 ... 2nd detection part, 25 ... Image processing part after imaging | photography, 251 ... 1st image processing part, 252 ... 2nd image processing part , 26 ... Panning detection unit, 27 ... Imaging condition setting unit, 28 ... Shooting control unit, 100 ... Imaging device

Claims (5)

An imaging unit that images a subject and generates an image signal;
A first detector for detecting a moving body based on the image signal;
A second detection unit for detecting information identifying the moving body detected by the first detection unit in the image to be captured;
An image processing unit that performs image processing on an image other than the image area of the moving body among the captured images based on information that identifies the moving body detected by the second detection unit after shooting;
An imaging apparatus comprising: The imaging apparatus according to claim 1,
The image processing unit
A first image processing unit that performs a movement blurring process on an image other than the moving body based on information identifying the moving body detected by the second detection unit;
A second image processing unit that synthesizes the image blurred by the first image processing unit and the image of the moving body;
An imaging apparatus comprising: The imaging apparatus according to claim 2,
The information for specifying the moving body includes information indicating the moving direction of the moving body,
The first image processing unit
Based on the information indicating the moving direction of the moving body included in the information for identifying the moving body, the blurring process is performed by determining a blur direction.
An imaging apparatus characterized by that. The imaging apparatus according to any one of claims 1 to 3,
The first detection unit includes:
Detecting a motion vector of the subject based on an image signal output from the imaging unit, and detecting the moving body based on the detected motion vector;
An imaging apparatus characterized by that. The imaging apparatus according to claim 4,
A panning detector for detecting device panning,
With
The first detection unit includes:
Based on the panning detected by the panning detection unit and the detected motion vector, the moving body is detected from an image based on an image signal output from the imaging unit,
An imaging apparatus characterized by that.

Images