JP2007067555A - Imaging apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and an imaging method in which camera control such as the setting of a focus, zooming, or the like, can be carried out certainly for an object to be photographed intended by a user. <P>SOLUTION: A control section 15 performs the frame indication of a moving object based on calculated frame information, performs highlighting for an object which is the current candidate of a target, waits an input from a user by bringing the highlighting of a frame into such a state (selectable state) as the target can be altered and determined at a user interface 17, notifies a camera control section for which object the user desires camera control by the input of determination from the user, and then the camera control section controls a camera control object 18 on a post stage. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a digital camera and an imaging method.

  In an imaging apparatus, when a moving subject is photographed in the past, as disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2003-241065), movement within a specific image region is performed using motion detection. Autofocus technology that automatically finds an object and focuses on the subject is adopted.

Here, lens control in a general camera using moving object detection will be described.
FIG. 1 is a flowchart for explaining lens control in a general camera using moving object detection.

As shown in FIG. 1, a moving object is detected in a specific area (often a central part is used) of the captured image area (ST1), and the camera automatically determines that the most likely object is the object. (ST2).
Then, the lens control is performed (ST3). If the set subject is within the detection range, the lens control is continued (ST4). If the subject cannot be detected, the moving object setting is deleted and the process is repeated from step ST1. A search is performed (ST5).
JP 2003-241065 A

However, in the control technique described above, since the camera automatically determines as seen in the process of step ST2, there is a possibility that an object not intended by the user may be determined as a subject, for example, during autofocus, The possibility of focusing on an object that is not the object intended by the user, such as misrecognition of a so-called crossing object, could not be excluded.
Further, for example, even in a zoom operation, it is often difficult to perform an appropriate zoom operation on a moving object.

A specific example of erroneous recognition in lens control using moving object detection will be described with reference to FIGS.
FIG. 2 shows a subject given as an example. FIG. 3 shows a state after focus setting.
In FIG. 2, moving objects 2 and 3 are included in the captured image 1. Even if the user wants to focus on the moving object 2, if the camera automatically focuses on the moving object 3 near the center, the captured image 1 is essentially the same as after the focus set in FIG. 3. The moving object 2 to be focused is out of focus and the moving object 3 is in focus.
This is far from the user's intention.

  An object of the present invention is to provide an imaging apparatus and an imaging method capable of reliably performing camera control such as focus setting and zooming processing on a subject intended by a user.

  An imaging apparatus according to a first aspect of the present invention includes an optical system, an imaging element that receives an optical image from the optical system and obtains a captured image, and detects movement of one or more objects in the captured image by the imaging element. And a control unit capable of calculating frame information surrounding a moving object based on movement information detected by the detection unit, and a captured image captured by at least the image sensor under the control of the control unit And an image display unit capable of displaying a frame based on the frame information.

  Preferably, in order to perform frame display for the purpose of emphasizing the moving object on the image display unit, the control unit displays another image superimposed on the captured image.

  Preferably, it has a user interface capable of selecting any displayed frame.

  Preferably, the user interface is capable of changing and determining the highlighted target.

  Preferably, the control unit displays a frame of a moving object on the image display unit based on the frame information, highlights a subject that is currently a target candidate, and displays the object from the user interface. The decision input recognizes the subject to be controlled, and instructs the control subject to control the subject.

  Preferably, the control unit performs feature extraction of the movement tendency of the subject.

  An imaging apparatus according to a second aspect of the present invention includes an optical system including a focus lens, an imaging element that receives an optical image by the optical system and obtains a captured image, and one or a plurality of objects in an image captured by the imaging element A detection unit that detects the movement of the camera, a control unit that can calculate frame information that surrounds the moving object based on the motion information detected by the detection unit, and imaging by at least the imaging device under the control of the control unit An image display unit capable of displaying a frame based on the captured image and the frame information, a user interface capable of selecting any displayed frame, and capable of changing and determining a target for highlighting; A focus lens control unit that performs drive control of the focus lens in accordance with an instruction from the control unit, and the control unit includes the frame information in the image display unit. Then, the frame of the moving object is displayed, the object that is currently the target candidate is highlighted, the object to be controlled is recognized by the decision input from the user interface, and the focus lens control is performed. To instruct the subject to perform drive control of the focus lens.

  An image pickup apparatus according to a third aspect of the present invention includes an optical system including a zoom lens, an image pickup device that receives an optical image by the optical system and obtains a picked-up image, and one or a plurality of objects in the image picked up by the image pickup device A detection unit that detects the movement of the camera, a control unit that can calculate frame information that surrounds the moving object based on the motion information detected by the detection unit, and imaging by at least the imaging device under the control of the control unit An image display unit capable of displaying a frame based on the captured image and the frame information, a user interface capable of selecting any displayed frame, and capable of changing and determining a target for highlighting; A zoom lens control unit that controls driving of the zoom lens in accordance with an instruction from the control unit, and the control unit moves to the image display unit based on the frame information The body frame is displayed, the subject that is currently the target candidate is highlighted, the subject to be controlled is recognized by the decision input from the user interface, and the zoom lens control unit Instructs the subject to perform zoom lens drive control.

  An imaging method according to a fourth aspect of the present invention is a first step of obtaining a captured image by an imaging device based on an optical image by an optical system, and a second step of detecting a motion of one or a plurality of objects in the captured image. And a third step of calculating frame information surrounding the moving object based on the motion information detected in the second step, and a first step of displaying on the image display unit at least the captured image and the frame based on the frame information. 4 steps.

  Preferably, in the fourth step, a frame of a moving object is displayed on the image display unit based on the frame information, a subject that is currently a target candidate is highlighted, and a user interface is displayed. It further has a fifth step of recognizing the subject to be controlled by the decision input and instructing the control subject to control the subject.

  According to the present invention, it is possible to reliably perform camera control such as focus setting and zooming processing on a subject intended by a user.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 4 is a block diagram illustrating a configuration example of the imaging apparatus according to the embodiment of the present invention.

  As shown in FIG. 4, the imaging apparatus 10 includes a lens system (optical system) 11, an imaging element 12, a memory 13, a motion information detection unit 14, a control unit 15, an image display unit 16, a user interface 17, and camera control. Has a subject 18.

  The lens system 11 includes, for example, a focus lens and a zoom lens, and condenses light (image) from the imaging target on the imaging element 12.

The imaging device 12 is configured using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) sensor, or the like, and forms (receives) light of a subject incident through, for example, a shutter (not shown).
The image sensor 12 converts the amount of received light reaching each light receiving element on the imaging surface into an electrical signal by photoelectric conversion. The image sensor 12 is, for example, a single-plate sensor, and an electrical signal to be output is any one color signal (data) of R signal, G signal, and B signal per pixel.
In the image sensor 12, the image signal converted into an electrical signal is input to a sampling circuit (not shown).
The sampling circuit removes the noise component of the image signal (electric signal) supplied from the image sensor 12 by, for example, correlated double sampling. The image signal by the sampling circuit is sampled and quantized by an analog / digital (A / D) conversion circuit.

The memory 13 stores a digital image signal from the image sensor 12.
The image signal stored in the memory 13 is accessed by the motion information detection unit 14, the control unit 15, and the image display unit 16.

The motion information detection unit 14 calculates the motion information of one or more objects in the captured image stored in the memory 13 and supplies the calculated information to the control unit 15.
Various methods can be employed as the motion detection processing of the motion information detection unit 14. For example, motion information is generated in the captured image area by motion detection processing as disclosed in Japanese Patent No. 3263794. I do.

The control unit 15 calculates frame information for the moving object based on the movement information of the moving object calculated by the movement information detection unit 14.
The control unit 15 displays one or more frames based on the image data stored in the memory 13 and the calculated frame information for the moving object on the image display unit 16 so as to be easily seen by the user. The control unit 15 performs display control such that another image is superimposed on the captured image in order to display a frame on the image display unit 16 for the purpose of emphasizing the moving object.
As another example, a configuration in which the motion information detection unit 14 is not provided and motion information is calculated by the control unit 15 and frame information is also calculated can be employed. Conversely, the motion information detection unit 14 may be configured to perform calculation of the frame information.

  The image display unit 16 is formed by, for example, a liquid crystal display (LCD), an organic EL display, or the like, and displays a captured image, frame information, and the like under control of the control unit 15.

The user interface 17 is configured so that the user can select a part or the whole of an arbitrary screen of the image display unit 16.
The user selects a desired camera control target using the user interface 17 while viewing the display image on the image display unit 16.

In the present embodiment, the control unit 15 displays a frame of a moving object based on the calculated frame information, highlights a subject that is currently a target candidate, and displays the above-described frame highlight display on the user interface. In step 17, the target can be changed and determined (selectable state), the input from the user is waited, and the camera control unit determines which subject the user desires to control the camera by the input from the user. It is possible to give an instruction to the subsequent camera control target 18 from the camera control unit.
Here, camera control refers to, for example, focus setting or zooming processing. The control unit 15 controls the subsequent camera control target 18 (for example, a focus lens or a zoom lens) with respect to the selection target using the subject selection information of the user.

  In addition, the control unit 15 may change the subject when the moving object selected by the user via the user interface 17 moves or is likely to move outside the shooting range, or when the moving object moves behind another object. The feature extraction of the movement tendency is performed.

Further, when the camera control target is focus lens control, the control unit 15 predicts and sets the focus area in the next frame based on movement information of the locked focus target in the current frame.
In addition, when the camera control target is zoom lens control, the control unit 15 performs adjustment so that the locked zoom target is displayed at an appropriate zoom magnification on the screen.

Next, a specific operation example of the imaging apparatus of FIG. 4 will be described with reference to FIGS.
FIG. 5 is a flowchart for explaining the main operation of the imaging apparatus of FIG.
FIG. 6 is a diagram illustrating an example of frame setting.
FIG. 7 is a diagram illustrating a screen display example in the image display unit.
FIG. 8 is a diagram illustrating an example of subject change and selection.

In the imaging apparatus 10, first, image light is collected by the lens system 11, converted into an image by the imaging device 12, and stored in the memory 13.
Using the data stored in the memory 13, motion information in the captured image data is calculated in the motion information detection unit 14, and frame information is calculated for the moving object in the control unit 15 based on the information.
Under the control of the control unit 15, a single or a plurality of frames based on the image data as the imaging result and the calculated frame information for the moving object are displayed on the image display unit 16 so as to be easily seen by the user.
The user selects a desired camera control target 18 using the user interface 17 while viewing the image displayed on the image display unit 16.
Here, the camera control refers to, for example, a focus setting or zooming process as described above. The control unit 15 controls the subsequent camera control target 18 (for example, a focus lens or a zoom lens) with respect to the selection target using the subject selection information of the user.

  Next, the operation from the motion information generation process will be described in more detail in association with the flowchart of FIG.

First, the motion information generation unit 14 generates motion information in the captured image region by a predetermined motion detection process (ST11).
Next, the control unit 15 calculates frame information for one or more moving objects using the motion information described above (ST12).
When deciding a frame, a group of blocks having a large movement or a group of blocks having similar movement directions is grouped, and the frame is set so as to surround the area.
The frame information refers to the position and size of the frame.

FIG. 6 shows an example of setting the frame.
In FIG. 6, reference numeral 20 denotes a captured image divided into blocks. The captured image 20 includes block groups 21 and 22 that satisfy the grouping conditions as described above.
Frames 23 and 24 are set so as to surround the block groups 21 and 22.
In this example, block division is performed. However, it is not always necessary to do so, and it is only necessary to specify the area of the moving object using another method. In this example, a rectangular frame is used, but the shape of the frame is not limited.

Next, if there is a feature in the subject inside the frame 23 or 24, the feature extraction for user assistance is performed (ST13).
For example, for a subject that is located at the very end of the shooting range and is moving in the direction outside the shooting range, it is possible to extract features such as the possibility of going out of the shooting range.
After that, the captured image and the frame calculated in step ST12 are displayed on the image display unit 16 so as to overlap with the captured image.
At that time, subject characteristics for user assistance may be displayed in an appropriate form as necessary in step ST13 to improve usability (ST14).

An example of a screen to which the above is applied is shown in FIG.
In the example of FIG. 7, among the frames 23 and 24 for the moving object, the frame 24 is highlighted because it is the current target candidate, and the frame 23 is not a target candidate at present. At present, those that are not target candidates are displayed as normal frames.
Information indicated by reference numeral 25 is user assistance information that informs that the moving object surrounded by the frame 23 goes out of the shooting range.
Note that the target candidate here refers to a subject that can be determined by the user using the user interface 17 as information for subsequent camera control. Others cannot be determined as information for camera control at that time.

If the user wants to change the target candidate from the highlighted subject to another subject because it is currently a target candidate, the user can select the desired subject using the user interface 17 such as a jog dial or touch panel. (ST15).
The newly selected target candidate is highlighted, and the previously highlighted target is no longer a target candidate, and a normal frame is displayed (ST16).

  FIG. 8 shows a specific example using the jog dial 170 as the user interface 17.

As shown by an arrow 171 in FIG. 8, the user rotates the jog dial 170 in a predetermined direction (for example, counterclockwise) to change the subject 241 in the frame 24 that has been the target candidate to the subject 231 in the frame 23. The target candidate is changed. As a result, the subject 231 in the frame 23 becomes a new target candidate subject, and highlighting is performed, and the frame 24 becomes a normal frame display.
Here, when the subject is determined by the user (ST17), the position information of the current target candidate subject on the screen is notified to the control unit 15 as information for camera control, and the target Appropriate camera control according to the candidate is performed (ST18).
If no subject is determined, the system will wait for user input. For example, in the example of FIG. 8, when the user performs an operation (172) of pressing the jog dial 170, the subject 231 in the change destination frame 23 becomes the official control target.
Thus, for example, in the case of a follow-up focus system, follow-up focus control is performed by the camera control target 18 on the subject 231 in the frame 23.

  As described above, in this embodiment, it is possible to easily notify the camera of the moving object expected by the user, and the camera controls the moving object (subject) expected by the user even on the camera side. Can be performed.

By providing the user interface according to the present embodiment, for example, the user does not set the camera to match the camera's automatic subject detection algorithm in step ST2 of the flow of FIG. 1, but is shown with an easy user interface. When the user himself / herself selects a target from among moving object candidates, the user can reliably select a moving object to be photographed. Further, camera control can be performed on the subject that the user expects on the camera side.
For example, this makes it possible to avoid erroneous recognition of a moving object and to perform autofocus on an appropriate subject.
In addition, the camera can automatically perform zoom operations on difficult moving objects by calculating the appropriate zoom magnification based on the amount and direction of movement of the subject selected as the target and performing the zoom operation on the camera side. Etc. are also possible.

  In the above description, the camera control target 18 has been generalized and described. However, a case where the camera control target is an autofocus mechanism and an auto zoom mechanism will be described below.

FIG. 9 is a diagram illustrating a configuration example when the imaging apparatus of FIG. 4 is applied to an autofocus system.
The imaging apparatus 10A in FIG. 9 is different from the imaging apparatus 10 in FIG. 4 in that the camera control target 10 is the focus lens control unit 18A. Since the other configurations are the same, the same components in FIG. 9 as those in FIG. 4 are denoted by the same reference numerals.

  Hereinafter, the operation of the imaging apparatus 10A in FIG. 9 will be described with reference to the flowchart in FIG. FIG. 10 is a diagram for explaining the main operation of the imaging apparatus of FIG.

In the imaging apparatus 10 </ b> A of FIG. 9, first, image light is collected by the lens system 11, converted into an image by the imaging device 12, and stored in the memory 13.
Using the data stored in the memory 13, motion information (for example, motion vectors) in the captured image data is calculated by the motion information detection unit 14, and frame information for moving objects is calculated by the control unit 15 based on the information. Is calculated.
Under the control of the control unit 15, a single or a plurality of frames based on the image data as the imaging result and the calculated frame information for the moving object are displayed on the image display unit 16 so as to be easily seen by the user.
The user selects a desired focus target using the user interface 17 while viewing the image.
The control unit 15 controls the focus control unit 18A for the selection target using the user's subject selection information.

  Next, the operation from the motion information generation process will be described in more detail in association with the flowchart of FIG. FIG. 10 shows another processing example of the processing of FIG.

Moving object detection is performed in the photographed image area. If there is a moving object, registration is performed, and if there is no moving object, the moving object is searched again (ST21 to ST24).
When a desired moving object is set (ST24), the moving object is highlighted (ST25), and when it is not set, the process returns to step ST21 and redetection is performed.
In addition, when there is a thing which is moving largely in the central part during the process of step ST25, a judgment material for the user may be provided such as further highlighting.
The user selects a desired subject using the user interface 17 such as a touch panel or a jog dial (ST26).
In the imaging apparatus 10A, the focus lens is driven by calculating an appropriate lens setting value with the focus lens control unit 18A for the selected subject (ST27) (ST28).
If the set subject is within the detection range, the lens control is continued (ST29). If the subject cannot be detected, a warning is displayed (ST30) and the moving body setting is deleted. Then, the search is performed again from step ST21 (ST31).

By performing the focus lens control as described above, it is possible to avoid erroneous recognition of a moving object.
This makes it possible to perform autofocus on an appropriate subject.

  Next, a case where the camera control target is an auto zoom mechanism will be described.

FIG. 11 is a diagram illustrating a configuration example when the imaging apparatus of FIG. 4 is applied to an auto zoom system.
The imaging apparatus 10B in FIG. 11 differs from the imaging apparatus 10 in FIG. 4 in that the camera control target 10 is the zoom lens control unit 18B. 11 are the same as those in FIG. 4 and are denoted by the same reference numerals.

  Hereinafter, the operation of the imaging apparatus 10B of FIG. 11 will be described in association with the flowchart of FIG. FIG. 12 is a diagram for explaining the main operation of the imaging apparatus of FIG.

In the imaging apparatus 10 </ b> B of FIG. 11, first, image light is collected by the lens system 11, converted into an image by the imaging device 12, and stored in the memory 13.
Using the data stored in the memory 13, motion information (for example, motion vectors) in the captured image data is calculated by the motion information detection unit 14, and frame information for moving objects is calculated by the control unit 15 based on the information. Is calculated.
Under the control of the control unit 15, a single or a plurality of frames based on the image data as the imaging result and the calculated frame information for the moving object are displayed on the image display unit 16 so as to be easily seen by the user.
The user selects a desired zoom target using the user interface 17 while viewing the image.
The control unit 15 controls the zoom lens control unit 18B for the selection target using the user's subject selection information.

  Next, the operation from the motion information generation process will be described in more detail in association with the flowchart of FIG. FIG. 12 shows another processing example of the processing of FIG.

Moving object detection is performed in the photographed image area. If there is a moving object, registration is performed. If there is no moving object, the moving object is searched again (ST41 to ST44).
If a desired moving object is set (ST44), the moving object is highlighted (ST45), and if not set, the process returns to step ST41 and redetection is performed.
In addition, when there is a thing which is moving greatly in the central portion during the process of step ST45, a judgment material to the user may be provided such as further highlighting.
The user selects a desired subject using the user interface 17 such as a touch panel or a jog dial (ST46).
In the imaging apparatus 10B, the zoom lens is driven by calculating an appropriate lens setting value with the zoom lens control unit 18B for the selected subject (ST47) (ST48). At this time, the user may set how much the auto zooming is performed with respect to the display screen.
If the set subject is within the detection range, the lens control is continued (ST49). If the subject cannot be detected, a warning is displayed (ST50) and the moving body setting is deleted. The search is performed again from step ST41 (ST51).

  By performing the name zoom lens control as described above, erroneous recognition of a moving object can be avoided. This makes it possible to perform auto zoom on an appropriate subject.

It is a flowchart for demonstrating the lens control in the common camera using a moving object detection. It is a figure for demonstrating the specific example of misrecognition, Comprising: It is a figure which shows an example of a to-be-photographed object. It is a figure for demonstrating the specific example of misrecognition, Comprising: It is a figure which shows the state after a focus set. It is a block diagram which shows the structural example of the imaging device which concerns on embodiment of this invention. 6 is a flowchart for explaining main operations of the imaging apparatus in FIG. 4. It is a figure which shows an example of the setting of a frame. It is a figure which shows the example of a screen display in an image display part. It is a figure which shows an example of a change and selection of a to-be-photographed object. It is a figure which shows the structural example at the time of applying the imaging device of FIG. 4 to an autofocus system. It is a figure for demonstrating the main operation | movement of the imaging device of FIG. It is a figure which shows the structural example at the time of applying the imaging device of FIG. 4 to an auto zoom system. It is a figure for demonstrating the main operation | movement of the imaging device of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10, 10A, 10B ... Imaging device, 11 ... Lens system, 12 ... Imaging element, 13 ... Memory, 14 ... Motion information detection part, 15 ... Control part, 16 ... Image display unit, 17 ... user interface, 18 ... camera control target, 18A ... focus lens control unit, 18B ... zoom lens control unit.

Claims (10)

Optical system,
An image sensor that receives an optical image by the optical system and obtains a captured image;
A detection unit for detecting movement of one or more objects in an image captured by the image sensor;
A control unit capable of calculating frame information surrounding a moving object based on movement information detected by the detection unit;
An imaging apparatus comprising: an image display unit capable of displaying at least a captured image captured by the imaging element and a frame based on the frame information under the control of the control unit. The imaging apparatus according to claim 1, wherein the control unit displays a frame with another image superimposed on the captured image in order to display a frame for the purpose of emphasizing a moving object on the image display unit. The imaging apparatus according to claim 2, further comprising a user interface capable of selecting an arbitrary displayed frame. The imaging apparatus according to claim 3, wherein the user interface is capable of changing and determining a highlighted target. The control unit displays a frame of a moving object on the image display unit based on the frame information, highlights a subject that is currently a target candidate, and by a determination input from the user interface, The imaging apparatus according to claim 4, recognizing a subject to be controlled and instructing the control target to control the subject. The imaging apparatus according to claim 5, wherein the control unit performs feature extraction of a movement tendency of a subject. An optical system including a focus lens;
An image sensor that receives an optical image by the optical system and obtains a captured image;
A detection unit for detecting movement of one or more objects in an image captured by the image sensor;
A control unit capable of calculating frame information surrounding a moving object based on movement information detected by the detection unit;
An image display unit capable of displaying a frame based on at least a captured image captured by the imaging element and the frame information under the control of the control unit;
A user interface that allows you to select any displayed border, change the highlighting target, and determine
A focus lens control unit that performs drive control of the focus lens in accordance with an instruction from the control unit,
The control unit displays a frame of a moving object on the image display unit based on the frame information, highlights a subject that is currently a target candidate, and by a determination input from the user interface, An imaging apparatus that recognizes a subject to be controlled and instructs the focus lens control unit to perform focus lens drive control on the subject. An optical system including a zoom lens;
An image sensor that receives an optical image by the optical system and obtains a captured image;
A detection unit for detecting movement of one or more objects in an image captured by the image sensor;
A control unit capable of calculating frame information surrounding a moving object based on movement information detected by the detection unit;
An image display unit capable of displaying a frame based on at least a captured image captured by the imaging element and the frame information under the control of the control unit;
A user interface that allows you to select any displayed border, change the highlighting target, and determine
A zoom lens control unit that performs drive control of the zoom lens according to an instruction of the control unit,
The control unit displays a frame of a moving object on the image display unit based on the frame information, highlights a subject that is currently a target candidate, and by a determination input from the user interface, An imaging apparatus that recognizes a subject to be controlled and instructs the zoom lens control unit to perform zoom lens drive control on the subject. A first step of obtaining a captured image by an image sensor based on an optical image by an optical system;
A second step of detecting movement of one or more objects in the captured image;
A third step of calculating frame information surrounding the moving object based on the movement information detected in the second step;
A fourth step of displaying on the image display unit at least a frame based on the captured image and the frame information;
An imaging method comprising: In the fourth step, a frame display of the moving object is performed on the image display unit based on the frame information, and a highlighted target object is highlighted.
The imaging method according to claim 9, further comprising a fifth step of recognizing a subject to be controlled by a determination input from a user interface and instructing the control target to perform control on the subject.

Images