JP2007074279A - Imaging apparatus and control method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and a control method thereof with which manual photographing can simply and smoothly be switched into tracking photographing. <P>SOLUTION: The imaging apparatus detects tracking candidate objects from imaged image data and selects an optional tracking candidate object from the detected tracking candidate objects (steps S406, S407). Then the imaging apparatus selects the selected tracking candidate object as a tracking target object (step S408), and shifts to a first mode wherein the tracking target object is subjected to tracking photographing (step S409). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of performing tracking shooting of an arbitrary object in captured image data, and a control method thereof.

  Automatic tracking technology for imaging devices is being researched and developed for the purpose of reducing costs by automating shooting and reducing the burden on photographers. For example, Patent Document 1 discloses a method of performing automatic tracking by detecting a face area of a subject. The tracking technology itself has been researched and developed for a long time. For example, various methods such as a method based on color, contour, and motion information disclosed in Patent Document 2 and a method based on correlation calculation disclosed in Patent Document 3 have been proposed.

JP-A-5-219421 Japanese Patent Laid-Open No. 4-326677 JP-A-5-075909

  However, many methods have been proposed for the automatic tracking technology of an imaging apparatus for broadcasting, but there are limitations in actual use, and the tracking performance is not always satisfactory. Therefore, it is desirable to be able to switch between manual shooting and tracking shooting by a cameraman.

  Therefore, an object of the present invention is to enable switching to tracking photography smoothly and easily.

According to a first aspect of the imaging apparatus of the present invention, an imaging unit that images a subject, a detection unit that detects a tracking candidate object from image data output from the imaging unit, and a tracking candidate object detected by the detection unit Selection means for selecting an arbitrary tracking candidate object from the above, setting means for setting the tracking candidate object selected by the selection means as a tracking target object, and first tracking imaging of the tracking target object set by the setting means And control means for making a transition to the mode.
According to a second aspect of the imaging apparatus of the present invention, an imaging unit that images a subject, a detection unit that detects a tracking candidate object from image data output from the imaging unit, and a tracking candidate object detected by the detection unit And setting means for setting a tracking candidate object satisfying a predetermined condition as a tracking target object, and control means for making a transition to a first mode for tracking shooting of the tracking target object set by the setting means. And
A first aspect of an imaging device control method according to the present invention is an imaging device control method having an imaging means for imaging a subject, and a detection step of detecting a tracking candidate object from image data output from the imaging means. A selection step of selecting an arbitrary tracking candidate object from the tracking candidate objects detected by the detection step, a setting step of setting the tracking candidate object selected by the selection step as a tracking target object, and the setting step. And a control step of transitioning to a first mode in which the set tracking target object is tracked and photographed.
A second aspect of the imaging device control method of the present invention is an imaging device control method having an imaging means for imaging a subject, and a detection step of detecting a tracking candidate object from image data output from the imaging means. A setting step for setting a tracking candidate object satisfying a predetermined condition among the tracking candidate objects detected by the detection step as a tracking target object; and a first tracking image of the tracking target object set by the setting step. And a control step for transitioning to the mode.
The program according to the present invention causes a computer to execute the first or second aspect of the method for controlling the imaging apparatus.
The computer-readable recording medium of the present invention is characterized in that the program is recorded.

  According to the present invention, for example, by selecting an arbitrary tracking target object from the tracking candidate objects, a transition to the tracking shooting mode of the selected tracking target object is performed. Therefore, it is possible to switch to tracking shooting smoothly and easily.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

-First embodiment-
First, a first embodiment of the present invention will be described. In the present embodiment, a manual shooting mode and a tracking shooting mode are provided in an imaging apparatus having an automatic tracking function. In the manual shooting mode, a desired tracking candidate is selected and confirmed using a button from a plurality of tracking candidate objects displayed on the finder screen, and tracking of the confirmed tracking target object is started. The end of tracking is performed by applying a load to the imaging apparatus, that is, by forcibly applying a force against the movement of the imaging apparatus from the outside.

  A system configuration of the imaging apparatus according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating a system configuration of an imaging apparatus according to the present embodiment. The imaging apparatus has a configuration in which an optical system 101 and a pan / tilt pan head 120 are attached to the imaging apparatus main body 100. The imaging apparatus main body 100 includes an imaging system 102, an imaging control unit 103, a pan head control unit 104, a CPU 105, a ROM 106, a RAM 107, an image processing unit 108, a display 110, a display control unit 111, a button 112, and a button control unit 113. The imaging apparatus main body 100 includes an external storage medium 114 such as a memory card, an external storage control unit 115, and other input / output interfaces 116. The above-described configuration of the image pickup apparatus is an example, and a method of separately taking out the taken video as an analog video signal from the image pickup control system is also possible. Alternatively, the pan head control unit 104 may be connected to the input / output interface 116.

  Next, a flow from image data shooting to tracking processing of a tracking target object will be described. The captured image data is digitized by the imaging control unit 103 and sent to the RAM 107. Next, the image processing unit 108 refers to this, and performs detection of tracking candidate objects and image processing for tracking. The image processing can be replaced by the CPU 105, but since the processing amount is enormous, it is performed by the image processing unit 108 constituted by a dedicated image processing circuit. On the display 110, a finder screen including tracking candidate objects is displayed. Here, the viewfinder screen refers to a screen on which a real-time image taken by the imaging apparatus main body 100 is displayed. The button 112 is used for an operation for selecting an arbitrary tracking object from the tracking candidate objects displayed on the display 110 and confirming the selected tracking candidate object as a tracking target object. The pan / tilt movement target position obtained by the image processing for tracking the confirmed tracking target object is sent to the pan / tilt head control unit 104 to control the pan / tilt pan / tilt head 120 on which the imaging device is installed. .

  Next, the operation mode of the imaging apparatus according to the present embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a transition relationship between operation modes between the manual shooting mode and the tracking shooting mode.

  As an operation mode of the imaging apparatus according to the present embodiment, a manual shooting mode 201 and a tracking shooting mode 202 are provided. When the tracking target object is determined in the manual shooting mode 201, the tracking shooting mode 202 is entered. The method for determining the tracking target object is to designate the tracking candidate cursor displayed on the finder by moving the button 112, and a specific method will be described later. On the other hand, when the operator overloads the imaging apparatus during tracking of the tracking target object (in the tracking shooting mode), the tracking operation mode 202 is changed to the manual shooting mode 201. Similarly, when the tracking target object is lost, the tracking operation mode 202 is changed to the manual shooting mode 201.

  Next, a method for setting the tracking target object will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of a viewfinder screen 300 displayed on the display 110. Here, rectangular displays 301, 302, and 303 are made at the positions of the tracking candidate objects, respectively. Among these, the rectangular display 301 displayed at the position of the first tracking candidate object is highlighted compared to the rectangular displays 302 and 303 displayed at the positions of the other tracking candidate objects. This is a tracking candidate cursor. Note that the method of representing the tracking candidate object is not necessarily limited to the rectangular display, and it goes without saying that the present invention does not depend on the method of representing the tracking candidate object.

  Next, the user uses the button 112 to select a tracking candidate. Here, an example in which two buttons 112 are used will be described. One is a selection button. When this button is pressed, the tracking target cursor moves. In the example of FIG. 3, the state of moving from 301 to 302 is shown. When the button is pressed again, the tracking target cursor moves to the position 303, and when pressed again, the tracking target cursor returns to the first position 301 again. In this way, when the tracking target cursor is moved to the position of the desired tracking candidate object and another determination button is pressed, the tracking candidate object corresponding to that position is determined as the tracking candidate object. When the tracking target object is determined, the operation mode transitions from the manual shooting mode 201 to the tracking shooting mode 202. In the tracking shooting mode 202, the tracking candidate cursor is continuously displayed at the position of the tracking target object.

  Here, as a method of detecting a tracking candidate object or shooting a tracking target object, a known method may be used, and the present invention does not depend on a specific method. For example, the tracking candidate object can be detected by a method of extracting a moving region or a method of detecting a face region. Further, focus area information in autofocus processing used in the imaging control system may be used. Further, as a tracking shooting method of the tracking target object, a method based on various image features such as a color, a texture, and a template may be used, or a method of continuously recognizing a face image detected first can be used. is there.

  To end the tracking shooting mode, the tracking shooting mode is forcibly ended. This is equivalent to overloading the motor. In order to bring the current value closer to the control target value, the motor control circuit emits a pulse in the case of a stepping motor or controls torque in the case of a DC server motor. If the desired position is not reached as a result of the control, a pulse is repeatedly issued or a stronger torque is instructed, but if it still does not approach the target position, the control is stopped at a fixed position. This can be considered as overload detection. Such a technique is a known technique realized by controlling the robot. By realizing the overload detection as described above, it is possible to detect the stopping operation from the user and stop the tracking control.

  Next, the operation procedure of the imaging apparatus will be described with reference to FIGS. FIG. 4 is a flowchart showing main processes such as a mode transition management process and a viewfinder screen update process performed by the image processing unit 108, and is divided into processing procedures for the manual shooting mode and the tracking shooting mode. FIG. 5 is a flowchart showing a sub-process of image processing by the image processing unit 108. 4 and 5 show only the processes necessary for the description of the present embodiment.

  When the manual shooting mode is started in FIG. 4, the image processing unit 108 determines whether it is a candidate movement event for changing the tracking target object to be selected (step S <b> 401). This event occurs when the selection button 112 is pressed. If so, the image processing unit 108 confirms whether or not the tracking candidate object to be selected next actually exists in the finder screen (step S402). If it exists, the Y branch is made, and the image processing unit 108 moves the tracking candidate cursor to the position of the tracking candidate object to be selected next (step S403). After the process of step S403 is completed, or when N branches are made in step S402, the process returns to step S401 and waits for the next event.

  On the other hand, if it is determined in step S401 that the event is not a candidate movement event, the process branches to N, and the image processing unit 108 determines whether the event is a viewfinder screen update event (step S404). This event is issued when a tracking candidate object is newly detected or the position of the tracking candidate object is changed as a result of image processing. If so, the Y branch is made, and the image processing unit 108 updates the finder screen (step S405). On the other hand, if it is determined in step S404 that the event is not a display update event, the process branches to N, and the image processing unit 108 determines whether it is a target determination event for determining the tracking target object (step S406).

  The target confirmation event occurs when the confirmation button 112 is pressed. If so, the Y branch is made, and the image processing unit 108 determines whether or not the tracking candidate object on which the tracking candidate cursor is displayed exists in the finder screen (step S407). If it exists, the Y branch is made, and the image processing unit 108 determines the tracking candidate object at the position where the tracking candidate cursor is currently displayed as the tracking target object (step S408). In addition to this, for example, target setting processing such as storing the feature amount of the tracking target object is performed, and the tracking shooting mode 202 is shifted in step S409. If N branches are made in step S406 or step S407, the process returns to step S401 to wait for the next event.

  Here, the selection and confirmation operations using two buttons have been described. However, the present embodiment does not depend on the type of button, and buttons of other types can be used. For example, using a two-stage push button, half-press selection processing may be performed, full-press determination processing may be performed, and a shuttle switch with a push function may be used to move back and forth or left and right to the next candidate, previous candidate It is also possible to set the pressing as a confirmation process.

  Next, in the tracking shooting mode, the image processing unit 108 determines whether it is a target amount update event (step S420). This is an event emitted from the image processing process, and notifies the update of the target amount of pan / tilt control. If it is the event, the process branches to Y and the image processing unit 108 updates the target amount (step S421). The target amount is referred to from a camera control process (not shown) and used for motor control of the pan / tilt head 120. If it is determined in step S420 that the event is not the target amount update event, the process branches to N, and the image processing unit 108 determines whether the event is a target loss event (step S422). This is also an event from the image processing process. If so, the Y branch is made, and the image processing unit 108 ends the tracking shooting mode 202 and shifts the operation mode to the manual shooting mode 201.

  If it is determined in step S422 that the event is not a target loss event, the process branches to step S423 after branching N, and the image processing unit 108 determines whether the event is a stop event (step S423). This event is issued from a motor control process (not shown) when the imaging apparatus is overloaded. If so, the Y branch is made, and the image processing unit 108 shifts the operation mode from the tracking shooting mode 202 to the manual shooting mode 201. If not, the process branches to N and the image processing unit 108 determines whether it is a display update event (step S425). If so, the display image of the finder is updated (step S426). Here, the tracking candidate object is detected, and the finder screen is updated. Thereafter, the process returns to step S420 to wait for the next event.

  Next, an image processing sub-process will be described with reference to FIG. The image processing unit 108 waits for an image input after the process starts (step S501). When an image is input, the Y branch is performed, and the image processing unit 108 determines whether or not the tracking shooting mode is set (step S502). If so, the Y branch is performed, and the image processing unit 108 detects the tracking target object. Processing is performed (step S503). Next, if the tracking target object is successfully detected, the Y branch is made, and the image processing unit 108 calculates a camera control target value corresponding to the tracking target object (step S505). Next, after issuing a viewfinder screen update event (step S506), the image processing unit 108 returns to step S501 and waits for the next image input event.

  On the other hand, if the detection of the tracking target object fails in step S504, the process branches to step S507, and the image processing unit 108 determines whether the number of tracking target object detection failures is equal to or greater than a predetermined value. (Step S507). If it fails continuously for a predetermined value or more, the Y branch is made, and the image processing unit 108 issues a target loss event (step S508). If it is determined in step S507 that the number of failures is less than the predetermined value, the process branches to N, and the image processing unit 108 issues a viewfinder screen update event, and then returns to step S501 to display the next image input event. wait. If it is determined in step S502 that the mode is not the tracking shooting mode 202, the current operation mode is the normal manual shooting mode 201. Therefore, N branching is performed, a tracking candidate object is detected, and a finder screen update event ( Proceed to step S506).

  As described above, according to the present embodiment, in the imaging apparatus having the automatic tracking shooting function, the manual shooting mode and the tracking shooting mode are provided, and the tracking target object is selected using the button when shifting from the manual shooting mode to the tracking shooting mode. Can be selected / confirmed. In this way, it is possible to easily start tracking shooting. Further, the tracking photographing operation can be terminated by overloading the motor of the pan / tilt pan head 120 during the tracking photographing. Therefore, it is possible to smoothly and easily switch from the means shooting mode to the tracking shooting mode.

-Second Embodiment-
Next, a second embodiment of the present invention will be described. The configuration of the imaging apparatus according to the present embodiment is the same as the configuration shown in FIG. In the first embodiment described above, the tracking candidate cursor is displayed at the position of the tracking target object in the tracking shooting mode to notify the user of the tracking target object. On the other hand, in the second embodiment of the present invention, not only the tracking target object but also other objects in the finder screen are notified of which is a tracking candidate object. Even in the tracking shooting mode, it is possible to select and confirm a tracking candidate object by a button operation and switch the tracking target object.

  FIG. 6 is a diagram showing a display example of the finder screen 600 according to the second embodiment of the present invention. As shown in FIG. 6, rectangular displays 601 to 603 are made for each object. The tracking target object is displayed with a rectangular display 601 that is emphasized as compared with other objects. A tracking candidate cursor for selecting a tracking target object is also displayed during tracking in a display mode different from other rectangular displays. In the example of FIG. 6, the tracking cursor is displayed at the position of the rectangular display 602. By performing the determination operation by operating the button 112, the tracking target object can be changed to the object at the position where the tracking candidate cursor is displayed at that time. In the example of FIG. 6, the tracking target object is changed to an object at a position from 601 to 602.

  Regarding the operation procedure of the process, the candidate movement event and the target determination event occur in the tracking shooting mode (steps S420 to S426) in FIG. When these events occur in the tracking shooting mode, the same processing as steps S401 to S403 in FIG. 4 is performed when a candidate movement event occurs. That is, when there is a tracking candidate object to be moved, the tracking candidate cursor is moved to the tracking candidate object. On the other hand, if there is no tracking candidate object to be moved, the process proceeds to processing for another event. When the target confirmation event occurs, processing similar to that in steps S406 to S408 in FIG. 4 is performed. That is, when the tracking candidate object on which the tracking candidate cursor is displayed exists on the finder screen, the tracking candidate object is determined as the tracking target object. On the other hand, when the tracking candidate object on which the tracking candidate cursor is displayed when the target confirmed event occurs does not exist on the finder screen, the process proceeds to processing for other events.

  In the present embodiment, the method for stopping tracking (the method for transitioning from the tracking shooting mode to the manual shooting mode) is the same as in the first embodiment. In the present embodiment, the tracking target object can be changed by a button operation even in the tracking shooting mode by the method described above.

-Third embodiment-
Next, a third embodiment of the present invention will be described. Since the configuration of the imaging apparatus according to the present embodiment is the same as the configuration illustrated in FIG. 1, the description thereof is omitted.

  When the camera is pointed at a moving object, tracking shooting can be automatically started. At this time, in the manual shooting mode, the tracking candidate object detection process is performed by the above-described method, and a rectangular display representing the tracking candidate object is displayed on the finder screen. In the present embodiment, the tracking target object is not selected and confirmed by a button operation as in the first and second embodiments. Next, when the tracking candidate object is located in a specific area such as the central portion of the finder screen, the tracking candidate object is automatically set as the tracking target object, and the tracking target object transitions to the tracking shooting mode.

  When the tracking shooting mode is started, rectangular display is not performed for other tracking candidate objects as in the second embodiment, but rectangular display is performed only at the position of the current tracking target object. In this embodiment, the button 112 is not used to specify the tracking target object. Accordingly, as a method for stopping the tracking shooting, the tracking shooting mode can be ended by operating the button 112 in addition to the method of manually forcibly stopping the tracking shooting as in the first and second embodiments.

  As described above, according to the present embodiment, it is possible to automatically shift to the tracking shooting mode without imposing a load on the photographer.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and a computer such as the system reads and executes the program codes from the storage medium. Is also achieved.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  In addition, the case where the functions of the above-described embodiment are realized by performing part or all of the actual processing by an OS or the like running on the computer based on the instruction of the program code read by the computer. It is.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion unit connected to the computer, the CPU or the like performs actual processing based on the instruction of the program code, and the above-described processing is performed. The case where the functions of the embodiment are realized is also included.

It is a figure which shows the system configuration | structure of the imaging device which concerns on the 1st thru | or 3rd embodiment of this invention. It is a figure which shows the transition relationship of the operation mode between manual imaging mode and tracking imaging | photography mode of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the finder screen displayed on the display in the 1st Embodiment of this invention. It is a flowchart which shows main processes, such as the management process of the mode transition by an image process part, and the update process of a finder screen. It is a flowchart which shows the subprocess of the image process by an image process part. It is a figure which shows an example of the finder screen displayed on the display in the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device main body 101 Optical system 102 Imaging system 103 Imaging control part 104 Pan head control part 105 CPU
106 ROM
107 RAM
108 Image processing unit 110 Display 111 Display control unit 112 Button 113 Button control unit 114 External storage medium 115 External storage control unit 116 Input / output interface 120 Pan / tilt pan head 300, 600 Viewfinder screens 301-303, 601-603 Rectangular display

Claims (12)

Imaging means for imaging a subject;
Detecting means for detecting a tracking candidate object from image data output from the imaging means;
Selection means for selecting an arbitrary tracking candidate object from the tracking candidate objects detected by the detection means;
Setting means for setting the tracking candidate object selected by the selection means as a tracking target object;
An image pickup apparatus comprising: a control unit that shifts to a first mode for tracking and shooting the tracking target object set by the setting unit.   The imaging apparatus according to claim 1, wherein the selection unit selects an arbitrary tracking candidate object from the tracking candidate objects detected by the detection unit in response to an operation input by a predetermined operation member.   The detection means detects a tracking candidate object in the first mode, and the selection means can select an arbitrary tracking candidate object from the tracking candidate objects detected by the detection means in the first mode. The setting means switches and sets the selected tracking candidate object as a tracking target object when a tracking candidate object different from the currently set tracking target object is selected by the selection means. The imaging apparatus according to 1 or 2. Imaging means for imaging a subject;
Detecting means for detecting a tracking candidate object from image data output from the imaging means;
Setting means for setting a tracking candidate object that satisfies a predetermined condition among the tracking candidate objects detected by the detection means as a tracking target object;
An image pickup apparatus comprising: a control unit that shifts to a first mode for tracking and shooting the tracking target object set by the setting unit.   The setting unit sets a tracking candidate object displayed in a predetermined display area as a tracking target object on a display unit that displays image data output from the imaging unit. The imaging device described.   The imaging apparatus according to claim 5, wherein the predetermined display area is a display area substantially in the center of the display unit. Load detection means for detecting a load due to an external force for driving the imaging apparatus in the first mode;
7. The control unit according to claim 1, wherein when the load is detected by the load detection unit, the control unit shifts from the first mode to a second mode for performing manual photographing. The imaging device according to item.   The control means, when there is an operation input from a predetermined operation member in the first mode, makes a transition from the first mode to a second mode for performing manual photographing. The imaging device according to any one of 4 to 6. A method for controlling an imaging apparatus having imaging means for imaging a subject,
A detection step of detecting a tracking candidate object from the image data output from the imaging means;
A selection step of selecting an arbitrary tracking candidate object from the tracking candidate objects detected by the detection step;
A setting step of setting the tracking candidate object selected in the selection step as a tracking target object;
A control step of transitioning to a first mode in which the tracking target object set in the setting step is tracked and photographed. A method for controlling an imaging apparatus having imaging means for imaging a subject,
A detection step of detecting a tracking candidate object from the image data output from the imaging means;
A setting step of setting a tracking candidate object that satisfies a predetermined condition among the tracking candidate objects detected by the detection step as a tracking target object;
A control step of transitioning to a first mode in which the tracking target object set in the setting step is tracked and photographed.   The program for making a computer perform the control method of the imaging device of Claim 9 or 10.   The computer-readable recording medium which recorded the program of Claim 11.

Images