JP4742901B2 - Electronic camera - Google Patents

Description

  The present invention relates to an electronic camera having a camera shake correction function.

Conventionally, in camera control, a real-time operating system (hereinafter referred to as RTOS) has been used in order to efficiently process a program group necessary for controlling each unit. An example of such a camera RTOS is disclosed in Patent Document 1, for example. In addition, in an electronic camera having a camera shake correction function, it is common to provide a processor specialized for camera shake correction control. However, a control unit that controls the entire camera may be burdened with camera shake correction control. It is being considered.
JP 2002-131811 A

  By the way, the importance of the camera shake correction process of the electronic camera changes depending on the operation state in the shooting mode (for example, when shooting standby, shooting, and processing of image data after shooting). Therefore, when camera shake correction is controlled by RTOS, it is required to optimize the camera shake correction process according to the operation state in the shooting mode and improve the performance of the entire electronic camera. Note that Patent Document 1 does not disclose any camera shake correction.

  The present invention is to solve the above-described problems of the prior art, and an object of the present invention is to provide an electronic camera capable of optimizing camera shake correction processing in accordance with the operation state in the shooting mode.

An electronic camera of one embodiment of the present invention includes an imaging element, an image processing unit, a camera shake correction unit, and a control unit. The image sensor captures a subject in the shooting screen. The image processing unit generates image data based on the output of the image sensor. The camera shake correction unit detects the amount of camera shake and corrects the image blur based on the camera shake amount. The control unit controls each part of the camera according to the priority in the shooting mode for shooting the subject, and when the execution timing of the camera shake correction task related to the control of the camera shake correction unit and other processing tasks conflicts. The processing priority of the camera shake correction task is changed according to the operation state in the shooting mode. The control unit is at least one of a first operation state in which a shooting instruction from the user is waited, a second operation state in which shooting is performed with an image sensor in response to the shooting instruction, and a third operation state in which image data is generated after exposure. The processing priority of the camera shake correction task is changed between these states. In addition, the control unit lowers the processing priority of the camera shake correction task in the third operation state to be lower than the processing priority of the task related to the image processing in the image processing unit. Continue to detect quantity.

According to the present invention, the processing priority of the camera shake correction task when the execution timing of the camera shake correction task and another processing task conflicts is changed according to the operation state in the shooting mode, and the camera shake correction is performed. control Ru are optimized for each operating condition.

  FIG. 1 is a block diagram showing the configuration of the electronic camera of this embodiment. The electronic camera includes a photographing lens 11, a first lens driving unit 12, a blur correction lens 13, a lens position detecting unit 14, a blur detecting unit 15, a second lens driving unit 16, an image sensor 17, and an A. A / D conversion unit 18, an image processing unit 19, a buffer memory 20, a recording I / F 21, a monitor 22, an operation unit 23, a control unit 24, and a bus 25. The A / D conversion unit 18, the image processing unit 19, the buffer memory 20, the recording I / F 21, the monitor 22, and the control unit 24 are connected via a bus 25.

The taking lens 11 is composed of a plurality of lens groups including a focusing lens for adjusting the in-focus position. The photographing lens 11 is configured so that the position in the optical axis direction can be adjusted by the first lens driving unit 12.
The blur correction lens 13 is a lens for correcting camera shake occurring in the electronic camera. The blur correction lens 13 is configured to be swingable in the direction perpendicular to the optical axis.

The lens position detector 14 detects the vertical position and the horizontal position of the shake correction lens 13 in a plane perpendicular to the optical axis. The output of the lens position detection unit 14 is connected to the control unit 24.
The shake detection unit 15 includes a vertical angular velocity sensor that detects the pitch of the electronic camera and a horizontal angular velocity sensor that detects the roll of the electronic camera. As each of the above angular velocity sensors, for example, a piezoelectric vibration type angular velocity sensor that detects Coriolis force due to rotation is used. The output of the blur detection unit 15 is connected to the control unit 24.

  The second lens driving unit 16 includes a vertical swing mechanism that swings the blur correction lens 13 in the vertical direction and a horizontal swing mechanism that swings the image blur correction lens 13 in the horizontal direction. The second lens driving unit 16 swings the blur correction lens 13 based on the output of the control unit 24. As a result, the imaging position of the subject image is shifted, and the image blur of the subject on the image sensor 17 is suppressed.

  The image sensor 17 photoelectrically converts a subject image by a light beam that has passed through the photographing lens 11 and the like to generate an analog image signal. The output of the image sensor 17 is connected to the A / D converter 18. In the shooting mode for shooting a subject, the image sensor 17 captures the subject at the time of release and also outputs an analog image signal (through image signal) at predetermined intervals even during standby for shooting. The through image signal is used for various arithmetic processes by the control unit 24 and display on the monitor 22.

The A / D converter 18 converts the analog image signal output from the image sensor 17 into a digital image signal.
The image processing unit 19 performs image processing (defective pixel correction, gradation correction, interpolation, color conversion, edge enhancement, etc.) on the digital image signal at the time of release to generate photographed image data. In addition, the image processing unit 19 can execute a light amount correction process for adjusting the gradation of a dark part or a highlight part of an image by image analysis, a red-eye correction process by image processing, or the like.

  Further, the image processing unit 19 generates a display image (view image) from the through image signal according to an instruction from the control unit 24. This view image is displayed as a moving image on the monitor 22 during shooting standby. Furthermore, the image processing unit 19 generates a display image (also referred to as a freeze image) displayed on the monitor 22 when the captured image data is generated. This freeze image corresponds to a photographed image at the time of release. The image processing unit 19 also executes processing for compressing captured image data in the JPEG format at the time of recording, and processing for decompressing and restoring the compressed captured image data at the time of reproduction.

The buffer memory 20 temporarily stores data in a pre-process or post-process of image processing by the image processing unit 19. It is also possible to provide a recording area for recording captured image data in the buffer memory 20.
A connector for connecting the recording medium 26 is formed on the recording I / F 21. The recording I / F 21 executes writing / reading of the captured image data with respect to the recording medium 26 connected to the connector. The recording medium 26 includes a hard disk, a memory card with a built-in semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the recording medium 26.

  The monitor 22 displays a screen instructed by the control unit 24 via a monitor driver (not shown). For example, the view image is displayed as a moving image on the monitor 22 at the time of shooting standby. The monitor 22 displays various information necessary for shooting (for example, the number of frames that can be shot, the position of the focus detection area, and the frame display of the face area detected by the face detection process described later) by the on-screen function. Etc.) can be superimposed on the view image.

Further, the monitor 22 can display a reproduced image of captured image data, a menu screen that allows input in a GUI (Graphical User Interface) format, and the like. In the example of the present embodiment, the monitor 22 is a liquid crystal monitor (view images and menu screens on the monitor 22 are not shown).
The operation unit 23 has a release button, an operation button, and the like. The release button of the operation unit 23 receives a release timing instruction input from the user. The operation button of the operation unit 23 receives an input on the menu screen or the like from the user, for example.

  The control unit 24 controls the operation of each part of the camera. As shown in FIG. 1, the control unit 24 includes a ROM 27, a CPU 28, and a work memory 29. The ROM 27 stores a kernel serving as the core of the RTOS function and a plurality of application programs. The CPU 28 executes arithmetic processing of a program related to RTOS. The work memory 29 is used as a work area when the CPU 28 calculates a program. Also, in the work memory 29, a part of the application program can be resident to perform various processes. The work area of the work memory 29 is managed by the RTOS. The work memory 29 can be read and written at high speed, but its capacity is small due to the cost. For this reason, most of the application programs constituting various tasks are executed on a large-capacity memory (not shown) provided outside the control unit 24.

  Here, the RTOS kernel has a task scheduling function, an interrupt control function, a system call function, a memory pool function, and the like. The RTOS executes a plurality of application programs in units of processing called tasks. At this time, the RTOS kernel changes the task state by a system call in response to a request from an interrupt handler or task. When the task state is changed, the task scheduler in the kernel is called to reselect the task to be executed (scheduling).

  The execution order of tasks by the RTOS depends on the priority of each task, and the RTOS executes in order from the task with the highest priority. As an example, when task B having a high priority appears during execution of task A in RTOS, task B having a high priority enters an execution state by interrupt processing, while task A being executed enters a wait state. Thereafter, when the interrupt process of task B is completed, the RTOS resumes the process of task A that was temporarily suspended. As will be described later, the priority of the task in the present embodiment varies depending on the operation state in the shooting mode.

  The tasks processed by the RTOS of this embodiment include a camera shake correction task, a through image processing task, a release operation task, a captured image data generation task, a captured image display task, a GUI control task, and an information management task. (See FIG. 3). The control unit 24 controls each unit of the electronic camera and executes various calculations according to each task processed by the RTOS.

  (1) The camera shake correction task is a task for causing the electronic camera to perform camera shake correction. In the camera shake correction task, the control unit 24 calculates the shake amount (blur direction and blur magnitude) of the electronic camera based on the output of the shake detection unit 15. Then, the control unit 24 instructs the second lens driving unit 16 to drive the blur correction lens 13 based on the blur amount information and the output of the lens position detection unit 14. Here, the camera shake correction of the electronic camera requires periodic control in order to ensure the correction accuracy. However, if the control cycle varies, the correction accuracy decreases. For this reason, in this embodiment, the priority of the camera shake correction task is made higher than other tasks in accordance with the operation state in the shooting mode.

  (2) The through image processing task is a task for causing the electronic camera to execute a plurality of processes related to the through image at the time of shooting standby. In this through image processing task, the control unit 24 executes drive control of the image sensor 17 related to acquisition of the through image, control of the image processing unit 19 related to generation of the view image, and display control of the view image on the monitor 22. . In the through image processing task, the control unit 24 performs various calculations such as AF (autofocus) calculation by contrast detection method, AE (automatic exposure) calculation, and AWB (auto white balance) calculation based on the data of the view image. Is performed in a known manner.

  Here, in the AF calculation described above, the control unit 24 can perform face detection processing on the view image and determine a focus detection area based on the detected face area. For example, the control unit 24 can extract a face region from a view image by a feature point extraction process such as Japanese Patent Application Laid-Open No. 2001-16573. Examples of the feature points include eyebrow, eye, nose, and lip end points, face contour points, head apexes, and chin lower end points. Alternatively, as disclosed in Japanese Patent Application Laid-Open No. 8-63597, the control unit 24 detects the face area by extracting the outline of the skin color area based on the color information of the subject, and further matching with a template of facial parts prepared in advance. May be.

(3) The release operation task is a task for causing the electronic camera to shoot a subject in response to a user's shooting instruction by fully pressing the release button. In the release operation task, the control unit 24 executes exposure control of the image sensor 17 according to a shooting instruction, control related to A / D conversion processing and buffering of an image signal, and the like.
(4) The captured image data generation task is a task for causing the electronic camera to generate captured image data based on the image signal read out in the release operation task. In this image data creation task, the control unit 24 executes control necessary for various types of image processing of the image processing unit 19 and control for recording captured image data on a recording medium or the like.

(5) The captured image display task is a task for causing the monitor 22 to display the freeze image when generating captured image data. In this captured image display task, the control unit 24 executes control for causing the image processing unit 19 to generate a freeze image and display control for the freeze image on the monitor 22.
(6) The GUI control task is a task for causing the control unit 24 to perform GUI display processing on the menu screen and display control of the monitor 22 by the on-screen function.

(7) The information management task is a task for detecting an input state from the operation unit 23 (release button and operation button), a battery level of a power source (not shown), and causing the control unit 24 to manage the information. is there.
Hereinafter, the operation in the shooting mode of the electronic camera of the present embodiment will be described. First, the transition of the operation state in the photographing mode will be described based on the flowchart of FIG. In the present embodiment, the description will be made on the assumption that the camera shake correction function is set to ON.

Step 101: When the photographing mode of the electronic camera is activated, the RTOS of the control unit 24 operates in a first operation state that is a photographing standby state. In the first operation state, a camera shake correction task, a through image processing task, a GUI control task, an information management task, and the like can be executed on the RTOS of the control unit 24.
Specifically, in the electronic camera in the first operation state, a view image based on the through image is displayed as a moving image on the monitor 22, and the control unit 24 executes AF calculation, AE calculation, AWB calculation, and the like to prepare for shooting. Then, the control unit 24 waits for a user's shooting instruction by fully pressing the release button. In the first operation state, the control unit 24 drives the shake correction lens 13 to acquire a through image and performs camera shake correction.

Step 102: The control unit 24 determines whether or not the release button has been fully pressed by the user. When the release button is fully pressed (YES side), the process proceeds to S103. On the other hand, when the release button is not fully pressed (NO side), the process proceeds to S105.
Step 103: The RTOS of the control unit 24 shifts to the second operation state for photographing the subject according to the release. In the second operation state, a camera shake correction task, a release operation task, a GUI control task, an information management task, and the like can be executed by the RTOS.

Specifically, the electronic camera in the second operation state takes an image of the subject by controlling the image sensor 17 based on the exposure time obtained by the AE calculation. In the second operation state, the control unit 24 drives the shake correction lens 13 to acquire a captured image and performs camera shake correction.
Step 104: The RTOS of the control unit 24 shifts to the third operation state for generating captured image data after the exposure of the image sensor 17 is completed. After the end of the third operation state, the RTOS of the control unit 24 returns to S101 and returns to the first operation state. In the third operation state, a camera shake correction task, a release operation task, a captured image data generation task, a captured image display task, and the like can be executed by the RTOS.

Specifically, in the electronic camera in the third operation state, the image processing unit 19 generates captured image data, and the control unit 24 records the captured image data on a recording medium or the like. In the third operation state, a freeze image is displayed on the monitor 22 until the next shooting is possible.
Here, since the imaging operation is not performed in the third operation state, the control unit 24 stops driving the blur correction lens 13. However, in order to quickly resume camera shake correction after the return to the first operation state, the control unit 24 in the third operation state executes detection of the shake amount in the background.

Step 105: The control unit 24 determines whether or not an operation for starting a menu screen has been executed by the user. When the above operation is executed (YES side), the process proceeds to S106. On the other hand, when the above operation is not executed (NO side), the process returns to S101, and the RTOS of the control unit 24 continues the operation in the first operation state.
Step 106: The RTOS of the control unit 24 shifts to a fourth operation state for performing various processes relating to the menu screen. After the end of the fourth operation state, the RTOS of the control unit 24 returns to S101 and returns to the first operation state. In the fourth operation state, a camera shake correction task, a through image processing task, a GUI control task, an information management task, and the like can be executed by the RTOS.

Specifically, a menu screen is displayed on the monitor 22 in the electronic camera in the fourth operation state. The user can change various settings of the electronic camera by operating icons on the screen with the operation buttons.
Here, since the imaging operation is not performed in the fourth operation state, the control unit 24 stops driving the blur correction lens 13. However, in order to restart the camera shake correction quickly after the return to the first operation state, the control unit 24 detects the amount of shake in the background even in the fourth operation state. Similarly, the display of the view image is interrupted in the fourth operation state, but the control unit 24 periodically acquires a through image and executes AE calculation and the like in the background in preparation for returning to the first operation state. In the fourth operation state, the control unit 24 may make the through image generation frequency lower than that in the first operation state.

Next, FIG. 3 shows task priorities in the respective operation states of the above-described shooting modes.
The priority of each task in the first operation state is set so as to decrease in the order of camera shake correction task, through image processing task, information management task, and GUI control task (see FIG. 3A). . In this first operating state, camera shake correction has an adverse effect on view image display, AF calculation, and the like, so it is necessary to perform camera shake correction with high accuracy. Therefore, in the first operation state, the camera shake correction task has priority over the through image processing task, and the camera shake correction task is given the highest priority.

  The priority of each task in the second operation state is set so as to decrease in the order of the camera shake correction task, the release operation task, the information management task, and the GUI control task (see FIG. 3B). In this second operation state, if image blurring occurs in the captured image due to camera shake during exposure, shooting fails, so it is necessary to perform camera shake correction with high accuracy. Therefore, in the second operation state, the camera shake correction task is prioritized over the release operation task, and the highest priority is given to the camera shake correction task.

  The priority of each task in the third operation state is set to decrease in the order of the release operation task, the captured image data generation task, the captured image display task, and the camera shake correction task (FIG. 3C). reference). In the third operation state, it is necessary to generate and record captured image data as quickly as possible in order to enable the next shooting. On the other hand, since there is no need for camera shake correction in the third operation state, the importance of the camera shake correction task is relatively low. Therefore, in the third operation state, the priority of the release operation task and the captured image data generation task is set higher than that of the camera shake correction task. The reason why the release operation task has the highest priority is that, in the image processing of the captured image data generation task, image signal buffering for one frame by the release operation task is a precondition.

  The priority of each task in the fourth operation state is set so as to decrease in the order of GUI control task, information management task, camera shake correction task, and through image processing task (see FIG. 3D). . In the fourth operation state, it is preferable to improve the responsiveness of the display on the menu screen to the user input as much as possible. On the other hand, in the fourth operation state, there is no need for camera shake correction, and display of the view image is interrupted, so the importance of the camera shake correction task and the through image processing task is relatively low. Therefore, in the fourth operation state, the priority of the GUI control task and the information management task is set higher than that of the camera shake correction task.

  Next, changes in the usage status of the work memory 29 in accordance with the operation state in the shooting mode will be described with reference to FIG. The control unit 24 of the present embodiment speeds up task processing by making a program resident in the work memory 29. And the control part 24 has replaced the part or all of the program which occupies the work memory 29 according to the operation state. Note that the upper limit of the occupation ratio of the camera shake correction task in the work memory 29 changes according to the priority of the task.

  An example of the usage status of the work memory 29 in the first operation state and the second operation state is shown in FIG. In these operating states, the camera shake correction task operates as the highest priority task, and the control unit 24 periodically executes camera shake correction processing. Therefore, in the first operation state and the second operation state, the control unit 24 basically provides the entire area of the work memory 29 to the camera shake correction task program. In this way, by using the work memory 29 to improve the processing speed of the camera shake correction task, the control unit 24 suppresses the influence on other tasks associated with the priority process of the camera shake correction task.

  FIG. 4B shows an example of the usage status of the work memory 29 when face detection AF is performed in the first operation state. In this case, in order to improve the processing speed of the face detection process, the control unit 24 provides a part (for example, 20%) of the work area of the work memory 29 to the program for the face detection process. However, since the priority of the task is higher in the camera shake correction task than in the face detection AF process, in this case, the control unit 24 assigns more work areas in the work memory 29 to the camera shake correction task.

  On the other hand, FIG. 4C shows an example of the usage state of the work memory 29 in the third operation state. In this case, in order to quickly perform the image processing of the captured image data, the control unit 24 uses most of the work area (for example, 80%) of the work memory 29 for the image processing program (for example, light amount correction processing, red-eye correction processing, etc. For large processing load). Then, the control unit 24 provides a part (for example, 20%) of the work area of the work memory 29 to the camera shake correction task program, and continuously executes the shake amount calculation and the like.

For reference, FIG. 4D shows an example of the usage status of the work memory 29 in a playback mode in which captured image data is played back. In this case, since it is not necessary to perform blur correction, the control unit 24 provides the entire area of the work memory 29 to the image processing program to improve the display speed of the reproduced image.
Hereinafter, functions and effects of the electronic camera of the present embodiment will be described. In the present embodiment, the control unit 24 dynamically changes the priority of tasks executed by the RTOS and the occupation ratio of the work memory 29 according to the operation state of the shooting mode. Thereby, control of camera shake correction of the electronic camera is optimized for each operation state.

Therefore, in the electronic camera of the present embodiment, the camera size and power consumption can be reduced by reducing the circuit scale while ensuring substantially the same performance as the electronic camera having a processor specialized in camera shake correction control. Suppression and cost reduction can be achieved.
(Supplementary items of the embodiment)
(1) In the above embodiment, the example of the electronic camera that performs the blur correction by driving the blur correction lens has been described. However, the blur correction configuration of the electronic camera of the present invention is not limited to the above embodiment. For example, the present invention can also be applied to an electronic camera for electronic camera shake correction that obtains a partial image subjected to shake correction by shifting the cutout position from the entire image in accordance with the amount of shake. The present invention can also be applied to an electronic camera provided with a shake correction mechanism that mechanically shifts an image sensor (all illustrations relating to the configuration of the camera are omitted).

  (2) The task priority and the work memory occupancy in each operation state described in the above embodiment are merely examples, and can be appropriately adjusted according to the configuration and use conditions of the electronic camera.

The block diagram which shows the structure of the electronic camera of this embodiment Flow chart for explaining transition of operation state in shooting mode The figure which shows an example of the priority of the task in each operation state of photography mode The figure which shows an example of the usage condition of the work memory according to the operating state in the shooting mode

Explanation of symbols

DESCRIPTION OF SYMBOLS 13 ... Blur correction lens, 14 ... Lens position detection part, 15 ... Blur detection part, 16 ... 2nd lens drive part, 17 ... Image pick-up element, 19 ... Image processing part, 22 ... Monitor, 23 ... Operation part, 24 ... Control 27: ROM, 28: CPU, 29: Work memory

Claims (6)

An image sensor for photographing the subject in the shooting screen;
An image processing unit that generates image data based on the output of the imaging device;
A camera shake correction unit that detects a camera shake amount and corrects a camera shake based on the camera shake amount;
While controlling each part of the camera according to the priority in the shooting mode for shooting the subject, the execution time of the camera shake correction task related to the control of the camera shake correction unit and other processing tasks conflicts with each other. A control unit that changes the processing priority of the camera shake correction task according to the operation state in the shooting mode ,
The control unit includes a first operation state in which a shooting instruction from a user is waited for, a second operation state in which shooting is performed with the image sensor in response to the shooting instruction, and a third operation state in which the image data is generated after exposure. And change the processing priority of the camera shake correction task between at least one of the states,
The control unit sets the processing priority of the camera shake correction task in the third operation state to be lower than the processing priority of the task related to image processing in the image processing unit, and when in the third operation state In addition, the detection of the blur amount is continued . The electronic camera according to claim 1 ,
A fourth operation state for accepting a setting change operation by GUI;
The control unit sets a processing priority of the camera shake correction task in the fourth operation state to be lower than a processing priority of a task related to the GUI control, and the shake amount in the fourth operation state. The electronic camera is characterized by continuing the detection . The electronic camera according to claim 1 or 2 ,
A monitor for displaying an image showing the state of the shooting screen in the shooting mode;
Wherein the control unit, the processing priority of the camera shake correction task in the first operating state, the electronic camera, which comprises greater than the processing priority of the filter disk relates to a display control of the monitor. The electronic camera according to any one of claims 1 to 3 ,
Wherein the control unit, an electronic camera, characterized in that the processing priority of the camera shake correction task in the second operating state, is higher than the processing priority of the filter disk relates to a control of the exposure of the imaging device . The electronic camera according to any one of claims 1 to 4,
It is further equipped with a memory used as a work area for arithmetic processing,
The electronic camera according to claim 1, wherein the control unit changes an upper limit of the occupation ratio of the memory for the camera shake correction task according to an operation state in the photographing mode. The electronic camera according to claim 5,
In the operation state in which the image processing unit generates the image data after exposure, the control unit sets the memory occupation rate for the camera shake correction task to be higher than the memory occupation rate for the image processing on the image data. An electronic camera characterized by being lowered.

Images