JP2010193202A - Digital camera, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital camera that can obtain a photographed image only at an optional timing even when a timing of pressing a release button is delayed. <P>SOLUTION: A display image acquisition section 16a creates RAW data and display image data at regular time intervals according to the image signal output by an imaging element 15 when the digital camera 1 is in live view mode. A differential creation section 16b picks up only differences between two RAW data and creates differential data. An image restoration section 16c uses the differential data and one of the RAW data of the two RAW data as a basis for creating the differential data, so as to restore the other RAW data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a digital camera having a live view function.

  Some digital cameras currently in use have a continuous shooting function. The continuous shooting function is a function that repeats shooting while pressing the release button. When shooting a moving object, if the user starts pressing the release button at an early timing, an image shot at an appropriate timing can be obtained from a plurality of images shot by the continuous shooting function.

  In order to use the continuous shooting function, a storage area larger than a certain size is required to record a plurality of images. As a technique for suppressing the size of a necessary storage area, for example, an imaging apparatus that compares two consecutive captured images in continuous shooting mode and records only the difference is known (Patent Document 1).

JP 2001-320623 A

  The conventional continuous shooting function cannot deal with a case where the action of pressing the release button for a plurality of images is delayed.

According to a first aspect of the present invention, there is provided imaging means for acquiring still image data of a subject at predetermined intervals in response to a predetermined camera operation, first still image data acquired by the imaging means, and the imaging means. Based on the second still image data acquired immediately before the first still image, when the data of the corresponding pixels is different, the difference data generating means for generating difference image data and the imaging means acquired A storage unit that records the latest still image data and one or more difference image data created by the difference data creation unit, and a release based on the still image data and the difference image data recorded in the storage unit. A digital camera comprising: restoration means for restoring still image data that has already been erased when the button is pressed.
According to a seventh aspect of the present invention, there is provided a receiving step of receiving still image data at predetermined intervals from an imaging device that shoots a subject and creating still image data, and the first still image data received in the receiving step. And a difference data creation step of creating difference image data when the data of corresponding pixels is different based on the second still image data received immediately before the first still image data in the reception step. A storage step for recording the latest still image data received in the reception step and one or a plurality of difference image data created in the difference data creation step, and still image data recorded in the storage step And a restoration step of restoring still image data that has already been erased based on the difference image data. Which is the image processing program characterized.

  According to the present invention, even when the timing of pressing the release button is later than the timing intended by the user, a captured image at the timing as intended can be obtained.

It is a block diagram which shows the circuit structure of the digital camera by 1st Embodiment. It is a figure explaining a still picture restoration function. It is a figure which shows a mode that difference data are produced. It is a figure which shows a mode that difference data are produced when a to-be-photographed image changes large. 4 is a flowchart of still image restoration processing executed by the digital camera 1. It is a figure which shows a mode that difference data are produced when a to-be-photographed image changes large in 2nd Embodiment.

-First embodiment-
FIG. 1 is a block diagram showing a circuit configuration of a digital camera according to this embodiment. The digital camera 1 includes an input device 13, a focus adjustment device 14, an image sensor 15, a control circuit 16, a DRAM 17, a flash memory 18, an LCD drive circuit 19, a liquid crystal monitor 20, a memory card interface (I / F) 21, and an acceleration sensor. 23.

  The imaging element 15 is configured by a CCD or CMOS image sensor including a plurality of photoelectric conversion elements corresponding to pixels. The imaging element 15 captures a subject image formed on the imaging surface and outputs a photoelectric conversion signal (image signal) corresponding to the brightness of the subject image. R (red), G (green), and B (blue) color filters are provided on the imaging surface of the imaging element 15 so as to correspond to the pixel positions, respectively. Since the image pickup device 15 picks up a subject image through the color filter, the image signal output from each image pickup device has RGB color system color information.

  The image signal output from the image sensor 15 is input to the control circuit 16. The control circuit 16 performs various image processing on the image signal to generate image data. Then, the control circuit 16 performs compression processing on the generated image data by a predetermined method such as JPEG, and records it in the storage medium 22 as an image file in a format such as EXIF. The control circuit 16 is composed of, for example, a RISC and controls each circuit shown in FIG.

  The focus adjustment device 14 performs a known focus evaluation value calculation based on the imaging signal output from the imaging element 15, and outputs the calculation result to the control circuit 16. Based on the calculation result, the control circuit 16 moves the photographing lens 10 to a position where the focus evaluation value is maximized.

  The DRAM 17 is used to temporarily store data during or after image processing, image compression processing, and display image data creation processing. The display image data is generated by the control circuit 16 based on the image data generated by the control circuit 16 based on the output from the image sensor 15 or the image data recorded in the storage medium 22. The generated display image data is stored in the DRAM 17 by the control circuit 16. The flash memory 18 is a non-volatile memory in which various processing programs for the operation of the control circuit 16 are recorded, for example.

  The LCD drive circuit 19 drives the liquid crystal monitor 20 based on a command from the control circuit 16, and the liquid crystal monitor 20 displays an image based on display image data stored in the DRAM 17. The liquid crystal monitor 20 displays various setting menu screens of the digital camera. The input device 13 includes various operation buttons such as a release button and a playback button, and outputs an operation signal of each operation button by the user to the control circuit 16. The acceleration sensor 23 is a triaxial acceleration sensor that detects movements in the vertical direction, the horizontal direction, and the front-rear direction applied to the digital camera 1.

  The control circuit 16 includes a display image acquisition unit 16a, a difference creation unit 16b, and an image restoration unit 16c. When the digital camera 1 is in the live view mode, the display image acquisition unit 16a creates RAW data and display image data at regular intervals based on the image signal output from the image sensor 15. Here, the RAW data is data obtained by simply performing A / D conversion on an image signal, and includes a plurality of pixel values including color information. These data are stored in the DRAM 17.

  When the digital camera 1 is in the live view mode, the control circuit 16 displays the display image data on the liquid crystal monitor 20 as a so-called through image every time the display image acquisition unit 16a creates the display image data. In this embodiment, the display image acquisition unit 16a creates display image data 30 times per second and RAW data 10 times per second.

  The difference creation unit 16 b creates difference data from two pieces of RAW data stored in the DRAM 17. The difference data is data obtained by extracting only the difference between the two RAW data, and is composed of address data representing a location where the pixel value differs between the two RAW data and a numerical value representing the difference between the pixel values. The data amount of the difference data depends on the magnitude of the difference between the two RAW data. The smaller the difference is, the smaller the data amount of the difference data is. In most cases, the difference data is much smaller than the RAW data.

  The image restoration unit 16c restores the other RAW data from the difference data and one RAW data of the two RAW data that is the basis for creating the difference data. For example, when the difference creation unit 16b creates difference data from the RAW data A and the RAW data B, the image restoration unit 16c restores the RAW data B using the difference data and the RAW data A, RAW data A can be restored using the data and RAW data B.

  Next, the still image restoration function of the digital camera 1 will be described. When the digital camera 1 is in the live view mode, the still image restoration function is enabled. When the still image restoration function is enabled, when the user performs shooting by pressing the release button, an image including a subject image before the time of shooting is displayed separately from the captured image. One or more are displayed. The user can record any image data corresponding to the selected image in the storage medium 22 by selecting any of these images.

  FIG. 2 is a diagram illustrating the still image restoration function. Initially, a through image 31 is displayed on the liquid crystal monitor 20. The through image 31 includes an automobile 34 that moves from the right to the left of the screen. Shortly thereafter, the through image 32 is displayed on the liquid crystal monitor 20. The user wants to take a picture at the time of the through image 32 and presses the release button. However, the image 33 has actually been taken because the timing for pressing the release button is delayed. That is, although the automobile 34 is located on the right side of the screen and the photographed image is not overlapped with the person 35, the automobile 34 is actually hidden behind the person 35 as shown in the image 33. The captured image was obtained.

  If the still image restoration function of the digital camera 1 is used, as described above, when a photographed image having a timing different from that intended by the user has been obtained, a photographed image as desired by the user can be obtained. it can. After pressing the release button, the captured image 33 is displayed on the liquid crystal monitor 20. At this time, when the user operates the input device 13, the liquid crystal monitor 20 can display an image before the captured image 33 such as an image corresponding to the through image 32 or an image corresponding to the through image 31. it can.

  Thereafter, when the user presses the release button again, the control circuit 16 records image data corresponding to the image displayed on the liquid crystal monitor 20 in the storage medium 22. In this way, the user can record in the storage medium 22 image data different from the photographed image 33 that should have been originally obtained by photographing.

  Next, a method for realizing the still image restoration function in the digital camera 1 will be described. If the RAW data periodically recorded by the display image acquisition unit 16a is simply recorded in the DRAM 17, the DRAM 17 needs a huge storage capacity. Therefore, in the present embodiment, the subject image before the release button is pressed is efficiently recorded using the difference data created by the difference creating unit 16b.

  FIG. 3 is a diagram illustrating how difference data is created. This figure shows the state of the RAW data and the difference data in the DRAM 17 when the time elapses from the time t0. The state at the time t0 is shown at the top of the figure, the state at the time t1 after the elapse of the unit time is shown below, and the state at the times t2 and t3 are shown further below.

  First, at time t0, the RAW data P0 (41) is created by the display image acquisition unit 16a. Thereafter, at time t1 when the unit time has elapsed, the display image acquisition unit 16a further creates RAW data P1 (42). At this time, according to the creation of the RAW data P1 (42), the difference creating unit 16b creates the difference data P1-P0 (41d) between the RAW data P1 (42) and the RAW data P0 (41). Then, the RAW data P0 (41) is replaced with the difference data P1-P0 (41d).

  Furthermore, when it becomes time t2 after unit time, the display image acquisition part 16a produces RAW data P2 (43). In response to this, the difference creating unit 16b creates the difference data P2-P1 (42d) between the RAW data P2 (43) and the RAW data P1 (42), and the RAW data P1 (42) is the difference. Replaced with data. Similarly, at time t3, new RAW data P3 (44) is created, and RAW data P2 (43) is replaced with difference data P3-P2 (43d).

  In this way, the RAW data created last and the difference data obtained by replacing the RAW data created so far are recorded in the DRAM 17 every unit time. Note that when the difference data recorded in the DRAM 17 is 40 or more, the control circuit 16 deletes old difference data so that the difference data is less than 40.

  If the difference data is recorded in the DRAM 17 by the above procedure, the RAW data replaced with the difference data can be restored by the image restoration unit 16c. For example, assume that the user has taken a picture by pressing the release button at time t3 shown in FIG. At this time, the user can obtain an image including the subject image at each time of t0 to t3. Hereinafter, a method for obtaining an image at each time will be described.

  First, an image at time t3 can be obtained by performing predetermined image processing on the RAW data P3 (44). When obtaining an image at time t2, first, the RAW data P3 (44) and the difference data P3-P2 (43d) are given to the image restoration unit 16c to create the RAW data P2 (43). Next, if predetermined image processing is performed on the RAW data P2 (43), an image at time t2 can be obtained.

  Similarly, when an image at time t1 is obtained, first, RAW data P3 (44) and difference data P3-P2 (43d) are given to the image restoration unit 16c to create RAW data P2 (43). Next, the RAW data P2 (43) and the difference data P2-P1 (42d) are given to the image restoration unit 16c to create the RAW data P1 (42). If predetermined image processing is performed on the RAW data P1 (42), an image at time t1 can be obtained. The same applies to time t0.

  The control circuit 16 displays each image obtained as described above according to a user operation. When the user presses the release button again, the currently displayed image is recorded in the storage medium 22.

  The method for sequentially recording the difference data in the DRAM 17 described above assumes that the difference data has a sufficiently small size. That is, it is assumed that there is no significant change in RAW data for each unit time. When the RAW data changes greatly with the elapse of the unit time, the difference data does not become so small, and as a result, the effect of reducing the storage area by the above-described method is reduced. Therefore, the digital camera 1 measures the magnitude of change in the RAW data for each unit time. If the RAW data has changed greatly, that is, if the subject has moved greatly or a large movement has been applied to the digital camera 1, the data recorded in the DRAM 17 is cleared and the difference is determined. Start recording data again from the beginning.

  FIG. 4 is a diagram showing how difference data is created when the subject image changes greatly. From time t0 to t2, it is the same as in FIG. Now, it is assumed that the subject image has changed significantly after time t2, for example, because the position of the digital camera 1 has been moved greatly. After the RAW data P3 (44) is created by the display image acquisition unit 16a, normally, difference data between the RAW data P3 (44) and the RAW data P2 (43) is created by the difference creation unit 16b.

  However, as a result of comparing the two RAW data, the difference creating unit 16b creates the difference data when the pixels having different pixel values exist between these RAW data by 50% or more. Not performed. In this case, the control circuit 16 deletes all RAW data and difference data except the latest RAW data P3 (44). As a result, the state of the DRAM 17 at time t3 is as shown in FIG. Thereafter, as in the case of time t0, the creation of difference data is continued.

  Next, processing for realizing the still image restoration function by the digital camera 1 will be described with reference to the drawings.

  FIG. 5 is a flowchart of still image restoration processing executed by the digital camera 1. First, in step S11, it is determined whether or not the live view mode is set for the digital camera 1. If the live view mode is not set, step S11 is repeated. When the live view mode is set, the process proceeds to step S12.

  In step S 12, RAW data is acquired by the display image acquisition unit 16 a and recorded in the DRAM 17. In step S13, the process waits until a predetermined time elapses. In step S <b> 14, RAW data is acquired again by the display image acquisition unit 16 a and recorded in the DRAM 17. In step S15, the difference creating unit 16b checks the difference between the RAW data recorded in the DRAM 17 before step S14 and the RAW data recorded in step S14. If there is no difference between the two RAW data, the process proceeds to step S20.

  If the difference between the two RAW data is less than 50% in step S15, that is, if the number of pixels having different pixel values is less than 50% in the two RAW data, the process proceeds to step S16. In step S16, difference data is created by the difference creating unit 16b based on the two pieces of RAW data. In step S17, the older one of the two RAW data recorded in the DRAM 17 is replaced with the difference data created in step S16.

  On the other hand, if there are 50% or more of the total number of pixels between the two RAW data in step S15, the process proceeds to step S18. In step S18, all the data except the latest RAW data is deleted from the RAW data and the difference data recorded in the DRAM 17.

  In step S20, it is determined whether or not the release button has been pressed by the user. If the release button has not been pressed, a negative determination is made in step S20, and the process returns to step S13. On the other hand, if a positive determination is made in step S20, the process proceeds to step S21. In step S21, a plurality of image data before the current time are created based on the RAW data and the difference data recorded in the DRAM 17.

  In step S22, one image data is selected by the user from the image data created in step S21 and the image data created by photographing performed in step S20. In step S23, the image data selected in step S22 is recorded on the storage medium 22. In step S24, all RAW data and difference data recorded in the DRAM 17 are discarded, and the process is terminated.

  In step S15, the process may proceed to step S18 when the acceleration sensor 23 detects a movement of a predetermined threshold value or more. This is because a large movement applied to the digital camera 1 means that the subject image formed on the imaging surface also changes greatly, and as a result, the difference between the RAW data is considered to increase. is there.

According to the digital camera according to the first embodiment described above, the following operational effects can be obtained.
(1) Each time the display image acquisition unit 16a creates RAW data, the difference creation unit 16b creates difference data to replace the old RAW data. Thereby, the storage area required for recording the RAW data before the release button is pressed is reduced.

(2) The image data created based on the RAW data restored by the image restoration unit 16c can be recorded in the storage medium 22 instead of the image data created in response to pressing of the release button. Thereby, even if the timing of pressing the release button is delayed, a captured image as intended by the user can be obtained.

(3) Unlike the continuous shooting function, the still image restoration function records only the image data selected by the user in the storage medium 22. This eliminates the trouble of deleting unnecessary image data unlike the continuous shooting function.

(4) If a large change is detected in the RAW data, no difference is created. Thereby, the storage capacity of the DRAM 17 is not occupied by unnecessary data.

  In the first embodiment described above, when the difference between RAW data is 50% or more, the data that has been recorded so far is immediately deleted. In the second embodiment described in detail below, a certain number of difference data having a large difference is allowed.

-Second embodiment-
FIG. 6 is a diagram showing how difference data is created when the subject image changes greatly in the second embodiment. In FIG. 6, no significant change appears in the RAW data from time t0 to time t2.

  It is assumed that a large change has occurred in the RAW data when the unit time has elapsed from time t2 and has reached time t3. Here, the difference data P3-P2 (43d) created based on the RAW data P3 (44) and the RAW data P2 (43) has a difference in 60% of the pixels of the entire RAW data. In the second embodiment, a total of two difference data including such a large difference is allowed. Therefore, at this time point, the data recorded in the DRAM is not deleted.

  Assume that a large change occurs again in the RAW data at time t4. Here, the difference data P4-P3 (44d) includes a difference of 70%. At this point, since there are two difference data including a large difference, the data recorded in the DRAM is not yet deleted.

  Thereafter, at time t5, it is assumed that a large change has occurred in the RAW data. Here, there is a 55% difference between the RAW data P5 (46) and the RAW data P4 (45). If the difference data P5-P4 (45d) is created based on these two RAW data, the DRAM has three difference data including a difference of 50% or more which is a threshold value. . Therefore, as in the first embodiment, the control circuit deletes all RAW data and difference data except the latest RAW data P5 (46). As a result, at time t5, only the RAW data P5 (46) is recorded in the DRAM.

According to the digital camera according to the second embodiment described above, the following functions and effects can be obtained in addition to the functions and effects obtained by the digital camera according to the first embodiment.
(1) Even when a large change appears in the subject image, the RAW data and the difference data are not immediately deleted. Thereby, even when a large change occurs temporarily in the subject image, the image data can be restored.

The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.
(1) It may be displayed on the liquid crystal monitor 20 or other display device how many seconds ago the image after the release can be restored. Further, this display may be a number or a display using a bar or the like.

(2) When a difference of 50% or more occurs, a flag indicating that a large change is included may be set in the difference data. In this case, if a difference of 50% or more occurs again, all the difference data set with the previous flag and the difference data before the difference data may be cleared.

(3) After the release, a plurality of image data may be recorded on the storage medium 22 at the same time. Further, the photographed image data is always recorded, and in addition to this, the restored image data may be recorded.

(4) The difference data may have a data format other than those described above. For example, the difference data may be obtained by extracting a rectangular area including a change in pixel value from image data.

(5) When creating difference data, a pixel whose pixel value difference is not greater than or equal to a predetermined threshold value may not be treated as a changed pixel. In this case, an image including a large change is determined only when 50% or more of the pixels whose pixel values have changed by a predetermined threshold value or more exist. In addition, pixels whose pixel values have greatly changed although the pixel values of surrounding pixels have not changed may be ignored.

(6) When the display image data is acquired, a location where the pixel value is greatly changed may be registered, and only the comparison of the region may be performed when creating the difference data.

(7) RAW data before color information is included by image processing may be targeted. In this case, since each pixel has only a single luminance value, this luminance value is treated as a pixel value.

(8) The control circuit of each embodiment described above may be a CPU that executes a predetermined control program stored in the flash memory 18. In this case, each process executed by the control circuit in each embodiment is realized by the CPU executing a predetermined control program. In addition, in a computer to which an external camera is connected as an imaging unit, a program that executes the same processing as the control circuit described above may be used.

  As long as the characteristics of the present invention are not impaired, the present invention is not limited to the above-described embodiments, and other forms conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. .

1 Digital Camera 16 Control Circuit 16a Display Image Acquisition Unit 16b Difference Creation Unit 16c Image Restoration Unit 17 DRAM
22 Storage media

Claims (7)

Imaging means for acquiring still image data of the subject at predetermined intervals in response to a predetermined camera operation;
When the data of corresponding pixels is different based on the first still image data acquired by the imaging unit and the second still image data acquired by the imaging unit immediately before the first still image, Differential data creating means for creating differential image data;
Storage means for recording the latest still image data acquired by the imaging means and one or a plurality of difference image data created by the difference data creating means;
Based on the still image data and the difference image data recorded in the storage means, restoring means for restoring still image data that has already been erased when the release button is pressed;
A digital camera comprising: The digital camera according to claim 1, wherein
The difference data creation means erases the second still image data recorded in the storage means according to creation of difference image data. The digital camera according to claim 1 or 2,
A still picture data restored by the restoration means is recorded on a storage medium. The digital camera according to any one of claims 1 to 3,
Detecting means for detecting that the photographing range of the digital camera has changed to a predetermined threshold value or more;
A deletion unit that deletes the difference image data in accordance with a detection result of the detection unit;
A digital camera further comprising: The digital camera according to claim 4, wherein
The digital camera according to claim 1, wherein the deletion unit deletes the difference image data in response to a predetermined number of detections by the detection unit. In the digital camera according to any one of claims 1 to 5,
Acquisition of still image data by the imaging means starts in response to activation of a display means for displaying still image data acquired by the imaging means. A receiving step of receiving still image data at predetermined intervals from an imaging device that shoots a subject and creates still image data;
Based on the first still image data received in the receiving step and the second still image data received immediately before the first still image data in the receiving step, data of corresponding pixels is obtained. When they are different, a difference data creation step for creating difference image data;
A storage step of recording the latest still image data received in the reception step and one or a plurality of difference image data created in the difference data creation step;
Based on the still image data and the difference image data recorded in the storage step, a restoration step of restoring still image data that has already been erased;
An image processing program for causing a computer to execute.

Images