JP5561044B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program suitable for a digital camera or the like having a continuous shooting function.

  Conventionally, each image taken during continuous shooting is sequentially compressed into a file and recorded on a medium, so that the thinned image can be displayed on the display unit so that the image can be recorded while checking the image quality. Technology. (For example, Patent Document 1)

JP 2007-074475 A

  However, in the case of the technique of the above-mentioned patent document, since all displayed images are to be recorded, there is no means to take for each image even if the photographer confirms the reflection of each image during continuous shooting. Even if it is determined that there is no need to do so, there is a problem that all the data is recorded.

  The present invention has been made in view of the above circumstances, and the purpose of the present invention is to effectively utilize the standby time at the time of image recording immediately after continuous shooting, and the intention of the photographer with respect to the image data to be recorded. It is to be able to perform an operation reflecting the above.

  According to the first aspect of the present invention, there is provided a photographing means for photographing a plurality of temporally continuous images, a holding means for temporarily holding a plurality of image data photographed by the photographing means, and a temporary holding means by the holding means. First image processing means for irreversibly encoding a plurality of image data held in the memory, a period of processing by the first image processing means, and each of the plurality of image data held by the holding means A selection receiving unit that receives a selection of one of the reduced images displayed in the list, and image data corresponding to the reduced image that has been selected by the selection receiving unit. Second image processing means for converting into or lossless compression encoding, and recording means for recording the image data processed by the first image processing means and the second image processing means Characterized by comprising.

  According to a second aspect of the present invention, in the first aspect of the invention, the second aspect further includes an image evaluation unit that performs image evaluation on a plurality of image data temporarily held by the holding unit. The image processing means is characterized in that when the selection is not accepted by the selection accepting means, the image data based on the evaluation result by the image evaluating means is subjected to non-compression coding or lossless compression coding.

  According to a third aspect of the present invention, a photographing step for causing the imaging unit to photograph a plurality of temporally continuous images, and a holding unit for temporarily holding a plurality of image data photographed in the photographing step in the holding unit. A first image processing step for irreversibly encoding a plurality of image data temporarily held in the holding step, and a processing period in the first image processing step. Each of a plurality of stored image data is reduced and displayed in a list on the display unit, and a selection receiving step for receiving selection of any one of these reduced images, and a reduced image for which selection has been received in the selection receiving step Are processed in a second image processing step that performs non-compression coding or lossless compression coding, and the first image processing step and the second image processing step. Characterized in that it comprises a recording step of recording in the recording unit the image data.

  Invention of Claim 4 is the program which the computer with which the apparatus provided with the imaging | photography part runs, the imaging | photography control means which makes the said imaging part image | photograph several time continuous images with respect to an imaging part, A holding unit for temporarily holding a plurality of image data shot by the shooting control unit in a holding unit; a first image processing for irreversibly encoding the plurality of image data temporarily held by the holding unit; Means, during the period of processing by the first image processing means, each of the plurality of image data held in the holding unit is reduced and displayed in a list on the display unit, and selection of any one of these reduced images is accepted A second image processing unit that performs non-compression encoding or lossless encoding on the image data corresponding to the reduced image received by the selection receiving unit; And characterized in that to function as a recording control means for recording the recording unit image data processed by the first image processing means and second image processing means.

  According to the present invention, it is possible to effectively use the standby time at the time of image recording immediately after continuous shooting, and to perform an operation reflecting the photographer's intention on the recorded image data.

It is a block diagram which shows the structure of the functional circuit of the digital camera which concerns on one Embodiment of this invention. 4 is a flowchart showing a series of processing contents from shooting to recording in a continuous shooting mode according to the embodiment. FIG. 3 is a flowchart of a subroutine showing detailed contents of an uncompressed image selection receiving process of FIG. 2 according to the same embodiment. It is a figure which illustrates the photographic subject displayed on a display part during the stand-by of continuous shooting concerning the embodiment. It is a figure which shows the concept of the Bayer data hold | maintained at the buffer memory by the continuous shooting which concerns on the same embodiment. FIG. 6A shows a display example of thumbnail images displayed on the display unit, and FIG. 6B shows a thumbnail image display screen for four focus frame portions. FIG. It is a figure which shows the example of a display which concerns on the embodiment, and shows the display screen of the image based on one Bayer data selected as a non-compressed recording candidate.

  Hereinafter, an embodiment in which the present invention is applied to a digital camera having a continuous shooting function will be described with reference to the drawings.

  FIG. 1 shows a circuit configuration of a digital camera 10 according to the present embodiment. As shown in the figure, a solid-state imaging device (IS) composed of, for example, a CCD (Charge Coupled Device) or a CMOS image sensor is provided via an optical lens unit 11 disposed on the front surface of the camera housing. ) The light image of the subject is incident on the image pickup surface 12 and formed.

  In a monitor state, also referred to as through image display or live view image display, an image signal obtained by imaging with the solid-state imaging device 12 is sent to the AGC / A / D conversion unit 13 to perform correlation square sampling, automatic gain adjustment, A / D conversion processing is executed and digitized. The digital image data is held in the DRAM 14 which is a buffer memory via the system bus SB.

  The image processing unit 15 appropriately performs necessary image processing on the image data held in the DRAM 14. Specifically, the image processing unit 15 performs matrix calculation, pixel interpolation processing, and gamma correction on image data (hereinafter referred to as “Bayer data”) corresponding to the configuration of a Bayer array color filter provided in the solid-state imaging device 12. By performing demosaic processing such as processing, the Bayer data, which is RAW (raw) data, is subjected to “development” and converted into luminance / color difference (YUV) image data.

  The image processing unit 15 creates image data in which the number of pixels and gradation bits are greatly reduced for display from the image data, and sends the image data to the display controller 16. The display controller 16 drives the display unit 17 based on the received image data, and displays the content captured by the solid-state image sensor 12 as a through image.

  Note that the image processing unit 15 has a face recognition function, recognizes a human face portion from image data, and extracts a region including the portion. When the face recognition function is executed, if a human face is recognized in the image data, a frame can be added to an area including the face portion and displayed.

  Similarly to the optical lens unit 11, a pair of microphones 18L and 18R are provided on the front surface of the camera casing, and the sound in the subject direction is input in stereo. The microphones 18 </ b> L and 18 </ b> R convert the input sound into electrical signals and output them to the sound processing unit 19.

  The sound processing unit 19 converts the sound signals input from the microphones 18L and 18R into digital data when recording a single sound, capturing a still image with sound, and capturing a moving image. Further, the sound processing unit 19 detects the sound pressure level of the digitized sound data, and compresses the sound data in a predetermined data file format, for example, AAC (moving picture experts group-4 Advanced Audio Coding) format. An audio data file is created and sent to a recording medium to be described later.

  In addition, the sound processing unit 19 includes a sound source circuit such as a PCM sound source, uncompresses and converts the sound data file sent during sound reproduction into an analog signal, and a speaker 20 provided on the rear side of the housing of the digital camera 10. To sound a loud sound.

  The CPU 21 performs overall control of the above circuit. The CPU 21 is directly connected to the main memory 22 and the program memory 23. The main memory 22 is composed of, for example, an SRAM and functions as a work memory. The program memory 23 is composed of an electrically rewritable non-volatile memory such as a flash memory, for example, and fixedly stores an operation program and data including control at the time of continuous shooting described later.

  The CPU 21 reads the necessary program, data, etc. from the program memory 23 and executes the control operation of the entire digital camera 10 while temporarily developing and storing it in the main memory 22.

  Further, the CPU 21 executes control operations in response to various key operation signals input directly from the key operation unit 24. The key operation unit 24 includes, for example, a power key, shutter key, zoom up / down key, shooting mode key, playback mode key, menu key, cursor (“↑” “→” “↓” “←”) key, set key, Includes display keys and the like.

  In addition to the AGC / A / D conversion unit 13, DRAM 14, image processing unit 15, display unit 17, and audio processing unit 19, the CPU 21 further includes a lens driving unit 25, a flash driving unit 26, an image via the system bus SB. The sensor (IS) drive unit 27, the touch input processing unit 28, and the memory card controller 29 are connected.

  The lens driving unit 25 controls the rotation of the lens DC motor (M) 30 in response to a control signal from the CPU 21, and a part of a plurality of lens groups constituting the optical lens unit 11, specifically zoom. The positions of the lens and the focus lens are individually moved along the optical axis direction.

  The flash drive unit 26 receives a control signal from the CPU 21 during still image shooting, and drives the flash unit 31 composed of a plurality of white high-intensity LEDs in synchronization with the shooting timing.

  The image sensor driving unit 27 performs scanning driving of the solid-state imaging device 12 in accordance with imaging conditions set at that time.

  The image processing unit 15 performs demosaic processing on the image data sent from the AGC / A / D conversion unit 13 and held in the DRAM 14 at the time of image capturing associated with the operation of the shutter key of the operation unit 24, and further receives predetermined data. If the file format is, for example, JPEG (Joint Photographic Experts Group), data compression processing such as DCT (Discrete Cosine Transform) or Huffman coding is performed to create an image data file with a greatly reduced data amount. The created image data file is recorded on the memory card 32 via the system bus SB and the memory card controller 29.

Further, the image processing unit 15 receives the image data read from the memory card 32 via the memory card controller 29 in the reproduction mode via the system bus SB, holds it in the DRAM 14, and then holds it in the DRAM 14. Image data of the original size is obtained by decompression processing that uncompresses the image data in a procedure reverse to that at the time of recording.
The touch input processing unit 28 receives an input from the touch input unit 33 made of a transparent film integrally formed on the display unit 17, generates coordinate data corresponding to the touch position, and sends the coordinate data to the CPU 21. To do.

  The memory card controller 29 is connected to the memory card 32 via the card connector 34. The memory card 32 is a memory for recording image data and the like, which is detachably attached to the digital camera 10 and serves as a recording medium for the digital camera 10, and has a nonvolatile memory that can be electrically rewritten in units of blocks. A flash memory and a driving circuit thereof are provided.

Next, the operation of the above embodiment will be described.
The following operations are stored in the program memory 23 when the continuous shooting mode is set and the shutter key is pressed for an arbitrary time to shoot a plurality of still images that are temporally continuous. The operation program and data are read out, developed in the main memory 22 and stored, and then executed.

  For example, when the digital camera 10 is upgraded, the operation program stored in the program memory 23 is stored in the program memory 23 when the digital camera 10 is shipped from the factory. To download and store new operation programs, data, and the like from the outside by connecting to.

2 and 3 show a series of processing contents from shooting to recording in the continuous shooting mode. At the beginning of the process, the CPU 21 repeatedly determines whether or not the shutter key of the key operation unit 24 has been operated (step S101). Thus, the CPU 21 waits for the shutter key of the key operation unit 24 to be operated.
FIG. 4 is a diagram illustrating a subject displayed on the display unit 17 while waiting for the operation of the shutter key. As shown in the figure, the display unit 17 displays, for example, a symbol CP, which is a combination of a plurality of rectangular frames, indicating that the continuous shooting mode is set at the upper center of the screen.

  At the same time, the CPU 21 causes the image processing unit 15 to extract the face portion in the image with respect to the image data held in the DRAM 14 and displays the focus frame FF that surrounds the extracted face portion superimposed on the image. This is displayed on the part 17.

  When the shutter key is operated, the CPU 21 determines that in step S101, immediately executes shooting with the solid-state imaging device 12 (step S102), and holds the Bayer data acquired by shooting in the DRAM 14 which is a buffer memory. (Step S103).

Then, it is determined whether the operation of the shutter key is stopped and turned off (step S104).
If it is determined that the shutter key is still operated and not turned off, the Bayer data held in the DRAM 14 by continuous shooting is a prescribed number, for example, the continuous shooting speed is 6 [sheets / second]. If it is possible to hold the maximum of 10 [seconds], it is determined whether or not the maximum of 60 sheets has been reached (step S105).

  The number of images that can be continuously shot may be determined each time depending on, for example, the image size of the captured image set at that time, the number of gradation bits per pixel, and the capacity of the DRAM 14, and the control is simplified. Therefore, it is possible to uniformly determine the maximum image size and the maximum number of gradation bits per pixel regardless of the image size or the like so that they can be held reliably.

  If the prescribed number of Bayer data is not yet held in the DRAM 14, it is determined in step S105, and the process returns to step S102 again.

  Thus, the processes of steps S102 to S105 are repeatedly executed until the operation of the shutter key is stopped and turned off or the number of Bayer data acquired by photographing reaches the specified number.

  FIG. 5 shows the concept of Bayer data that is temporally continuous and held in the DRAM 14. Here, four Bayer data BD1 to BD4 that are temporally continuous are illustrated, and these data are sequentially held in the DRAM 14.

  If it is determined in step S104 that the operation of the shutter key is stopped and turned off, or if it is determined that the number of Bayer data acquired in step S105 has reached the specified number, the shooting operation is performed at that time. The data is stopped and read out from the plurality of Bayer data held in the DRAM 14 by the image processing unit 15 in order from the earliest acquisition timing, and data such as DCT or Huffman coding is used in a predetermined lossy compression format, for example, JPEG. The compression encoding process and recording of the compressed data file to the memory card 32 are started (step S106).

  At the same time, the CPU 21 creates a thumbnail image for display by greatly reducing the image size by pixel thinning for each Bayer data held in the DRAM 14 (step S107).

  Then, a list display is performed on the display unit 17 using the created thumbnail images, and selection of uncompressed recording candidate images is accepted (step S108).

  FIG. 3 is a subroutine showing the detailed processing contents for accepting the selection of the uncompressed recording candidate image in step S108. In the figure, initially, a list screen with thumbnail images is displayed on the display unit 17 (step S301).

  FIG. 6A illustrates a thumbnail image list screen displayed on the display unit 17 at this time. Here, the screen of the display unit 17 is divided into a total of nine (3 × 3), and a total of nine thumbnail images are collectively displayed.

  At this time, the display unit 17 also displays a guide message of the character string “Writing card” at the center of the upper part of the screen so that a series of image data held by shooting is irreversibly compressed and encoded. And informing that the process of writing to the memory card 32 is being executed.

  When the number of image data held in the DRAM 14 is larger than the number of thumbnail images displayed as a list on the display unit 17 and not all can be displayed at once, thumbnail images are displayed as a list from the top in accordance with the order of shooting, for example. Further, the fact that there is a thumbnail image that cannot be displayed is expressed by displaying a right arrow on the right side of the last thumbnail image to be displayed.

  If there is a touch input from the cursor input (“→” key) of the key operation unit 24 or the touch input processing unit 28 according to the display position on the display unit 17 for this display, the operation is performed. Scroll the contents to be displayed in a list accordingly.

  Further, from the display state of the thumbnail image list screen shown in FIG. 6A, for example, by operating the zoom-up key of the key operation unit 24, only the thumbnail corresponding to the portion in the focus frame FF of each image is displayed. Images may be created, displayed as a list screen, and displayed as shown in FIG.

  In FIG. 6B, as shown in the figure, thumbnail images of four focus frame portions divided into a total of 4 pieces of 2 × 2 in total are smaller than the list image shown in FIG. 6A. Displayed all at once.

  In this way, by displaying only the focus frame portion on the larger list screen, it is possible to grasp the contents of photographing in a state where the face portion in the image by the face recognition process is enlarged. Therefore, the photographer of the digital camera 10 can more accurately determine how each image is captured.

  It is determined whether any image is selected in a state where the list screen as shown in FIG. 6A or 6B is displayed (step S302).

  For example, a combination of a cursor (“↑”, “→”, “↓”, “←”) key and a set key of the key operation unit 24 is performed, or touch input processing according to the image position on the display unit 17 is performed. Judgment is made by touch input at the unit 28.

  If the above selection operation is not performed, the data compression encoding process for all the Bayer data held in the DRAM 14 and the recording of the compressed data file to the memory card 32 are started in the next step S106. (Step S303).

  If it is determined that the process has not been completed, the process returns to step S301.

  By repeatedly executing the processes of steps S301 to S303 in this way, one of the displayed items is selected while displaying the list screen, or the data compression encoding process of all the Bayer data held in the DRAM 14 And waits for the recording to the memory card 32 to end.

  When the process of steps S301 to S303 is repeatedly executed and an operation for selecting one of the images from the state in which the list screen is displayed is performed, the CPU 21 determines that in step S302, and selects the selected thumbnail image. The Bayer data held in the corresponding DRAM 14 is selected and set as an uncompressed recording candidate (step S304).

  In accordance with this selection setting, the corresponding thumbnail images in the list screen displayed on the display unit 17 are displayed with the surrounding and frame colors changed (step S305).

  Thereafter, similarly to step S303, it is determined whether the data compression encoding process for all the Bayer data held in the DRAM 14 and the recording of the compressed data file to the memory card 32 have been completed (step S306).

  If it is determined that the process has not been completed, the process returns to step S301. In this way, even after selecting an uncompressed recording candidate image, the process returns to the process from step S301 again so that a plurality of images can be selected as uncompressed recording candidates.

  When it is determined in step S306 that the data compression encoding process for all the Bayer data held in the DRAM 14 and the recording of the compressed data file to the memory card 32 have been completed, the subroutine of FIG. End and return to the main routine of FIG.

  When the processes of steps S301 to S303 are repeatedly executed and the list screen is displayed, the data compression encoding process of all the Bayer data stored in the DRAM 14 and the recording to the memory card 32 are completed. This is determined in step S303, and it is then determined whether there is at least one image applied as an uncompressed recording candidate in the Bayer data held in the DRAM 14 (step S307).

  If it is determined that at least one piece of Bayer data has been selected as an uncompressed recording candidate, the subroutine in FIG. 3 is terminated and the process returns to the main routine in FIG.

  If it is determined in step S307 that no Bayer data has been selected as an uncompressed recording candidate so far, all the Bayer data stored in the DRAM 14 is subjected to face recognition processing by the image processing unit 15. Image evaluation is performed, and an evaluation value is calculated for each Bayer data (step S308).

Here, the image evaluation value is evaluated based on at least one of the position, area, blur amount, and facial expression of the face portion of the subject, for example.
As the position of the face portion of the subject, the higher the evaluation is in the center of the image.
The larger the area of the subject's face, the higher the evaluation.
The smaller the amount of blurring of the subject's face, the higher the evaluation.
As the expression of the subject, the higher the smile level (the lower end of both eyes is lowered and the both ends of the lips are raised), the higher the evaluation is.
The technical content of the face recognition process is a well-known technique that is standardly installed in most models of digital cameras that have been generally sold recently, and a description thereof will be omitted.

  By calculating the image evaluation value using the face recognition processing technique in this way, it is possible to accurately evaluate the appearance of the person who is highly likely to be selected as the subject, and to obtain a more “good” person image. Can be selected automatically.

When the image evaluation values of all the Bayer data held in the DRAM 14 are calculated, the highest Bayer data among the calculated evaluation values is then selected and set as an uncompressed recording candidate (step S309). The subroutine is terminated and the process returns to the main routine of FIG.
FIG. 7 illustrates one piece of Bayer data BDA selected as an uncompressed recording candidate in the subroutine of FIG.

  In the main routine of FIG. 2, after selecting the uncompressed recording candidate in step S108, the newly selected Bayer data is encoded as it is without compression, and the obtained data file is recorded in the memory card 32 (step S108). S109).

Thereafter, the contents of the DRAM 14 are erased all at once (step S110), and then the process returns to the standby processing from step S101 again to prepare for the next continuous shooting. Effectively use the waiting time for image recording immediately after, reflects the photographer's intention to the image data to be recorded, and also records uncompressed image data files for images that seem particularly good Therefore, image data without image quality deterioration can be effectively used later.

  In the above embodiment, when the uncompressed recording candidate is not selected by the photographer, the apparatus automatically selects the image with the highest evaluation value from the evaluation values of each image as the uncompressed recording candidate. Even if there is no selection operation by the photographer, it automatically selects an image that seems to have an objectively high evaluation value, and later makes image data without image quality degradation effective. Can be used.

  Furthermore, in the above embodiment, the evaluation of each image has been described as being calculated by face recognition processing. However, the present invention is not limited to this. For example, the entire image or a predetermined range of contrast values in the image (= after orthogonal transformation) The image evaluation value can also be obtained by image processing based on the amount of the high-frequency component), the composition, the luminance balance value calculated from the luminance histogram, and the like.

  In the above embodiment, the uncompressed RAW data is stored for the selected image. However, the present invention is not limited to this, and is in a format such as PNG (Portable Network Graphics) or GIF (Graphics Interchange Format). In principle, image data that has been subjected to lossless compression processing, which is also called lossless compression, may be recorded. Although the amount of data cannot be reduced as much as irreversible compression processing, image data that has been compressed without degrading image quality. Can be recorded.

  Furthermore, although the said embodiment demonstrated the case where it applied to a digital camera, this invention is not restricted to this, As long as it is an electronic device which has a camera function, a mobile telephone terminal and PDA (Personal Digital Assistants) will be used. : Personal information portable terminal), portable media player, electronic book, mobile computer, and other devices.

  In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention in the implementation stage. Further, the functions executed in the above-described embodiments may be combined as appropriate as possible. The above-described embodiment includes various stages, and various inventions can be extracted by an appropriate combination of a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 11 ... Optical lens unit, 12 ... Solid-state image sensor (IS), 13 ... AGC / A / D conversion part, 14 ... DRAM, 15 ... Image processing part, 16 ... Display controller, 17 ... Display part, 18L, 18R ... microphone, 19 ... audio processing unit, 20 ... speaker, 21 ... CPU, 22 ... main memory, 23 ... program memory, 24 ... key operation unit, 25 ... lens drive unit, 26 ... flash drive unit, 27 ... Image sensor driving unit, 28 ... touch input processing unit, 29 ... memory card controller, 30 ... lens DC motor (M), 31 ... flash unit, 32 ... memory card, 33 ... touch input unit, 34 ... card connector, BD1 ~ BD4, BDA ... Bayer data, SB ... System bus.

Claims (4)

Photographing means for photographing a plurality of temporally continuous images;
Holding means for temporarily holding a plurality of image data photographed by the photographing means;
First image processing means for irreversibly encoding a plurality of image data temporarily held by the holding means;
During the period of processing by the first image processing means, each of the plurality of image data held by the holding means is reduced and displayed as a list, and selection of any one of the reduced images displayed in the list is accepted. A selection receiving means;
Second image processing means for performing non-compression coding or lossless compression coding of image data corresponding to the reduced image whose selection is accepted by the selection receiving means;
An image pickup apparatus comprising: a recording unit that records image data processed by the first image processing unit and the second image processing unit. Image evaluation means for evaluating images for a plurality of image data temporarily held by the holding means;
The second image processing means performs non-compression encoding or lossless encoding on image data based on the evaluation result by the image evaluation means when the selection is not accepted by the selection acceptance means. The imaging device according to claim 1. An imaging step for causing the imaging unit to capture a plurality of temporally continuous images;
A holding step for temporarily holding a plurality of image data shot in the shooting step in a holding unit;
A first image processing step for irreversibly encoding a plurality of image data temporarily held in the holding step;
Selection for accepting selection of any one of the reduced images by reducing each of the plurality of image data held in the holding unit and displaying the list on the display unit during the processing period in the first image processing step A reception step;
A second image processing step of performing non-compression coding or lossless compression coding of image data corresponding to the reduced image whose selection is accepted in the selection acceptance step;
An imaging method comprising: a recording step of causing the recording unit to record the image data processed in the first image processing step and the second image processing step. A program executed by a computer built in a device having a photographing unit,
The computer
An imaging control unit that causes the imaging unit to capture a plurality of temporally continuous images,
Holding means for temporarily holding a plurality of image data shot by the shooting control means in a holding unit;
First image processing means for irreversibly encoding a plurality of image data temporarily held by the holding means;
During the processing period of the first image processing means, a selection reception for reducing each of a plurality of image data held in the holding unit and displaying a list on the display unit and accepting selection of any one of these reduced images means,
Second image processing means for performing non-compression coding or lossless compression coding of image data corresponding to a reduced image whose selection is accepted by the selection acceptance means;
A program for causing a recording unit to record image data processed by the first image processing means and the second image processing means.

Images