JP4329473B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program for processing image data.

  Conventionally, there has been an image photographing device such as a digital movie camera for photographing a digital moving image. In recent years, digital still cameras that shoot digital still images have been sold that have a moving image shooting function.

  In moving image shooting, together with the captured moving image data, a thumbnail as a reduced image for grasping the contents of the moving image data at a glance is stored in a recording medium such as a memory card. As the thumbnail, for example, the first frame after the start of shooting of moving image data, the first frame after the elapse of a predetermined time since the start of shooting of moving image data, or the frame selected and input by the user is used.

  In addition to a configuration in which one thumbnail corresponds to one moving image data, a configuration in which a plurality of thumbnails correspond to one moving image data is considered. For example, in order to easily find the start of an arbitrary shooting time during playback of moving image data after shooting, thumbnail image data is created from a frame at that moment every time a predetermined time elapses during shooting of the moving image data. Appended to the data. Thus, in the configuration in which thumbnails are automatically assigned, the created thumbnails may not be appropriate.

In addition, when an announcer has few changes in the image itself, such as a video that reads a plurality of news, and subtitles in the image change one after another, a representative image extraction device that uses a frame with subtitles in moving image data as a representative image (For example, refer to Patent Document 1).
JP-A-8-212231

  However, the representative image extraction device described in Patent Document 1 cannot be applied to moving image shooting without captions. Further, in the conventional thumbnail automatic generation method, for example, there is a possibility that thumbnails that are pure white, pure black, overexposed, or underexposed are extracted regarding the luminance value of a moving image. In addition, in the conventional thumbnail selection method based on the selection input by the user, it is necessary to check images that are clearly inappropriate as thumbnails, and there is a possibility that the operation input operation of the user becomes troublesome.

  An object of the present invention is to automatically select an appropriate thumbnail corresponding to moving image data.

In order to solve the above-described problems, the invention described in claim 1 is directed to a processing load on image data of each moving image frame obtained by sequentially capturing images with the imaging unit during capturing of moving image data. A thumbnail extraction unit that sequentially executes a first image analysis to select a candidate frame for a thumbnail , and image data of each moving image frame that has already been recorded after shooting of the moving image data is completed. The second image analysis, which has a larger processing load than the first image analysis, is executed, and the final thumbnail frame is determined from the plurality of frames selected as thumbnail candidates by the thumbnail extraction unit. a thumbnail determination unit for recording a thumbnail corresponding to a frame determined by the thumbnail determining unit in association with the moving image data A thumbnail storage unit that, characterized in that it comprises a.

The invention described in claim 2 further includes a shooting control unit that performs a third image analysis for determining shooting and recording conditions, and the thumbnail extraction unit uses the third image analysis. The first image analysis is performed.

According to a third aspect of the present invention, the photographing control unit further determines at least one photographing condition among photographing and recording conditions including a shutter speed, an aperture value, auto focus, automatic exposure, and auto white balance. Therefore, the third image analysis is performed.

The invention described in claim 4 is further characterized in that the first image analysis is an analysis of a luminance value distribution of the image data.

The invention described in claim 5 is further characterized in that the second image analysis is an analysis of a hue component distribution of the image data.

According to a sixth aspect of the present invention, the thumbnail extraction unit records a thumbnail image of a frame selected in real time during moving image shooting, and the thumbnail determination unit records the thumbnail image recorded by the thumbnail extraction unit. Among them, a thumbnail image recorded corresponding to a frame that has not been determined as a final thumbnail is deleted.

The invention according to claim 7 is characterized in that the thumbnail determination unit further includes a process of determining a frame to be a final thumbnail from a plurality of frames by a user's selection input.

According to an eighth aspect of the present invention, in the first image analysis, a processing load is less than or equal to a predetermined value for image data of each moving image frame obtained by sequentially capturing images by the imaging unit during capturing of moving image data. A thumbnail extraction function that executes frames in real time and sequentially selects thumbnail candidate frames;
After the shooting of the moving image data is completed, a second image analysis that has a larger processing load than the first image analysis is performed on the image data of each moving image frame that has already been recorded, and the thumbnail extraction function A thumbnail determination function for determining a frame as a final thumbnail from a plurality of frames selected as thumbnail candidates, and a thumbnail corresponding to the frame determined by the thumbnail determination function in association with the moving image data It is an image processing program for realizing a thumbnail storage function for recording.

According to the present invention, during shooting of moving image data, image analysis with a low processing load is executed in real time to select frames as thumbnail candidates sequentially, and after the shooting of moving image data is completed, the image data is already recorded. For each video frame, image analysis with a heavy processing load is performed and the final thumbnail is determined from the selected thumbnail candidates. Therefore, a thumbnail with an appropriate image quality corresponding to the moving image data is selected. Both accuracy and speed can be achieved .

  Embodiments according to the present invention will be described below with reference to the accompanying drawings.

  First, the apparatus configuration of the present embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 shows an appearance of a digital camera 1 according to the present embodiment, where (a) mainly shows a front configuration, and (b) mainly shows a rear configuration. FIG. 2 shows the internal configuration of the digital camera 1.

  As shown in FIG. 1, a digital camera 1 as an image processing apparatus has a photographing lens 2, a self-timer lamp 3, an optical viewfinder window 4, and a strobe light emitting unit 5 disposed on the front surface of a substantially rectangular body. Are provided with a power key 6 and a shutter key 7. The digital camera 1 has a moving image shooting function in addition to the still image shooting function.

The photographic lens 2 is a single focal lens, and the focal position and the like can be optically changed by a driving unit of a lens optical system 22 described later.
The power key 6 is a key for turning on / off the power each time a pressing operation is performed. The shutter key 7 can be used as a key for instructing release in the shooting mode, and for setting / execution in menu selection or the like. It shall function. In the moving image shooting mode, the shutter key 7 instructs to start shooting when pressed, and instructs the end of shooting when the pressing is stopped (released).

Further, a mode switch (SW) 8, a menu key 9, a cross key 10, an optical finder 11, a strobe charge lamp 12, and a display unit 13 are arranged on the back of the digital camera 1.
The mode switch 8 is constituted by a slide switch, for example, and switches between the photographing mode “R” and the reproduction mode “P”. The shooting mode is divided into a still image shooting mode and a moving image shooting mode, and the still image shooting mode and the moving image shooting mode are selected and input by user input from the menu key 9 and the cross key 10.

The menu key 9 is operated when various menus are selected.
The display unit 13 is composed of a color liquid crystal panel with a backlight, and performs monitor display as an electronic viewfinder in the photographing mode, while reproducing and displaying the selected image in the reproduction mode.

  Although not shown, a memory card slot with a lid is provided on the lower surface of the body, and a memory card as a recording medium of the digital camera 1 is detachably attached. Although not shown, a connector slot for connecting a cable for communicating with an external device is also provided.

Next, the electronic circuit configuration of the digital camera 1 will be described with reference to FIG.
In the monitoring state in the photographing mode, the CCD 23 which is an image pickup element of the lens optical system 22 including the photographing lens 2 whose focal length, aperture position, and the like are moved by driving the motor (M) 21 is a timing generator (TG). 24) A scanning drive is performed by the vertical driver 25, and a photoelectric conversion output corresponding to a light image formed at fixed intervals is output for one screen.

  The photoelectric conversion output is appropriately gain-adjusted for each primary color component of RGB in the state of an analog value signal, then sampled and held by a sample hold circuit (S / H) 26, and digital data by an A / D converter 27 The color process circuit 28 performs color process processing including pixel interpolation processing and γ correction processing to generate a luminance signal Y and color difference signals Cb, Cr, and a DMA (Direct Memory Access) controller 29. Is output.

  The DMA controller 29 once uses the luminance signal Y and the color difference signals Cb and Cr output from the color process circuit 28 by using the composite synchronization signal, the memory write enable signal and the clock signal from the color process circuit 28 once. The data is written in the buffer, and DMA transfer is performed to the DRAM 31 used as a buffer memory via the DRAM interface (I / F) 30.

The control unit 32 includes a CPU, a ROM that fixedly stores an operation program executed by the CPU including color enhancement processing described later, a RAM that is used as a work memory, and the like. After the DMA transfer of the luminance and color difference signals to the DRAM 31, the luminance and color difference signals are read from the DRAM 31 via the DRAM interface 30 and written to the VRAM 34 via the VRAM controller 33.
The digital video encoder 35 periodically reads out the luminance and color difference signals from the VRAM 34 via the VRAM controller 33, generates a video signal based on these data, and outputs the video signal to the display unit 13.

As described above, the display unit 13 functions as a monitor display unit (electronic finder) in the shooting mode. By performing display based on the video signal from the digital video encoder 35, the display unit 13 receives from the VRAM controller 33 at that time. The through image based on the included image information is displayed in real time.
As described above, the shutter key that configures the key input unit 36 as the input unit at a timing when still image shooting or moving image shooting is desired in a state where the image at that time is displayed in real time as the monitor image on the display unit 13. When 7 is operated, a trigger signal is generated.

In the still image shooting mode, the controller 32 immediately after the DMA transfer of the luminance and color difference signals for one screen captured from the CCD 23 to the DRAM 31 at that time in response to the trigger signal, to the DRAM 31 from the CCD 23. Is stopped, and a transition is made to the record storage state.
In this record storage state, the control unit 32 outputs the luminance and color difference signals for one frame written in the DRAM 31 through the DRAM interface 30 for each of Y, Cb, and Cr components of 8 pixels × 8 pixels horizontally. Are read in units called basic blocks and written to the compression circuit 37. The compression / decompression circuit 37 uses the JPEG (Joint Photograph coding Experts Group) method or the like to perform ADCT (Adaptive Discrete Cosine Transform), entropy. Data compression is performed by processing such as Huffman encoding, which is an encoding method.

The obtained code data is read out from the compression circuit 37 as a data file of one image and stored in a flash memory 38 which is a non-volatile memory enclosed in a memory card that is detachably mounted as a recording medium of the digital camera 1. Write. Further, the flash memory 38 may be separately incorporated in the digital camera 1.
The controller 32 re-enables the path from the CCD 23 to the DRAM 31 when the luminance and color difference signals for one frame are compressed and all the compressed data is written to the flash memory 38.

  In the moving image shooting mode, the control unit 32 writes the luminance and color difference signals for one frame from the CCD 23 to the DRAM 31 from the CCD 23 to the DRAM 31 at a predetermined time interval from when the shutter key 7 is pressed to when it is released. The luminance and color difference signals are read out by the circuit 37, compressed by a method such as MPEG (Motion Picture Experts Group) 4, and stored in the flash memory 38. A moving image shooting instruction button is provided separately from the shutter key 7, and moving image shooting is started when this moving image shooting instruction button is pressed, and moving image shooting is ended when the moving image shooting instruction button is pressed again during moving image shooting. It is good also as such a structure.

The key input unit 36 includes a power key 6, a shutter key 7, a mode switch 8, a menu key 9, and a cross key 10, and signals associated with these key operations are sent directly to the control unit 32.
In the playback mode, the control unit 32 selectively reads out still image or moving image data recorded in the flash memory 38 and compresses the data by the compression / decompression circuit 37 in a procedure completely opposite to the procedure of data compression in the image shooting mode. The decompressed image data is decompressed, the decompressed image data is expanded and stored in the VRAM 34 via the VRAM controller 33, and then periodically read out from the VRAM 34 to generate a video signal based on the image data. Then, the display unit 13 reproduces and outputs.

  In still image shooting, the shutter key 7 of the key input unit 36 operates with a two-stage stroke, and an AE (automatic exposure) process or the like is performed in a first-stage operation state generally expressed as “half-press”. It is assumed that preparations for shooting such as AF (autofocus) processing are performed, and shooting is executed in a second-stage operation state in which a stronger pressing operation is generally expressed as “full press”.

  In addition, it is assumed that a moving image shooting program is stored in the internal ROM of the control unit 32. The control unit 32 has functions as a thumbnail creation unit, a thumbnail extraction unit, and a thumbnail determination unit.

Note that the above configuration is merely a typical configuration example of a digital camera, and it is needless to say that the configuration may have other configurations. For example, a digital camera may not be thin.
The image photographing device is not limited to a digital camera, and may be a digital movie camera having a moving image photographing function, a mobile phone, a PHS (Personal Handyphone System), a PDA (Personal Digital Assistants), or the like.
The recording medium for recording moving image data is not limited to a flash memory and a memory card including the flash memory, but may be a hard disk, a CD-RW, a DVD-RAM, or the like. Further, in addition to the example of the recording medium such as the writable CD-RW and DVD-RAM already described, for example, the following using a blue laser such as a Blu-ray Disc (R) or an AOD (Advanced Optical Disc) It is possible to implement the present invention using various large-capacity storage media to be developed in the future, such as next-generation optical disc storage media, HD-DVD9 using a red laser, and Blue Laser DVD using a blue-violet laser.

  Next, the operation of the digital camera 1 will be described with reference to FIGS. FIG. 3 shows the moving image shooting process. FIG. 4 shows real-time thumbnail storage processing during moving image shooting processing. FIG. 5 shows the first real-time thumbnail storage process. FIG. 6 shows the second real-time thumbnail storage process. FIG. 7 shows a distribution example of the area ratio in the image with respect to the luminance, (a) shows a distribution example where the area ratio is large at high luminance, and (b) shows a distribution example where the area ratio is large at low luminance. FIG. 8 shows a distribution example of the luminance level of the image with respect to the hue component, (a) shows a distribution example of the low luminance level regardless of the hue component, and (b) shows a distribution example of the luminance level having variations.

  With reference to FIG. 3, the moving image shooting process executed in the digital camera 1 will be described. The moving image shooting process is a process of shooting moving image data and selecting an appropriate thumbnail. In the digital camera 1, triggered by the user (photographer) being set to the moving image shooting mode by the key input unit 36 (mode key 8, menu key 9, cross key 10, etc.) and the execution instruction of moving image shooting processing being input. The control unit 32 reads out the moving image shooting program stored in the internal ROM or the like and appropriately expands it in the work area of the internal RAM to execute the moving image shooting process. Hereinafter, the subject of each step is the control unit 32 unless otherwise specified.

  First, it is determined whether or not moving image shooting is started by the user pressing the shutter key 7 (step S1). When the moving image shooting has not been started (step S1; NO), the process returns to step S1.

  When moving image shooting is started (step S1; YES), one frame (one frame) of the current moment is shot, and one frame of image data is generated (step S2). Then, the image data of one frame taken in step S2 is analyzed in real time (step S3). Here, the real-time image analysis is an image analysis with a low processing load that can be executed in real time in parallel with the shooting of a moving image. For example, AF, AE, and white balance of the liquid crystal display of the display unit 13 are set. It is image analysis for following. These image analyzes are usually performed in real time in parallel with video recording even when it is not necessary to generate thumbnails, so thumbnails can be obtained using the results of these image analysis. By generating / selecting, there is an advantage that the processing efficiency can be improved.

  Based on the analysis result of the real-time image analysis in step S2, the log file stored in the DRAM 31 in the digital camera 1 is updated (step S4). The log file is a file that stores analysis results of real-time image analysis of each frame for use in parameter calculation processing after shooting (step S9). Then, it is determined whether or not it is the timing for thumbnail acquisition (step S5). Whether or not it is the timing of thumbnail acquisition is determined based on the result of the image analysis in step S3, for example, based on whether or not the exposure is excessive, whether or not the exposure is insufficient, and whether or not autofocus is possible. It is determined whether a thumbnail can be generated as a candidate for a correct thumbnail. If the exposure is excessive, the exposure is insufficient, or the autofocus is not possible, it is determined that it is not the timing for obtaining the thumbnail.

  If it is the timing of thumbnail acquisition (step S5; YES), a thumbnail is created from the image data of one frame generated in step S2 (step S6). Then, real-time thumbnail storage processing as described below is executed using the created thumbnail image (step S7).

  Here, the real-time thumbnail storage process will be described with reference to FIG. First, real-time parameters are calculated (step S71). Here, the real-time parameters refer to various parameters required in the image analysis process with a low processing load that can be executed in real time, such as AF, AE, and white balance, as described above. Since the processing for obtaining these parameters is usually performed in real time in parallel with the shooting of a moving image even when it is not necessary to generate a thumbnail, in the next step (step S72) It is efficient to be able to use it in the judgment process. Then, it is determined whether or not the thumbnail created in step S6 is left as an appropriate thumbnail candidate (step S72). When leaving as a thumbnail (step S72; YES), the thumbnail image data created in step S6 is stored in the flash memory 38 (step S73). If not left as a thumbnail (step S72: NO), the thumbnail storage process is terminated without storing the thumbnail image data, and the process proceeds to the next step (step S8). An example of the real-time thumbnail storage process will be described later in detail as the first and second real-time thumbnail storage processes.

  Further, referring back to FIG. 3, it is determined whether or not the moving image shooting has been terminated by the user's opening input of the shutter key 7 (step S8). If the moving image shooting has not been completed (step S8; NO), the process proceeds to step S2, and the loop for continuously shooting the moving image is processed. On the other hand, returning to the determination process in step S5, if the current timing is not the timing for obtaining the thumbnail (step S5; NO), the process immediately proceeds to step S8 without performing the thumbnail creation and storage processes in steps S6 and S7. When the moving image shooting is finished (step S8; YES), parameters after moving image shooting are calculated (step S9). In addition, after moving image shooting, moving image data is stored in the flash memory 38.

  Here, the parameter after moving image shooting refers to a parameter that is difficult to calculate in parallel with moving image shooting due to a relatively large amount of computation. Calculation of parameters after shooting is a process after shooting, so that there is no problem even if the calculation is complicated and the processing load is large compared to the real-time image analysis processing in step S3. The calculation of parameters after shooting is performed using, for example, the log file data updated in step S4, and the evaluation of moving image data using an evaluation function or the like is calculated using the parameters. Such parameters vary depending on the performance of the control unit 32, the pixel size of the captured image, and the like, and thus it cannot be said what parameters are generally applicable. For example, the distribution of hue components (light spectrum) In addition, as an evaluation using this, for example, there are things such as whether the hue is rich or the primary color system is vivid, and if the evaluation is high, it is considered a promising candidate for a thumbnail Become.

  Then, based on the parameter calculated in step S9 among the one or more thumbnails stored in the flash memory 38, it is determined whether there is a thumbnail to be deleted that is not an appropriate thumbnail (step S10). If there is a thumbnail to be deleted (step S10; YES), the thumbnail image data to be deleted is deleted from the flash memory 38.

  Then, one or more thumbnails stored in the flash memory 38 are displayed as a list on the display unit 13 (step S12). Then, a selection input of a desired thumbnail among the thumbnails displayed in a list is received from the user via the key input unit 36 (step S13). Then, the thumbnail image data of the thumbnail selected in step S13 is stored in the flash memory 38 in association with the moving image data in the flash memory 38 (step S14), and the image photographing process is ended.

  Next, a first real-time thumbnail storage process as a specific example of the real-time thumbnail storage process shown in FIG. 4 will be described with reference to FIG. The first real-time thumbnail storage process is a process of selecting and storing a thumbnail that is not overexposed or underexposed among the thumbnails created in step S6.

  As shown in FIG. 5, first, after executing step S6, the luminance value of each pixel of the thumbnail created in step S6 is calculated (step S74). Then, the average value of the luminance values of each pixel of the calculated thumbnail is calculated (step S75). Then, it is determined whether or not the calculated average value of luminance values is within an allowable range (step S76). For example, the determination is made based on whether the average value of the luminance values is between two predetermined threshold values and is within an allowable range.

  If the average value of the luminance values is within the allowable range (step S76; YES), the thumbnail image data of the thumbnail created in step S6 is stored in the flash memory 38 (step S77), and the process proceeds to step S8. When the average value of the luminance values is not within the allowable range (step S76; NO), the process proceeds to step S8. Thereby, when the average value of the luminance values of the thumbnail images created in step S6 is not within a suitable range, that is, an image that is too dark or too bright is created as a thumbnail image. Even in such a case, such an image can be discarded without being stored as a thumbnail.

  Next, a second real-time thumbnail storage process as a specific example of the real-time thumbnail storage process shown in FIG. 4 will be described with reference to FIG. Similarly to the first real-time thumbnail storage process described above, the second real-time thumbnail storage process is a process of selecting and storing a thumbnail that is not overexposed or underexposed among the thumbnails created in step S6.

  As shown in FIG. 6, first, after executing step S6, the luminance value of each pixel of the thumbnail created in step S6 is calculated (step S78). Then, the number of pixels in which the calculated luminance value of each pixel of the thumbnail is within an allowable range is counted (step S79). That is, in step S76 in FIG. 5, the average value of the luminance values is obtained and determined, whereas in step S79 in this figure, the determination is made using the frequency distribution of the luminance values.

  And it is discriminate | determined whether the counted pixel number is more than a predetermined threshold value (step S80). If the counted number of pixels is equal to or greater than the predetermined threshold (step S80; YES), the thumbnail image data of the thumbnail created in step S6 is stored in the flash memory 38 (step S81), and the process proceeds to step S8. If the counted number of pixels is not equal to or greater than the predetermined threshold (step S80; NO), the process proceeds to step S8.

  In the first or second real-time thumbnail storing process, over-exposed or under-exposed thumbnails are excluded as stored thumbnails. As described above, the present invention is not limited to the configuration in which whether or not the exposure is appropriate is used as an appropriate thumbnail selection determination condition.

  For example, whether or not the automatically set shutter speed is in an appropriate range may be used as an appropriate thumbnail selection determination condition. The digital camera corrects to keep the shutter speed slow and open brightly when the image is too dark, whether moving image tracking or still image tracking, and conversely when the image is too bright, open the shutter speed quickly to keep it dark. Finally correct. Whether or not the shutter speed is appropriate may be determined as an appropriate thumbnail selection determination condition based on whether or not the correction level is too strong. In other words, a thumbnail that is extremely white and a thumbnail that is extremely dark and noisy has a large correction amount and is excluded from appropriate thumbnails.

  Similarly to the shutter speed, whether or not the aperture automatically set is appropriate may be used as an appropriate thumbnail selection determination condition. Further, whether or not the histogram as the distribution of the area occupied by the thumbnail image with respect to the luminance value (frequency distribution of the luminance value) is too tight may be set as a determination condition for appropriate thumbnail selection. For example, it is also possible to make a condition whether or not the distribution of the luminance value distribution in the histogram is equal to or greater than a predetermined value. By doing so, for example, a monotonous image such as almost the same color on one side can be displayed as a thumbnail. You can avoid choosing as.

  Also, for example, whether or not there are a predetermined number or more of pixels having luminance values outside a predetermined range (for example, near the center) in the histogram can be used as an inappropriate thumbnail exclusion determination condition. For example, when there are a predetermined number or more of pixels with luminance values outside the predetermined range, the pixels may be concentrated in an inappropriate luminance value area and may be an inappropriate thumbnail. In addition, if there are too few pixels with luminance values outside the predetermined range, it may be determined that the image is a monotonous image and inappropriate as a thumbnail.

  Further, for example, whether or not the luminance number where the number of pixels is the largest (mode) in the histogram is equal to or greater than a predetermined threshold may be used as an appropriate thumbnail selection determination condition. This is because it is a condition whether or not a certain level of brightness is at a peak.

  Further, for example, whether or not there are a predetermined number or more of pixels in the high luminance area in the histogram may be used as an inappropriate thumbnail exclusion determination condition. For example, the histogram shown in FIG. 7A is inappropriate because the number of pixels is concentrated in a high-brightness region, so that it is so bright that overexposure may occur. In addition, for example, whether or not there are a predetermined number or more of pixels in the low luminance region in the histogram may be used as an inappropriate thumbnail exclusion determination condition. The histogram shown in FIG. 7B is inappropriate because it is too dark because the number of pixels is concentrated in a low luminance area. These can be determined by counting the frequency distribution of the luminance value of each pixel of the image and counting how many pixels are above or below the threshold compared to a predetermined threshold.

  Further, as described above, in addition to the method of determining an inappropriate thumbnail using the histogram (frequency distribution) of the luminance value of the image, the luminance level (luminance signal) with respect to the hue component (here, the wavelength of visible light). The spectrum as the distribution of (amplitude) may be used as a determination material, and whether or not this spectrum is appropriate may be used as a determination condition for appropriate thumbnail selection. For example, whether or not there is a certain luminance level over the entire hue component in the spectrum may be used as an appropriate thumbnail selection determination condition. Specifically, it is determined whether or not the integrated value of the luminance level is greater than or equal to a predetermined value. The spectrum shown in FIG. 8A is unsuitable because it is too dark for the hue component as a whole and is too dark.

  Further, for example, whether or not the peak of the luminance level in the spectrum is equal to or higher than a predetermined threshold may be used as an appropriate thumbnail selection determination condition. The spectrum shown in FIG. 8B is appropriate when the luminance level peak P1 is equal to or greater than a predetermined threshold. Further, for example, whether or not a certain high luminance level is widely distributed in the spectrum may be used as an appropriate thumbnail selection determination condition. The spectrum shown in FIG. 8B is appropriate when the luminance level equal to or higher than the predetermined value is distributed over the predetermined length L1. When the luminance level equal to or higher than the predetermined value is distributed over the predetermined length L1, it is considered that the image is appropriate as a thumbnail, not a monotonous image such as a solid image (an image having the same color on the entire surface). Because.

  Further, for example, whether or not the unevenness of the luminance level is moderately distributed in the spectrum may be used as an appropriate thumbnail selection determination condition. For example, whether or not the number of inflection points is greater than or equal to a predetermined number may be used as an appropriate thumbnail selection determination condition. It is assumed that the spectrum shown in FIG. 8B has moderate unevenness.

  In addition, in the spectrum, whether or not the luminance level is concentrated in one place of a certain hue component may be used as an inappropriate thumbnail removal determination condition. This is because when the luminance level is concentrated at one location of a certain hue component, the image is too simple and cannot be regarded as an appropriate thumbnail. That is, the thumbnail requires a certain level of complexity. However, those that are too complex are excluded. In the histogram of FIG. 8A, the luminance levels are distributed almost uniformly and are reasonably complex. When making such a determination, for example, it is appropriate when the peak P1 of the luminance level is equal to or greater than a predetermined threshold. Further, for example, whether or not a certain high luminance level is widely distributed in the spectrum may be used as an appropriate thumbnail selection determination condition. The spectrum shown in FIG. 8B is appropriate when the luminance level equal to or higher than a predetermined value is distributed over a predetermined predetermined length L1.

  Note that two or more of the plurality of determination conditions described above may be combined as a determination condition for selecting a thumbnail with an appropriate image quality. In this case, an appropriate thumbnail can be selected more accurately. Further, the parameter calculation using the histogram and spectrum may be executed in the real-time parameter calculation process in step S71.

  As described above, according to the present embodiment, during moving image shooting with the digital camera 1, thumbnails are created one after another from the captured moving image data at a predetermined timing, and the parameters for narrowing down thumbnails of the created thumbnails are real-time. And thumbnails with appropriate image quality are extracted based on the calculation results. After shooting the video, parameters for further thumbnail selection are calculated, and thumbnails with more appropriate image quality are extracted based on the calculation results. So you can extract thumbnails with appropriate quality.

  Then, the extracted thumbnail is determined based on the selection input by the user via the key input unit 36, and the thumbnail image data of the determined thumbnail is stored in the flash memory 38 in association with the moving image data. It is possible to select and store a thumbnail having a more appropriate image quality corresponding to the image data.

  In step S71, parameter calculation with a relatively small processing load is performed in real time, so that delay or interruption of the moving image shooting process and the parameter calculating process at the time of moving image shooting is prevented, and parameter calculation processing after moving image shooting is performed. The burden can be reduced. Further, in step S9, parameter calculation with a relatively large processing load is performed after moving image shooting, so that thumbnails can be further appropriately narrowed down after moving image shooting.

Further, since it is determined whether or not the thumbnail is appropriate based on the luminance value distribution of the thumbnail image data, for example, the luminance value distribution such as the thumbnail being extremely bright, extremely dark, overexposed, or underexposed is visually You can remove thumbnails that are inappropriate.
In addition, since it is determined whether or not the thumbnail is appropriate based on the hue component distribution of the luminance level of the thumbnail image data, it is possible to exclude visually inappropriate thumbnails such as thumbnails being monotonous images, for example. it can.

  In step S9, since various parameters are calculated using information stored in the log file after moving image shooting, the information used during moving image shooting can be used effectively. Note that if the process of removing inappropriate thumbnails (steps S9, S10, S11) is not performed after moving image shooting, the log file update process of step S4 is not necessary.

The description in the above-described embodiment is an example of a suitable image processing apparatus according to the present invention, and the present invention is not limited to this.
In the above embodiment, the offense using the digital camera 1 having the moving image shooting function has been described. However, the present invention is not limited to this. For example, in a configuration in which parameter calculation processing and thumbnail extraction processing are performed only after shooting without performing parameter calculation processing in real time with moving image shooting, the image shooting apparatus such as the digital camera 1 does not have parameter calculation and thumbnail extraction functions. Further, a configuration may be adopted in which an external device such as a PC (Personal Computer) is provided with a parameter calculation and thumbnail extraction function. In this case, moving image data is input to the external device from the image capturing device, and parameter calculation and thumbnail extraction processing are performed on the input moving image data in the external device.

It is a figure which shows the external appearance of the digital camera 1, (a) is a figure which mainly shows the structure of a front surface, (b) is a figure which mainly shows the structure of a back surface. 2 is a block diagram showing an internal configuration of the digital camera 1. FIG. It is a flowchart which shows a moving image shooting process. A real-time thumbnail storage process during the moving image shooting process is shown. It is a flowchart which shows a 1st real-time thumbnail storage process. It is a flowchart which shows a 2nd real-time thumbnail storage process. It is a figure which shows the example of a distribution of the area ratio in the image with respect to a brightness | luminance, (a) is a figure which shows a distribution example with a large area ratio in high brightness, (b) shows a distribution example with a large area ratio in low brightness. FIG. It is a figure which shows the example of distribution of the luminance level of the image with respect to a hue component, (a) is a figure which shows the example of distribution of a low-luminance level irrespective of a hue component, (a) is an example of distribution of the luminance level which has dispersion | variation. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Shooting lens 3 Self-timer lamp 4 Optical finder window 5 Strobe light emission part 6 Power key 7 Shutter key 8 Mode switch 9 Menu key 10 Cross key 11 Optical finder 12 Strobe charge lamp 13 Display part 21 Motor (M)
22 Lens optical system 23 CCD
24 Timing generator (TG)
25 Vertical driver 26 Sample hold circuit (S / H)
27 A / D converter 28 Color process circuit 29 DMA controller 30 DRAM interface (I / F)
31 DRAM
32 Control unit 33 VRAM controller 34 VRAM
35 Digital Video Encoder 36 Key Input Unit 37 Compression / Expansion Circuit 38 Flash Memory

Claims (8)

A frame that is a candidate for a thumbnail by executing in real time the first image analysis with a processing load equal to or less than a predetermined value on image data of each moving image frame obtained by sequentially capturing images with moving image data during shooting of moving image data A thumbnail extraction unit that sequentially selects
After the shooting of the moving image data is completed, a second image analysis that has a larger processing load than the first image analysis is performed on the image data of each moving image frame that has already been recorded, and the thumbnail extracting unit A thumbnail determination unit for determining a frame to be a final thumbnail from a plurality of frames selected as thumbnail candidates by
A thumbnail storage unit that records a thumbnail corresponding to the frame determined by the thumbnail determination unit in association with the moving image data;
An image processing apparatus comprising: A shooting control unit for performing a third image analysis for determining shooting and recording conditions;
The image processing apparatus according to claim 1, wherein the thumbnail extraction unit performs the first image analysis using the third image analysis. The shooting control unit performs the third image analysis to determine at least one shooting condition among shooting and recording conditions including shutter speed, aperture value, autofocus, automatic exposure, and auto white balance. The image processing apparatus according to claim 2. The first image analysis, image processing apparatus according to any one of claims 1 to 3, characterized in that said an analysis of the distribution of luminance values of the image data. The second image analysis, image processing apparatus according to any one of claims 1 to 3, characterized in that said an analysis of the color component distribution of the image data. The thumbnail extraction unit records a thumbnail image of a frame selected in real time during moving image shooting,
The thumbnail determination unit deletes a thumbnail image recorded corresponding to a frame that has not been determined as a final thumbnail from the thumbnail images recorded by the thumbnail extraction unit. The image processing apparatus according to claim 5. The image processing according to claim 1 , wherein the thumbnail determination unit includes a process of determining a frame to be a final thumbnail from a plurality of frames according to a selection input by a user. apparatus. On the computer,
A frame that is a candidate for a thumbnail by executing in real time the first image analysis with a processing load equal to or less than a predetermined value on image data of each moving image frame obtained by sequentially capturing images with moving image data during shooting of moving image data A thumbnail extraction function that sequentially selects
After the shooting of the moving image data is completed, a second image analysis that has a larger processing load than the first image analysis is performed on the image data of each moving image frame that has already been recorded, and the thumbnail extraction function A thumbnail determination function for determining a final thumbnail frame from a plurality of frames selected as thumbnail candidates by
A thumbnail storage function for recording a thumbnail corresponding to the frame determined by the thumbnail determination function in association with the moving image data;
An image processing program for realizing

Images