JP5525703B2 - Image playback display device - Google Patents

Description

The present invention relates to an image reproducing display equipment.

  Most digital cameras have a display using a TFT liquid crystal display device or an organic EL display device (hereinafter referred to as “built-in display”), and image data recorded and recorded (hereinafter referred to as “recorded image”). Data ") can be read from the storage medium and displayed. Similarly, devices such as a storage viewer that can store and carry recorded image data and a digital photo frame that can be placed on a desktop to display and view images can also read recorded image data from a storage medium. The above-mentioned built-in display. Many of these built-in displays have a display resolution (number of display pixels) of about QVGA (horizontal 320 pixels × vertical 240 pixels) to VGA (horizontal 640 pixels × vertical 480 pixels).

  On the other hand, the number of pixels of recording image data generated by a digital camera (hereinafter referred to as “recording pixel number”) tends to increase year by year as the number of pixels of the image sensor increases. As a result, in many cases, the number of recorded pixels greatly exceeds the number of display pixels of the built-in display. Therefore, before the recorded image data is read from the storage medium and displayed on the built-in display, the recorded image data needs to be resized to reduce the number of pixels.

  An image sensor incorporated in a digital camera has a large number of pixels (photoelectric conversion elements) two-dimensionally arranged in a discrete (non-continuous) manner at a predetermined arrangement pitch. When a subject image formed on the image sensor is captured and sampled, moire may occur depending on the spatial frequency of the subject image. This moire occurs when the subject image has a spatial frequency component equal to or higher than the Nyquist limit determined by 1/2 of the pixel pitch (spatial sampling period) of the pixels arranged on the image sensor to be used. It has been known.

  As a method for suppressing the above moire, Patent Document 1 discloses a method of setting an upper limit on the spatial frequency of a subject image using an optical low-pass filter (hereinafter, the optical low-pass filter is referred to as “OLPF”). ing. Further, in Patent Document 1, an OLPF cut-off frequency can be switched according to a selected resolution in a video camera capable of performing image signal processing at either one of a low resolution and a high resolution. The structure is disclosed.

As another method for suppressing moire, Patent Document 2 discloses that the focus position is shifted from the image data obtained when the focus position of the subject image formed on the light receiving surface of the image sensor matches. A technique is disclosed in which image processing is performed using image data obtained when the moiré is made inconspicuous.
JP-A-3-226078 Japanese Patent Laid-Open No. 10-276350

  With the techniques disclosed in Patent Document 1 and Patent Document 2 described above, it is possible to suppress the occurrence of moire in an image obtained by photographing.

  By the way, if the process of resizing the image data to reduce the number of pixels (this process is simply referred to as “resizing” in this specification), it is equivalent to the number of pixels related to sampling (imaging) being reduced equivalently. It becomes a state. That is, it becomes the same state as when the arrangement pitch of the pixels of the image sensor increases (becomes coarse). As a result, the spatial sampling frequency decreases and the spatial frequency at the Nyquist limit also decreases. This means that even when the spatial frequency of the subject image is such that moire does not occur in an image having the number of pixels (resolution) obtained at the time of photographing, moire may occur in an image obtained by resizing. Means that.

  In the cited document 1 and the cited document 2 described above, there is no mention of a solution to the occurrence of moire in an image displayed by resizing the recorded image data.

  The present invention has been made in view of the above-described problems, and an object thereof is to make it possible to suppress the occurrence of moire in an image displayed by resizing recorded image data.

(1) A first aspect of the present invention includes a plurality of resizing algorithms for resizing processing, and uses any one of the plurality of resizing algorithms to perform resizing processing on recorded image data to display an image. The present invention is applied to an image reproduction display device that generates display image data suitable for the number of display pixels of a unit and displays a display image based on the display image data on the image display unit. And this image reproduction display device
A moire determination unit that determines the degree of moire occurrence of display image data generated using any of the plurality of resizing algorithms;
A display image data generation unit that generates display image data using any one of the plurality of resizing algorithms and outputs the display image data to the image display unit;
The display image data generation unit prioritizes the resizing algorithm in the plurality of resizing algorithms in a priority order that can be expected to obtain a display image with high resolution or in a priority order that can be processed in a short time. Run to
When the moire determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm is equal to or less than a predetermined reference, the first resizing algorithm is used. The generated display image data is output to the image display unit,
If the moiré determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm exceeds the predetermined reference, the remaining including the second resizing algorithm The display image data is generated using all of one or a plurality of resizing algorithms, and the degree of moire generation in the generated display image data is determined in the display image data generated using the first resizing algorithm. Including the degree of moire occurrence, and is configured to output display image data generated by the resizing algorithm having the lowest degree of moire generation among the plurality of resizing algorithms to the image display unit.
Or
When the moire determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm is equal to or less than a predetermined reference, the first resizing algorithm is used. The generated display image data is output to the image display unit,
If the moiré determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm exceeds the predetermined reference, the remaining including the second resizing algorithm The resize algorithm is sequentially used to generate the display image data, and each time the moire determination unit determines the degree of moiré in the display image data. Even if there is a remaining resizing algorithm by outputting the display image data generated using the image display unit, the generation processing of the display image data using them or the same is terminated,
As described above, the display image data generated by the resize algorithm having the lowest degree of moire generation among the plurality of resize algorithms is output to the image display unit when the predetermined reference is exceeded. Solve the problem.
(2) The second aspect of the present invention includes two types of resizing algorithms including a first resizing algorithm and a second resizing algorithm for resizing processing. Among the two types of resizing algorithms for recording image data, Any one of them is used to generate a display image data adapted to the number of display pixels of the image display unit by performing a resizing process, and applied to an image reproduction display device that displays a display image based on the display image data on the image display unit Is done. The first resizing algorithm is a resizing algorithm that can be expected to obtain a display image with a higher resolution than the second resizing algorithm, or a resizing algorithm that can be processed in a shorter time. This image reproduction display device
A moire determination unit that determines the degree of moire occurrence of display image data generated using the first resizing algorithm;
A display image data generation unit that generates display image data using one of the two types of resizing algorithms and outputs the display image data to the image display unit, the display image generated using the first resizing algorithm When the moire determination unit determines that the degree of occurrence of data moiré is below a predetermined reference, display image data generated using the first resize algorithm is output to the image display unit. On the other hand, if the moire determination unit determines that the predetermined reference is exceeded, display image data is generated using the second resizing algorithm and output to the image display unit. A display image data generation unit.

  According to the present invention, when resize processing is performed on recorded image data to generate display image data suitable for the number of display pixels of the image display unit, and a display image based on the display image data is displayed on the image display unit In addition, it is possible to suppress the occurrence of moire.

  FIG. 1 is a block diagram illustrating a schematic configuration of a photographing apparatus 100 according to an embodiment of the present invention. The image capturing apparatus 100 can capture a still image, can capture a moving image, or can capture both. This photographing apparatus 100 may be used alone or may be incorporated in a device such as a mobile phone and function as a photographing apparatus.

  The photographing apparatus 100 includes a photographing lens 102, an image sensor 104, an image signal processing unit 106, a timing generation unit 122, an image display unit 124, a system bus 140, a lens driving unit 142, and a CPU 146. The above components are electrically connected via a system bus 140. The photographing apparatus 100 also includes a memory control unit 152, a flash memory 153, a RAM 154, and an interface 156. The memory control unit 152 and the interface 156 are also electrically connected to the above-described components via the system bus 140. The RAM 154 is electrically connected to the above-described components via the memory control unit 152 and the system bus 140. The interface 156 is configured such that a storage medium 158 composed of a flash memory, a small hard disk drive, or the like can be detachably connected.

  The photographing lens 102 forms a subject image on the imaging surface of the image sensor 104. As the image sensor 104, a two-dimensional image sensor composed of a CCD, a CMOS image sensor, or the like can be used. In the present embodiment, the image pickup device 104 will be described as a CMOS image sensor that has processing blocks such as CDS and A / D conversion therein and can output digital image signals. The image sensor 104 is also described as a single-plate image sensor having a Bayer array on-chip color filter.

  The image sensor 104 is configured to be able to output image signals with a plurality of types of resolutions. For example, the image sensor 104 can output an image signal at a resolution higher than the resolution for displaying the live view image and the resolution for displaying the live view image used for generating the recorded image data. Although the image signal for live view image display has a relatively low resolution (eg, horizontal 640 pixels, vertical 480 pixels), the amount of information read from the image sensor 104 is reduced, so that a relatively high frame rate (field reading In this case, the image signal can be read at a field rate. Accordingly, it is possible to display a smooth live view image by setting the frame rate (field rate) of the live view image to be relatively high.

  The image signal used for generating the recorded image data has a higher resolution (for example, horizontal 4,000 pixels and vertical 3,000 pixels) than the resolution obtained with the image signal for live view image display. The image sensor 104 can be configured to output an image signal in an arbitrary form among a plurality of types of resolutions and a plurality of types of aspect ratios as image image signals used for generating recorded image data. . However, for the purpose of simplifying the description in this specification, the image signal for live view image display has 640 pixels in the horizontal direction and 480 pixels in the vertical direction, and the image signal for generating recorded image data. Will be described as having 4,000 pixels in the horizontal direction and 3,000 pixels in the vertical direction.

  The digital image signal output from the image sensor 104 is temporarily stored in the RAM 154 via the memory control unit 152. Since the RAM 154 is also used as a buffer memory when the image signal processing unit 106 performs image processing, it is desirable that the writing / reading speed be fast, and it can be constituted by, for example, a DRAM. The imaging device 104, the image signal processing unit 106, the CPU 146, and the image display unit 124 are configured to be accessible to the RAM 154 via the system bus 140. The memory control unit 152 has a function of arbitrating memory access requests from the above-described components.

  The image signal processing unit 106 can be configured by an application specific integrated circuit (ASIC) or the like. The image signal processing unit 106 includes a synchronization processing unit 108, a recorded image data generation unit 110, a display image data generation unit 112, and a moire detection processing unit 116. The synchronization processing unit 108 performs synchronization processing (demosaic processing) on the digital image signal output from the image sensor 104 and temporarily stored in the RAM 154.

  The recording image data generation unit 110 performs processing such as white balance adjustment, gradation / level correction, unsharp masking, shading correction, and the like on the image data that has been demosaiced by the synchronization processing unit 108. Is generated. The recording image data generated by the recording image data generation unit 110 is JPEG-compressed as necessary by the image signal processing unit 106 and is recorded on the storage medium 158 via the interface 156.

  The display image data generation unit 112 performs a resizing process on the recording image data, and generates display image data suitable for the number of display pixels of a display element 128 of the image display unit 124 described later. The display image data generation unit 112 includes a resizing processing unit 114, and the resizing processing unit 114 performs the resizing process described above. The resizing processing unit 114 has a plurality of processing algorithms for resizing processing (hereinafter referred to as “resizing algorithm”), and performs resizing processing using a resizing algorithm selected based on a command issued from the CPU 146 described later. Do.

  The moiré detection processing unit 116 detects the level of moire generated in the display image based on the display image data generated by the display image data generation unit. As a method of moiré level detection processing in the moiré detection processing unit 116, it is possible to use a method of performing two-dimensional Fourier transform processing on the display image data generated by the display image data generation unit 112.

  Here, the two-dimensional Fourier transform will be described with reference to FIG. FIG. 2A shows an example of an image that does not cause moire. FIG. 2B conceptually depicts a state in which a power spectrum obtained by two-dimensional Fourier transform of the image data of the image shown in FIG. 2A is plotted. FIG. 2C shows an example of an image in which moire is generated in the body portion of the subject. FIG. 2D conceptually depicts a power spectrum obtained by two-dimensional Fourier transform of the image data of the image shown in FIG.

  As shown in FIGS. 2A and 2C, two-dimensional Fourier transform F is performed when x and y coordinates are taken and an image (pixel value at coordinates x and y) is represented by F (x, y). (U, v) is defined as follows.

In the above equation, j = √ (−1), u represents the spatial frequency in the x direction, and v represents the spatial frequency in the y direction. The images shown in (a) and (c) of FIG. 2 are represented by F (x, y), and two-dimensional Fourier transform is performed using the above equation (1) to obtain F (u, v). What is plotted in the coordinate system of u and v is the power spectrum shown in FIGS.

  In FIGS. 2B and 2D, when there is a plot point at the center of the graph, it is u = v = 0, that is, a component having a spatial frequency of 0 (lines / mm or line pairs / mm) in the image. Is included. On the other hand, when there is a plot point in the peripheral portion of the graph, it indicates that the image contains a component having a relatively high spatial frequency. A power spectrum as shown in FIG. 2B is generally obtained from an image as shown in FIG. That is, the image includes a relatively large spectrum of low spatial frequencies, and the spectrum intensity tends to decrease as the spatial frequency increases. On the other hand, if the image contains many linear components extending along a specific direction such as an image such as a window frame or a horizontal line, the pattern of the power spectrum also exhibits a specific shape accordingly.

  If a person as a subject wears clothes with a relatively fine and periodic pattern such as a check pattern, the image may cause moire as shown in FIG. is there. When moire is generated in the image, as shown in FIG. 2 (d), a unique spectral pattern S appears at a symmetrical position in the vertical and horizontal directions of the graph. The moiré detection processing unit 116 performs the above-described two-dimensional Fourier transform on the image data that has been resized by the resize processing unit 114. The moire detection processing unit 116 further detects the occurrence level of moire based on the result obtained by performing the two-dimensional Fourier transform process. At this time, the image data is subdivided into a plurality of areas, and the two-dimensional Fourier transform is performed for each subdivided area to obtain the spatial frequency spectrum. It is possible to detect how much is.

  The image display unit 124 includes a display element 128 configured by a color liquid crystal display (LCD) panel or an organic EL (OEL) display panel, and a display control unit 126 that performs processing for displaying an image on the display element 128. . The display control unit 126 performs processing for displaying an image based on the image data generated by the image signal processing unit 106 on the display element. The display element 128 may be an EVF (= Electronic View Finder) for viewing an image displayed on a small display panel through an enlargement optical system. Or the monitor display panel provided in the back surface, the side surface, etc. of the imaging device 100 may be sufficient. Further, both the EVF and the monitor display panel may be provided so that the photographer can select which one to use.

  The display resolution of the display element 128, that is, the number of display pixels in the horizontal direction and the vertical direction is smaller than the number of pixels of the recorded image data. Therefore, the display image data generation unit 112 performs resize processing on the recording image data, and generates display image data suitable for the number of display pixels of the image display unit 124 (display element 128).

  For the reason described above, the display image data generated by performing the resizing process on the recorded image data has a smaller number of pixels than the number of pixels of the recorded image data. This is equivalent to a decrease in the spatial sampling frequency of the subject image. When the spatial sampling frequency is lowered, the spatial frequency at the Nyquist limit is also lowered. This means that even if the spatial frequency of the subject image is such that moire does not occur in an image based on recorded image data, moire may occur in a display image based on display image data. The moiré detection processing unit 116 executes the moiré detection process in response to the CPU 146 issuing a command signal for executing the moiré detection process. The result of moiré detection by the moiré detection processing unit 116 is output to the CPU 146.

A method for detecting the moire level will be described with reference to FIGS. 3A, 3B, 4A, 4B, 5A, and 5B. Figures 3A, 4A, 5A shows an example of a display image that moire intensity of occur are moire strongly Figure 3A is the most weak Figure 5A is top, and Figure 4A is medium intensity It has become. 3B, 4B, and 5B show plots of spatial frequency spectrum intensities obtained by two-dimensional Fourier transform of the display image data shown in FIGS. 3A, 4A, and 5A on uv coordinates.

  Referring to FIG. 3B, the spectrum patterns to which reference numerals S31, S32, S33, and S34 are attached are derived from moire. In FIG. 3B, only the spectrum pattern existing in the first quadrant of the uv coordinate system is given a reference numeral. The same applies to FIGS. 4B and 5B.

  Referring to FIG. 4B, the spectrum patterns to which symbols S41, S42, S43, and S44 are attached are derived from moire. Comparing FIG. 3B and FIG. 4B, there is almost no difference in the spectral distribution of one zone centered on u = 0 and v = 0. That is, there is no difference in the spectrum distribution in the component region with a low spatial frequency. However, it can be seen that the spectral intensities of S43 and S44 are weaker than the spectral intensities of S33 and S34.

  Referring to FIG. 5B, the spectrum patterns with S51 and S52 are derived from moire. Compared with FIG. 3B and FIG. 4B, there is almost no difference in the spectral distribution of one band centered on u = 0 and v = 0. However, it can be seen that the spectrum pattern derived from moire is greatly reduced as compared with those shown in FIGS. 3B and 4B.

  The moire generated on the display image changes depending on the relationship between the number of pixels of the recording image data and the number of pixels of the display image data, the resizing algorithm used for the resizing process, the spatial frequency of the subject image, and the like. If the moire changes, the frequency spectrum included in the moire-derived spectrum pattern also changes. The moiré detection processing unit 116 analyzes the spatial frequency spectrum of the display image data obtained by performing the two-dimensional Fourier transform process on the display image data generated by the display image data generation unit 112. When a spectrum pattern derived from moire is detected, the degree of moire generation in the display image is obtained from the intensity of the spectrum pattern and output to the CPU 146.

  The CPU 146 has a function of generally controlling the operation of the photographing apparatus 100 as a whole. A program interpreted and executed by the CPU 146 is recorded in the flash memory 153 as firmware. When the photographing apparatus 100 is turned on, this program is copied to the RAM 154. The CPU 146 interprets and executes the program copied to the RAM 154. For example, a series of operations such as photometry related to automatic exposure, exposure amount calculation, charging of the main capacitor for the flash unit, adjustment of the flash emission amount, focusing control of the photographing lens 102, acceptance of a photographing mode setting operation by the photographer, The CPU 146 performs transmission of a control signal to the display unit 124, opening / closing of a shutter (not shown), control of an opening of a diaphragm (not shown), and the like. The lens driving unit 142 drives each actuator for zoom driving, focusing driving, shutter driving, and aperture driving incorporated in the photographing lens 102 based on a control signal output from the CPU 146.

  The CPU 146 also issues a command to the image signal processing unit 106 to perform any of live view image display processing, image recording processing, and reproduction image display processing. When the CPU 146 issues a command to the image signal processing unit 106 to perform the reproduction image display processing, the CPU 146 provides information specifying any one of the plurality of resizing algorithms included in the resizing processing unit 114 to the image signal processing unit 106. Output to. The display image data generation unit 112 generates display image data using a resize algorithm specified by the CPU 146. The moiré detection processing unit 116 performs processing for detecting moiré from the display image data as described above, and outputs the detection result to the CPU 146. The CPU 146 determines the degree of moire generation based on the moire detection result output from the moire detection processing unit 116. Based on the determination result of the degree of occurrence of moire, the CPU 146 determines which resize algorithm out of a plurality of resize algorithms is used to output to the image display unit 124. Then, the CPU 146 outputs the display image data generated using the determined resizing algorithm to the image signal processing unit 106 so as to output it to the image display unit 124. As a result, it is possible to reduce moire of the display image displayed on the display element 128 of the image display unit 124.

  The imaging apparatus 100 according to the first to sixth embodiments described below has the same configuration as described above with reference to FIGS. The difference between the first to sixth embodiments lies in the contents of processing performed in the photographing apparatus 100, as will be apparent from the following description.

-First embodiment-
Hereinafter, the photographing apparatus 100 according to the first embodiment of the present invention will be described focusing on the processing contents of the image signal processing unit 106 and the CPU 146. In the first embodiment, the resize processing unit 114 will be described as having five resize algorithms. Hereinafter, these five resizing algorithms are referred to as “resizing algorithm 1”, “resizing algorithm 2”,..., “Resizing algorithm 5”. As these resizing algorithms, the present invention is not limited to these, but resizing algorithm 1 is a bicubic method, resizing algorithm 2 is a nearest neighbor method, resizing algorithm 3 is a bilinear method, resizing algorithm 4 Can use the Lagrangian method, and the resizing algorithm 5 can use the Lanchos method. Although the present embodiment is described as having five resize algorithms, the present invention is not limited to this. The number of resizing algorithms can be two or more.

  In the embodiment of the present invention, the resizing algorithm 1 is performed first, and then the resizing algorithms 2, 3,... Are performed in the embodiment of the present invention. When setting the priority for executing these resizing algorithms, a resizing algorithm that can be expected to obtain a display image with higher resolution even if the possibility of moiré is high is preferentially used. Can be. Alternatively, a resizing algorithm that can be processed in a shorter time can be preferentially used. Furthermore, a resizing algorithm that can obtain a good result relatively stably may be preferentially executed for a wide variety of images.

  FIG. 6 is a flowchart schematically showing a procedure of image reproduction display processing executed by the CPU 146. The process shown in the flowchart of FIG. 6 may be executed when, for example, the photographing apparatus 100 is set to the image reproduction display mode and the user selects a desired image from the thumbnail images.

  In S <b> 602, the CPU 146 reads JPEG-compressed image data (hereinafter referred to as “JPEG image data”) from the storage medium 158 and stores it in the RAM 154. In step S <b> 604, the CPU 146 issues a command to the image signal processing unit 106 to perform JPEG expansion processing. In response to this instruction, the image signal processing unit 106 performs JPEG decompression processing on the JPEG image data stored in the RAM 154 in the processing of S <b> 602, and stores the generated recorded image data in the RAM 154.

  In S606, the CPU 146 sets a variable n to 1. This variable n is for designating the type of resizing algorithm to be applied when generating display image data. For example, when the variable n is set to 1, the resizing algorithm 1 is applied in the display image data generation process described later.

  In step S <b> 608, the CPU 146 issues a command to the image signal processing unit 106 to generate display image data. At this time, the CPU 146 outputs information (algorithm designation information) for designating a resizing algorithm based on the value set in the variable n to the image signal processing unit 106. The image signal processing unit 106 that has received these commands and algorithm designation information performs processing for generating display image data using a designated resizing algorithm (here, resizing algorithm 1 is designated).

  In S610, the CPU 146 issues a command to the image signal processing unit 106 to perform the moire detection process. Upon receiving this instruction, the image signal processing unit 106 performs moiré detection processing on the reproduced image data generated in step S <b> 608 and outputs the moiré detection result to the CPU 146.

  In S612, the CPU 146 determines whether or not the moire occurrence level of the display image data detected in the process of S610 is equal to or lower than a predetermined reference based on the moire detection result output from the image signal processing unit 106. judge. If the determination in S612 is affirmative, that is, if it is determined that the moire occurrence level of the display image data generated in S608 is equal to or less than a predetermined reference, the process proceeds to S614.

  In S614, the CPU 146 selects display image data generated by using the resizing algorithm 1 as data to be output to the image display unit 124, and issues a command to display a reproduced image on the image display unit 124 in subsequent S616. . The image display unit 124 receives this command from the CPU 146, and performs a process of displaying a display image based on the display image data generated in S608 on the display element 128.

  If the determination in S612 is negative, that is, if it is determined that the moire occurrence level of the display image data detected in the process of S610 exceeds a predetermined reference, the process proceeds to S618. In S618, the CPU 146 stores the moiré occurrence level detected in S610 in the RAM 154.

  In S620, the CPU 146 determines whether or not the variable n is 5. If this determination is negative, that is, it is determined that the value of the variable n is less than 5, the process proceeds to S624. In S624, the CPU 146 performs a process of incrementing the variable n, that is, a process of adding increment 1 to the variable n.

  In S626, the CPU 146 issues a command to the image signal processing unit 106 to generate display image data. At this time, the CPU 146 outputs information (algorithm designation information) for designating a resizing algorithm based on the value set in the variable n to the image signal processing unit 106. The image signal processing unit 106 that has received these commands and algorithm designation information performs a process of generating display image data using a designated resizing algorithm (resizing algorithm n).

  In S628, the CPU 146 issues a command to the image signal processing unit 106 to perform the moire detection process. Upon receiving this instruction, the image signal processing unit 106 performs moire detection processing on the reproduced image data generated in S626 and outputs the moire detection result to the CPU 146. The process of the CPU 146 returns to S618, and the loop process including S624, S626, S628, S618, and S620 is repeated until it is determined in S620 that the variable n is 5.

  If the determination in S620 is affirmative, the process proceeds to S622. At this point, the RAM 154 stores display image data generated using the resizing algorithms 1, 2,..., 5 and moire generation levels corresponding to these display image data. In S <b> 622, the CPU 146 selects display image data generated using the resizing algorithm having the smallest moire generation level among the resizing algorithms 1 to 5 as data to be output to the image display unit 124. The CPU 146 issues a command to display the reproduced image on the image display unit 124 in S616. The image display unit 124 receives this command from the CPU 146, and performs a process of displaying a display image based on the display image data selected in S622 on the display element 128.

  As described above, according to the photographing apparatus according to the first embodiment of the present invention, the following operational effects can be achieved.

  When it is determined in S612 that the moire generation level detected from the display image data generated using the resizing algorithm 1 is equal to or less than a predetermined reference, the display image data is generated using the resizing algorithms 2 to 5. No processing is performed. For this reason, it is possible to complete the reproduction image display processing within a relatively short time.

  If it is determined in S612 that the moire occurrence level detected from the display image data generated using the resizing algorithm 1 exceeds a predetermined reference, display image data is generated using the resizing algorithms 2 to 5. . At this time, the moire occurrence level is detected for each of the generated display image data. Then, the display image data generated using the resizing algorithm having the minimum moire generation level among the resizing algorithms 1 to 5 is selected as data to be output to the image display unit 124 and displayed on the display element 128. Accordingly, since the display image based on the display image data having the smallest moire generation level among the display image data generated using the five resize algorithms is displayed on the display element 128, the user can display the display image with reduced moire. Can be observed.

  7A and 8A show examples of images before resize processing (hereinafter, these images are referred to as “original images”). 7B, 8B, 7C, and 8C show examples of images after the resizing process. Note that the images shown in FIGS. 7B, 8B, 7C, and 8C are originally small images as a result of the resizing process and the number of pixels being reduced, but here for ease of viewing, 7A and 8A are shown to have the same display magnification. For example, it is assumed that the image generated by the resizing process using the resizing algorithm 1 is as illustrated in FIGS. 7B and 8B. If it is determined in S612 in FIG. 6 that the moiré generation level of these images exceeds a predetermined reference, resizing processing using resizing algorithm 2 to resizing algorithm 5 is performed. The images illustrated in FIGS. 7C and 8C that are generated using the resize algorithm having the minimum moire generation level are selected.

-Second Embodiment-
Hereinafter, the imaging apparatus 100 according to the second embodiment of the present invention will be described focusing on the processing contents of the image signal processing unit 106 and the CPU 146. Also in the second embodiment, similarly to the first embodiment, the resize processing unit 114 will be described as having five resize algorithms. Assume that as these five resizing algorithms 1, resizing algorithm 2,..., Resizing algorithm 5, resizing algorithms similar to those described in the first embodiment are set. As for the execution order of the resizing algorithms, the resizing algorithm 1 is performed first, and then the resizing algorithms 2, 3,... Are performed. Further, although the present embodiment is described as having five resizing algorithms, the present invention is not limited to this. The number of resizing algorithms can be 3 or more.

  FIG. 9 is a flowchart schematically showing a procedure of image reproduction display processing executed by the CPU 146. The process shown in the flowchart of FIG. 9 can be executed, for example, when the photographing apparatus 100 is set to the image reproduction display mode and the user selects a desired image from the thumbnail images. In FIG. 9, the same reference numerals are given to the processing procedures similar to the processing contents performed by the image signal processing unit 106 and the CPU 146 in the photographing apparatus 100 according to the first embodiment of the present invention shown in FIG. 6. Detailed description thereof is omitted. The description will focus on the differences from the first embodiment.

  In the first embodiment, the resizing algorithm 1 is first used when generating the display image data. When the moire generation level of the generated display image data is equal to or lower than a predetermined reference, the resizing algorithm 1 is used. A display image based on the display image data generated by using the display image data is displayed. This is the same in the second embodiment.

  Further, in the first embodiment, when the moiré generation level of the display image data generated using the resizing algorithm 1 exceeds a predetermined standard, the second to fifth resizing algorithms are used. Display image data generation processing is performed. Then, a display image based on the display image data with the minimum moire occurrence level is displayed from the display image data generated using the first to fifth resizing algorithms.

  On the other hand, in the second embodiment, when the moiré generation level of the display image data generated using the resizing algorithm 1 exceeds a predetermined reference, the second to fifth resizing algorithms are used. When the display image data is generated using, the moire detection process is sequentially performed. At this time, for example, when the moire generation level of the display image data generated using the third resizing algorithm is equal to or lower than a predetermined reference, the display images using the remaining fourth and fifth resizing algorithms are used. The data generation process is terminated. A display image based on the display image data generated using the third resizing algorithm is displayed. In other words, if there is a display image data generation / moire detection process using a plurality of resizing algorithms in sequence and the moire generation level is below a predetermined reference value, there are still resizing algorithms that have not yet been tried. However, the display image data generation process using them (they) is aborted.

  As a result of executing the above processing procedure, when the moire generation levels of all display image data generated using the second to fifth resizing algorithms exceed a predetermined standard, the first embodiment The same processing is performed. That is, a display image based on the display image data with the minimum moire occurrence level is displayed from the display image data generated using the first to fifth resizing algorithms.

  In S606, variable n is set to 1, display image data is generated using resizing algorithm 1 in S608, moiré detection processing is performed in S610, and it is determined in S612 that the moiré occurrence level exceeds a predetermined standard. From now on, it demonstrates with reference to FIG. In this case, the process proceeds to S618, and the CPU 146 stores the moiré occurrence level of the display image generated using the resizing algorithm 1 in the RAM 154.

  In S620, the CPU 146 determines whether or not the variable n is 5. If this determination is negative, that is, it is determined that the value of the variable n is less than 5, the process proceeds to S924. In S924, the CPU 146 performs a process of incrementing the variable n, that is, a process of adding increment 1 to the variable n. Thereafter, the process returns to S608, and display image data is generated using the resize algorithm specified by the updated variable n.

  In S610, the CPU 146 issues a command to the image signal processing unit 106 to perform the moire detection process. Upon receiving this instruction, the image signal processing unit 106 performs moiré detection processing on the reproduced image data generated in step S <b> 608 and outputs the moiré detection result to the CPU 146.

  In S612, the CPU 146 determines whether or not the moire occurrence level of the display image data generated in the process of S608 is equal to or lower than a predetermined reference based on the moire detection result output from the image signal processing unit 106. judge. If the determination in S612 is affirmative, that is, if it is determined that the moire occurrence level of the display image data detected in the process of S610 is equal to or less than a predetermined reference, the process proceeds to S914. In S914, the CPU 146 selects display image data generated using the resizing algorithm n as data to be output to the image display unit 124, and issues a command to display a reproduced image on the image display unit 124 in subsequent S616. . The image display unit 124 receives this command from the CPU 146, and performs a process of displaying a display image based on the display image data generated in S608 on the display element 128.

  The above points are different from the first embodiment. That is, as described above, in the process of generating display image data using all of the plurality of resize algorithms prepared, the moiré detection process is sequentially performed in S610. At this time, if it is determined in S612 that the moire generation level is equal to or lower than a predetermined reference, image display processing is performed in S616 using the display image data. Even if there are remaining resizing algorithms, the display image data generation processing using them is aborted.

  On the other hand, when it is determined that all the moire occurrence levels of the display image data generated using the resizing algorithms 1 to 5 exceed a predetermined standard, the process in S622 is performed. At the time of executing the processing of S622, the RAM 154 stores display image data generated using the resizing algorithms 1, 2,..., 5 and moire generation levels corresponding to these display image data. In step S <b> 622, the CPU 146 selects display image data generated using the resizing algorithm having the smallest moire generation level among the resizing algorithms 1 to 5 as data to be output to the image display unit 124. The CPU 146 issues a command to display the reproduced image on the image display unit 124 in S616. The image display unit 124 receives this command from the CPU 146, and performs a process of displaying a display image based on the display image data selected in S622 on the display element 128.

  As described above, in the photographing apparatus 100 according to the second embodiment of the present invention, three or more types of resizing algorithms are prepared in advance. In this resizing algorithm, display image data is generated using resizing algorithm 1. When the moiré occurrence level of the display image data is below a predetermined reference, a display image based on the display image data generated using the resizing algorithm 1 is displayed. On the other hand, when the moire generation level of the display image data generated using the resizing algorithm 1 exceeds a predetermined standard, display image data is generated using a plurality of other resizing algorithms in sequence. If the degree of occurrence of moire in the generated display image data is equal to or less than a predetermined reference, a display image based on the display image data generated using this resizing algorithm is displayed. Even if there are remaining resizing algorithms, the display image data generation processing using them is aborted. On the other hand, when the degree of moire generation in the display image data generated using three or more types of resizing algorithms exceeds a predetermined standard, the degree of moire generation is the highest of these three or more types of resizing algorithms. A display image based on the display image data generated by the low resizing algorithm is displayed.

  In this way, even when the moiré generation level of the display image data generated using the resizing algorithm 1 exceeds a predetermined reference, the moiré generation level is determined in advance within a shorter required time. It is possible to search for a resizing algorithm that is less than or equal to the standard. When the moiré occurrence levels of display image data generated using three or more types of resizing algorithms prepared in advance exceed all predetermined criteria, a resizing algorithm that minimizes the moiré occurrence level is used. A display image based on the display image data generated in this way is displayed. Therefore, it is possible to suppress the moire of the display image while suppressing the waiting time until the display image is displayed on the display element after the user operates the image display from being unnecessarily long. .

-Third embodiment-
Hereinafter, the photographing apparatus 100 according to the third embodiment of the present invention will be described focusing on the processing contents of the image signal processing unit 106 and the CPU 146. In the third embodiment, description will be made assuming that the resizing processing unit 114 has two resizing algorithms. As the resizing algorithm 1, the present invention is not limited to this, but the bicubic method can be used. Regarding the resizing algorithm 2, when the moire generation level of the display image data generated by using the resizing algorithm 1 exceeds a predetermined standard, it is desirable to make the algorithm capable of effectively suppressing the moire. The resizing algorithm 2 will be described in detail later.

  FIG. 10 is a flowchart schematically showing a procedure of image reproduction display processing executed by the CPU 146. The process shown in the flowchart of FIG. 10 may be executed when the photographing apparatus 100 is set to the image reproduction display mode and the user selects a desired image from the thumbnail images, for example. In FIG. 10, processing steps similar to those performed by the image signal processing unit 106 and the CPU 146 in the photographing apparatus 100 according to the first embodiment of the present invention shown in FIG. Detailed description thereof is omitted. The description will focus on differences from the first or second embodiment.

  In the first embodiment, the resizing algorithm 1 is first used when generating display image data. When the generated moire generation level of the display image data is equal to or less than a predetermined reference, a display image based on the display image data generated using the resizing algorithm 1 is displayed. With respect to these points, the third embodiment is the same.

  On the other hand, in the first embodiment, when the moiré generation level of the display image data generated using the resizing algorithm 1 exceeds a predetermined standard, the display image data is generated using the second to fifth resizing algorithms. The moire generation level in the displayed display image data is detected. In the first to fifth resizing algorithms, a display image is displayed based on the display image data generated by the resizing algorithm that minimizes the moire generation level.

  On the other hand, in the third embodiment, when the moiré generation level of the display image data generated using the resizing algorithm 1 exceeds a predetermined reference, the display image data is generated using the resizing algorithm 2. A display image is displayed based on the displayed image data. This is different from the first embodiment.

  The JPEG image data read from the storage medium 158 in S602 is subjected to JPEG expansion processing in S604 to generate recording image data, which is recorded in the RAM 154. In step S <b> 1008, the CPU 146 issues a command to the image signal processing unit 106 to generate display image data using the resizing algorithm 1. The image signal processing unit 106 that has received this command performs processing for generating display image data using the resizing algorithm 1.

  The display image data generated using the resizing algorithm 1 is subjected to moire detection processing in S610, and it is determined in S612 whether or not the detected moire generation level is equal to or less than a predetermined reference. If it is determined in S612 that the moire occurrence level of the display image data generated using the resizing algorithm 1 is equal to or less than a predetermined reference (determination in S612: Yes), the process proceeds to S614. In S614, a display image display process based on the display image data generated using the resizing algorithm 1 is performed. These processes are the same as those in the first and second embodiments.

  The difference from the first and second embodiments is a case where it is determined in S612 that the moire occurrence level of the display image data generated using the resizing algorithm 1 exceeds a predetermined reference (determination in S612: No). If the determination in S612 is negative, the process proceeds to S1026. In step S <b> 1026, the CPU issues a command to the image signal processing unit 106 to generate display image data using the resizing algorithm 2. The image signal processing unit 106 that has received this command performs a process of generating display image data using the resizing algorithm 2.

  In S1028, the CPU 146 selects display image data generated by using the resizing algorithm 2 as data to be output to the image display unit 124, and issues a command to display a reproduced image on the image display unit 124 in subsequent S616. . The image display unit 124 receives this command from the CPU 146 and performs a process of displaying a display image based on the display image data generated in S1026 on the display element 128.

  Here, the resizing algorithm 2 will be described. As a technique used in the resizing algorithm 2, the following can be used.

  Method 1. Although the processing content is more complicated than the method used in the resizing algorithm 1, a method in which the moire generation level is expected to be lower than when the method used in the resizing algorithm 1 is used (for example, the Lanczos method). .

  Method 2. Processing for attenuating or removing spatial frequency components (relatively high spatial frequency components) that cause moire by resizing processing in the spatial frequency components of the image based on the recorded image data before the resizing processing (in this specification, This method is referred to as “low-pass filtering processing”), and the resize processing is performed after the recording image data is applied in advance. As a specific method of the low-pass filtering process, there is a method of performing an averaging process on the recorded image data before the resizing process. As a method used for the resizing process, a method used in the resizing algorithm 1 (bicubic method in the present embodiment) may be used, or a method different from the method used in the resizing algorithm 1 may be used. May be.

  Method 3. A method of detecting the area where moire occurs (hereinafter referred to as “display image moire area”) in the display image based on the display image data generated using the resizing algorithm 1 and performing the following processing. That is, after the resize processing is selectively performed only on the data corresponding to the display image moire area in the recorded image data before the resize processing, the resize processing is performed. Do. The above-described low-pass filtering process is possible as a process for suppressing the generation of moire when the resizing process is performed. As a specific method of the low-pass filtering process, a method of performing an averaging process on the recorded image data before the resizing process can be used. At this time, the moving average calculation process is performed only on the data corresponding to the display image moire area in the recorded image data before the resizing process. For this reason, it is possible to maintain the resolution of the image other than the display image moire region.

  Method 4. A method of performing two-dimensional Fourier transform and inverse transform processing on display image data generated using the resizing algorithm 1. In this method, display image data generated using the resizing algorithm 1 is subjected to a two-dimensional Fourier transform process to obtain a frequency spectrum. Then, after removing the moire-derived spectral components from the obtained frequency spectrum, the display image data with reduced moire can be obtained by performing inverse two-dimensional Fourier transform. Note that the display image data to be subjected to the two-dimensional Fourier transform and the inverse transform may be other than the one generated using the resizing algorithm 1. That is, display image data generated using another resizing algorithm may be subjected to two-dimensional Fourier transform and inverse transform processing. In Method 4, since processing is performed on image data having a relatively small data size, the time required for processing can be shortened.

  An example of incorporating the four methods described above into algorithm 2 is shown in the flowchart of FIG. The flowchart shown in FIG. 11 shows the processing procedure of S1026 in the flowchart of FIG. 10 in more detail. 11 (a) shows the above method 1, FIG. 11 (b) shows the above method 2, FIG. 11 (c) shows the above method 3, and FIG. 11 (d) shows the above method 4. This corresponds to the flowchart when used as the algorithm 2.

  Referring to FIG. 11A, the CPU 146 issues a command to the image signal processing unit 106 in S1102 to perform resizing processing using a method (for example, the Lanczos method) in which moire is less likely to occur. In response to this instruction, the image signal processing unit 106 performs a resize process of the recorded image data, and generates display image data.

  Referring to FIG. 11B, in step S1104, the CPU 146 issues a command to the image signal processing unit 106 to perform low-pass filtering processing on the recorded image data. The image signal processing unit 106 receives the above command from the CPU 146 and performs low-pass filtering processing on the recorded image data recorded in the RAM 154. In step S <b> 1106, the CPU 146 issues a command to the image signal processing unit 106 to perform the resizing process on the recorded image data. Receiving this command, the image signal processing unit 106 performs a resizing process using a resizing method prepared in advance for the resizing algorithm 2, and generates display image data.

  Referring to FIG. 11C, the CPU 146 issues a command to the image signal processing unit 106 to detect the display image moire area in S1110. Receiving this command, the image signal processing unit 106 detects a display image moire area from the display image data generated by using the resizing algorithm 1 in S1008.

  In S <b> 1112, the CPU 146 issues a command to the image signal processing unit 106 so as to selectively perform low-pass filtering only on the portion of the recorded image data corresponding to the display image moire area. Upon receiving this command, the image signal processing unit 106 performs the above-described selective low-pass filtering on the recorded image data. In S <b> 1114, the CPU 146 issues a command to the image signal processing unit 106 to perform the resizing process on the recorded image data. Receiving this command, the image signal processing unit 106 performs a resizing process using a resizing method prepared in advance for the resizing algorithm 2, and generates display image data.

  Referring to FIG. 11D, the CPU 146 issues a command to the image signal processing unit 106 to perform a two-dimensional Fourier transform process on the display image data in S1120. Receiving this instruction, the image signal processing unit 106 performs a two-dimensional Fourier transform process on the display image data generated using the resizing algorithm 1 in S1008.

  In S <b> 1122, the CPU 146 issues a command to the image signal processing unit 106 to detect and remove moire-derived spectral components from the spatial frequency spectrum obtained by the two-dimensional Fourier transform process. Receiving this command, the image signal processing unit 106 analyzes the spatial frequency spectrum of the display image data, detects a spectral component derived from moire, and performs a process of removing this spectral component.

  In S <b> 1124, the CPU 146 issues a command to the image signal processing unit 106 to perform a two-dimensional inverse Fourier transform process on the spatial frequency spectrum of the display image data from which moire-derived spectral components have been removed as described above. Receiving this command, the image signal processing unit 106 performs a two-dimensional inverse Fourier transform process on the spatial frequency spectrum of the display image data to generate display image data with reduced moire.

  According to the third embodiment of the present invention, a display image generated using the resizing algorithm 1 if the moire generation level of the display image data generated using the resizing algorithm 1 is equal to or lower than a predetermined reference. An image based on the data is displayed. On the other hand, when the moire generation level of the display image data generated using the resizing algorithm 1 exceeds a predetermined reference, an image based on the display image data generated using the resizing algorithm 2 is displayed. By using any one of the above-described methods 1 to 4 as the resizing algorithm 2, it is possible to effectively suppress the occurrence of moire in the display image.

-Fourth embodiment-
Hereinafter, the imaging apparatus 100 according to the fourth embodiment of the present invention will be described focusing on the processing contents of the image signal processing unit 106 and the CPU 146. In the fourth embodiment, recorded image data generated by performing a series of photographing operations is subjected to JPEG compression (encoding) processing, and display image data is generated when it is written to the storage medium 158. When this display image data is subjected to moiré detection processing and the moiré occurrence level exceeds a predetermined reference, the moiré occurrence information and the moiré occurrence level are included in the tag information added to the JPEG-compressed image data. Is recorded.

  When image reproduction display is performed, image data is read from the storage medium 158 together with tag information. When the moire occurrence information and the moire occurrence level are recorded in the tag information, an image reproduction display process is performed as will be described in detail below depending on the case where the moire occurrence information and the moire occurrence level are not recorded.

  Also in the fourth embodiment, the resize processing unit 114 will be described as having five resize algorithms, as in the first embodiment.

  FIG. 12 is a flowchart schematically showing a procedure of image recording processing executed by the CPU 146. The process shown in the flowchart of FIG. 12 is executed when the photographing apparatus 100 is set to the photographing mode and a photographing operation is performed by the user.

  In S1202, the CPU 146 performs exposure control. That is, the CPU 146 issues a control signal to the lens driving unit 142 and performs a series of operation control related to exposure, such as focus adjustment, aperture control, and shutter opening / closing control. In S1204, the CPU 146 issues a command to the image sensor 104 to read out the image signal. Upon receiving this instruction, the image sensor 104 transfers an image signal to the RAM 154.

  In S1206, the CPU 146 issues a command to the image signal processing unit 106 to perform demosaic processing. Receiving this command, the image signal processing unit 106 performs demosaic processing on the image signal temporarily stored in the RAM 154 to generate RGB image data. In S1208, the CPU 146 outputs a command to the image signal processing unit 106 so as to process the RGB image data. In response to this command, the image signal processing unit 106 performs processing such as white balance adjustment, gradation / level correction, unsharp masking, and shading correction on the RGB image data to generate recorded image data.

  In S1210, the CPU 146 issues a command to the image signal processing unit 106 to generate display image data using the resize algorithm 1 from the recording image data obtained in S1208. Receiving this command, the image signal processing unit 106 performs processing for generating display image data using a designated resizing algorithm.

  In step S <b> 1212, the CPU 146 issues a command to the image signal processing unit 106 to perform moiré detection processing. Upon receiving this instruction, the image signal processing unit 106 performs moire detection processing on the reproduced image data generated in S1210, and outputs the moire detection result to the CPU 146.

  In S1214, the CPU 146 determines whether or not the moiré occurrence level of the display image data detected in the process of S1212 is equal to or lower than a predetermined reference based on the moiré detection result output from the image signal processing unit 106. judge. If the determination in S1214 is affirmative, that is, if it is determined that the moiré occurrence level of the display image data generated in S1210 is equal to or less than a predetermined reference, the process proceeds to S1216.

  In step S1216, the CPU 146 issues a command to the image signal processing unit 106 to perform JPEG encoding processing on the recorded image data. Upon receiving this command, the image signal processing unit 106 performs JPEG compression processing on the recorded image data to generate JPEG image data.

  If the determination in S1214 is negative, that is, if it is determined that the moire occurrence level of the display image data detected in the process of S1212 exceeds a predetermined reference, the process proceeds to S1220. In S1220, the CPU 146 records the moire occurrence information and the moire occurrence level in the tag information. When the process of S1220 is completed, the process proceeds to S1216. In step S1216, JPEG compression processing is performed as described above to generate JPEG image data.

  In step S1218, the CPU 146 outputs a command to the image signal processing unit 106 so as to record the JPEG image data together with the tag information in the storage medium 158. In response to this instruction, the image signal processing unit 106 performs processing for recording the JPEG image data together with the tag information in the storage medium 158.

  FIG. 13 is a flowchart schematically showing a procedure of image reproduction display processing executed by the CPU 146. The process shown in the flowchart of FIG. 13 can be executed when the photographing apparatus 100 is set to the image reproduction display mode and the user selects a desired image from the thumbnail images, for example. In FIG. 13, processing steps similar to those performed by the image signal processing unit 106 and the CPU 146 in the photographing apparatus 100 according to the first embodiment of the present invention shown in FIG. Detailed description thereof is omitted. The description will focus on the differences from the first embodiment.

  The JPEG image data read from the storage medium 158 in S602 is subjected to JPEG expansion processing in S604 to generate recorded image data, which is recorded in the RAM 154. In step S1302, the CPU 146 reads tag information from the JPEG image data. In S <b> 1304, the CPU 146 determines whether or not moire occurrence tag information is included in the tag information. If it is determined in S1304 that there is no moire occurrence tag information, the processing in S614 and S616 is executed. A display image based on the display image data generated using the resizing algorithm 1 is displayed on the display element 128.

  If it is determined in S1304 that there is moire occurrence tag information, the process proceeds to S1306. In S1306, the CPU 146 sets a variable n to 1. Thereafter, the loop process formed in S618, S620, S624, S626, and S628 is performed until the variable n becomes 5. As a result, display image data is generated using the resizing algorithms 1 to 5, and a moire detection process is performed.

  In step S <b> 622, the CPU 146 selects display image data generated using the resizing algorithm having the smallest moire generation level among the resizing algorithms 1 to 5 as data to be output to the image display unit 124. The CPU 146 issues a command to display the reproduced image on the image display unit 124 in S616. The image display unit 124 receives this command from the CPU 146, and performs a process of displaying a display image based on the display image data selected in S622 on the display element 128.

  As described above, according to the photographing apparatus according to the fourth embodiment of the present invention, the following operational effects can be obtained.

  When it is determined that there is no moiré occurrence tag information, display image data generation processing using the resizing algorithms 2 to 5 is not performed. For this reason, it is possible to complete the reproduction image display processing within a relatively short time.

  When it is determined that there is moire generation tag information, display image data is generated using resizing algorithms 1 to 5. At this time, the moire occurrence level is detected for each of the generated display image data. Then, the display image data generated using the resizing algorithm having the minimum moire generation level among the resizing algorithms 1 to 5 is selected as data to be output to the image display unit 124 and displayed on the display element 128. Accordingly, since the display image based on the display image data having the smallest moire generation level among the display image data generated using the five resize algorithms is displayed on the display element 128, the user can display the display image with reduced moire. Can be observed.

-Fifth embodiment-
Hereinafter, the photographing apparatus 100 according to the fifth embodiment of the present invention will be described focusing on the processing contents of the image signal processing unit 106 and the CPU 146. In the fifth embodiment as well, the processing shown in the flowchart of FIG. 12 is performed in advance, as in the fourth embodiment. That is, the recording image data generated by performing a series of photographing operations is subjected to JPEG compression (encoding) processing, and display image data is generated when writing to the storage medium 158. When the display image data is subjected to moiré detection processing and the moiré generation level exceeds a predetermined reference, moiré occurrence information and moiré occurrence are included in the tag information added to the JPEG-compressed image data. The level is recorded.

  When image reproduction display is performed, JPEG image data is read from the storage medium 158 together with tag information. When the moire occurrence information and the moire occurrence level are recorded in the tag information, an image reproduction display process is performed as will be described in detail below depending on the case where the moire occurrence information and the moire occurrence level are not recorded.

  Also in the fifth embodiment, similarly to the first embodiment and the fourth embodiment, the resize processing unit 114 will be described as having five resize algorithms. Further, the processing from the start of shooting until the JPEG image is recorded on the storage medium 158 described with reference to FIG. 12 in the fourth embodiment is the same as that in the fifth embodiment, and thus detailed description thereof will be given. Is omitted. Only the procedure for reading and displaying the JPEG image data recorded in the storage medium 158 will be described below.

  FIG. 14 is a flowchart schematically showing a procedure of image reproduction display processing executed by the CPU 146. The process shown in the flowchart of FIG. 14 may be executed when, for example, the photographing apparatus 100 is set to the image reproduction display mode and the user selects a desired image from the thumbnail images. In FIG. 14, processing steps similar to those performed by the image signal processing unit 106 and the CPU 146 in the photographing apparatus 100 according to the first embodiment of the present invention shown in FIG. Detailed description thereof is omitted.

  The JPEG image data read from the storage medium 158 in S602 is subjected to JPEG expansion processing in S604 to generate recorded image data, which is recorded in the RAM 154. In S1402, the CPU 146 reads tag information from the JPEG image data. In step S <b> 1404, the CPU 146 determines whether or not moire occurrence tag information is included in the tag information. If it is determined in S1404 that there is no moire tag information, the CPU 146 sets a variable n to 1 in S1406.

  In step S1408, the CPU 146 issues a command to the image signal processing unit 106 to generate display image data. At this time, the CPU 146 outputs information (algorithm designation information) for designating a resizing algorithm based on the value set in the variable n to the image signal processing unit 106. The image signal processing unit 106 that has received these commands and algorithm designation information performs processing for generating display image data using a designated resizing algorithm (here, resizing algorithm 1 is designated).

  In step S <b> 1410, the CPU 146 selects display image data generated using the resizing algorithm n as data to be output to the image display unit 124. When the process reaches S1410 via S1408, display image data generated using the resizing algorithm 1 is selected in S1410.

  The CPU 146 issues a command to display the reproduced image on the image display unit 124 in S616. The image display unit 124 receives this command from the CPU 146, and performs a process of displaying a display image based on the display image data selected in S622 on the display element 128.

  Returning to the description of the determination process in S1404. If it is determined in S1404 that there is moire occurrence tag information, the CPU 146 sets a variable n to 1 in S1412.

  Thereafter, the loop process formed in S608, S610, S612, S618, S620, and S1414 is performed until the variable n becomes 5. In this process, if it is determined in S612 that the moire occurrence level is below the reference, the process goes out of the loop process, and the process proceeds to S1410. In step S <b> 1410, the CPU 146 selects display image data generated using the resizing algorithm n as data to be output to the image display unit 124. When the process goes from S612 to S1410, the display image data determined in S612 that the moire occurrence level is below the reference is selected in S1410.

  On the other hand, if no moiré occurrence level equal to or lower than the reference level is found until the variable n reaches 5, the process branches from S620 to S622. Then, among the display image data generated using the resizing algorithms 1 to 5, the display image data having the minimum moire generation level is selected in S622, and the process proceeds to S616.

  Subsequently, the process of S616 is performed, and as a result, a display image based on the display image data selected in S622 or S1410 is displayed on the display element 128.

  As described above, according to the photographing apparatus according to the fifth embodiment of the present invention, the following operational effects can be obtained.

  When it is determined that there is no moiré occurrence tag information, display image data generation processing using the resizing algorithms 2 to 5 is not performed as in the fourth embodiment. For this reason, it is possible to complete the reproduction image display processing within a relatively short time.

  If it is determined that there is moiré occurrence tag information, display image data is sequentially generated using resizing algorithms 1 to 5 and moiré detection processing is performed. At this time, when display image data whose moire generation level is determined to be below the reference is found, display image data generation processing using the remaining resize algorithms is terminated. Therefore, it is possible to shorten the time until the reproduction image display processing is completed.

  Further, when the moire generation level detected in all display image data generated using the resizing algorithms 1 to 5 exceeds a predetermined reference, the same as in the second embodiment. That is, display image data having the minimum moire generation level among the display image data generated using the resizing algorithms 1 to 5 is selected as data to be output to the image display unit 124. For this reason, even if any resize algorithm is used to generate moire, it is possible to display a display image based on display image data having the minimum moire generation level.

-Sixth embodiment-
Hereinafter, the imaging apparatus 100 according to the sixth embodiment of the present invention will be described focusing on the processing contents of the image signal processing unit 106 and the CPU 146. In the sixth embodiment, similarly to the fourth and fifth embodiments, the processing shown in the flowchart of FIG. 12 is performed in advance. That is, the recording image data generated by performing a series of photographing operations is subjected to JPEG compression (encoding) processing, and display image data is generated when writing to the storage medium 158. When the display image data is subjected to moiré detection processing and the moiré generation level exceeds a predetermined reference, moiré occurrence information and moiré occurrence are included in the tag information added to the JPEG-compressed image data. The level is recorded.

  When image reproduction display is performed, JPEG image data is read from the storage medium 158 together with tag information. When the moire occurrence information and the moire occurrence level are recorded in the tag information, an image reproduction display process is performed as will be described in detail below depending on the case where the moire occurrence information and the moire occurrence level are not recorded.

  Also in the sixth embodiment, similarly to the third embodiment, the resizing processing unit 114 has two resizing algorithms. Further, the processing from the start of shooting until the JPEG image is recorded on the storage medium 158 described in the fourth embodiment with reference to FIG. 12 is the same as that in the sixth embodiment, and thus the detailed description thereof will be given. Is omitted. Only the procedure for reading and displaying the JPEG image data recorded in the storage medium 158 will be described below.

  FIG. 15 is a flowchart schematically showing a procedure of image reproduction display processing executed by the CPU 146. The process shown in the flowchart of FIG. 15 may be executed when, for example, the photographing apparatus 100 is set to the image reproduction display mode and the user selects a desired image from the thumbnail images. In FIG. 15, processing steps similar to those performed by the image signal processing unit 106 and the CPU 146 in the photographing apparatus 100 according to the first embodiment of the present invention shown in FIG. Detailed description thereof is omitted.

  The JPEG image data read from the storage medium 158 in S602 is subjected to JPEG expansion processing in S604 to generate recorded image data, which is recorded in the RAM 154. In S1502, the CPU 146 reads tag information from the JPEG image data. In S <b> 1504, the CPU 146 determines whether or not moire occurrence tag information is included in the tag information. If it is determined in step S1504 that there is no moire generation tag information, the CPU 146 issues a command to the image signal processing unit 106 to generate display image data using the resizing algorithm 1 in step S1506. In response to this instruction, the image signal processing unit 106 performs processing for generating display image data using the resizing algorithm 1. Subsequent to the processing of S1506, the processing of S614 and S616 is performed, and a display image based on the display image data generated using the resizing algorithm 1 is displayed on the display element 128.

  If it is determined in S1504 that there is moire occurrence tag information, the CPU 146 issues a command to the image signal processing unit 106 to generate display image data using the resizing algorithm 2 in S1508. The image signal processing unit 106 that has received this command performs a process of generating display image data using the resizing algorithm 2.

  In S1510, the CPU 146 selects display image data generated by using the resizing algorithm 2 as data to be output to the image display unit 124, and issues a command to display a reproduced image on the image display unit 124 in subsequent S616. . As a result, a display image based on the display image data generated using the resizing algorithm 2 is displayed on the display element 128. As the resizing algorithm 2, any one of those described with reference to FIGS. 11A to 11D in the third embodiment can be used.

  As described above, according to the photographing apparatus according to the sixth embodiment of the present invention, the following operational effects can be obtained.

  If it is determined that there is no moiré occurrence tag information, display image data generation processing using other resizing algorithms is not performed, and resizing algorithm 1 is used, as in the fourth and fifth embodiments. The display image data is generated. For this reason, it is possible to complete the reproduction image display processing within a relatively short time.

  If it is determined that there is moire generation tag information, an image based on the display image data generated using the resizing algorithm 2 is displayed. By using any one of the resizing algorithms 2 shown in FIGS. 11A to 11D, it is possible to effectively suppress the occurrence of moire in the display image.

  As described above, according to the present invention, it is possible to effectively suppress the occurrence of moire in the display image. Further, when the moire occurrence level of the display image data generated using the resizing algorithm 1 exceeds a predetermined standard, the moire occurrence tag information is recorded together with the moire occurrence level in the tag information. Therefore, it is possible to omit the process of generating display image data using the resizing algorithm 1 and detecting the moire occurrence level. Therefore, it is possible to reduce the time required for the processing until the display image data is generated using the resizing algorithm 2 and the display image is displayed on the monitor display element 128.

  The criteria for determining the moire occurrence level in the first to sixth embodiments of the present invention described above will be described. This determination criterion can be determined in advance based on the number of pixels of the recorded image data, the number of display pixels of the display element 128, the specifications of the photographing apparatus 100, and the like. Alternatively, the user may be configured to be able to set in advance while looking at the sample image displayed on the display element 128.

  As described above, in the first to sixth embodiments, the example in which the present invention is applied to the photographing apparatus has been described. Applicable. Moreover, although the example which displays a still image as a display image was demonstrated, this invention is applicable also when displaying a moving image of a high pixel on a display apparatus with few display pixels.

  The present invention is also applicable to a storage viewer that can store and carry recorded image data. Alternatively, the present invention is applied to a digital photo frame that can be viewed by displaying an image on a liquid crystal display device or the like disposed in the photo frame, instead of the traditional photo frame for viewing by putting a print photo in the photo frame. Is possible. The present invention can also be used for a photographing apparatus such as a digital still camera and a digital video camera, a mobile phone with a photographing function, and the like. Furthermore, the present invention can be applied when displaying a display image on a portable personal computer, a PDA, a portable video game apparatus, or the like having a display element with a relatively small screen size and display pixel number (display resolution). .

It is a block diagram which shows roughly the internal structure of the imaging device to which this invention is applied. It is a figure which illustrates the power spectrum obtained by carrying out two-dimensional Fourier-transform processing of the image and the image data which form the image, (a) is an example of the image which does not produce moiré, (b) is (a) (C) is a figure which shows the example of the image which has produced the moire, (d) is a figure which shows the power spectrum corresponding to the image illustrated in (c). It is a figure which shows the example of an image with a comparatively high moire generation level. It is a figure which shows the power spectrum corresponding to the image of FIG. 3A. It is a figure which shows the example of an image with a moderate moire generation level. It is a figure which shows the power spectrum corresponding to the image of FIG. 4A. It is a figure which shows the example of an image with a comparatively low moire generation level. It is a figure which shows the power spectrum corresponding to the image of FIG. 5A. It is a flowchart explaining the image reproduction display processing procedure performed within the imaging device which concerns on the 1st Embodiment of this invention. It is a figure which illustrates the image before performing a resizing process. It is a figure which shows the example of the image after performing the resizing process, and shows the example with a comparatively high moire generation level. It is a figure which shows the example of the image after performing a resizing process, and shows the example whose moire generation level is comparatively low. It is a figure which illustrates the image before performing a resizing process. It is a figure which shows the example of the image after performing the resizing process, and shows the example with a comparatively high moire generation level. It is a figure which shows the example of the image after performing a resizing process, and shows the example whose moire generation level is comparatively low. It is a flowchart explaining the image reproduction display processing procedure performed within the imaging device which concerns on the 2nd Embodiment of this invention. It is a flowchart explaining the image reproduction display processing procedure performed within the imaging device which concerns on the 3rd Embodiment of this invention. It is a flowchart explaining the process sequence performed as the resizing algorithm 2 in the image reproduction display process sequence performed within the imaging device which concerns on the 3rd Embodiment of this invention. It is a flowchart explaining the JPEG image recording process procedure performed within the imaging device which concerns on the 4th Embodiment of this invention. It is a flowchart explaining the image reproduction display process procedure performed within the imaging device which concerns on the 4th Embodiment of this invention. It is a flowchart explaining the image reproduction display processing procedure performed within the imaging device which concerns on the 5th Embodiment of this invention. It is a flowchart explaining the image reproduction display processing procedure performed within the imaging device which concerns on the 6th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image pick-up device 102 Imaging lens 104 Image pick-up element 106 Image signal processing part 108 Synchronization processing part 110 Image data generation part 112 Display image data generation part 114 Resize processing part 116 Moire detection processing part 122 Timing generation part 124 Image display part 126 Display control Part 128 display element 140 system bus 142 lens driving part 146 CPU
152 Memory control unit 153 Flash memory 154 RAM
156 Interface 158 Storage medium

Claims (6)

A plurality of resizing algorithms are provided for resizing processing, and any one of the plurality of resizing algorithms is used to resize the recorded image data to display image data that matches the number of display pixels of the image display unit. An image reproduction display device that generates and displays a display image based on the display image data on the image display unit,
A moire determination unit that determines the degree of moire occurrence of display image data generated using any of the plurality of resizing algorithms;
A display image data generation unit that generates display image data using any one of the plurality of resizing algorithms and outputs the display image data to the image display unit;
The display image data generation unit
The resizing algorithm in the plurality of resizing algorithms is preferentially executed in a priority order that can be expected to obtain a display image with high resolution, or in a priority order that can be processed in a short time,
When the moire determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm is equal to or less than a predetermined reference, the first resizing algorithm is used. The generated display image data is output to the image display unit,
If the moiré determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm exceeds the predetermined reference, the remaining including the second resizing algorithm The display image data is generated using all of one or a plurality of resizing algorithms, and the degree of moire generation in the generated display image data is determined in the display image data generated using the first resizing algorithm. Including the degree of moire occurrence, and is configured to output display image data generated by the resizing algorithm having the lowest degree of moire generation among the plurality of resizing algorithms to the image display unit.
Or
When the moire determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm is equal to or less than a predetermined reference, the first resizing algorithm is used. The generated display image data is output to the image display unit,
If the moiré determination unit determines that the degree of moiré in the display image data generated using the first resizing algorithm exceeds the predetermined reference, the remaining including the second resizing algorithm The resize algorithm is sequentially used to generate the display image data, and each time the moire determination unit determines the degree of moiré in the display image data, and if the resize algorithm is equal to or less than the predetermined reference, Even if there is a remaining resizing algorithm by outputting the display image data generated using the image display unit, the generation processing of the display image data using them or the same is terminated,
When the display image data generated by using the remaining resize algorithms in sequence exceeds the predetermined standard, the display image data is generated by the resize algorithm having the lowest degree of moire generation among the plurality of resize algorithms. An image reproduction display device configured to output the displayed image data to the image display unit. Two types of resizing algorithms comprising a first resizing algorithm and a second resizing algorithm are provided for resizing processing, and image processing is performed by applying resizing processing to recorded image data using one of the two resizing algorithms. An image reproduction display device that generates display image data suitable for the number of display pixels of the unit and displays a display image based on the display image data on the image display unit,
The first resizing algorithm is a resizing algorithm that can be expected to obtain a display image with a higher resolution than the second resizing algorithm, or a resizing algorithm that can be processed in a shorter time,
A moire determination unit that determines the degree of moire occurrence of display image data generated using the first resizing algorithm;
A display image data generation unit that generates display image data using one of the two types of resizing algorithms and outputs the display image data to the image display unit, the display image generated using the first resizing algorithm When the moire determination unit determines that the degree of occurrence of data moiré is below a predetermined reference, display image data generated using the first resize algorithm is output to the image display unit. On the other hand, when the moire determination unit determines that the predetermined reference is exceeded, display image data is generated using the second resizing algorithm and output to the image display unit And a display image data generation unit.   The second resizing algorithm performs processing for attenuating or removing the spatial frequency component that causes moiré due to the resizing processing in the spatial frequency component of the image based on the recorded image data. The image reproduction display device according to claim 2, further comprising a process of resizing later.   The second resizing algorithm detects a region where moire occurs in a display image based on display image data generated using the first resizing algorithm, and corresponds to a region where moire occurs in the display image. 3. The image reproduction display device according to claim 2, further comprising a step of performing a process of attenuating or removing a spatial frequency component that causes moire due to the resizing process only on the data in the recorded image data. .   The moire determining unit analyzes a spatial frequency spectrum of display image data obtained by performing a two-dimensional Fourier transform process on the display image data to detect a moire-derived spectrum pattern, and from the intensity of the spectrum pattern in the display image The image reproduction display device according to claim 1, wherein the degree of occurrence of moiré is determined. An image data recording processing section for performing processing for recording the recorded image data;
A moire information addition processing unit for performing processing for adding moire information, which is information related to the determination result of the moire determination unit, to the display image data and recording the image data on the image data recording processing unit;
Further comprising
When the recording image data is recorded in the image data recording processing unit, the moire determination unit detects the moire of the display image data generated using the first resizing algorithm, and the generation level of the moire is determined in advance. When the specified reference is exceeded, the moire information addition processing unit adds moire information, which is information related to the determination result of the degree of occurrence of moire, to the recorded image data,
6. The moire determination unit according to any one of claims 1 to 5, wherein the moire generation degree of display image data generated using a first resizing algorithm is determined with reference to the moire information. The image reproduction display device according to one.