JP2011130112A - Display support device and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily adjust focus with high resolution which is originally equipped in an imaging apparatus even when a display system whose resolution is lower than imaging resolution is used. <P>SOLUTION: A display support device includes: a subband division part 105 which extracts a high-resolution component image D105-HL/LH/HH from image data D102 output from an imaging element 102 by using subband conversion in which the bandwidth is halved; and a display signal output part 109 which outputs the high-resolution component image D105-HL/LH/HH as a display signal D109 to be displayed on the low resolution display system 110 whose resolution is lower than that of the image data D102. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display support apparatus suitable for performing focus adjustment using a monitor system having a resolution lower than the imaging resolution, and an image pickup apparatus including the display support apparatus.

In recent years, due to the evolution of imaging devices, imaging devices have been developed that generate images with a resolution equivalent to or exceeding that of movie films. However, in a high-resolution imaging system compatible with 4K or the like, a monitor system that can display a high-resolution image such as 4K at a shooting site becomes huge, and it is not realistic to use it in actual operation. For this reason, an operation style using a small monitor system with a lower resolution such as 2K is often taken at the shooting site. However, since it is a low resolution monitor, it is very difficult to achieve just focus at an original high resolution such as 4K.
Note that 4K is an example of high resolution specifications such as 4096 samples × 2160 lines. 2K is an example of a specification with a resolution lower than 4K, such as 2048 samples × 1080 lines.

For example, in the technique described in Patent Document 1, the focus adjustment of an imaging device having a high-definition video output can be adjusted easily and comfortably while confirming the video of a display device that can handle only a lower-definition video signal. It is described that an enlargement display method is used in order to provide an imaging device that can be enabled.
In this method, an image is cut out from a high-definition video and displayed on a lower-definition display device, so that only a part of the image can be displayed in detail, and high-definition focus adjustment can be performed. However, on the other hand, it is impossible to adjust the focus more precisely while always checking the entire screen.

In the technique described in Patent Document 2, the mixing ratio of detail signals for contour enhancement obtained by a plurality of boost (peak) frequencies can be set in accordance with the resolution of the viewfinder.
According to the technique described in Patent Document 2, it is possible to provide a camera that can be more easily focused in accordance with the resolution of the viewfinder. However, in an environment where only a display device that can handle only a lower definition video signal can be used for high-definition video output, it is limited by the resolution of the display device. Can not be confirmed.

With reference to FIG. 1, an example in which a small monitor system having a lower resolution such as 2K is used in a shooting site in a high-resolution imaging system compatible with 4K or the like will be described.
FIG. 1A shows a 4K frequency band inherent to a high-resolution imaging system. The right side of the horizontal axis indicates a higher-resolution frequency component. The vertical axis represents the intensity (amplitude) of the signal.
In order to display the video signal having 4K resolution using a display device having 2K low resolution, the video signal is reduced and displayed. However, since it is necessary to display after limiting to the 2K frequency band in advance, the outline of the 4K high-frequency component becomes invisible as shown in FIG. 1A. This is clear from the sampling theorem and will not be discussed in detail here.

  Further, in order to focus an image of 4K resolution, as shown in FIG. 1B, using the method disclosed in Patent Document 2, it is used for contour enhancement obtained by a plurality of boost (peak) frequencies in the 4K band. Assume that the detail signal mixing ratio is set. However, as shown in FIG. 1B, the frequency band of the display device corresponding to low-definition 2K is eventually limited. Therefore, all the emphasized frequency components of the video signal having the 4K band are dropped out of the 2K band. Of course, this usage does not correspond to the resolution of the viewfinder to be used, so the technique described in Patent Document 2 is not aimed at.

  Further, as shown in FIG. 1C, the method disclosed in Patent Document 2 is obtained by a plurality of boost (peak) frequencies in the 2K band corresponding to the resolution of a display device that can handle only a low-definition video signal. It is assumed that the mixing ratio of the detail signal for contour enhancement is set. However, as shown in FIG. 1C, the low-resolution 2K signal is merely emphasized. Therefore, it is impossible to confirm the focus at the resolution of the high-definition 4K video, and the focus can be confirmed only at the resolution of the low-definition display device.

JP 2007-336257 A Japanese Patent Application Laid-Open No. 8-16312

  The present invention has been made in view of the above-described problems, and makes it possible to easily perform high-resolution focusing that is inherent to even a display system having a resolution lower than the imaging resolution.

The display support apparatus of the present invention
A subband splitting unit that extracts a high-resolution component image from image data output from the image sensor using a subband splitting conversion method in which the band is halved;
And a display signal output unit that outputs a high-resolution component image extracted by the subband dividing unit as a display signal for display on a display system having a resolution lower than that of the image data.

The imaging device of the present invention is
An image sensor;
A subband splitting unit that extracts a high-resolution component image from the image data output from the image sensor using a subband splitting conversion method with a band of 1/2;
And a display signal output unit that outputs a high-resolution component image extracted by the subband dividing unit as a display signal for display on a display system having a resolution lower than that of the image data.

  According to the present invention, the extracted high-resolution component image is an image whose image size is halved by subband division whose bandwidth is halved, and the high-resolution component image whose image size is halved is And output as a display signal for a low-resolution display system. Therefore, by displaying this display signal on the low-resolution display system, it is possible to easily perform the high-resolution focusing that is inherent to the display system.

  According to the present invention, even when a display system having a resolution lower than the imaging resolution is used, it is possible to easily perform high-resolution focusing that is inherently provided.

It is a figure which shows the frequency band of a high-definition image. 1 is a diagram showing a camera-integrated recording / reproducing apparatus according to an embodiment of the present invention. 3 is a block diagram of a compression / decompression processing unit 106 at the time of compression encoding. FIG. It is a figure which illustrates the division | segmentation level of wavelet transformation. It is a figure which illustrates the division | segmentation level of wavelet transformation. FIG. 6 is a block diagram when the compression / decompression processing unit 106 performs compression decoding. FIG. 6 is a diagram illustrating processing at the time of wavelet transform of the compression / decompression I / F unit 105. FIG. 10 is a diagram illustrating processing at the time of reverse conversion of the compression / decompression I / F unit 105. 3 is a block diagram of a viewfinder signal processing unit 109. FIG.

Hereinafter, an example of an embodiment for carrying out the present invention will be described with reference to the accompanying drawings.
In the following embodiments, a display support system (an example of a display support apparatus) for easily performing focusing at an imaging site is provided. In particular, a high-resolution imaging system compatible with 4K or the like provides a display support system for easily performing high-resolution focusing that is inherent even when a small monitor system having a resolution lower than the imaging resolution is used.

In this display support system, a high-resolution component image is extracted using a subband division conversion method in which the band is halved, such as wavelet conversion. Since the extracted high-resolution component image is an image whose image size is halved by subband division, the high-resolution component image is displayed as it is on the low-resolution display system as it is. I can do it.
For example, when wavelet transform is performed on a 4K image, the image is decomposed into a 2K size low-frequency image and a high-frequency image. Since the high-frequency image is a 2K-sized image having 4K resolution, it can be displayed at the same magnification on a 2K resolution display system. That is, it becomes possible to display a high-frequency component and contour component of 4K resolution on a display system of 2K resolution, and support more detailed focusing.

  This will be explained with reference to FIG. 1. As shown in FIG. 1D, in order to focus an image with 4K resolution, a sub-band division conversion method in which the band is halved, such as wavelet conversion, is used. Used to separate the low resolution component image and the high resolution component image. Among these, the high-resolution component image (shaded portion in the figure) includes 4K resolution focus information. Since the extracted high-resolution component image is an image whose image size is halved by subband division, the high-resolution component image is directly displayed on the 2K low-resolution display system at the same magnification as shown in FIG. 1D. 4K high resolution components can be displayed.

Furthermore, this display support system is realized with a simple hardware configuration that does not have a special contour extraction circuit, using wavelet transform in compression encoding, which is used when a captured image is recorded.
In particular, by highlighting a high-resolution component image, it is possible to easily highlight an original high-resolution contour component and to perform display support specialized for focusing.
In addition, the high resolution component image is emphasized on the low frequency component image and then superimposed, so that display support that enables simultaneous camera work and focusing to determine the composition can be performed.
The high-resolution component image can be selected from horizontal, vertical, and diagonal components, and can be used alone or in combination depending on the subject to be photographed. The photographer can select the appropriate combination.
Further, the high-resolution component image can be displayed after being converted into a luminance component image, or can be displayed as RGB, and can be selected by the photographer according to the photographing conditions.

  FIG. 2 is a block diagram of a camera-integrated recording / reproducing apparatus that handles 4K images and 2K images, which is an embodiment when the present invention is applied. The camera-integrated recording / reproducing apparatus 10 is an example of a camera-integrated recording / reproducing apparatus to which the display support system of the present invention is applied.

In FIG. 2, a lens block 101 controls the aperture and zoom, and forms an optical image on the image sensor unit 102. The image sensor unit 102 converts the optical image input from the lens block 101 into a video digital signal and outputs it as recorded RAW data (D102).
Further, the camera signal processing unit 103 reads recorded RAW data (D102) from the image sensor unit 102 such as a Bayer array or reproduced RAW data (D105) read from the recording media unit 108 and subjected to expansion processing, which will be described later, into an image. As a result, an RGB full 4K image is created (so-called color separation). Then, camera image adjustment such as white balance and brightness is performed, and a recording / playback 4K image (D103) is output to the monitor-out unit 104.
The monitor-out unit 104 outputs a 4K signal to an external 4K monitor or the like.

The compression / decompression I / F unit 105 is an example of a subband division unit that extracts a high-resolution component image from image data using a predetermined subband division conversion method. In this example, recorded RAW data (D102) from the image sensor unit 102 is decomposed into 2K band subband images by wavelet transform.
A 2K band sub-band image that is a low-frequency component in both the horizontal and vertical directions is output as D105-LL.
A 2K band subband image having a high frequency component in the horizontal direction and a low frequency component in the vertical direction is output as D105-HL.
A subband image in the 2K band having a low frequency component in the horizontal direction and a high frequency component in the vertical direction is output as D105-LH.
A 2K band sub-band image that is a high-frequency component in both the horizontal and vertical directions is output as D105-HH.
Details of this process will be described separately.
Further, the compression / decompression I / F unit 105 performs inverse wavelet transform on the subband image (D105-LL / HL / LH / HH) of the 2K band that is read from the recording media unit 108 and decompressed, Output as reproduction RAW data (D105). Details of this will also be described separately.

  The compression / decompression processing unit 106 compresses each 2K band sub-band image (D105-LL / HL / LH / HH) using a compression encoding method, and each corresponding code stream (D106-LL / HL / (LH / HH). Since the compression / decompression I / F unit 105 uses wavelet transform, JPEG2000 or the like using the same wavelet transform is optimal, but other existing compression encoding methods may be used. The compression / decompression processing unit 106 decompresses each compressed subband image data recorded in the recording media unit 108, and outputs the decompressed subband image (D105-LL / HL / LH / HH) in 2K band.

  The recording media interface unit 107 performs an interface for accessing the recording media unit 108 at high speed and reading / writing compressed image data. The recording medium unit 108 is a recording medium for recording and reproducing the image compression data, and a nonvolatile memory such as a flash memory is applied.

  The viewfinder signal processing unit 109 is an example of a display signal output unit that outputs an input image as a display signal for displaying on the display system. Of the 2K subband images, D105-LL is a low-frequency subband image in both the horizontal and vertical directions, and can be monitored as a 2K size RGB full image. Similar to the camera signal processing unit 103, camera image adjustment such as white balance and brightness is performed at 2K. Furthermore, setting information for photographing, peaking processing for facilitating focusing, and the like are performed, and a recording / reproducing 2K image (D109) is output to the viewfinder unit 110. At this time, by using subband images D105-HL / LH / HH other than D105-LL, it is possible to perform focusing of a 4K resolution image with a 2K resolution display device. Details will be described separately.

  The viewfinder unit 110 displays the recorded / reproduced 2K image from the viewfinder signal processing unit 109.

The system control unit 111 has a control software program, and controls the entire camera-integrated recording / reproducing apparatus 10 according to the program. Further, in response to an input from the operation unit 112, each block is connected to a data bus, data is exchanged, and settings and states for photographing are controlled. Further, various settings for focusing and the contour enhancement level are transmitted to the viewfinder signal processing unit 109 and controlled by the photographer's settings from the operation unit 112.
The operation unit 112 receives an operation on the camera-integrated recording / reproducing apparatus 10 and transmits the operation to the system control unit 111 as an electric signal.

  As described above, one of the blocks for realizing the present invention is the compression / decompression I / F unit 105. However, since the wavelet transform is used, the compression / decompression processing unit 106 that similarly performs the compression / decompression using the wavelet transform. First of all, I will explain.

FIG. 3 shows a detailed block diagram of the compression / decompression processing unit 106 during compression encoding.
The wavelet transform unit 32 performs wavelet transform on the 2K band sub-band image D105-LL and outputs a wavelet transform coefficient D32-LL. The wavelet transform unit 32 is usually realized by a filter bank including a low-pass filter and a high-pass filter. Since a digital filter usually has an impulse response (filter coefficient) having a length of a plurality of taps, it is necessary to buffer in advance an input image or coefficient that can be filtered. Similarly, when the wavelet transform is performed in multiple stages, it is necessary to buffer only the coefficients that allow the wavelet transform coefficients generated in the previous stage to be filtered.

Here, the subband image generated by the wavelet transform will be described.
FIG. 4 shows an example of a subband image. In this wavelet transform, the low frequency components are usually repeatedly transformed and divided as shown in FIG. 4, because most of the image energy is concentrated in the low frequency components. This can also be seen from the fact that the sub-band images are formed as shown in the figure as the division level is advanced from division level = 1 shown in FIG. 5A to division level = 3 shown in FIG. 5B. .
The division level of the wavelet transform in FIG. 4 is 3, and as a result, a total of 10 subband images are formed. Here, in FIG. 4, L and H represent the low frequency and the high frequency, respectively, and the numbers before L and H represent the division level. That is, for example, 1LH represents a subband image of division level = 1 in which the horizontal direction is a low frequency and the vertical direction is a high frequency.

Returning to the description of FIG. The wavelet transform coefficient D32-LL is then quantized by the quantizing unit 33 and the quantized coefficient D33-LL is output. As the quantization means here, the scalar quantization used in JPEG2000 may be used. As shown in the following (Equation 1), the value obtained by dividing the wavelet transform coefficient W by the quantization step size Δ may be the value of the quantization coefficient q.
q = W / Δ ··········· (Formula 1)

The quantization step size ΔD37-LL is given from a code amount measuring unit 37 described later.
The quantized coefficient D33-LL is then output to the entropy coding unit 35, and the entropy coding unit 35 performs an operation of compressing the quantized coefficient D33-LL using an arbitrary information source compression unit. As the entropy encoding means, commonly used Huffman encoding (used in MPEG and JPEG, refer to the Huffman encoding table created in advance according to the appearance frequency of the symbols appearing in the data, Code generation method) and arithmetic coding (methods adopted in H.264 and JPEG2000) may be used. At this time, although details are not described here, the quantization coefficient may be combined with EBCOT (Embedded Block Coding with Optimal Truncation) which is entropy coding in bit plane units, as in JPEG2000.

  The result encoded by the entropy encoding unit 35 is output as an encoded code stream D35-LL, becomes an output D106-LL of the compression / decompression processing unit 106, and is also input to the code amount measurement unit 37.

The code amount measuring unit 37 compares the code amount within one frame of the encoded code stream D35-LL with the target code amount D36-LL given from the control unit 36. When the target code amount is likely to be exceeded, the quantization step size D37-LL of the quantization unit 33 is changed to be increased by one step.
On the contrary, when the accumulation of the code amount within one frame of the encoded code stream D35-LL is likely to be less than the target code amount, the quantization step size D37-LL of the quantization unit 33 is decreased by one step. Change as follows.

The above is the operation at the time of compression encoding of the compression / decompression processing unit 106. In addition to the subband image LL component of 2K band, the compression / decompression processing unit 106 receives HL component, LH component, and HH component. The
As for the HL / LH / HH component, similarly to the LL component, the wavelet transform unit 32 may be used to increase the division level. However, in the example of FIG. 3, since the first-level wavelet transform has already been performed in the compression / decompression I / F unit 105, the energy of the image shown in FIG. 4 is concentrated in the low frequency component. In accordance with the normal wavelet transform used, only the low-frequency component subband image D105-LL is repeatedly transformed and divided.

For example, when processing from wavelet transform to entropy coding can be realized by a single circuit by hardware, for example, an existing one is not developed without newly developing a circuit that does not perform wavelet transform. There is no restriction on configuring the compression / decompression processing unit 106 using four circuits in parallel.
In the example of FIG. 3, only the quantization and entropy coding are performed on the subband images D105-HL, D105-LH, and D105-HH in the 2K band other than the LL component, and the wavelet division level is not increased. .

  The subband images D105-HL, D105-LH, and D105-HH in the 2K band other than the LL component are output as encoded code streams D35-HL, D35-LH, and D35-HH, respectively, and the compression / decompression processing unit 106 Outputs D106-HL, D106-LH, and D106-HH. In addition, the code amount measuring unit 37 accumulates the code amount in one frame of each encoded code stream, while the target code amounts D36-HL, D36-LH, D36-HH given from the control unit 36 are Make a comparison. Similarly to the LL component, when the code amount in one frame of the encoded code stream is likely to exceed the target code amount, the quantization step size of the quantization unit 33 is changed to be increased by one step. . On the other hand, when the accumulation of the code amount within one frame of the encoded code stream is likely to be less than the target code amount, the quantization step size of the quantization unit 33 is changed to be one step smaller.

As described above, the code amount for each sub-band image in the 2K band is controlled.
The control unit 36 has been described so that the target code amount for each sub-band image in the 2K band is set in advance in the corresponding code amount measurement unit 37. For example, the code amount integration unit 37 receives the code amount integration from each code amount measurement unit 37. Information on the situation may be sent to the control unit 36, and the target code amount for each subband may be dynamically interchanged and changed in accordance with the respective code amount integration situations. Since a natural image is an image containing a lot of low-frequency components, or conversely an image containing many high-frequency components, optimal code amount control according to the properties of the image is possible.
The above is the description of the operation at the time of compression encoding.

Next, FIG. 6 shows a detailed block diagram of the compression / decompression processing unit 106 at the time of compression decoding, and the operation at the time of compression decoding will be described.
The entropy decoding unit 38 to which the encoded code streams D106-LL, D106-HL, D106-LH, and D106-HH are input performs decoding according to the means corresponding to the entropy encoding described in FIG. As a result of the entropy decoding, quantization coefficients D38-LL, D38-HL, D38-LH, and D38-HH are generated.
The quantization coefficients D38-LL, D38-HL, D38-LH, and D38-HH are converted from the quantization coefficients D38-LL, D38-HL, D38-LH, and D38-HH by the inverse quantization unit 39 to the wavelet transform coefficient D39-. LL, D39-HL, D39-LH, and D39-HH. The inverse quantization means here is the reverse operation of (Equation 1) and can be expressed by the following (Equation 2).
W = q × Δ (Equation 2)
(W is a wavelet transform coefficient, q is a quantization coefficient, Δ is a quantization step size)

The wavelet transform coefficient D39-LL is returned to the 2K band subband image D105-LL of the LL component by the wavelet inverse transform unit 40 and output.
When wavelet subdivision is performed on subband images D105-HL, D105-LH, and D105-HH in 2K bands other than the LL component at the time of encoding, the wavelet inverse transform unit 40 is provided for each subband. Obviously, it is only necessary to return to the subband images of each 2K band.
The above is the description of the operation at the time of compression decoding.

The compression / expansion I / F unit 105, which is one of the blocks for realizing the present invention, will be described.
The present invention can maximize its effect when wavelet transform is used. Therefore, although the basic configuration is the same as that of the wavelet transform unit 32 of the compression / decompression processing unit 106, the processing of the compression / decompression I / F unit 105 will be described again with reference to FIG.
The recording RAW data D102 is input as an input image of the compression / decompression I / F unit 105. Here, a 4096 × 2160 size image will be described.

  First, horizontal wavelet transform is performed on the input image. The wavelet transform is performed using a low-pass filter (LPF: Low Pass Filter) and a high-pass filter (HPF: High Pass Filter). Each of the low-pass filter and the high-pass filter is down-sampled 2: 1 in the horizontal direction (represented by a downward arrow and number 2 in FIG. 7), and is 2048 × 2160 in which the horizontal direction is halved. Low-band sub-band image (L) and high-band sub-band image (H).

Next, vertical wavelet transform is performed on the 2048 × 2160 low-frequency subband image (L). Each of the low-pass filter and the high-pass filter performs a down-sampling of 2: 1 in the vertical direction, and a low-frequency subband image (L, L) D105-LL of 2048 × 1080 in which the vertical direction is halved. It becomes a regional subband image (L, H) D105-LH.
In addition, vertical wavelet transform is performed on the 2048 × 2160 high-frequency subband image (H). Each of the low-pass filter and the high-pass filter is down-sampled 2: 1 in the vertical direction, and the low-pass sub-band image (H, L) D105-HL of 2048 × 1080 whose half size is in the vertical direction is high. It becomes a regional subband image (H, H) D105-HH.

As described above, the compression / decompression I / F unit 105 performs wavelet transform on three components of R / G / B of 4096 × 2160 size, and transforms them into the following four types of subband images of 2048 × 1080 size. .
(1) 2048 × 1080 size subband image (L, L) D105-LL that has passed through a low-pass filter in both the horizontal and vertical directions
(2) 2048 × 1080 size subband image (L, H) D105-LH that has passed through a low-pass filter in the horizontal direction and a high-pass filter in the vertical direction
(3) 2048 × 1080 size subband image (H, L) D105-HL that has passed through a high-pass filter in the horizontal direction and a low-pass filter in the vertical direction
(4) 2048 × 1080 size subband image (H, H) D105-HH that has passed through a high-pass filter in both the horizontal and vertical directions

Next, processing at the time of inverse conversion of the compression / decompression I / F unit 105 will be described again with reference to FIG.
As an input image of the compression / decompression I / F unit 105, a 2048 × 1080 size subband image (D105-LL / HL / LH / HH) read from the recording media unit 108 and subjected to the expansion process is input.

  Then, a subband image (L, L) D105-LL having a size of 2048 × 1080 that has passed through a low-pass filter in both the horizontal and vertical directions, and a 2048 × 1080 through a low-pass filter in the horizontal direction and a high-pass filter in the vertical direction. The wavelet inverse transform in the vertical direction is performed on the 1080-size subband image (L, H) D105-LH. In the inverse wavelet transform, the subband image D105-LL is subjected to 1: 2 upsampling (represented by an upward arrow and the number 2 in FIG. 8) to form a low-pass filter (LPF) and to the subband image D105-LL. LH is upsampled 1: 2 and passed through a high pass filter (HPF). Then, by combining the outputs of the low-pass filter and the high-pass filter, a low-frequency subband image (L) of 2048 × 2160 having a vertical resolution doubled is obtained.

  Further, a 2048 × 1080 size subband image (H, L) D105-HL that has passed through a high-pass filter in the horizontal direction and a low-pass filter in the vertical direction, and 2048 × that has passed through a high-pass filter in both the horizontal and vertical directions. Wavelet inverse transform in the vertical direction is performed on the 1080-size subband image (H, H) D105-HH. In the inverse wavelet transform, the subband image D105-HL is up-sampled 1: 2 to the low-pass filter (LPF), and the sub-band image D105-HH is up-sampled 1: 2 to the high-pass filter (HPF). Pass through. Then, by synthesizing the outputs of the low-pass filter and the high-pass filter, a 2048 × 2160 high-frequency subband image (H) in which the vertical resolution is doubled in size is obtained.

  Next, the wavelet inverse transform in the horizontal direction is performed on the 2048 × 2160 low-frequency subband image (L) and the 2048 × 2160 high-frequency subband image (H) on which the vertical wavelet inverse transform has been performed. . In the inverse wavelet transform, the low-pass subband image (L) is up-sampled by 1: 2, and the low-pass filter (LPF) is up-sampled, and the high-pass sub-band image (H) is up-sampled by 1: 2. Pass through (HPF). Then, by combining the outputs of the low-pass filter and the high-pass filter, 4096 × 2160 reproduction RAW data D105 having a size twice the horizontal direction is obtained.

As described above, the processing at the time of inverse conversion of the compression / decompression I / F unit 105 is performed by reading from the recording media unit 108 and performing the decompression processing on the subband image (D105-LL / HL / LH / D105-size). HH) can be used to perform inverse wavelet transform in the vertical and horizontal directions, thereby decoding into a 4096 × 2160 size image.
The above is the description of the operation at the time of reverse conversion.

Next, the viewfinder signal processing unit 109, which is another block for realizing the present invention, will be described with reference to FIG.
The compression / decompression I / F unit 105 converts the 4K band image into a 2K band subband image, and D105-LL of the subband images is a low band subband image in both the horizontal and vertical directions. The viewfinder unit 110 can display a 2K size RGB full image.
The sub-band image D105-LL is processed by the camera signal processing unit 41 that performs camera image adjustment such as white balance and brightness at 2K in the same manner as the camera signal processing unit 103 in FIG. 2, and the 2K monitor image (D41). To the selection circuit 42. The selection circuit 42 is an example of a selection unit described in the claims. By selecting this signal by the selection circuit 42 and outputting it as a recording / reproduction 2K image (D109) to the viewfinder unit 110, a 2K size RGB full image monitor is possible.

One of the features of the present invention is that the subband images D105-HL / LH / HH other than D105-LL are also used to support focusing of 4K resolution images.
Specifically, wavelet transform is performed on a 4096 × 2160 size image, and a 2048 × 1080 size subband image D105-HL is generated through a high-pass filter in the horizontal direction and a low-pass filter in the vertical direction. Since this subband image D105-HL includes a horizontal outline component, this is used.
However, since the subband image D105-HL includes a high frequency component, it also includes a lot of noise components. Therefore, first, the noise component is removed by the coring circuit 43. As shown in the figure, the coring is to remove a minute amplitude such as a noise component so that a signal less than ± Coring Level is forcibly output as 0 with respect to the input signal. By performing the coring, it is possible to prevent the amplification circuit 44 described later from amplifying noise components.

The Coring Level can be adjusted by the photographer operating the volume knob or the like of the operation unit 112 in FIG. From the system control unit 111 in FIG. 2, control data indicating Coring Level adjusted by the operation unit 112 is transmitted to the coring circuit 43.
The amplification circuit 44 increases the gain of the subband image including the high frequency component for the purpose of enhancing the contour component of the image and facilitating focusing.
The gain (Gain Level) can be adjusted by the photographer operating the volume knob or the like of the operation unit 112 in FIG. Control data indicating Gain Level adjusted by the operation unit 112 is transmitted to the amplifier circuit 44 from the system control unit 111 in FIG.

In addition to the subband image D105-HL, the 2048 × 1080 size subband image D105-LH having a low-pass filter in the horizontal direction and a high-pass filter in the vertical direction contains many contour components in the vertical direction. Further, the 2048 × 1080 size subband image D105-HH that has passed through the high-pass filter in both the horizontal direction and the vertical direction includes a large number of diagonal contour components.
Similarly, the subband images including these high frequency components are processed by the coring circuit 43 and the amplification circuit 44 for the purpose of facilitating the focusing.

The selection & mix & matrix circuit 45 is an example of a selection / addition circuit that selects any one of the subband images input from each amplification circuit 44 or adds all of them. In this circuit, a high frequency component that is most easily focused from three types of 2048 × 1080 size subband images D105-HL / LH / HH processed by the coring circuit 43 and the amplification circuit 44 according to the subject. Select a subband image containing.
For example, D105-HL including a large amount of contour components in the horizontal direction is suitable for a subject (vertical stripe pattern or the like) having a resolution in the horizontal direction (high frequency component) and a pattern in the vertical direction. On the other hand, D105-LH including a large amount of contour components in the vertical direction is suitable for a subject having a resolution (high frequency component) in the vertical direction and having a pattern in the horizontal direction (such as a horizontal stripe pattern). For an object having a resolution in the oblique direction (high frequency component) and a pattern in the oblique direction, D105-HH including a large number of contour components in the oblique direction is preferable.

  The selection & mix & matrix circuit 45 not only selects one of the three types of subband images including the high frequency components, but also uses all three types of subband images and mixes them (pixels). By adding), almost any subject can be handled. This is effective in that it does not take the trouble of selecting the photographer.

  Further, in the selection & mix & matrix circuit 45, in addition to selecting or mixing the subband image with the RGB signal (three primary color signals) as it is, the internal matrix circuit can also convert the RGB signal to the luminance signal. To do. This is because in many natural images, the contour component is represented by a change in luminance component.

  The photographer operates the switch or the like of the operation unit 112 in FIG. 2 to select the subband image including the high frequency component, the mix, or the luminance component to output and display the contour information for focusing. By doing so, selection is possible. From the system control unit 111 in FIG. 2, control data indicating the selection result in the operation unit 112 is transmitted to the selection & mix & matrix circuit 45.

  The contour-enhanced image (D45) output from the selection & mix & matrix circuit 45 in this way is sent to the selection circuit 42. By selecting the contour-enhanced image (D45) by the selection circuit 42 and outputting it as a recorded / reproduced 2K image (D109) to the viewfinder unit 110, the high resolution of the entire 4K resolution image is output to the 2K resolution viewfinder unit 110. Ingredients can be displayed. In this case, since only the outline-enhanced image is displayed, it is preferable when focusing on focusing. On the other hand, when the focus is not achieved, a clear contour component cannot be obtained, and there is a concern that it is difficult to confirm the entire framing.

  Therefore, a mix circuit 46 as an adder circuit is provided. The mix circuit 46 mixes (adds pixels) a 2K monitor image (D41) suitable for framing and monitoring and a contour-enhanced image (D45) including many contour components suitable for focusing. As a result, a contour component is highlighted on a normal image, and an image signal (D46) suitable for both framing and focusing support can be generated. At this time, the mix ratio between the two can be adjusted by the photographer operating the volume of the operation unit 112 in FIG. From the system control unit 111 in FIG. 2, control data indicating the mix ratio adjusted by the operation unit 112 is transmitted to the mix circuit 46.

  In the selection circuit 42, as the recording / playback 2K image (D109) output to the viewfinder unit 110, only the framing image (D41), only the contour emphasis image (D45) for focusing, or an image obtained by mixing both ( D46) is selected. At this time, which image the selection circuit 42 selects and outputs can be selected by the photographer operating a switch or the like of the operation unit 112 in FIG. From the system control unit 111 in FIG. 2, control data indicating the selection result in the operation unit 112 is transmitted to the selection circuit 42.

  In the selection circuit 42, an evaluation value calculation circuit 47 is provided as an evaluation value calculation unit. The evaluation value calculation circuit 47 calculates a focus evaluation value used in, for example, the autofocus technique based on the contour emphasis image (D45) output from the selection & mix & matrix circuit 45.

In the selection circuit 42, an automatic mode for automatically switching the recording / playback 2K image (D109) to be output to the viewfinder unit 110 can be selected by operating the switch of the operation unit 112 in FIG. is there.
When control data indicating that the automatic mode has been selected is transmitted from the system control unit 111, the selection circuit 42 has reached the value at which the focus evaluation value calculated by the evaluation value calculation circuit 47 indicates the in-focus state. If not, the contour-enhanced image (D45) is output as a recorded / reproduced 2K image (D109). On the other hand, when the focus evaluation value reaches a value indicating the in-focus state, the selection circuit 42 outputs the framing image (D41) as a recording / reproduction 2K image (D109).

As a result, if the automatic mode is selected prior to the start of shooting, the contour-enhanced image (D45) is automatically generated in a state where focus is not achieved even if the photographer does not operate the operation unit 112 during shooting. Since the image is displayed on the viewfinder unit 110, the photographer can concentrate on focusing. On the other hand, in the focused state, the display of the viewfinder unit 110 automatically switches to the framing image (D41), so that the photographer can concentrate on the camera work for determining the composition.
When the automatic mode is selected, if the focus evaluation value does not reach the value indicating the in-focus state, a clear contour component cannot be obtained, and therefore it is difficult to confirm the entire framing. Therefore, the contour-enhanced image (D45) Instead of outputting the mixed image (D46) as a recording / reproducing 2K image (D109), the operation unit 112 may be selected.

According to the camera-integrated recording / playback apparatus having the above-described configuration, a high-resolution component (contour component) image is extracted using wavelet transform, and is directly applied to the viewfinder unit 110, which is a low-resolution display system. By displaying, it becomes possible to display a high resolution component.
In addition, by selecting the optimum contour component image according to the subject or mixing them and displaying them on the viewfinder unit 110, the 4K resolution image can be easily focused on the 2K resolution display device. A display support system that can do this can be provided.
Furthermore, since there is a concern that it is difficult to confirm the entire framing in the state where the focus is not achieved only by the contour component image, a signal containing a large amount of the contour component is mixed with the image for framing and monitoring. Accordingly, it is possible to provide a display system that can perform both framing and support for focusing.
The wavelet transform is a conversion method used by the compression / expansion I / F unit 105 in the compression process, does not generate a special additional circuit, and normally uses a high frequency component that is not used as a display. Can be realized with the system.

  As described above, according to the present invention, in a high-resolution imaging system such as 4K, display support for easily performing high-resolution focusing that is inherent to a small monitor system having a resolution lower than the imaging resolution. A system can be provided. Therefore, it is possible to reduce the size of equipment at the shooting site and to perform high-resolution focus confirmation even with low-budget monitor equipment.

In the above embodiment, wavelet transform is used as a subband division conversion method in which the band is halved, but a conversion method other than wavelet conversion (for example, Haar transform) may be used.
In the above embodiment, the present invention is applied to a camera-integrated recording / reproducing apparatus. However, the present invention can also be applied to a camera (imaging apparatus) that is not integrated with a recording / reproducing apparatus. .
The present invention can also be applied to a camera-integrated recording / playback apparatus and camera in which a low-resolution display system corresponding to the viewfinder unit 110 is not integrated with the camera body (attached to the camera body detachably). it can.
Further, according to the present invention, in a device that is not equipped with an image sensor, when a detachable image sensor is attached to the device, the image data is output from the attached image sensor and input to the device. The present invention can also be applied to focus using a low-resolution display system.

  Further, the present invention is not limited to the above-described embodiments, and it is needless to say that other various application examples and modifications can be taken without departing from the gist of the present invention described in the claims.

  DESCRIPTION OF SYMBOLS 10 ... Camera integrated recording / reproducing apparatus, 32 ... Wavelet transformation part, 33 ... Quantization part, 35 ... Entropy encoding part, 36 ... Control part, 37 ... Code amount measurement part, 38 ... Entropy decoding part, 39 ... Inverse Quantization unit, 40 ... Wavelet inverse transformation unit, 41 ... Camera signal processing unit (2K), 42 ... Selection circuit, 43 ... Coring circuit, 44 ... Amplification circuit, 45 ... Selection & mix & matrix circuit, 46 ... Mix circuit , 47 ... evaluation value calculation circuit, 101 ... lens block, 102 ... image sensor unit, 103 ... camera signal processing unit, 104 ... monitor-out unit, 105 ... compression / decompression I / F unit, 106 ... compression / decompression processing unit, 107 ... Recording media interface unit 108... Recording media unit 109 109 Viewfinder signal processing unit 110 110 Viewfinder unit 111 System control unit, 112 ... operating unit, D102 ... recording RAW data, D 105-LL ... subband image, D 105-HL ... subband images, D 105-LH ... subband images, D 105-HH ... subband image

Claims (8)

A subband splitting unit that extracts a high-resolution component image from image data output from the image sensor using a subband splitting conversion method in which the band is halved;
A display support apparatus comprising: a display signal output unit that outputs a high-resolution component image extracted by the subband dividing unit as a display signal for displaying on a display system having a resolution lower than that of the image data. The display signal output unit
An addition unit that adds the high-resolution component image extracted by the subband division unit and the low-resolution component image extracted by the subband division unit;
The display support apparatus according to claim 1, further comprising: a selection unit that can select both the image added by the addition unit and the high-resolution component image extracted by the subband division unit as the display signal. The display support apparatus according to claim 2, wherein the adding unit adds the high-resolution component image and the low-resolution component image at a specified ratio. The display signal output unit
Of the high resolution component images extracted by the subband splitting unit,
A high-resolution component image that passes through a low-pass filter during horizontal sub-band division by the sub-band division unit and passes through a high-pass filter during vertical sub-band division;
Or a high-resolution component image that passes through a high-pass filter during horizontal sub-band division by the sub-band division unit and passes through a low-pass filter during vertical sub-band division;
Alternatively, either the horizontal subband division by the subband division unit or the vertical subband division is selected for selecting either one of the high-resolution component images that have passed through the high-pass filter or adding all of them. The display support apparatus according to claim 2, further comprising an addition unit. The display signal output unit
The display support apparatus according to claim 4, further comprising a matrix unit that converts the high-resolution component image extracted by the subband dividing unit from a three primary color signal into a luminance signal. The display signal output unit
An evaluation value calculation unit that calculates a focus evaluation value of the high-resolution component image extracted by the subband division unit;
When the focus evaluation value calculated by the evaluation value calculation unit has not reached a value indicating an in-focus state, the selection unit outputs the high-resolution component image extracted by the subband division unit as the display signal. The display support apparatus according to claim 2, wherein when the focus evaluation value reaches a value indicating an in-focus state, the low-resolution component image extracted by the subband dividing unit is output as the display signal. The display support apparatus according to claim 1, wherein the subband division unit uses wavelet transform. An image sensor;
A subband splitting unit that extracts a high-resolution component image from the image data output from the image sensor using a subband splitting conversion method with a band of 1/2;
An imaging apparatus comprising: a display signal output unit that outputs a high-resolution component image extracted by the subband dividing unit as a display signal for displaying on a display system having a resolution lower than that of the image data.