JP5924932B2 - Image processing apparatus and control method thereof - Google Patents

Description

  The present invention relates to an image processing apparatus and a control method thereof.

In recent years, displays that can display stereoscopic images (three-dimensional images, three-dimensional images, three-dimensional images) from observers have become widespread in stereoscopic movie systems, commercial monitors, amusement game machines, CG creation, medical sites, etc. It's getting on. With respect to conventional techniques for improving the image quality of non-stereoscopic images (two-dimensional images, two-dimensional images), there is a need to correspond to stereoscopic images.
In a stereoscopic image, different images are used for the left and right eyes (hereinafter, an image entering the left eye is referred to as an L image, and an image entering the right eye is referred to as an R image), thereby allowing the viewer to perceive parallax and create a stereoscopic effect. ing.

  In the conventional high image quality processing for 2D images, for example, as shown in FIG. 11A, a frequency distribution such as image color and brightness is obtained by hardware (H / W) processing, and information on the frequency distribution is obtained. Based on the above, gamma correction of the image is adaptively performed by firmware (F / W) processing. In the example of FIG. 11A, 60 frames of image data are input per second, and image data reflecting the calculation result of the image quality improvement processing is output at a timing delayed by 2 frames (1/30 seconds) from the input. Is done.

  The average luminance level and dynamic range of the input L image and R image of the stereoscopic image are individually acquired, and based on this, image processing is performed on the L image or R image to match the characteristics of the L image and R image. A technique has been proposed (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2-058933

  There is a stereoscopic image transmission method in which an L image and an R image are transmitted in a frame sequential manner. FIG. 11B is a diagram illustrating an example in which high-quality processing such as gamma correction is performed on a stereoscopic image transmitted by such a frame sequential method based on the frequency distribution of each of the L image and the R image. In the example of FIG. 11B, 60 frames of image data for each of the L image and the R image are input per second, for a total of 120 frames. As in the case of FIG. 11A, it is assumed that the image after the image quality enhancement processing is output at a timing delayed by 1/30 second from the input. That is, the image data to which the image quality improvement processing calculation is applied is output at a timing delayed by 4 frames for the L image and the R image. For this purpose, for example, when the H / W acquires the frequency distribution 1L of the L image 1, the F / W sets the frequency distribution 1L of the L image 1 to H at the timing (after one frame) when the next R image 1 is input. / W and the image quality enhancement processing calculation is performed using the frequency distribution 1L. At this time, the F / W processing must be completed before the next L image 2 is input (after two frames). That is, it is necessary to perform the image quality improvement processing calculation within the time of one frame.

However, if the number of pixels of the L image and the R image is the same as the number of pixels of the 2D image in FIG. 11A, for example, the time 1302 required for the high image quality processing of the L image 1 is as shown in FIG. This is the same as the time 1304 required for the high image quality processing calculation of the image of one frame of the 2D image shown in FIG. In this case, as shown in FIG. 11B, the time required for the image quality improvement processing calculation may exceed the time of one frame. Then, as shown in FIG. 11B, the image quality enhancement processing calculation of the L image 1 is not completed before the input timing of the next L image 2. Then, the start timing of the image quality improvement processing calculation is gradually delayed, and at the stage of the L image 2, there is a state (overflow 1303) in which the image quality improvement processing operation is not completed even when the image output timing arrives. End up. In order to avoid overflow, the period of 1 frame (1/120 seconds) in FIG. 11B is half of the period of 1 frame (1/60 seconds) in FIG. Therefore, it is necessary to use an expensive CPU having a processing capacity twice that in the case of A).

  On the other hand, as shown in FIG. 11C, H / W acquires the frequency distribution for two images of L image and R image and writes them to the memory, and F / W reads the L image and R image from the memory. It is conceivable to read the frequency distribution information for two images and perform a high image quality calculation of the L and R images. In this case, overflow can be avoided if the image quality enhancement processing operation for the L image and the R image is completed within the time of 2 frames (1/60 seconds), so that the required performance to the CPU is higher than in the case of FIG. Can be lowered.

  However, since the F / W needs to read the frequency distribution information for two images of the L image and the R image from the memory, the time required for reading the frequency distribution information is one image of only the L image or only the R image. This is twice as much as when the frequency distribution is read out. Therefore, in some cases, there is a possibility that the image quality improvement processing calculation is not completed within the time of 2 frames. Further, since it is necessary to store the frequency distribution information for two images of the L image and the R image in the memory, the necessary memory capacity increases, which causes an increase in cost.

  Therefore, the present invention provides a technique capable of avoiding overflow without increasing CPU performance or memory capacity in image processing of a stereoscopic image in which L images and R images are input in a frame sequential manner. Objective.

The present invention includes an input means for the left eye images and the right-eye image picture are successively input alternately,
The left eye image and the right eye image that are continuously input are each divided into a plurality of blocks, and the frequency distribution for each gradation of each block of the left eye image and the right eye image is added to each block. and the frequency distribution acquisition means that Tokusu taken for,
Detecting means for detecting a position of an object image included in each of the left-eye image and the right-eye image;
Applying the in the sum frequency distribution of each block, the frequency is, the blocks of the basis of the rupees click tone exceeds a threshold determined in accordance with the position of the object image, the left eye image and the right-eye image Determining means for determining a gradation conversion function for
Wherein said each block of the left-eye image and the right-eye image, and facilities to the image processing means the image processing using the gradation conversion function corresponding,
An image processing apparatus comprising:

The present invention includes an input step of left eye image and the right-eye image picture are successively input alternately,
The left eye image and the right eye image that are continuously input are each divided into a plurality of blocks, and the frequency distribution for each gradation of each block of the left eye image and the right eye image is added to each block. and the frequency distribution acquisition step that Tokusu taken for,
A detection step of detecting a position of an object image included in each of the left-eye image and the right-eye image;
Applying the in the sum frequency distribution of each block, the frequency is, the blocks of the basis of the rupees click tone exceeds a threshold determined in accordance with the position of the object image, the left eye image and the right-eye image A determination step of determining a gradation conversion function for
Wherein said each block of the left-eye image and the right-eye image, and facilities to the image processing step the image processing using the gradation conversion function corresponding,
A control method for an image processing apparatus, comprising:

  According to the present invention, in image processing of a stereoscopic image in which L images and R images are input in a frame sequential manner, overflow can be avoided without increasing CPU performance or memory capacity.

1 is a block diagram illustrating a schematic configuration of a stereoscopic image display device according to Embodiment 1. FIG. 3 is a flowchart illustrating a high quality image processing of a stereoscopic image according to the first embodiment. The figure which shows frequency distribution of L image and R image, and frequency distribution which added them together The figure which shows notionally the flow of the image quality improvement process in Example 1. The figure which shows the example in case an image exists in a different block by L image and R image The figure which shows frequency distribution of L image and R image, and frequency distribution which added them together FIG. 6 is a block diagram illustrating a schematic configuration of a stereoscopic image display apparatus according to the second embodiment. 7 is a flowchart illustrating a high quality image processing of a stereoscopic image according to the second embodiment. Conceptual diagram of parallax measurement of L and R images The figure which shows frequency distribution of L image and R image, and frequency distribution which added them together A diagram conceptually showing the flow of high image quality processing in the prior art The figure which shows the case where the image of an object exists in a block boundary by L image and R image FIG. 12 is a diagram showing a method of lowering the peak detection threshold of block 1 in FIG. FIG. 12 is a diagram showing a method for lowering the peak detection threshold of block 2 in FIG. FIG. 12 is a diagram showing a method of lowering the peak detection threshold of block 3 in FIG.

Example 1
Embodiment 1 of the present invention will be described below.
In the first embodiment, the frequency distributions of the L image and the R image of the stereoscopic image are added together by H / W, and the combined frequency distribution is stored in a memory (SRAM), and the F / W is based on the combined frequency distribution. R image quality enhancement processing is performed. According to this, the capacity of the memory may be equal to the capacity for storing the frequency distribution information for one image, and the time required for the F / W to read the frequency distribution information from the memory is also equivalent to one image. This is equivalent to the time required to read the frequency distribution information. Therefore, it is possible to suppress an increase in the time required for the image quality improvement processing calculation due to the time for reading the frequency distribution information from the memory. Details will be described below.

  FIG. 1 is a block diagram illustrating a functional configuration of the image processing apparatus 100 according to the present embodiment. In this embodiment, the input unit 101, the decoding unit 102, the frequency distribution acquisition unit 103, the image processing unit 105, and the memory 106 in FIG. 1 are configured by H / W, and the image quality improvement processing calculation unit 104 performs processing by F / W. Shall be. F / W processing is software processing performed by the control unit 107 (CPU) reading and executing a program stored in a ROM (not shown) or the like in a RAM (not shown). The control unit 107 controls operations of the input unit 101, the decoding unit 102, the frequency distribution acquisition unit 103, the image processing unit 105, and the memory 106 that constitute the H / W part. Hereinafter, description will be made with reference to the block diagram of FIG.

  The image processing apparatus 100 inputs image data of a stereoscopic image composed of an L image and an R image from an external connection device or the like (not shown). The image data is input to the input unit 101 and then decoded by the decoding unit 102. The decoded image data is transferred to the frequency distribution acquisition unit 103 via the internal bus. As the image data, left-eye image data (L image data) and right-eye image data (R image data) are alternately input.

  The frequency distribution acquisition unit 103 acquires a frequency distribution for each gradation of the L image and the R image that are continuously input. The frequency distribution acquisition unit 103 acquires the frequency distribution of the L image and the frequency distribution of the R image from the image data of the L image and the R image that are alternately input continuously, and adds the frequency distribution of the L image and the R image. The information on the combined frequency distribution is acquired and stored in the memory 106. The image data of the L image and the R image for which the frequency distribution is acquired by the frequency distribution acquisition unit 103 is input to the image processing unit 105.

  The image quality improvement processing calculation unit 104 acquires the information of the combined frequency distribution calculated by the frequency distribution acquisition unit 103 by reading it from the memory 106, and performs an operation for improving the image quality of the image using the combined frequency distribution. Process. For example, the image quality improvement processing calculation unit 104 detects a gradation whose frequency exceeds a predetermined threshold in the combined frequency distribution as a peak gradation, and based on the detected peak gradation, the input gradation of the L image and the R image is determined. A gradation conversion function for converting to output gradation is determined. The F / W sets the calculation result in a register included in the image processing unit 105 to execute image quality enhancement processing.

  The image processing unit 105 reads out the image quality improvement processing calculation result calculated by the image quality improvement processing calculation unit 104 from the register, and based on the register value, the image data of the input L image and R image Perform image processing. The image processing unit 105 outputs the image data of the L image and the R image subjected to the image processing. The output destination is an image display device such as a liquid crystal display device that displays an image based on the image data, a storage device that stores the image data, an image processing device that performs other image processing on the image data, and the like.

FIG. 2 is a flowchart showing image quality improvement processing of the image processing apparatus 100 according to the first embodiment of the present invention.
In step S <b> 201, the decoding unit 102 inputs L image data of the stereoscopic image data obtained by decoding to the frequency distribution acquisition unit 103.
In S202, the frequency distribution acquisition unit 103 acquires the frequency distribution of the input L image data.
In step S <b> 203, the frequency distribution acquisition unit 103 stores the acquired frequency distribution information of the L image data in the memory 106. The frequency distribution acquisition processing of the L image data is performed from S201 to S203.

Next, in S <b> 204, the decoding unit 102 inputs R image data among the three-dimensional images obtained by decoding to the frequency distribution acquisition unit 103.
In S205, the frequency distribution acquisition unit 103 acquires the frequency distribution of the input R image data.
In S206, the frequency distribution acquisition unit 103 adds the frequency distribution of the acquired R image data with the frequency distribution of the L image data stored in S203, and calculates a combined frequency distribution.

In S <b> 207, the frequency distribution acquisition unit 103 stores the information on the combined frequency distribution calculated in S <b> 206 in the memory 106.
In S208, the image quality improvement processing calculation unit 104 reads the information of the combined frequency distribution stored in S207 from the memory 106, and performs the image quality improvement processing calculation. Examples of the high image quality processing calculation include a process of calculating a γ curve based on a combined frequency distribution.
In step S209, the image processing unit 105 performs image processing on the input image data based on the calculation result in step S208. As a result, adaptive image quality enhancement processing is performed on the input image data.

FIG. 3 is a diagram illustrating an application example of the image processing according to the first embodiment to an actual image.
In the example of FIG. 3, the L image 301 and the R image 302 are images in which two low-brightness objects (objects) are drawn on a high-brightness background image. In the graph of the frequency distribution in FIG. 3, the horizontal axis indicates the gradation, and the vertical axis indicates the frequency at each gradation. In the frequency distribution 303 of the L image and the frequency distribution 304 of the R image, there are peaks corresponding to the exhaust picture image and the object image in the high luminance region and the low luminance region, respectively. It can also be seen that the same tendency is seen in the combined frequency distribution 305 obtained by adding up the frequency distribution 303 of the L image and the frequency distribution 304 of the R image.

  In the image display apparatus of this embodiment, a combined frequency distribution 305 obtained by adding the frequency distribution 303 of the L image and the frequency distribution 304 of the R image is stored in the memory, and the F / W reads the combined frequency distribution 305 from the memory, Based on this, high quality processing of the L image and the R image is performed.

FIG. 4 is a diagram conceptually showing a flow of image processing in the first embodiment. In FIG. 4, the horizontal axis represents time.
As shown in FIG. 4, in this embodiment, it is assumed that the stereoscopic image is input to the image processing apparatus 100 by a frame sequential method in which an L image and an R image are 60 frames each per second and a total of 120 frames. Therefore, it takes 1/120 seconds from the input of the L image to the input of the next R image. In this embodiment, it is assumed that the image data after the image quality enhancement processing is output at a timing delayed by 4 frames after the image data is input, that is, at a timing delayed by 1/30 second after the input. In the H / W process, the frequency distribution of the L image and the R image is acquired, the combined frequency distribution is calculated, and the combined frequency distribution is stored in the memory. In the F / W process, the total frequency distribution is acquired from the memory, and the image quality improvement processing calculation is performed based on the total frequency distribution.

  As shown in FIG. 4, the H / W acquires the frequency distribution 1L of the L image 1, acquires the frequency distribution 1R of the input R image 1 and stores the information of the combined frequency distribution 1S obtained by adding them in the memory. To do. F / W reads the information of the combined frequency distribution 1S from the memory at the timing when the next L image (L image 2) is input, and performs an image quality improvement processing calculation based on the combined frequency distribution 1S. In order to prevent this F / W processing from overflowing, within a period of 2 frames (1/60 seconds) from the timing of reading the information of the combined frequency distribution 1S to the timing of inputting the next L image (L image 3) It suffices to complete the image quality improvement processing calculation. The time required for the F / W to read the information of the combined frequency distribution 1S from the memory is equivalent to the time required to read the frequency distribution information for one image of the L image or the R image from the memory. Therefore, the high image quality processing calculation of the present embodiment has the same load as that in the case of performing the image processing of the 2D image as shown in FIG. 11A and needs only to be executed within the same time. Even if a CPU having a processing performance equivalent to that in the case of FIG.

  By controlling the image processing of the image processing apparatus 100 as described above, the information on the frequency distribution of the two images of the L image and the R image is not acquired and held for the stereoscopic image, but the L image and the R image are stored. The information of the combined frequency distribution obtained by adding the frequency distributions is acquired and held. Since the image quality enhancement processing of the L image and the R image is performed using the information of the combined frequency distribution of the F / W, the CPU and the capacity of the memory have the same processing performance as the image enhancement processing of the 2D image. Overflow can be avoided.

(Example 2)
In the second embodiment, a case will be described in which an image is divided into a plurality of blocks and image quality improvement processing is performed according to the frequency distribution of each block. In particular, in this embodiment, a process (dynamic γ process) for changing the gradation conversion characteristics according to the frequency distribution of each block will be described.

In the first embodiment, the method of adding the frequency distributions of the L image and the R image and performing the image quality enhancement processing calculation on the basis of the obtained combined frequency distribution has been described. In the present embodiment, the peak of the frequency distribution for each block is further detected and dynamic γ processing is performed. For this reason, in this embodiment, in addition to the summation of the frequency distributions, for example, the block in which an image that can affect the peak of the frequency distribution, such as the object image in FIG. It is determined whether or not.

FIG. 5 is a diagram illustrating an example of a case where a block in which an image that can affect the peak of the frequency distribution in the L image and the R image is different between the L image and the R image.
In the example of FIG. 5, as in FIG. 3, images of low-luminance objects A and B exist in a high-luminance background image. Each of the L image and the R image is divided into a total of 9 blocks of 3 horizontal blocks × 3 vertical blocks. The numbers (1 to 9) given to the respective blocks in the L image and the R image indicate block division identifiers.

  The images of the object A and the object B in FIG. 5 have parallax so that they can be seen stereoscopically. Therefore, depending on the position of the image of the object, there are cases where the blocks in which the image exists are different between the L image and the R image due to parallax. In the example of FIG. 5, the image of the object A has a small parallax and is located at the center portion of the block 1, so that the L image and the R image belong to the same block 1. On the other hand, since the image of the object B has a large parallax and is located near the boundary of block division, it belongs to the block 2 in the L image and to the block 3 in the R image.

In the case of the image shown in FIG. 5, let us consider adding the frequency distribution of the L image and the R image for each block.
FIG. 6 conceptually shows a process of adding the frequency distribution of the L image and the R image shown in the example of FIG. 5 for each block and finding the peak of the frequency distribution used in the dynamic γ processing based on the combined frequency distribution. FIG. The peak of the frequency distribution is a portion (reference numeral 601 in FIG. 6) where the peak is locally peaked in the combined frequency distribution of FIG. In the dynamic γ processing of this embodiment, a threshold (reference numeral 602 in FIG. 6) is provided, and a gradation having a frequency equal to or higher than the threshold is detected as a peak gradation. In this embodiment, the detected peak gradation is determined as the gradation of the object of interest (hereinafter referred to as the attention gradation), and gradation conversion is performed so that many gradations are assigned to the attention gradation. A γ curve that is a gradation conversion function is dynamically generated.

  In the case of a stereoscopic image, like the image of the object B in FIG. 5, the image of the same object may exist in different blocks due to the parallax between the L image and the R image. In such a case, the frequency distribution of the L image and the R image is summed up. For example, in the case of the block 2, the frequency distribution of the L image has a peak corresponding to the image of the object B, but the frequency distribution of the R image has that peak. There are no peaks. Similarly, in the case of block 3, the frequency distribution of the R image has a peak corresponding to the image of the object B, but the frequency distribution of the L image has no peak. Therefore, neither the sum frequency distribution of block 2 nor the sum frequency distribution of block 3 has a peak corresponding to the image of the object B exceeding the threshold value 602, and as a result, is not detected as a peak. On the other hand, it is conceivable to lower the peak detection threshold. However, if the threshold for peak detection is lowered, especially when images of the same object exist in the same block for the L image and the R image, the detection sensitivity becomes too high, and a gradation that is not a peak is erroneously detected as a peak. There is a possibility that.

  Therefore, in the second embodiment, it is detected that the same object image exists in different blocks in the L image and the R image, and the threshold value for peak detection is set to a predetermined value only for the block to which the image of the object belongs. Lower the process. Thereby, when the image of the same object exists in different blocks in the L image and the R image, the gradation corresponding to the image of the object can be detected as a peak in any block. Hereinafter, the second embodiment will be described in detail.

FIG. 7 is a block diagram illustrating an apparatus configuration of the image processing apparatus 700 according to the second embodiment. In the present embodiment, the input unit 701, the decoding unit 702, the frequency distribution acquisition unit 703, the image processing unit 704, the parallax measurement unit 705, and the memory 709 are configured by H / W. In addition, the block boundary determination unit 706, the peak detection unit 707, and the image quality improvement processing calculation unit 708 are processed by F / W. F / W processing is software processing performed by the control unit 709 (CPU) reading a program stored in a ROM (not shown) or the like into a RAM (not shown) and executing it. Note that the control unit 709 controls operations of the input unit 701, the decoding unit 702, the frequency distribution acquisition unit 703, the image processing unit 704, the parallax measurement unit 705, and the memory 709 that constitute the H / W part.

  The image processing apparatus 700 receives image data from an external connection device or the like (not shown). The image data is input to the input unit 701 and then decoded by the decoding unit 702. The decoded image data is passed to the frequency distribution acquisition unit 703 and the parallax measurement unit 705 via the internal bus.

  The frequency distribution acquisition unit 703 acquires the frequency distribution for each gradation of the L image and the R image that are continuously input. The frequency distribution acquisition unit 703 acquires the frequency distribution of the L image and the R image for each block from the image data of the L image and the R image that are input alternately and continuously. Then, the frequency distribution acquisition unit 703 adds the frequency distribution of the L image and the R image for each block to acquire the total frequency distribution, and stores the information of the total frequency distribution in a memory (not shown).

  The parallax measurement unit 705 compares the image data of the L image and the R image of the stereoscopic image decoded by the decoding unit 702, and measures the parallax. The parallax measurement unit 705 detects an object image in the stereoscopic image by a known method such as pattern matching or edge detection, and calculates the parallax from the display position in the L image and the display position in the R image for each object image. To do. The amount of parallax measured by the parallax measuring unit 705 is used to detect whether there are images of the same object that exist in different blocks between the L image and the R image.

  Based on the amount of parallax measured by the parallax measurement unit 705, the block boundary determination unit 706 determines whether there is an image that is an image of the same object and that exists in different blocks between the L image and the R image, The determination result is passed to the peak detection unit 707 in the subsequent stage.

  The peak detection unit 707 detects the peak gradation of the frequency distribution for each block based on the information on the combined frequency distribution acquired by the frequency distribution acquisition unit 703 and the determination result by the block boundary determination unit 706. In the present embodiment, the threshold value for peak detection of the frequency distribution is lowered from a predetermined value for blocks that are images of the same object and that have different images in the L image and the R image.

  The image quality improvement processing calculation unit 708 performs calculation processing for improving the image quality of the image based on the information of the peak gradation detected by the peak detection unit 707. The F / W can execute image quality enhancement processing by setting the calculation result in a register included in the image processing unit 704. Here, the image quality improvement processing calculation unit 708 adaptively creates a γ curve that assigns many gradations to the peak gradation detected by the peak detection unit 707.

  The image processing unit 704 reads the result calculated by the image quality enhancement processing calculation unit 708 from the register, and performs image processing (dynamic γ processing) based on the register value. Thereby, the γ curve created by the image quality enhancement processing calculation unit 708 is reflected in the image data.

FIG. 8 is a flowchart illustrating image quality improvement processing of the image processing apparatus 700 according to the second embodiment.
Since the processing from S801 to S807 is the same as the processing from S201 to S207 in FIG. However, the L image and the R image are both divided into blocks, and the frequency distribution acquisition and summing processing in S802, S805, and S806 is performed for each block, which is different from the first embodiment. In addition, a frequency distribution and a combined frequency distribution memory 709
Saving is also performed for each block.

  In step S808, the parallax measurement unit 705 measures the parallax between the L image and the R image of the stereoscopic image. In this embodiment, the parallax between the L image and the R image is obtained by pattern matching, and the obtained parallax is used as a parameter to determine whether there is an image of the same object that exists in different blocks between the L image and the R image. Judge whether or not. FIG. 9 is a diagram illustrating an example in which parallax is measured for each image of an object by pattern matching between an L image and an R image. As shown in FIG. 9A, the image includes low-brightness object images A and B in a high-brightness background image as in FIG. 3, and pattern matching between the L image and the R image results in FIG. As shown in B), the parallax PA of the object A and the parallax PB of the object B are measured.

  The pattern matching itself can use a general method as a method for acquiring parallax of a stereoscopic image. Further, in this embodiment, it is only necessary to detect images of the same object that are present in different blocks between the L image and the R image. Therefore, the parallax may be measured using a method other than pattern matching. In addition, when the parallax amount of the stereoscopic image is added to the stereoscopic image data as metadata, information on the parallax amount may be acquired from the metadata.

  In step S809, the block boundary determination unit 706 determines whether there is an image of the same object that exists in different blocks between the L image and the R image based on the parallax measured in step S808. The position in the image of the object and its parallax are obtained by pattern matching. Based on information on the position and parallax of the object in this image and information on the size of each block (the number of pixels in the vertical and horizontal directions), the image of the object spans two blocks of an L image and an R image (block It can be determined whether it exists (beyond the boundary).

  FIG. 9C is a diagram showing that it is determined that the image of the object B exists in different blocks (blocks 2 and 3 in FIG. 9C) between the L image and the R image. For these blocks, the peak detection unit 707 performs processing for lowering the peak detection threshold at the time of the next peak detection in S810.

In step S810, the peak detection unit 707 determines the peak detection threshold for blocks that are determined to be in the same object and exist in different blocks in the L image and the R image as a result of the determination in step S809. Perform processing to lower.
On the other hand, in step S811, the peak detection unit 707 determines the default for blocks that are determined to have no image in the different blocks of the L image and the R image as a result of the determination in step S809. Decide to use peak detection threshold (default).

  FIG. 10 is a diagram illustrating the peak detection result when the peak detection threshold of a block having an image of the same object that is present in different blocks of the L image and the R image is lowered. In the above example, for block 2 and block 3, there are images that are images of the same object but in different blocks in the L image and the R image. Therefore, as shown in FIG. To threshold B. On the other hand, with respect to the block 1, since there is no image that is an image of the same object and exists in different blocks between the L image and the R image, the default threshold A is used.

  As a result, the peak based on the image of the object B is also reduced by reducing the peak detection threshold for the blocks 2 and 3 in which the peak based on the image of the object B is not as high as the block 1 in the combined frequency distribution. It can be detected as a key. FIG. 10 shows that the peak based on the image of the object A or the object B can be detected as the target gradation in any of the blocks 1, 2, and 3.

  In the present embodiment, the example in which the threshold for peak detection is lowered for blocks having the same object and having images in different blocks in the L image and the R image has been described. However, a correction may be performed to increase the frequency distribution of the block or the combined frequency distribution, and the threshold value may be common to all blocks. By increasing the frequency, it is possible to detect a peak in a block having an image of the same object and existing in different blocks of the L image and the R image without lowering the peak detection threshold.

In S812, the peak detection unit 707 detects, for each block, the peak of the combined frequency distribution obtained by adding the frequency distributions of the L image and the R image using the peak detection threshold value determined in S810 and S811.
In step S813, the image quality improvement processing calculation unit 708 creates a γ curve so as to assign a large number of gradations to the gradation in which the peak exists based on the peak information detected in step S812 for each block. To do.
In step S814, the image processing unit 704 applies the γ curve created in step S813 to the input image data.

  As described above, the image processing apparatus 700 according to the present exemplary embodiment acquires the combined frequency distribution obtained by adding the frequency distributions of the L image and the R image for each block, and determines the gradation in which the frequency exceeds the threshold in the combined frequency distribution at the peak level. It is detected as a tone (notable tone). Then, adaptive gradation conversion is performed for each block so that many gradations are assigned near the peak gradation. At this time, a process of detecting a block having an image of the same object in a stereoscopic image and having an image existing in different blocks in the L image and the R image, and lowering a threshold for detecting the peak gradation for the block I do. This makes it possible to detect a peak based on an image of the same object that is present in different blocks between the L image and the R image, and has a large parallax. Can optimally generate and apply an optimal γ curve.

  In FIG. 5, the image of the object B exists in the block 2 in the L image, exists in the block 3 in the R image, and the image of the object B and the block boundary do not overlap in either the L image or the R image. However, as shown in FIG. 12, the block boundary and the object image may be overlapped. In this case, the object image exists over a plurality of blocks in at least one of the L image and the R image. In such a state, in step S809 in FIG. 8, the block boundary determination unit 706 detects how much the object image exists in each of the plurality of blocks (area, the ratio of the number of pixels), and is most present. It is determined whether or not a block with a large ratio is different between the L image and the R image. If they are different, the process may proceed to S810, and if they are not different, the process may proceed to S811.

  A method for determining the threshold value of each block when the object crosses the block boundary will be described with reference to FIGS. FIGS. 13 to 15 are diagrams illustrating threshold value determination methods in blocks 1 to 3 when the stereoscopically viewed image is FIG. 12.

As for block 1, all the objects A in the L image are included in block 1 from FIG. 13, but object A in the R image straddles the boundary between block 1 and block 2. Therefore, the threshold value of block 1 is determined so that the image of the object A is lowered by an amount exceeding the boundary between block 1 and block 2 toward the block 2 side. That is, from the diagram of FIG. 13, in the R image, the threshold value is lowered by a value corresponding to the distance d1AR (the amount of protrusion) that the object A exceeds the block 2 side from the boundary between the block 1 and block 2. However, when the distance d1AR is smaller than the predetermined value, it is not necessary to use the distance for determination (correction) of the threshold because the influence on the detection of the peak value of the frequency distribution is small. The value corresponding to the amount exceeding the block 2 side can be, for example, the number of pixels of the portion included in the block 2 in the image of the object A. Alternatively, the value can be set so as to increase as the distance in the direction perpendicular to the boundary (the horizontal direction in the relationship between the block 1 and the block 2) increases. The same applies to the following description.

  Next, block 2 will be described with reference to FIG. In block 2, the object B of the L image straddles the boundary with the adjacent block 3, the object A of the R image straddles the boundary with the adjacent block 1, and the object B straddles the boundary with the adjacent block 3. . In this case, in the L image, the threshold value is lowered by the distance d2BL that the object B has passed from the boundary between the block 2 and the block 3 to the block 3 side. However, when the distance d2BL is smaller than the predetermined value, it is not necessary to use the distance for determination of the threshold because the influence on the detection of the peak value of the frequency distribution is small.

  In the R image, the object A exceeds the boundary between the block 2 and the block 1 by the distance d2AR from the boundary between the block 2 and the block 1, but the object A is not included in the block 2 in the L image. It is not necessary to detect the peak of the object A. Therefore, the distance d2AR that the object A protrudes to the block 1 in the R image is not used for determining the threshold value of the block 2. Therefore, the distance d2AR does not affect the threshold value of block 2. In the R image, the object B exceeds the boundary between the block 2 and the block 3 toward the block 3 by the distance d2BR. Since the block B is also included in the block 2 in the L image, the threshold value is lowered by the distance d2BR. As described above, the threshold value of the block 2 is lowered by (d2BL + d2BR).

Finally, block 3 will be described with reference to FIG. In the case of block 3, since the amount of object B is large, the peak detection threshold is lowered by the distance that object B exceeds the block boundary. In FIG. 15, the object B protrudes by the distance d3BL toward the block 2 in the case of the L image, and protrudes by the distance d2BR toward the block 2 in the case of the R image. Therefore, the peak detection threshold can be lowered by (d3BL + d3BR).
As described above, even when the object image extends over the block boundary as shown in FIG. 12, it is possible to obtain the same effect as in the second embodiment.
An example of the high image quality processing of the above embodiment is performing gamma correction set based on the frequency distribution for each frame. Further, the image quality enhancement processing is not limited to gamma correction, and may be processing such as high resolution, noise removal, and edge enhancement, or image processing that combines these.

DESCRIPTION OF SYMBOLS 100 Stereoscopic image display apparatus 101 Input part 103 Frequency distribution acquisition part 104 Image quality improvement process calculating part 105 Image processing part

Claims (7)

Input means for the left eye image and the right-eye image picture are successively input alternately,
The left eye image and the right eye image that are continuously input are each divided into a plurality of blocks, and the frequency distribution for each gradation of each block of the left eye image and the right eye image is added to each block. and the frequency distribution acquisition means that Tokusu taken for,
Detecting means for detecting a position of an object image included in each of the left-eye image and the right-eye image;
Applying the in the sum frequency distribution of each block, the frequency is, the blocks of the basis of the rupees click tone exceeds a threshold determined in accordance with the position of the object image, the left eye image and the right-eye image Determining means for determining a gradation conversion function for
Wherein said each block of the left-eye image and the right-eye image, and facilities to the image processing means the image processing using the gradation conversion function corresponding,
An image processing apparatus comprising: Depending on the position of the object image detected by the detection unit, a first block the object picture image in the left eye image is present, and a second block wherein the object image picture in said right eye image exists , but example Bei determining means for determining whether differ,
If the determination means determines that said first block and the second block are different, said determining means, said threshold value in at least the first block, the determining means and the first block and the second block The image processing apparatus according to claim 1, wherein the image processing apparatus is smaller than a case where it is determined that they are not different . Depending on the position of the object image detected by the detection unit, a first block the object picture image in the left eye image is present, and a second block wherein the object image picture in said right eye image exists , but example Bei determining means for determining whether differ,
The said frequency distribution acquisition means performs correction | amendment which makes frequency increase at least with respect to the total frequency distribution of a said 1st block, when the said determination means determines with the said 1st block and the said 2nd block differing. The image processing apparatus described. Comprising a parallax acquisition means for acquiring a parallax between the object image picture in said object picture image and the right eye image in the left eye image,
The determination means, the parallax acquired by the parallax acquisition unit, a position of the object image picture, the position of each block, based on said object image image the left eye image and the right-eye image The image processing apparatus according to claim 2 , wherein each of the image processing apparatuses determines whether the blocks are present in different blocks. When the object image exists across a plurality of the blocks of the left-eye image, the determination unit determines the threshold value of the plurality of blocks where the object image exists, and the object image indicates the left-eye. 5. The image processing apparatus according to claim 1, wherein the image processing apparatus is determined so as to be smaller than a case where the image does not exist across the plurality of blocks of the image for use. When the object image exists across a plurality of the blocks of the right-eye image, the determination unit determines the threshold value of the plurality of blocks where the object image exists, and the object image indicates the right-eye. 5. The image processing apparatus according to claim 1, wherein the image processing apparatus is determined so as to be smaller than a case where the image does not exist across the plurality of blocks of the image for use. An input step of left eye image and the right-eye image picture are successively input alternately,
The left eye image and the right eye image that are continuously input are each divided into a plurality of blocks, and the frequency distribution for each gradation of each block of the left eye image and the right eye image is added to each block. and the frequency distribution acquisition step that Tokusu taken for,
A detection step of detecting a position of an object image included in each of the left-eye image and the right-eye image;
Applying the in the sum frequency distribution of each block, the frequency is, the blocks of the basis of the rupees click tone exceeds a threshold determined in accordance with the position of the object image, the left eye image and the right-eye image A determination step of determining a gradation conversion function for
Wherein said each block of the left-eye image and the right-eye image, and facilities to the image processing step the image processing using the gradation conversion function corresponding,
A control method for an image processing apparatus, comprising:

Images