JP2017033545A - Image processing device and control method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of obtaining the outcome of highly accurate image processing without bringing in cost increase in an image processing device.SOLUTION: An image processing device of the present invention includes: first processing means for applying a first image process to input image data; communication means for communicating with another image processing device including second processing means for performing a second image process different from the first image process to have the second processing means apply the second image process to the input image data, and for obtaining a result of applying the second image process to the input image data from the other image processing device; and synthesis means for performing a synthesis process of synthesizing the result of the first image process by the first processing means and the result of the second image process obtained by the communication means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus and a control method thereof.

  In recent years, the resolution of image data that can be displayed on an image display device has been increasing. Accordingly, it is desired to increase the resolution of image data that can be processed by the image processing apparatus. As a method for processing high-resolution image data, there is a method in which a plurality of partial image data corresponding to a plurality of partial images constituting the original image represented by the original image data are processed in parallel by a plurality of image processing devices. . Each partial image is an image in a partial region of the original image. In this method, for example, by using two image processing apparatuses capable of processing image data of 5 million pixels, it is possible to realize image processing for original image data of 10 million pixels. Construction of an image processing system (a system having a plurality of image processing devices) that performs image processing by the above method is easier than development of a new image processing device capable of processing high-resolution image data.

  In addition, opportunities for handling image data having a wide dynamic range of luminance called HDR (High Dynamic Range) image data are increasing. When the HDR image data is an object of image processing, the image processing apparatus processes image data with various luminances from low luminance to very high luminance. However, when image processing for HDR image data is realized using a conventional LUT (Look Up Table), image quality deterioration occurs due to insufficient accuracy of image processing. For example, blackout occurs due to the low luminance pixel value (gradation value) of the input image data, which is image data before image processing, being converted to the lower limit value by image processing. The conventional LUT is, for example, an LUT created in consideration of only image data (SDR (Standard Dynamic Range) image data) having a standard luminance dynamic range.

  A method for increasing the number of words in the LUT is conceivable as a method for preventing the image quality deterioration that occurs in image processing on HDR image data. Specifically, a method of expanding the address range of the LUT and increasing the number of addresses that can be referred to the LUT can be considered. However, the number of words in the LUT is limited by the capacity of a memory (for example, SRAM (Static Random Access Memory)) constituting the LUT. Since the increase in memory capacity causes an increase in the cost of the image processing apparatus, the total amount of memory cannot be increased easily.

  A method of dynamically rewriting an LUT during image processing is conceivable as a method of realizing image processing using addresses larger than usual and a wider range than usual as addresses that can be referred to. For example, when the luminance of each pixel of the input image data is concentrated in a part of the luminance range, addresses in the range corresponding to the luminance range are referred to. For this reason, a method of rewriting an address set (mapped) in the LUT from an address in a range that is unlikely to be referred to to an address in a range that is likely to be referenced. However, in this method, an image disturbance (deterioration in image quality) occurs due to reading of the address during rewriting of the LUT.

A conventional technique for rewriting an LUT during image processing is disclosed in Patent Document 1, for example. In the technique disclosed in Patent Document 1, an address near the address being rewritten is referred to instead of the address being rewritten. Thereby, the occurrence of large image disturbance is suppressed. However, in the technique disclosed in Patent Document 1, a desired address may not be referred to at a desired timing. For this reason, image quality degradation may occur when the luminance range of the input image data changes.

  In order to prevent the occurrence of the image quality deterioration due to the rewriting of the LUT, it is preferable that a certain range of addresses can always be accessed during image processing.

  As a method of increasing the number of words in the LUT, a method of reducing the bit depth per word can be considered. However, reducing the bit depth reduces the accuracy of image processing. In addition, the LUT that is desired to be used may differ among a plurality of regions of the image. However, if a plurality of LUTs are configured using one memory, the memory capacity per area is reduced, and the accuracy of image processing is reduced. As a matter of course, if a plurality of memories each configuring a plurality of LUTs are provided in the image processing apparatus, the cost of the image processing apparatus increases.

JP 2005-266576 A

  An object of the present invention is to provide a technique capable of obtaining a highly accurate image processing result without increasing the cost of an image processing apparatus.

The first aspect of the present invention is:
An image processing apparatus,
First processing means for performing first image processing on input image data;
The second image processing on the input image data is performed by performing communication with another image processing apparatus having a second processing unit that performs second image processing different from the first image processing. Communication means for causing the second processing means to perform the second image processing on the input image data, and acquiring the result from the other image processing apparatus;
Combining means for combining the result of the first image processing by the first processing means and the result of the second image processing acquired by the communication means;
An image processing apparatus comprising:

The second aspect of the present invention is:
An image processing system having a first image processing device and a second image processing device,
The first image processing apparatus includes:
First processing means for performing first image processing on first input image data which is image data input to the first image processing apparatus;
First communication means for performing communication with the second image processing apparatus;
First combining means for performing first combining processing for combining the result of the first image processing by the first processing means and the result of communication by the first communication means;
The second image processing apparatus includes second processing means for performing second image processing different from the first image processing;
The first communication unit causes the second processing unit to perform the second image processing on the first input image data by performing communication with the second image processing apparatus. A result obtained by performing the second image processing on the first input image data is obtained from the second image processing apparatus;
The first synthesizing process is a process of synthesizing the result of the first image processing on the first input image data and the result of the second image processing acquired by the first communication unit. An image processing system is characterized by being.

The third aspect of the present invention is:
A control method for an image processing apparatus having first processing means for performing first image processing,
A processing step for causing the first processing means to perform the first image processing on input image data which is image data input to the image processing apparatus;
The second image processing on the input image data is performed by performing communication with another image processing apparatus having a second processing unit that performs second image processing different from the first image processing. A communication step of causing the second processing means to perform the second image processing on the input image data, and acquiring the result from the other image processing device;
A synthesis step for performing a synthesis process for synthesizing the result of the first image processing acquired in the processing step and the result of the second image processing acquired in the communication step;
A control method for an image processing apparatus, comprising:

The fourth aspect of the present invention is:
A program for causing a computer to execute a control method of an image processing apparatus having first processing means for performing first image processing,
The control method is:
A processing step for causing the first processing means to perform the first image processing on input image data which is image data input to the image processing apparatus;
The second image processing on the input image data is performed by performing communication with another image processing apparatus having a second processing unit that performs second image processing different from the first image processing. A communication step of causing the second processing means to perform the second image processing on the input image data, and acquiring the result from the other image processing device;
A synthesis step for performing a synthesis process for synthesizing the result of the first image processing acquired in the processing step and the result of the second image processing acquired in the communication step;
It is a program characterized by having.

  According to the present invention, a highly accurate image processing result can be obtained without increasing the cost of the image processing apparatus.

1 is a diagram illustrating an example of a configuration of an image processing apparatus according to a first embodiment. 1 is a diagram illustrating an example of a configuration of an LUT according to the first embodiment. 1 is a diagram illustrating an example of a configuration of an LUT according to the first embodiment. 1 is a diagram illustrating an example of a configuration of an image processing system according to a first embodiment. 1 is a diagram illustrating an example of a configuration of an LUT according to the first embodiment. FIG. 3 is a diagram illustrating an example of a processing flow of the image processing system according to the first embodiment. FIG. 10 is a diagram illustrating an example of processing timing of the image processing system according to the first embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an image processing apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an image processing system according to the second embodiment. The figure which shows an example of the processing flow of the LUT setting process which concerns on Example 2. FIG. The figure which shows an example of the 1st input image which concerns on Example 2, and a 2nd input image The figure which shows an example of the 1st input image which concerns on Example 2, and a 2nd input image The figure which shows an example of the 1st input image which concerns on Example 2, and a 2nd input image FIG. 10 is a diagram illustrating an example of a processing flow of the image processing system according to the second embodiment. The figure which shows an example of the processing flow of the cooperation process which concerns on Example 2. FIG. FIG. 10 is a diagram illustrating an example of processing timing of the image processing system according to the second embodiment. FIG. 10 is a diagram illustrating an example of processing timing of the image processing system according to the second embodiment. FIG. 10 is a diagram illustrating an example of the configuration of an image processing apparatus according to a third embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an image processing system according to the third embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an LUT according to the third embodiment. FIG. 10 is a diagram illustrating an example of a processing flow of the image processing system according to the third embodiment. FIG. 10 is a diagram illustrating an example of processing timing of the image processing system according to the third embodiment. FIG. 10 is a diagram illustrating an example of the configuration of an image processing apparatus according to a fourth embodiment. FIG. 10 is a diagram illustrating an example of the configuration of an image processing system according to a fourth embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an LUT according to the fourth embodiment. Explanatory drawing of the delay adjustment part which concerns on Example 4. FIG. FIG. 10 is a diagram illustrating an example of a processing flow of the image processing system according to the fourth embodiment. FIG. 10 is a diagram illustrating an example of the configuration of an image processing apparatus according to a fifth embodiment. FIG. 10 is a diagram illustrating an example of an LUT configuration according to the fifth embodiment. FIG. 10 is a diagram illustrating an example of an LUT configuration according to the fifth embodiment. FIG. 10 is a diagram illustrating an example of the configuration of an image processing system according to a fifth embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an LUT according to the fifth embodiment. FIG. 10 is a diagram illustrating an example of a processing flow of the image processing system according to the fifth embodiment.

<Example 1>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram illustrating an example of the configuration of the image processing apparatus 100 according to the present embodiment. The image processing device 100 is used, for example, in an image processing system having a plurality of image processing devices.

  The address determination unit 101 determines an address corresponding to the pixel value of the input image data (address determination process). The input image data is image data input to the image processing apparatus 100. The address determination process is performed for each pixel of the input image data. In the address determination process, for example, the address is determined in accordance with the value of the upper few bits of the pixel value of the input image data.

  The address exchange unit 102 performs communication with other image processing apparatuses. Specifically, the address exchanging unit 102 outputs the address determined by the address determining unit 101 to another image processing apparatus or acquires an address from another image processing apparatus.

  The address selection unit 103 selects one of the address determined by the address determination unit 101 and the address acquired from another image processing apparatus by the address exchange unit 102, and outputs the selected address to the LUT 104.

  The LUT 104 is a look-up table (LUT) that performs first image processing on input image data. When an address (input value) is input, the LUT 104 outputs an output value (a first processing value that is a pixel value after the first image processing) corresponding to the input address. When the address determined by the address determination unit 101 is input to the LUT 104, the LUT 104 performs the first image processing on the input image data.

  The LUT setting unit 105 sets the LUT 104.

The processing value exchange unit 106 communicates with the other image processing apparatus. Specifically, the process value exchanging unit 106 outputs the process value obtained by the LUT 104 to another image processing apparatus or acquires the process value from the other image processing apparatus.

  In this embodiment, the other image processing apparatus includes an LUT that performs second image processing different from the first image processing. The LUTs possessed by other image processing apparatuses are hereinafter referred to as “other LUTs”. The address exchange unit 102 causes the other LUT to perform the second image processing on the input image data by performing communication with another image processing apparatus. Specifically, when the address determined by the address determining unit 101 is output from the address exchanging unit 102 to another image processing apparatus, the address determined by the address determining unit 101 is input to another LUT. As a result, as the output value of the other LUT, a second processing value that is a result of performing the second image processing on the pixel value of the input image data is obtained. Therefore, it can be said that “a process for causing the other LUT to perform the second image processing for the input image data” is a “a process for causing the other LUT to output an output value corresponding to the address determined by the address determination unit 101”. Then, the processing value exchanging unit 106 performs communication with another image processing device, thereby obtaining a second processing value that is a result of performing the second image processing on the pixel value of the input image data. From the image processing apparatus. Further, the processing value exchanging unit 106 outputs a processing value corresponding to the address acquired from the other image processing apparatus to the other image processing apparatus.

  Instead of the address exchange unit 102 and the processing value exchange unit 106, one communication unit having the functions of the address exchange unit 102 and the processing value exchange unit 106 may be used.

  The processing value combining unit 107 performs a combining process for combining the result of the first image processing by the LUT 104 and the result of the second image processing acquired by the processing value exchanging unit 106. The combining process is a process for combining the result of applying the first image processing to the input image data and the result of applying the second image processing to the input image data. In this embodiment, the composition processing is performed by using the first processing value that is the output value of the LUT 104 corresponding to the address corresponding to the pixel value of the input image data and the second output value that is the output value of the other LUT corresponding to the address. Is a process of combining the process value. In this embodiment, a composition process is performed for each pixel of the input image data. Then, the process value synthesis unit 107 outputs a process value after the synthesis process (synthesis process value).

  The processing value selection unit 108 selects one of the first processing value that is the output value of the LUT 104 and the synthesis processing value that is the output value of the processing value synthesis unit 107, and selects the selected processing value (selection processing value). Output.

  The interpolation unit 109 performs an interpolation calculation of the selection process value based on the pixel value of the input image data. Note that all the bits of the pixel value of the input image data may be used for the interpolation calculation, or some bits of the pixel value of the input image data may be used for the interpolation calculation. For example, when the address determination unit 101 determines an address, it is assumed that the upper few bits among a plurality of bits constituting the pixel value of the input image data are used. In this case, of the plurality of bits constituting the pixel value of the input image data, the lower-order bits that are not used when the address determination unit 101 determines the address may be used for the interpolation calculation. The interpolation operation may be an operation that realizes linear interpolation or an operation that realizes nonlinear interpolation. The interpolation unit 109 outputs the processed value after the interpolation calculation as the pixel value of the output image data.

  Note that the processing value selection unit 108 and the interpolation unit 109 may be omitted, and the synthesis processing value may be used as the pixel value of the output image data.

The setting of the LUT 104 performed by the LUT setting unit 105 will be described with reference to FIGS.
Here, it is assumed that the LUT 104 is configured by an SRAM (Static Random Access Memory) having a data capacity of X × Y bits. The LUT setting unit 105 determines the bit width of each first processing value constituting the LUT 104 and the number of words (address range) of the LUT 104 so as to fit within the SRAM size. Then, the LUT setting unit 105 maps the address and the first processing value on the SRAM according to the determined bit width and the determined number of words. Thereby, the LUT 104 indicating the address in the determined range (number of words) and the first processing value having the determined bit width associated with each address is set.

  FIG. 2 shows an example of the configuration of the LUT 104. FIG. 2 shows an example where the bit width of the first processing value per word (one address) is X bits and the number of words is Y words. In order to set the LUT 104 shown in FIG. 2, a data capacity of X × Y bits (= X bits × Y words) is required. As described above, the data capacity of the SRAM is X × Y bits. Therefore, the LUT 104 shown in FIG. 2 can be set. In FIG. 2, “ADR = x” indicates an address x, and dat [x] indicates a first processing value corresponding to the address x.

  FIG. 3 also shows an example of the configuration of the LUT 104. FIG. 3 shows an example in which the number of words is 2Y words (twice that of FIG. 2). However, if X bits (the same value as in FIG. 2) are used as the bit width of the first processing value per word, the data capacity necessary for setting the LUT 104 exceeds the data capacity of the SRAM (X × Y bits). Therefore, the LUT 104 cannot be set. Specifically, a data capacity of 2 × X × Y bits (= X bits × 2Y words) is necessary for setting the LUT 104. Therefore, in the example of FIG. 3, X / 2 bits are used as the bit width of the first processing value per word. Thereby, the data capacity necessary for setting the LUT 104 can be maintained at X × Y bits (= X / 2 bits × 2Y words), and the LUT 104 can be set. In this embodiment, an example in which the LUT 104 shown in FIG. 3 is set will be described. In FIG. 3, “ADR = x” indicates an address x, and dat1 [x] indicates a first processing value corresponding to the address x.

  When image processing is performed on image data having a wide luminance dynamic range called HDR (High Dynamic Range) image data, it is conceivable to increase the number of words (address range) of the LUT. However, as shown in FIG. 3, when the address range is expanded, the bit width of the processing value per word is reduced, and the accuracy of the processing value per word is lowered. Increasing the data capacity of the SRAM can suppress a decrease in accuracy of the processing value, but increases the cost of the image processing apparatus.

  Therefore, in this embodiment, the accuracy of the LUT used for image processing is improved by using another image processing apparatus. Specifically, the address range (number of words) is expanded without lowering the accuracy of the processing value per word using another image processing apparatus. Thereby, it is possible to obtain highly accurate image processing results without increasing the cost of the image processing apparatus.

  FIG. 4 is a block diagram illustrating an example of the configuration of the image processing system according to the present embodiment. As shown in FIG. 4, the image processing system according to the present embodiment includes an image processing apparatus 100 and an image processing apparatus 200. Note that the image processing system may include three or more image processing apparatuses. The image processing apparatus 200 includes an address determination unit 201, an address exchange unit 202, an address selection unit 203, an LUT 204, an LUT setting unit 205, a processing value exchange unit 206, a processing value synthesis unit 207, a processing value selection unit 208, and an interpolation unit. 209.

The address determination unit 201 has the same function as the address determination unit 101. The address exchange unit 202 has the same function as the address exchange unit 102. The address selection unit 203 has the same function as the address selection unit 103. The LUT 204 has the same function as the LUT 104. The LUT setting unit 205 has the same function as the LUT setting unit 105. The processing value exchange unit 206 has the same function as the processing value exchange unit 106. The processing value synthesis unit 207 has the same function as the processing value synthesis unit 107. The processing value selection unit 208 has the same function as the processing value selection unit 108. The interpolation unit 209 has the same function as the interpolation unit 109. That is, the image processing apparatus 200 has the same configuration as the image processing apparatus 100. In the image processing device 200, processing is performed in which “first” and “second” in the above description regarding the processing of the image processing device 100 are interchanged.

  Communication for address transmission / reception is performed between the address exchange unit 102 and the address exchange unit 202, and communication for processing value transmission / reception is performed between the processing value exchange unit 106 and the processing value exchange unit 206. Done.

  FIG. 5 shows an example of the LUT 104, the LUT 204, and a combined LUT obtained by combining the LUT 104 and the LUT 204. The LUT 104 is shown on the left side of FIG. 5, the LUT 204 is shown in the middle of FIG. 5, and the composite LUT is shown on the right side of FIG. The LUT 104 in FIG. 5 is the same as the LUT 104 in FIG. In the LUT 204 of FIG. 5, the bit width of the processing value (second processing value) per word is X / 2 bits and the number of words is 2Y words, as in the LUT 104 of FIG. In FIG. 5, “ADR = x” indicates an address x, dat1 [x] indicates a first processing value corresponding to the address x, and dat2 [x] indicates a second processing value corresponding to the address x. .

  In the present embodiment, as the first processing value dat1, a first partial value that is one of a plurality of partial values constituting a pixel value obtained by performing predetermined image processing (high-precision image processing) on the pixel value is used. use. Then, as the second processing value dat2, a second partial value that is one of the plurality of partial values and is different from the first partial value is used. By synthesizing the first partial value and the second partial value, it is possible to obtain a processing value with higher accuracy than the first processing value dat1 and the second processing value dat2 as a combined processing value. The number of the plurality of partial values matches, for example, the number of image processing apparatuses included in the image processing system. By synthesizing a plurality of partial values obtained by a plurality of image processing apparatuses, it is possible to obtain the same pixel value as the result of high-precision image processing.

  In the present embodiment, an LUT that assumes that the bit width of the processing value per word is X bits and the number of words is 2Y words is assumed as an LUT that realizes predetermined image processing. Then, the upper X / 2 bits of the X-bit processed value are used as the first processed value dat1, and the lower X / 2 bits of the X-bit processed value are used as the second processed value dat2. Then, as shown in FIG. 5, by synthesizing the first processing value dat1 and the second processing value dat2, a high-precision LUT (the bit width is X bits and the number of words is 2Y words). The result of image processing using (LUT) can be obtained.

  Next, an example of a processing flow of the image processing system according to the present embodiment will be described. FIG. 6 is a flowchart illustrating an example of a processing flow of the image processing system according to the present embodiment. Here, processing relating to input image data input to the image processing apparatus 100 will be described. Note that the processing related to the input image data input to the image processing apparatus 200 is the same as the processing related to the input image data input to the image processing apparatus 100, and thus description thereof is omitted.

  First, in S101, the address determination unit 101 determines an address from the pixel value of the input image data. Address determination unit 101 outputs the determined address to address exchange unit 102 and address selection unit 103.

Next, in S <b> 102, the address exchange unit 102 starts communication (address exchange) with the address exchange unit 202 of the image processing apparatus 200. For example, the process in which the address exchange unit 102 outputs the address determined in S101 to the address exchange unit 202 is performed as an address exchange. The address exchange includes a process in which the address exchange unit 102 acquires an address output from the address exchange unit 202 in addition to a process in which the address exchange unit 102 outputs the address determined in S101 to the address exchange unit 202. Also good. In this embodiment, the processes of S103 and S104 are performed in parallel with the execution of address exchange. After the address exchange is completed, the process may proceed to S103.

  In step S <b> 103, the address selection unit 103 selects the address determined by the address determination unit 101 and outputs the selected address to the LUT 104. That is, the address selection unit 103 outputs the address determined in S101 to the LUT 104.

  In S104, the LUT 104 outputs the first processing value dat1 corresponding to the address output from the address selection unit 103 in S103 to the processing value synthesis unit 107 (generation of the first processing value dat1). The process value synthesis unit 107 holds the input process value until the synthesis process is completed.

  In step S <b> 105, the address exchange unit 202 of the image processing apparatus 200 determines whether or not address exchange with the address exchange unit 102 has been completed. For example, when the address exchange unit 202 completes the process of acquiring the address determined in S101 from the address exchange unit 102, it is determined that the address exchange is completed. If the address exchange unit 202 has not completed the process of acquiring the address determined in S101 from the address exchange unit 102, it is determined that the address exchange has not been completed. The process of S105 is repeated until it is determined that the address exchange is completed. If it is determined that the address exchange is completed, the process proceeds to S106.

  In S106, the address exchange unit 202 outputs the address determined in S101 to the LUT 204 via the address selection unit 203. Thereby, in the LUT 204, the second processing value dat2 corresponding to the address determined in S101 is generated. Then, the LUT 204 outputs the second processing value dat2 corresponding to the address determined in S101 to the processing value exchanging unit 206.

  In step S <b> 107, the process value exchange unit 206 starts communication (process value exchange) with the process value exchange unit 106 of the image processing apparatus 100. For example, the process value exchange unit 206 outputs the second process value dat2 obtained in S106 to the process value exchange unit 106 as a process value exchange. In the process value exchange, in addition to the process value exchange unit 206 that outputs the second process value dat2 obtained in S106 to the process value exchange unit 106, the first process value output from the process value exchange unit 106 The processing value exchanging unit 206 may include processing for acquiring dat1.

  In S108, process value exchanging section 106 determines whether or not the process value exchange with process value exchanging section 206 has been completed. For example, when the process value exchanging unit 106 acquires the second process value dat2 obtained in S106 from the process value exchanging unit 206, it is determined that the process value exchange has been completed. Then, when the process value exchange unit 106 has not completed the process of acquiring the second process value dat2 obtained in S106 from the process value exchange unit 206, it is determined that the process value exchange has not been completed. The process of S108 is repeated until it is determined that the process value exchange has been completed. If it is determined that the process value exchange has been completed, the process proceeds to S109. When it is determined that the process value exchange has been completed, the process value exchange unit 106 outputs the second process value dat2 obtained in S106 to the process value synthesis unit 107.

In S109, the process value synthesizing unit 107 synthesizes the first process value dat1 input in S104 and the second process value dat2 input in S108 (synthesizing process). This
A composite processing value having a bit width of X bits as shown in FIG. 5 is obtained. The process value synthesis unit 107 outputs the synthesis process value to the process value selection unit 108.

  Next, in S110, the processing value selection unit 108 selects the output value of the processing value synthesis unit 107, and outputs the selected output value to the interpolation unit 109. That is, the processing value selection unit 108 outputs the combined processing value obtained in S109 to the interpolation unit 109. Then, the interpolation unit 109 performs interpolation calculation of the synthesis processing value based on the pixel value of the input image data.

  FIG. 7 is a time chart illustrating an example of processing timing of the image processing system according to the present embodiment. FIG. 7 is a time chart corresponding to the processing flow described with reference to FIG.

  At time t1, image data is input to the address determination unit 101 of the image processing apparatus 100, and address determination is started. Next, at time t2, address determination is completed, and the determined address is input to the address exchange unit 102 and the LUT 104. Then, at time t3, the LUT 104 reads the first processing value dat1 corresponding to the address determined at time t2. Next, the read first processing value dat1 is input to the processing value synthesis unit 107 at time t4. The first process value dat1 input to the process value synthesis unit 107 is held by the process value synthesis unit 107 until the synthesis process is completed.

  At time t5, the LUT 204 reads out the second processing value dat2 corresponding to the address input from the address exchange unit 102 to the LUT 204 via the address exchange unit 202 and the address selection unit 203. Thereafter, the read second processing value dat2 is input to the processing value exchanging unit 206. Then, at time t6, a process of outputting the read second processing value dat2 from the processing value exchanging unit 206 to the processing value exchanging unit 106 is started.

  At time t7, the second processing value dat2 acquired from the processing value exchanging unit 206 by the processing value exchanging unit 106 is input to the processing value combining unit 107, and the input second processing value dat2 and the time t4 The held first processing value dat1 is synthesized. Then, at time t8, the interpolation unit 109 performs an interpolation operation on the combined processing value obtained by the processing value combining unit 107.

  Here, the processing value synthesis unit 107 needs to hold the first processing value dat1 input at time t4 until time t7. Also, the interpolation unit 109 needs to hold the pixel value of the input image data until the interpolation calculation timing. Although not shown in the drawing, the holding can be realized by using a delay circuit using a latch, a FIFO (First In First Out) memory, a DRAM (Dynamic Random Access Memory), and the like.

  Note that the image processing apparatus 200 may perform processing on the input image data of the image processing apparatus 200 during a period in which the processing of the input image data of the image processing apparatus 100 is performed. In such a case, the LUT 204 reads the second processing value corresponding to the address determined by the image processing apparatus 100 and the second processing value corresponding to the address determined by the image processing apparatus 200. Are read twice (two types). Therefore, in such a case, it is necessary to operate the LUT 204 at a rate that is at least twice the rate of the input image data. For example, if the operating frequency (operating speed) of the LUT 204 is increased to more than twice the operating frequency of other functional units, the LUT 204 can be operated at a rate that is at least twice that of the input image data.

Further, the LUT 104 may be used (referenced) by the image processing apparatus 200 during a period in which the image processing apparatus 100 performs processing on input image data of the image processing apparatus 100. In that case, it is necessary to operate the LUT 104 at a rate more than twice the rate of the input image data for the same reason as described above.

  As described above, according to the present embodiment, it is possible to obtain a high-precision image processing result without causing an increase in cost of the image processing apparatus by operating a plurality of image processing apparatuses in cooperation with each other. it can. Specifically, an image processing result equivalent to the case of using an LUT that performs high-precision image processing can be obtained without increasing the cost of the image processing apparatus. The image processing system of the present embodiment can cope with high-precision image processing for HDR image data. In this embodiment, an example in which image processing is performed using an LUT has been described. However, the present invention is not limited to this. The image processing method is not particularly limited. For example, image processing for converting pixel values by calculation using a function may be performed.

<Example 2>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 2 of the present invention will be described. FIG. 8 is a block diagram illustrating an example of the configuration of the image processing apparatus 300 according to the present embodiment.

  The area detection unit 301 distinguishes and detects a plurality of specific areas from the area of the image (input image) represented by the input image data. In this embodiment, the area detection unit 301 detects a first area and a second area from the area of the input image. In this embodiment, the second area is an area corresponding to normal display in which the dynamic range of luminance is a normal value. The first area is an area corresponding to a display (HDR display) having a wider dynamic range of luminance than the display corresponding to the second area. In this embodiment, the first area is described as “HDR area”, and the second area is described as “normal area”. The region detection unit 301 outputs input image data in a region including at least one region A1 of the HDR region and the normal region to the input selection unit 305. In addition, the region detection unit 301 outputs input image data in a region including at least the other region A2 of the HDR region and the normal region to the image exchange unit A303. In this embodiment, the input image data in the area A1 is output to the input selection unit 305, and the input image data in the area A2 is output to the image exchange unit A303. Hereinafter, the input image data in the area A1 is described as “image data D1”, and the input image data in the area A2 is described as “image data D2”.

  Note that the first region and the second region are not limited to the above-described regions. For example, the first region may be a region including a large amount of the first color, and the second region may be a region including a large amount of the second color. The first color and the second color are not particularly limited. For example, the first color is an achromatic color, and the second color is a chromatic color.

  In addition, the detection method of a specific area is not specifically limited. For example, the specific area can be detected using the upper several bits of each pixel value of the input image data. In addition, when region information indicating a specific region is added to the input image data, the specific region can be detected by referring to the region information.

  The control unit 302 determines which type of specific region is large based on the type and size of each specific region detected by the region detection unit 301. Further, the control unit 302 can perform communication with other image processing apparatuses, and the type and size of each specific area detected by the image processing apparatus 300 can be detected by each image processing apparatus detected by the other image processing apparatus. It can be compared with the type and size of a specific area.

  The image exchange unit A303 performs communication with other image processing apparatuses. Specifically, the image exchange unit A303 outputs the image data D2 output from the region detection unit 301 to another image processing device, or acquires the image data D3 from another image processing device.

  The image holding unit 304 holds (stores) the image data D3 acquired from another image processing apparatus by the image exchange unit A303. The image data D3 held by the image holding unit 304 is read from the image holding unit 304 by the input selection unit 305 at a specific timing.

  The input selection unit 305 selects one of the image data D1 output from the region detection unit 301 and the image data D3 held by the image holding unit 304, and outputs the selected image data to the LUT 306.

  The LUT 306 is an LUT that performs first image processing. The LUT 306 outputs an output value (a first processing value that is a pixel value after the first image processing) corresponding to the pixel value of the image data input to the LUT 306. In the LUT 306, for each pixel of the image data input to the LUT 306, a process for converting the pixel value into the first processing value is performed. In the LUT 306, the first image processing is performed on the image data D1, or the first image processing is performed on the image data D3. Hereinafter, the image data obtained by applying the first image processing to the image data D1 will be referred to as “processed image data PD1”, and the image data obtained by applying the first image processing to the image data D3 will be described. This is described as “processed image data PD3”. Note that, as described in the first embodiment, an interpolation calculation of the first processing value may be performed.

  The LUT setting unit 307 sets the LUT 306 so that the image processing corresponding to the area A1 is executed as the first image processing. “LUT setting” can also be said to be “image processing setting”. For example, “a process for setting an LUT so that an image process corresponding to the area A1 is executed as the first image process” is also referred to as “a process for setting the image process corresponding to the area A1 as the first image process”. I can say that. In the present embodiment, the control unit 302 selects the HDR region as the region A1, or selects the normal region as the region A1. Therefore, the LUT setting unit 307 sets the LUT 306 between the LUT that executes image processing corresponding to the HDR area and the LUT that executes image processing corresponding to the normal area, according to the selection result of the area A1. Switch. Specifically, the control unit 302 instructs the LUT setting unit 307 according to the selection result of the area A1. Then, the LUT setting unit 307 sets the LUT 306 in accordance with an instruction from the control unit 302.

  The image exchange unit B308 performs communication with the other image processing apparatus. Specifically, the image exchange unit B308 outputs the processed image data PD3 obtained by the LUT 306 to another image processing apparatus or acquires image data from another image processing apparatus.

  In this embodiment, the other image processing apparatus includes an LUT that performs second image processing different from the first image processing. The second image processing is image processing corresponding to the area A2. The LUTs possessed by other image processing apparatuses are hereinafter referred to as “other LUTs”. The image exchange unit A303 causes the other LUT to perform the second image processing on the input image data by performing communication with another image processing apparatus. Specifically, when the image data D2 output from the region detection unit 301 is output from the image exchange unit A303 to another image processing device, the image data D2 output from the region detection unit 301 is transferred to another LUT. Entered. As a result, the second image processing is performed on the image data D2 output from the area detection unit 301 by another LUT. Therefore, “a process for causing another LUT to perform the second image processing on the input image data” is “a process for causing the other LUT to perform the second image processing on the image data D2 output from the area detection unit 301”. It can also be said. Hereinafter, the image data obtained by performing the second image processing on the image data D2 is referred to as “processed image data PD2”. Then, the image exchange unit B308 acquires the processed image data PD2 from the other image processing device by performing communication with the other image processing device.

  Note that instead of the image exchange unit A303 and the image exchange unit B308, one communication unit having the functions of the image exchange unit A303 and the image exchange unit B308 may be used.

  The composition unit 309 performs composition processing for combining the result of the first image processing by the LUT 306 and the result of the second image processing acquired by the image exchange unit B308. The combining process is a process for combining the result of applying the first image processing to the input image data and the result of applying the second image processing to the input image data. In the present embodiment, the synthesis process is a process for synthesizing the processed image data PD1 and the processed image data PD2. Therefore, a composite image that is an image obtained by performing the first image processing on the area A1 of the input image by the composition processing, and an image obtained by performing the second image processing on the area A2 of the input image. Represented composite image data is generated. Then, the processing value synthesis unit 107 outputs synthesized image data.

  Depending on the data capacity of the LUT possessed by the image processing apparatus, it may not be possible to simultaneously set a plurality of LUTs corresponding to a plurality of image processes. For example, in one image processing apparatus, only the LUT corresponding to the first image processing can be set, and the LUT corresponding to the second image processing may not be set. In such a case, the desired image processing can be performed on the area A1 of the input image, but the desired image processing cannot be performed on the area A2 of the input image. There is a risk of deterioration.

  Therefore, in this embodiment, an increase in the number of LUTs used for image processing is realized using another image processing apparatus. Thereby, it is possible to obtain highly accurate image processing results without increasing the cost of the image processing apparatus.

  FIG. 9 is a block diagram illustrating an example of the configuration of the image processing system according to the present embodiment. As shown in FIG. 9, the image processing system according to the present embodiment includes an image processing device 300 and an image processing device 400. Note that the image processing system may include three or more image processing apparatuses. The image processing apparatus 400 includes an area detection unit 401, a control unit 402, an image exchange unit A 403, an image holding unit 404, an input selection unit 405, an LUT 406, an LUT setting unit 407, an image exchange unit B 408, and a composition unit 409.

  The area detection unit 401 has the same function as the area detection unit 301. The control unit 402 has the same function as the control unit 302. The image exchange unit A403 has the same function as the image exchange unit A303. The image holding unit 404 has the same function as the image holding unit 304. The input selection unit 405 has the same function as the input selection unit 305. The LUT 406 has the same function as the LUT 306. The LUT setting unit 407 has the same function as the LUT setting unit 307. The image exchange unit B408 has the same function as the image exchange unit B308. The combining unit 409 has the same function as the combining unit 309. That is, the image processing apparatus 400 has the same configuration as the image processing apparatus 300. In the image processing apparatus 400, “first” and “second” in the above description regarding the processing of the image processing apparatus 300 are replaced, and “area A1” in the above description regarding the processing of the image processing apparatus 100 is replaced with “area A2”. Processing is performed.

  In the present embodiment, the communication is performed between the control unit 302 and the control unit 402, and the area is set so that the data size of the image data transmitted and received between the image processing device 300 and the image processing device 400 is reduced. A1 and A2 are determined. Then, LUT setting (image processing setting) is performed according to the determination results of the areas A1 and A2.

Specifically, when the HDR area detected by the area detection unit 301 is larger than the HDR area detected by the area detection unit 401, the HDR area is selected as the area A1 and the normal area is selected as the area A2. Is done. As a result, the LUT that performs image processing corresponding to the HDR area is set as the LUT 306, and the LUT that performs image processing corresponding to the normal area is the LUT 406.
Set as When the HDR area detected by the area detection unit 301 is smaller than the HDR area detected by the area detection unit 401, the normal area is selected as the area A1 and the HDR area is selected as the area A2. As a result, the LUT that performs image processing corresponding to the normal area is set as the LUT 306, and the LUT that performs image processing corresponding to the HDR area is set as the LUT 406.

  Note that the types of the areas A1 and A2 may be determined in advance by the manufacturer or may be set according to a user operation. For example, the HDR area may always be used as the area A1, and the normal area may always be used as the area A2.

  Communication for transmission / reception of image data before image processing is performed between the image exchange unit A303 and the image exchange unit A403, and communication for transmission / reception of image data after image processing is performed by the image exchange unit B308. And the image exchange unit B408.

  Next, an example of a processing flow of LUT setting processing (processing related to LUT setting) according to the present embodiment will be described. FIG. 10 is a flowchart illustrating an example of a process flow of the LUT setting process according to the present embodiment.

  First, in S <b> 201, the area detection unit 301 of the image processing apparatus 300 detects a plurality of specific areas from the first input image area, and notifies the control unit 302 of the type and size of each specific area. Similarly, the area detection unit 401 of the image processing apparatus 400 detects a plurality of specific areas from the area of the second input image, and notifies the control unit 402 of the type and size of each specific area. The first input image is an image represented by the input image data input to the image processing apparatus 300, and the second input image is an image represented by the input image data input to the image processing apparatus 400. is there. In this embodiment, an HDR area and a normal area are detected as a plurality of specific areas.

  Next, in S202, communication between the control unit 302 and the control unit 402 is performed. Thereby, the processing target area of each image processing apparatus is determined according to the detection result of S201 (the type and size of each specific area). In this embodiment, the areas A1 and A2 are determined. The determination of the processing target area may or may not be performed in each of the control unit 302 and the control unit 402. The determination of the processing target region may be performed by one of the control unit 302 and the control unit 402, and the determination result of the processing target region may be notified from one of the control unit 302 and the control unit 402 to the other.

  In step S <b> 203, the control unit 302 uses the LUT setting unit 307 to set the LUT corresponding to the processing result in step S <b> 202 as the LUT 306. Similarly, the control unit 402 uses the LUT setting unit 407 to set the LUT corresponding to the processing result of S202 as the LUT 406.

  A specific example of the processing of S202 and S203 will be described with reference to FIGS. 11 to 13 are diagrams illustrating examples of the first input image and the second input image. 11 to 13 show the two partial images constituting the original image as the first input image and the second input image, but the first input image and the second input image are partial. It is not limited to images. The first input image and the second input image may be two independent images that are independent of each other. 11 to 13, the size of the first input image is equal to the size of the second input image, but the size of the first input image may be different from the size of the second input image.

FIG. 11 shows a case where the first input image includes the HDR region and the normal region, and the second input image includes only the normal region. In this case, in S202, the HDR area is selected as the area A1, and the normal area is selected as the area A2. In S203, H
The LUT that performs image processing corresponding to the DR area is set as the LUT 306, and the LUT that performs image processing corresponding to the normal area is set as the LUT 406.

  When the selection and setting as described above are performed, desired image processing (image processing corresponding to the HDR region) is performed on the HDR region of the first input image by the LUT 306 of the image processing apparatus 300. Will be given. Then, desired image processing (image processing corresponding to the normal region) is performed on the normal region of the first input image and the entire region of the second input image by the LUT 406 of the image processing apparatus 400. It becomes. That is, desired image processing can be performed on the entire area of the first input image and the entire area of the second input image.

  Here, consider a case where the normal region is selected as the region A1 and the HDR region is selected as the region A2. In this case, the image corresponding to the HDR region of the first input image and the second input image (entire) are transmitted and received between the image processing device 300 and the image processing device 400. On the other hand, when the HDR area is selected as the area A1 and the normal area is selected as the area A2, an image corresponding to the normal area of the first input image is transmitted / received. The size of the normal area of the first input image is smaller than the sum of the size of the HDR area of the first input image and the size of the second input image (entire). Therefore, when the HDR area is selected as the area A1 and the normal area is selected as the area A2, the data size of the image to be transmitted / received can be reduced.

  FIG. 12 shows a case where only the normal area is included in the first input image and the HDR area and the normal area are included in the second input image. In this case, in S202, the normal area is selected as the area A1, and the HDR area is selected as the area A2. In S203, the LUT that performs image processing corresponding to the normal area is set as the LUT 306, and the LUT that performs image processing corresponding to the HDR area is set as the LUT 406.

  When the selection and setting as described above are performed, the LUT 306 of the image processing apparatus 300 performs a desired image for the entire area of the first input image and the normal area of the second input image. Processing (image processing corresponding to the normal area) is performed. Then, the desired image processing (image processing corresponding to the HDR region) is performed on the HDR region of the second input image by the LUT 406 of the image processing apparatus 400. That is, desired image processing can be performed on the entire area of the first input image and the entire area of the second input image.

  Here, consider a case where the HDR region is selected as the region A1 and the normal region is selected as the region A2. In this case, the first input image (entire) and the image corresponding to the HDR region of the second input image are transmitted and received between the image processing device 300 and the image processing device 400. On the other hand, when the normal area is selected as the area A1 and the HDR area is selected as the area A2, an image corresponding to the normal area of the second input image is transmitted / received. The size of the normal area of the second input image is smaller than the sum of the size of the first input image (entire) and the size of the HDR area of the second input image. Therefore, the data size of an image to be transmitted / received can be reduced by selecting the normal region as the region A1 and selecting the HDR region as the region A2.

  FIG. 13 shows a case where the first input image includes an HDR region and a normal region, and the second input image also includes an HDR region and a normal region. The HDR area of the first input image is larger than the HDR area of the second input image. In this case, in S202, the HDR area is selected as the area A1, and the normal area is selected as the area A2. In step S <b> 203, the LUT that performs image processing corresponding to the HDR area is set as the LUT 306, and the LUT that performs image processing corresponding to the normal area is set as the LUT 406.

  When the selection and setting as described above are performed, a desired image is selected for the HDR area of the first input image and the HDR area of the second input image by the LUT 306 of the image processing apparatus 300. Processing (image processing corresponding to the HDR area) is performed. Then, desired image processing (image processing corresponding to the normal region) is performed on the normal region of the first input image and the normal region of the second input image by the LUT 406 of the image processing apparatus 400. It becomes. That is, desired image processing can be performed on the entire area of the first input image and the entire area of the second input image.

  Here, consider a case where the normal region is selected as the region A1 and the HDR region is selected as the region A2. In this case, an image corresponding to the HDR area of the first input image and an image corresponding to the normal area of the second input image are transmitted and received between the image processing apparatus 300 and the image processing apparatus 400. It will be. On the other hand, when the HDR region is selected as the region A1 and the normal region is selected as the region A2, the image corresponding to the normal region of the first input image and the image corresponding to the HDR region of the second input image Are transmitted and received. The sum of the size of the normal area of the first input image and the size of the HDR area of the second input image is the size of the HDR area of the first input image and the size of the normal area of the second input image. Smaller than the sum of Therefore, when the HDR area is selected as the area A1 and the normal area is selected as the area A2, the data size of the image to be transmitted / received can be reduced.

  Next, an example of the overall processing flow of the image processing system according to the present embodiment will be described. FIG. 14 is a flowchart illustrating an example of the overall processing flow of the image processing system according to the present embodiment. Here, processing relating to input image data input to the image processing apparatus 300 will be described. Note that the processing related to the input image data input to the image processing apparatus 400 is the same as the processing related to the input image data input to the image processing apparatus 300, and thus the description thereof is omitted. The input image data input to the image processing device 300 represents the first input image, and the input image data input to the image processing device 400 represents the second input image.

  First, in S204, the area detection unit 301 detects a plurality of specific areas by distinguishing them from the area of the first input image. The process of S204 is the process of S201 already described. After the process of S204 (S201), the process of S202 and the process of S203 are performed, and the process proceeds to S205. If the type and size of the area do not change, the processes in S202 and S203 can be omitted.

  In S205, the region detection unit 301 determines whether a region A1 (a specific region processed by the image processing device 300) is detected from the region of the first input image. When the area A1 is detected, the area detection unit 301 outputs the image data D1 that is input image data in the area A1 to the input selection unit 305, and the process of S206 is performed. Further, the region detection unit 301 determines whether or not a region A2 (a specific region processed by the image processing device 400) is detected from the region of the first input image. When the region A2 is detected, the region detection unit 301 outputs the image data D2 that is the input image data in the region A2 to the image exchange unit A303, and the process of S207 is performed. When both the area A1 and the area A2 are detected, the process of S206 and the process of S207 are performed in parallel or in order.

  In S206, the input selection unit 305 outputs the image data D1 output from the region detection unit 301 in S205 to the LUT 306. Thereby, in the LUT 306, the first image processing (image processing corresponding to the area A1) is performed on the image data D1, and the processed image data PD1 is generated.

In S207, cooperative processing using the image processing apparatus 400 is performed. In the linkage processing, the second image processing (region A2) is performed on the image data D2 output from the region detection unit 301 in S205.
Is processed), and processed image data PD2 is generated. In the linkage processing, the image exchange unit A303 outputs the image data D2 to the image processing device 400, and the image exchange unit B308 acquires the processed image data PD2 from the image processing device 400.

  In step S208, the synthesis unit 309 acquires processed image data necessary for the synthesis process. When the area A1 is detected from the area of the first input image and the area A2 is not detected from the area of the first input image, the synthesis unit 309 uses the processed image data PD1 generated in S206. Obtained from the LUT 306. When the area A1 is not detected from the area of the first input image and the area A2 is detected from the area of the first input image, the synthesis unit 309 uses the processed image data PD2 generated in S207. Obtained from the image exchange unit B308. When both the area A1 and the area A2 are detected from the area of the first input image, the synthesis unit 309 acquires both the processed image data PD1 and the processed image data PD2. In that case, in the combining unit 309, both the processed image data PD1 and the processed image data PD2 are not always acquired simultaneously. The synthesizing unit 309 holds the acquired processed image data.

  Next, in S209, the synthesis unit 309 determines whether all the processed image data necessary for the synthesis process has been acquired. If there is processed image data that has not been acquired, the process returns to S208. If all the processed image data necessary for the synthesis process has been acquired, the process proceeds to S210.

  In S210, the composition unit 309 generates composite image data by performing a composition process using the processed image data acquired in S208. Then, the synthesis unit 309 outputs the generated synthesized image data. When the area A1 is detected from the area of the first input image and the area A2 is not detected from the area of the first input image, the combining unit 309 outputs the processed image data PD1 as combined image data. To do. When the area A1 is not detected from the area of the first input image and the area A2 is detected from the area of the first input image, the synthesis unit 309 outputs the processed image data PD2 as synthesized image data. To do. When both the area A1 and the area A2 are detected from the area of the first input image, the synthesis unit 309 outputs synthesized image data obtained by synthesizing the processed image data PD1 and the processed image data PD2.

  Next, an example of the processing flow of the processing of S207 (cooperation processing) will be described. FIG. 15 is a flowchart illustrating an example of the processing flow of the cooperation processing.

  First, in S211, the image exchange unit A303 of the image processing apparatus 300 outputs the image data D2 output from the area detection unit 301 in S205.

  Next, in S212, the image exchange unit A403 of the image processing apparatus 400 acquires the image data D2 output from the image exchange unit A303 in S211 and outputs the acquired image data D2 to the image holding unit 404. Thereby, the image data D2 output from the image exchange unit A303 in S211 is held by the image holding unit 404.

  In step S <b> 213, the input selection unit 405 determines whether the current timing is a timing at which the LUT 406 can be used. If it is determined that the current timing is not a timing at which the LUT 406 can be used, the process of S213 is repeated until it is determined that the current timing is a timing at which the LUT 406 can be used. If it is determined that the current timing is a timing at which the LUT 406 can be used, the process proceeds to S214.

In S214, the input selection unit 405 reads the image data D2 held in S212 from the image holding unit 404, and outputs the read image data D2 to the LUT 406. Thereby, in the LUT 406, the second image processing (image processing corresponding to the area A2) is performed on the image data D2, and processed image data PD2 is generated.

  The timing when the LUT 406 can be used is a timing within a period during which the LUT 406 is not processing. For example, the timing at which the LUT 406 can be used is a timing within a blank period in which input image data is not input to the image processing apparatus 400. The timing within the period in which the area B is not detected by the area detection unit 401 is also a timing at which the LUT 406 can be used. Here, consider a case where the LUT 406 can operate at a clock rate that is twice or more the clock rate of other functional units. In this case, the rising edge of the clock of the LUT 406 and the falling edge of the clock of the LUT 406 can be detected, and the processing of the LUT 406 can be time-division multiplexed. For example, when the input selection unit 405 detects the rising edge of the clock of the LUT 406, the input selection unit 405 outputs the image data from the region detection unit 401 to the LUT 406. When the input selection unit 405 detects the falling edge of the clock of the LUT 406, the input selection unit 405 outputs the image data held by the image holding unit 404 to the LUT 406. Thereby, the processing of the LUT 406 is time-division multiplexed. In this case, the timing at which the falling edge of the clock of the LUT 406 is detected can also be determined as the timing at which the LUT 406 can be used in S213.

  Next, in S215, the LUT 406 outputs the processed image data PD2 generated in S214 to the image exchange unit B408. Then, the image exchange unit B408 outputs the acquired processed image data PD2 to the image exchange unit B308.

  FIG. 16 is a time chart illustrating an example of processing timing of the image processing system according to the present embodiment. FIG. 16 is a time chart corresponding to the processing flow described with reference to FIGS. In the example of FIG. 16, images are input to both the image processing apparatus 300 and the image processing apparatus 400 during the period from time t1 to time t4, and the image processing apparatus 300 and the image processing are performed during the period from time t9 to time t10. Images are output from both devices 400. In FIG. 16, it is assumed that the area A1 and the area A2 are predetermined. Note that the timing at which an image is input to the image processing apparatus 300 may be different from the timing at which an image is input to the image processing apparatus 400. The timing at which an image is output from the image processing apparatus 300 may be different from the timing at which an image is output from the image processing apparatus 400.

  In the example of FIG. 16, the region detection unit 301 inputs the image D1-1 input to the image processing apparatus 300 during the period from time t1 to time t2 and input to the image processing apparatus 300 during the period from time t3 to time t4. Each of the images D1-2 is determined to be an image of the area A. An area A is an area processed by the image processing apparatus 300. Then, the first image processing is performed on the images D1-1 and D1-2 by the LUT 306. The processed image PD1-1 is generated by performing the first image processing on the image D1-1, and the processed image PD1-2 is generated by performing the first image processing on the image D1-2. Is done. The processed image PD1-1 is input to the combining unit 309 at time t5, and the processed image PD1-2 is input to the combining unit 309 at time t6. The processed images PD1-1 and PD1-2 are held by the synthesis unit 309.

  Further, the area detection unit 301 determines that the image D2 input to the image processing apparatus 300 during the period from the time t2 to the time t3 is an image of the area B. Area B is an area processed by the image processing apparatus 400. Then, the image exchange unit A303 outputs the image D2 to the image processing apparatus 400.

On the other hand, the area detection unit 401 determines that the image D4 (entire image) input to the image processing apparatus 400 during the period from time t1 to time t4 is an image of the area B. Then, the LUT 406 performs second image processing on the image D4. The processed image PD4 is generated by performing the second image processing on the image D4. The processed image PD4 is input to the synthesis unit 409 at time t5. The processed image PD4 is held by the synthesis unit 409.

  Here, the image D2 output from the image processing apparatus 300 is input to the image exchange unit A403 of the image processing apparatus 400 at time t11 and is held by the image holding unit 404. However, this timing is a timing within a period during which the LUT 406 performs processing on the image D4, and is not a timing at which the LUT 406 can be used for processing on the image D2. Therefore, the input selection unit 405 waits until the timing when the LUT 406 can be used for processing the image D2. Then, the input selection unit 405 reads the image D2 from the image holding unit 404 at time t12 within the blank period during which no image is input to the image processing apparatus 400, and outputs the read image D2 to the LUT 406. Thereafter, the LUT 406 performs second image processing on the image D2. By performing the second image processing on the image D2, a processed image PD2 is generated. The processed image PD2 is output to the image processing apparatus 300 by the image exchange unit B408.

  Then, at time t7, the processed image PD2 is input to the image exchanging unit B308 of the image processing apparatus 300, and the image exchanging unit B308 outputs the processed image PD2 to the synthesizing unit 309. Thereafter, the composition processing by the composition unit 309 is performed by time t9, and images are output from the composition unit 309 of the image processing device 300 and the composition unit 409 of the image processing device 400 at time t9. A synthesized image obtained by synthesizing the processed images PD1-1, PD1-2, and PD2 is output from the synthesizing unit 309, and a processed image PD4 is output from the synthesizing unit 409.

  If these series of processes are completed within one frame period of the input image, the input image of each frame can be processed without any problem. If the composition unit 309 has a configuration capable of holding images of a plurality of frames, the input image of the next frame can be processed without waiting for the completion of image output by the composition unit 309.

  FIG. 17 is also a time chart corresponding to the processing flow described with reference to FIGS. FIG. 17 shows an example in which the LUT 406 can operate at a clock rate more than twice the clock rate of the other functional units, and the processing of the LUT 406 is time-division multiplexed.

  In the example of FIG. 17, since the processing of the LUT 406 is time-division multiplexed, the image D2 output from the image processing device 300 is applied to the image D2 at time t11 when input to the image exchange unit A403 of the image processing device 400. LUT 406 can be used for processing. Therefore, the image holding unit 404 is substantially unnecessary. In FIG. 17, the image holding unit 404 is omitted.

  In the example of FIG. 17, the image D2 is input to the LUT 406 at a time t13 earlier than the time t12 in FIG. Then, at time t <b> 14, the processed image PD <b> 2 is output from the image exchange unit B <b> 408 to the image processing apparatus 300.

  Then, at time t15 earlier than time t7 in FIG. 16, the processed image PD2 is input to the image exchanging unit B308 of the image processing apparatus 300, and the processed image PD2 is input to the synthesizing unit 309 at time t16. Thereafter, the composition processing by the composition unit 309 is performed by time t17, and images are output from the composition unit 309 of the image processing device 300 and the composition unit 409 of the image processing device 400 during a period from time t17 to time t18. The

In FIG. 17, the processed image PD2 is input to the image processing apparatus 300 at time t15 earlier than time t7 in FIG. Therefore, a time earlier than the time t9 in FIG. 16 can be set as the time t17. That is, the time from input to output of an image in the image processing apparatus can be shortened to a shorter time than that in FIG. As a result, an image with a short one frame period can be processed without any problem.

  Note that there is a technique called frame interleaving that performs image processing with skipping several pixels for one image (frame) and realizes image processing for all pixel positions by image processing for a plurality of frames. In the present embodiment, the area on which image processing is performed may be a plurality of areas (a plurality of areas having different pixel combinations) that are individually processed by frame interleaving. Each LUT may store a table value individually for each area processed by frame interleaving.

  As described above, according to the present embodiment, it is possible to obtain a high-precision image processing result without causing an increase in cost of the image processing apparatus by operating a plurality of image processing apparatuses in cooperation with each other. it can. Specifically, an image processing result equivalent to the case of using a plurality of types of LUTs can be obtained without increasing the cost of the image processing apparatus.

<Example 3>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 3 of the present invention will be described. In image processing that refers to an LUT, there are cases where two processing values respectively corresponding to two even and odd addresses adjacent to each other are read from the LUT and an intermediate value between the two processing values is obtained by interpolation. In the present embodiment, an example of an image processing system capable of performing such image processing with higher accuracy by linking a plurality of image processing apparatuses will be described.

  FIG. 18 is a block diagram illustrating an example of the configuration of the image processing apparatus 500 according to the present embodiment. Functional parts that are equivalent to the functional parts described in the above-described embodiments are assigned the same reference numerals, and detailed descriptions thereof are omitted.

  The address determination unit 501 selects the address exchange unit 102 or the address selection unit 103 according to the address attribute determined by the address determination unit 101. Then, the address determination unit 501 outputs the address determined by the address determination unit 101 to the selected function unit. Note that all addresses may be output to one of the address exchange unit 102 and the address selection unit 103. Further, the address determination unit 101 may determine two or more addresses for one pixel of the input image. For example, the address determination unit 101 may determine two even and odd addresses adjacent to each other for one pixel.

  When the image processing apparatus 500 is used alone, a lookup table equivalent to the LUT 104 is used as the LUT 502, and the LUT setting unit 503 performs settings equivalent to the LUT setting unit 105. The LUT setting unit 503 forms a lookup table different from the LUT 104 as the LUT 502 when the image processing apparatus 500 is linked with another apparatus.

  The delay adjustment unit 504 can hold the processing value output from the LUT 502. As the delay adjustment unit 504, for example, a memory element such as a FIFO (First In First Out) memory is used.

  The process value exchange unit 505 has the same function as the process value exchange unit 106. Further, the processing value exchanging unit 505 can notify the delay adjustment unit 504 of the timing at which the processing value is received from another image processing apparatus. As a result, the processing value held by the delay adjustment unit 504 can be read from the delay adjustment unit 504 in synchronization with the timing at which the processing value is received from another image processing apparatus.

FIG. 19 is a block diagram illustrating an example of the configuration of the image processing system according to the present embodiment. As shown in FIG. 19, the image processing system according to this embodiment includes an image processing device 500 and an image processing device 600. Note that the image processing system may include three or more image processing apparatuses. The image processing apparatus 600 includes an address determination unit 201, an address determination unit 601, an address exchange unit 202, an address selection unit 203, an LUT 602, an LUT setting unit 603, a delay adjustment unit 604, a processing value exchange unit 605, and a processing value synthesis unit. 207.

  The address determination unit 601 has a function equivalent to that of the address determination unit 501. The LUT 602 has the same function as the LUT 502. The LUT setting unit 603 has the same function as the LUT setting unit 503. The delay adjustment unit 604 has a function equivalent to that of the delay adjustment unit 504. The process value exchange unit 605 has a function equivalent to that of the process value exchange unit 505.

  Communication for address transmission / reception is performed between the address exchange unit 102 and the address exchange unit 202, and communication for processing value transmission / reception is performed between the processing value exchange unit 505 and the processing value exchange unit 605. Is called.

  In the image processing system according to the present embodiment, the address determination unit 501 selects the address selection unit 103 when the address is an even number (even address), and the address exchange unit 102 when the address is an odd number (odd address). Select. When both the even address and the odd address are determined, both the address selecting unit 103 and the address exchanging unit 102 are selected, the even address is output to the address selecting unit 103, and the odd address is output to the address exchanging unit 102. Is done. The address determination unit 601 selects the address selection unit 203 when the address is an odd address, and selects the address exchange unit 202 when the address is an even address. When both the even address and the odd address are determined, both the address selection unit 203 and the address exchange unit 202 are selected, the odd address is output to the address selection unit 203, and the even address is output to the address exchange unit 202. Is done.

  FIG. 20 shows an example of the LUT 502, the LUT 602, and a combined LUT obtained by combining the LUT 502 and the LUT 602. A processing value corresponding to an even address is set in the LUT 502, and a processing value corresponding to an odd address is set in the LUT 602. In this embodiment, the address determination unit 501 and the address determination unit 601 output addresses so that the LUT 502 is referred to when an even address is determined, and the LUT 602 is referred to when an odd address is determined. Select the destination. For this reason, the image processing system of this embodiment can perform image processing with higher accuracy than when only one of the LUT 502 and the LUT 602 is used.

  Note that a processing value corresponding to an odd address may be set in the LUT 502, and a processing value corresponding to an even address may be set in the LUT 602. In that case, the address determination unit 501 may select the address selection unit 103 when the address is an odd address, and may select the address exchange unit 102 when the address is an even address. The address determination unit 601 may select the address selection unit 203 when the address is an even address, and may select the address exchange unit 202 when the address is an odd address.

  As described above, in this embodiment, the processing value held by the delay adjustment unit 504 can be read from the delay adjustment unit 504 in synchronization with the timing at which the processing value exchange unit 505 receives the processing value of the LUT 602. Therefore, the process value synthesis unit 107 can simultaneously use the process value corresponding to the even address and the process value corresponding to the odd address for the interpolation calculation (synthesis process).

FIG. 21 is a flowchart illustrating an example of the processing flow of the present embodiment. Processes equivalent to those described in the above-described embodiment are denoted by the same reference numerals (step numbers), and detailed description thereof is omitted.

  First, the processing of S101 to S103 is performed. Here, it is assumed that both the even address and the odd address are determined in S101. Therefore, in S102, the address determination unit 501 outputs the odd address determined in S101 to the address exchange unit 102, and the address exchange unit 102 outputs the odd address determined in S101 to the address exchange unit 202. Start (address exchange). The address determination unit 501 outputs the even address determined in S101 to the address selection unit 103. In step S <b> 103, the address selection unit 103 outputs the even address determined in step S <b> 101 to the LUT 502. As a result, the processing value corresponding to the even address determined in S101 is output from the LUT 502 to the delay adjustment unit 504.

  Next, in S301, the delay adjustment unit 504 acquires the processing value corresponding to the even address determined in S101 from the LUT 502 and holds it. And the process of S105-S107 is performed. In S105, the address exchange unit 202 of the image processing apparatus 600 determines whether or not the process (address exchange) for acquiring the odd address determined in S101 from the address exchange unit 102 has been completed. In S106, the address exchange unit 202 outputs the odd address determined in S101 to the LUT 602 via the address selection unit 203. As a result, the processing value corresponding to the odd address determined in S101 is output from the LUT 602 to the processing value exchanging unit 605. In S107, the process value exchanging unit 605 starts a process (process value exchange) of outputting the process value corresponding to the odd address determined in S101 to the process value exchanging unit 505.

  In step S302, the process value exchanging unit 505 determines whether the process (process value exchange) for acquiring the process value corresponding to the odd-numbered address determined in step S101 from the process value exchanging unit 605 is completed. The process of S302 is repeated until the process value exchange is completed. At the timing when the process value exchange is completed, the timing is notified from the process value exchange unit 505 to the delay adjustment unit 504, and the process proceeds to S303. When the processing value exchange is completed, the processing value exchanging unit 505 outputs the processing value acquired from the processing value exchanging unit 605 (the processing value corresponding to the odd address determined in S101) to the processing value combining unit 107. .

  In S303, the delay adjustment unit 504 outputs the retained processing value (the processing value corresponding to the even address determined in S101) to the processing value synthesis unit 107 in synchronization with the notification generated in S302. .

  In step S <b> 304, the process value synthesis unit 107 performs an interpolation operation using the process value output from the LUT 502 and the process value output from the LUT 602. That is, an interpolation operation is performed using the processing value corresponding to the even address determined in S101 and the processing value corresponding to the odd address determined in S101. This interpolation calculation can also be said to be “combining processing for combining two processing values”. When only an even address is determined, only the processing value corresponding to the even address (the processing value output from the LUT 502) is used. When only the odd address is determined, only the processing value corresponding to the odd address (processing value output from the LUT 602) is used. The pixel value of the output image data is determined and output by the interpolation calculation in S304. Note that, in the interpolation calculation of S304, input image data may be used as in the processing of the interpolation unit 109 of the first embodiment.

  FIG. 22 is a time chart illustrating an example of processing timing of the image processing system according to the present embodiment. FIG. 22 is a time chart corresponding to the processing flow described with reference to FIG.

  At time t1, image data is input to the address determination unit 101 of the image processing apparatus 500, and address determination is started. At time t2, address determination is completed, and the determined address is input to address determination unit 501. Then, the address determination unit 501 outputs the even address to the LUT 502 of the image processing apparatus 500 and outputs the odd address to the LUT 602 of the image processing apparatus 600. At time t3, the LUT 502 reads (outputs) the processing value corresponding to the even address output from the address determination unit 501. At time t4, the processing value read at time t3 is input to the delay adjustment unit 504 and held.

  At time t5, the LUT 602 reads the processing value corresponding to the odd address input from the address exchange unit 102 to the LUT 602 via the address exchange unit 202 and the address selection unit 203. Thereafter, the read processing value is input to the processing value exchanging unit 605. At time t6, a process of outputting the read process value from the process value exchange unit 605 to the process value exchange unit 505 is started.

  At time t7, the processing value exchange unit 505 completes the processing value read by the LUT 602. At this time, the processing value exchange unit 505 notifies the delay adjustment unit 504, and in synchronization with the notification, the processing value held by the delay adjustment unit 504 (the processing value read by the LUT 502) is delayed. The process of reading from 504 is started. At time t8, the processing value read by the LUT 502 and the processing value read by the LUT 602 are input to the processing value synthesis unit 107, and interpolation calculation starts.

  Note that the image processing apparatus 600 may perform processing on the input image data of the image processing apparatus 600 during a period in which the processing of the input image data of the image processing apparatus 500 is performed. In such a case, in the LUT 602, the processing value corresponding to the address determined by the image processing apparatus 500 is read twice and the processing value corresponding to the address determined by the image processing apparatus 600 is read twice (2). Type) is read out. Therefore, in such a case, it is necessary to operate the LUT 602 at a rate that is at least twice the rate of the input image data. For example, if the operating frequency (operating speed) of the LUT 602 is increased to at least twice the operating frequency of the other functional units, the LUT 602 can be operated at a rate that is at least twice the rate of the input image data.

  In addition, the LUT 502 may be used (referenced) by the image processing apparatus 600 during a period in which the image processing apparatus 500 performs processing on input image data of the image processing apparatus 500. In that case, it is necessary to operate the LUT 502 at a rate more than twice the rate of the input image data for the same reason as described above.

  As described above, according to the present embodiment, it is possible to obtain a high-precision image processing result without causing an increase in cost of the image processing apparatus by operating a plurality of image processing apparatuses in cooperation with each other. it can. Specifically, an image processing result equivalent to the case of using an LUT that performs high-precision image processing can be obtained without increasing the cost of the image processing apparatus. The image processing system of the present embodiment can cope with high-precision image processing for HDR image data. In this embodiment, an example in which image processing is performed using an LUT has been described. However, the present invention is not limited to this. The image processing method is not particularly limited. For example, image processing for converting pixel values by calculation using a function may be performed.

<Example 4>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 4 of the present invention will be described. FIG. 23 is a block diagram illustrating an example of the configuration of the image processing apparatus 700 according to the present embodiment. Functional parts that are equivalent to the functional parts described in the above-described embodiments are assigned the same reference numerals, and detailed descriptions thereof are omitted.

  The address determination unit 701 selects the address exchange unit 102 or the address selection unit 103 according to the address attribute determined by the address determination unit 101. Then, the address determination unit 701 outputs the address determined by the address determination unit 101 to the selected function unit. Note that all addresses may be output to one of the address exchange unit 102 and the address selection unit 103. Further, the address determination unit 701 determines an identification number corresponding to the address determined by the address determination unit 101 (address used for processing). When the address determination unit 701 outputs an address to the address selection unit 103, the address determination unit 701 notifies the delay adjustment unit 704 described later of the identification number of the address.

  When the image processing apparatus 700 is used alone, a lookup table equivalent to the LUT 104 is used as the LUT 702, and the LUT setting unit 703 performs settings equivalent to the LUT setting unit 105. The LUT setting unit 703 forms a lookup table different from the LUT 104 as the LUT 702 when the image processing apparatus 700 is linked with other apparatuses.

  The delay adjustment unit 704 can hold the processing value output from the LUT 702. As the delay adjustment unit 704, for example, a memory element such as a FIFO (First In First Out) memory is used. The delay adjustment unit 704 can control reading of the retained processing value based on the identification number from the address determination unit 701 and the notification from the processing value exchange unit 505. Details of this will be described later.

  FIG. 24 is a block diagram illustrating an example of the configuration of the image processing system according to the present embodiment. As shown in FIG. 24, the image processing system according to the present embodiment includes an image processing apparatus 700 and an image processing apparatus 800. Note that the image processing system may include three or more image processing apparatuses. The image processing apparatus 800 includes an address determination unit 201, an address determination unit 801, an address exchange unit 202, an address selection unit 203, an LUT 802, an LUT setting unit 803, a delay adjustment unit 804, a processing value exchange unit 605, and an interpolation unit 209. Have.

  The address determination unit 801 has a function equivalent to that of the address determination unit 701. The LUT 802 has the same function as the LUT 702. The LUT setting unit 803 has the same function as the LUT setting unit 703. The delay adjustment unit 804 has the same function as the delay adjustment unit 704.

  Communication for address transmission / reception is performed between the address exchange unit 102 and the address exchange unit 202, and communication for processing value transmission / reception is performed between the processing value exchange unit 505 and the processing value exchange unit 605. Is called.

  In the image processing system according to the present embodiment, the address determination unit 701 divides the address space into two groups. That is, the address determination unit 701 sets two partial ranges that constitute a range of values that can be taken by the address. Three or more partial ranges may be set as a plurality of partial ranges constituting a range of values that can be taken by the address. The plurality of partial ranges may be arbitrarily determined. The address determination unit 501 selects the address selection unit 103 when the address belongs to the group with the smaller address value (first group), and the address belongs to the group with the larger address value (second group). In this case, the address exchange unit 102 is selected. The “address value” means “address address size”. Further, the address determination unit 801 selects the address selection unit 203 when the address belongs to the second group, and selects the address exchange unit 202 when the address belongs to the first group. The group with the larger address value may be used as the first group, and the group with the smaller address value may be used as the second group.

FIG. 25 illustrates an example of the LUT 702, the LUT 802, and a combined LUT in which the LUT 702 and the LUT 802 are combined. A processing value corresponding to an address belonging to the first group is set in the LUT 702, and a processing value corresponding to an address belonging to the second group is set in the LUT 802. The address determination unit 701 and the address determination unit 801 refer to the LUT 702 when an address corresponding to the first group is determined, and refer to the LUT 802 when an address corresponding to the second group is determined. Select the output destination of the address. For this reason, in the image processing system of the present embodiment, it is possible to perform image processing with higher accuracy than when only one of the LUT 702 and the LUT 802 is used.

  In this embodiment, the interpolation unit 109 generates and outputs output image data in which each pixel value is a pixel value based on the processing value of the LUT 702 or a pixel value based on the processing value of the LUT 802 by the synthesis process. Specifically, every time a processing value is input, the interpolation unit 109 outputs the input processing value (a pixel value based on the input processing value) as a pixel value of the output image data.

  The time required for the image processing apparatus 700 to acquire the processing value of the LUT 802 may be much longer than the time required for the image processing apparatus 700 to acquire the processing value of the LUT 702. If such a difference in processing time is not taken into account, the interpolation unit 109 performs processing based on the processing value of the LUT 702 at a timing at which processing based on the processing value of the LUT 802 is to be performed, and the pixel of the output image data An incorrect value may be obtained as a value.

  Therefore, in this embodiment, the delay adjustment unit 704 is used to adjust the time taken for the processing value to reach the interpolation unit 109. Specifically, transmission of the processing value of the LUT 702 to the interpolation unit 109 is intentionally delayed using the delay adjustment unit 704. Thereby, the interpolation unit 109 can use the processing value to be used more reliably. As a result, pixel values (pixel values of input image data) corresponding to input values belonging to the first group can be converted into pixel values (pixel values of output image data) based on the processing values of the LUT 702. Then, the pixel value (pixel value of the input image data) corresponding to the input value belonging to the second group can be converted into the pixel value (pixel value of the output image data) based on the processing value of the LUT 802.

  The processing of the delay adjustment unit 704 will be described with reference to FIG. The address determination unit 701 determines an identification number corresponding to the address used for the processing of the address determination unit 701. The identification number increases by one each time the address determination unit 701 is processed. Then, it is initialized at a predetermined timing such as the timing of the horizontal synchronizing signal. When the address determination unit 701 outputs an address to the LUT 702 via the address selection unit 103, the address determination unit 701 notifies the delay adjustment unit 704 of the identification number of the address. When an address is input to the LUT 702, a processing value corresponding to the address is output from the LUT 702 to the delay adjustment unit 704. The delay adjustment unit 704 holds the input identification number and processing value in association with each other.

  The delay adjustment unit 704 reads out (outputs) the processing values in the order of identification numbers. In the example of FIG. 26, the processing values of identification numbers # 001, # 002, and # 003 are read in order. Here, the identification number next to the identification number # 003 is # 004, but the delay adjustment unit 704 does not hold a processing value corresponding to the identification number # 004. The processing values not held by the delay adjustment unit 704 are output from the LUT 802. For this reason, the delay adjustment unit 704 stops reading the processing value until the processing value of the identification number # 004 is acquired by the processing value exchanging unit 505 and output to the interpolation unit 109. The process value exchange unit 505 notifies the delay adjustment unit 704 at the timing when the process value exchange is completed. Based on this notification, the delay adjustment unit 704 resumes reading of the processing value. In the example of FIG. 26, after the processing value corresponding to the identification number # 004 is output from the processing value exchanging unit 505 to the interpolation unit 109, the delay adjustment unit 704 reads the processing value corresponding to the identification number # 005. .

By performing such processing, the interpolation unit 109 can perform processing based on the processing value in the order of identification numbers (processing order of the address determination unit 701). In some cases, a difference between the time required to acquire the processing value of the LUT 702 and the time required to acquire the processing value of the LUT 802 is determined in advance. In that case, the delay adjustment unit 704 may determine a time to stop reading the processing value (processing value of the LUT 702) based on the difference.

  FIG. 27 is a flowchart illustrating an example of the processing flow of the present embodiment. Processes equivalent to those described in the above-described embodiment are given the same step numbers, and detailed descriptions thereof are omitted.

  First, the process of S101 is performed. Next, in S401, the address determination unit 701 determines whether the address determined in S101 belongs to the first group or the second group. If the address determined in S101 belongs to the first group, the address determination unit 701 outputs the address determined in S101 to the address selection unit 103, and the process proceeds to S402. If the address determined in S101 belongs to the second group, the address determination unit 701 outputs the address determined in S101 to the address exchange unit 102, and the process proceeds to S102.

  In S402, the address determination unit 701 notifies the delay adjustment unit 704 of the identification number corresponding to the address determined in S101. Next, the process of S103 is performed. In S103, the address selection unit 103 outputs the address determined in S101 to the LUT 702. As a result, the processing value corresponding to the address determined in S101 is output from the LUT 702 to the delay adjustment unit 704. In step S403, the delay adjustment unit 704 holds the identification number input in step S402 and the processing value input in step S103 in association with each other. Thereafter, the process proceeds to S404.

  When the process has proceeded to S102, the process of S102, the process of S106, and the process of S107 are performed. Thereby, the process value exchanging unit 505 acquires the process value of the LUT 802 from the process value exchanging unit 605 as the process value corresponding to the address determined in S101. Thereafter, the process proceeds to S404.

  In S404, the delay adjustment unit 704 adjusts the output timing of the processing value of the LUT 702 so that the processing value is transmitted to the interpolation unit 109 according to the processing order of the address determination unit 701 by using an identification number or the like. , The processing value of the LUT 702 is output to the interpolation unit 109. Then, the interpolation unit 109 performs processing based on the processing values according to the identification number order (processing order of the address determination unit 701).

  Note that the image processing apparatus 800 may perform processing on the input image data of the image processing apparatus 800 during a period in which the processing of the input image data of the image processing apparatus 700 is performed. In such a case, in the LUT 802, the processing value corresponding to the address determined by the image processing apparatus 700 is read twice and the processing value corresponding to the address determined by the image processing apparatus 800 is read twice (2). Type) is read out. Therefore, in such a case, it is necessary to operate the LUT 802 at a rate that is at least twice the rate of the input image data. For example, if the operating frequency (operating speed) of the LUT 802 is increased to at least twice the operating frequency of the other functional units, the LUT 802 can be operated at a rate that is at least twice the rate of the input image data.

  Further, the LUT 702 may be used (referenced) by the image processing apparatus 800 during a period in which the image processing apparatus 700 performs processing on input image data of the image processing apparatus 700. In that case, it is necessary to operate the LUT 702 at a rate more than twice the rate of the input image data for the same reason as described above.

  As described above, according to the present embodiment, it is possible to obtain a high-precision image processing result without causing an increase in cost of the image processing apparatus by operating a plurality of image processing apparatuses in cooperation with each other. it can. Specifically, an image processing result equivalent to the case of using an LUT that performs high-precision image processing can be obtained without increasing the cost of the image processing apparatus. The image processing system of the present embodiment can cope with high-precision image processing for HDR image data. In this embodiment, an example in which image processing is performed using an LUT has been described. However, the present invention is not limited to this. The image processing method is not particularly limited. For example, image processing for converting pixel values by calculation using a function may be performed.

<Example 5>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 5 of the present invention will be described. FIG. 28 is a block diagram illustrating an example of the configuration of the image processing apparatus 900 according to the present embodiment. Functional parts that are equivalent to the functional parts described in the above-described embodiments are assigned the same reference numerals, and detailed descriptions thereof are omitted.

  The color-specific address determination unit 901 determines a plurality of addresses respectively corresponding to a plurality of color components as addresses corresponding to the pixel values of the input image data. Specifically, the address determination unit 901 for each color determines a value (color component value) corresponding to the color component from the pixel value for each of the plurality of color components, and determines an address according to the determined color component value. To do. In this embodiment, an address corresponding to the first color component and an address corresponding to the second color component are determined. Then, the color-specific address determination unit 901 outputs an address corresponding to the first color component to the address selection unit 103, and outputs an address corresponding to the second color component to the address exchange unit 102. Note that all addresses may be output to one of the address exchange unit 102 and the address selection unit 103. Also, three or more color components may be considered as the plurality of color components.

  The LUT 902 is a lookup table that performs image processing on input image data. When an address (input value) is input, the LUT 902 outputs an output value (a processing value that is a pixel value after image processing) corresponding to the input address. When the address determined by the color-specific address determination unit 901 is input to the LUT 902, the LUT 902 performs image processing on the input image data. In this embodiment, the color-specific address determination unit 901 determines a plurality of addresses corresponding to a plurality of color components. Therefore, a plurality of LUTs corresponding to a plurality of color components can be set as the LUT 902. A detailed configuration of the LUT 902 will be described later. The LUT setting unit 903 sets the LUT 902.

  29 and 30 show an example of the configuration of the LUT 902. 29 and 30 includes an LUT corresponding to the first color component (Cb) and an LUT corresponding to the second color component (Cr). 29 and 30, “ADR1” is an address corresponding to the value of the first color component, and “ADR2” is an address corresponding to the value of the second color component. Here, the capacity of the LUT 902 (SRAM data capacity) is X × Y bits. The LUT 902 shown in FIGS. 29 and 30 is used when the image processing apparatus 900 is used alone, for example. The first color component is not limited to “Cb”, and the second color component is not limited to “Cr”.

  In FIG. 29, in each of the LUT corresponding to the first color component and the LUT corresponding to the second color component, the bit width of the processing value per word (address) is X / 2 bits, and the word The number is a Y word. Therefore, the total capacity of the two LUTs is X × Y bits, and the LUT 902 shown in FIG. 29 can be set.

In FIG. 30, in each of the LUT corresponding to the first color component and the LUT corresponding to the second color component, the bit width of the processing value per word is X bits, and the number of words is Y
/ 2 words. Therefore, the total capacity of the two LUTs is X × Y bits, and the LUT 902 shown in FIG. 30 can also be set.

  FIG. 31 is a block diagram illustrating an example of the configuration of the image processing system according to the present embodiment. As shown in FIG. 31, the image processing system according to this embodiment includes an image processing apparatus 900 and an image processing apparatus 1000. Note that the image processing system may include three or more image processing apparatuses. The image processing apparatus 1000 includes a color-specific address determination unit 1001, an address exchange unit 202, an address selection unit 203, an LUT 1002, an LUT setting unit 1003, a delay adjustment unit 604, a process value exchange unit 605, and a process value composition unit 207. .

  The color-specific address determination unit 1001 has the same function as the color-specific address determination unit 901. The LUT 1002 has the same function as the LUT 902. The LUT setting unit 1003 has the same function as the LUT setting unit 903.

  Communication for address transmission / reception is performed between the address exchange unit 102 and the address exchange unit 202, and communication for processing value transmission / reception is performed between the processing value exchange unit 505 and the processing value exchange unit 605. Is called.

  In the image processing system according to the present embodiment, the color-specific address determination unit 901 outputs an address corresponding to the first color component to the address selection unit 103, and an address corresponding to the second color component is the address exchange unit 102. Output to. Further, the color-specific address determination unit 1001 outputs an address corresponding to the second color component to the address selection unit 203 and outputs an address corresponding to the first color component to the address exchange unit 202.

  FIG. 32 shows an example of the LUT 902 and the LUT 1002. In FIG. 32, an LUT 902 is an LUT corresponding to the first color component, and an LUT 1002 is an LUT corresponding to the second color component. When the LUT 902 does not include an LUT corresponding to another color component (second color component), a high-precision LUT corresponding to the first color component can be set as the LUT 902. Similarly, when the LUT 1002 does not include an LUT corresponding to another color component (first color component), a high-precision LUT corresponding to the second color component can be set as the LUT 1002. In FIG. 32, in each of the LUT 902 and the LUT 1002, the bit width of the processing value per word is X bits, and the number of words is Y words. Therefore, the LUT 902 in FIG. 32 is more accurate than the LUT in FIG. 29 and 30 (LUT corresponding to the first color component), and the LUT 1002 in FIG. 32 corresponds to the LUT in FIG. 29 and 30 (the second color component). More accurate than LUT).

  In this embodiment, the color-specific address determination unit 901 and the color-specific address determination unit 1001 refer to the LUT 902 for the address corresponding to the first color component, and refer to the LUT 1002 for the address corresponding to the second color component. As described above, the output destination of the address is selected. For this reason, the image processing system of the present embodiment can perform image processing with higher accuracy than when only one of the LUT 902 and the LUT 1002 is used.

  FIG. 33 is a flowchart illustrating an example of a processing flow of the present embodiment. Processes equivalent to those described in the above-described embodiment are given the same step numbers, and detailed descriptions thereof are omitted. First, in S501, the color-specific address determination unit 901 determines an address corresponding to the first color component and an address corresponding to the second color component from the pixel value of the input image data. The color-specific address determination unit 901 outputs the address corresponding to the first color component (the address determined in S501) to the address selection unit 103, and the address corresponding to the second color component (the address determined in S501). The data is output to the address exchange unit 102. And the process of S102-S304 is performed.

  This embodiment is effective when image processing for each color component is performed individually, and can also be applied to RAW image debayer processing. For example, when there are color components such as R, G1, G2, and B, a plurality of image processing apparatuses can share and execute image processing for each color component.

  Note that the image processing apparatus 1000 may perform processing on the input image data of the image processing apparatus 1000 during the period in which the processing of the input image data of the image processing apparatus 900 is performed. In such a case, in the LUT 1002, the processing value corresponding to the address determined by the image processing apparatus 900 is read and the processing value corresponding to the address determined by the image processing apparatus 1000 is read twice (2 Type) is read out. Therefore, in such a case, it is necessary to operate the LUT 1002 at a rate that is at least twice the rate of the input image data. For example, if the operating frequency (operating speed) of the LUT 1002 is increased to at least twice the operating frequency of other functional units, the LUT 1002 can be operated at a rate that is at least twice the rate of the input image data.

  Further, the LUT 902 may be used (referenced) by the image processing apparatus 1000 during the period in which the image processing apparatus 900 performs processing on the input image data of the image processing apparatus 900. In that case, it is necessary to operate the LUT 902 at a rate more than twice the rate of the input image data for the same reason as described above.

  As for the color components of an image, there are those in which one pixel value is set for every two pixels, such as Cb and Cr of Y: Cb: Cr = 4: 2: 2 format. When handling such Cb and Cr, the LUT can be operated at the same rate as the rate of the input image data.

  Further, Cb and Cr information may be alternately input for each line, such as Cb and Cr in the Y: Cb: Cr = 4: 2: 0 format. In this case, one of the processing for the input image data of the image processing apparatus 900 and the processing for the input image data of the image processing apparatus 1000 may be delayed by one line by using a line memory. Accordingly, it is possible to prevent the image processing apparatus 900 and the image processing apparatus 1000 from using the same LUT (at least one of the LUT 902 and the LUT 1002) at the same time. Even in such a case, the lookup table can be operated at the same rate as the rate of the input image data.

  Further, there may be a case where only one color component value exists at each coordinate forming an image, like a Bayer array in a RAW image. In this case, when image processing at a certain coordinate is performed, only the LUT of one color component is referred to. Accordingly, two LUTs having different color components may be set as the LUT 902 and the LUT 1002 so that each image processing apparatus does not refer to the same LUT at the same time. Thereby, the lookup table can be operated at the same rate as the rate of the input image data.

  In the case of the Y: Cb: Cr = 4: 2: 0 format, it is possible to prevent the Cb image processing and the Cr image processing from occurring simultaneously. In such a case, of the LUT 902 and the LUT 1002, the operation of the LUT that is not used for image processing can be stopped. Thereby, it is possible to obtain effects such as low power consumption of the image processing system and reduction of heat generation while suppressing local power concentration.

As described above, according to the present embodiment, it is possible to obtain a high-precision image processing result without causing an increase in cost of the image processing apparatus by operating a plurality of image processing apparatuses in cooperation with each other. it can. Specifically, an image processing result equivalent to the case of using an LUT that performs high-precision image processing can be obtained without increasing the cost of the image processing apparatus. The image processing system according to the present embodiment can cope with high-accuracy image processing on HDR image data. In this embodiment, an example in which image processing is performed using an LUT has been described. However, the present invention is not limited to this. The image processing method is not particularly limited. For example, image processing for converting pixel values by calculation using a function may be performed.

  In addition, Examples 1-5 are an example to the last, and the structure obtained by changing suitably and changing the structure of Examples 1-5 within the range of the summary of this invention is also contained in this invention. Configurations obtained by appropriately combining the configurations of Examples 1 to 5 are also included in the present invention.

<Other examples>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100, 200, 300, 400, 500: Image processing device 600, 700, 800, 900, 1000: Image processing device 102, 202: Address exchange unit 104, 204, 306, 406, 502: LUT
602, 702, 802, 902, 1002: LUT
106, 206, 505, 605: processing value exchanging unit 107, 207: processing value combining unit 303, 403: image exchanging unit A
308, 408: Image exchange unit B
309, 409: synthesis unit 504, 604, 704, 804: delay adjustment unit

Claims (23)

An image processing apparatus,
First processing means for performing first image processing on input image data;
The second image processing on the input image data is performed by performing communication with another image processing apparatus having a second processing unit that performs second image processing different from the first image processing. Communication means for causing the second processing means to perform the second image processing on the input image data, and acquiring the result from the other image processing apparatus;
Combining means for combining the result of the first image processing by the first processing means and the result of the second image processing acquired by the communication means;
An image processing apparatus comprising: The first processing unit and the second processing unit output an output value corresponding to the input value when an input value corresponding to a pixel value is input,
An input value corresponding to a pixel value of the input image data is input to the first processing means,
The communication means includes
By outputting the input value corresponding to the pixel value of the input image data to the other image processing device, an output value corresponding to the input value is output to the second processing unit,
Obtaining the output value output from the second processing means from the other image processing apparatus;
The combining process includes: an output value of the first processing unit corresponding to the input value corresponding to the pixel value of the input image data; and an output value of the second processing unit corresponding to the input value. The image processing apparatus according to claim 1, wherein the image processing apparatus is a composition process. The first processing means outputs a first partial value that is one of a plurality of partial values constituting a pixel value obtained by performing predetermined image processing on the pixel value;
3. The second processing means outputs a second partial value that is one of the plurality of partial values and that is a partial value different from the first partial value. An image processing apparatus according to 1. The input value corresponding to the pixel value of the input image data includes a first input value corresponding to the first color component value of the pixel value and a second input value corresponding to the second color component value of the pixel value. Including
The first input value is input to the first processing means,
The communication means causes the second processing means to output an output value corresponding to the second input value by outputting the second input value to the other image processing apparatus. The image processing apparatus according to 3. Detecting means for detecting a first area and a second area from the area of the image represented by the input image data;
The first image processing is image processing corresponding to one of the first region and the second region;
The second image processing is image processing corresponding to the other region of the first region and the second region;
The communication means includes
Based on the detection result of the detection means, the output image data, which is the input image data in a region including at least the other region, is output to the other image processing device, whereby the second image with respect to the output image data is output. Causing the second processing means to perform image processing;
Obtaining the result of performing the second image processing on the output image data from the other image processing device,
The synthesizing unit is an image obtained by performing the first image processing on the one area by the synthesis process, and an image obtained by performing the second image process on the other area. The image processing apparatus according to claim 1, wherein composite image data representing a composite image is generated. The image processing apparatus according to claim 5, wherein the output image data is the input image data in the other region. The image processing apparatus according to claim 5, wherein the first area is an area corresponding to a display having a wider dynamic range than a display corresponding to the second area. The first processing unit and the second processing unit output an output value corresponding to the input value when an input value corresponding to a pixel value is input,
When the first input value is included in one or more input values corresponding to the pixel value of the input image data, the first input value is input to the first processing means,
When the one or more input values corresponding to the pixel values of the input image data include a second input value, the communication unit includes:
By outputting the second input value to the other image processing device, an output value corresponding to the second input value is output to the second processing unit,
Obtaining the output value output from the second processing means from the other image processing apparatus;
The combining process corresponds to the first input value when the one or more input values corresponding to the pixel value of the input image data include the first input value and the second input value. The image processing apparatus according to claim 1, wherein the image processing apparatus is a process of combining the output value of the first processing unit and the output value of the second processing unit corresponding to the second input value. The first processing unit and the second processing unit output an output value corresponding to the input value when an input value corresponding to a pixel value is input,
The possible range of the input value includes a plurality of partial ranges including a first range and a second range,
When the input value according to the pixel value of the input image data belongs to the first range, the input value according to the pixel value of the input image data is input to the first processing means,
When the input value according to the pixel value of the input image data belongs to the second range, the communication means
By outputting the input value corresponding to the pixel value of the input image data to the other image processing device, an output value corresponding to the input value corresponding to the pixel value of the input image data is set to the second value. Output to the processing means,
Obtaining the output value output from the second processing means from the other image processing apparatus;
The synthesizing unit generates, by the synthesizing process, image data in which each pixel value is a pixel value based on an output value of the first processing unit or a pixel value based on an output value of the second processing unit. The image processing apparatus according to claim 1, wherein: The pixel value of the input image data corresponding to the input value belonging to the first range is converted into a pixel value based on the output value of the first processing means, and the input value belonging to the second range corresponds to the input value belonging to the second range. Delay means for delaying transmission of the output value of the first processing means to the synthesizing means so that the pixel value of the input image data is converted into a pixel value based on the output value of the second processing means; The image processing apparatus according to claim 9, further comprising: An image processing system having a first image processing device and a second image processing device,
The first image processing apparatus includes:
First processing means for performing first image processing on first input image data which is image data input to the first image processing apparatus;
First communication means for performing communication with the second image processing apparatus;
First combining means for performing a first combining process for combining the result of the first image processing by the first processing means and the result of communication by the first communication means;
Have
The second image processing apparatus includes second processing means for performing second image processing different from the first image processing;
The first communication unit causes the second processing unit to perform the second image processing on the first input image data by performing communication with the second image processing apparatus. A result obtained by performing the second image processing on the first input image data is obtained from the second image processing apparatus;
The first synthesizing process is a process of synthesizing the result of the first image processing on the first input image data and the result of the second image processing acquired by the first communication unit. An image processing system characterized by being. The second image processing apparatus includes:
Second communication means for performing communication with the first image processing apparatus;
Second combining means for performing second combining processing for combining the result of the second image processing by the second processing means and the result of communication by the second communication means;
Further comprising
The second processing means performs the second image processing on second input image data that is image data input to the second image processing device,
The second communication unit causes the first processing unit to perform the first image processing on the second input image data by performing communication with the first image processing apparatus, A result obtained by performing the first image processing on the second input image data is obtained from the first image processing apparatus;
The second combining process is a process of combining the result of the second image processing on the second input image data and the result of the first image processing acquired by the second communication unit. The image processing system according to claim 11, wherein there is an image processing system. The first processing unit and the second processing unit output an output value corresponding to the input value when an input value corresponding to a pixel value is input,
An input value corresponding to a pixel value of the first input image data is input to the first processing means,
The first communication means includes
By outputting the input value corresponding to the pixel value of the first input image data to the second image processing device, an output value corresponding to the input value is output to the second processing unit,
Obtaining the output value output from the second processing means from the second image processing apparatus;
The first combining process includes an output value of the first processing unit corresponding to the input value corresponding to the pixel value of the first input image data, and a second process corresponding to the input value. The image processing system according to claim 11, wherein the image processing system is a process of combining the output value of the means. The second image processing apparatus includes:
Second communication means for performing communication with the first image processing apparatus;
Second combining means for performing second combining processing for combining the result of the second image processing by the second processing means and the result of communication by the second communication means;
Further comprising
An input value corresponding to a pixel value of second input image data that is image data input to the second image processing apparatus is input to the second processing means,
The second communication means includes
By outputting the input value corresponding to the pixel value of the second input image data to the first image processing device, the output value corresponding to the input value is output to the first processing unit,
Obtaining the output value output from the first processing means from the first image processing apparatus;
The second synthesis process includes an output value of the second processing unit corresponding to the input value corresponding to the pixel value of the second input image data, and the first process corresponding to the input value. The image processing system according to claim 13, wherein the image processing system is a process of combining the output value of the means. The first processing means outputs a first partial value that is one of a plurality of partial values constituting a pixel value obtained by performing predetermined image processing on the pixel value;
14. The second processing means outputs a second partial value which is one of the plurality of partial values and which is a partial value different from the first partial value. Or the image processing system of 14. The first image processing apparatus further includes first detection means for detecting a first area and a second area from an area of the image represented by the first input image data,
The first image processing is image processing corresponding to one of the first region and the second region;
The second image processing is image processing corresponding to the other region of the first region and the second region;
The first communication means includes
Based on the detection result of the first detection means, the first output image data that is the first input image data in the region including at least the other region is output to the second image processing device. , Causing the second processing means to perform the second image processing on the first output image data,
A result obtained by performing the second image processing on the first output image data is obtained from the second image processing apparatus;
The first combining means is an image obtained by performing the first image processing on the one area by the first combining process, and the second image on the other area. The image processing system according to claim 11, wherein first composite image data representing a first composite image which is a processed image is generated. The second image processing apparatus includes:
Second communication means for performing communication with the first image processing apparatus;
Second detection for detecting the first region and the second region from the region of the image represented by the second input image data which is the image data input to the second image processing device Means,
Second combining means for performing second combining processing for combining the result of the second image processing by the second processing means and the result of communication by the second communication means;
Further comprising
The second processing means performs the second image processing on the second input image data,
The first image processing is image processing corresponding to one of the first region and the second region;
The second image processing is image processing corresponding to the other region of the first region and the second region;
The second communication means includes
By outputting second output image data, which is the second input image data in a region including at least the one region, to the first image processing device based on the detection result of the second detection unit. , Causing the first processing means to perform the first image processing on the second output image data,
A result obtained by performing the first image processing on the second output image data is obtained from the first image processing apparatus;
The second combining means is an image obtained by performing the first image processing on the one area by the second combining process, and the second image on the other area. The image processing system according to claim 16, wherein second composite image data representing a second composite image that is an image that has been processed is generated. A first setting unit configured to set, as the first image processing, image processing corresponding to the first region or image processing corresponding to the second region; In addition,
A second setting unit configured to set, as the second image processing, image processing corresponding to the first region or image processing corresponding to the second region; In addition,
When the first area detected by the first detection means is larger than the first area detected by the second detection means,
The first setting means sets image processing corresponding to the first region as the first image processing,
The second setting means sets the image processing corresponding to the second region as the second image processing;
When the first area detected by the first detection means is smaller than the first area detected by the second detection means,
The first setting means sets image processing corresponding to the second region as the first image processing,
The image processing system according to claim 17, wherein the second setting unit sets image processing corresponding to the first region as the second image processing. The image processing system according to claim 16, wherein the first output image data is the first input image data in the other region. The first output image data is the first input image data in the other region,
The image processing system according to claim 17 or 18, wherein the second output image data is the second input image data in the one region. 21. The image processing system according to claim 16, wherein the first area is an area corresponding to a display having a wider dynamic range than a display corresponding to the second area. . A control method for an image processing apparatus having first processing means for performing first image processing,
A processing step for causing the first processing means to perform the first image processing on input image data which is image data input to the image processing apparatus;
The second image processing on the input image data is performed by performing communication with another image processing apparatus having a second processing unit that performs second image processing different from the first image processing. A communication step of causing the second processing means to perform the second image processing on the input image data, and acquiring the result from the other image processing device;
A synthesis step for performing a synthesis process for synthesizing the result of the first image processing acquired in the processing step and the result of the second image processing acquired in the communication step;
A control method for an image processing apparatus, comprising: A program for causing a computer to execute a control method of an image processing apparatus having first processing means for performing first image processing,
The control method is:
A processing step for causing the first processing means to perform the first image processing on input image data which is image data input to the image processing apparatus;
The second image processing on the input image data is performed by performing communication with another image processing apparatus having a second processing unit that performs second image processing different from the first image processing. A communication step of causing the second processing means to perform the second image processing on the input image data, and acquiring the result from the other image processing device;
A synthesis step for performing a synthesis process for synthesizing the result of the first image processing acquired in the processing step and the result of the second image processing acquired in the communication step;
The program characterized by having.

Images