JP2013047935A - Video processing device, video display device, and video processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve video quality more suitably.SOLUTION: In a video processing system, a video processing device, or a video display device, there are provided a video input unit which receives an input video signal, and a video processing unit which calculates a contrast correction amount for making the luminance value of input video signal correspond to a visual property curve and performs brightness emphasis processing of the input video signal on the basis of the calculated contrast correction amount and the luminance value of input video signal.

Description

  The technical field relates to video signal processing.

  Patent Document 1 discloses a technique for determining, on the basis of an L * a * b * signal, a region where the chromaticity change is small and a region where the L * signal changes suddenly within a certain area as a region that is exposed to light. It is disclosed.

JP 2002-51226 A

  Since the texture of the object displayed in the video varies depending on the brightness around the object and the like, it is necessary to perform texture enhancement processing considering the periphery of the target object. In addition, when there is a portion where only one pixel is shining as an image, it is difficult to express gloss even if only one pixel is enhanced. In the technique disclosed in Patent Document 1, there is a problem that gloss according to the brightness around the pixel to be processed cannot be detected. Further, this is a method for expressing gloss only for the extracted glossy region, and the concept of further expanding the extracted glossy region and enhancing the gloss is not disclosed.

  The present invention has been made in view of the above problems, and an object of the present invention is to more suitably improve the quality of an image.

In order to solve the above-described problem, for example, a video input unit to which an input video signal is input, a contrast correction amount for making the luminance value of the input video signal correspond to a visual characteristic curve, and the calculated contrast correction are calculated. A video processing unit that performs gloss enhancement processing on the input video signal based on the amount and the luminance value of the input video signal may be provided.

  According to the present invention, it is possible to improve the texture of the video more preferably.

It is a block diagram which shows an example of the video processing system which concerns on one Example of this invention. It is a block diagram which shows an example of the contrast correction amount calculation part which concerns on one Example of this invention. It is a block diagram which shows an example of the glossiness enhancement amount calculation part which concerns on one Example of this invention. It is a flowchart which shows an example of operation | movement of the video processing system which concerns on one Example of this invention. It is a figure which shows an example of the gloss enhancement in the video processing system which concerns on one Example of this invention. It is a figure which shows an example of the filter used by the glossiness emphasis part which concerns on one Example of this invention. It is the figure which showed the state of the glossiness of the periphery of the reflected light of the object after the image processing which concerns on one Example of this invention. It is a block diagram which shows an example of the video processing system which concerns on one Example of this invention. It is a block diagram which shows an example of the video processing system which concerns on one Example of this invention. It is a block diagram which shows an example of the video display apparatus which concerns on one Example of this invention. It is an image figure of an example of the glossy part of the video of L and R of 3D video. It is an example of the display image at the time of making the video processing system which concerns on one Example of this invention cooperate with a vehicle-mounted camera. It is a block diagram which shows an example of the glossiness enhancement amount calculation part which concerns on one Example of this invention. It is a flowchart which shows an example of operation | movement of the video processing system which concerns on one Example of this invention. It is an example of the image | video which the video processing system which concerns on one Example of this invention outputs. It is an example of the application system of the video processing system which concerns on one Example of this invention. It is an example of the application system of the video processing system which concerns on one Example of this invention.

  Examples will be described below.

  FIG. 1 is a block diagram illustrating an example of a video processing system according to the first embodiment of the present invention.

An input video signal 101 is input to the video processing system 100. In this example, it is assumed that the input is in the YCbCr format, and the luminance signal calculation unit 102 calculates the magnitude of the luminance (Y) signal. The contrast correction amount calculation unit 103 calculates a correction amount of contrast that is easy for a person to visually recognize for each region using the Retinex theory. For example, in the case of an image in which an object is placed in a dark area (low brightness), the placed object may not be visible to humans because the placed object has a lower brightness than the surrounding brightness. is there. However, since the eyes adjust according to the brightness of the surrounding space in the actual space, not the image, the object in the dark can be recognized. Using the Retinex theory that utilizes this human visual characteristic, it is separated into light (reflected light) that is reflected when light hits an object from the image and brightness (ambient light) such as ambient lighting, From the visual characteristic curve, it is calculated how much the reflected light is enhanced to make it easier to see, and the contrast correction amount is calculated. As a result, an effect of increasing the apparent dynamic range can be obtained even for an image having a low dynamic range. Retinex theory is described in Reference 1.
[Reference 1] EDWIN H. LAND and JOHN J. McCANN Lightness and Retinex Theory JOSA, Vol. 61, Issue 1, pp. 1-11 (1971)
The contrast correction amount calculation unit 103 transmits the input luminance value from the luminance signal calculation unit 102 and the contrast correction amount based on the Retinex theory to the gloss enhancement amount calculation unit 104. The gloss enhancement amount calculation unit 104 calculates the gloss enhancement amount based on the input luminance value calculated from the luminance signal calculation unit 102 and the contrast correction amount calculated from the contrast correction amount 103. For example, when the brightness value of which the input brightness is corrected for contrast is greater than a certain threshold value, the human visual characteristic is that light reflected from the object takes into account the surrounding brightness. Therefore, the region is extracted as a glossy portion (glossy region). Further, the amount of how bright the glossy portion is visually shining (gloss amount) is represented by the contrast correction amount calculated by the contrast correction amount calculation unit 103 and the contrast correction represented by the input luminance + contrast correction amount. It calculates using the luminance value after. A gloss enhancement amount for enhancing gloss is calculated according to the gloss amount based on the visual characteristics. Depending on the gloss enhancement amount, a spatial filter is applied to express spatial gloss enhancement. The composition process 105 composes the contrast correction amount and the gloss enhancement amount with the input image to create the output image 106. In the configuration of FIG. 1, the composition processing 105 has been described as performing the image processing on the input image at once using the calculated contrast correction amount and the calculated gloss enhancement amount, but the contrast correction amount calculation unit 103 is described. After that, it is also possible to perform a contrast correction process on the input video and then further perform a gloss enhancement process. Although not shown in FIG. 1, a configuration including a display unit that displays the output video 106 may be employed. In this case, the video processing system 100 constitutes a video display device.

  In the present embodiment, processing for a YCbCr signal will be described as an example, and processing for contrast enhancement and gloss enhancement will be described. If the incoming signal is an RGB signal, the same processing can be performed for each of R, G, and B. In addition, with various video signals such as HSV, it is possible to perform glossy extraction and gloss enhancement by similar processing.

  FIG. 2 is a block diagram illustrating an example of a contrast correction amount calculation unit according to the present embodiment.

  The input luminance value 200 from the luminance signal calculation unit 102 is input, and one of the input luminance values is output through. One is separated into a reflected light component 201 and an ambient light component 202 in the Retinex processing unit. According to Retinex theory, if the captured image is I (x, y), the reflected light component is R (x, y), and the ambient light component is L (x, y), I (x, y) = Expressed as R (x, y) × L (x, y). R (x, y) = I (x, y) / L (x, y) where the average value around the pixel to be processed (for example, an area in a predetermined range) is the ambient light component L (x, y) The reflected light component R (x, y) can be obtained by the following equation. The reflected light component 201 and the ambient light component 202 are matched with a human visual characteristic curve 203. The visual characteristic curve 203 is a curve showing the relationship between the surrounding luminance and the luminance that the processing target pixel is easy to see, and when the surrounding is in a low luminance state, For example, when it is desired to emphasize a pixel to be processed in a case where is a medium luminance or a high luminance, a curve suitable for the purpose is used. In contrast correction amount 204, a correction amount for adjusting the luminance value of the processing target pixel to the visual characteristic curve 203 is calculated, and a contrast correction amount 205 is output. Accordingly, the correction amount of the luminance value of the object in the video that matches the visual characteristics is calculated for each video region.

  FIG. 3 is a block diagram illustrating an example of the gloss enhancement amount calculation unit according to the present embodiment.

  The gloss enhancement amount calculation unit 104 calculates the gloss enhancement amount by using the input luminance value 200 and the contrast correction amount 205 output from the contrast correction amount calculation unit 103 as inputs. The gloss amount calculation unit 300 calculates the gloss amount from the input luminance value 200 and the contrast correction amount 205. For example, when the brightness value after contrast correction represented by input brightness + contrast correction amount is larger than a certain threshold value, the area is determined to be a glossy part. This is because it is an easy-to-view part that takes into account the ambient brightness according to the Retinex theory, and the high-brightness part that is expressed as an image is a part that should be exposed to light to express gloss. How much the portion is emphasized is calculated from the contrast correction amount 205. The larger the contrast correction amount 205 for the extracted glossy portion, the easier it is to see the surroundings. Therefore, it can be estimated that the gloss amount can be expressed more strongly, so the gloss enhancement amount is increased. To calculate. In accordance with the gloss amount calculated here, the filter processing unit 301 performs filter processing to spatially emphasize the gloss. The filter performs a Gaussian filter, a tent filter, and the like, and outputs a correction amount of luminance spread by the filter as a gloss enhancement amount 302. Depending on the gloss expression method, the filter may be dynamically switched according to the gloss amount. Accordingly, the gloss amount corresponding to the visual characteristic is calculated, and it is possible to enhance the gloss spatially according to the gloss amount. As a result, in the video expression, it is possible to realize an inevitable glossy expression using human visual characteristics and physical phenomena around the light of the projected object.

  In addition to the method of evaluating the input luminance + contrast correction amount, the glossy portion is determined, for example, when the input luminance is equal to or higher than a predetermined first threshold and the contrast correction amount is equal to or higher than a predetermined second threshold. In some cases, the glossy portion may be determined. The same applies to any of the following embodiments.

  FIG. 4 is a flowchart showing an example of the operation in this embodiment.

  Image reading is performed in S401, and input luminance is calculated in S402. In S403, the contrast correction amount calculation unit 103 calculates the contrast correction amount Δy (205) of the processing target pixel from the luminance y (200) of the processing target pixel and the surrounding luminance using the Retinex theory. In S <b> 404, the gloss enhancement amount calculation unit 104 extracts a gloss portion based on the brightness threshold value, and calculates the gloss enhancement amount ΔY (302) from the input brightness y and the contrast correction amount Δy. In S405, the output luminance Y (Y = y + Δy + ΔY) is calculated. In S406, color components such as CbCy are combined to output an output video (106).

  FIG. 5 is a diagram illustrating an example of gloss enhancement in the present embodiment.

  FIG. 5A is an 8-bit grayscale image obtained by extracting a part (5 × 5) of the input image, where black represents luminance 0 and white represents luminance 255. The example of FIG. 5A is an example in which there is a pixel of luminance 85 in the center of a background of luminance 80. The center pixel is not clearly visible. FIG. 5B is an image diagram in which the central luminance is increased by 55 and the luminance is 140 by correcting the contrast of FIG. 5A using the Retinex theory so that the visual correction is easy. If it is determined that a luminance of 120 or more is a glossy portion, this pixel emphasizes the glossiness, and the enhancement amount is calculated according to the contrast increase amount 55, and the glossiness is enhanced on the surface by the filter processing. If only the brightness of one point is increased, it may look like noise rather than video. Therefore, the gloss enhancement amount calculation unit 104 of the image processing system according to the present embodiment further expresses gloss in consideration of not only human visual characteristics but also the physical phenomenon of light hitting an object as shown in FIG. The gloss enhancement amount is calculated as follows.

  FIG. 6 is a diagram illustrating an example of a filter used in the gloss enhancement processing according to the present embodiment.

The figure is an image diagram showing a Gaussian filter. The glossy portion extracted by the gloss amount calculation unit 300 in FIG. 3 is multiplied by a filter according to the gloss amount calculated by the filter processing unit 301 in FIG. Accordingly, it is possible to express the image so that the light spreads.

It is desirable to select the size and type of filter suitable for the video by extracting video features such as whether the image is a natural image with a small change in luminance or chromaticity or a smooth image such as an animation. For natural images with small changes in brightness and chromaticity, use a filter with a small size, use a large filter for images with low resolution, and select no filter for animation.

FIG. 7 is a diagram illustrating a glossy state around the reflected light of an object after image processing according to the present embodiment.
The figure shows a situation where light hits a sphere from a light source in real space. A portion where the sphere receives light and reflects light by specular reflection is A, and a portion where light diffuses around A and is shining is B. Depending on the material of the object, the areas of A and B differ, but there are not only a few parts that dimly shine around the mirror-reflected surface. The portion B is the gloss enhancement amount calculation unit 10 already described.
This can be expressed by a spatial gloss enhancement process using the filter No. 4.

If you can determine the material of the object in the video using image recognition or metadata,
A database storing the ratio of specular reflection and diffuse reflection for each material may be held in a memory (not shown), and the size and characteristics of the filter may be changed for each material of the determination result. Thereby, there is an advantage that a glossy feeling with higher texture can be expressed.

FIG. 8 is a block diagram showing another configuration example of the video processing system in the present embodiment, and is a configuration in which color determination processing is added to the video processing system of FIG.

  When the input signal is YCbCr, the correction of only the luminance signal may cause the color to become white due to the increase in luminance due to the above-described gloss improvement processing. In order to solve this problem, a method of weakening the contrast correction amount and the gloss enhancement amount may be used for a portion colored from the value of CbCr. In the gloss enhancement amount calculation unit 104, a color close to white is not erased for a location determined as a glossy portion, and thus the luminance is increased to enhance the gloss. On the other hand, in order to prevent the color from becoming white in the colored portion, the gloss enhancement expression is not performed, or the contrast correction amount and the gloss enhancement amount are weakened. Thereby, a more natural gloss expression can be achieved. Although not shown in FIG. 8, a configuration including a display unit that displays the output video 106 may be employed. In this case, the video processing system 100 constitutes a video display device.

  When expressing gloss enhancement in RGB, apply the above gloss enhancement processing to each of R, G, and B, and take measures to reduce the contrast correction amount and gloss enhancement amount in consideration of the ratio of R, G, and B. To do.

  According to the video processing system or the video processing method of the first embodiment of the present invention described above, the gloss enhancement processing of the video is performed by detecting the glossy portion of the video and correcting the contrast using the Retinex theory. Thereby, the image quality can be improved more suitably.

  FIG. 9 is a block diagram illustrating an example of a video processing system according to the second embodiment of the present invention.

  The video processing system 100 in FIG. 9 is connected to a display (video display device) 902. The video processing system 100 in FIG. 9 has a configuration in which a light sensor 900 and a brightness adjustment unit 901 for the display 902 are added to the video processing system in FIG. Since other configurations are the same as those of the first embodiment, the description thereof is omitted.

  The optical sensor 900 can be installed on a display that displays video to check the viewing environment of the user. In this embodiment, a brightness adjustment unit 901 that adjusts the brightness of the display 902 by adjusting the output of a backlight or a self-light emitting element according to the output of the optical sensor 900 is provided. At this time, by performing contrast correction using the Retinex theory according to the adjustment amount of the brightness adjustment by the brightness adjustment unit 901, even if the backlight output is lowered and the dynamic range is lowered, The appearance is easy to see and the dynamic range of appearance can be increased. For example, if the optical sensor 900 is in a situation where an image is being viewed on a liquid crystal display in a dark room, it will look beautiful even if the backlight output is lowered by Δ. In the contrast correction amount calculation unit 103, the input signal is compressed within the dynamic range corresponding to Δ where the output is reduced, and the corrected input signal is subjected to contrast correction using the Retinex theory. By applying, it becomes easy to see the appearance of each area, and the power consumption of the backlight is reduced. By adjusting the brightness and performing gloss enhancement in addition to contrast correction, even if the backlight output is low, it is possible to express an image in which the glossy portion in the image is emphasized. In addition, by reducing the luminance threshold (input luminance + contrast correction amount) for detecting the glossy area in accordance with the magnitude of Δ that reduced the backlight output, it is possible to extract the glossy part even in a low dynamic range image. To do. For example, even when the video processing system is applied to a mobile device, there is an advantage that an apparent dynamic range can be improved regardless of outdoors or indoors.

  Note that the video processing system 100 and the display 902 of FIG. 9 can be integrated to form a display device.

  According to the video processing system or video processing method of the second embodiment of the present invention described above, it is possible to perform a suitable gloss enhancement process according to the viewing environment.

  FIG. 10 is a block diagram of a display device according to Embodiment 3 of the present invention. FIG. 10 illustrates an example in which the video processing system 100 according to the first embodiment or the second embodiment is incorporated as a video processing unit 100 in a display device.

  In FIG. 10, when a 2D or 3D input video 1000 is input via the user I / F 1001, the user performs setting for selecting whether to display in 2D or 3D. As the setting, it may be configured to manually select when inputting video, or may be selected and set in advance to display in 2D or 3D, and configured to be automatically selected when inputting video. May be. The 2D / 3D display switching unit 1002 stores the display switching information in the video display state management unit 1003 according to the selection setting from the user I / F 1001, and performs 2D video processing or 3D display processing. Switch what to do. The display device unit 1008 can perform 2D display and 3D display, and the display device unit 1008 switches between 2D display and 3D display in accordance with information stored in the video display state management unit 1003. The 2D video processing 1006 outputs video by matching the video stream with the display frequency of the display device.

  The 3D video processing 1007 outputs the video by switching the outputs of L and R in conjunction with the display frequency of the display device. Prior to 2D or 3D video processing, the video processing unit 100 described in the first or second embodiment performs contrast correction and gloss enhancement processing. At this time, the video processing unit 100 is configured to include different parameter 1 (1004) and parameter 2 (1005) for 2D and 3D, respectively. Thereby, in the determination of the contrast correction amount of the Retinex processing unit of the video processing unit 100, it is possible to reflect the characteristics of video expression in 2D display and 3D display. For example, in the case of shutter glasses 3D display, glasses are added between the eyes and the display, so that the amount of light from the display is reduced, so that the image appears dark. Further, since only L or R is visible instantaneously for switching the shutter, the amount of video that enters the vision is reduced, so that it looks dark. Therefore, if the same video processing as that in 2D display is performed for 3D display, the effect is reduced. Therefore, when switching to 3D display, the visual characteristic curve of the Retinex processing unit is changed, and parameter settings are used so that the contrast correction amount is larger than that in 2D. Regarding the gloss enhancement amount, the threshold value of the luminance value for judging the glossy portion is lowered to switch to a parameter for increasing the glossy portion, and the filter is also switched to a filter with a high peak in a wide range (may be narrow). .

  In the configuration of this embodiment, the display device and the video processing unit may be separated. For example, Retinex parameters may be switched when switching between 2D and 3D in a video processing system on the side of a signal transmission device such as a recorder connected to a television.

  Note that a video output unit that outputs the 2D video processing 1006 or the 3D video processing 1007 to an external device may be provided instead of or in addition to the display device unit 1008. In this case, even when the 2D video or 3D video is output to an external device, it is possible to improve the glossiness.

  FIG. 11 is a diagram illustrating an example of a shift in the gloss emphasis position of L and R (left and right images) during 3D display.

  For 3D images taken simultaneously with the two left and right cameras, the angle at which the light rays reflected by the object hit the camera are different, so if the L, R left and right images are superimposed, there will be a shift in the position of the glossy part. It may have occurred. At this time, if the gloss emphasis of the present invention is performed, the gloss at different positions on the left and right may appear alternately, and the gloss emphasis portion may appear as crosstalk. A similar phenomenon may occur in the contrast correction unit. Therefore, in the contrast correction processing and gloss enhancement processing using the Retinex processing, the contrast correction amount and the gloss enhancement amount for each pixel are calculated. Here, with respect to a pixel having a greatly different correction amount or enhancement amount at the corresponding pixel position of the L and R images, a process of not performing contrast correction or gloss enhancement or reducing the correction amount and enhancement amount is performed. Thereby, it is possible to prevent the crosstalk from increasing.

  According to the video processing system, display device, or video processing method of the third embodiment of the present invention described above, the gloss enhancement processing can be more suitably applied to 3D display.

  The video processing system according to the fourth embodiment of the present invention displays additional information regarding the glossy part using the video that has been subjected to gloss enhancement using the video processing systems according to the first and second embodiments.

  First, a system when the video processing system is linked with an in-vehicle camera will be described with reference to FIG. FIG. 12 is an example of a display image by the system.

  The camera, the video processing system and the display device according to the present embodiment are mounted on the car, and the Retinex processing according to the first and second embodiments is applied using the video from the camera as the input video. At this time, a frozen part or a puddle on the road made of rain or snow can be recognized as a glossy part by being illuminated by light or a car light. Therefore, at the stage where the glossy part has been extracted, the display device displays “There is a possibility that there is a place where it is easy to slip forward” and visually expresses the glossy part in freezing. It is easy to see where the danger is. As the numerical value representing this possibility, a value proportional to the gloss amount in the gloss enhancement amount calculation unit 104 is displayed. The reason is that it can be estimated that the larger the gloss amount is, the higher the ratio of specular reflection is.

  Further, in order to inspect whether the portion detected as the glossy portion is the glossy portion to be detected, such as a puddle or a frozen portion, the gloss enhancement amount calculation unit 104 determines the gloss amount during normal road travel. May be calculated, and the difference from the normal gloss amount may be calculated. If the difference is large, it may be estimated that the detected glossy portion is a puddle or a frozen portion.

  Further, if the gloss enhancement amount calculation unit 104 calculates the positional relationship of the gloss extraction unit with the road and a glossy portion is detected at a location that does not appear normally, there is a risk of traveling. Therefore, a warning to that effect may be displayed on the display or warned with sound.

  When video from two left and right (LR) cameras is used as the input video, a glossy part is found in the L video, but a glossy part is found in the R video, depending on the camera position and light reflection. In some cases, such as when there is no glossy part, the glossy part is detected only in one image. In this case, the detected glossy part of the L video is trusted and a warning is displayed, and the glossy emphasized part is displayed on the R video based on the L video. May be. At this time, even when the 3D image is displayed, the detected glossy portion may be highlighted. Thereby, the probability of overlooking detection in the way of light reflection is reduced, and an easy-to-understand emphasis expression is possible even when a dangerous part is displayed to the user.

  The above-described camera, video processing system, and display device may be configured as an integrated vehicle-mounted video display system, or may be configured as a vehicle-mounted video display system in which each is connected by wired or wireless communication means.

  Similarly, the video processing system of the present embodiment can be applied to a camera video in an endoscope. As a means to detect the glossy part and detect inflammation or swelling of the affected area by analyzing the reflected light component by the light irradiated to the internal organs, skin, body fluid, etc. Can also be applied. Depending on the part to be photographed, the space is large and the amount of irradiation light applied to the tip of the endoscope cannot be clearly captured, or conversely, the space is too small and the amount of light is too strong, making it difficult to see the affected area. Therefore, it is possible to make the image easy to see by correcting the contrast using the Retinex theory for the low dynamic range image. Furthermore, the fact that the light that is reflected when irradiated with light is strong means that there is a certain amount of unevenness (not fine unevenness), and the affected area and swelling degree are calculated from the amount of gloss calculated from the reflected light component. There is a possibility of estimating the size of. Therefore, the glossy part extraction process by the video processing system of this embodiment is effective. Further, when the video processing system according to the present embodiment is applied to an endoscopic video, a gloss amount is extracted for each of a plurality of colors in the color determination process of FIG. The affected area can be detected for each color, such as whether it is a violet or shiny part. Furthermore, there is a possibility that the affected part can be easily diagnosed by emphasizing the affected area.

  In these embodiments, the same processing is possible even if the input signal is YCbCr, RGB, or HSV. Further, the program that operates in the video processing system may be installed in the video processing system, may be provided by being recorded on a recording medium, or may be provided by being downloaded via a network. Anyway. By using these various distribution forms, provision in various use forms becomes possible, and there is an effect of increasing the number of users.

  In each of the embodiments described above, the high-luminance portion is detected as a glossy region, but conversely, the low-luminance portion can be detected as a dark portion region.

  Using Retinex theory that utilizes human visual characteristics, light is reflected from objects in the image (reflected light) and the brightness of the surrounding lighting (environmental light) is separated into visual By emphasizing the reflected light from the characteristic curve, it is possible to increase the apparent dynamic range even for an image with a low dynamic range. Although the gloss enhancement technique using this effect has been described, it is considered that a glossy video expression can be realized more subjectively by reflecting the glossiness actually felt by a person.

First, there is Reference 2 as an example of confirming the correlation between the glossiness perceived by humans and the numerical value. There is a correlation between the reflectance, the skewness, and the vision of the image sample, and a strong correlation was found between the skewness and the vision. The result is that the higher the distortion, the more glossy it feels. Therefore, in this embodiment, gloss enhancement based on human visual characteristics based on Retinex theory is combined with evaluation judgment processing using numerical values that correlate with improvement in glossiness due to human subjectivity such as the above-mentioned distortion. Thus, a method for realizing a more suitable video expression will be described.
[Reference 2] Motoyoshi et.al., Nature, vol47, pp206-209 (2007)
In the video processing apparatus according to the fifth embodiment of the present invention, the configuration of the gloss enhancement amount calculation unit 104 in the configuration illustrated in FIG. 1 is changed to the configuration illustrated in FIG. 13 instead of FIG. Other configurations are the same as those already described, and a description thereof will be omitted.

  The gloss enhancement amount calculation unit of the fifth embodiment illustrated in FIG. 13 includes a gloss evaluation / determination unit 1300 in addition to the gloss enhancement amount calculation unit of FIG. Reflecting the brightness value of the pixel to be processed and the gloss enhancement amount calculated from the contrast correction amount using Retinex theory, the value of the result of gloss enhancement on the filter surface is input, and the subjectivity such as glossiness Compared with the mathematical expression result that can be correlated, the comparison result is fed back to the gloss enhancement processing, thereby realizing a video effect that improves the glossiness more subjectively.

  FIG. 14 shows a processing sequence. First, in S1401, the numerical value (S0) of glossiness around the pixel to be processed is calculated using the result of Reference 2 or the like. As an example, the equation of skewness shown in Reference Document 2 is shown in Equation 1.

I(x,y)：Luminance of each pixel
ｍ ： average luminance
N ： number of pixels
s.d.： standard deviation

次に、図３で示した方法により、入力輝度２００、コントラスト補正量２０５を入力として、光沢量算出３００を行い、その光沢量値に応じたフィルタ処理を３０１にて行い、光沢強調量を算出する（Ｓ１４０２）。次に、光沢感評価・判定部１３００が、光沢が強調されたＳ１４０２の結果に対して、対象画素周囲の光沢感の数値（Ｓ１）を数１等を利用して算出する（Ｓ１４０３）。Ｓ１４０４にて、Ｓ０とＳ１を比較する。もし、処理結果前のＳ０よりも、Ｓ１が高く、図３での光沢強調処理により光沢感が向上している場合には、Ｓ１４０２での算出結果を利用した光沢強調を行う（Ｓ１４０５）。Ｓ１４０４にて、Ｓ１４０２の結果、光沢感の数値が下がる場合には、Ｓ１４０６にて、Ｓ１４０２の結果を利用しない処理（例えば、光沢感の強調を行わない）を行う。

I (x, y): Luminance of each pixel
m: average luminance
N: number of pixels
sd: standard deviation

Next, by the method shown in FIG. 3, the glossiness calculation 300 is performed with the input luminance 200 and the contrast correction amount 205 as inputs, and the filter processing corresponding to the glossiness value is performed at 301 to calculate the gloss enhancement amount. (S1402). Next, the glossiness evaluation / determination unit 1300 calculates a numerical value (S1) of glossiness around the target pixel using the number 1 or the like with respect to the result of S1402 in which glossiness is enhanced (S1403). In S1404, S0 and S1 are compared. If S1 is higher than S0 before the processing result and the glossiness is improved by the gloss enhancement processing in FIG. 3, gloss enhancement using the calculation result in S1402 is performed (S1405). In S1404, if the numerical value of glossiness decreases as a result of S1402, in S1406, processing that does not use the result of S1402 (for example, glossiness is not emphasized) is performed.

  As a result, out of the portions that are subjected to contrast correction in the Retinex theory, the portions that are determined to be able to be expected to improve glossiness by performing evaluation and determination using numerical values such as the skewness calculated in Reference Document 2 are selectively used. Further, the gloss enhancement processing by the contrast correction can be reflected in the output video.

  In S1404 of FIG. 14, it may be added that the input luminance value (y) is a predetermined value or more as a condition for proceeding to S1405.

  In S1405 of FIG. 14, the calculation result in S1402 may be used as it is, or adjustment for changing the degree of emphasis may be performed by reflecting S1.

  As described above, for example, evaluation judgment using numerical values that correlate with improvement in glossiness due to human subjectivity such as skewness, etc., for gloss enhancement processing suitable for human visual characteristics such as contrast correction using Retinex theory By performing feedback by processing, it is possible to select the part that can be expected to improve the subjective glossiness of human beings from the part that is enhanced by the human visual characteristics and perform the gloss enhancement processing of the video It is.

  In the above-described configuration or processing method, skewness is used as a criterion for evaluation and determination of glossiness that matches human visual characteristics. Any numerical value may be used as long as it is calculated by the above. It is considered that the subjective evaluation of appearance can be improved by using such numerical values.

  It should be noted that the configuration or processing method of the fifth embodiment of the present invention described above can be applied to the configuration and processing of the gloss enhancement calculator of other embodiments. In this case, the system, the display device, or the video processing method in another embodiment to which the configuration or the processing method of the fifth embodiment is applied can obtain the effects of the fifth embodiment.

  In the fifth embodiment, the method using the skewness as an evaluation criterion for enhancing glossiness has been described. In contrast, the sixth embodiment of the present invention provides gloss enhancement processing that can reflect the subjectivity of the user of the video processing apparatus and system.

  The sixth embodiment of the present invention is configured, for example, by mounting the gloss enhancement amount calculation unit of FIG. 13 on the video processing device of FIG. The output video 106 in FIG. 1 may be output to an external device, or may be displayed on a display provided in the video processing apparatus. When the video processing apparatus includes a display, the video display apparatus is configured.

  The video processing apparatus according to the sixth embodiment of the present invention differs from the fifth embodiment in the processing of the gloss enhancement amount calculation unit in FIG. First, the gloss enhancement amount calculation unit according to the sixth embodiment creates an image in which the degree of gloss enhancement is changed to a plurality of levels (for example, 3 levels). For example, as shown in FIG. 15, images with different glossiness enhancement levels are output to an external device or a display. At this time, a display that prompts the user to select the one of the plurality of stages of video (video 1, video 2, video 3 in the example of FIG. 15) and select which correction feels the most glossy is also included. Output.

  The video processing apparatus according to the sixth embodiment of the present invention further includes an instruction input unit (not shown) for inputting an instruction from a user, and one video among a plurality of displayed videos is displayed on the instruction input unit. When video selection information indicating selection is input, the degree of gloss enhancement applied to the selected video is set as the degree of gloss enhancement suitable for the user's subjectivity. It is configured to perform an emphasis process.

  Here, the degree of gloss enhancement may be a coefficient to be multiplied by the gloss amount calculated by the gloss amount calculation unit 300 in FIG. 3, or may be the skewness of the gloss enhancement amount after the filter 301 described in the fifth embodiment.

  When the coefficient multiplied by the gloss amount is set to the degree of gloss enhancement, a plurality of images are created by setting the coefficient in a plurality of stages and presented to the user in the display shown in FIG. The coefficient used to create the video selected by the user may be set as a coefficient to be multiplied by the gloss amount calculated by the gloss amount calculation unit 300. In the subsequent gloss enhancement processing of the video, the coefficient may be multiplied by the gloss amount calculated by the gloss amount calculation unit 300.

  Further, when the degree of distortion is set to the degree of gloss enhancement, a plurality of videos having different degrees of gloss enhancement of the gloss enhancement amount in a predetermined area of the image after the filter 301 are created and presented to the user with the display shown in FIG. The skewness of the video selected by the user is set as the target skewness of the gloss enhancement amount calculation unit in FIG. Specifically, for example, the gloss enhancement amount is changed, and a three-stage video image having Sa, Sb, Sc (Sa <Sb <Sc) in a predetermined area is created and presented to the user as the video image of FIG. . The skewness of the video selected by the user via the instruction input unit is set as the target skewness St. In the subsequent gloss enhancement processing of the video, when the glossiness evaluation / determination unit 1300 determines that the skewness of the gloss enhancement amount after the filter 301 is within a predetermined range from the target skewness St, Gloss emphasis is performed using the gloss emphasis amount after 301 in the filter process. When the glossiness evaluation / determination unit 1300 determines that the skewness of the gloss enhancement amount after the filter 301 is not within a predetermined range (± δ) from the target skewness St, the filter processing is performed after 301. The gloss enhancement amount is not used (for example, glossiness is not enhanced in the corresponding area). An example of the flowchart of the process in this case can be realized, for example, by changing the determination condition of S1404 to | S1-St | ≦ δ in the process of the flowchart of FIG. In this case, as in the fifth embodiment, in S1404 of FIG. 14, as a condition for proceeding to S1405, it may be added that the input luminance value (y) is a predetermined value or more. In S1405 of FIG. 14, the calculation result in S1402 may be used as it is, or adjustment for changing the degree of enhancement may be performed by reflecting S1.

  It should be noted that the “Glossy Setting” item is provided in the “Image Quality Setting Menu” of the video processing apparatus and video processing system that can be operated via the instruction input unit, and the above-described multiple video output of FIG. What is necessary is just to comprise so that selection and the degree of gloss emphasis can be set.

  15 may be created in real time from the input video, but a plurality of sample videos with different gloss enhancements in multiple stages are prepared in advance and stored in a memory or the like. It may be presented to the user.

  Note that according to the configuration or the processing method according to the sixth embodiment described above, it is possible to realize the gloss enhancement process having a correlation with the glossiness based on the subjectivity of the user of the video processing apparatus and system.

  It should be noted that the configuration or processing method of the sixth embodiment of the present invention described above can be applied to the configuration and processing of the gloss enhancement calculation unit of other embodiments. In this case, the system, the display device, or the video processing method in another embodiment to which the configuration or the processing method of the sixth embodiment is applied can obtain the effects of the sixth embodiment.

  A seventh embodiment of the present invention will be described with reference to FIG.

  FIG. 16 shows an example in which the video processing apparatus of the present invention is used in an inspection system. For example, the object 1601 flowing on the conveyor is photographed by the camera 1600, and the image from the camera is used as the input image 101, and the image processing apparatus 100 (according to the first, second, third, fifth, sixth embodiments described above). In this case, the gloss degree obtained from the contrast correction amount (Δy) and the gloss enhancement amount (ΔY) may be digitized and displayed on the display 1602 together with the video. With this system, it is possible to inspect the object with numerical values indicating the image and glossiness. For example, it can be used for inspecting the brightness of pearls and the finished condition of painted objects.

  In FIG. 16, glossiness information 1603 output from the video processing unit 100 includes a contrast correction amount (Δy), a gloss enhancement amount (ΔY), or a value calculated based on this value (glossiness). (Degree).

  In FIG. 16, as the output video 1604 output from the video processing unit 100, the output luminance (y + Δy + ΔY) with gloss enhancement may be used, or the output luminance (gloss enhancement) ( y + Δy) may be used. Moreover, you may comprise so that the video processing part 100 can switch and output these according to the instruction | indication from a user.

  According to the inspection system of the seventh embodiment of the present invention described above, the inspection target is photographed by the camera, information indicating the glossiness of the inspection target is calculated from the input video, and the gloss of the inspection target is set to a numerical value or the same. In addition, the inspection can be performed in consideration of the video.

  An eighth embodiment of the present invention will be described with reference to FIG. FIG. 17 is an example of use in an Internet search service or a system for searching for a glossy object from video in a local database. In FIG. 17, a server 1702 for holding video / images is connected to a network 1701 such as the Internet, and the video processing apparatus 100 (according to the above-described embodiments 1, 2, 3, 5, 6, etc.) An image is acquired from the server 1702 or the like via 1701, and an output image 1604 and glossiness information 1063 similar to those of the seventh embodiment are output to the display 1602 and displayed. Further, the video source may be acquired from the storage 1703 locally connected to the video processing apparatus 100 in addition to being acquired via the network 1701. The storage 1703 and the video processing apparatus 100 may constitute an integrated information processing apparatus.

  A plurality of videos are obtained from these video sources, and the videos and glossiness information are stored in a database in the storage 1703, for example, so that a video / image having a predetermined glossiness as shown in FIG. Can be displayed as a list.

  Such a system may be used for a service such as displaying a list of the freshness of fish by numerically calculating the degree of glossiness of an image, for example, in online shopping. In this case, there is a merit that the user can estimate the quality of the material, the degree of completion of the finish, the freshness, etc. from the situation where there are only videos or images.

  The following modifications can be applied to any of the embodiments described above.

In any of the embodiments described above, a method of separating specular reflection components using a dichroic reflection model as shown in Reference 3 for the processing in the reflected light component detection result unit 201 in the Retinex processing unit in FIG. Can be used.
[Reference 3] GJ Klinker, SA Shafer, and T. Kanade: The measurement of highlight in color images, International Journal of Computer Vision (IJCV), Vol.2, No.1, pp.7-32, 1988. Carnegie Mellon University)
The separation of reflected light components in the dichroic reflection model uses a color space, so the light source needs to be white, and can only be applied when the surface color of the object and the hue of diffuse reflection are the same. There are constraints. The method of extracting the glossy part based on the intensity of the reflected light using Retinex described in the above-mentioned embodiment and taking into account the situation of light components and unevenness around the pixel of interest is more human. There is a feature that can be extracted according to visual characteristics. Since both have different characteristics, the Retinex method or the dichroic reflection model method may be selected for the reflected light component detection processing according to the system to be applied. Moreover, both may be switched according to an input image | video and an environment, and it is good also as a system which uses both together suitably. Thus, it is considered that gloss extraction combining the physical aspect of the object and the visual characteristics can be realized.

DESCRIPTION OF SYMBOLS 100 Video processing part, 101 Input video, 102 Luminance signal calculation part, 103 Contrast correction amount calculation part, 104 Gloss emphasis amount calculation part, 105 Composition processing part, 106 Output image, 201 Reflected light component detection result part, 202 Ambient light detection Result section, 203 visual characteristic curve processing section, 204 contrast correction amount calculation section, 205 contrast correction amount, 300 gloss amount calculation section, 301 filter processing section, 302 gloss enhancement amount, 800 color determination process, 900 optical sensor, 901 brightness Adjustment unit, 902 display, 1000 input video, 1001 user I / F, 1002 2D / 3D switching processing unit, 1003 video display state management unit, 1004 parameter 1 (for 2D), 1005 parameter 2 (for 3D), 1006 2D video Processing unit, 1007 3D video processing unit, 1008 display device unit,
1300 Glossiness Evaluation / Determination Unit, 1600 Camera, 1601 Object Group, 1602 Display Device (Display), 1701 Network, 1702 Server, 1703 Storage

Claims (18)

A video input unit to which an input video signal is input;
A video for calculating a contrast correction amount for making the luminance value of the input video signal correspond to the visual characteristic curve, and performing gloss enhancement processing on the input video signal based on the calculated contrast correction amount and the luminance value of the input video signal A video processing apparatus comprising: a processing unit. In the calculation process of the contrast correction amount, the brightness value of the input video signal is separated into a reflected light component and an ambient light component, and contrast correction is performed so as to correspond to the visual characteristic curve based on the separated information. The video processing apparatus according to claim 1, wherein an amount is calculated. In the gloss enhancement process, a gloss region in the video is detected from the luminance value of the input video signal and the calculated contrast correction amount, and the gloss amount corresponding to the contrast correction amount is detected for the detected gloss region. The video processing apparatus according to claim 1, wherein the filter processing is performed to perform spatial gloss enhancement processing. 4. The video processing apparatus according to claim 3, wherein, in the glossy region detection process, a region where the sum of the luminance value of the input video signal and the contrast correction amount is a predetermined threshold value or more is detected as a glossy region. A video display device comprising the video processing device according to claim 1 and a video display unit. 6. The video display device according to claim 5, further comprising: an optical sensor capable of measuring the intensity of ambient light; and a brightness adjustment unit of the video display unit, wherein the brightness adjustment unit includes the optical sensor. The brightness of the video display unit is adjusted based on the output of the video display device. 5. The video processing apparatus according to claim 1, wherein 2D video and 3D video can be input to the video input unit,
And a switching unit for switching between 2D display video processing and 3D display video processing,
The video processing unit switches parameters used for the contrast correction amount calculation processing or the gloss enhancement processing according to switching between 2D display video processing and 3D display video processing by the switching unit. apparatus. The video processing apparatus according to claim 7, wherein the parameter is switched so that the contrast correction amount or the gloss amount is larger in 3D display than in 2D display. 9. A video display device comprising: the video processing device according to claim 7; and a display unit capable of displaying 2D video and 3D video. A video processing device according to claim 3 and a video display unit,
The video display device, wherein the video display unit displays information on the detected glossy area. A video input step in which an input video signal is input;
A video for calculating a contrast correction amount for making the luminance value of the input video signal correspond to the visual characteristic curve, and performing gloss enhancement processing on the input video signal based on the calculated contrast correction amount and the luminance value of the input video signal A video processing method comprising: a processing step. In the calculation process of the contrast correction amount, the brightness value of the input video signal is separated into a reflected light component and an ambient light component, and contrast correction is performed so as to correspond to the visual characteristic curve based on the separated information. The video processing method according to claim 11, wherein the amount is calculated. In the gloss enhancement process, a gloss region in the video is detected from the luminance value of the input video signal and the calculated contrast correction amount, and the gloss amount corresponding to the contrast correction amount is detected for the detected gloss region. The image processing method according to claim 11, wherein the filter processing is performed to perform spatial gloss enhancement processing. 14. The glossy region detection process detects a region where a sum of a luminance value of the input video signal and the contrast correction amount is a predetermined threshold or more as a glossy region.
Video processing method. The gloss enhancement processing includes a processing unit that calculates a numerical value indicating glossiness by subjectivity, and a numerical value indicating the glossiness calculated for the input image and an image after the spatial gloss enhancement processing is performed. Compare the calculated glossy values for
Than the numerical value indicating the glossiness calculated for the input video,
4. The spatial gloss enhancement process is not performed when a numerical value indicating glossiness calculated for an image after the spatial gloss enhancement process is small. Video processing device.   In the gloss enhancement process, a filter process is performed on the detected gloss area with respect to the gloss amount corresponding to the contrast correction amount in a plurality of stages to create a spatial gloss enhancement image and use it. 16. The video processing apparatus according to claim 15, wherein a video image that gives a person a subjective gloss feeling is selected, and spatial gloss enhancement processing is performed according to the selection result.   16. The video processing apparatus according to claim 15, wherein in the gloss enhancement process, the video is classified according to the gloss amount corresponding to the contrast correction amount in a plurality of stages with respect to the detected gloss region. .   In the gloss enhancement process, for the detected gloss region, the gloss value corresponding to the contrast correction amount is compared with a numerical value indicating subjective glossiness, and correlation is obtained to indicate subjective glossiness. The video processing apparatus according to claim 15, wherein a numerical value is evaluated.

Images