GB2509692A

GB2509692A - Method and apparatus for detection of chromatic aberration

Info

Publication number: GB2509692A
Application number: GB1219920.4A
Authority: GB
Inventors: Jonathan Mckinnell; Mark Glanville; Phil Tudor
Original assignee: British Broadcasting Corp
Current assignee: British Broadcasting Corp
Priority date: 2012-11-05
Filing date: 2012-11-05
Publication date: 2014-07-16
Also published as: GB201219920D0

Abstract

Method and apparatus for analysing an image to determine the presence of chromatic aberration comprising: applying an edge mask to the image to produce a colour edge image; producing a histogram of the hues in the colour edge image and a histogram of the hues in the original image; determining a mean ratio of the two histograms in a range of hues; and asserting the presence of chromatic aberration if the mean ratio is above a threshold. The mean ratio may be found by determining a ratio at each hue in the range and then calculating the mean of these ratios. The range of hues may comprise pink hues. Edge detection may be carried out by applying a Sobel filter to the luminance component of a YUV image. The centre of the image may be masked, or a pixels contribution to the histograms may be weighted based on its distance from the centre, to improve the detection of transverse chromatic aberration.

Description

Method and Apparatus for Detection of Chromatic Aberration

BACKGROUND OF THE INVENTION

The present invention relates to systems and methods for monitoring the quality of audio video content from each of multiple sources. In particular, the invention relates to checking for chromatic aberration.

Programme makers are being required to produce greater volumes of high quality content for a larger variety of delivery platforms and audiences with ever decreasing budgets. Alerting production staff of potential problems with audio or video content could help save time and money both on the production and in the editing suite by expediting the process whilst potentially avoiding the failure of a technical review. There are a variety of possible technical errors or potential problems which can occur in a production.

Programmes can fail a technical review due to excessive amounts of chromatic aberration. By detecting this faults in a production, such failures could be avoided whilst also saving time in the post-production process by storing such detections as metadata along with the content. This may be implemented as part of a wider suite of software of automated quality checking tools.

SUMMARY OF THE INVENTION

We have appreciated the need for improved systems and methods for alerting users to the fact that a source of audio video content may not be providing that content at a required quality level.

In broad terms, the invention resides in alerting potential problems with footage to production staff could help save time and money both on the production and in the editing suite by expediting the process whilst potentially avoiding the failure of a technical review. An embodiment of the invention uses an edge detection algorithm in conjunction with detection of hue to determine a measure of chromatic aberration * 2

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described in more detail by way of example with reference to the accompanying drawings, in which: Figure 1: is a block diagram showing the main functional components of a system embodying the invention; Figure 2: is an example image of a scene; Figure 3: is a close up of the image of Figure 2; Figure 4: shows an algorithm enbodying the invention; Figure 5: is a luminance image of the scene of Figure 2; Figure 6: is a hue image of the scene of Figure 2; Figure 7: is a filtered edge image of the luminance image of Figure 5; Figure 8: is a binary thresholded version of the image of Figure 7; Figure 9: shows the original image of Figure 2 filtered usind the binary threshold image of Figure 8; Figure 10: shows the histogram of hues of entire image; and Figure 11: shows the histogram of hues of edge pixels.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION

Automatic shot classification is a general term for the tagging of images with metadata that can improve the overall workflow within a production. This metadata could provide cost or time saving benefits, better archiving retrieval or be useful for other non-production purposes.

An example of time and cost saving would be if images with chromatic aberration could be logged. With the continuing uptake of digital media and digital workflows, traditional methods of checking the technical quality of content are becoming increasingly limited, in part due to the sheer volume of content now being produced. Both academic and broadcasting standardisation work has been carried out into a perception based quality measurement. In many cases it is becoming increasingly impossible for human eyes to view all content for errors even at the delivery stage of the programme making chain. The UK public service broadcasters, through the Digital Production Partnership initiative, have agreed a set of technical standards that programmes must achieve.

The technical quality of a programme is assessed before it is broadcast. Failure to meet the standards required in this technical review can result in significant extra costs. In some cases it may not be possible to correct mistakes, as content cannot always be re-shot. The ability to detect poor quality footage during production could enable a system to warn production teams that something may be wrong, potentially prompting immediate action to be taken to correct the error.

Equally useful would be a system that can review footage before the edit and flag unusable footage so that the editor does not waste time and money viewing it.

A type of poor quality video is where a high level of chromatic aberration is present (which tends to be most noticeable around edges). This is due to the prism effect of a lens, and is common when lenses are operated at the highest end of their focal range. Unacceptably large amounts of unintended chromatic aberration can also cause the final version of the programme to fail a technical review. If the lens properties are known, it is possible to correct some forms of chromatic aberration in post-production. However, detection of chromatic aberration in the production stage would be desirable to save time and money in post-production.

A system embodying the invention provides automated quality checking, particularly for chromatic aberration. A variety of possible checks can be performed in order to ascertain the quality of footage. Each of these checks can be carried out separately, and subsequently combined and presented to the production staff via a Graphical User Interface (GUI). The present disclosure relates to determining the presence of chromatic aberration.

A system embodying the invention will now be described in relation to Figure 1 which shows the functional components of an entire system embodying the invention. Multiple sources of audio video content provide that audio video content to one or more displays 12 via a connection path 7. Typically, the sources will be cameras either in a studio or at a remote location connected via a wired or wireless communication link 7 direct to the displays 12 of a production team. In addition, the cameras will connect to a variety of other studio equipment including recorders, controllers, and so on that are not shown here for simplicity of explanation. In addition to connecting to the displays 12, the cameras 2 have S 4 an input 3 to a monitoring system 4. The input 3 may comprise multiple separate hardware inputs, one for each camera, or more typically will be a communication network over which audio video content from each of the cameras will be streamed and received at a single network input port to the monitoring system 4.

The monitoring system 4 has an output 5 that can be asserted to the displays 12 to provide a visual indication of the existence of a quality problem with one or more of the cameras 2. This visual indication can be separate from the display of image on the display, but is preferably an overlay on the relevant display or part of the display. In this regard, the output signal on the output 5 may include the audio video content itself combined with the visual indicator. In such an arrangement the monitoring system 4 would be included as part of a larger system such as a production suite The monitoring system 4 also has an output 9 providing data indicating the nature of any quality issues regarding the audio video sources coupled to a metadata store 13 arranged receive the data and to as store metadata describing any quality problems with the corresponding feeds. The metadata may be periodically stored, such as at a certain interval of a number of-frames, or may be stored whenever there is a change indicated by the signal.

The monitoring system 4 comprises an analysis module 6, a settings database 8 and an output control 10 The settings database 8 stores user definable values to indicate the sources 2 for which monitoring is required and the types of analysis that should be performed to determine if those sources are correctly functioning. The types of analysis will be described later.

The analysis module 6 undertakes the analysis of each audio video signal in accordance with the settings from the settings database 8 and provides the results to output control 10 which asserts the output signal either separately or as part of the audio video signal at the output 5 as already mentioned.

The settings database B may be implemented by any known database technology and functionally provides a table with a row for each source 2 and columns for each of the types of analysis that are to be performed. The settings database 8 may be updated by an input at input line 9, which would typically be by a graphical user interface software controller. In this way, the user can select exactly which of the analysis techniques should be applied to which of the camera sources 2. In addition, the settings database stores for each source analysis combination the nature of the alert that should be given, for example green, amber or red. The user thus has control over the nature of the analysis for each source as well as the nature of the indication given.

The type of analysis that will be performed will now be described in greater detail with reference to the remaining figures The nature of chromatic aberration will first be discussed for ease of understanding.

Refractive index is the property of a material that provides a measure of the speed of light travelling through it as a function of wavelength. The angle of refraction of light at the surface boundary between two materials depends upon their relative refractive indices, according to Snell's Law. Lenses are designed very carefully and manufactured to high tolerances to mitigate this effect, but no lens is perfect. Sometimes light of different wavelengths from the same point in space can be directed by the lens onto a different point on the image sensor and result in the red, green and blue channels of an image being offset from each other.

There are two types of chromatic aberration. Axial chromatic aberration is due to different colours of light being focused at different focal distances. This can result in red and blue defocus (assuming green is in focus) which can be difficult to remedy. Transverse chromatic aberration is due to different colours of light being focused at different points on the focal plane resulting in red, green and blue being at different magnifications which it is possible to correct in post-production with appropriate scaling. However, in practice chromatic aberration results in a permanent loss of some detail. Correction is further complicated in single-sensor cameras, which use Bayer filters and electronic processing to produce multiple colour channels. Transverse aberration is more pronounced at the edges of the image than towards the centre, whereas axial aberration appears everywhere.

On a local scale however, the two forms have similar characteristics, so in terms of detection they can be treated as one.

Chromatic aberration results in coloured fringes at the edges of objects whose colour is composed of more than one of the primary colours. These fringes are unsightly and do not accurately represent the reality of the scene. Broadcasters require that the picture is free of excessive lens aberrations.

Detecting chromatic aberration The majority of research into chromatic aberration in imaging concentrates on its correction, but work has also been published on its use in forensics. It is possible to correct transverse chromatic aberration based on knowledge of the optics used to produce the image, without having to inspect the image at all. Software is available to apply appropriate distortions to the image channels in order to fix chromatically aberrated images in this way. The same approach can be taken without prior knowledge of the image or optics and instead using computer vision to analyse the distortion of, generally, the red and blue colour channels relative to the green. However, this is ultimately an image registration problem, which tends to be complicated to implement and computationally expensive.

Another approach is to look for non-smooth gradients in colour difference signals at image edges. Such algorithms are functional, but can give a high false positive rate. It is likely this would be considered acceptable for the application of correcting chromatic aberration so long as the correction method did not adversely alter already good" pixels. However, for the generation of metadata and live quality checking warnings, a low false positive rate is desired. In fact, a high false negative rate would be preferable to a high false positive rate.

The method of detecting chromatic aberration of this disclosure follows a different method than before and is computationally fast. The method uses the principle that chromatic aberration often produces visible magenta fringes around edges.

The proportion of pink hues present in the edge regions is compared to the proportion of pink hues in the entire image. While chromatic aberration can result in fringes of any colour, depending on how the lens is constructed, pink is less likely to appear in highly detailed areas of images without chromatic aberration.

Trees and grass, for example result in very green edges. Detecting pink edges has the added bonus of spotting primary axial colour', which has not been mentioned here because it is often so well corrected for in broadcast lenses. The pink and green fringes around prominent edges can be seen in Figure 2 and can be more dearly seen in the enlarged section in Figure 3.

Algorithm outline Considering Figure 3, where chromatic aberration is clearly seen occurring at the edges in the image, a first stage to detecting chromatic aberration is to detect the edges in the image and then use this image to form a representation of the chromatic aberration present. This is done by combining an edge image with the original colour image to form a coloured edge image. The hue histogram of this coloured edge image is then compared to the hue histogram of the original image and it is this comparison which can be used to detect chromatic aberration. An overview of the algorithm is shown in Figure 4.

Edge detection can be carried out using a Sobel mask on the luminance component of a VUV image. In this method we apply the Sobel filter separately in horizontal and vertical directions and then combine the two by summing, which is allowed to saturate the 8 bit unsigned type in which it is stored. The horizontal and vertical Sobel filters apply the following filter kernels respectively: 1-1 0 1 -I -2 -L S=I-z 0 2 S, 0 0 0 t-iOt 1. 2 1.

Figure 5 shows the luminance component of an image and Figure 6 shows the hue component of that image. The resulting edge image of these is shown in Figure 7 which is then subjected to a binary threshold, for example at 96 (8 bit values range 0-255) so that small edges are ignored, resulting in a binary image of edge pixels shown in Figure 8.

Applying the edge mask of Figure 8 to the original colour image the pink and green fringes become very obvious as shown in Figure 9. Using this edge mask, the method produces two histograms -one of the hues present in edge pixels, Figure 11, and one of the hues present in the entire image, Figure 10. A comparison of the ratios between the "pink" hues in these histograms tells us how pink the edges are compared to the whole image. * 8

The histogram values are scaled based on the proportion of the image considered edge or non-edge, so we have "mean pixel" hue histograms We take a ratio of the values in the two histograms for each bin in given range, here chosen to be the range 127-161. This represents the pink hues on a scale from 0-179 with the red primary at zero. The mean of these ratios is considered to be a measure of the chromatic aberration present in the image. This mean is subject to a threshold to determine if the image contains chromatic aberration or not.

The threshold level may be adjusted using the settings database, or may be a preset value for the system.

Disproportionately pink edges may indicate chromatic aberration, It may also indicate that a stripy pink object is in the image, so additional measures may be used for a more robust detection algorithm. The benefits of the disclosed method over those involving image registration or colour gradient analyses are that it is fast and computationally simple The effects of transverse chromatic aberration are more pronounced at the edge of the image than they are at the centre As such, the measure of aberration may be is weakened in its detection of transverse aberration by the inclusion of the central portion of the image. Simply masking this region may improve performance, but it may further improve performance to weight a pixel's contribution to the histograms based on its distance from the centre of the image.

Claims

CLAIMS1. A method of analysing an image to determine the presence or absence of chromatic aberration, comprising: -applying an edge mask to the image to produce a colour edge image; -producing a first histogram of the hues in the colour edge image and a second histogram of the hues in the image; -determining a mean ratio of the first and second histograms in a range of hues; and -asserting that chromatic aberration is present if the mean ratio is above a th resh old.
2. A method according to claim 1, wherein the mean ratio is determined by determining a ratio at each hue in the range and determining the mean of the ratios so determined
3. A method according to claim 1 or 2, wherein the range of hues comprises pink hues.
4. Apparatus for analysing an image to determine the presence or absence of chromatic aberration, comprising: -means for applying an edge mask to the image to produce a colour edge image; -means for producing a first histogram of the hues in the colour edge image and a second histogram of the hues in the image; -means for determining a mean ratio of the first and second histograms in a range of hues; and -means for asserting that chromatic aberration is present if the mean ratio is above a threshold.
5. Apparatus according to claim 4, wherein the mean ratio is determined by determining a ratio at each hue in the range and determining the mean of the ratios so determined.
6. Apparatus according to claim 4 or 5, wherein the range of hues comprises pink hues.