GB2532346A

GB2532346A - Method of reducing digital video flicker

Info

Publication number: GB2532346A
Application number: GB1518581.2A
Authority: GB
Inventors: Rabyking Serge
Original assignee: Apical Ltd
Current assignee: Apical Ltd
Priority date: 2014-10-20
Filing date: 2015-10-20
Publication date: 2016-05-18
Anticipated expiration: 2035-10-20
Also published as: WO2016063023A1; GB2532346B; GB201418619D0; GB201518581D0; US20170201669A1; US10063787B2

Abstract

Reducing flicker in digital video, in which, at constant time-averaged illumination intensity, video frames are recorded unevenly with respect to time and in synchronisation with a lighting flicker frequency. Preferably a pixel rate frequency of an image sensor is increased so as to permit more rapid video frame recordal than for the average time between video frames. Preferably the pixel rate frequency is increased to a fixed value, before the interval between frames is varied by a blanking value from frame to frame. Preferably the time interval between successive frames is adjusted so that the start of a frame will have the same light intensity as the corresponding start of all successive frames, whilst maintaining the overall frame rate. Preferably the video is assembled with frames which are equally spaced in time. Also disclosed is Reducing flicker in digital video, in which, at constant time-averaged illumination intensity, video frames are recorded unevenly with respect to time and either in synchronisation with a lighting flicker frequency or ninety degrees out of phase with the lighting flicker frequency, where image processing is then applied so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions.

Description

Intellectual Property Office Application No. GII1518581.2 RTM Date:14 March 2016 The following terms are registered trade marks and should be read as such wherever they occur in this document: iPhone (page 2, Lines 15-16) Red Digital Cinema (Page 2, Lines 26) Intellectual Property Office is an operating name of the Patent Office www.gov.uk /ipo

METHOD OF REDUCING DIGITAL VIDEO FLICKER

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention relates to methods of reducing digital video flicker, and to related systems, devices and computer program products.

2. Technical Background

A known problem in digital video recording is that flickering of the image may be seen during playback if the video is recorded under lighting powered by alternating current. The alternating current has a frequency of 60 Hz or 50 Hz. Because power varies as the square of the current, flickering lighting will have a flicker frequency which is double the mains frequency, which is therefore 120 Hz or 100 Hz. But this 120 Hz or 100 IL flicker frequency is not the flickering seen during video playback. The flicker during video playback may arise because of a beat effect between the constant but mis-matched frequencies of the flickering light source and the frame capture rate of the sensor.

Flickering in video playing may be readily perceived if the video playback flicker frequency is below about 60Hz. Flickering in video playing may be weakly perceived if the video playback flicker frequency is greater than 60Hz and below 100 Hz. Flickering in video playing will not be perceived if the video playback flicker frequency is 100 Hz or more.

Digital image cameras may include a digital image sensor, including an active imaging pixel array. The digital image sensor may be programmable, through an interface. The senor may be operated in a default mode, or it may be user-programmed to control the frame size, exposure, or gain setting, for example. The pixel array may include optical black columns and rows around the edges of the array, to monitor the black level, for black level adjustment. Image data may be read out in a progressive scan. Valid image data may be next to horizontal blanking and vertical blanking.

The Pixel Clock is a high frequency pulse train that may determine when the image sensor's data lines have valid data. On the active edge of the pixel clock (which can be either the rising edge or the falling edge, depending on the camera), the digital lines should all have a constant value that is input into the image acquisition device, which latches in the data. The data changes to the next pixel value before the next active edge of the pixel dock, so that the next pixel value will get latched into the image acquisition device. During the image capture process, each pixel accumulates light for a certain time and is then road.

3. Discussion of Related Art It is known to remove flicker by altering the frame rate of a digital video camera to equal the artificial lighting flicker rate divided by an integer. The idea is that the start of each frame will then have the same light intensity as the corresponding start of all successive frames -so that there will he no light flicker. So in the US, an iPhone6 with native 1080p filming at 60 frames per second or an iPhonc5 with native 720p filming at 30 frames per second will exhibit no flicker when filming in lighting flickering with a flicker rate of 120Hz, because 120/60 is exactly 2, and because 120/30 is exactly 4. But those same devices recording video in Europe, with a 1001Iz light flicker frequency, may exhibit flicker during video playback because 100/60 is not an integer, and because 100/30 is not an integer. And smartphones do not provide the end-user with any native capability to alter the frame rate to reduce flicker: a typical smartphone might permit recording at one frame rate, so 30 fps, or 60 fps, and another much higher rate for slow-motion recording. But they typically do not permit the kinds of adjustment needed to reduce flicker; this kind of adjustment is however possible in a professional-grade video camera, such as the cameras from the Red Digital Cinema Camera Company.

If the formula fps44N=1-c2 is valid, where fps is the frame rate, and F is the mains current frequency (501I2 or 6011z), and the integer number N=1,2,... then there will be no visible flicker. This is because in this condition, the frame rate of a digital video camera is equal to the artificial lighting flicker rate divided by an integer.

It is also known to remove flicker by altering the shutter speed of a digital video camera to equal the artificial lighting flicker rate divided by an integer. The idea is that each frame will then include a whole number of cycles of varying light intensity -so again there will be no light flicker. So if your camera shook at 60 frames per second, and you're in Europe with a 100Hz light flicker, then you may set your shutter to a constant 1/100 s. Or if you are shooting at 30fps with a 100 Hz light flicker, you can set your shutter to any of I /100s, I/50s, 1/33.3 s. But this approach is only suitable when the environment is not bright. The brighter the environment, the faster the shutter speed should be, otherwise the image will be overexposed.

A drawback of these known approaches to reducing flicker seen in video playback is that they may lead to under-utilization of device video recording capabilities. For example, for a given image resolution, a video recording device may be capable of filming at up to about 50 fps. But under 120Hz light flickering, to eliminate flicker, the frame rate would have to he reduced to 40 fps, because 120/40 is an integer, whereas 120/50 is not an integer. OF course 60 Fps would also eliminate Flicker, because 120/60 is an integer, but 60 Fps is not achievable by the device in this example. Tt is desirable to fully utilize device video recording capabilities, without leading to flickering in video playback.

SUMMARY OF THE INVENTION

According to a first aspect of the invention, there is provided a method of reducing human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, video frames are recorded unevenly with respect to time and in synchronization with a lighting flicker frequency. An advantage is that it is possible to fully utilize video recording capabilities of a device, without leading to flickering in video playback. The method has a technical effect on a process which is carried on outside the computer, because flicker is reduced during video play.

Optional implementation features include any one or more of the following: * a pixel rate frequency of an image sensor is increased relative to a pixel rate frequency of the image sensor corresponding to recording video frames evenly with respect to time at an average time between video frames, so as to permit more rapid video frame recordal than for the average time between video frames.

* the pixel rate frequency is increased to a fixed value, and then the interval between frame capture times is varied by varying a V blanking value from frame to Frame.

* some frames are recorded closer together in time than an average time between video frames, and other frames are recorded further apart in time than the average time between video frames.

* each frame is recorded at the same light intensity as the corresponding start of all successive frames.

* the time interval between successive frames is adjusted so that the start of a frame will have the same light intensity as the corresponding start of all successive frames, whilst maintaining the overall frame rate.

* human-perceivable flicker in the digital video is eliminated.

* the video is assembled with frames which are equally spaced in time.

* a position is determined using a position determining system, and the lighting flicker frequency is identified based on the determined position.

* the local mains frequency is a user settable parameter for use in video recording, and in which the lighting nicker frequency is double the local mains frequency.

* the lighting flicker Frequency is 120 Hz.

* the lighting flicker frequency is 100 Hz.

According to a second aspect of the invention, there is provided a system including a camera and a computer, the system configured to reduce human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, video frames are recorded unevenly with respect to time and in synchronization with a lighting flicker frequency, so as to reduce human-perceivable flicker in a video assembled using the computer from the video frames. An advantage is that it is possible to fully utilize video recording capabilities of the system, without leading to flickering in video playback. The system has a technical effect on a process which is carried on outside the computer, because flicker is reduced during video play.

The system may be one in which a pixel rate frequency of an image sensor of the camera is increased relative to a pixel rate frequency of the image sensor corresponding to recording video frames evenly with respect to time at an average time between video frames, so as to permit more rapid video frame recordal than for the average time between video frames.

The system may implement any one or more of the features defined above.

The system may be a computer vision system, an object recognition system, a human detection system, an autonomous vehicle computer vision system, a robotics computer vision system, a smartphone, a digital camera.

According to a third aspect of the invention, there is provided a computer program product for reducing human-perceivable flicker in a digital video, the computer program product when executing on a computer configured to record video frames unevenly with respect to time and in synchronization with a lighting flicker frequency, at constant time-averaged illumination intensity.

The computer program product may be further configured to implement any one or more of the features defined above.

According to a fourth aspect, there is a video recording device, such as a smartphone, which displays to the end-user a user-selectable option to reduce light flicker, and the device is programmed so that when the option is selected, then video frames are automatically recorded unevenly with respect to time and in synchronization with a lighting flicker frequency, at constant time-averaged illumination intensity. The smartphone may use a light sensor to measure the lighting flicker frequency, or may use knowledge of its geographic location to infer the lighting flicker frequency, or the end-user may define the AC current frequency. The device may implement any one or more of the features defined above.

According to a fifth aspect of the invention, there is provided a method of reducing human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, in a cycle of video recording, video frames are recorded unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and image processing is applied so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions. An advantage is that it is possible to fully utilize video recording capabilities of a device, without leading to flickering in video playback. An advantage is that this method may use less battery power than the method according to the first aspect of the invention. The method has a technical effect on a process which is carried on outside the computer, because flicker is reduced during video play.

According to a sixth aspect of the invention, there is provided a system including a camera and a computer, the system configured to reduce human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, in a cycle of video recording, video frames are recorded unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and image processing is applied so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions, so as to reduce human-perceivable flicker in a video assembled using the computer from the video frames.

According to a seventh aspect of the invention, there is provided a computer program product for reducing human-perceivable flicker in a digital video, the computer program product when executing on a computer configured to, at constant time-averaged illumination intensity, in a cycle of video recording, En record video frames unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and (ii) apply image processing so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions.

According to an eighth aspect of the invention, there is provided a video recording device, such as a smartphone, which displays to the end-user a user-selectable option to reduce light flicker, and the device is programmed so that when the option is selected, then at constant time-averaged illumination intensity, in a cycle of video recording, video frames are automatically recorded unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and image processing is applied so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects of the invention will now he described, by way of example only, with reference to the following Figures, in which: Figure 1 shows a schematic solution example in which a video is recorded at 50fps under 60T Tz mains frequency, without flickering.

Figure 2 shows a schematic solution example in which a video is recorded at 60fps under 50TP/ mains frequency, without flickering. 11)

DETAILED DESCRIPTION

In our approach, we keep the headline frame rate the same (e.g. if the camera shoots at 30fps, then we preserve that as an average frame rate), but we decrease the frame interval for some frames and increase the frame interval for others, whilst ensuring that the start of a frame will then have the same light intensity as the corresponding start of all successive frames, so there is no flicker in video playback. The frame start may be defined as the start of capturing the first pixel of the frame.

So we adjust the frame rate or time interval between successive frames so that the start of a frame will have the same light intensity as the corresponding start of all successive frames, whilst maintaining the overall frame rate, e.g. measured over a second or more.

It is normal to assume that if your video camera shoots at say 30Ips, then the time interval between successive frames must he a constant 33.3 ms (this assumption applies even when dealing with professional-grade video cameras). But instead, with an implementation of this invention, we can locally, i.e. within a time period of a second, adjust the time intervals so that they are 30ms, 30ms and 40ms, for example, it we have a 1001Iz flickering light source, which flickers every 10 ms. So here we speed up the frame rate for the first two frames, but slow it down for the last one. For all frames, the start of a frame will then have the same light intensity as the corresponding start of all successive frames -so that there will be no light flickering seen during video playback.

In an example, a video is recorded at 50Eps under 60Hz mains frequency lighting, which provides lighting flickering at 120 Hz. The average time between frames is 1/50 s which is 20 ms. The time between light flickers is 1/120 s which is about 8.3 ms. A conventional video camera would record an image every 20 ms. A flickering frequency may be visible in a recorded video at a beat frequency which is 120/2 I lz -50 1 lz which is 10 1 lz. A schematic frame recordal solution example is shown in Figure 1, in which the sinusoidal line represents the light intensity modulation, and the black rectangles represent image recordal. Instead of recording images every 20 ins, which could lead to flicker in a played-back video, images are instead recorded starting at 0 ms, 16.7 ms, 41.7 ms, 58.3 ms, and 83.3 ms. This provides the required 5 frames in 100 ms, which provides fps on average. 'the images are all recorded under equal intensity, which eliminates flicker.

In an example, a video is recorded at 60fps under 50Hz mains frequency, which provides a light flickering at 100 TTz. The average time between frames is 1/60 s which is about 16.7 ms. A conventional video camera would record an image every 16.7 ms. A flickering frequency may be visible in a recorded video at a beat frequency which is 60T Tz -100/2 TTz which is 10 TTz. A schematic frame recordal solution example is shown in Figure 2, in which the sinusoidal line represents the light intensity modulation, and the black rectangles represent image recordal. Instead of recording images every 16.7 ms, which could lead to flicker in a played-back video, images are instead recorded starting at 0 rns, 20 ms, 40 ms, 50 ms, 70 ms and 90 ms. this provides the required 6 frames in 100 rns, which provides 60 fps on average. The images are all recorded under equal light intensity, which eliminates flicker. Note that the actual shutter speed used is not relevant since it is constant over the short-time intervals relevant to reducing flicker; what is key here is the time interval between the start of successive frames.

An implementation issue can arise if we record a frame earlier than dictated by the video filming frequency. For example, the first two frames in Figure 1 are recorded 16.7 ms apart, whereas the 50 fps video filming frequency would dictate that these frames should have been recorded 20 ms apart. The 'pixel clock' or 'pixel rate frequency of the sensor might not be fast enough to permit recording a frame earlier in this way: if it is set for recording frames 20 ms apart it might not be able to record two frames just 16.7 ms apart.

We should clarify the image recording process. There are two processes: the capture and readout. The capture process is basically connected to the shutter. And readout is the process of reading data from the sensor. These processes do not occur at the same time, but are shifted by the shutter time. To simplify somewhat, when the shutter closes the readout starts. These two processes are working in parallel but are shifted by the shutter time. More specifically, a rolling shutter may be used -when a line of pixels stops light accumulation, the line of pixels is ready for readout of the pixel values. There is only one readout process and we cannot read the first pixel of the next frame until we have read out the last pixel of current frame. So we need to have a faster readout capability to be able to start reading the next frame earlier than the average time between frames, when required.

The flicker during video playback arises because of a beat effect between the constant but mis-matched frequencies of the flickering light source and the frame capture rate of the sensor. If we alter the time interval between successive frames so that they are not all the same, then we can reduce or eliminate the beat effect because we no longer have both the light source and frame capture operating at constant but mis-matched frequencies. So under 100 TTz flickering light, we could maintain a 30fps rate, but alter the successive frames to be taken at time 0 ins, then +30ms, +30ms, and +40rns (and repeating in this cycle). Key here is that not all intervals are the same: the +40ms interval is different to the 30ms interval. The 30ms is the more challenging for image capture, because it is less than the average time of 33.3ms between frames.

The skilled person will understand that if illumination intensity T as a function of time t is given by I(t)=I0 cos'(2aft), where 2f is the intensity flicker frequency, then the rime-averaged intensity is constant and is given by I-12.

Implementations in detail We assume we have a sensor working with a frame rate of N fps. Frames are read from the sensor at intervals of T=1/N seconds. During each interval we have a sensor read out of Ta seconds and a blanking time V such that T=Ta+V. it is important to note that it is possible to change the blanking interval V for most sensors in run time.

e assume Vna n is the minimum allowed blanking value for our system.

We may decrease the interval time T by decreasing V blanking down to Vmin. Also we could increase the interval time T by increasing V blanking to practically any value.

Suppose we have light powered with an alternating current with a frequency F. Actually the light flicker effect will be with frequency 2xF, because power varies with the square of the current. So the time intervals between light flickers will be Tf=1/(2xF). For example: for 501-Ix mains frequency, Tf=10ms, and for 601-Iz mains frequency Tf=8.3ms.

if we manipulate the value of V blanking in the way that 't will be a multiple of 'if will obtain pictures when all frames are at equivalent points in the light flickering cycle. This will eliminate the flicker artifact in the played-hack video. To achieve this we need a big enough V blanking range in the sensor.

The upper value of V blanking should be at least Vm IHT mod Tt). Here we come to limitation of: Ta=T-V<T4T mod TO-Vinin=1/N1-((1/N) mod 1 (2xF))-Vmin.

Let n=int((1/N)/(2x14))=int(2xF/N), where int(x) is the integer part of the expression x.

Then ((1/N) mod 1"/(2xF))=1/N -n x (1/(2 x F))= N -int(2xF/N)/(2xF), so Ta<(int(2 x F / IN)"/(2 x F) -Vmin) To satisfy this requirement, we need to program the light image sensor to a higher pixel rate frequency Fp.

Suppose we have a sensor with the following configuration: W -number of horizontal active pixels, Wb -horizontal blanking in pixels, H -number of vertical active pixels, then the time for read out will be Ta= (W+Wb)k41/Fp.

From here we get the minimum required pixel frequency Fp= (W (b)*11/(int(2 x F / N)/(2 x F)-Vmin).

For example, we have a sensor with active image pixel output of 192031080 at 60fps with \Vb=280 and 5011z mains frequency.

So Fp=(1920+280)(1080/(int(100/60)/100-Vmin)=2200* 080/ (1/100-Vmin) and roughly it is about 238IN1Hz instead of original 148.3MHz.

In this example it means roughly that the sensor needs to be programmed to the pixel frequency corresponding to the double of the flicker frame rate but some frames will he further apart in time than the average time between frames, to achieve the required average frame rate, while the recorded frames will he recorded at the equivalent points in the light flickering cycle.

in another example with a lower frame rate, we have a sensor with an active output of 1920 \ 1080 at 30fps with NVI]=280 and 5011z mains frequency. So Fp = (1920+ 280)* 1080 / ( int(100/30) / 100-Vmin) = 2200' 1080/(3 /100-Vrnm) and roughly it is about 79.2MHz instead of the original 74.25MHz.

in implementations, the pixel rate frequency is increased to a fixed value, and then the interval between frame capture times is varied by varying the V blanking value from frame to frame.

Even though the frames are captured at a non-constant rate, the video may be assembled with frames which are equally spaced in time. In typical implementations, the jerkiness which could arise because the video frame display times are not exactly the same as the frame capture times is too small to sec.

The variation of V blanking is necessary, if the pixel rate frequency is kept constant. This is because since T = Ta +V, if Ta is constant you can't change T without changing V. We can define the time instant when the very first pixel of the frame starts integration as "shutter start". We can define the time instant when the very last pixel of the frame stops integration as "shutter stop". The capture process is the internal process of the sensor that we control in terms of the "shutter start" and "shutter stop".

As a result we have four variables to drive the capture process: (a) the time difference between one "shutter start" and the next "shutter start", (b) the time difference between a "shutter stop" and the next "shutter start", (c) the time difference between a "shutter start" and the next "shutter stop" and (d) the time difference between one "shutter stop" and the next "shutter stop".

For the process in which video frames are recorded unevenly with respect to time and in synchronization with a lighting flicker frequency, this may be achieved by controlling "shutter starts" or "shutter stops" that is by controlling (a) or by controlling (d).

The sensor integration time directly impacts on (c).

The values of (a), (c) or (d) could be driven by modifying sensor parameters like total frame height or vertical blanking.

Because in most sensors there is access to a rolling shutter, we can eliminate the flicker by modulating phase ("shutter starts" or "shutter stops"), for the process in which video frames are recorded unevenly with respect to time and in synchronization with a lighting flicker frequency, but this does not remove uneven lighting across an image. However, any such uneven lighting effect is removable with a shading correction across the image, because this effect will he static and will repeat from frame to frame.

When the sensor shutter parameters are changed, which is basically an integration time change, therefore (c) is changed. If a change is applied to (d), the effect of uneven light distribution may shift across a frame. To avoid this shift, we should keep sum (b)+(c) constant by modifying (b) accordingly. However, if a change is applied to (a), the shift will not happen.

In case of a high frame rate it could use a lot of battery power to nth a sensor at a very high frequency. An alternative approach could be applied to partly remove flicker, that is to the case of flicker frequency G but with parameters for the flicker frequency n*G, where n is an integer which is 2, or which is greater than 2.

An example for the case n=2 is: with 60 fps and mains frequency 5041z, ie flicker frequency G of 100 Ilz, we can apply an approach for flicker reduction as if there were 24G (200112) frequency flicker. As a result we will have time intervals in multiples of 5ms and we will have two types of frames related to the flicker in terms of lighting. Basically we will have "ideal flicker", the flicker that will constantly repeat and not move across the image from frame to frame. The flicker will turn into blinking. This type of flicker could be easily removed by extra image processing in hardware or software. Also this blinking effect could be used to autodetect flicker.

In more detail, when in the presence of flicker frequency G we apply the approach for flicker reduction with parameters for the flicker frequency 2<G, we will have two types of frames that are related to each other by half a period, or we could say by 90 degrees phase difference.

For example if we have 60 fps and 50Th: mains frequency, 100 ITi is the flicker frequency, 200TT2 is double the flicker frequency, and we could put frames at time intervals of 15ms, 15ms, 20ms, 15ms, 15ms, 20ms, and so on. So the frame time positions will be 1st frame -Oms, 2nd frame -15ms, 3rd frame -30ms, 4th frame -50ms, 5th frame -65ms, 6th frame -80ms, 7th frame 10Orns and so on. We can see that 2nd, 5th, ... (3*m-1)th frames are related by 90 degrees phase difference to the other, non(3vm-1)th frames, where m is an integer. This is because (3*m-1)th frames occur at a multiple of 5 ms in time, but not at a multiple of 10 ms in time (eg. because 15 ins and 65 ms are multiples of 5 ms, but are not multiples of 10 ms), whereas the non-(3"m-1)th frames occur at multiples of 10 ms in time (eg. 0 ms, 30 ms, 50 ms, 80 ms). So we have one of every three frames blinking. In this example, three successive frames is a cycle of video recording, as would be clear to one skilled in the art.

First of all this makes it possible to detect flicker easily, because each pixel will have only two states of brightness in relation to the flicker. 'the difference between blinking frames can be measured in each pixel and compensation gain can be applied in image processing to eliminate blinking completely. Colourisation can be removed if the measurement is made per colour channel and the respective gain is applied per colour channel.

This additional approach is very important because we therefore have an approach to remove all possible effects produced by flicker and to remove it completely.

Applications Applications include those in which a video is recorded that might be affected by flicker associated with artificial light. Such applications include: * Smartph ones * Computer vision systems * Object recognition systems * Human detection systems * Autonomous vehicle computer vision systems * Robotics computer vision systems Devices which may utilize the method of reducing flicker in a video for playback include smartphones including a video camera, tablet computers including a video camera, desktop computers including a video camera, laptop computers including a video camera and digital video cameras, for example. The method may also be used by a computer in connection with, and controlling, a digital video camera.

Implementations for a Traveller In today's globalized world, users of devices including a digital video camera may travel from a region with an electricity supply which has a mains frequency of 50 Hz, to a region with an electricity supply which has a mains frequency of 60 Hz, or vice versa. An example is that North America has a mains frequency of 60 Hz, whereas Europe has a mains frequency of 50 Hz. Another example is that Japan has two regions, one with a mains frequency of 50 Hz, and the other with a mains frequency of 60 Hz.

It is desirable that such a device can continue to eliminate video playback flicker, even if the device travels between regions with different mains frequencies.

The device may determine its position using a position determining system, which may be a global positioning system (GPS) sensor, a system which can determine a position based on identified cellular phone network towers, or a system which can determine a position based on identified Wi stations, for example. The device may then identify the local mains frequency based on the determined position, and then implement a method of reducing human-perceivable flicker in a digital video, based on the local lighting flicker frequency, which is double the mains frequency. In an example, the local mains frequency is identified based on consulting a database on the device. For example, such a database could be a map database, in which various areas of the map each has an associated mains frequency. In an another example, the determined position could be sent to a server, the server then determining the mains frequency which corresponds to the determined position, and the server returning the mains frequency to the device, so that the device can then implement a method of reducing human-perceivable flicker in a digital video, based on the correct local lighting flicker frequency.

A device may be configured to determine the local mains frequency each time it is turned on. A device may be configured to determine the local mains frequency each time the digital camera is prepared for filming.

A device may include a setting which sets the local mains frequency which is assumed by the device when implementing a method of reducing human-perceivable flicker in a digital video. This may be useful when a device is located near a border between two regions with different mains frequencies, so that the local mains frequency might be identified incorrectly, such as due to position determination inaccuracy, for example. The setting may also be useful if the device is located somewhere in which its position cannot easily be determined, such as in a tunnel. The setting may also be useful if the mains frequency has drifted from a standard value, such as from 60 Hz to 60.5 Hz. Note

It is to be understood that the above-referenced arrangements are only illustrative of the application for the principles of the present invention. Numerous modifications and alternative arrangements can be devised without departing from the spirit and scope of the present invention. While the present invention has been shown in the drawings and fully described above with particularity and detail in connection with what is presently deemed to be the most practical and preferred example(s) of the invention, it will he apparent to those of ordinary skill in the art that numerous modifications can be made without departing from the principles and concepts of the invention as set forth herein.

Claims

CLAIMS1. Method of reducing human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, video frames are recorded unevenly with respect to time and in synchronization with a lighting flicker frequency.
2. Method of Claim 1, in which a pixel rate frequency of an image sensor is increased relative to a pixel rate frequency of the image sensor corresponding to recording video frames evenly with respect to time at an average time between video frames, so as to permit more rapid video frame recordal than for the average time between video frames.
3. Method of Claim 2, in which the pixel rate Frequency is increased to a fixed value, and then the interval between frame capture times is varied by varying a V blanking value from frame to frame.
4. Method of any previous Claim, in which some frames are recorded closer together in time than an average time between video frames, and other frames are recorded further apart in time than the average time between video frames.
Method of any previous Claim, in which each frame is recorded at the same light intensity as the corresponding start of all successive frames.
6. Method of any previous Claim, in which the time interval between successive frames is adjusted so that the start of a frame will have the same light intensity as the corresponding start of all successive frames, whilst maintaining the overall frame rate.
7. Method of any previous Claim, in which human-perceivable flicker the digital video is eliminated.
8. Method of any previous Claim, in which the video is assembled with frames which are equally spaced in time.
9. Method of any previous Claim, in which a position is determined using a position determining system, and the lighting flicker frequency is identified based on the determined position.
10. Method of any of Claims 1 to 8, in which the local mains frequency is a user settable parameter for use in video recording, and in which the lighting flicker frequency is double the local mains frequency.
11. Method of any previous Claim, in which the lighting flicker frequency is 120 ITz.
12. Method of any of Claims 1 to 10, in which the lighting flicker frequency is 100 Hz.
13. System including a camera and a computer,the system configured to reduce human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, video frames are recorded unevenly with respect to time and in synchronization with a lighting flicker frequency, so as to reduce human-perceivable flicker in a video assembled using the computer from the video frames.
14. System of Claim 13, in which a pixel rate frequency of an image sensor of the camera is increased relative to a pixel rate frequency of the image sensor corresponding to recording video frames evenly with respect to time at an average time between video frames, so as to permit more rapid video frame recordal than for the average time between video frames.
15. System of Claims 13 or 14, adapted to perform a method of any of Claims 3 to 12.
16. System of any of Claims 13 to 15, wherein the system is a computer vision system.
17. System of any of Claims 13 to 15, wherein the system is an object recognition system.
18. System of any of Claims 13 to 15, wherein the system is a human detection system
19. System of any of Claims 13 to 15, wherein the system is an autonomous vehicle computer vision system.
20. System of any of Claims 13 to 15, wherein the system is a robotics computer vision system.
21. Device including the system of any of Claims 13 to 15.
22. Device of Claim 21, in which the device is a smartphone.
23. Computer program product for reducing human-perceivable flicker in a digital video, the computer program product when executing on a computer configured to record video frames unevenly with respect to time and in synchronization with a lighting flicker frequency, at constant time-averaged illumination intensity.
24. Computer program product of Claim 23, further configured to perform a method of any of Claims 2 to 12.
25. A video recording device, such as a smartphone, which displays to the end-user a user-selectable option to reduce light flicker, and the device is programmed so that when the option is selected, then video frames are automatically recorded unevenly with respect to time and in synchronization with a lighting flicker frequency, at constant time-averaged illumination intensity.
26. The device of Claim 25 that uses a light sensor to measure the lighting flicker frequency, and/or uses knowledge of its geographic location to infer the lighting flicker frequency, and/or permits the end-user to define the AC current frequency.
27. Method of reducing human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, in a cycle of video recording, video frames are recorded unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and image processing is applied so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions.
28. Method of Claim 27, in which it is possible to detect flicker easily, because each pixel will have only two states of brightness in relation to the flicker.
29. Method of Claims 27 or 28, in which a difference between blinking frames can be measured in each pixel and compensation gain can be applied in image processing to eliminate blinking completely.
30. Method of any of Claims 27 to 29, in which colourisation can be removed by making measurements per colour channel and the respective gain is applied per colour channel.
31. Method of any of Claims 27 to 30, in which a pixel rate frequency of an image sensor is increased relative to a pixel rate frequency of the image sensor corresponding to recording video frames evenly with respect to time at an average time between video frames, so as to permit more rapid video frame recordal than for the average time between video frames.
32. Method of Claim 31, in which the pixel rate frequency is increased to a fixed value, and then the interval between frame capture times is varied by varying a 12-blanking value from frame to frame.
33. Method of any of Claims 27 to 32, in which some frames are recorded closer together in time than an average time between video frames, and other frames are recorded further apart in time than the average time between video frames.
34. Method of any of Claims 27 to 33, in which human-perce vable flicker n the digital video is eliminated.
35. Method of any of Claims 27 to 34, in which the video is assembled with frames which are equally spaced in time.
36. Method of any of Claims 27 to 35, in which a position is determined using a position determining system, and the lighting flicker frequency is identified based on the determined position.
37. Method of any of Claims 27 to 35, in which the local mains frequency is a user settable parameter for use in video recording, and in which the lighting flicker frequency is double the local mains frequency.
38. Method of any of Claims 27 to 37, in which the lighting flicker Frequency is 120 Hz.
39. Method of any of Claims 27 to 37, in which the lighting flicker frequency is 100
40. System including a camera and a computer,the system configured to reduce human-perceivable flicker in a digital video, in which, at constant time-averaged illumination intensity, in a cycle of video recording, video frames are recorded unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker Frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and image processing is applied so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions, so as to reduce human-perceivable flicker in a video assembled using the computer from the video frames.
41. System of Claim 40, in which a pixel rate frequency of an image sensor of the camera is increased relative to a pixel rate frequency of the image sensor corresponding to recording video frames evenly with respect to time at an average time between video frames, so as to permit more rapid video frame recordal than for the average time between video frames.
42. System of Claims 40 or 41, adapted to perform a method of any of Claims 28 to 39.
43. System of any of Claims 40 to 42, wherein the system is a computer vision system.
44. System of any of Claims 40 to 42, wherein the system is an object recognition system.
45. System of any of Claims 40 to 42, wherein the system is a human detection system
46. System of any of Claims 40 to 42, wherein the system is an autonomous vehicle computer vision system.
47. System of any of Claims 40 to 42, wherein the system is a robotics computer vision system.
48. Device ncluding the system of any of Claims 40 to 42.
49. Device of Claim 48, in which the device is a smartphone. 25
50. Computer program product for reducing human-perceivable flicker in a digital video, the computer program product when executing on a computer configured to, at constant time-averaged illumination intensity, in a cycle of video recording, (i) record video frames unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and (ii) apply image processing so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions.
51. Computer program product of Claim 50, further configured to perfotui a method of any of Cairns 28 to 39.
52. A video recording device, such as a smartphone, which displays to the end-user a user-selectable option to reduce light flicker, and the device is programmed so that when the option is selected, then at constant time-averaged illumination intensity, in a cycle of video recording, video frames are automatically recorded unevenly with respect to time, and all frames of the cycle are recorded either in synchronization with a lighting flicker frequency or 90 degrees out of phase with the lighting flicker frequency, and at least one frame in the cycle is recorded in synchronization with the lighting flicker frequency, and at least one frame in the cycle is recorded 90 degrees out of phase with the lighting flicker frequency, and image processing is applied so that all frames of the cycle are provided as if recorded under identical illumination intensity conditions.33. The device of Claim 52 that uses a light sensor to measure the lighting flicker frequency, and/or uses knowledge of its geographic location to infer the lighting flicker frequency, and/or permits the end-user to define the AC current frequency.