GB2537826A

GB2537826A - Image capture system

Info

Publication number: GB2537826A
Application number: GB1506897.6A
Authority: GB
Inventors: Stephen Cole Richard; Anthony Eves David
Original assignee: Ambx UK Ltd
Current assignee: Ambx UK Ltd
Priority date: 2015-04-23
Filing date: 2015-04-23
Publication date: 2016-11-02
Also published as: GB201506897D0

Abstract

A system 10 having an image capture system 12 arranged to capture a plurality of image frames per second (FPS), a lighting system 14 arranged to emit modulated light 16, and a synchronisation system 18 connected to the image capture system and the lighting system. The synchronisation system operates the lighting system to adjust the light level between image frames (20, fig 2) captured by the image capture system relative to the light level captured in the image frames. The adjusted light level may be used to produce an additional visual element (22, fig 5), which may provide a visual prompt/aid for an actor working in front of a green screen or scene (24, fig 4). The modulated light may be produced from different coloured light emitting diodes (LEDs), which when synchronized with the frame capture rate of the camera, would create a lighting effect or scene that is invisible in the final filmed result. The lighting system may emit outside the visible spectrum, such as in the Ultraviolet (UV) or infrared (IR) spectrums.

Description

DESCRIPTION

IMAGE CAPTURE SYSTEM

This invention relates to a system including an image capture system and to a method of operating the system.

Image capture systems are well known in many fields of technology. For example, in the television and film industries live and non-live events are io recorded using image capture devices such as video cameras. Such cameras generally work by capturing a fixed number of images (image frames) per second, which are then either recorded for later broadcast or are transmitted immediately to end viewers. For example, at a major sporting event that is to be televised, a large number of fixed and mobile cameras are present that will capture the action as it occurs, with a live broadcast of the captured images being selected by a television producer.

In the film industry, video cameras are used to capture scenes being shot, usually with numerous takes, which are then edited and selected to form the final finished product. The increase in computing and software power has meant that a large amount of on-screen content can be created electronically, without being captured via a video camera. This is very common in big budget films, with the term "CGI" being used for the content that is created in addition to the actual human content delivered by actors that is captured using conventional video cameras. A common technique that is used is called "green screen", whereby live action acting is captured by a video camera while the actor(s) is placed in front of a green background. Post-production techniques then add the CGI material in place of the green areas in each image frame. This known and widely used technique has the problem that it is more difficult for the actor(s) to work in such an environment, as they have far fewer visual clues to work from, when acting through the scene being captured.

It is therefore an object of the invention to improve upon the known art.

According to a first aspect of the present invention, there is provided a system comprising an image capture system arranged to capture a plurality of image frames per second, a lighting system arranged to emit modulated light, and a synchronisation system connected to the image capture system and the lighting system and arranged to operate the lighting system to adjust the light level in the spaces between image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system.

According to a second aspect of the present invention, there is provided io a method comprising capturing a plurality of image frames per second with an image capture system, emitting modulated light with a lighting system, and adjusting the light level in the spaces between image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system.

Owing to the invention, it is possible to use modulated light such as pulse width modulated (PWM) lighting (for example provided by LEDs) synchronised with the frame capture rate of digital film cameras to create lighting effects that are invisible in the final filmed result. By embedding a certain selected portion of the light contribution into the gaps between frames, the lighting contribution can differ between that perceived by the live actors and the result captured by the camera and displayed in the cinematic playback. This can allow additional cue or comfort lighting to be included on set. The modulated light is produced using a digital signal that switches from on and off periodically and in the control of LED lights this is achieved using pulse width modulation.

For example, a dark scene may be more brightly lit to assist the actors, colour temperature may be shifted or special effect cues embedded into the lighting for the actor to react to. This will be particularly useful in "green screen" scenes, where an actor has few cues as to what will be happening in the final resulting post production scene and will benefit from timing and effect cues to act against. These however cannot interfere with the final material or complicate the composite post production process. A far flatter light effect is possible for the filmed result.

A similar approach could be used in TV production of live events where the difference in lighting between that wished to be experienced by those in the venue and as suitable for capturing the event for a viewer may be achieved by altering the interleaved proportions of different wavelengths of light.

Embodiments of the present invention will now be described, by way of io example only, with reference to the accompanying drawings, in which:-Figure 1 is a schematic diagram of a system, Figure 2 is a schematic diagram of image frames, Figure 3 is a flowchart of a method of operating the system, Figure 4 is a schematic diagram of scene as viewed from the naked is eye, Figure 5 is a schematic diagram of scene as viewed from an image capture device, and Figure 6 is a schematic diagram of light and image capture phasing.

Figure 1 shows schematically a system 10 that comprises an image capture system 12 which is arranged to capture a plurality of image frames per second, a lighting system 14 which is arranged to emit pulse width modulated (PWM) light 16, and a synchronisation system 18 which is connected to the image capture system 12 and the lighting system 14. The lighting system 14 comprises a plurality of LED lights. The system 10 can be used in a film studio, for example, where filming is taking place that requires the use of "green screen" technology, as discussed above. The image capture device 12 is a video camera that captures image frames at a specific frame rate per second.

LED light systems work by emitting PWM light. There will be two types of frame of light in the PWM rendering of a colour in the lighting system 14.

The PWM is a proportion of on and off of various emitters for example, red, green and blue LEDs. Some frames of the PWM signal will be those captured by the frame rate of the camera 12, some will be between those frames. For example, a camera 12 that captures twenty-four frames a second (a standard cinema frame rate) will only capture light that is emitted from the LEDs at the exact moment in each second that a frame is captured.

The synchronisation system 18 operates an algorithm that will adjust the colour or intensity of the camera visible frames to achieve the desired colour of the filmed light, while delivering a corresponding addition or reduction of contribution by the invisible frames which will be perceived only by the live viewers. The live viewer's eye will perceive the light as a full combination of all contributing PWM frames, while the camera 12 will capture only the light contribution on the sub-sampled frames. Essentially, the light emitted from the lighting system 14 is controlled so that the light visible to a normal viewer of the scene is different from that captured by the camera 12.

PWM light works on the basis that the human eye combines all of the contributions of light over a period resulting in a different perceived result. The red, green, and blue emitters in a typical coloured LED light are being turned on and off at 1000s of Hz in different proportions of on to off to make different perceived mixtures. By altering the mixture in the captured and un-captured frames it is possible for the live effect to differ from the captured effect. Figure 2 illustrates a one quarter of a second film sequence, which at a frame rate of twenty-four frames a second, results in six frames 20 being captured by the video camera 12.

In general, the loss of light being captured by the camera 12 is seen to a detrimental effect where the pulse frequency of the LED PWM light is in phase with the camera capture rate, since this results in dark bands or flicker effects where the camera sees mainly the "off" phases of the LED lights. By providing a synchronisation between the two systems (the camera 12 and the lighting system 14) it is possible to algorithmically manage the contribution of the lighting to the scene while the camera 12 is recording. The synchronisation system 18 controls the synchronisation and the light output levels of the lighting system 14 to achieve the desired effect.

The contributions are combined for a viewer in the live space so the contribution between frames 20 may have to be adjusted to compliment the desired recorded effect or vice versa. For example, if extra red pulse contribution is being used in the captured view it might be lessened in the non-captured frames. With light being additive some effects will not be possible to achieve, for example the live viewer could not see a saturated green scene with the camera capturing one that was saturated red. However, where the live view is well lit with white light there is considerable option to subtract certain colours from the captured view. Potentially multiple differently synchronised fo cameras 12 could be capturing different colour views with more complex algorithms to split the total pulse modulation.

High frequencies are generally used to give smooth colour mixing with minimal perceived flicker. This will lend itself well to combination with the high frequency capture of high definition (HD) video cameras. In general many on/off pulses of each light emitter might be generated within each captured or non-captured period for the camera and the mix of contributing pulses varied across these to avoid "beat" effects. The system 10 is well suited to use with HD video cameras and the synchronisation system 18 can ensure that the necessary light levels are outputted by the lighting system 14 such that at even at high frame rates it is still possible to emit additional light that will be seen by the naked human eye, but not captured by the camera 12.

It may be necessary to "over specify" the lightings maximum output to achieve the desired effect as it will be necessary to have the pulses off for certain contributing colours within the recorded frame and then to compensate in the non-recorded period. This may require that a level of pulse considerably greater than the total perceived result is necessary in these short windows. The emitters do not only have to be in the visible range, there could be IR, UV or other emissions within the transmission. Then, for example, special reflectors or detectors could be used for these emissions but isolated or excluded in the video capture.

The system 10 is also applicable in security situations where a UV or IR reflective marker may be followed by the camera 12 which does not see the visible spectrum light transmitted in the non-camera-visible frames but that washes out the UV effect and provides illumination for the live viewer. On a film set, the path of a special effect or computer rendered character that will only be added in post-production could be simulated by a lighting effect that is only visible on set but not in the captured material. Similarly cues could be done by lighting for the actors without affecting the final result. In a stadium lighting levels can be held higher in seating areas for audience safety but fade out when shown on TV to allow greater concentration on the action.

Figure 3 shows a flowchart of the method of operating the system 10.

The method comprises, at step S3.1, capturing a plurality of image frames 20 per second with the image capture system 12, at step S3.2, emitting modulated light 16 with the lighting system 14, and, at step S3.3, adjusting the light level in the spaces between image frames 20 captured by the image capture system 12 relative to the light level at the image frames 20 captured by the image capture system 12. The synchronisation system 18 is responsible for controlling the output of the lighting system 14 to ensure that the necessary light is emitted out of phase with the image capture system 12.

In a preferred embodiment, the lighting system 14 is arranged to output a plurality of different colour components of light and the synchronisation system 18 is arranged to operate the lighting system 14 to increase at least one colour component in the spaces between image frames 20 captured by the image capture system 12 relative to the light level at the image frames 20 captured by the image capture system 12. In this way, individual colour levels can be raised for the light that will be seen by the human eye, but this additional coloured light will not be captured by the camera 12, which is out of phase with the raised light levels for the specific colour component(s).

The synchronisation system 18 can be a dedicated piece of hardware that is connected to the other components or could be built into a lighting control system that will control all of the lights that make up the lighting system 14. Alternatively, the synchronisation system 18 could be a conventional piece of computing hardware such as a desktop computer or server that is controlled by a computer program product on a computer readable medium. The computer program product comprises instructions that can be used to control the operation of the physical hardware that provides the necessary control functions for the lighting system 14, thereby controlling the individual lights to emit light as discussed above.

Figures 4 and 5 are schematic views of a scene 24 being recorded, as viewed from the point of view of the naked eye (Figure 4) and as captured by the image capture system 12 (Figure 5). The scene is an example of how the system 10 of Figure 1 would work in a studio environment, for example where a film is being recorded that is using a "green scene" technology for at least to some of the scenes that make up the film. In this case, it is desirable to provide additional visual cues for the actors performing in the film, so that they are able to work in an environment that has more than just a green background.

The scene is illuminated using the lighting system 14, which comprises a plurality of LED lights which are controlled to emit PWM light 16. The phasing of this light 16 is controlled by the synchronisation system 18, which is able to introduce additional visual elements 22, which are visible with the naked eye, since the eye will capture all of the light 16 emitted by the lighting system 14, but are not captured by the camera 12. The camera 12 only captures image frames 20 at specific moments in each second, in the film environment this is typically twenty-four frames per second. Light outputted out of phase with the camera 12 is not captured.

The lighting system 14, as controlled by the synchronisation system 18, essentially emits light that varies in amplitude throughout a specific period (which is simplest to consider as one second, since camera frame rates are normally defined on a frame per second basis). During the specific time period, the lighting system 14 adjusts the light level in the spaces between image frames 20 captured by the image capture system 12 relative to the light level at the image frames 20 captured by the image capture system 12. As seen in Figure 5, the elements 22 are not present and not captured by the camera 12, although they are seen by the naked eye.

Figure 6 shows the phasing of different light colour components set against the phasing of the camera 12. The lighting system 14 is controlled so that two of the colour components of the light are outputted in phase with the image capture by the camera 12. The third component is outputted out of phase with the image capture by the camera 12. The three colour components can be the regular RGB (red, green, blue) used in many colour systems and outputted by the LEDs of the lighting system 14. The top two colour components are in phase with the camera 12 and the bottom colour component is out of phase with the camera 12.

In this example, the three colour components will be visible to the naked human eye, the fact that two of the components are in phase and one of the fo components is out of phase does not affect human vision, since all of the components will be seen by the eye, which essentially has a much faster sampling rate of visual content than the camera 12. However, the camera 12 will not capture the component that is out of phase, since the camera 12 is not capturing an image frame 20 at the particular points in time when the bottom component is actually being emitted from the lighting system 14, as controlled by the synchronisation system 18.

The phasing shown in Figure 5 is a simplification, as different light (amplitude) levels of the individual colour components will be present in order to achieve different colours. In an LED system that uses a large number of individual LEDs, different colour components at different lights will be in and out of phase at different times, in order to achieve the desired colours that should be visible to the naked eye and as seen by the camera 12. The fundamental feature is that the light outputted uses the same phasing as the image capture device 12, with different light levels in the spaces between image frames 20 captured by the image capture system 12 relative to the light level at the image frames 20 captured by the image capture system 12. The PWM frames of light can be at a very high frame rate per second even within the spaces between the interleaved film frames 20.

Claims

CLAIMS1. A system comprising: * an image capture system arranged to capture a plurality of image frames per second, * a lighting system arranged to emit modulated light, and * a synchronisation system connected to the image capture system and the lighting system and arranged to operate the lighting system to adjust the light level in the spaces between io image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system.
2. A system according to claim 1, wherein the lighting system I s comprises a plurality of LED lights.
3. A system according to claim 1 or 2, wherein the lighting system is arranged to output a plurality of different colour components of light and wherein the synchronisation system is arranged to operate the lighting system to increase at least one colour component in the spaces between image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system.
4. A system according to claim 1, 2 or 3, wherein the synchronisation system is arranged to operate the lighting system to increase the light level between image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system to produce an additional visual element which is not captured by the image capture system.
5. A system according to any preceding claim, wherein at least part of the lighting system is arranged to emit light outside of the visible spectrum.
6. A system according to any preceding claim, wherein the lighting system is arranged to emit pulse width modulated light.
7. A method comprising: * capturing a plurality of image frames per second with an image capture system, * emitting modulated light with a lighting system, and * adjusting the light level in the spaces between image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system.
8. A method according to claim 7, wherein the lighting system comprises a plurality of LED lights.
9. A method according to claim 7 or 8, wherein the emitted light comprises a plurality of different colour components of light and wherein the adjusting of the light level comprises increasing at least one colour component in the spaces between image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system.
10. A method according to claim 7, 8 or 9, wherein the adjusting of the light level comprises increasing the light level between image frames captured by the image capture system relative to the light level at the image frames captured by the image capture system to produce an additional visual element which is not captured by the image capture system.
11. A method according to any one of claims 7 to 10, wherein the emitting of modulated light comprises, at least in part, light outside of the visible spectrum.
12. A method according to any one of claims 7 to 11, wherein the emitting of modulated light comprises emitting pulse width modulated light.