AT522631A1 - Image recording device with exposure device - Google Patents

Abstract

The present invention relates to an image recording device (1) together with a lighting device such as a flash device system (2, 3, 4) or a series of smartphones which make it possible to adjust the direction and strength of the exposure of the image after the recording.

Description

BACKGROUND OF THE INVENTION The present invention relates to an image capturing device together with a lighting structure consisting, for example, of a flash lamp system

or even some smartphones.

2 State of the art

When taking a photo with a camera, one of the most important things to consider is the light setting. There is no right or wrong in the lighting arrangement, but most arrangements use a combination of different light sources to create the naturalness of everyday lighting. Light sources can be viewed as point sources that create very sharply defined shadows. This type of light is called hard light and does not reflect typical daylight, but can be compared to direct sunlight. When clouds obscure the sun, the sunlight is scattered in millions of light sources that make light too soft and therefore too soft

Lead shadows.

In portrait photography, a very widespread technique is used, the so-called softbox, which is comparable to the cloud situation mentioned above. This structure disperses the light from a flash unit into many light sources that are evenly distributed on a large screen. This results in very soft shadows, but still a main direction of lighting, which is sometimes supplemented by a reflector or a second flash unit, to add to the spatiality of the

Scene.

1102-68-98 we 40,000

the 1SO settings of the sensor.

Sophisticated flash unit systems use additional pre-flashes to calculate the correct output of all flash units involved and thus to determine the correct scene.

to optimize lighting even further.

Once the image is captured, a 2D version of the three-dimensional scenery is mapped through millions of color and light values related to the position of each pixel on the sensor chip. At this stage post-processing is able to change the lighting, color and many other properties of the 2D image, which is also known as image processing.

To change the lighting after the picture has been taken, a prior art method uses the fact that the picture is in 3 black-and-white

is saved in images assigned to the colors red, blue and green.

If the exposure uses colored light from different directions, it is possible to use post-processing that weights each of the 3 black and white channels of the image to accentuate a particular direction of light. This method leads to a black and white picture with adjustable spatial lighting function. Since the differentiation between the directions of light is based on color, the resulting black and white image is still not a classic black and white image for the following reason:

When the object to be photographed is green, the red light is hardly reflected on that object and therefore the red channel shows hardly any reflections from that object. If the object is gray, this approach will give reasonable results

for any color evenly represented in gray.

A completely different situation applies to a non-static scenery like the one in

Portrait photography can be found. The subject can be instructed in this case

3102-68-98 we 40e0Fe0

Identify hair on a face.

Current frame rates of a film are in the range of around 24 frames per second - within a time frame of 300ms, several images are recorded in which a subject will move more than a few dozen micrometers and therefore a shift of the image from frame to frame will be inevitable. Shifting the image is still the best case scenario, in practice there are changes in the 3D scenery that cannot even be modeled by a set of 2D transformations that represent the 3D scenery, e.g. when the subject turns their head away from the camera.

Compensating for the displacement of the imaged subject between frames is one of the fundamental features of prior art video tracking and video coding techniques that can be used to compensate for these artifacts. The name of these techniques reveals about it

beyond the area of application and thus its limits: They are optimized to find correlations between individual images in which the lighting is more or less

niger is the same.

If the above mentioned setup with e.g. 2 flash units are used that illuminate a face from the left for frame 1 and from the right for frame 2, these video-tracking algorithms will fail because there is no correlation.

tion between these two images there.

It is therefore the object of this invention to solve this problem by inserting a so-called master frame into the set of recorded frames, which has the following properties:

3102-68-98 we 400Ee0590

the camera itself is transmitted.

The temporal positioning of the master frame within the image sequence can be chosen as desired, but is advantageously placed in the middle of the image sequence around the slightest changes between the first and the last

Frame.

After the image sequence has been recorded, all frames are processed with reference to the master frame, which defines the target position of the subject, contains all directions of exposure and thus can be used to identify any

of the frames to correlate with it.

The compensation techniques mentioned above are not part of this invention as they represent the state of the art. Depending on the level of complexity, the prior art includes simple methods such as e.g. moving the whole image by a few pixels to very sophisticated techniques that appreciate the underlying three-dimensional geometry of the scene and so on

Compensate found motion sequences for each pixel of each frame.

Up to this point it was assumed that the camera had already focused correctly before the first frame, which for most cameras requires some light to ensure optimal focus. If different light directions are of great interest, however, the ambient light must be kept at a very low level. To get enough light for focusing and at the same time reducing the ambient light while recording the image

A modification of the control system is required to achieve this.

E62 68/98 we 400E08a0

switched on again at the end of the recording of the entire image sequence.

This means that the first frame captured by the camera is not illuminated by the first flash unit, but by the ambient light that was used to focus the subject or object to be photographed.

lighten.

The basic principle of time-shifted recordings implies some restrictions for this invention; the more complex the motif movements during the recording, the more difficult it is to compensate for advanced post-processing

processing functions.

Today's smartphones already offer an image quality that is comparable to that of a typical SLR camera. The invention therefore not only relates to the classic studio setup with a camera and external flash units,

but can easily be adapted to one or more smartphones.

While the camera module of the smartphone can easily be identified as an image recording module according to the invention, the control module is distributed over many modules defined by hardware and software.

One possible implementation of the invention uses the smartphone as image recording and a simple software application to convert the times of the image recording into electrical signals that are available at one of the given interfaces (e.g. telephone plug for data transfer or USB connection). All other components remain the same as described above. The above-mentioned problem with the focus in low ambient light can also be solved with this smartphone setup. The software application in question can be used before the image sequence is recorded to determine the optimal optical

Find parameters. Due to the fact that a lightning bolt usually in one

1102-68-98 We 4 + 00e08a0

kept constant during the acquisition process.

Another implementation goes one step further and uses a more complex software application to take over the control function and the dedicated distribution of the trigger to each of the available flash units via wireless communication, be it e.g. NFC, WLAN or Bluetooth, which are hereinafter referred to as standard wireless communication protocols. With this implementation, the flash units must have a reception function for the

chose wireless communication protocol to be added.

Another implementation goes one step further and uses other smartphones than flash units, in which the built-in standard wireless communication protocols of these other smartphones are activated via a software application.

that control the flash units built into the smartphones.

Another implementation uses the above-mentioned setup with multiple smartphones, with the difference that all smartphones use their camera module for each trigger pulse of an image in the image sequence in order to capture an image. This results in a set of images, each of which can be used to create any lighting from a particular perspective

To synthesize smartphones that are involved in the recording.

Another implementation uses a different method of communication between the devices involved, called the time delay approach. So far, all of the setups described require very precise synchronization between the time the image is taken and the triggering of the flash units. While this is quite easy to accomplish in a wired setup, most of the standard wireless communication protocols available do not support low latency behavior as they are dedicated to transmitting data at high data rates while reducing the latency of a single data packet from low.

ger importance and the transfer only within a certain period of time

7732

E62 68/98 we 400e0880

running flash unit. This works like this:

Each smartphone operates an application that is connected via standard wireless communication protocols. In the initialization process, these applications negotiate the role of each device involved, e.g. which of the smartphones is to act as master and which flash units are to be addressed with which ID - an approach that can already be used in several network-compatible devices

is found. In addition, the individual time delay is distributed for each ID.

The master device starts the image recording process by recording the first frame, which is supported by the standard flash function to set the optical system (exposure time, ISO sensitivity, etc.). At the same time, all participating smartphones are time-triggered by this flash pulse via their camera system and trigger a timer in their application. The previously communicated individual time delay for each device involved between this received trigger and the individual flash device activation is based only on a synchronized time base among all smartphones, which is significantly more precise than the sum of the latencies as used in standard protocols for

wireless communication can be found.

3 BRIEF DESCRIPTION OF THE FIGURES

Figure 1 shows a schematic representation of an embodiment of the flash unit of the photo camera and its electrical connections;

Figure 2 shows a schematic representation of a studio setup with the

Flash unit of the photo camera in Figure 1 including the structure;

Figure 3 shows a schematic representation of the various states of the signal levels during the recording of the image;

3182-68-98 we 40080080

Figure 5 shows a schematic representation of a studio setup with smartphones only, in which each smartphone also functions as a source of illumination.

yaws and adopts an image sequence according to his position;

Figure 6 shows a schematic representation of the various states of the

Signal level while taking pictures with smartphones;

Figure 7 shows a schematic representation of the different states of the signal levels during the recording of the image, if a time delay mentioned above

lag approach is used to synchronize the images;

Figure 8 shows a schematic representation of a hybrid studio setup with ambient light, external flash units and smartphones;

4 DETAILED DESCRIPTION OF THE FIGURES

Figure 1 shows a schematic representation of an embodiment of the flash unit of the photo camera and its electrical connections with an external control module. This means that the signal flow from the camera to the control module through the X-Sync signal or another signal that signals the time of a recorded frame is unidirectional. This means that the camera records frame by frame without increasing the lighting for each frame

Taxes.

The power supply for the camera (1), the control module (2), the ambient light (4) and the flash units (3a, 3b, 3c, 3d) is not shown. For safety reasons, all parts are galvanically separated with separate power packs, signal connections are made via optocouplers to prevent damage from a possible failure of the high-voltage circuit inside the flash units.

change.

3182-68-98 we 4U0ESSN

Figure 3 shows a series of diagrams showing the different states of the signal levels at the input of each component, e.g. B. one of the flash units, while taking the pictures.

The first diagram shows the pulse train that comes from the camera (1) while the release button is pressed for approx. 1s. Each pulse shows one from the camera

recorded frame, which in this example comprises 6 images.

The first pulse of the pulse train (6) triggers the ambient light (4) to switch itself off (9) - it remains dark until the image sequence has been recorded. A simple time delay circuit ensures that the camera uses all flash units

triggers while the ambient light remains off.

The second pulse of the pulse train (6) comes from the camera (1) and triggers the first flash unit (3a), the duration of the pulse being matched to the required light output by the control module. There are so many ways to achieve optimized lighting by rapid flash circuits that are not part of the present invention and are therefore not described in detail here. The pulse duration in this example stands for any type of light power management and can also be set manually before the actual image sequence is recorded.

The third camera pulse of the pulse train (6) triggers the second flash unit (3b), while the 4th pulse (8) triggers all flash units (3a, 3b, 3c, 3d) at once to illuminate the master frame. As mentioned above, the pulse duration shown schematically stands for the light output. So you can see that the duration of the 4th

Pulses (8), which is delivered to the flash units, is significantly shorter because the image of

1102-68-98 we 4 SERIES

secure energy is discharged.

The fifth and sixth camera pulse in the pulse train triggers (7) the third (3c) and fourth flash units (3d) accordingly.

This setup described above describes an embodiment in which the control mechanism is located in the control module. The camera takes over all optical tasks such as focusing, which was determined using the ambient light situation, as well as all other settings such as aperture and shutter speed, which are automatically or manually adjusted by the photographer.

A completely different situation arises when the camera takes over control and triggers the flash units via a parallel or serial data interface. In this case, the correct lighting can be obtained from the camera in real time without manual

all default settings are calculated.

Figure 4 shows a schematic representation of a studio landscape, including the setup for use with smartphones only (100-104). All smartphones are aligned to the subject - in order to minimize the differences between the frames due to the portrait situation, a holder for the smartphone that takes pictures (100) is recommended. The control unit (2a-2e) is distributed over all smartphones involved, the master smartphone (100) uses the camera module (1) to capture the images and a software application for the flash units (3a-3d) via the wirelessly connected control -

devices (2a-2e) to control.

3102-68.-98 We 4 + 00E0890

and the flash unit triggering.

Figure 6 shows a schematic representation of the various states of the signal levels as it takes the set of images with smartphones (100-104) along with ambient lights (4a, 4b), as can be found in the schematic diagram of Figure 8. The distributed control devices (2a-2e) are not only found within the smartphones involved, but also in an external control

unit (2b) which is connected to the ambient lights (4a, 4b).

Figure 7 shows a schematic representation of the various states of the signal levels during the acquisition of the image when an above-mentioned time delay approach is used to synchronize the image acquisition process.

ren.

An application runs on each smartphone (100-104) that is connected via standard wireless communication protocols. In the initialization process, the master smartphone (100) negotiates the ID or address of each device. In addition, two time delay parameters are distributed to each slave smartphone (101-104): The time delay (20) between the first flash of the master smartphone (100) and the trigger to illuminate the master frame is the same for all devices involved. The second time delay parameter is different for each smartphone in order to generate the expected lighting scenario. This means that smartphone (101) is set to time delay parameters (20) and (21) during the initialization phase, while smartphone (102) is set to time delay parameters (20) and (22), smartphone 103 to (20) and (23), smartphone 104 is set to (20) and (24).

1102-68-98 we 40,000

Reference throughout the specification to "various embodiments", "some embodiments", "an embodiment" or "an embodiment" or the like means that a particular feature, structure, or feature described in connection with the embodiment relates to at least one embodiment. Thus, throughout the specification, the appearances of the phrases “in various embodiments,” “in some embodiments,” “in one embodiment,” or “in one embodiment,” or the like, do not necessarily all refer to the same embodiment. In addition, the special features, structures or features can be combined in a suitable manner in one or more embodiments. Thus, the special features, structures or features that are shown or described in connection with an embodiment can be combined in whole or in part with the features, structures or features of one or more other embodiments without restriction. because such a combination is not illogical or not functional

is.

It should be noted that, as used in this specification and the appended claims, the singular forms "a" and "the" include plural references,

unless the content clearly dictates otherwise.

1102-68-98 we 4 SERIES

Object or device are inherent.

All direction references (e.g. "plus", "minus", "up", "down", "up", "down", "left", "right", "front", "back", "above", "below" "," vertical "," horizontal "," counterclockwise ") are used for identification purposes only to aid the reader's understanding of the present disclosure and do not create any restrictions, particularly as to the position, orientation, or use of any aspect of the Epiphany. It is to be understood that the terms so used are interchangeable in appropriate circumstances so that the embodiments of the invention described herein may e.g. in other than those shown or otherwise act. in this

wrote.

As used herein, the phrase "configured for" and similar expressions indicate that the device, device or system concerned is designed and / or constructed (e.g., by suitable hardware, software and / or components) to perform a or to fulfill several specific purposes and not only

Lich is able to fulfill the purpose.

Connection references (e.g., "connected," "coupled," "connected" and the like) are to be interpreted broadly and may be intermediate elements between a

1102-68-98 we 40,000

All numbers that use measurements and so on in the specification and claims are to be replaced by the term "about" in all cases.

3102-68-98 We 4 + 00e0850

Claims (1)

1. A digital image capture device for capturing a scene consisting of: a. an imaging optical system adapted to capture the plurality of images sequentially; b. a variety of lighting sources for illuminating that scene; c. Means for controlling these lighting sources and this optical imaging system, which provides a different lighting during the acquisition of these plurality of images by the lighting direction or a combination of lighting directions and weighting of this combination by lighting power or lighting duration or a combination of lighting of power and duration; d. Means for controlling these lighting sources in order to illuminate at least one of the recorded images, which are designated as master frames during the image sequence acquisition, by activating all available lighting sources; e. Means for compensating for movement-induced differences between the recorded images and one or more of said master frames by post-processing the images; f. Means for combining the processed images or areas of the processed images or a subset of the processed images o- the areas of a subset of the processed images to a final DZ 68-98 we 40000 2. The image recording device according to claim 1, wherein one or more people are portrayed. 3. The image pickup device according to claim 1, wherein one or more objects are objects of the image. 4. The image pickup apparatus according to claim 1, wherein the imaging optical system is a digital still camera. 5. The image pickup device of claim 1, wherein the imaging optical system is a digital photo camera module in a mobile device such as a smartphone or a tablet computer. 6. The image pickup device according to claim 1, wherein the lighting sources are flash devices. 7. The image pickup device according to claim 1, wherein the sources of illumination are flash units located in a mobile device such as a smartphone or a tablet computer. 8. The image recording device according to claim 1, wherein the control of the lighting sources is carried out by a module that receives trigger signals from the image recording device and sends them to the respective connected flash devices, the trigger signal of the master frame being an exception in which the module is all connected Flash device ten sends a trigger signal. 3102-68.-98 we 4U0E0SS0 The ambient light source of claim 9 connected to the master device via standard wireless communication protocols. 11.The control module according to claim 8, wherein said lighting sources are controlled by a software application in a mobile device, referred to as the master device, which is connected to the camera module of this mobile device as well as to external flash devices via standard wireless communication protocols. 12.The control module of claim 8, wherein said lighting sources are controlled by a software application in a mobile device, referred to as the master device, which is wired to the camera module of that mobile device as well as external flash devices. 13.The control module of claim 11, wherein the external flash units are flash modules such as those found in mobile devices such as a smartphone or tablet computer that are controlled by a software application that receives the trigger signals for triggering the flash from the master device Standard wireless communication protocols obtained. 14.The image recording device according to claim 1, in which these illumination sources are controlled such that at least one of the recorded captured images, called master frames, during the 3102-68-98 we 4U0ERSSN .. .. ® .. .. ... Image sequence recording can be evenly illuminated with all available lighting sources. 15.The image recording device according to claim 5, wherein it is a software application, which runs on the master smartphone and is connected to the software application of the slave smartphones via standard wireless communication protocols and initially distributes information on the time offset of the flash pulses of the devices involved in relation to the very first flash pulse of the master smartphone the individual images recorded during the image sequence with the flashes of each of the slave devices involved are transmitted in accordance with the initial phase the schedule are synchronized. 1102-68-98 We 4LE0FeN