WO2017211511A1

WO2017211511A1 - Method and device for creating a panoramic image

Info

Publication number: WO2017211511A1
Application number: PCT/EP2017/060747
Authority: WO
Inventors: Arno Schmidt
Original assignee: Rheinmetall Defence Electronics Gmbh
Priority date: 2016-06-10
Filing date: 2017-05-05
Publication date: 2017-12-14
Also published as: DE102016110686A1

Abstract

The invention relates to a method for creating a panoramic image from individual images from a number of cameras, comprising the following steps: selecting (101) a zoom level for the recording of the individual images by means of the number of cameras, setting (102) a configuration for the number of cameras in such a way the cameras are configured to record the respective individual images at the selected zoom level with maximum imaging performance, recording (103) the individual images by means of the number of cameras at the selected zoom level with the set configuration, determining (104) a transformation specification for the selected zoom level for transforming the recording individual images into the panoramic image, and transforming (105) the recorded individual images into the panoramic image by means of the determined transformation specification.

Description

METHOD AND DEVICE FOR CREATING A

PANORAMA IMAGE

The present invention relates to a method and apparatus for obtaining a panoramic image from frames of a number of cameras.

The technical field of the invention relates to the creation of panoramic ^{stable ¬} countries. A panoramic image is characterized by the cover of a large Be ^¬ trachtungswinkels. For example, a panorama image can be created by using different frames of a scene by means of a stitching

Procedure to be assembled. For example, a panoramic image of an airport area is helpful for a pilot of the airport to get a good overview of the recorded airport area. For example, a camera and stitching software are used to create panoramic images. Stitching by the software, the Pa ^¬ noramabild from each of the camera recorded camera images will ^¬ sammengefügt. The manner of co ^¬ conventionally journal notebook static. In other words, a static transformation rule is used to the panoramic image zusammenzufü ^¬ gen. A static transformation rule changes during operation, the individual pictures are not.

Furthermore, electronic methods can also be used in which the geometric juxtaposition of the individual images or individual videos takes place via image transformations that have been defined once. The resulting Panora ^¬ mabild is then fixed in the frame and / or the image orientation and can not be changed. In a static transformation only the possibility remains digital in the

To zoom image into it, because the apparent optical alignment beziehungswei ^¬ se configuration can not be changed. A zoom in conditionally but loss of image quality, especially in terms of resolution. ^Such qualitative losses are at least undesirable for the user, and in ^certain cases are also not acceptable under regulatory law. Against this background, an object of the present invention is to improve the creation of a panoramic image from frames of a number of cameras.

Accordingly, a method is proposed for producing a panoramic image from individual images of a number of cameras. The method comprises the following steps a) to e):

a) selecting a zoom level for taking the frames by the number of cameras,

b) Setting a configuration for the number of cameras such that the respective camera to record the respective frame with the selected

Zoom level is set at maximum imaging power,

c) receiving the frames by the number of cameras to the ge ^¬ selected zoom level and the set configuration,

d) determining a transformation rule for the selected zoom stage for transforming the captured frames into the panoramic image; and e) transforming the captured frames to the panorama image using the determined transformation rule.

Advantageously, the configuration for the number of cameras is present aufneh- Menden set such that the respective camera with maximum image quality, especially with maximum resolution that can receive respective ^¬ A zelbild. Furthermore, in the present case, the transformation rule for the selected zoom level is determined in each case, that is to say determined in particular dynamically. This results in a dynamic adjustment of the transformation tion regulation and the selected configuration for receiving Kame ^¬ ras. This is advantageous, for example, if the receiving camera or the receiving cameras basically have the airspace of an airport to monitor, but in some cases especially only enlarged the landing ^strip is to be displayed. Another example is the application on schif ^¬ fen in which the recording camera or recording cameras should ge ^¬ Nerell observe the surrounding sea area, in special cases, comparable enlarges intended to represent the horizon. In contrast, remain at a static arrangement, as set forth above in detail, only the adverse possibility digi tal ^¬ to back in the recorded panorama image into it.

A panoramic image is understood to be an image from a plurality of individual images, which covers a large viewing angle. A single image is an image which will be ^¬ taken by a camera at a certain time. Under a single image, a single video can be ver ^¬ were present. Accordingly, a Pano ^¬ ramavideo can be understood as a panorama.

The respective camera can for example also be a thermal imaging camera or an infrared camera.

According to one embodiment, step a) comprises:

Set at least one angle of the camera to take the picture

Frame.

By determining the at least one angle of view, the zoom level for recording the individual images can be determined in a simple manner. The image angle refers to those angles in the object space which are bounded by the edges of the recording format.

According to another embodiment, step a) comprises ^:

Set a horizontal angle of the camera, a vertical angle of the camera, and / or a diagonal angle of the camera to capture the still image. In a rectangular recording format, the diagonal angle of view is the largest of the three angles of view. The direction of the respective image angle versus Ho ^¬ rizontalen is at the horizontal angle between 0 ° and 180 °, when Vertika ^¬ len angle of between + 90 ° and - 90 °, and in diagonal angle of +/- arctan (height / width).

According to a further embodiment, step b) comprises:

Using a specific array of the number of cameras for recording ^¬ the individual images for the panoramic image.

In order to set the optimum configuration in this embodiment, a specific array, in particular a specific subset, of the number of cameras for capturing the individual images for the panoramic image is selected and aligned accordingly to the user's image ^request , for example, to a specific detail view of an airport area airport.

According to a further embodiment, step b) comprises:

Setting an alignment of at least a portion of the number of Kame ^¬ ras.

Here, the cameras which receive current frames are sol ^¬ len, aligned according to the recording requested by the user. For this orientation zoom optics, pan-tilt heads or pan-tilt systems and / or beam guidance systems can be used.

According to a further embodiment, step b) comprises:

Setting a beam guidance system for the number of cameras.

According to a further embodiment, step b) comprises:

Setting at least one zoom lens for the number of cameras. According to a further embodiment, at least a portion of the number of cameras is used to adjust the alignment of a pan-tilt system ver ^¬ turns. The pan-tilt system includes, for example, a number of pan-tilt heads. In this case, a respective pan-tilt-head is coupled to one of Kame ^¬ ras. The pan / tilt head can then be used to adjust the orientation of the paired camera. According to another embodiment) to be picked up by the individual cameras with a predetermined overlap of the A ^¬ zelbilder the Einzelbil ^¬ in step c.

The predetermined overlap of the frames allows the use of a stitching method or a stitching method to combine the frames into the panoramic image. Under the stitching together ^¬ menfügen a plurality of frames or individual images to the ge ^¬ genüber larger each individual image panorama image is understood in the present case. When Sti ^¬ tching the individual recordings of excerpts of the image to be created are joined to the larger panoramic image.

According to a further embodiment, the transformers ^¬ tions regulation stage is selected depending on the selected zoom of a plurality of predetermined transform rules in step d).

The predetermined transformation rules can be stored for example in ei ^¬ nem memory. One of the stored in the memory transformation rules can then be selected in Depending ^¬ ness of the selected zoom level for the implementation of step d).

According to a further embodiment, the transformers ^¬ tions regulation is dynamically generated in step d). In this embodiment, the transformation rule is generated in each case dyna ^¬ mixed. Examples of this dynamic generation are explained in detail in the ^¬ consequences.

According to another embodiment, dynamically generating the transformation rule comprises:

Search and definition of common control points in neighboring ^¬ single frames of recorded individual images.

Generating control samples in neighboring frames of taken on ^¬ frames.

According to another embodiment, a laser projection is used to generate the control pattern.

The laser projection artificially generates control points in the individual images. These control points can then in turn be used for the appli ^¬ dung of stitching.

According to another embodiment, step e) comprises ^:

Assembling the received frames using the control points or the control pattern by means of stitching to the Panora ^¬ mabild.

According to a further embodiment, the steps b) to e) are always performed when, according to step a), the zoom level is newly selected.

Whenever the user re-selects or resets the zoom level according to step a), the execution of steps b) to e) is triggered. hereby In the present case, it is reacted dynamically and during operation to changes in the zoom level.

According to another embodiment, the maximum imaging power is a maximum resolution.

According to another embodiment, the camera has a wide-angle lens. The generated panoramic image is preferably output to a pilot of the airport via one or more screens.

According to a further embodiment, the panoramic image is formatted in a predetermined format and at least the formatted panoramic image via a communication network to an ATM system (ATM; Air Traffic Ma ^¬ management) to be transmitted.

Furthermore, a computer program product is proposed, which causes the execution of the method as explained above on a program-controlled device.

A computer program product, such as a computer program means, for example, can be provided or supplied as a storage medium, such as memory card, USB stick, CD-ROM, DVD, or in the form of a downloadable file from a server in a network. This can be done, for example, in a wireless communication network by the transmission of a ent ^¬ speaking file with the computer program product or program Computerpro ^¬ agent. Furthermore, a device for creating a panoramic image from Einzelbil ^¬ countries a number of cameras, it is proposed, comprising: a first unit for selecting a zoom level for receiving the A ^¬ zelbilder by the number of cameras,

a second unit for setting a configuration for the number of cameras in such a way that the respective camera is set up to record the respective individual image by means of the selected zoom level with maximum imaging power,

a third unit for controlling the number of cameras in such a way that they take up the frames with the selected zoom level and the set confi guration ^¬,

a fourth unit for determining a transformation rule for the selected zooming step for transforming the captured frames into the panoramic image, and

a fifth unit for transforming the recorded Einzelbil ^¬ the means of the determined transformation rule to the panoramic image.

The device can also be referred to as a panoramic camera or as a dynamic panora ^¬ camera.

The embodiments and features described for the proposed method apply accordingly to the proposed device.

The respective unit, for example the fifth unit, can be implemented in terms of hardware and / or software. In a hardwaretechni ^¬ rule implementation, the unit may be designed as a device or as part of a device, such as a computer or a microprocessor. In a software implementation, the unit may Computer program product as ^¬, as a function, as a routine, be formed as part of a program ^¬ codes or as an executable object. Other possible implementations of the invention include non-expli ^¬ zit Combinations of above or below with respect to the exporting approximately ^¬ Examples features or embodiments described. It will the skilled person also add individual aspects as improvements or additions to the respective basic form of the invention.

Further advantageous embodiments and aspects of the invention are the subject of the dependent claims as well as the execution ^¬ embodiments of the invention described below. Furthermore, the invention by way of Favor ^¬ th embodiments with reference will be explained in more detail to the accompanying figures. Fig. 1 is a schematic flow diagram showing an embodiment of a method for creating a panoramic image from Einzelbil ^¬ countries a number of cameras!

2 shows a schematic view of an exemplary embodiment of a device for producing a panoramic image from individual images of a

Number of cameras! and

Fig. 3 shows a schematic representation of an example of one through the

Device according to Fig. 2 recordable airport area.

In the figures, identical or functionally identical elements have the same reference numbers have been provided ^¬ Be, unless otherwise specified.

FIG. 1 shows a schematic flow diagram of an exemplary embodiment of a method for producing a panoramic image from individual images of a number of cameras.

The method of Fig. 1 comprises the following steps 101 to 105. In step 101, a zooming step for taking the frames by the number of cameras is selected. The number of cameras can also be one without limita ^¬ effect of the general public. Furthermore, the number of camera can also be a plurality of cameras. At step 101 an image angle of the camera for the recording of the image frame is preferably set at least ^¬. The fixing of the at least one image angle of the camera may comprise determining the horizontal image angle of the camera, the vertical image angle of the camera and the diagonal angle of the camera.

In step 102, a configuration for the number of cameras is so turned ^¬ provides that the respective camera for recording of each individual image with the selected zoom level is set at the maximum imaging performance. The maximum imaging performance is designed in particular as the maximum resolution of the camera.

For example, step 102 includes using a particular array of the number of cameras to capture the frames for the panoramic image. For example, may be formed as a plurality of Kame ^¬ ras, the number of cameras. Depending on the application and focusing a limited hours ^¬ th subset of said plurality of cameras can be used. These limited hours ^¬ te subset can be described as an array. Additionally or alternatively, step 102 may include aligning at least a portion of the number of cameras currently being used to create the panoramic image. In this orientation, a pan-tilt system, a rocker-tilt head, or a plurality of pan-tilt heads can be used.

Furthermore, the step 102 that a Strahlfüh ^¬ assurance system for the number of cameras is set may preferably comprise.

Alternatively or additionally, step 102 may further include adjusting one or more zoom optics for the cameras to provide the desired configuration for capturing the frames. In step 103, the frames are captured by the number of cameras with the selected zoom level and configuration set. In particular ^¬ sondere the individual images are absorbed such that they are present with a predetermined overlap. The predetermined overlap is meaningless explained fully for the joining of the individual images into a panoramic image as is the Fol ^¬ constricting detail using the example stitching.

In step 104, a transformation rule for the selected zoom level for transforming the captured frames to the panoramic image is determined. For example, in step 104, the transformation rule may be selected based on the selected zoom level from a plurality of predetermined transformation rules. The plurality of predetermined transformation rules can for example be created in advance and stored in a memory device.

Alternatively, the transformation rule can also be generated dynamically ^¬ to. The dynamic generation of the transformation procedure may comprise at ^¬ play that common control points are searched in neighboring ^¬ images of the captured frames and determines which are useful for subsequent stitching.

In step 105, the recorded individual images are transformed to the panoramic image by means of the determined transformation rule. The step 105 may include, in particular, that the images using the control points or the like Kontrollmuster- un ter ^¬ step 104 recorded individual ^¬ erläutert- be added to the panoramic image by stitching together. In particular, the steps 102 to 105 are always performed when the user has re-selected the zoom level according to step 101. With others Words, the application of step 101 triggers the execution of steps 102-105.

FIG. 2 shows a schematic view of an exemplary embodiment of a device 10 for producing a panoramic image from individual images of a number of cameras. The device 10 has the number of cameras or is coupled to the ^¬ sen cameras. The device 10 comprises a first unit 11, a second unit 12, a third unit 13, a fourth unit 14 and a fifth unit 15.

The first unit 11 is configured to select a zoom level for taking the frames by the number of cameras. Here, the first unit 11 can receive a user input of a user and select the zoom level in Depending ^¬ ness of the received user input.

The second unit 12 is adapted to set a configuration for the number of cameras such that each camera for receiving the time jewei ^¬ single image by the selected zoom level is set at the maximum imaging ^¬ performance.

The third unit 13 is adapted to the number of cameras in such a way ^¬ control that these take up the frames with the selected zoom level and turned ^¬ set configuration. The fourth unit 14 is configured to determine a transformation rule for the selected zoom level for transforming the captured frames into the panoramic image.

The fifth unit 15 is arranged to process the received frames with- means of form ^¬ the determined transformation rule to the panoramic image to be transplanted. Fig. 3 shows a schematic representation of an example by the pre ^¬ device 10 of Figure 2 recordable airport surface 20. On the Flughafenflä ^¬ che 20 are aircraft, such as airplanes. 1 -. 4. Without loss of generality, Figure 3 four aircraft located on the airport surface 20 1 - 4. Each of the aircraft 1 - 4 has a unique identi ^¬ fikations information, which in particular as a carrier indicator of the aircraft 1 - is formed. 4 The airport area 20 includes, for example, the apron, at least one runway and / or at least one taxiway of the airport.

The apparatus 10 is adapted to receive a panoramic image of the flight ^¬ harbor area twentieth The captured panoramic image can be provided to one pilot via one or more screens. The screens are arranged, for example, in a tower or airport tower of the airport. The captured panoramic image can also have a communications network to an ATM (ATM; Air Traffic Management) the übertra gen ^¬.

Although the present invention has been described with reference to exemplary embodiments, it can be modified in many ways.

LIST OF REFERENCE NUMBERS

1 plane

2 plane

3 plane

4 plane

10 device

11 first unit

12 second unit

13 third unit

14 fourth unit

15 fifth unit

20 airport area

101 process step

102 process step

103 process step

104 process step

105 process step

Claims

PATENT CLAIMS

1. Method for creating a panoramic image from individual images from a ^number of cameras, with:

a) selecting (101) a zoom level for recording the individual images using the number of cameras,

b) setting (102) a configuration for the number of cameras such that the respective camera is set up to record the respective individual image with the ^selected zoom level at maximum imaging power, c) recording (103) the individual images by the number of Cameras with the selected zoom level and the set configuration,

d) Determining (104) a transformation rule for the selected zoom level for transforming the recorded individual images into the panoramic ^image , and

e) transforming (105) the recorded individual images into the panoramic image using the ^determined transformation rule,

wherein step b) comprises:

Using a specific array of the number of cameras to ^record the individual images for the panoramic image,

Setting an alignment of at least part of the number of ^cameras ,

Setting a beam delivery system for the number of cameras, and/or

Setting at least one zoom lens for the number of cameras.

2. Method according to claim 1,

characterized,

that step a) includes:

Determining at least one image angle of the camera for recording the individual image.

3. Method according to claim 1, characterized,

that step a) includes:

Specify a horizontal camera angle of view, a vertical camera angle of view, and/or a diagonal camera angle of view for capturing the frame.

4. Method according to claim 1,

characterized,

that a pan-tilt system is used to adjust the alignment of at least part of the number of ^cameras .

5. Method according to one of claims 1 to 4,

characterized,

that in step c) the individual images are recorded by the individual cameras with a predetermined overlap of the individual images.

6. Method according to one of claims 1 to 5,

characterized,

that in step d) the transformation rule is selected from a plurality of predetermined transformation rules depending on the selected zoom level.

7. Method according to one of claims 1 to 5,

characterized,

that in step d) the transformation rule is generated dynamically.

8. Method according to claim 7,

characterized,

that the dynamic generation of the transformation rule includes:

Find and set common control points in neighboring ones

Individual images of the recorded individual images.

9. Method according to claim 7,

characterized,

that the dynamic generation of the transformation rule includes:

Generating control patterns in neighboring individual images of the recorded individual images.

10. Method according to claim 9,

characterized,

that a laser projection is used to generate the control patterns.

11. Method according to one of claims 8 to 10,

characterized,

that step e) includes:

Assembling the recorded individual images using the control points or the control patterns by stitching to form the panoramic ^image .

12. Method according to one of claims 1 to 11,

characterized,

that steps b) to e) are always carried out when the zoom level is reselected according to step a).

13. Method according to one of claims 1 to 12,

characterized,

that the maximum imaging performance is a maximum resolution.

14. Device (10) for creating a panoramic image from individual images from ^a number of cameras, with:

a first unit (ll) for selecting a zoom level for recording the individual images by the number of cameras,

a second unit (12) for setting a configuration for the ^number of cameras such that the respective camera is used to record the respective Single image is set up using the selected zoom level at maximum imaging performance,

a third unit (13) for controlling the number of cameras in such a way that they record the individual images with the selected zoom level and the set configuration,

a fourth unit (14) for determining a transformation rule for the selected zoom level for transforming the recorded individual ^images into the panoramic image, and

a fifth unit (15) for transforming the recorded individual images into the panoramic image using the determined transformation rule, the second unit (12) being set up to do this:

to use a specific array of the number of cameras to record the ^individual images for the panoramic image,

to set an alignment of at least part of the number of cameras,

to set a beam delivery system for the number of cameras, and/or

to set at least one zoom lens for the number of cameras.