FR2964203A1 - Image acquiring device e.g. camera, for use in photograph field of two-dimensional effects, has target determining unit determining part of panoramic, where viewing axis of device is passed through optical center of lens and sensor - Google Patents

Abstract

The device has a target determining unit (3) determining a part of a panoramic i.e. target (11). A control screen or electronic viewfinder i.e. screen (4), displays primary images (101, 102). The primary images are positioned on the screen such that representation of the target in all the primary images is placed appreciably at a same site of the screen, where the panoramic is set on landscape elements viewed from the device. A viewing axis of the device is passed through an optical center of a photographic lens (1) and center of a sensor. An independent claim is also included for a method for acquiring images.

Description

TECHNICAL FIELD The present invention relates to an image acquisition device for obtaining one or more images called "final image (s)" by making a progressive synthesis of multiple images. say "primary images" acquired successively. The problem to be solved is to acquire an image as clear as possible, even under difficult conditions (for example when using a camera with a telephoto lens or when you want to scan a part of the 10 panorama larger than that which can be grasped at once by the lens). The device of the present invention can also replace a spotting scope, or binoculars of high quality, while using only inexpensive components and without the need to mechanically stabilize the camera. PRIOR ART Numerous mechanical stabilization devices are known which aim at reducing the movement of the image acquisition device or of certain of its constituents. Numerous methods of digital image stabilization are also known, and in particular those which consist in correcting the framing of the images of a film in order to limit the differences in angle of view or framing of an image with the image. next. There are also many video surveillance systems that know how to track a moving object like a person or a vehicle. Such systems can use either a mobile camera or a high definition fixed camera. Only part of the acquired image is retained in order to center it on the object being monitored. Such devices are also used for automatic framing, for example for video conferencing. With all these devices, the operator - if not replaced by a robot - sees in real time in his viewfinder, or in the control screen that serves as a viewfinder, the image 30 seen through the objective, if necessary cropped to represent the targeted target in a stabilized way.

The present invention operates quite differently, by adding images taken successively. Panorama building methods are known by adding multiple images. In particular the document US6677981 (Bi) "Motion play-back of stil) pictures of MANCUSO MASSIMO [IT] and LUSINCHI EMMANUEL [US] published on January 13, 2004 and other subsequent documents which repeat this principle - W02010017476 (Al) In-camera panorama image stitching of LEVY NOAM [IL] - EP 2128868 (A2) Image pickup apparatus, image pickup method, playback control apparatus, playback control method, and program of HAYASHI NAOKI [] P] - US 2007160311 (A1) System and method for capturing adjacent images by utilizing a panorama of EDWARDS ERIC D [US]; FISHER CLAY [US] - US2005179787 (A1) Method for providing image alignment feedback for panorama (composite) images in WEBB CHRISTOPHER [US] digital cameras using edge detection These documents describe various similar techniques, with the goal of making great panoramas by moving the camera, but do not provide an effective way to to allow the operator to view a stable image in the viewfinder or control panel. DESCRIPTION OF THE INVENTION The objective of the present invention is that the image seen in the viewfinder is as stable as possible, and as clear as possible, even when the lens 1 of the device is oriented in another direction. that of the target 11. The operator determines the main part 11 of the so-called "final image" image 10 that he wishes to acquire. This main part 11 comprises a central part called "target". The final image 10 is progressively built up by steps corresponding to as many successive acquisitions of so-called "primary" images 101, 102 and following.

Whenever the objective 1 of the device targets all or part of the target 11, the part of the image whose acquisition is sought is added to the final image 10. What the operator sees in his viewfinder does not correspond not directly to the movements of the device and may even be very different.

Since the objective realizes several successive acquisitions and adds the images to each other, the resolution of the final image 10 may even be greater than that of the sensor 2 of the device. In a preferred version, the device comprises a means of searching through all said primary images the best pixels, those which are the sharpest, 10 to retain only them in the constitution of said final image 10. In an improved version, the objective and / or the sensor of the device are movable to automatically orient closer to the axis in which the target is located, and thus to correct the wrong position given to the device by the operator. The present invention is an image acquisition device - a so-called "objective" photographic lens, - a so-called "sensor" acquisition means 2 of images 101, 102 and following called "primary images", seen at through said lens 1, - a means 3 for determining a part of the so-called "target 11" panorama, said "target determination means", - a control screen or electronic viewfinder called "screen" 4 displaying all or part of the primary images 101, 102 and following, - means for comparing the axis of view of said image acquisition device with the direction in which said target 11 is located, 25 characterized by the fact that: comprises a means of communication to the operator of said difference between the line of sight of said image acquisition device and the direction in which said target 11 is located; and that said primary images 101, 102 and following are positioned on said 30 control screen 4 so that the representation of said target 11 in all of said primary images is placed substantially at the same location of said screen 4, it being understood that above - by "panorama" is meant all the elements of the landscape that can be seen from said acquisition device, and by "line of sight" of said image acquisition device the line passing through the optical center of said lens 1 and the center of said sensor 2. The present invention is also the method described as that implemented by the present device, in its various versions described. It differs from the known image stabilization devices in that the image that appears on the screen 4 is a synthesis or addition of primary images 101, 102 and following. It differs from panorama construction methods by adding multiple images in that the image appearing on the screen is stabilized. In fact, it constitutes a very significant advance for each of these two technologies. BRIEF DESCRIPTION OF THE DRAWINGS The invention will be better understood, and other objects, advantages and features thereof will become more clearly apparent from the following description, which is illustrated by the accompanying figures 1 and 2. Figure 1 is a perspective view of a device according to the invention according to a first embodiment of the invention. It allows to see the objective 1 and the control screen 4 which serves as a viewfinder. The means 3 for determining the target is constituted by the pointing device 3 which is displayed on the screen 4, which is here tactile. This pointing device makes it possible to modify the position on the screen 4 of the representation symbol 12 of the target 11. A cursor 6 makes it possible to determine the surface of said final image 10 with respect to the surface of a primary image 101 , the surface of said final image 10 being represented by the symbol 14. The last image acquired by the device is surrounded by a rectangle 15 which makes it possible to identify its position relative to that of said final image 10.

Figure 2 is a diagram describing what appears on the screen 4 in a second embodiment of the invention, simpler and therefore more ergonomic, but also powerful. The periphery 13 of the surface of said final image 10 coincides with the perimeter of the screen 4. The rectangle 15 identifies the position of the image hereinafter called "target image", which is the image which is the last image been acquired by the device. The cursor 6 makes it possible to determine the surface of said final image 10 on the screen 4. The hatched portion indicates the part of the final image 10 which has not yet been acquired by the device. DETAILED DESCRIPTION OF PARTICULAR EMBODIMENTS OF THE INVENTION The method implemented by the device according to the invention comprises a comparison of the line of sight of said image acquisition device with the direction in which a target is located. This comparison can be done by an analysis of the primary images 101, 102 and following before their display on the screen 4, so as - if necessary - to move them, to reframe them, to change their orientation with respect to vertically, enlarge or shrink. It can also be done more simply by calculating the vector of the displacement of the camera between two successive primary images, on the basis of the information provided by a known means of measuring the motion of the image acquisition device or of one of its components such as lens 1 or sensor 2. Such motion measuring devices are known and are also present on many small advanced telephones (often referred to as "Smartphones") They may comprise a gyroscope and / or a compass and / or an accelerometer An image acquisition device according to the invention thus advantageously comprises a known means for measuring the movement of at least one of the components of the image acquisition device. adapted to simple image acquisition devices, the method according to the invention is implemented in several successive steps, and for example: 1. a first step in which said primary images 101, 102 and following are positioned on said screen 4 real-time control from the information obtained from said motion measuring means, which gives an approximate result. It is possible to simplify the workload of the processor by not seeking at this step to modify the images to correct a rotation of the acquisition device between two successive primary images, or to equalize the light or the contrast between them, or to Correct perspective 2. and a second step in which images are compared to each other to be more finely combined, with only the complex task of correcting primary images at this time.

The first of these two steps can be started when the operator presses the shutter release button, and the second when he releases the button. The main goal sought during this analysis is to superimpose two primary images 101 and 102 as accurately as possible, despite the possible movement of the acquisition device in the space between the two corresponding shots. Each acquired image is moved on the screen 4 according to the vector of the displacement of the camera between two successive images, so that the representation of said target in all said primary images is placed substantially in the same location of said screen 4.

The two methods of calculating the camera displacement vector between two successive images described in the preceding paragraph - the measurement of the movement of a component of the image acquisition device and the comparative analysis of the images - are advantageously combined to improve the calculation precision of said vector. One can for example take into account the average, weighted or not, of the result obtained by each of the two methods. This analysis also makes it possible to suppress poor quality images, for example those rendered fuzzy by the movement of the device. A good way to know if an image is blurred is to compare the characteristics (brightness and color) of one of its pixels with those of neighboring pixels. The clearer the image, the more important the difference. We can compare several primary images with the same part of the panorama to see which is the clearest.

The images of the best quality can also be selected by measuring the speed of movement of the device with said means for measuring the movement of a component of the image acquisition device, to retain in the composition of the final image 10 than those that were acquired when the device had a weak movement. Calculating the average between two pixels corresponding to the same location of said final image 10 in several so-called primary images also makes it possible to identify and suppress the noise of the images acquired by the sensor 2. It is also possible to take advantage of this analysis to correct the said primary images with 10 of the many known image correction or enhancement software. In a basic embodiment, said control screen 4 displays a stack of images taken successively, the last image 109 hiding all or part of the previous ones. The final image 10 thus appears in the form of the combination of primary image portions 101, 102 and following. Advantageously, the latter image is surrounded by a colored frame 15 visible on the screen 4, to induce the operator to center the colored frame 15 around the target 11, by changing the orientation of the acquisition device. The device according to the invention can be used in real time and / or include means for recording the primary images 101, 102 and following, and / or the final image 10 and / or all the displayed images. successively on the screen 4. A procedure for using the device may comprise the successive capture of several so-called final images to constitute a film. In this case, it is recommended to acquire said primary images at a faster rate than the rate predicted for the diffusion of the film. This makes it possible to use several primary images to produce one of said final images. It is also recommended to use a sensor with a surface in pixels greater than the area in pixels of the video output format, in order to be able to crop these primary images without damage. An essential step is the determination of the target 11. In a first embodiment of the invention, said means 3 for determining the target 11 is a known pointing device, for example a mouse, a touchpad , a joystick, but it is preferred to use as a screen 4 a touch screen on which it is sufficient to include a set of 3 arrows for changing the position of the target with the finger, as shown in Figure 1.

In an automated version, said target 11 is determined by the average position of the line of sight of said image acquisition device during the last period. This average position is advantageously weighted so that the most recent positions of the device have more influence than older positions. This average position can be calculated using a known means for measuring the movement of at least one of the components of the image acquisition device, which is an additional reason for providing the acquisition device with such a means. . The position of the line of sight can also be determined by analyzing the primary images 101, 102 and following acquired by the device from a determined period. Hereinafter referred to as "target image" the image seen by the sensor 2 through the lens 1 when no correction is made to the sensor 2 or the objective 1. We will see later than in a version perfected, such a correction is possible. Advantageously, said target 11 is centered on the weighted barycentre of all or part of the pixels of said images targeted by the device for a given period, it being specified that the weighted barycentre is understood above to mean the barycenter of all the pixels. of all of said target images considered, each pixel of each image being assigned a mass determined according to any parameter. This parameter can be freely chosen by the user. For example, a pixel from a newer primary image 100 may be assigned a higher mass. The target 11 is thus automatically determined without user intervention, and it is sufficient for this operator to modify briefly the orientation of the device in a direction chosen to slightly move the target 11 in this direction. The target 11 can thus be changed automatically during the shooting, but its displacement is damped with respect to that of the acquisition device.

There is software to detect and even recognize faces. The pixels of a target image representing a face may for example be assigned a higher mass. The fact of aiming for a certain time this face can also suffice to determine it as being a significant part of the target 11. Advantageously, said target 11 is indicated by a symbol 12 superimposed on the image displayed by said screen. the case where a touch screen is used as screen 4, the operator can move this symbol 12 with the finger to move the target.

The symbol 12 of the target 11 may be a circle of color around the target 11 as shown in FIG. 1. It is also possible to use a different color or brightness to enable the operator to easily locate the target's location. that he must strive to aim. In an improved mode, the device according to the invention comprises means for determining the surface of said final image 10. Final image 10 may in fact have a larger or smaller surface area than that of a primary image. It is for example much larger to make panoramas and smaller or substantially equal to that of a primary image to achieve. This means for determining the registration may be a known pointing device, for example a mouse, a touch zone, or a joystick. When the screen 4 comprises a tactile surface used as a pointing device, known means may be used. to determine the framing with one or two fingers. A symbol 14 of said periphery 13 may be superimposed on said screen 4. Said periphery 13 may for example be seen on the screen in the form of a colored frame 14. The portion of the panorama not belonging to said final image 10 may be displayed blurry or darker. The central portion of said periphery 13 may also be represented in a symbolic form to allow the comparison of its position with the symbol 12 of the target 11.

It is possible to agree that the target is at the center of said periphery 13. In this case, the symbol 12 of the target 11 and the symbol 14 of the periphery 13 may be represented by a single symbol as in FIG. a drawing appearing on the screen 4. Advantageously, said periphery 13 of said final image 10 coincides with the perimeter of the screen 4 as shown in FIG.

The appreciation by the operator of the difference between the axis of view of said image acquisition device and the direction in which the target 11 is located may be the display of any graphic element, such as, for example a rectangle 15 which gives an indication of the position of the image that would be acquired by the device along the line of sight of said image acquisition device. This rectangle may for example be the perimeter of this image. It is also possible to display with a brightness or a slightly different color the portion of the screen 4 corresponding to the axis of view of the image acquisition device, or to use any other information mode, for example visual and / or sound without departing from the scope of the present invention.

An improvement of the present invention is that said screen 4 displays in a particular way the areas belonging to said final image 10 which are not yet acquired satisfactorily. For example, a fuzzy input can be considered unsatisfactorily entered. These parts may for example be colored in a particular way to encourage the operator to aim to eliminate this particular coloration. This method makes it possible to make panoramic images comprising many more pixels than the sensor 2 comprises, by scanning a large part of a panorama with a device according to the invention. In an improved embodiment, said control screen 4 represents a composite image consisting of the association of pixels originating from different primary images 101, 102 and following, the origin of each pixel displayed on said screen 4 being chosen in accordance with FIG. function of a predetermined rule. This predetermined rule may for example be to display preferably the sharpest pixels, or from primary images sharper than the others. One can also give the preference according to any other rule for example to privilege the pixels having not varied between two successive primary images, to eliminate objects moving in the panorama. A device according to the invention can be used to produce so-called final images of a resolution higher than that of the sensor 2. Said control screen 4 in this case advantageously has a resolution greater than that of the portion of the primary image 100 that he displays. The device may comprise software calculating the value of the intermediate pixels between the pixels of the primary images acquired by said sensor 2. A device according to the invention is in particular compatible with a digital zoom, and thus makes it possible to obtain sharper images. than those obtained by the simple use of such a function. A device according to the invention may of course be provided with two lenses 1a and 1b respectively associated with two sensors 2a and 2b to produce stereoscopic images. It may also be provided with a larger number of these lens / sensor assemblies to produce auto-stereoscopic images. In an improved embodiment, the device comprises a physical correction means 7 for framing. The term "physical correction of the framing" means a modification of the geometry of the image acquisition device having the effect of modifying the effective axis of the shooting relative to said axis of view of said acquisition device. picture. For example, it may be a means of modifying the orientation of the assembly constituted by said objective 1 and said sensor 2 with respect to the image acquisition device. It can also be a means of modifying the position of only one of these two elements, which are objective 1 and sensor 2. In a preferred version, objective 1 is fixed relative to the device of FIG. acquisition, while the sensor 2 is movable in the focal plane of said lens 1, and its position is determined by inverse piezoelectric effect actuators. Many physical image stabilization devices include such means. The objective pursued is to modify the frame so that the primary images acquired are as much as possible centered on the target 11 and aligned with the final image 10. The characteristics of the registration correction obtained by said means 7 are then taken into account. physical correction of framing, for determining a new target 11 by analyzing the portion of said primary images corresponding to said line of sight of said image acquisition device.

In this case, the rectangle 15 advantageously makes it possible to identify the position not of the last acquired primary image, but that which would be acquired by the device along the axis of view of said image acquisition device and the zoom coefficient used before any physical correction of the framing. Without departing from the scope of the invention, the operator can be replaced by a robot that changes the position and / or orientation of the device - or the parameters of the physical correction of the framing described above - according to the information provided by the device (eg: possible presence of the target 11 on a primary image and, if appropriate, its position in a primary image 100 - unvested or poor quality zones that must be acquired again, appearance of a object in motion, recognition of a face or hot zone, etc.). The combination of physical framing correction and automation opens up considerable applications in the field of automatic interviewing, videoconferencing and surveillance. The use of a device according to the invention can be advantageously completed by a postproduction sequence, consisting for example of recomposing the final image 1 with other primary images than those selected automatically, for example to choose an expression of a portrait, or a particular moment of a sporting event. The corresponding software is advantageously implemented by the device according to the invention itself.

The applications of the present invention are generally in the field of photography and cinema in 2D and 3D, in particular of still or moving shots under difficult conditions (surgery and functional exploration for example), automatic framing (videoconferences, interviews, surveillance, etc.), etc., but also in the fields of remote viewing (binoculars and telescopes with or without recording acquired or final images).

Claims (2)

REVENDICATIONS1. An image acquisition device comprising: a photographic lens 1 called "objective", an acquisition means 2 called "sensor" of images 101, 102 and following called "primary images", seen through said objective 1, a means 3 for determining a part of the so-called "target 11" panorama, called "means for determining the target", a control screen or electronic viewfinder called "screen" 4 displaying all or part of the primary images. 101, 102 and following, - a means for comparing the axis of view of said image acquisition device with the direction in which said target 11 is located, characterized in that: - it comprises a communication means to the operator of said difference between the line of sight of said image acquisition device and the direction in which said target 11 is located, and that said primary images 101, 102 and following are positioned on said screen 4 of control so that the representation of ladit e target 11 in all said primary images is placed substantially at the same location of said screen 4, it being understood that it is understood above: - by "panorama" all the elements of the landscape likely to be seen from said device d acquisition, - and by "line of sight" of said image acquisition device the line passing through the optical center of said lens 1 and by the center of said sensor 2. An image acquisition device according to claim 1 characterized in that it comprises a means for measuring the movement of at least one of its components.A device for image acquisition according to claim 1. characterized in that said target 11 is determined by analyzing the position of the line of sight of said image acquisition device during the last period 4. An image acquisition device according to claim 2 characterized in that said primary images 101, 102 and subsequent are positioned on said control screen 4 - at least initially - using the information obtained from said motion measuring means. 5. An image acquisition device according to claim 1 characterized in that it comprises a means 7 for physical correction of framing, it being specified that the term "physical correction of the framing" a modification of the geometry of the image acquisition device having the effect of modifying the effective axis of the shooting relative to said line of sight of said image acquisition device. An image acquisition device according to claim 1 characterized in that it comprises a means 6 for determining the surface 13 of said final image 10 with respect to the surface of a primary image 101. 7. Device for image acquisition according to claim 5 characterized in that said periphery 13 of said final image 10 is substantially coincident with the perimeter of the screen 4. 8. Image acquisition device according to claim 5 characterized by the fact said screen 4 signals in a particular way the zones belonging to said periphery 13 not yet satisfactorily acquired. 9. An image acquisition device according to claim 1 characterized in that said control screen 4 represents a composite image consisting of the association of pixels from different primary images 101, 102 and following. 10. An image acquisition method according to any preceding claim characterized in that - said target determination means is used to determine part of the panorama.les said primary images 101, 102 and following are positioned on said control screen 4 so that the representation of said target 11 in all said primary images is placed substantially at the same location of said screen 4.