FR2983996A1 - Method for displaying portion of image on screen of display device in cell phone, involves displaying extracted image on screen of display device, and determining crop window based on position of observer relative to screen - Google Patents

Abstract

The method involves determining a crop window (12) in an image, and extracting the image (14) defined by the crop window. The extracted image is displayed (16) on a screen of a display device. The crop window is determined and analyzed (10) based on a position of an observer relative to the screen. A displacement of the observer in a plane parallel to the screen of the display device is compared to a reference position. The reference position is located in the image, and a part of a face of the observer is determined based on the crop window in the image. An independent claim is also included for a display device.

Description

FIELD OF THE DISCLOSURE 1. Field of the Invention The invention relates to the general field of image cropping.

The invention more particularly relates to a device and a method for displaying a cropped image. 2. State of the art For an application to broadcast video content on a display device having a small screen, eg a mobile phone, it is necessary to generate a small version of each image of the video in order to display them on the small screen. It is known to obtain such small images from downsampling the initial images to the size of the display screen. Such a solution is not satisfactory because some objects visually interesting for the observer, eg a ball in the case of a football match, then have a very small size in the subsampled image. It is also known to obtain such small images from cropping the original images to the size of the display screen. Referring to Figure 1, it is known to crop an initial image to determine a cropping window in this image, for example from a model of visual attention. The portion of the initial image delimited by the cropping window is then extracted for display on the small screen. The advantage of such a method is to take into account the content of the initial image to determine the cropping window. The cropping window is positioned in the initial image so as to keep in this window the areas of the image that are most visually interesting for an observer. The image to be displayed can also be obtained by simultaneously applying the cropping and downsampling processes, in order to make a compromise between these two solutions. In the remainder of this document the term cropping may also involve resampling when the number of pixels of the cropping window is different from the number of pixels of the display device.

Figure 2 shows a high resolution image to be displayed on a small screen. In this figure, a cropping window is shown in solid white line. The position of this cropping window in the image is for example determined by a method such as that described in the patent application EP1748385. The position of this cropping window therefore depends on the analysis of the content of the image. Visual attention models are statistical models that take into account, for example, movement in images and contrast. This information is used to determine the areas of interest in the image, i.e., which attract the observer's eye, in this case the football players and the ball. The cropped image depends only on the analysis of the content of the image, eg the visual attention model. The content of the image thus cropped and displayed on the screen is identical regardless of the observer and his preferences. SUMMARY OF THE INVENTION The object of the invention is to overcome at least one of the drawbacks of the prior art. The invention relates to a method for displaying at least part of an image on a screen comprising the following steps: determining a window of reframing in the image which depends on the position of an observer relative to the 'screen; extracting from the image the portion delimited by the cropping window; and - display the extracted image portion on the screen. According to a first embodiment, determining the cropping window comprises: determining a displacement of the observer in a plane parallel to the screen with respect to a reference position; and - determining the position of the cropping window in the image from the displacement. Advantageously, determining a displacement of the observer in a plane parallel to the screen with respect to a reference position comprises: - acquiring at least one image of the observer's face; and - determining the position of at least part of the face in the acquired image.

According to a particular characteristic, the at least part of the face comprises the eyes of the observer. Advantageously, determining the position of the cropping window in the image from the displacement of the observer comprises determining a first cropping window and moving the first cropping window k times the displacement of the observer where k is a parameter of gain. According to another embodiment, determining the window of reframing in the image comprises: determining a distance of the observer relative to the screen; and - determining the size of the cropping window in the image from the distance of the observer from the screen. According to one particular aspect of the invention, determining the size of the cropping window in the image from the distance of the observer from the screen comprises: determining a second cropping window; and - determining in the second cropping window, a centered reframing window of size Ox (1-k'.FaceZ) where k 'is a gain parameter. Advantageously, the method is used in an image processing device. The invention further relates to a device for displaying at least a portion of an image on a screen comprising: - processing means for determining a cropping window in the image; means for extracting from the image the portion delimited by the cropping window; and display means for displaying the image portion extracted on the screen. The processing means determine the window of reframing in the image from the position of an observer relative to the screen. Advantageously, the processing means comprise: means for determining a displacement of the observer in a plane parallel to the screen with respect to a reference position; and - means for determining the position of the cropping window in the image from the displacement. According to a particular aspect of the invention, the means for determining a displacement of the observer in a plane parallel to the screen with respect to a reference position comprise: means for acquiring at least one image of the face of the observer; and means for determining the position of at least part of the face in the acquired image.

According to a particular characteristic of the invention, the at least part of the face comprises the eyes of the observer. Advantageously, the means for determining the position of the cropping window in the image from the displacement of the observer comprise means for determining a first cropping window and means for moving the first cropping window k times the displacement. of the observer where k is a gain parameter. Advantageously, the processing means for determining the cropping window in the image comprise: means for determining a distance from the observer with respect to the screen; and - means for determining the size of the cropping window in the image from the distance of the observer from the screen. Advantageously, the means for determining the size of the cropping window in the image from the distance of the observer from the screen comprise: means for determining a second cropping window; and means for determining, in the second cropping window, a centered reframing window of size Ox (1-k'.FaceZ) where k 'is a gain parameter. 4. Lists of the Figures The invention will be better understood and illustrated by means of examples of advantageous embodiments and implementations, in no way limiting, with reference to the appended figures in which: FIG. 1 illustrates a display method according to the state of the art; FIG. 2 shows a high resolution image on which is represented a cropping window determined according to the prior art; - Figure 3 shows a large image on which is represented a cropping window can be moved, according to the principles of the invention, within a wider window; - Figure 4 shows large images on which are represented cropping windows moved according to the principles of the invention; FIG. 5 illustrates a display method according to the invention; FIGS. 6 to 8 illustrate the determination of the position of an observer relative to a screen; FIGS. 9 to 11 illustrate the reframing of an image as a function of the displacement of an observer in a plane parallel to the screen; FIG. 12 illustrates the reframing of an image as a function of the longitudinal displacement of an observer with respect to the screen (along the axis of the distance to the screen); and - Figure 13 illustrates a display device according to the invention. 5. DETAILED DESCRIPTION OF THE INVENTION The invention relates to a method and a device for displaying a cropped image from an initial image in which the position of a cropping window is controlled by the observer. The method and the device according to the invention are also adapted to display a video of which each image is reframed according to the principles of the invention. The control of the reframing by the observer behaves as if the image was observed through a window. When the observer is placed facing the screen in an initial position or reference position, eg the observer is centered with respect to the screen, the cropping window corresponds to an initial reframing window. The initial position of the observer is that from which are determined displacements of the latter relative to the screen. According to one variant, this initial position of the observer is defined by a user command. For example, the observer is placed in a certain position relative to the screen and then validates this position as being the initial position eg by clicking the key of a keyboard / remote control. The initial reframing window is for example derived from the analysis of the content of the image, eg using a visual attention model. According to one variant, this initial cropping window is centered on the larger image. This last variant does not require any analysis of the content of the image but is defined a priori. The invention is not limited by the way in which this initial cropping window is defined.

In the same way as by shifting to the right with respect to a window, it is possible to reveal what is more left behind this window, a displacement to the right of the observer with respect to the screen moves the window cropping to the left and thus reveals the contents of the larger image that is on the left. This mechanism can be applied in each direction. According to the invention, the observer therefore moves the cropping window around the initial window in a very intuitive manner, as when looking at a scene through a window. Figure 3 shows a high resolution image to be displayed on a small screen. In this figure, the initial crop window is shown in solid white line. The user can by moving in front of the screen or by turning his mobile screen because it is here the relative position of the observer with respect to the screen, move this window of reframing inside a wider window, indicated in white dashed lines. The window indicated in white dotted lines may correspond to the whole of the original image. In this case the observer can move in all the content transmitted in high resolution. However, the high resolution content must be transmitted in full to the display device, involving a large bandwidth, for a final display in low resolution. The white dashed window may be constrained by a first crop resulting for example from a model of visual attention. In this case, the observer can move in the video content only around a position resulting from this model of visual attention, but only the white dashed window must be transmitted to the display device. In this embodiment, we would have then, a first "wide" refraction carried out before transmission for example by a visual attention model, a transmission of the dashed window, followed by a display resulting from the reframing according to the invention. invention which is driven by the user.

Figure 4 illustrates in more detail the principle of the invention. In the top left of Figure 4 is shown the initial cropping window. In the state of the art this part of the image is that which is displayed on the screen of small size whatever the relative position of the observer with respect to this screen. According to the invention and as illustrated at the top right of Figure 2 when the user moves up the image, the cropping window is itself moved downwards, revealing the content being more below. According to the invention and as illustrated at the bottom left of Figure 42 when the user moves to the right of the screen, the cropping window is itself moved to the left, thus revealing the content being more to the left. In these examples, the cropping window is moved when the observer moves in a plane parallel to the screen. According to an alternative embodiment, when the observer approaches the screen, the cropping window can be enlarged in the same way that the field of view is wider when approaching a window. Since the cropping window would have a pixel size different from the size of the final screen, this cropping is associated with resampling. The approximate distance information can be obtained for example by considering the average interocular distance.

The method of reframing according to the invention takes into account, to a certain extent, the preference of the user. Two observers, with the same image content, can focus on different areas thanks to the invention. Cropping is improved since it takes into account the observer. FIG. 5 represents a display method according to the invention.

In a step 10, the position of the observer relative to the screen he is watching is determined. The invention is not limited by the manner in which this position is determined. This position is then used in a step 12 to determine in the initial image a cropping window. During a step 14, the portion of the initial image delimited by the cropping window resulting from step 12 is extracted and then displayed during a step 16 on a screen. Since the cropping window would have a pixel size different from the size of the final screen, this cropping is associated with resampling. According to a particular embodiment, the position of the observer relative to the screen is determined using an image acquisition module such as a camera or webcam. The acquisition module is preferably positioned on the screen and films the observer. The observer's images are then analyzed to determine the observer's position relative to the screen. As a simple illustrative example, the images of the observer are analyzed by an eye tracking method or a face detection method. The eye tracking method makes it possible to detect the observer's eyes and their position in the acquired image. On the other hand, the eye tracking method does not consider either the direction of the gaze or the area of the screen observed. Such an eye tracking method is for example implemented by image processing, on images from for example a camera such as a webcam. Alternatively, the eye tracking method is replaced by a face detection method. Face detection is less expensive in terms of signal processing than eye detection. Face detection methods are known to those skilled in the art of image processing and are not further detailed. Indeed, the invention is in no way limited by the face detection method used. Some image processing libraries provide face detection functions (eg OpenCV). Other methods may be used if they make it possible to determine a position of the observer relative to the screen. Figures 6 to 8 illustrate a particular embodiment. The left side of the figures represents an observer in front of a screen, eg a screen of a digital tablet above which a camera is positioned. The camera can be an integral part of the screen or external to it. The camera films the observer watching the screen. The images of the observer acquired by the camera are shown on the right-hand part of FIGS. 6 to 8. In FIG. 6, the observer is in front of the screen, his face is thus centered in the acquired image. In Figure 7, the observer has moved to the left of the screen and in Figure 8 on the right of the screen. The images acquired by the camera are analyzed by eye tracking or face detection. In FIGS. 6 to 8, a face detector is used to determine the face position of the user's face in the acquired image and in particular its displacement along the x axis between two extreme positions -Dmax and + Dmax. This information is then used to control the cropping of the initial image. Figure 9 shows an input image I of horizontal dimension lx pixels. This image should be displayed on a lower resolution screen of horizontal size Ox pixels. For this purpose, the initial image is cropped.

Cropping, for example based on a model of visual attention, provides the position of the origin (Cropx, Cropy) of the cropped image in image I. When the crop is not controlled by the position of the observer or when the observer is placed in front of the screen, the pixels of the image I of Cropx position to Cropx + Ox-1 are displayed on the screen of size Ox as illustrated on the right part of Figure 9. The cropping window being bounded by the input image I, 0 Cropx Ix-Ox. Referring to FIG. 10, the position of the face observer is used to change the position of the crop window, as if the observer were watching his video content through a window. A new crop window position Cropx 'is determined from the position of the user. By way of example, Cropx '= f (Cropx, Facex). In this case, the pixels of the image I of position Cropx' to Cropx '+ Ox-1 are displayed on the screen of size Ox. The cropping window is bounded by the input image I, 0 Cropx 'Ix-Ox. The function f is for example a linear function. By way of example, f is defined such that Cropx '= Cropx + k.Facex. In this case, the pixels of the Cropx + k.Facex position I image at Cropx + k.Facex + Ox-1 are displayed on the Ox size screen. The gain parameter k makes it possible to define the amplitude (and the direction) of the displacement of the reframing window as a function of the amplitude of the displacement of the observer. Since the position of the cropping window is fixed by Cropx '= Cropx + k.Facex, a displacement of the observer's face pixels in the image acquired by the camera associated with the screen (FIGS. 7 and 8) results in a moving k.Facex pixels from the crop window. In the case where this camera would be replaced by a device giving the position of the observer relative to the screen for example in cm, k would be expressed in pixels per cm. Of course, the same method can be applied along the y axis for a movement of the observer from top to bottom or vis and versa. FIG. 11 illustrates the cropping method described above in connection with FIGS. 6 to 8 but viewed from above as if the image were viewed through an imaginary window placed at the screen. An image of size Ox, corresponding to the resolution of the screen is extracted from the original image of size lx. The position of the cropping window is controlled by the position of the observer O in front of the screen. Thus, a displacement of the observer of a distance Face implies a displacement of this cropping window in the image I of a value k. Facex. Figure 12 illustrates another embodiment. According to this embodiment, the cropping is controlled by the distance of the user to the screen. Thus, a displacement of the observer along the z axis causes a modification of the initial reframing window. The distance of the user to the screen is evaluated accurately by triangulation using two cameras. According to a variant, it is evaluated by considering the size of the face detected in front of the screen, or the interocular distance. When the observer moves along the z-axis, the number of pixels from the initial image I to be displayed on the screen of size Ox varies according to the distance of the observer on the screen. This amounts to zooming in the original image. The relation linking the number of pixels from the original image to be displayed to the observer's distance variation is g (Ox, Face). By way of example, g is defined as follows: g (Ox, Face) = 25 Ox (1-k'.Face7) with k '<0. Indeed, when k 'is positive, a distance from the observer implies a zoom in the original content on the screen at lower resolution. But it is better because more intuitive than a rapprochement of observer induces a zoom in the original content (we get closer to see better). The gain parameter k 'makes it possible to set the amplitude (and the direction) of the zoom in the reframing window according to the amplitude of displacement of the observer. When the distance of the observer from the screen corresponds to the initial position Face = 0, the window for reframing Ox pixels is to be displayed on the screen size Ox. Otherwise, Ox. (1- k'.FaceZ) pixels are displayed on the Ox size screen. In FIG. 12, the reframing window when the observer has moved from Face, is therefore a window of size Ox (1-k'.FaceZ) pixels which is centered in the window. initial cropping. In this case, the initial reframing window is either that obtained when the user is in the initial position (FIG. 12 on the left), or the reframing window obtained according to the first embodiment after a displacement in a plane parallel to screen (Figure 11 right). FIG. 13 diagrammatically illustrates an exemplary embodiment 10 of a display device 2 according to an example of a particular and nonlimiting implementation of the invention. The display device 2 is for example a tablet, a PDA (acronym for "personal digital assistant") or a mobile phone. The display device 2 comprises, interconnected by a bus 24 of addresses and data: a microprocessor 21 (or CPU); a non-volatile ROM type memory (of the "Read Only Memory") 22; a random access memory (Random Access Memory) 23; One or more I / O (of the "Input / Output" or "Input / Output") interfaces 25, such as for example a keyboard, a mouse; and - a power supply 26. The display device 2 also comprises a camera, eg webcam, which makes it possible to obtain images of the observer or any other position sensor of the observer. This camera can also be external to the display device is connected to it by a cable. The device 2 also comprises display means such as a screen for displaying the cropped images. It will be observed that the word "register" used in the description of the memories 22 and 23 designates in each of the memories mentioned, as well a memory area of low capacity (a few binary data) and a memory zone of large capacity (allowing store an entire program or all or part of the representative data of a received audio / video service). The algorithms implementing the steps of the method according to the invention and described above are stored in the ROM 22. The RAM 23 includes in particular in a register, the operating program of the microprocessor 21 loaded at power up of the display device. At power up, the microprocessor 21 loads and executes the instructions of these algorithms. The function of one of the input / output interfaces 25 is notably to receive the initial images to be cropped and to transmit the cropped images according to the invention. Cropped images are stored in RAM. According to variants, cropping devices compatible with the invention are implemented in a purely hardware ("hardware") embodiment, for example in the form of a dedicated component (for example in an ASIC or FPGA or VLSI). (respectively "Application Specific Integrated Circuit" in English, meaning "Specific Application-Specific Integrated Circuit", "Field-Programmable Gate Array" in English, meaning "In-Situ Programmable Gate Network", "Very Large Scale Integration" In English, meaning "very large-scale integration") or of several electronic components integrated in a device or in the form of a mixture of hardware elements and software elements ("software" in English).

Of course, the invention is not limited to the embodiments mentioned above. In particular, those skilled in the art can make any variant in the exposed embodiments and combine them to benefit from their various advantages. In particular, the method according to the invention can combine the two embodiments described with reference to FIGS. 9 to 12, taking into account both a displacement in a plane parallel to the screen and a displacement perpendicular to the screen to get closer to it or move away from it.

Claims (15)

REVENDICATIONS1. A method of displaying at least a portion of an image on a screen comprising the steps of: - determining (10, 12) a cropping window in the image; extracting (14) from said image the portion delimited by said cropping window; displaying (16) said image portion extracted on said screen; Characterized in that the determination (10, 12) of said cropping window in the image depends on the position of an observer with respect to said screen. The method of claim 1, wherein determining (10, 12) said cropping window comprises: - determining (10) a displacement of the observer in a plane parallel to the screen with respect to a reference position; and - determining (12) the position of said cropping window in said image from said displacement. 20 The method of claim 2, wherein determining a movement of the observer in a plane parallel to the screen with respect to a reference position comprises: - acquiring at least one image of the observer's face; and determining the position of at least a portion of the face in said acquired image. The method of claim 3, wherein said at least a portion of said face comprises the eyes of said observer. 30 The method according to one of claims 2 to 4, wherein determining the position of said cropping window in said image from said observer displacement comprises determining a first cropping window and moving said first cropping window of k times the displacement of said observer where k is a gain parameter. The method of one of claims 1 to 5, wherein determining said cropping window in said image comprises: - determining a FaceZ distance of the observer from said screen; and - determining the size of said cropping window in said image from said observer's distance from said screen. The method of claim 6, wherein determining the size of said cropping window in said image from said observer's FaceZ distance from said horizontal resolution screen Ox comprises: - determining a second cropping window; and - determining in said second cropping window, a centered reframing window of size Ox (1-k'.FaceZ) where k 'is a gain parameter. 8. Method according to one of claims 1 to 7, said method being used in an image processing device. 9. Apparatus for displaying at least part of an image on a screen comprising: processing means for determining a cropping window in the image; means for extracting from said image the portion delimited by said cropping window; display means for displaying said extracted image part on said screen; characterized in that the processing means determines said cropping window in the image from the position of an observer with respect to said screen. 10. Device according to claim 9, wherein said processing means comprise: means for determining a displacement of the observer in a plane parallel to the screen with respect to a reference position; and- means for determining the position of said cropping window in said image from said displacement. 11. Device according to claim 10, wherein the means for determining a displacement of the observer in a plane parallel to the screen with respect to a reference position comprise: means for acquiring at least one image of the face of the 'observer; and means for determining the position of at least part of the face in said acquired image. The device of claim 11, wherein said at least a portion of said face comprises the eyes of said observer. 13. Device according to one of claims 10 to 12, wherein the means for determining the position of said cropping window in said image from said displacement of the observer comprises means for determining a first cropping window and means to move said first cropping window k times the displacement of said observer where k is a gain parameter. 14. Device according to one of claims 9 to 13, wherein the processing means for determining said cropping window in said image comprise: means for determining a distance FaceZ of the observer relative to said screen; and - means for determining the size of said cropping window in said image from said observer's distance from said screen. The apparatus of claim 14, wherein the means for determining the size of said cropping window in said image from said observer's distance from said horizontal resolution screen Ox comprises: - means for determining a second cropping window; and means for determining, in said second cropping window, a centered reframing window of size Ox (1-k'.FaceZ) where k 'is a gain parameter. 10