WO2016005252A1

WO2016005252A1 - Method and device for image extraction from a video

Info

Publication number: WO2016005252A1
Application number: PCT/EP2015/065031
Authority: WO
Inventors: Lorenzo Sorgi; Gwenael Dumusc
Original assignee: Thomson Licensing SAS
Current assignee: Thomson Licensing SAS
Priority date: 2014-07-10
Filing date: 2015-07-01
Publication date: 2016-01-14
Anticipated expiration: 2017-01-10
Also published as: EP2966594A1

Abstract

A method for extracting images from a video and a device (20) configured to perform the method are described. An acquiring unit (21) of the device acquires (10) a video including a plurality of images in which a reference object is captured. An analysing unit (22) analyses (11) features of the reference object in the images of the video, segments (14) the video into clusters, and determine one key image for each cluster based on the analysis of the features. An extracting unit (23) then extracts (12) the key images from the video.

Description

METHOD AND DEVICE FOR IMAGE EXTRACTION FROM A VIDEO

TECHNICAL FIELD A method and a device for image extraction from a video are proposed. The extracted images provide an image dataset required for camera calibration, which is a field that collects algorithms and technologies aimed at the

characterization of a mathematical projection model for the image formation process of a camera.

BACKGROUND

A camera calibration process in general consists of

radiometrical and geometrical stages. The radiometrical calibration is to acquire information on how a camera distorts the luminous properties of a scene, e.g. color and luminance. It plays a fundamental and important role for applications such as astronomical imaging and color

processing, but is generally bypassed in most of typical 3D vision applications. The geometrical calibration leads to the estimation of a suitable model for the image formation geometry, namely the camera projection and the optical distortion, and is crucial for most 3D vision applications.

Calibration techniques can be generally classified into self-calibration and object-based calibration methods.

Self-calibration attempts to infer the camera model from the transformation of the image appearance under the action of a rigid motion, while object-based calibration relies on a certain a-priori known calibration object [I, II] .

The most common object-based techniques assume the

availability of images of a specific object, which has a known shape and can be easily detected. The calibration object is captured by a camera from different points of view. This prerequisite eases the collection of an error- free set of correspondences between 3D points and 2D image projections for a subsequent camera calibration procedure

[III] .

The pioneer technique in this field was proposed by Tsai

[IV] and has been followed by a large number of other algorithms, which differentiate from each other for attributes such as the geometry of the calibration object, the features represented on the object surface, the number of required images, the constraints on camera motion and the estimated camera projection model [V] . Among the huge proliferation of calibration algorithms, Zhang's approach [VI] deserves a remark as it has become the basis of many open-source as well as commercial calibration tools.

Meanwhile, with a large number of standard solutions, several tools addressing specific problems like fisheye lenses and omnidirectional imaging systems and camera clusters calibration are also available.

Despite the large number of solutions, not much attention has been given to the preliminary processing task of gathering an optimal calibration image dataset from a video, which is actually crucial for and can influence the result and accuracy of the expected camera calibration. In most of the techniques, the task of collecting the image dataset is committed to end users. However, an end user is normally without specific knowledge about how to acquire an optimal image dataset for camera calibration. In addition, many low cost cameras are equipped merely with a minimal interface, sometimes even without a live-view feature, which makes it impossible for a user to control the images actually captured. The manual capture and decision for the image dataset is a tedious and time consuming task

especially when a minimal interface is used. To acquire an optimal calibration image dataset, one essential step is to identify the reference object captured in the given video or images. It is sometimes burdensome and tedious for an operator to perform a manual

identification. For example, when a checkerboard is used as the reference object, which is common because of its easily detectable a-priori known geometry, the number of pixel points to be identified is tremendous and the

identification result might be poor and thus unreliable. An alternative to the manual identification is a

calibration tool equipped with a fully automatic function for checkerboard corner detection or a light-assisted one, which is nowadays commonly available on the market, e.g. Camera Calibration Toolbox for Matlab, OcamCalib,

EasyCamCalib, Omnidirectional Calibration Toolbox, etc. Nevertheless, most of the tools are limited to certain conditions and are not feasible in some cases, e.g. when the calibration object is only partially visible in the images and when highly distorted images are used. Since wide angle lenses introduce a severe nonlinear distortion into the image formation process, all those techniques that assume a roughly regular geometry of the checkerboard pattern would be inevitably infeasible for such cases, especially when a linear homography mapping between the object space and the image space is used for an approximate location of the checker corners. The high curvature of the checkerboard structure on the image indeed makes the corner prediction unreliable. Another problem is the partial visibility of the

calibration object. A strong calibration of a camera system requires a well-spread distribution of the 2D/3D

correspondences, namely the source 3D points should span a volume with sufficient extension in depth with respect to the camera location, and the corresponding 2D image

projections should cover with approximately equal density the whole retinal plane. This is an essential condition to achieve an accurate estimation of the calibration

parameters, which requires a detection system able to cope with partially occluded patterns. In these cases, most of the calibration tools do not support a human interaction for a removal of false detections or for the integration of supplementary data for missed detections.

SUMMARY

Therefore, it is an objective to propose an improved solution for extracting a calibration image dataset from a video. More specifically, the proposed method and device is for automatic image extraction from a video.

According to the one embodiment, a method for extracting images from a video is introduced. The video includes a plurality of images in which a reference object is

captured. The method comprises: acquiring the video;

analyzing features of the reference object in the images of the video; segmenting the video into clusters, each cluster including at least one image; determining one key image for each cluster based on the analysis of the features; and extracting the key images from the video.

In one embodiment, segmenting the video into clusters includes using the techniques of K-means clustering. In one embodiment, the reference object includes a

reference checker of a planar checkerboard, and the

features of the reference object include one or more of velocity, orientation and area of the reference checker, and a representing nodal point closest to the geometric center of the reference checker.

In one embodiment, in particular, analyzing the features includes labeling the images of the video in a

tridimensional space defined according to the orientation, area and closest representing nodal point of the reference checker in the images. Segmenting the video into clusters includes using the techniques of K-means clustering on the labeled images in the tridimensional space. Determining one key image for each cluster includes selecting the image in which the reference checker has a lowest velocity for each cluster . In one embodiment, the method further comprises identifying the reference object in the video. Preferably, identifying the reference object includes using Gauss-Laguerre Circular Harmonic Filters. Accordingly, a device configured to extract images from a video is introduced. The device comprises: an acquiring unit configured to acquire the video; an analyzing unit configured to analyze features of the reference object in the images of the video, to segment the video into clusters and to determine one key image for each cluster based on analysis of the features; and an extracting unit configured to extract the key images from the video. In one embodiment, the device further comprises a user input unit configured to receive a user input for

identifying the reference object. Also, a computer readable storage medium has stored therein instructions enabling extracting images from a video, which when executed by a computer, cause the computer to: acquire a video including a plurality of images in which a

reference object is captured; analyze features of the reference object in the images of the video; segment the video into clusters, each cluster including at least one image; determine one key image for each cluster based on the analysis of the features; and extracting the key images from the video.

The embodiments of the proposed method provide an improved solution to automatically acquire a calibration image dataset. Since the method can be performed automatically in a single device, it is time saving and can provide a higher suitability of the extracted image dataset. The images are extracted according to some optimality criteria.

BRIEF DESCRIPTION OF THE DRAWINGS For a better understanding the proposed solution shall now be explained in more detail in the following description with reference to the figures. It is understood that the solutions are not limited to this disclosed exemplary embodiments and that specified features can also

expediently be combined and/or modified without departing from the scope of the present solutions as defined in the appended claims. In the figures: Fig. 1 is a flow chart illustrating one preferred embodiment of the proposed image extraction method . Fig. 2 is a flow chart illustrating other preferred embodiments of the image extraction method.

Fig. 3 is a flow chart illustrating an exemplary

tracking of a reference object according to one exemplary embodiment of the method.

Fig. 4 shows the tracking results obtained from the exemplary embodiment of the method shown in Fig. 3.

Fig. 5 shows one exemplary measurement for a grid of nodal points implemented in one exemplary embodiment of the method. Figs. 6-7 show two implementation examples of the

proposed image extraction method.

Fig. 8 is a flow chart illustrating the checkerboard detection of one exemplary embodiment of the image extraction method.

Figs. 9-11 show one implementation example of one

embodiment of the proposed method for automatic identification of a calibration object.

Fig. 12 shows an exemplary result obtained from the

implementation example shown in Figs. 9-11. Fig. 13 is a schematic diagram illustrating a device configured to perform the image extraction method . DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Fig. 1 schematically illustrates a preferred embodiment of the proposed method for extracting images from a video. The method comprises: acquiring 10 a video including a

plurality of images in which a reference object is

captured; analyzing 11 visual features of the reference object in the images of the video; and extracting 12 images from the video based on the analysis of the features. The method is preferably performed in a single device such as a personal computer, a server, a mobile device, a camera, or the like. In other words, the method is advantageously and automatically conducted by the device without external inputs .

In one preferred embodiment, acquiring 10 the video

includes filming the reference object, particularly by a same device which performs the analyzing and extracting steps. In this case, the device is equipped with the functions such as filming and editing a video, and thus can acquire a video or an image sequence by its own. For example, a multi-functional mobile phone or a mobile device can be used for the implementation of this preferred embodiment. Of course, the video can also be acquired from an external device such as a supplementary camera, a database, or any other suitable apparatus. If the captured video is used for a subsequent camera calibration

procedure, the same is captured by the camera which is to be calibrated. As shown in Fig. 2, in another preferred embodiment, the method further comprises identifying 13 the reference object in the video. The identification 13 of the reference object can be automatically performed by the device or alternatively by a user via a user input.

In yet another preferred embodiment, the method optionally comprises segmenting 14 the video into several clusters each including at least one image. In this embodiment, extracting 12 images from the video includes determining one key image for each of the clusters and then extracting the key images.

In the following one exemplary embodiment of the proposed method is described in detail. In this embodiment, a video is acquired by capturing a calibration object from various distances and orientations using a camera. A checkerboard is used as the calibration object for this embodiment, while a central checker is defined as the reference object which preferably remains visible across the images of the whole video. Of course, any one of the checkers of the checkerboard can be defined and chosen as the reference obj ect . In this embodiment, the central checker is identified 13 by a user through a user input by localization of the four corners thereof in a clockwise or a counterclockwise order in the first image of the video. Alternatively, the

reference object can be automatically defined by an

operation device depending on different situations and demands. More details about automatic identification are given below in an implementation example. The analysis 11 of the features of the reference object, i.e., the central checker in this embodiment,

advantageously comprises temporally tracking the reference object across the images of the video and extracting and analyzing the features of the reference object. In

particular, the positions of the four checker corners of the reference checker are tracked and extracted, which are subsequently used to extract four features related to the motion state and visual appearance of the reference object.

Fig. 3 shows a flowchart illustrating the procedure of an exemplary tracking of the reference checker. First, the locations of the corners of the reference checker are predicted 15. The predicted position, the estimated

position and the velocity of the i-th corner in the j-th image of the video are denoted as {>i,j, Vi,j_> ^vi,j) · I^R the first image of the video, the predicted positions are identified via a user interaction or by a device automatically, and the velocity is initialized to zero. Then the tracking of the reference checker proceeds across the successive images using a prediction-measurement scheme. A constant velocity dynamical model is applied to compute the predicted corner locations in the images,

where the velocity is computed as

The measurement step is implemented by searching 17 for the local maxima of a cornerness measure within a small search region, denoted as W_{i r} centered on the corner predicted location Pi . To compute the cornerness measure, many different solutions are available in the literature. Among them, it is opted in this embodiment for the magnitude of the complex response of the image to the Gauss-Laguerre Circular Harmonic Filter (2,0), denoted as o( ), since it has been shown to be an effective detector of a checker pattern .

In order to achieve a subpixel accuracy of the corner detection, the center of mass of all pixels with response magnitude higher than a given threshold is computed,

Vu =

¹ ∑_peJ/₀ ²(p) l where Ω denotes the set of pixels fulfilling the

condition. In this implementation, max{||/₀ ²(p)||}

is used as threshold for the definition of the set Ω. Fig. 4 shows an exemplary result of the tracking of the reference checker, wherein Fig. 4(a) shows the shape of the corner detection filter, Fig. 4(b) shows the magnitude of the response and Fig. 4(c) shows the extracted corner position, which is accurately located at the corner of the checkers.

Subsequently, the temporal tracking of the reference checker is used to compute four quality measures of the reference checker in the images of the video and to populate the calibration image dataset from the video. The four quality measures are defined according to the features that are preferably retrieved in the final image dataset, namely low speed, slanted and front-to-parallel orientation of the reference object, exhaustive coverage of the image retinal plane with the reference object, and the distance of the reference object from the camera.

The overall measure of the velocity of the reference checker is defined as the sum magnitudes of the velocities of the checker corners:

It is assumed that the formula provides an indirect

indication of image blur likelihood and, therefore, is used to identify the sharpest images of the video, where the corner localization can reach the highest accuracy.

The second measure deals with the orientation of the reference checker. It is observed that in an ideal case where a camera is not affected by optical distortion, a front-to-parallel configuration maintains the rectangular shape of the checker in the captured images. Therefore, in this embodiment, the projective distortion of the reference checker is defined and taken as an indicator of the

relative slant between the camera and the reference object. The distortion measure, denoted as 9_j , is derived from two simple geometrical features, namely the ratio between the checker diagonals and the ratio between the longest and shortest checker side:

where d_j and D_j are the short and long diagonals, and s^- is the length of the i-th checker side. An exhaustive coverage is indeed helpful in order to guarantee a reliable estimation of the camera projection model also in the image periphery. For this purpose, a regular grid of nodal points are denoted as c_k and defined as representatives of the target image areas, and each image is labeled according to the nodal point closest to the reference checker:

argmin{||py - c_k II)

where p_j is the geometric center of the reference checker. Fig. 5 shows the grid of nodal points used in this

embodiment and also the resulting space partitioning as a Voronoi diagram.

The fourth measure deals with the distance of the reference object from the camera. The area of the central reference checker, denoted as σ,· , is used as a simple distance indicator. It is reasonable to assume that the bigger the checker is, the closer the reference object is. The area is simply computed using a diagonal to split the checker in two triangles and summing up the corresponding areas.

After the above computation of the four quality measures, the same are analyzed in order to decide which image of the video should be selected and extracted from the given video .

A tridimensional labeling space L = LQ X L_A X L_K is defined and implemented for the whole given video, where LQ = [θ₃, θ_ρ j is used to indicate the slanted or front-to-parallel orientation of the reference object, ί_σ = {σ_Ν, σ_ρ] is the distance of the reference object from the camera, and

L_K = {1, ... , K] is the corresponding image region.

The L_K labeling is a relatively simple task as it

corresponds to the closest nodal point label kj . The labeling LQ and L_A serve as two numerical thresholds that are required to be compared with the distance and

distortion measures 9_j and σ,· . K-means clustering algorithm is used in this exemplary embodiment, which is aimed to partition the 2D observation vectors ( 9_j and σ,· ) in k = 4 clusters. The cluster centers, needed as algorithm

initialization, are provided by the four possible

combinations of the extreme values assumed by 9_j and σ,· , i.e., four extremities spanned by the scaled measures of 9_j and Oj .

Once the whole acquired video is labeled, the image of the video with lowest velocity is selected and extracted from each cluster of the video. The extracted images then provide the image dataset for subsequent camera calibration process . Figs. 6 and 7 show two implementation examples for the above method, where a Panasonic stereo camcorder with normal 50mm-like field of view (Fig. 6) and a GoPro HD Hero2 camera with a 170° wide-angle lens (Fig. 7) are respectively used. The black squares shown in the curve diagrams represent the extracted images which are displayed at the bottom of the respect figures. In an ideal case, the extracted images from the video should contain a representative for each label combination, in order to meet the requirements regarding the volumetric extension of the virtual calibration structure and the coverage of image retinal plane. This leads to an image dataset with a maximum cardinality of 4xK. In practice, it sometimes occurs that not all labels are represented in the final dataset. However, based on the above exemplary embodiment of the proposed method, the extracted image dataset is the one which best meets the suitability

requirements, given the available video.

The proposed image extraction method provides an improved solution to automatically acquire a calibration image dataset by a simple and normal device or computer. The initial video input of the method can be easily acquired from a supplementary camera or by a device itself directly without special requirements or limitations, which is also feasible for end consumers. For example, the video can be captured by simply waving a reference object in different orientations and distances in front of a filming camera, where the camera can stand at one fixed position without moving . Given the video, the image extraction method simplifies the procedure to acquire an image dataset from the video and optimize the dataset suitability for a subsequent camera calibration process. Comparing to typical solutions which acquire an image dataset from a video by manual review and selection of the images from an operator, the proposed image extraction method is time saving and can be widely used in either normal or critical situations such as an outdoor environment where a qualitative visual analysis of a calibration dataset might not be possible. Automatic identification of calibration object

As described above, in the proposed image extraction method, the preliminary identification of the reference object can be advantageously and automatically performed by a device, a computer, or the like. In the following

implementation example of the automatic identification, a checkerboard is used as the calibration object.

Fig. 8 shows an exemplary procedure of the checkerboard detection in this example. First, the identification of seed corners, e.g. three corners of a central checker, is performed, either by a user via a user input or

automatically by a device. This information is used to initialize the reference corner location and the analysis of local velocity in the direction of the grid axis, which represent the seed for the following image exploration. An automatic labeling of the whole set of checkerboard corners is subsequently performed by means of a simple corner extractor integrated within a recursive framework.

Essentially, the checkerboard is progressively discovered by walking along the grid edges from corner to corner in the direction of the four connected neighbors. The

exploration directions and the labels of the searched corners are completely defined by the information available for the initiating corner. The local response to a pair of circular harmonic filters, which are specifically tailored to edges and corners, is used to update the discovered velocities and to localize the corners with sub-pixel precision. By means of this framework, the geometrical information are propagated from one seed corner, and the set of detected corners is progressively updated using only - li ^¬ the local image information, which make the technique insensible to image distortion or partial visibility.

The grid dataset is denoted as G = {rii = [Xi,Yi,Xi,yi,v_ix,v_iY]}_{i=1 N}, where N is the number of corners, (¾¾ and (Xi,yi) are the coordinates of the i-th corner in object and image spaces and ix,v_iY) are the local velocity in X and Y directions, namely the directions in image space towards the next corners (X_t + 1, ¾ and (X₀ Y_t + 1) .

For the identification of seed corners, three corners

°f the reference checker are selected and used to instantiate the seed corners and the local velocities:

where function re (.) identifies the sub-pixel refinement the corner localization. Starting from this seed the checkerboard is discovered by recursively calling a corner-to-corner detection method which alternates edge walking and velocity update steps until the next corner is detected or the image border is reached. This recursion is invoked from the calling node in the direction of the corners belonging to the four- connected neighborhood. For example, from a node n_ir the search for the corners (¾Y^"i + l) and (¾ Y±— 1) is invoked using the velocities v_iY and —v_iY, which is analogously done for the other two connected nodes in the X direction. The edge walking step is performed in a filtering based context using the Gauss-Laguerre Circular Harmonic Filters (GL-CHF) . The set of GL polynomials {L^(p,e);n,k E N] provides a complex basis suitable for the orthogonal expansion of square integrable functions represented in polar

coordinates, under the weighting of a Gaussian window

V(p) = _e-ⁿP² [VII] :

In literature the set of coefficients f^io) is referred as the Gauss Laguerre Transform (GLT) of f{p,9) at scale σ and is obtained by projection of f{p,9) onto the basis provided by the Gaussian-weighted GL polynomials. Equivalently, in the case of digital images, the GLT can be obtained by filtering through a bank of GL-CHFs .

The GLT provides a representation of an image as a stack of sub-bands, and the first of which are selectively tuned on different low level features, such as corners, borders or lines. In particular, the sub-bands _Q ¹ and /₀ ² may be acounted as edgeness and cornerness maps and are used for the edge walking and the corner localization in this implementation example. The high selectivity of the GL-CHFs to specific image features suggests the possibility to exploit the local response fo ip ) to follow an edge between two consecutive corners. Indeed, the magnitude of the local response supports the edge localization and provides an estimate for its local direction [VIII] . Fig. 9(a) shows a sample template of a checker corner of a checkerboard, and Fig. 9 (b) shows the magnitude and phase of the edgeness map extracted from the sample template of Fig. 9(a) using GL- CHFs techniques. Since the edges are not straight on the image plane due to distortion introduced by the optical system, the edge walking is performed by iteratively taking short steps and updating the the edge direction according to the local response fd(x) .

Given the current edge position p = (x,y), a step is taken in the direction of the local velocity v_p . The predicted edge location p' = p + δν_ρ is used to identify a segment denoted as L and orthogonal to the edge itself. Along this segment, the pixel with the highest edgeness magnitude is selected and the local velocity is update according to the local phase of fo{x):

Fig. 10 illustrates an exemplary graphic for the above step. The point A represents the location p' of the edge predicted using the current velocity estimate. From this point a search ID space (the solid line) segment is

inspected in order to refine the edge location by searching for the maximum of the local edgeness map (point B) . The local velocity is updated using the phase of the local edgeness map. The edge is travelled until the update of the direction of the local velocity exceeds a given threshold. This happens when a corner is crossed, as the inversion of the checker colors determines a rotation of approximate π radians in the phase of the filter response. Once the edge walking termination condition is met, the corner location is locally refined using the cornerness map foip). The corner candidates are extracted by searching for the local maxima of the cornerness map within a small neighborhood of the current edge location. In highly distorted images and in particular in the peripheral areas, it may happen that a set of multiple candidates is

detected, from which only the most likely ones are

retained. For this purpose, the distribution of the

edgeness map around each corner candidate is inspected and the coherence with the grid topology is measured.

The set of corner candidates is denoted as C = fc,) , with Γ_[ denoted as the closed square contour centered on each corner candidate and Dj = \ lm_{j k} E Γj,· ) j_{fc=1 K} . is the set of local maxima of the edgeness map detected on the same contour. If the candidate is correctly centered on the checkerboard corner, the set of velocities directly sampled from the edgeness on the location given by D_j should be aligned with the directions analytically defined by the coordinate of the points in a same set D_j , which is

schematically shown in Fig. 11(a).

Following the above assumption, a quality measure q is defined for each c

The candidate with the highest score is selected. To achieve a sub-pixel accuracy of the pixel position, an elliptical paraboloid is fitted using a 3 X 3 window

centered on the selected candidate and the local minima are analytically computed. This simple refinement step is introduced as described above in (1) with the notation re/(.).

Fig. 12 shows an exemplary result of this implementation example for automatic identification of a checkerboard. Fig. 12(a) shows the identification of the seed corners and Fig. 12(b) shows the final result of the checkerboard identification. The images are captured using a GO-PRO Hero camera which is equipped with a very wide angle lens able to operate in two different configurations, 120° and 170° field of view. The latter, i.e., the narrowest

configuration, is used in this implementation example.

It can be seen that the checker corners are accurately identified, even including those located close to the image border. In addition, even though the captured checkerboard is severely distorted due to the wide angle lens and is only partially visible, the detection and identification of the checkerboard are still reliable. The missed detection rate of the checker corners is highly reduced, contributing to a superior collection of 2D/3D correspondences in subsequent camera calibration procedures. In other words, the automatic identification technique can automate the creation of the 2D/3D correspondences dataset and is suitable for exploitation with a wide range of lenses, from the regular field of view up to very wide-angle fisheye lenses and full panoramic catadioptric systems.

Fig. 13 schematically shows a device 20 configured to perform the embodiments of the proposed image extraction method. The device 20 comprises an acquiring unit 21 configured to acquire a video; an analyzing unit 22

configured to analyze features of a reference object captured in the images of the video; and an extracting unit 23 configured to extract images from the video based on the analysis of the features.

Preferably, the acquiring unit 21 is configured to film a reference object. More preferably, the analyzing unit 22 is configured to identify the reference object and segment the video into several clusters each including at least one image, and the extracting unit 23 is configured to

determine one key image for each of the clusters and extract the key images. Optionally, the device 20 further comprises a user input unit 24 configured to receive a user input for identifying the reference object captured in the images of the given video. The analyzing unit 22 and the extracting unit 23 can be optionally combined into a single device or function separately.

In one embodiment, the reference object includes a

reference checker of a planar checkerboard. The analyzed unit 22 is configured to analyze 11 the features by

labeling the images of the video in a tridimensional space defined according to the orientation, area and closest representing nodal point of the reference checker in the images, and to segment 14 the video into clusters by using the techniques of K-means clustering on the labeled images in the tridimensional space. The extracting unit 23 is configured to determine one image for each cluster by selecting the image in which the reference checker has a lowest velocity for each cluster. REFERENCES

[I] D. Liebowitz, "Camera Calibration and Reconstruction of Geometry from Images," D.Phil, thesis, University of Oxford, 2001

[II] E. Hemayed, "A survey of camera self-calibration,"

Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance, pp.351-357, 2003

[III] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision (2 ed.). Cambridge University

Press, New York, NY, USA, 2003

[IV] R.Y. Tsai, "An Efficient and Accurate Camera

Calibration Technique for 3D Machine Vision, " Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 364-374, 1986

[V] J. Salvi, X. Armangue and J. Batlle, "A comparative review of camera calibration methods with accuracy evaluation," Pattern Recognition, Vol. 35, Issue 7, pp. 1617-1635, 2002 [VI] Z. Zhang, "A flexible new technique for camera

calibration." IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11) : 1330-1334, 2000

[VII] G. Jacovitti, A. Neri, "Multiresolution circular

harmonic decomposition", IEEE Transactions on Signal Processing, 48 (11) : 3242-3247, 2000

[VIII] L. Sorgi, "Edgelet tracking using Gauss-Laguerre

Circular Harmonic filters". ICIP 2011: 2897-2900, 2009

Claims

A method for extracting images from a video including plurality of images in which a reference object is captured, the method comprising:

- acquiring (10) the video;

- analyzing (11) features of the reference object in the images of the video;

- segmenting (14) the video into clusters based on the analysis of the features, each cluster including at least one image;

- determining one image for each cluster based on the analysis of the features; and

- extracting (12) the determined images from the video .

The method of claim 1, wherein segmenting (14) the video into clusters includes using the techniques of fe^¬ means clustering.

The method of claim 1 or 2, wherein the reference object includes a reference checker of a planar

checkerboard, and the features of the reference object include one or more of velocity, orientation and area of the reference checker, and a representing nodal point closest to the geometric center of the reference checker .

The method of claim 3, wherein

- analyzing (11) the features includes labeling the images of the video in a tridimensional space defined according to the orientation, area and closest

representing nodal point of the reference checker in the images, - segmenting (14) the video into clusters includes using the techniques of K-means clustering on the labeled images in the tridimensional space, and

- determining one image for each cluster includes selecting the image in which the reference checker a lowest velocity for each cluster.

The method of one of the preceding claims, wherein analyzing (11) features of the reference object

includes tracking the reference object across the images of the video and extracting features of the tracked reference object.

The method of one of the preceding claims, wherein acquiring (10) the video includes filming the reference obj ect .

The method of one of the preceding claims, further comprising identifying (13) the reference object in video .

The method of claim 7, wherein identifying (13) the reference object includes using Gauss-Laguerre Circular Harmonic Filters.

A device (20) configured to extract images from a video including a plurality of images in which a reference object is captured, the device comprising:

- an acquiring unit (21) configured to acquire (10) the video ;

- an analyzing unit (22) configured to analyze (11) features of the reference object in the images of the video, to segment (14) the video into clusters based on the analysis of the features, each of the clusters including at least one image; and

- an extracting unit (23) configured to determine one image for each cluster based on the analysis of the features and to extract (12) the determined images from the video.

The device (20) of claim 9, wherein the reference object includes a reference checker of a planar

checkerboard, and the analyzing unit (22) is configured to

- analyze (11) the features by labeling the images of the video in a tridimensional space defined according to the orientation, area and closest representing nodal point of the reference checker in the images, and

- segment (14) the video into clusters by using the techniques of K-means clustering on the labeled images in the tridimensional space,

and the extracting unit (23) is configured to

- determine one image for each cluster by selecting the image in which the reference checker has a lowest velocity for each cluster.

The device (20) of claim 9 or 10, wherein the analyzing unit (22) is configured to track the reference object across the images of the video and to extract features of the tracked reference object.

12. The device (20) of one of claims 9 to 11, wherein the acquiring unit (21) is configured to film the reference obj ect .

13. The device (20) of one of claims 9 to 12, wherein the analyzing unit (22) is configured to identify (13) the reference object in the video.

14. The device (20) of one of claims 9 to 13, further

comprising a user input unit (24) configured to receive a user input for identifying the reference object.

15. A computer readable storage medium having stored

therein instructions enabling extracting images from a video, which when executed by a computer, cause the computer to:

- acquire a video including a plurality of images in which a reference object is captured;

- analyze features of the reference object in the images of the video;

- segment the video into clusters based on the analysis of the features, each cluster including at least one image ;

- determine one image for each cluster based on the analysis of the features; and

- extracting the determined images from the video.