Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture

Definitions

• Method and device for processing image sequences of a scene and system for monitoring a scene comprising such a device
• the present invention relates to a method for processing image sequences of a scene, acquired by at least one camera, of the type comprising a step of reconstructing a panoramic view of the scene from the acquired image sequences and a step of displaying the panoramic view.
• the invention also relates to a device associated with the image sequence processing method and a monitoring system comprising such a device.
• the invention applies in the field of video surveillance systems.
• video surveillance systems are nowadays equipped with more and more numerous, rich and complex functionalities, with the aim of automating, simplifying and making reliable the work of an operator in control and command centers equipped with such systems.
• CCTV CCTV
• the operator must generally interpret a large number of video streams as well as information from ancillary sensors or video algorithms.
• the operator must then be expert in the system and know in a remarkable way the disposition of the cameras and third devices in the environment filmed by the cameras and monitored by the monitoring system. He then makes a considerable mental effort to build a coherent (mental) visualization of the situation of his environment.
• mapping system used as a data representation medium.
• This mapping system includes a global data visualization tool containing several basic information such as the position and state of the cameras, their field of view ...
• this mapping system is also used as an interface support Man / Machine (HMI) for basic operational functions in command centers, eg for camera reservation.
• HMI Man / Machine
• mapping systems are two-dimensional and dissociated from the video stream.
• the document WO 2010/1 12 604 describes a method of virtual extension and enrichment of the field of observation of the current image of a scene described by a video comprising several images.
• the field of view is extended by a mosaic obtained from the images of the video illustrating the scene at different resolutions.
• the method further comprises a step of insertion in the repository of the contextual data image and a step of enriching the extended field of view by georeferenced information in the contextual data.
• the object of the invention is to propose another type of method for contextualization in the form of a panoramic view of the stream or streams of at least one camera.
• the subject of the invention is a method for processing image sequences of a scene of the aforementioned type, characterized in that the panoramic view comprises at least one area of interest that can be moved in the panoramic view and refreshed at a higher frequency than the rest of the panoramic view.
• the method for processing image sequences of a scene comprises one or more of the following characteristics taken alone or in combination:
• the area of interest is dynamically refreshed at the acquisition frequency of the image sequences
• the method comprises a step of displaying additional contextual information on the displayed panoramic view
• the additional contextual information belongs to the group consisting of: georeferenced devices, mobile units, georeferenced events, points of interest;
• the panoramic view is reconstructed from sequences of images acquired by at least one mobile camera performing a scan of the scene intended to be represented on the panoramic view;
• the panoramic view is reconstructed from sequences of images acquired by at least two fixed cameras having fields of view of the scene that are at least contiguous;
• the method comprises a step of filtering the image sequences of the scene in order to extract information therefrom.
• the invention also relates to a device for processing image sequences of a scene, acquired by at least one camera, comprising means for reconstructing a panoramic view of the scene from the sequences acquired images and means for displaying the panoramic view, characterized in that the device further comprises means for defining an area of interest in the panoramic view, means for moving the area of interest in the panoramic view and means for refreshing the area of interest at a frequency greater than that of the rest of the panoramic view and is adapted to implement a method of processing image sequences as described above.
• the image sequence processing device comprises one or more of the following characteristics taken alone or in combination:
• the moving means of the zone of interest can be manipulated by an operator
• the moving means of the zone of interest are chosen from the group comprising a control lever, a touch screen, a touchpad and buttons.
• the subject of the invention is also a system for monitoring a scene, comprising at least one camera oriented towards the scene and acquiring sequences of images of the scene, characterized in that it comprises a device for processing image sequences as described above.
• FIG. 1 is a block diagram illustrating a scene monitoring system according to the invention comprising a device for processing image sequences of the scene,
• FIG. 2 is an exemplary illustration of a panoramic view of a scene obtained by an image sequence processing device of the scene according to one embodiment of the invention
• FIGS. 3 and 4 are block diagrams illustrating steps of a method for processing image sequences of a scene implemented by the device of FIG. 1.
• the invention relates to a system 10 for monitoring a scene as illustrated in FIG.
• This monitoring system 10 comprises at least one camera 12 oriented towards the scene and acquiring sequences of images of the scene and a device 14 for processing the scene sequences of the scene.
• image sequences are also called video streams.
• the camera or each camera 12 is of the PTZ camera type for "Pan-Tilt-Zoom" in English, that is to say remotely controllable by an operator of the surveillance system 10 to realize panoramic, tilting or zoom in an adjustable way.
• the instantaneous fields of view of each camera are distinct from each other.
• the or each camera 12 is a fixed camera whose field of view can not be adjusted in size, inclination or zoom.
• the instantaneous fields of view of each camera are at least contiguous and preferably overlap partially so as to be able to reconstruct a continuous panoramic view from the images acquired by the cameras at a given moment.
• the device 14 for processing the scene sequences of the scene comprises a unit 16 for processing the images acquired by each camera 12 and a man / machine interface comprising means 18 for displaying in graphical form the images processed by the unit 16.
• the processing unit 16 is able to process the image sequences acquired by the cameras 12, to reconstruct a panoramic view from the acquired image sequences and to command the display of this panoramic view in graphic form by the means 18 display.
• the processing unit 16 comprises means 20 for reconstructing the panoramic view of the scene from the image sequences acquired.
• the device 14 further comprises means for defining an area of interest in the panoramic view, means for moving the area of interest in the panoramic view and means for refreshing the area of interest at a frequency. higher than the rest of the panoramic view.
• the displacement means of the zone of interest can be manipulated by an operator. They are chosen from the group comprising a control lever, a touch screen, a touchpad, buttons, etc.
• the device 14 comprises a data storage unit 28 representative of the acquired image sequences and data to be displayed in graphical form by the display means 18 such as the reconstructed panoramic view and the area of interest.
• the storage unit 28 comprises a first memory 30 and a second temporary memory 32.
• the first memory 30 is adapted to store the initial and final data, that is to say before and after treatment by the treatment unit 16.
• the second temporary memory 32 is adapted to store the temporary data, that is to say, being processed by the processing unit 16.
• FIG. 2 illustrates an example of a panoramic view 50 of a scene reconstructed by the means 20 for reconstructing a device 14 for processing image sequences of the scene as described above.
• an area of interest 52 is defined by the means for defining the device 14.
• This area of interest 52 is representative of the field of view of a camera 12 of a surveillance system according to the invention comprising the device 14.
• the area of interest 52 is refreshed at a higher frequency than the rest of the panoramic view by the refreshing means of the device 14.
• the panoramic view is refreshed locally by the video stream from the or each camera associated with an area of interest.
• the dynamic panoramic view is formed of the refreshed area of interest and the panoramic view, that is to say the bottom, said static with respect to a lower refresh rate than that of the area of interest.
• an operator using a monitoring system 10 enters data via the man / machine interface concerning the panoramic view that he wishes to display on the display means 18 of the human-machine interface.
• these data relate to the dimensions of the field of view and the resolution of the desired panoramic view.
• the image sequence processing device implements a method for processing image sequences of a scene according to the invention. This is described in the case where the image sequences are acquired by a single camera 12 in order to simplify the description.
• This camera 12 is preferably a PTZ camera type mobile camera.
• This method comprises a step of creating or reconstructing a panoramic view of the scene from the image sequences acquired by the moving camera having performed a scan of the scene.
• the panoramic view represents a field of view of 360 °.
• An exemplary embodiment of this step 100 is illustrated in FIG.
• the processing unit 16 After receiving a request from the operator for the generation of a panoramic view at 102, the processing unit 16 reads from the storage unit 28 information relating to the camera to be used to generate the desired panoramic view, during of a step 104.
• this information is the IP address of the camera, its geographical coordinates ...
• the processing unit 16 sends commands to the camera so that it acquires a sequence of images making it possible to reconstruct the desired panoramic view in both resolution and field of view.
• the camera acquires this sequence of images for a defined time according to the desired resolution and field of view and acquisition parameters of the camera.
• the acquired images are transmitted to the device 14 and stored in the temporary memory 32.
• the processing unit 16 filters and cleans the images of the sequence to remove information, at 1 10. For example, a raw image, that is to say acquired by the camera, is cleaned and filtered to remove from the image unwanted dynamic elements in the representation of the scene illustrating the city such as cars, characters or others. This step is performed automatically.
• the processing unit 16 implements video analysis methods for the purpose of automatically detecting moving objects, for example by applying bottom segmentation methods and keeping in the image only the static background. of the scene in order to form the so-called static panoramic view.
• the filtered and cleaned images are stored in the temporary memory 32. Then, if the camera has not scanned a field of view sufficient to form the desired static panoramic view, the device 14 again performs steps 106 through 110. there are enough images to form the desired static panoramic view.
• the processing unit 16 builds the static panoramic view from the cleaned and filtered images stored in the temporary memory 32, during a step 1 12.
• the images from the mobile camera, taken at different angles (Pan and Tilt), after being cleaned and filtered, are juxtaposed by image processing techniques known to those skilled in the art to align them and to form a global image of the whole scene: the static panoramic view.
• This image is then recorded in the first memory 30 of the storage unit 28 and transmitted to the display means 18.
• the operator defines an area of interest in the panoramic view using the means for defining this area, for example using a touch screen, a touchpad and / or buttons of the input means of the screen.
• This area of interest is refreshed dynamically from the video stream of the camera. The whole formed by the area of interest refreshed in the panoramic view is called thereafter a dynamic panoramic view.
• FIG. 4 illustrates an exemplary embodiment of a step 200 for displaying the dynamic panoramic view of the image sequence processing method of the camera implemented by the device 14.
• the processing unit 16 After receiving the definition of the area of interest by the operator, the processing unit 16 reads in the storage unit 28 information about the camera corresponding to the desired area of interest, during a step 202.
• this information is the IP address of the camera, its geographical coordinates ...
• a step of embedding the video stream in the panoramic view is performed in order to relocate the video stream in the panoramic view.
• This step takes place in two sub-steps:
• This "direct" method has the advantage of being robust to the brightness conditions of the scene since it is based only on mechanical values.
• the coordinates of the texture of the portion of the panoramic view defined by the area of interest to be updated are calculated in 208.
• this sub-step 210 of fine relocation is executed by relying on image processing techniques to refine the location.
• this sub-step 210 of fine relocation is carried out by the mapping of points of interest between the panoramic view and the images acquired by the camera.
• the texture of the portion of the panoramic view corresponding to the area of interest is updated, at 212, by embedding the video stream received by the camera associated with the area of interest.
• the area of interest is refreshed at a higher frequency than the rest of the panoramic view.
• it is refreshed dynamically at the acquisition frequency of the video stream of the camera associated with the area of interest.
• the area of interest is movable in the panoramic view by the operator using the means for moving this area, for example using a touch screen, a touchpad and / or buttons means of input of the man-machine interface.
• step 200 of displaying the dynamic panoramic view is then implemented again.
• the step 200 of displaying the panoramic view is performed using a primitive three-dimensional model (3D) such as for example a polyhedron (or quad) or preferably a sphere.
• 3D three-dimensional model
• the method further comprises a step of enriching the dynamic panoramic view with information on the environment, for example the names of the streets visible on the panoramic view, as well as the state of the system, for example localized alarms.
• the intrinsic and extrinsic parameters of each camera are accurately estimated in order to be able to integrate this information in a realistic way into the dynamic panoramic view.
• an additional contextual information display is displayed on the displayed panoramic view.
• This additional contextual information is selected from the group consisting of georeferenced devices, mobile units, georeferenced events, points of interest.
• step 100 of reconstructing a panoramic view is repeated regularly in order to update the static panoramic view. However, the update rate of the static panoramic view is lower than that of the area of interest. This update ensures a good match with the actual scene over a longer period.
• the method and the device for processing image sequences of a scene thus make it possible to obtain a dynamic panoramic view refreshed locally by the video (s) coming from one or more observation points which thus restores Realistic context of one or more real video streams.
• the system and the device according to the invention can minimize the cognitive load of the operator by reducing the need to readjust the frame of reference of the cameras when changing the point of view.
• the use of the panoramic context makes it possible to display on the same medium: the video stream, the static vicinity of the camera, and location-based information such as alarms or popups such as street names, if they are visible in the field of the panoramic view, to complete the perception of the operator.
• the monitoring device according to the invention also makes it possible to provide new functionalities with respect to state-of-the-art monitoring devices such as camera control, thus simplifying the use of the video surveillance device. .
• the operator controls his visual using for example a control lever (or "joystick” in English) classic.
• the updated area of interest moves according to the movements of the camera.
• the display of the other available cameras of the network in the panoramic field of view respectively represented by their associated area of interest allows the operator to choose / switch from the current camera to another camera simply by selecting the one in the panorama thanks to the man-machine interface.
• the operator selects the camera by double clicking on the area of interest representative of the camera's field of view in the panorama using a joystick, a touch screen or a mouse of the man-machine interface.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Closed-Circuit Television Systems (AREA)
• Studio Devices (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=44486864

Family Applications (1)

Country Status (4)

Cited By (1)

Families Citing this family (1)

Citations (1)

Family Cites Families (4)

• 2011
  • 2011-01-31 FR FR1100282A patent/FR2971074B1/fr not_active Expired - Fee Related
• 2012
  • 2012-01-31 EP EP12702022.0A patent/EP2671376A1/de not_active Withdrawn
  • 2012-01-31 WO PCT/EP2012/051537 patent/WO2012104290A1/fr active Application Filing
  • 2012-01-31 MX MX2013008816A patent/MX2013008816A/es active IP Right Grant

Patent Citations (1)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events