WO2013017387A1 - Methods for compressing and decompressing animated images - Google Patents

Methods for compressing and decompressing animated images Download PDF

Info

Publication number
WO2013017387A1
WO2013017387A1 PCT/EP2012/063606 EP2012063606W WO2013017387A1 WO 2013017387 A1 WO2013017387 A1 WO 2013017387A1 EP 2012063606 W EP2012063606 W EP 2012063606W WO 2013017387 A1 WO2013017387 A1 WO 2013017387A1
Authority
WO
WIPO (PCT)
Prior art keywords
encoded
image
decimated
animated
window
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2012/063606
Other languages
French (fr)
Inventor
Emmanuel GUINET
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Airbus DS SAS
Original Assignee
Cassidian SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cassidian SAS filed Critical Cassidian SAS
Publication of WO2013017387A1 publication Critical patent/WO2013017387A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115Selection of the code volume for a coding unit prior to coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object

Definitions

  • the invention relates to a method for compressing animated images and to a corresponding method for decompressing animated images.
  • Information is acquired via image sensors, infrared sensors and electro-optical sensors installed in the drone so as to be transmitted in real time to a remote station via a terrestrial or satellite data transfer link.
  • the information acquired by the sensors generally has a rate of flow greater than 1 Gbit/s.
  • Each terrestrial or satellite link attributed to a drone normally has a rate of flow between 2 and 10 Mbit/s, this link being shared by the different sensors. In this instance, the acquired information is compressed.
  • a known technique consists in subjecting the acquired animated images to temporal decimation, that is to say, instead of transmitting 25 images/s, no more than 4 images/s are transmitted for example.
  • temporal decimation that is to say, instead of transmitting 25 images/s, no more than 4 images/s are transmitted for example.
  • the object of the invention is to propose a method for compressing animated images and a method for decompressing animated images, making it possible to reduce the transmission rate of the acquired animated images whilst still maintaining good fluidity of movement and/or an optimum level of detail over those zones of the animated image considered to be of interest.
  • the invention relates to a method for compressing animated images, said method comprising the following steps:
  • the window comprising at least one region of interest is encoded uniquely and is not decimated.
  • the region(s) of interest (ROI) contained in the window thus maintain(s) optimum definition. Since the window only represents part of the acquired animated image, the size of this window once encoded is relatively small, especially since the fixed size of the window makes it possible to use a video codec, which is more efficient than an image codec.
  • the transmission of the encoded decimated animated image and of the encoded window to the remote station thus uses a reduced rate of flow of a data transfer link (also called a channel).
  • a data transfer link also called a channel.
  • the method for compressing animated images according to the invention may also have one or more of the features below, taken individually or in any technically feasible combination.
  • the method comprises the following additional steps:
  • said selection is made on the basis of said animated image or on the basis of the content of said decimated animated image of lower resolution illustrating a background.
  • said selection of at least one region of interest is made by a cropping or shrinking method.
  • said selection is made manually at the remote station or automatically at the encoder.
  • metadata are multiplexed with the encoded decimated animated image and the encoded window during said multiplexing step.
  • each animated image is encoded by video encoding.
  • a better image resolution compared to image coding is thus achieved at iso rate of flow.
  • the window is encoded by video coding.
  • a better encoding efficiency and therefore, at iso rate of flow, a better resolution of the region of interest are thus obtained compared to image encoding.
  • said encoding of the window takes place at constant rate of flow.
  • the method comprises an additional step of shrinking of said at least one region of interest before the grouping step. This makes it possible to reduce the size of those regions of interest which are too large compared to the configured size of the window.
  • the method comprises a step of temporal decimation of the animated images acquired by said image sensor. This makes it possible to reduce the rate of flow of the transmitted images.
  • the method comprises an additional step of instantaneous transmission of an image zone from said image sensor to said remote station.
  • the image zone can be formed by the entire acquired image or by part of the acquired image. This makes it possible to transmit a high-definition image of the entire scene acquired by the image sensor. This is especially beneficial in particular if the video encoding method used limits the number of bits per pixel component or else the chromatic definition.
  • said image zone transmitted instantaneously is encoded in accordance with image encoding.
  • said image zone is multiplexed with the encoded decimated animated image and the encoded window during said multiplexing step.
  • the instantaneous transmission of an image zone of said image sensor is carried out at constant rate of flow.
  • said image sensor is installed in an aerial, terrestrial or naval mobile device.
  • said image sensor is fixed.
  • the invention also relates to a computer program for compressing animated images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of said sequences of instructions enabling implementation of the method for compressing animated images according to the invention, when said program is loaded onto a computer.
  • the invention also relates to a device for compressing animated images, said device for compressing animated images comprising: - an image sensor adapted to acquire animated images;
  • - decimation means able to shrink the animated image acquired by said image sensor so as to obtain a decimated animated image of lower resolution illustrating a background
  • ROI region of interest
  • said encoder is also adapted to code said window comprising said at least one region of interest
  • a transmitter adapted to transmit the multiplexed encoded window and the multiplexed encoded decimated animated image.
  • the invention also relates to a method for decompressing animated images, said method comprising the following steps:
  • the superposition of said at least one region of interest and said decoded animated image makes it possible to obtain a resultant image comprising a zone of high definition (the region of interest) illustrating precisely identifiable elements and zones of low resolution (a region not comprising an element needing to be identified precisely).
  • the integration (or superposition) of the decimated animated image with the window makes it possible to have only a single window to be encoded.
  • the background image thus does not need a specific encoding channel and can use the same as that of the window. This feature also makes it possible to reduce the number of encoding procedures.
  • the integration of the decimated animated image with the window makes it possible to have only a single window to be decoded.
  • the background image thus does not need a specific decoding channel and can use the same as that of the window. This feature also makes it possible to reduce the number of decoding procedures.
  • the method comprises an additional step of ungrouping at least one region of interest in said window as a function of the metadata.
  • said method comprises an additional step of instantaneous reception of an image zone originating from an image sensor.
  • the invention also relates to a device for decompressing animated images, said device for decompressing images comprising:
  • a remote station adapted to receive an encoded decimated animated image illustrating a background and an encoded window comprising at least one region of interest; said encoded decimated animated image and said encoded window being multiplexed;
  • a demultiplexer adapted to demultiplex the multiplexed encoded window and the multiplexed encoded decimated animated image
  • a decoder adapted to decode said encoded window and adapted to decode said encoded decimated animated image
  • the invention also relates to a computer program for decompressing animated images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of said sequences of instructions enabling implementation of the method for decompressing animated images according to the invention, when said program is loaded onto a computer.
  • the invention also relates to a system for compressing and decompressing animated images, said system comprising the device for compressing images according to the invention and the device for decompressing animated images according to the invention.
  • the invention also relates to a method for compressing and decompressing animated images, comprising the following steps:
  • FIG. 1 shows a device for compressing animated images according to the invention in accordance with a non-limiting embodiment
  • FIG. 2 shows a block diagram of the steps of the method for compressing animated images according to the invention in accordance with a non-limiting embodiment
  • FIG. 3 shows a device for decompressing animated images according to the invention in accordance with a non-limiting embodiment
  • FIG. 4 shows a block diagram of the steps of the method for decompressing animated images according to the invention in accordance with a non-limiting embodiment
  • - figure 5 shows a method for compressing and decompressing animated images according to the invention
  • - figure 6 shows a device for compressing animated images and a device for decompressing animated images according to the invention, used in a non-limiting video surveillance application;
  • figure 7 shows a method for compressing and a method for decompressing animated images according to the invention implemented by the devices for compressing and decompressing animated images illustrated in figure 6.
  • Figure 1 shows a schematic view of a compression device DC for compressing animated images.
  • the compression device DC comprises:
  • an image sensor CI adapted to acquire animated images IM, said image sensor CI possibly being, for example, a camera installed in a device E, for example an aerial drone;
  • decimation means M_DECIM able to shrink an animated image IM acquired by the image sensor CI so as to obtain a decimated animated image IMD of lower resolution illustrating a background (or theatre of operations);
  • - selection means M_SELECT for selecting at least one region of interest ROI of variable size and position
  • - grouping means M_REGROUP for grouping the at least one region of interest ROI selected in the window F;
  • the encoder ENC is also adapted to code the window F comprising the at least one region of interest ROI ;
  • - multiplexing means MUX for multiplexing the encoded window FC and the encoded decimated animated image IMDC;
  • TX adapted to transmit the multiplexed encoded window FC and the multiplexed encoded decimated animated image IMDC.
  • the device DC for compressing animated images is adapted to communicate with a remote station SDT so as to transmit thereto the encoded decimated animated image IMDC and the encoded window FC, both of which are multiplexed MUX (FC&IMDC).
  • Animated images is understood to mean images which, when placed in sequence and displayed at regular interval, reproduce the activity and movements in a scene. Such a sequence is also called video. If the frequency of the images is high (that is to say the interval between two images is short), typically a few tens of Hertz, the content of two consecutive images in the sequence is very close. This small difference in content between two consecutive images is utilised by the video encoders with the aim of optimising efficiency in terms of compression rate.
  • the device DC is adapted to implement a method 100 for compressing animated images according to the invention and is described hereinafter with reference to figures 1 and 2.
  • the method 100 for compressing animated images comprises the following steps:
  • decimated animated image 104 (denoted ENCJMD in figure 2) by means of an encoder ENC; - creation 105 of a window F (also called a mosaic) of fixed and configurable size comprising at least one region of interest (denoted CREA_F in figure 2);
  • the compression method comprises the following additional steps:
  • the compression method comprises an additional step of shrinking 107 of said at least one region of interest before the grouping step (denoted DS-S-ROI in figure 2).
  • the compression method comprises a step of temporal decimation 102 of the animated images IM acquired by said image sensor CI (denoted DC-TP in figure 2).
  • the compression method comprises an additional step of instantaneous transmission 1 12 of an image zone from said image sensor CI to said remote station SDT (denoted TX-INT in figure 2).
  • the steps of the method 100 for compressing animated images are described in greater detail hereinafter. The description below also includes the steps of additional, non-limiting embodiments.
  • a first step 101 (denoted ACJM in figure 2), the image sensor CI acquires animated images IM.
  • the image sensor CI is a high-definition camera allowing acquisition of 25 images per second.
  • the image sensor CI can acquire images at a different rate, for example 30, 50 or 60 images per second.
  • this image sensor CI can be installed in an aerial, terrestrial or naval mobile device E so as to visualise theatres of operation, such as war scenes, and/or so as to follow a mobile vehicle
  • Each animated image acquired by the image sensor CI (also called a source image) is stored with its date of acquisition in a memory (also called a buffer) located in the drone E.
  • This buffer makes it possible to store a history of the last images acquired.
  • This history makes it possible to process orders, received by the remote station, for instantaneous image request and for tracking of the region of interest (tracking of the ROI).
  • an operator at the remote station orders a request for an image N
  • said order generally arrives at the encoder a few moments (seconds) after acquisition of the image N by the sensor.
  • Storage in the buffer of the image N after acquisition makes it possible to keep the image N accessible for processing and possible transmission to the remote station.
  • the operator at the remote station orders the tracking of a region of interest ROI on an element, also called an object, such as a car or person, of the scene present at a certain position in the image N
  • an element also called an object, such as a car or person
  • the position of this object in the last image acquired thus defines the current position of the ROI.
  • the animated image sensor CI can be fixed so as to carry out video surveillance, for example in a railway station.
  • the compression method 100 comprises a step of temporal decimation 102 of the animated images acquired by the image sensor CI. For example, 12.5 animated images per second will be received. This step makes it possible to reduce the number of animated images transmitted. It thus follows that this step of temporal decimation 102 makes it possible to reduce the rate of transfer.
  • a temporary increase in the temporal decimation can be implemented with the aim of transmitting an instantaneous image at a constant overall rate of flow.
  • the advantage of this mechanism is twofold:
  • decimation means M_DECIM spatially decimate an acquired animated image IM so as to obtain a decimated animated image IMD of lower resolution illustrating a background.
  • the shrinking 103 can be carried out by bilinear or bicubic filtering, which is well known to a person skilled in the art and makes it possible to reduce the number of pixels of the animated image initially acquired by the image sensor CI.
  • each calculated point is dependent on a zone of neighbouring points and not on a single point. This results in a smoothing of the image and prevents a loss of information in the case of a reduction of the image dimensions.
  • This type of decimated animated image IMD shows a theatre of operations (also called a "background"), such as a war scene, in its entirety.
  • the decimated animated image IMD may show the main hall of a railway station.
  • this decimated animated image IMD (background) does not make it possible to identify with precision an object in the image displayed subsequently at the remote station SDT.
  • an encoder ENC encodes the decimated animated image IMD.
  • the encoding process makes it possible to compress the decimated animated image IMD and therefore to still reduce the size thereof.
  • the decimated animated image IMD can be encoded by video coding.
  • this video coding may be in accordance with standard H264 (4:2:0, 8 bits) or VP8.
  • creation means M_CREA create a window F of fixed and configurable size, comprising at least one region of interest ROI.
  • a window F is also called a mosaic.
  • said at least one region of interest ROI is a region of interest by default and is created by a cropping or shrinking method, well known to a person skilled in the art.
  • the mosaic is thus an ROI vector, which can contain a plurality of regions of interest ROI of variable size and position.
  • selection means M_SELECT located at the drone E in the example illustrated in figure 1 ) select at least one region of interest ROI of variable size and position.
  • the selection 106 can be carried out manually, that is to say by an operator working from the remote station SDT and transmitting the coordinates of the regions of interest to the drone, or automatically, that is to say by an image processing algorithm installed in the drone E.
  • the selection 106 is carried out on the basis of the source animated image or on the basis of the content of the decimated animated image of lower resolution illustrating a background.
  • the selection 106 makes it possible to determine the position of the region(s) of interest in the decimated animated image of lower resolution or in the animated image not yet decimated. These positions are given for a given moment, corresponding to a given image of the video sequence.
  • the region(s) of interest is/are thus extracted from the source animated image (acquired animated image not yet decimated) corresponding to this moment.
  • the region(s) of interest is/are extracted from the same animated image as that of the background, this makes it possible to obtain coherence between the two bits of information - background and region of interest - which will be superposed before display on a screen at the remote station SDT (most conventional example of identical temporal decimation over the background and the regions of interest).
  • each of the regions of interest ROI can be of variable position, that is to say fixed or mobile.
  • its position may be dependent on the position of an object in the image, such as a car or a person.
  • This implementation called “tracking”, is well known to a person skilled in the art.
  • the method 100 for compressing images may comprise a step of shrinking of the regions of interest. This step 107 makes it possible to reduce slightly the size of the regions of interest so as to be contained within the window F of fixed size.
  • step 108 grouping means M_RGROUP group the regions of interest ROI in the window F.
  • the regions of interest ROI of variable size and position are grouped in the window F.
  • the regions of interest ROI are of variable size and position, but since the size of the window F is fixed, the rate of transfer of the sequence of the window F via the transfer link will be constant.
  • the grouping is carried out by a method well known to a person skilled in the art, called a "back-pack" method.
  • Basic implementation may consist in cutting the mosaic into squares of equal size as a function of the number of selected regions of interest, the aim being that each region of interest is contained in a square.
  • the size of the mosaic makes it possible to determine the target constant rate of flow to be obtained for a given image quality restored at the remote station SDT.
  • the encoder ENC encodes the window F.
  • the window F can be encoded by video coding.
  • this video encoding is carried out in accordance with standard H264. This makes it possible to benefit from the efficiency of a video codec (which utilises the temporal redundancy of the signal in a manner well known to a person skilled in the art), which is not normally possible with regions of interest of variable size. The bandwidth is thus limited since a lower rate of flow is generated.
  • the video encoding of the window F can be carried out at constant rate of flow or at variable rate of flow.
  • a multiplexer MUX multiplexes the encoded window FC and the encoded decimated animated image IMDC, for example in accordance with the data packet transfer standard MPEG2-TS.
  • Metadata are also multiplexed with the encoded window FC and the encoded decimated animated image IMDC during this multiplexing step 1 10.
  • these metadata may relate to:
  • the steps of multiplexing the encoded decimated animated image IMDC, the encoded window FC, and the metadata make it possible to combine independent signals so as to form a single signal, from which they can be restored in a demultiplexing step.
  • a transmitter TX transmits the multiplexed encoded decimated animated image IMDC and the multiplexed encoded window FC, and the multiplexed metadata to a remote station SDT.
  • a transmitter TX installed in the drone E transmits the encoded decimated animated image IMDC and the encoded window FC via a satellite or terrestrial link having, for example, a transfer rate between 2 and 10 Mbit/s. Since the animated image acquired initially by the image sensor CI is decimated and encoded, its size is reduced and can therefore be transmitted easily via the transfer link (also called a transfer channel). Likewise, since the window is of reduced size and is also encoded, it can easily be transmitted via the transfer link.
  • a twelfth step 112 (denoted TXJNT in figure 2), an instantaneous image zone is transmitted from the image sensor CI to the remote station SDT.
  • this instantaneous transmission 1 12 is triggered manually by an operator transmitting a request by means of the remote station SDT or is triggered automatically by image processing carried out at the drone E, for example by searching for an object moving in the scene, utilising the capacities of the encoder ENC, which itself searches for movement for the encoding process.
  • Such an instantaneous transmission is known by the term "snapshot" and makes it possible to transmit the entire acquired image, that is to say the source image, even if the signal comprises more than 8 bits per component (a component being luminance or chrominance) and is in (4:2:2) or (4:4:4) format. Thanks to this snapshot, the operator instantaneously obtains a precise image, possibly with no loss in terms of sensor performance.
  • the image zone transmitted instantaneously is encoded by an image coding process in accordance with the JPEG standard.
  • the image (whether encoded or not) is transmitted at a constant rate of flow.
  • the image zone transmitted instantaneously is multiplexed with the background images, the windows and the metadata and is also transmitted in the same transfer link as that devoted to the transmission of the background images and of the windows.
  • step of instantaneous transmission 1 1 2 can be carried out prior to, in parallel with, or subsequent to the following steps:
  • the typical gain compared to uniform compression H264 is approximately 2. This is with shrinking of approximately 40 (good comprehension of the scene maintained) and a mosaic size equivalent to an SD image (large surface available for the regions of interest ROI).
  • approximately 3 Mbits/s with H264 and 1 .5 Mbits/s with selective compression are required to encode 1080p@12.5 fps.
  • the cumulative surface of the ROIs is not very significant, it is possible to reduce the size of the mosaic and thus to reduce considerably the overall rate of flow. Beneficial functions can thus be obtained, even with a few hundred kbits/s, and even with an HD video at the input.
  • the transmission rate (also called overall rate of flow) can be calculated as follows: (F+M+D) * (1 +p)
  • the output stream rate of the encoder ENC is formed
  • the rate of flow of the metadata is linked to the implementation. Typically, this rate is a hundred octets per encoded animated image.
  • the rate of flow linked to the encapsulation is a percentage of the rates of flow of the transmitted data. Again, this depends on the implementation. In an MPEG2-TS/U DP/IP configuration, this is typically less than 15 %.
  • the rate of flow of the decimated animated image (that is to say the background image) or of the image of the mosaic is calculated by applying a rule of 3 compared to a reference rate of flow providing the desired visual quality for a certain known pixel rate at the input, called the reference rate.
  • a rule of 3 compared to a reference rate of flow providing the desired visual quality for a certain known pixel rate at the input, called the reference rate.
  • 6Mbits/s can typically be taken as a reference for 1080p@25fps.
  • the number of pixels per second for the decimated animated image (or the image of the mosaic) is used and compared to the reference rate.
  • the invention also relates to a computer program PGC (with reference to figure 1 ) for compressing images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of sequences of instructions making it possible to implement the method 100 for compressing animated images according to the invention, when said program is loaded onto a computer.
  • PGC computer program for compressing images
  • Figure 3 shows a schematic illustration of a device DD for decompressing animated images according to the invention.
  • this device DD for compressing animated images is included in the remote station SDT.
  • the device DD for decompressing animated images comprises:
  • a receiver RX adapted to receive an encoded decimated image IMDC illustrating a background, an encoded window FC comprising at least one region of interest, and possibly metadata, said encoded decimated image IMDC, said encoded window FC and, where applicable, said metadata being multiplexed;
  • demultiplexer DEMUX adapted to demultiplex the encoded window FC, the encoded decimated animated image IMDC, and possibly the metadata;
  • decoder DEC adapted to decode the encoded window FC and adapted to decode the encoded decimated animated image IMDC;
  • - means MJJNGROUP for ungrouping at least one region of interest ROI in the decoded window FD; - means M_UPS for increasing the resolution of the decoded decimated animated image IMDD;
  • M_SUP for superposing the at least one region of interest ROI and the decoded decimated animated image IMDD.
  • the device DD for decompressing images is adapted to implement a method 200 for decompressing animated images according to the invention, as described hereinafter on the basis of figures 3 and 4.
  • the method 200 for decompressing animated images comprises the following steps:
  • an encoded decimated animated image IMDC illustrating a background, an encoded window FC comprising at least one region of interest ROI, and metadata (denoted RXJMDC&FC in figure 4) via the receiver RX, said encoded decimated animated image IMDC, said encoded window FC, and said metadata being multiplexed;
  • the decompression method 200 comprises a step 204 of ungrouping at least one region of interest ROI in the decoded window FD (denoted DEGRP_ROI_FD in figure 4).
  • the steps of the method 200 for decompressing animated images are described in greater detail hereinafter. The description below also includes the steps of additional, non-limiting embodiments.
  • the receiver RX receives an encoded decimated animated image IMDC illustrating a background and an encoded window FC comprising at least one region of interest ROI, said encoded decimated animated image IMDC and said encoded window FC being multiplexed.
  • the metadata are also multiplexed, as described previously, with said encoded decimated animated image IMDC and said encoded window FC.
  • a demultiplexer DEMUX demultiplexes the multiplexed encoded decimated image IMDC, the multiplexed encoded window FC, and the metadata. This step makes it possible to restore the independent signals which were previously combined so as to facilitate the transport process.
  • a third step 203 the decoder DEC at the remote station SDT decodes the encoded window FC to obtain the decoded window FD.
  • the ungrouping means M-DEGROUP ungroup said at least one region of interest ROI in the decoded window FD.
  • This step is also called mosaicing.
  • the ungrouping 204 of at least one region of interest in the decoded window FD is carried out via metadata. Said metadata make it possible in particular to reposition in space said at least one region of interest within the decoded decimated animated image IMDD.
  • these metadata may relate to:
  • the window • the size of the window; the date ("time stamp") of the acquired animated image and that of the region(s) of interest contained in the window so as to combine the correct background image with the correct region(s) of interest (the time stamp of the background animated image is contained in the bit stream containing the background video);
  • the orientation of the line of vision of the animated image sensor CI the orientation of the line of vision of the animated image sensor CI ; the estimated distance between the image sensor CI and the target;
  • the decoder DEC decodes the encoded decimated animated image IMDC.
  • the decoded decimated animated image IMDD is also subjected to a step of reverse temporal decimation.
  • the decoded window FD is also subjected to a step of reverse temporal decimation.
  • the means M_UPS for increasing resolution increase the resolution of the decoded decimated animated image IMDD ("up- scaling").
  • the increase in resolution may be carried out by bilinear or bicubic filtering, which is well known to a person skilled in the art.
  • Such an increase in resolution consists in taking a predefined matrix and calculating each point of the resultant image by application of this matrix to the source image zone corresponding to the point of the resultant image.
  • the superposition means M_SUP superpose the at least one region of interest ROI and the decoded decimated animated image IMDD.
  • These superposition means M_SUP are software means.
  • the superposition of the at least one region of interest ROI and of the decoded decimated animated image IMDD makes it possible to obtain a resultant image comprising a high-definition zone/one of the high-definition zones (the region(s) of interest extracted from the window) illustrating precisely identifiable elements and smoothed zones of low resolution (the rest of the image not comprising any elements needing to be identified precisely).
  • the image is also displayed on a screen (not illustrated) at the remote station SDT so that the operator can view it.
  • the receiver RX receives an instantaneous image zone originating from the image sensor CI following a request emitted by the remote station SDT or following a request emitted automatically via an image processing procedure implemented at the compression device DC.
  • This image zone is multiplexed with the encoded decimated animated image IMDC and the encoded window FC.
  • the demultiplexer DEMUX demultiplexes the instantaneous image zone IMJNT.
  • This image zone is stored in an image zone bank (or "snapshot").
  • This image zone is also displayed (after possible decoding) on a screen (not illustrated) at the remote station SDT so that the operator can view it.
  • the invention also relates to a computer program PGD (illustrated in figure 3) for decompressing images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of sequences of instructions making it possible to implement the method 200 for decompressing animated images according to the invention, when said program is loaded onto a computer.
  • PGD illustrated in figure 3
  • the invention also relates to a system 300 for compressing and decompressing animated images, said system comprising the device DC for compressing animated images according to the invention and the device DD for decompressing animated images according to the invention.
  • This system 300 for compressing and decompressing is illustrated in figure 1 .
  • the invention also relates to a method 400 (see figure 5) for compressing and decompressing animated images, said method comprising the following steps:
  • the remote station SDT comprises a configuration interface IHM allowing an operator to configure, for example, the following parameters and to view, for example, the results of the configuration of the parameters directly on the screen at the remote station SDT:
  • quality is a ratio applied to a reference rate of flow, said rate of flow corresponds to the minimum rate of flow required to obtain correct visual quality under standard conditions of image complexity (neither too simple nor too complex). Typically 6Mbits/s for 1080p@25fps with H264 video coding;
  • o estimated total rate background, window, and metadata
  • the invention is in no way limited to the drone application presented above. It can therefore also be implemented in a video surveillance application.
  • the methods for compressing 100 and decompressing 200 animated images advantageously make it possible to replace a camera of the SD type (standard definition) currently used with a camera of the HD type (or any other resolution greater than SD), implementing a uniform compression technique using certain existing SD compression resources.
  • “Existing resources” is understood to mean existing infrastructure resources, such as:
  • IP type Internet Protocol
  • the SD camera used in the prior art is replaced by a compression device DC, the compression device DC comprising: - an animated image sensor CI, for example of the HD type, adapted to acquire animated images,
  • decimation means M_DECIM able to shrink an animated image acquired by the image sensor so as to obtain a decimated animated image IMD of lower resolution illustrating a background, such as the main hall of a station,
  • - multiplexing means MUX for multiplexing the window and the decimated animated image.
  • an encoder ENC adapted to code the decimated animated image and adapted to code the window comprising the at least one region of interest, the decimated animated image and the window being multiplexed;
  • TX a transmitter TX adapted to transmit the multiplexed encoded window and the multiplexed encoded decimated animated image
  • an IP network adapted to allow circulation of the information between the transmitter TX and a receiver RX;
  • the receiver RX is adapted to receive the multiplexed encoded decimated image and the multiplexed encoded window
  • decoder DEC adapted to decode the encoded window and adapted to decode the encoded decimated animated image.
  • a decompression device DD is added to the existing infrastructure and comprises:
  • DEMUX demultiplexer DEMUX adapted to demultiplex the decoded window and the decoded decimated animated window
  • the compression device 100 implements the following steps:
  • a window F also called a mosaic
  • a window F also called a mosaic
  • the existing infrastructure means implement the following steps:
  • the decompression device DD implements the following steps:
  • This superposition is displayed, for example, on a screen ECR at the remote station SDT so that the operator can view it.
  • a control channel Eth of the Ethernet type can be used to transmit instantaneous images (or "snapshots"), this channel also being used to transmit the metadata. This mode is used, for example, when a low transmission rate is used for the instantaneous images.
  • this control channel Eth allows a user of a control console to enter the parameters of the compression of the animated images, these parameters being described previously in the above embodiments.
  • the compression device thus makes it possible to control the SD-type encoder ENC.
  • a transmission channel parallel to the animated images can be used to transmit instantaneous images (or "snapshots").
  • this parallel transmission channel is not obligatory.
  • the instantaneous images can be transported in the window cut into small pieces, and aggregated in the regions of interest ROI and in the animated image. The transmission of the entire instantaneous image may be spread over a plurality of images.
  • a transmission channel parallel to the transmission channel of the animated images is desirable for transmission of the metadata (size and position of the region of interest, shrinking, and temporal decimation, ).
  • these metadata which are numerical values, are not naturally encoded without degradation through a video encoder.
  • One approach may consist in transferring an octet over the luminance components of two pixels and of only using the 4 most significant bits of these pixels; care should also be taken to ensure that a small variation in the value of the pixel does not have an effect on the 4 most significant bits.
  • two luminance components of 0x48 and 0x28 are used to convey a metadata 0x42. Even if the encoder introduced an error of +/- 7 LSB, it is thus still possible to regain the metadata values intact. It is noted that, if the rate of flow of the compressed animated image is very low and the visual quality is also low, 3 or even 2 most significant bits may be used for transmission of the metadata, and each octet may be spread over 3 or 4 components.
  • this image is transmitted in parallel with the encoded decimated animated images. To do this, it is necessary for an additional transmission channel to be available. However, this additional channel is only necessary in this specific embodiment of video surveillance using existing SD compression resources.
  • the use of a large region of interest ROI allows the operator to view with precision a large part of the theatre of operations (of the background), which allows him to easily track an object moving within the theatre of operations, and maintaining a constant rate of flow thanks to the fixed size of the window and of the decimated animated image.
  • a terrestrial or satellite channel (or link) of fixed size can thus be utilised in spite of the variability in size, position and number of the region of interest ROI.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention relates to a method for compressing animated images and to a corresponding method for decompressing animated images. More specifically, the method 100 for compressing animated images comprises the following steps in particular:• - acquisition (101) of an animated image (IM) by means of an image sensor (CI); • - shrinking (103) of said acquired animated image (IM) so as to obtain a decimated animated image (IMD) of lower resolution illustrating a background; • - encoding (104) of said decimated animated image (IMD) by means of an encoder (ENC_IMD); • - creation (105) of a window (F) of fixed and configurable size comprising at least one region of interest (ROI); • - encoding (109) of said window comprising said at least one region of interest (ROI) by means of said encoder (ENC_F); • - multiplexing (110) of said encoded decimated animated image (IMDC) and of said encoded window (FC); • - transmission (111) of said multiplexed encoded decimated animated image (IMDC) and of the multiplexed encoded window (FC) to a remote station (SDT).

Description

METHODS FOR COMPRESSING AND DECOMPRESSING ANIMATED
IMAGES
TECHNICAL FIELD OF THE INVENTION
The invention relates to a method for compressing animated images and to a corresponding method for decompressing animated images.
TECH N I CAL BACKG RO U N D O F TH E I NV ENTI ON
Such methods are implemented in drones for example. Information is acquired via image sensors, infrared sensors and electro-optical sensors installed in the drone so as to be transmitted in real time to a remote station via a terrestrial or satellite data transfer link.
The information acquired by the sensors generally has a rate of flow greater than 1 Gbit/s. Each terrestrial or satellite link attributed to a drone normally has a rate of flow between 2 and 10 Mbit/s, this link being shared by the different sensors. In this instance, the acquired information is compressed.
However, even by compressing all the acquired information using standard video compression techniques, the size of the information is not sufficiently reduced to be transmitted in real time via the transfer link.
To overcome this drawback, a known technique consists in subjecting the acquired animated images to temporal decimation, that is to say, instead of transmitting 25 images/s, no more than 4 images/s are transmitted for example. However, it is no longer possible to correctly perceive movement beyond a certain pace. Moreover, the operator viewing the animated images in real time becomes tired. On this basis, the object of the invention is to propose a method for compressing animated images and a method for decompressing animated images, making it possible to reduce the transmission rate of the acquired animated images whilst still maintaining good fluidity of movement and/or an optimum level of detail over those zones of the animated image considered to be of interest.
G E N E RAL D ESC R I PTI ON O F TH E I N VE NTI ON To this end, the invention relates to a method for compressing animated images, said method comprising the following steps:
- acquisition of an animated image by means of an image sensor;
- shrinking of said acquired animated image so as to obtain a decimated animated image of lower resolution illustrating a background;
- encoding of said decimated animated image by means of an encoder;
- creation of a window of fixed and configurable size comprising at least one region of interest;
- encoding of said window comprising said at least one region of interest by means of said encoder;
- multiplexing of said encoded decimated animated image and of said encoded window;
- transmission of said encoded decimated animated image and of said multiplexed encoded window to a remote station.
Thanks to the method for compressing animated images of the invention:
- an entire animated image is decimated then encoded (that is to say compressed), this feature making it possible to obtain an animated image of which the volume is highly reduced by means of the shrinking and encoding processes;
- the window comprising at least one region of interest is encoded uniquely and is not decimated. The region(s) of interest (ROI) contained in the window thus maintain(s) optimum definition. Since the window only represents part of the acquired animated image, the size of this window once encoded is relatively small, especially since the fixed size of the window makes it possible to use a video codec, which is more efficient than an image codec.
The transmission of the encoded decimated animated image and of the encoded window to the remote station thus uses a reduced rate of flow of a data transfer link (also called a channel). The method for compressing animated images according to the invention may also have one or more of the features below, taken individually or in any technically feasible combination.
In a non-limiting embodiment of the compression method, the method comprises the following additional steps:
- selection of at least one region of interest of variable size and position;
- grouping said at least one region of interest in said window. This makes it possible to allocate a sub-bandwidth of constant rate of flow to each image sensor.
In a non-limiting embodiment of the compression method, said selection is made on the basis of said animated image or on the basis of the content of said decimated animated image of lower resolution illustrating a background.
In a non-limiting embodiment of the compression method, said selection of at least one region of interest is made by a cropping or shrinking method.
In a non-limiting embodiment of the compression method, said selection is made manually at the remote station or automatically at the encoder.
In a non-limiting embodiment of the compression method, metadata are multiplexed with the encoded decimated animated image and the encoded window during said multiplexing step.
In a non-limiting embodiment of the compression method, each animated image is encoded by video encoding. A better image resolution compared to image coding is thus achieved at iso rate of flow.
In a non-limiting embodiment of the compression method, the window is encoded by video coding. A better encoding efficiency and therefore, at iso rate of flow, a better resolution of the region of interest are thus obtained compared to image encoding.
In a non-limiting embodiment of the compression method, said encoding of the window takes place at constant rate of flow.
In a non-limiting embodiment of the compression method, the method comprises an additional step of shrinking of said at least one region of interest before the grouping step. This makes it possible to reduce the size of those regions of interest which are too large compared to the configured size of the window. In a non-limiting embodiment of the compression method, the method comprises a step of temporal decimation of the animated images acquired by said image sensor. This makes it possible to reduce the rate of flow of the transmitted images.
In a non-limiting embodiment of the compression method, the method comprises an additional step of instantaneous transmission of an image zone from said image sensor to said remote station. The image zone can be formed by the entire acquired image or by part of the acquired image. This makes it possible to transmit a high-definition image of the entire scene acquired by the image sensor. This is especially beneficial in particular if the video encoding method used limits the number of bits per pixel component or else the chromatic definition.
In non-limiting embodiment of the compression method, said image zone transmitted instantaneously is encoded in accordance with image encoding.
In a non-limiting embodiment of the compression method, said image zone is multiplexed with the encoded decimated animated image and the encoded window during said multiplexing step.
In a non-limiting embodiment of the compression method, the instantaneous transmission of an image zone of said image sensor is carried out at constant rate of flow.
In a non-limiting embodiment of the compression method, said image sensor is installed in an aerial, terrestrial or naval mobile device.
In a non-limiting embodiment of the compression method, said image sensor is fixed.
The invention also relates to a computer program for compressing animated images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of said sequences of instructions enabling implementation of the method for compressing animated images according to the invention, when said program is loaded onto a computer.
The invention also relates to a device for compressing animated images, said device for compressing animated images comprising: - an image sensor adapted to acquire animated images;
- decimation means able to shrink the animated image acquired by said image sensor so as to obtain a decimated animated image of lower resolution illustrating a background;
- an encoder adapted to code said decimated animated image;
- means for creating a window of fixed and configurable size comprising at least one region of interest (ROI);
- said encoder is also adapted to code said window comprising said at least one region of interest;
- means for multiplexing said encoded window and said encoded decimated animated image;
- a transmitter adapted to transmit the multiplexed encoded window and the multiplexed encoded decimated animated image.
The invention also relates to a method for decompressing animated images, said method comprising the following steps:
- receiving, via a remote station, an encoded decimated animated image illustrating a background and an encoded window comprising at least one region of interest, said encoded decimated animated image and said encoded window being multiplexed;
- demultiplexing said encoded decimated animated image and said encoded window;
- decoding said encoded window by means of a decoder at said remote station;
- decoding said encoded decimated animated image by means of said decoder;
- increasing the resolution of said decoded decimated animated image;
- superposing said at least one region of interest and said decoded animated image.
Thanks to the method for decompressing animated images of the invention, the superposition of said at least one region of interest and said decoded animated image makes it possible to obtain a resultant image comprising a zone of high definition (the region of interest) illustrating precisely identifiable elements and zones of low resolution (a region not comprising an element needing to be identified precisely).
Furthermore, the integration (or superposition) of the decimated animated image with the window makes it possible to have only a single window to be encoded. The background image thus does not need a specific encoding channel and can use the same as that of the window. This feature also makes it possible to reduce the number of encoding procedures.
Similarly, the integration of the decimated animated image with the window makes it possible to have only a single window to be decoded. The background image thus does not need a specific decoding channel and can use the same as that of the window. This feature also makes it possible to reduce the number of decoding procedures.
In a non-limiting embodiment of the decompression method, the method comprises an additional step of ungrouping at least one region of interest in said window as a function of the metadata. In a non-limiting embodiment of the decompression method, said method comprises an additional step of instantaneous reception of an image zone originating from an image sensor.
The invention also relates to a device for decompressing animated images, said device for decompressing images comprising:
- a remote station adapted to receive an encoded decimated animated image illustrating a background and an encoded window comprising at least one region of interest; said encoded decimated animated image and said encoded window being multiplexed;
- a demultiplexer adapted to demultiplex the multiplexed encoded window and the multiplexed encoded decimated animated image; - a decoder adapted to decode said encoded window and adapted to decode said encoded decimated animated image;
- means for increasing the resolution of said decoded decimated animated image;
- means for superposing said at least one region of interest and said decoded animated image.
The invention also relates to a computer program for decompressing animated images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of said sequences of instructions enabling implementation of the method for decompressing animated images according to the invention, when said program is loaded onto a computer. The invention also relates to a system for compressing and decompressing animated images, said system comprising the device for compressing images according to the invention and the device for decompressing animated images according to the invention.
The invention also relates to a method for compressing and decompressing animated images, comprising the following steps:
- acquisition of an animated image by means of an image sensor;
- shrinking of said acquired animated image so as to obtain a decimated animated image of lower resolution illustrating a background;
- encoding of said decimated animated image by means of an encoder;
- creation of a window of fixed and configurable size comprising at least one region of interest;
- encoding of said window comprising said at least one region of interest by means of said encoder;
- multiplexing of said encoded decimated animated image and of said encoded window; - transmission of said multiplexed encoded decimated animated image and of the multiplexed encoded window to a remote station;
- reception by said remote station of said multiplexed encoded decimated animated image and of said multiplexed encoded window;
- demultiplexing of said encoded decimated animated image and of said encoded window;
- decoding of said encoded window by means of a decoder of said remote station;
- decoding of said encoded decimated animated image by means of said decoder;
- increase of the resolution of said decoded decimated animated image;
- superposition of said at least one region of interest and said decoded animated image.
BRIEF DESCRIPTION OF THE FIGURES
Further features and advantages of the invention will emerge clearly from the non-limiting description given hereinafter by way of indication and with reference to the accompanying figures, in which:
- figure 1 shows a device for compressing animated images according to the invention in accordance with a non-limiting embodiment;
- figure 2 shows a block diagram of the steps of the method for compressing animated images according to the invention in accordance with a non-limiting embodiment;
- figure 3 shows a device for decompressing animated images according to the invention in accordance with a non-limiting embodiment;
- figure 4 shows a block diagram of the steps of the method for decompressing animated images according to the invention in accordance with a non-limiting embodiment;
- figure 5 shows a method for compressing and decompressing animated images according to the invention; - figure 6 shows a device for compressing animated images and a device for decompressing animated images according to the invention, used in a non-limiting video surveillance application;
- figure 7 shows a method for compressing and a method for decompressing animated images according to the invention implemented by the devices for compressing and decompressing animated images illustrated in figure 6.
DESCRIPTION OF EMBODIMENTS OF THE INVENTION
For reasons of clarity, only those elements key to comprehension of the invention have been illustrated, more specifically not to scale and in a schematic manner.
Figure 1 shows a schematic view of a compression device DC for compressing animated images. In particular, the compression device DC comprises:
- an image sensor CI adapted to acquire animated images IM, said image sensor CI possibly being, for example, a camera installed in a device E, for example an aerial drone;
- decimation means M_DECIM able to shrink an animated image IM acquired by the image sensor CI so as to obtain a decimated animated image IMD of lower resolution illustrating a background (or theatre of operations);
- an encoder ENC adapted to code the decimated animated image IMD;
- selection means M_SELECT for selecting at least one region of interest ROI of variable size and position;
- creation means M_CREA for creating a window F of fixed and configurable size;
- grouping means M_REGROUP for grouping the at least one region of interest ROI selected in the window F;
- the encoder ENC is also adapted to code the window F comprising the at least one region of interest ROI ; - multiplexing means MUX for multiplexing the encoded window FC and the encoded decimated animated image IMDC;
- a transmitter TX adapted to transmit the multiplexed encoded window FC and the multiplexed encoded decimated animated image IMDC.
The device DC for compressing animated images is adapted to communicate with a remote station SDT so as to transmit thereto the encoded decimated animated image IMDC and the encoded window FC, both of which are multiplexed MUX (FC&IMDC).
"Animated images" is understood to mean images which, when placed in sequence and displayed at regular interval, reproduce the activity and movements in a scene. Such a sequence is also called video. If the frequency of the images is high (that is to say the interval between two images is short), typically a few tens of Hertz, the content of two consecutive images in the sequence is very close. This small difference in content between two consecutive images is utilised by the video encoders with the aim of optimising efficiency in terms of compression rate.
The device DC is adapted to implement a method 100 for compressing animated images according to the invention and is described hereinafter with reference to figures 1 and 2. The method 100 for compressing animated images comprises the following steps:
- acquisition of animated images 101 (denoted ACJM in figure 2) by means of an image sensor CI ;
- shrinking 103 (denoted DC_S in figure 2) of an acquired animated image IM so as to obtain a decimated animated image
IMD of lower resolution illustrating a background;
- encoding of the decimated animated image 104 (denoted ENCJMD in figure 2) by means of an encoder ENC; - creation 105 of a window F (also called a mosaic) of fixed and configurable size comprising at least one region of interest (denoted CREA_F in figure 2);
- encoding of the window 109 (denoted ENC_F in figure 2) comprising the at least one region of interest by means of the encoder ENC;
- multiplexing 1 10 of the encoded decimated animated image and of the encoded window FC (denoted MUXJMDC&FC);
- transmission 1 1 1 of the multiplexed encoded decimated animated image and of the multiplexed encoded region of interest (denoted TX_MUX(IMDC&FC) in figure 2) to the remote station SDT.
In a non-limiting embodiment, the compression method comprises the following additional steps:
- selection 106 of at least one region of interest ROI of variable size and position (denoted SELECT_ROI in figure 2);
- grouping 108 of said at least one region of interest ROI in said window F (denoted RGP_ROI in figure 2).
In a non-limiting embodiment, the compression method comprises an additional step of shrinking 107 of said at least one region of interest before the grouping step (denoted DS-S-ROI in figure 2).
In a non-limiting embodiment, the compression method comprises a step of temporal decimation 102 of the animated images IM acquired by said image sensor CI (denoted DC-TP in figure 2).
In a non-limiting embodiment, the compression method comprises an additional step of instantaneous transmission 1 12 of an image zone from said image sensor CI to said remote station SDT (denoted TX-INT in figure 2). The steps of the method 100 for compressing animated images are described in greater detail hereinafter. The description below also includes the steps of additional, non-limiting embodiments. In a first step 101 (denoted ACJM in figure 2), the image sensor CI acquires animated images IM.
In a non-limiting manner, the image sensor CI is a high-definition camera allowing acquisition of 25 images per second. In different, non-limiting embodiments, the image sensor CI can acquire images at a different rate, for example 30, 50 or 60 images per second.
In a non-limiting embodiment, this image sensor CI can be installed in an aerial, terrestrial or naval mobile device E so as to visualise theatres of operation, such as war scenes, and/or so as to follow a mobile vehicle
Each animated image acquired by the image sensor CI (also called a source image) is stored with its date of acquisition in a memory (also called a buffer) located in the drone E. This buffer makes it possible to store a history of the last images acquired. This history makes it possible to process orders, received by the remote station, for instantaneous image request and for tracking of the region of interest (tracking of the ROI). In fact, if an operator at the remote station orders a request for an image N, said order generally arrives at the encoder a few moments (seconds) after acquisition of the image N by the sensor. Storage in the buffer of the image N after acquisition makes it possible to keep the image N accessible for processing and possible transmission to the remote station.
Likewise, if the operator at the remote station orders the tracking of a region of interest ROI on an element, also called an object, such as a car or person, of the scene present at a certain position in the image N, it is then necessary, upon receipt of the order, to scour through the last images acquired and stored in the buffer since acquisition of the image N so as to analyse these images and to track the position of this object cited by the operator in the series of images. The position of this object in the last image acquired thus defines the current position of the ROI.
In a different, non-limiting embodiment, the animated image sensor CI can be fixed so as to carry out video surveillance, for example in a railway station.
In a second step 102 (denoted DC_TP and illustrated by a dotted line in figure 2), the compression method 100 comprises a step of temporal decimation 102 of the animated images acquired by the image sensor CI. For example, 12.5 animated images per second will be received. This step makes it possible to reduce the number of animated images transmitted. It thus follows that this step of temporal decimation 102 makes it possible to reduce the rate of transfer.
A temporary increase in the temporal decimation can be implemented with the aim of transmitting an instantaneous image at a constant overall rate of flow. To this end, the pace of the encoded animated images is reduced by implementing temporal decimation: typically, if one animated image in 2 is encoded, no more than one animated image in 4 will be encoded, and therefore 50 % (=2/4) bandwidth is recovered for instantaneous transmission. It may also be decided to switch to one animated image in 10, in which case 80 % (8 in 10) bandwidth is recovered. The advantage of this mechanism is twofold:
- transmission at a constant overall rate of flow (background + window comprising at least one region of interest + zone of image transmitted instantaneously in the same sub-bandwidth), and
- transmission of the entire source signal, even if the signal comprises more than 8 bits/component and is in 4:2:2 or 4:4:4 format - a function rarely supported by video encoders.
It will be noted that this temporal decimation is the least disruptive for the operator viewing the video feed at the remote station SDT.
In a third step 103 (denoted DC_S in figure 2), decimation means M_DECIM spatially decimate an acquired animated image IM so as to obtain a decimated animated image IMD of lower resolution illustrating a background.
In a non-limiting manner, the shrinking 103 can be carried out by bilinear or bicubic filtering, which is well known to a person skilled in the art and makes it possible to reduce the number of pixels of the animated image initially acquired by the image sensor CI.
Due to the use of a matrix measuring greater than 1 x1 (conventionally 5x5 or greater), each calculated point is dependent on a zone of neighbouring points and not on a single point. This results in a smoothing of the image and prevents a loss of information in the case of a reduction of the image dimensions.
This type of decimated animated image IMD shows a theatre of operations (also called a "background"), such as a war scene, in its entirety. In another non-limiting example, the decimated animated image IMD may show the main hall of a railway station.
In other words, this decimated animated image IMD (background) does not make it possible to identify with precision an object in the image displayed subsequently at the remote station SDT.
In a fourth step 104 (denoted ENC IMD in figure 2), an encoder ENC encodes the decimated animated image IMD. The encoding process makes it possible to compress the decimated animated image IMD and therefore to still reduce the size thereof.
In a non-limiting manner, the decimated animated image IMD can be encoded by video coding. By way of non-limiting example, this video coding may be in accordance with standard H264 (4:2:0, 8 bits) or VP8.
In a fifth step 105 (denoted CREA_F in figure 2), creation means M_CREA create a window F of fixed and configurable size, comprising at least one region of interest ROI. Such a window F is also called a mosaic.
In non-limiting embodiments, said at least one region of interest ROI is a region of interest by default and is created by a cropping or shrinking method, well known to a person skilled in the art.
It will be noted that the mosaic is thus an ROI vector, which can contain a plurality of regions of interest ROI of variable size and position.
In a sixth step 106 (denoted SELECT_ROI in figure 2), selection means M_SELECT (situated at the drone E in the example illustrated in figure 1 ) select at least one region of interest ROI of variable size and position.
The selection 106 can be carried out manually, that is to say by an operator working from the remote station SDT and transmitting the coordinates of the regions of interest to the drone, or automatically, that is to say by an image processing algorithm installed in the drone E. The selection 106 is carried out on the basis of the source animated image or on the basis of the content of the decimated animated image of lower resolution illustrating a background.
The selection 106 makes it possible to determine the position of the region(s) of interest in the decimated animated image of lower resolution or in the animated image not yet decimated. These positions are given for a given moment, corresponding to a given image of the video sequence.
The region(s) of interest is/are thus extracted from the source animated image (acquired animated image not yet decimated) corresponding to this moment. Thus, since the region(s) of interest is/are extracted from the same animated image as that of the background, this makes it possible to obtain coherence between the two bits of information - background and region of interest - which will be superposed before display on a screen at the remote station SDT (most conventional example of identical temporal decimation over the background and the regions of interest).
It should be noted that each of the regions of interest ROI can be of variable position, that is to say fixed or mobile. In fact, if the region of interest ROI is mobile, its position may be dependent on the position of an object in the image, such as a car or a person. This implementation, called "tracking", is well known to a person skilled in the art.
In a seventh step 107 (denoted by a dotted line DC_S_ROI in figure
2), if at least two regions of interest are selected, the method 100 for compressing images may comprise a step of shrinking of the regions of interest. This step 107 makes it possible to reduce slightly the size of the regions of interest so as to be contained within the window F of fixed size.
In an eighth step 108 (denoted RGP_ROI in figure 2), grouping means M_RGROUP group the regions of interest ROI in the window F.
This, if a plurality of regions of interest ROI is selected, the regions of interest ROI of variable size and position are grouped in the window F. The regions of interest ROI are of variable size and position, but since the size of the window F is fixed, the rate of transfer of the sequence of the window F via the transfer link will be constant. The grouping is carried out by a method well known to a person skilled in the art, called a "back-pack" method. Basic implementation may consist in cutting the mosaic into squares of equal size as a function of the number of selected regions of interest, the aim being that each region of interest is contained in a square.
It should be noted that the size of the mosaic makes it possible to determine the target constant rate of flow to be obtained for a given image quality restored at the remote station SDT. In a ninth step 109 (denoted ENC_F in figure 2), the encoder ENC encodes the window F.
In a non-limiting manner, the window F can be encoded by video coding. By way of non-limiting example, this video encoding is carried out in accordance with standard H264. This makes it possible to benefit from the efficiency of a video codec (which utilises the temporal redundancy of the signal in a manner well known to a person skilled in the art), which is not normally possible with regions of interest of variable size. The bandwidth is thus limited since a lower rate of flow is generated.
In a non-limiting embodiment, the video encoding of the window F can be carried out at constant rate of flow or at variable rate of flow.
In a tenth step 110 (denoted MUX IMDC&FC in figure 2), a multiplexer MUX multiplexes the encoded window FC and the encoded decimated animated image IMDC, for example in accordance with the data packet transfer standard MPEG2-TS.
In addition, metadata are also multiplexed with the encoded window FC and the encoded decimated animated image IMDC during this multiplexing step 1 10.
In a non-limiting manner, these metadata may relate to:
· the original size of the animated image so as to regain the original size thereof after decompression;
• temporal decimation of the background image;
• temporal decimation of the regions of interest;
• the size of the window; • the date ("time stamp") of the acquired animated image and that of the region(s) of interest contained in the window so as to combine the correct background image with the correct region(s) of interest (the time stamp of the background animated image is contained in the bit stream containing the background video);
• the number of regions of interest;
• for each region of interest:
• the size and position in the source animated image,
• the size and position in the window,
• the type of region of interest, that is to say fixed or mobile;
• the UTC time;
· the GPS position of the animated image sensor CI ;
• the orientation of the line of vision of the animated image sensor CI ;
• the estimated distance between the image sensor CI and the target;
· ...
The steps of multiplexing the encoded decimated animated image IMDC, the encoded window FC, and the metadata make it possible to combine independent signals so as to form a single signal, from which they can be restored in a demultiplexing step.
In an eleventh step 111 (denoted TX IMDC&FC in figure 2), a transmitter TX transmits the multiplexed encoded decimated animated image IMDC and the multiplexed encoded window FC, and the multiplexed metadata to a remote station SDT.
In a non-limiting manner, a transmitter TX installed in the drone E transmits the encoded decimated animated image IMDC and the encoded window FC via a satellite or terrestrial link having, for example, a transfer rate between 2 and 10 Mbit/s. Since the animated image acquired initially by the image sensor CI is decimated and encoded, its size is reduced and can therefore be transmitted easily via the transfer link (also called a transfer channel). Likewise, since the window is of reduced size and is also encoded, it can easily be transmitted via the transfer link.
It should be noted that these aforementioned steps of the compression method 100 are repeated for each animated image acquired by the image sensor CI. In other words, a sequence of animated images is formed of a plurality of windows.
In a twelfth step 112 (denoted TXJNT in figure 2), an instantaneous image zone is transmitted from the image sensor CI to the remote station SDT.
In a non-limiting example, this instantaneous transmission 1 12 is triggered manually by an operator transmitting a request by means of the remote station SDT or is triggered automatically by image processing carried out at the drone E, for example by searching for an object moving in the scene, utilising the capacities of the encoder ENC, which itself searches for movement for the encoding process.
Such an instantaneous transmission is known by the term "snapshot" and makes it possible to transmit the entire acquired image, that is to say the source image, even if the signal comprises more than 8 bits per component (a component being luminance or chrominance) and is in (4:2:2) or (4:4:4) format. Thanks to this snapshot, the operator instantaneously obtains a precise image, possibly with no loss in terms of sensor performance.
In a non-limiting manner, the image zone transmitted instantaneously is encoded by an image coding process in accordance with the JPEG standard. In a non-limiting embodiment, the image (whether encoded or not) is transmitted at a constant rate of flow.
It should be noted that the image zone transmitted instantaneously is multiplexed with the background images, the windows and the metadata and is also transmitted in the same transfer link as that devoted to the transmission of the background images and of the windows.
It should be noted that the steps of:
- temporal decimation 102;
- shrinking 103;
- encoding 104 of the decimated animated image;
can be carried out prior to, in parallel with, or subsequent to the steps of:
- creation 105 of a window F of fixed and configurable size;
- selection 106 of at least one region of interest ROI of variable size and position;
- shrinking 107 of the regions of interest;
- grouping 108 of the at least one region of interest ROI in the window F;
- encoding 109 of the window F.
Likewise, the step of instantaneous transmission 1 1 2 can be carried out prior to, in parallel with, or subsequent to the following steps:
- temporal decimation 102;
- shrinking 103;
- encoding 104 of the decimated animated image;
- creation 105 of a window F of fixed and configurable size;
- selection 106 of at least one region of interest ROI of variable size and position;
- shrinking 107 of the regions of interest.
- grouping 108 of the at least one region of interest ROI in the window F;
- encoding 109 of the window F. For example, it could be noted that, thanks to the compression method according to the invention, the typical gain compared to uniform compression H264 is approximately 2. This is with shrinking of approximately 40 (good comprehension of the scene maintained) and a mosaic size equivalent to an SD image (large surface available for the regions of interest ROI). Thus, under the conditions described above, approximately 3 Mbits/s with H264 and 1 .5 Mbits/s with selective compression are required to encode 1080p@12.5 fps. In applications in which the cumulative surface of the ROIs is not very significant, it is possible to reduce the size of the mosaic and thus to reduce considerably the overall rate of flow. Beneficial functions can thus be obtained, even with a few hundred kbits/s, and even with an HD video at the input.
The transmission rate (also called overall rate of flow) can be calculated as follows: (F+M+D)*(1 +p)
The output stream rate of the encoder ENC is formed
• of the rate of flow F of the decimated and encoded background image;
• of the rate of flow M of the encoded ROI mosaic;
· of the rate of flow D of the metadata;
• of the additional rate of flow p linked to the encapsulation of the network transmission protocols, for example MPEG2-TS, UDP, IP. This rate of flow is a percentage p applied to the transmitted data.
The rate of flow of the metadata is linked to the implementation. Typically, this rate is a hundred octets per encoded animated image.
The rate of flow linked to the encapsulation is a percentage of the rates of flow of the transmitted data. Again, this depends on the implementation. In an MPEG2-TS/U DP/IP configuration, this is typically less than 15 %.
The rate of flow of the decimated animated image (that is to say the background image) or of the image of the mosaic is calculated by applying a rule of 3 compared to a reference rate of flow providing the desired visual quality for a certain known pixel rate at the input, called the reference rate. With H264, 6Mbits/s can typically be taken as a reference for 1080p@25fps. To carry out the rule of 3, the number of pixels per second for the decimated animated image (or the image of the mosaic) is used and compared to the reference rate.
Once this rate of flow has been calculated, it is necessary to overestimate this rate a little so as to take into account the fact that the lower the number of pixels at the input, the less significant is the efficiency of the encoding (greater entropy, less temporal correlation, greater importance of the headers of fixed size, ...). This overestimation is dependent on the video encoder used and may require a phase of calibration.
The invention also relates to a computer program PGC (with reference to figure 1 ) for compressing images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of sequences of instructions making it possible to implement the method 100 for compressing animated images according to the invention, when said program is loaded onto a computer.
Figure 3 shows a schematic illustration of a device DD for decompressing animated images according to the invention. In a non-limiting embodiment, this device DD for compressing animated images is included in the remote station SDT.
In particular, the device DD for decompressing animated images comprises:
- a receiver RX adapted to receive an encoded decimated image IMDC illustrating a background, an encoded window FC comprising at least one region of interest, and possibly metadata, said encoded decimated image IMDC, said encoded window FC and, where applicable, said metadata being multiplexed;
- a demultiplexer DEMUX adapted to demultiplex the encoded window FC, the encoded decimated animated image IMDC, and possibly the metadata;
- a decoder DEC adapted to decode the encoded window FC and adapted to decode the encoded decimated animated image IMDC;
- means MJJNGROUP for ungrouping at least one region of interest ROI in the decoded window FD; - means M_UPS for increasing the resolution of the decoded decimated animated image IMDD;
- means M_SUP for superposing the at least one region of interest ROI and the decoded decimated animated image IMDD.
The device DD for decompressing images is adapted to implement a method 200 for decompressing animated images according to the invention, as described hereinafter on the basis of figures 3 and 4. The method 200 for decompressing animated images comprises the following steps:
- reception 201 via the remote station SDT of an encoded decimated animated image IMDC illustrating a background, an encoded window FC comprising at least one region of interest ROI, and metadata (denoted RXJMDC&FC in figure 4) via the receiver RX, said encoded decimated animated image IMDC, said encoded window FC, and said metadata being multiplexed;
- demultiplexing 202 of the encoded decimated animated image IMDC, of the encoded window FC, and of the metadata via the demultiplexer DEMUX;
- decoding 203 of the encoded window FC (denoted DEC_FC in figure 4) via the decoder DEC at the remote station SDT;
- decoding 205 of the encoded decimated animated image IMDC (denoted DECJMDC in figure 4) via the decoder DEC;
- increase 206 of the resolution of the decoded decimated animated image IMDD (denoted UPSJMDD in figure 4) via the means M_UPS for increasing resolution;
- superposition 207 of the window F and of the decoded decimated animated image IMDD (denoted SUP_ROI&IMDD in figure 4) via the superposition means M_SUP.
In a non-limiting embodiment, the decompression method 200 comprises a step 204 of ungrouping at least one region of interest ROI in the decoded window FD (denoted DEGRP_ROI_FD in figure 4). The steps of the method 200 for decompressing animated images are described in greater detail hereinafter. The description below also includes the steps of additional, non-limiting embodiments.
In a first step 201 , the receiver RX receives an encoded decimated animated image IMDC illustrating a background and an encoded window FC comprising at least one region of interest ROI, said encoded decimated animated image IMDC and said encoded window FC being multiplexed. The metadata are also multiplexed, as described previously, with said encoded decimated animated image IMDC and said encoded window FC.
In a second step 202, a demultiplexer DEMUX demultiplexes the multiplexed encoded decimated image IMDC, the multiplexed encoded window FC, and the metadata. This step makes it possible to restore the independent signals which were previously combined so as to facilitate the transport process.
In a third step 203, the decoder DEC at the remote station SDT decodes the encoded window FC to obtain the decoded window FD.
In a fourth step 204, the ungrouping means M-DEGROUP ungroup said at least one region of interest ROI in the decoded window FD. This step is also called mosaicing.
It should be noted that the ungrouping 204 of at least one region of interest in the decoded window FD is carried out via metadata. Said metadata make it possible in particular to reposition in space said at least one region of interest within the decoded decimated animated image IMDD.
In a non-limiting manner, these metadata may relate to:
· the original size of the animated image so as to regain the original size thereof after decompression;
• temporal decimation of the background image;
• temporal decimation of the regions of interest;
• the size of the window; the date ("time stamp") of the acquired animated image and that of the region(s) of interest contained in the window so as to combine the correct background image with the correct region(s) of interest (the time stamp of the background animated image is contained in the bit stream containing the background video);
the number of regions of interest;
for each region of interest:
• the size and position in the source animated image,
• the size and position in the window,
• the type of region of interest, that is to say fixed or mobile;
the UTC time;
the GPS position of the animated image sensor CI ;
the orientation of the line of vision of the animated image sensor CI ; the estimated distance between the image sensor CI and the target;
In a fifth step 205, the decoder DEC decodes the encoded decimated animated image IMDC.
In a non-limiting embodiment, if the animated image is subjected to a step of temporal decimation via the compression method, the decoded decimated animated image IMDD is also subjected to a step of reverse temporal decimation.
Likewise, in a non-limiting embodiment, if the window is subjected to a step of temporal decimation via the compression method, the decoded window FD is also subjected to a step of reverse temporal decimation.
In a sixth step 206, the means M_UPS for increasing resolution increase the resolution of the decoded decimated animated image IMDD ("up- scaling").
In a non-limiting manner, the increase in resolution may be carried out by bilinear or bicubic filtering, which is well known to a person skilled in the art. Such an increase in resolution consists in taking a predefined matrix and calculating each point of the resultant image by application of this matrix to the source image zone corresponding to the point of the resultant image.
In a seventh step 207, the superposition means M_SUP superpose the at least one region of interest ROI and the decoded decimated animated image IMDD. These superposition means M_SUP are software means.
The superposition of the at least one region of interest ROI and of the decoded decimated animated image IMDD makes it possible to obtain a resultant image comprising a high-definition zone/one of the high-definition zones (the region(s) of interest extracted from the window) illustrating precisely identifiable elements and smoothed zones of low resolution (the rest of the image not comprising any elements needing to be identified precisely). The image is also displayed on a screen (not illustrated) at the remote station SDT so that the operator can view it.
In an eighth additional, non-limiting step 208 (denoted RX INT in figure 4), the receiver RX receives an instantaneous image zone originating from the image sensor CI following a request emitted by the remote station SDT or following a request emitted automatically via an image processing procedure implemented at the compression device DC. This image zone is multiplexed with the encoded decimated animated image IMDC and the encoded window FC.
The demultiplexer DEMUX demultiplexes the instantaneous image zone IMJNT. This image zone is stored in an image zone bank (or "snapshot"). This image zone is also displayed (after possible decoding) on a screen (not illustrated) at the remote station SDT so that the operator can view it.
In addition, it should be noted that the steps of:
- decoding 203 the encoded window;
- ungrouping 204 the at least one region of interest ROI in the decoded window FD;
can be carrier out prior to, in parallel with, or subsequent to the steps of:
- decoding 205 the encoded decimated animated image IMDC; increasing 206 the resolution of the decoded decimated animated image IMDD.
The invention also relates to a computer program PGD (illustrated in figure 3) for decompressing images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of sequences of instructions making it possible to implement the method 200 for decompressing animated images according to the invention, when said program is loaded onto a computer.
The invention also relates to a system 300 for compressing and decompressing animated images, said system comprising the device DC for compressing animated images according to the invention and the device DD for decompressing animated images according to the invention. This system 300 for compressing and decompressing is illustrated in figure 1 .
The invention also relates to a method 400 (see figure 5) for compressing and decompressing animated images, said method comprising the following steps:
- acquisition 101 of an animated image IM by means of an image sensor CI ;
- shrinking 103 of said acquired animated image IM so as to obtain a decimated animated image IMD of lower resolution illustrating a background;
- encoding 104 of said decimated animated image IMD by means of an encoder ENC;
- creation 105 of a window F of fixed and configurable size comprising at least one region of interest ROI ;
- encoding 109 of said window comprising said at least one region of interest ROI by means of said encoder ENC;
- multiplexing 1 10 of said encoded decimated animated image IMDC, of said encoded window FC, and of and metadata; - transmission 1 1 1 of said multiplexed encoded decimated animated image IMDC and of the multiplexed encoded window FC to a remote station SDT, and of the metadata;
- reception 201 by said remote station SDT of said multiplexed encoded decimated animated image IMDC, of said multiplexed encoded window FC, and of any metadata;
- demultiplexing 202 of said encoded decimated animated image IMDC, of said encoded window FC, and of the metadata;
- decoding 203 of said encoded window FC by means of a decoder DEC_FC at said remote station SDT;
- decoding 205 of said encoded decimated animated image IMDC by means of said decoder DECJMDC;
- increase 206 of the resolution of said decoded decimated animated image IMDD;
- superposition 207 of said at least one ungrouped region of interest ROI and of said decoded animated image IMD.
It should be noted that the remote station SDT comprises a configuration interface IHM allowing an operator to configure, for example, the following parameters and to view, for example, the results of the configuration of the parameters directly on the screen at the remote station SDT:
- background:
o temporal decimation;
o shrinking percentage;
o quality: quality is a ratio applied to a reference rate of flow, said rate of flow corresponds to the minimum rate of flow required to obtain correct visual quality under standard conditions of image complexity (neither too simple nor too complex). Typically 6Mbits/s for 1080p@25fps with H264 video coding;
- window or mosaic:
o choice of encoding (JPEG-2000, H264, etc.);
o temporal decimation;
o shrinking percentage;
o mosaic size; o quality;
- instantaneous image zone:
o size of the instantaneous image zone;
o size of the sub-bandwidth allocated for the instantaneous image zone;
From these parameters, it is possible to display to the operator:
o estimated rate for transmission of the background;
o estimated rate for transmission of the window;
o estimated total rate (background, window, and metadata);
o lag time for display of an instantaneous image;
It should be noted that the above list of parameters is not limiting.
The invention is in no way limited to the drone application presented above. It can therefore also be implemented in a video surveillance application.
In a specific embodiment of the invention of which the objective is to reduce costs within the scope of video surveillance implementation over an existing infrastructure, the methods for compressing 100 and decompressing 200 animated images advantageously make it possible to replace a camera of the SD type (standard definition) currently used with a camera of the HD type (or any other resolution greater than SD), implementing a uniform compression technique using certain existing SD compression resources. "Existing resources" is understood to mean existing infrastructure resources, such as:
- an SD encoder,
- a communications network of the IP type (Internet Protocol),
- an SD decoder.
More specifically, such an implementation is illustrated in figure 6.
The SD camera used in the prior art is replaced by a compression device DC, the compression device DC comprising: - an animated image sensor CI, for example of the HD type, adapted to acquire animated images,
- decimation means M_DECIM able to shrink an animated image acquired by the image sensor so as to obtain a decimated animated image IMD of lower resolution illustrating a background, such as the main hall of a station,
- creation means M_CREA for creating a window of fixed and configurable size,
- multiplexing means MUX for multiplexing the window and the decimated animated image.
The existing infrastructure means listed below are retained so as to reduce costs:
- an encoder ENC adapted to code the decimated animated image and adapted to code the window comprising the at least one region of interest, the decimated animated image and the window being multiplexed;
- a transmitter TX adapted to transmit the multiplexed encoded window and the multiplexed encoded decimated animated image;
- an IP network adapted to allow circulation of the information between the transmitter TX and a receiver RX;
- the receiver RX is adapted to receive the multiplexed encoded decimated image and the multiplexed encoded window;
- a decoder DEC adapted to decode the encoded window and adapted to decode the encoded decimated animated image.
In addition, a decompression device DD is added to the existing infrastructure and comprises:
- a demultiplexer DEMUX adapted to demultiplex the decoded window and the decoded decimated animated window;
- means M_UPS for increasing the resolution of the decoded decimated animated image;
- means M_SUP for superposing the at least one region of interest and the decoded decimated animated image. This type of infrastructure is adapted for implementation of the aforementioned compression method 100 according to the invention and the aforementioned decompression method 200 according to the invention. Thus, all steps of the compression method 100 and of the decompression method 200 can be carried out.
More specifically, as illustrated in figure 7, the compression device 100 implements the following steps:
- acquisition of animated images 101 by means of the image sensor CI ;
- shrinking 103 of the acquired animated image IM so as to obtain a decimated animated image IMD of lower resolution illustrating a background;
- creation 105 of a window F (also called a mosaic) of fixed and configurable size comprising at least one region of interest ROI ;
- multiplexing 1 10 of the decimated animated image IMD and of the window F.
The existing infrastructure means implement the following steps:
- encoding 104 of the decimated animated image by means of the encoder ENC;
- encoding 109 of the window 109 comprising the at least one region of interest by means of the encoder ENC;
- transmission 1 1 1 of the multiplexed encoded decimated animated image IMDC and of the multiplexed window FC to the remote station SDT;
- reception 201 by the remote station SDT of the multiplexed encoded decimated animated image IMDC and of the multiplexed encoded window FC;
- decoding of the encoded window 203 by means of a decoder DEC at the remote station SDT;
- decoding of the encoded decimated animated image 205 by means of the decoder DEC. The decompression device DD implements the following steps:
- demultiplexing 202 of the decoded decimated animated image IMDD and of the decoded window FD;
- increase 206 of the resolution of the decoded decimated animated image;
- superposition 207 of the at least one region of interest and of the animated image. This superposition is displayed, for example, on a screen ECR at the remote station SDT so that the operator can view it.
It should be noted that the means of the compression and decompression devices and the steps described in this non-limiting embodiment of figures 6 and 7 are the same as those described previously. It should also be noted that the additional non-limiting steps described with reference to figures 1 to 5 also apply, such as the temporal decimation of the acquired image and also the selection of a plurality of regions of interest, etc. It should be noted that, in a non-limiting embodiment, a control channel Eth of the Ethernet type can be used to transmit instantaneous images (or "snapshots"), this channel also being used to transmit the metadata. This mode is used, for example, when a low transmission rate is used for the instantaneous images.
It should be noted that this control channel Eth allows a user of a control console to enter the parameters of the compression of the animated images, these parameters being described previously in the above embodiments. The compression device thus makes it possible to control the SD-type encoder ENC.
In another embodiment, a transmission channel parallel to the animated images can be used to transmit instantaneous images (or "snapshots"). However, this parallel transmission channel is not obligatory. In fact, the instantaneous images can be transported in the window cut into small pieces, and aggregated in the regions of interest ROI and in the animated image. The transmission of the entire instantaneous image may be spread over a plurality of images.
A transmission channel parallel to the transmission channel of the animated images is desirable for transmission of the metadata (size and position of the region of interest, shrinking, and temporal decimation, ...). In fact, these metadata, which are numerical values, are not naturally encoded without degradation through a video encoder. However, it is still conceivable in the absence of a specific channel to group these metadata in the first lines of the decimated animated image and to pass them through the chain of encoding and decoding. Care must be taken to prevent degradation of the information through the chain for encoding animated images. One approach may consist in transferring an octet over the luminance components of two pixels and of only using the 4 most significant bits of these pixels; care should also be taken to ensure that a small variation in the value of the pixel does not have an effect on the 4 most significant bits. Typically, two luminance components of 0x48 and 0x28 are used to convey a metadata 0x42. Even if the encoder introduced an error of +/- 7 LSB, it is thus still possible to regain the metadata values intact. It is noted that, if the rate of flow of the compressed animated image is very low and the visual quality is also low, 3 or even 2 most significant bits may be used for transmission of the metadata, and each octet may be spread over 3 or 4 components.
In this specific embodiment and if it is desired to transmit an instantaneous image, this image is transmitted in parallel with the encoded decimated animated images. To do this, it is necessary for an additional transmission channel to be available. However, this additional channel is only necessary in this specific embodiment of video surveillance using existing SD compression resources.
Generally, and in a non-limiting manner, the methods, devices and computer programs of the invention in particular afford the advantages of:
- enabling excellent efficiency of compression of each acquired animated image. A blurred effect is obtained over each animated image, whilst maintaining good perception of the background, even with shrinking up to 1 /50 or even 1/100,
maintaining the fluidity of the projection of the animated images for a temporal decimation of one image in two,
maintaining a very good level of detail of the region of interest, enabling modification of the size of the region of interest ROI and of the position of the region of interest ROI with each new animated image thanks to the selection,
enabling management of a large region of interest ROI thanks to the shrinking,
maintaining the resolution of the acquired animated image by capturing an instantaneous image or "snapshot",
maintaining a constant rate of transfer of the information (encoded decimated animated image + window comprising at least one region of interest ROI + snapshot (instantaneous image)),
maintaining a low rate of flow, even with a large region of interest ROI, thanks to the encoding of the window; the use of a large region of interest ROI allows the operator to view with precision a large part of the theatre of operations (of the background), which allows him to easily track an object moving within the theatre of operations, and maintaining a constant rate of flow thanks to the fixed size of the window and of the decimated animated image. A terrestrial or satellite channel (or link) of fixed size can thus be utilised in spite of the variability in size, position and number of the region of interest ROI.

Claims

A method (100) for compressing animated images, said method comprising the following steps:
- acquisition (101 ) of an animated image (IM) by means of an image sensor (CI);
- shrinking (103) of said acquired animated image (IM) so as to obtain a decimated animated image (IMD) of lower resolution illustrating a background;
- encoding (104) of said decimated animated image (IMD) by means of an encoder (ENCJMD);
- creation (105) of a window (F) of fixed and configurable size comprising at least one region of interest (ROI);
- encoding (109) of said window comprising said at least one region of interest (ROI) by means of said encoder (ENC_F);
- multiplexing (1 10) of said encoded decimated animated image (IMDC) and of said encoded window (FC);
- transmission (1 1 1 ) of said multiplexed encoded decimated animated image (IMDC) and of the multiplexed encoded window (FC) to a remote station (SDT).
The method (100) according to claim 1 , characterised in that it comprises the following additional steps:
selection (106) of at least one region of interest (ROI) of variable size and position;
grouping (108) of said at least one region of interest (ROI) in said window (F).
The method (100) according to claim 2, characterised in that said selection (106) is carried out on the basis of said animated image (IM) or on the basis of the content of said decimated animated image (IMD) of lower resolution illustrating a background.
4. The method (100) according to one of claims 2 or 3, characterised in that said selection (106) of at least one region of interest (ROI) is carried out in accordance with a cropping or shrinking method.
5. The method (100) according to one of claims 2 to 4, characterised in that said selection (106) is carried out manually or automatically.
6. The method (100) according to one of the preceding claims, characterised in that metadata are multiplexed with the encoded decimated animated image (IMDC) and the encoded window (FC) during said multiplexing step (1 10).
7. The method (100) according to one of the preceding claims, characterised in that each animated image is encoded by with video coding.
The method (100) according to one of the preceding claims, characterised in that the window (F) is encoded by video coding.
The method (100) according to claim 8, characterised in that said video encoding of the window (F) is carried out at a constant rate of flow.
10. The method (100) according to one of the preceding claims, characterised in that it comprises an additional step of shrinking of said at least one region of interest (107) before the grouping step (108).
1 1 . The method (100) according to one of the preceding claims, characterised in that it comprises a step of temporal decimation (1 02) of the animated images (IM) acquired by said image sensor (CI).
1 2. The method (100) according to one of the preceding claims, characterised in that it comprises an additional step of instantaneous transmission (1 1 2) of an image zone from said image sensor (CI) to said remote station (SDT).
1 3. The method (1 00) according to claim 1 2, characterised in that said image zone transmitted instantaneously is encoded by image coding.
14. The method (1 00) according to one of claims 1 2 or 1 3, characterised in that, during said multiplexing step (1 10), said image zone is multiplexed with the encoded decimated animated image (IMDC) and the encoded window (FC).
1 5. The method (1 00) according to one of claims 1 3 or 14, characterised in that the instantaneous transmission (1 1 2) of an image zone from said image sensor (CI) is carried out at a constant rate of flow.
1 6. The method (100) according to one of the preceding claims, characterised in that said image sensor (CI) is installed in an aerial, terrestrial or naval mobile device (E).
1 7. The method (1 00) according to one of claims 1 to 1 5, characterised in that said image sensor (CI) is fixed.
1 8. A computer program (PGC) for compressing images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of said sequences of instructions enabling implementation of the method (100) for compressing images according to one of claims 1 to 17, when said program is loaded onto a computer.
19. A device (DC) for compressing animated images, said device (DC) comprising:
- an image sensor (CI) adapted to acquire animated images (IM),
- decimation means (M_DECIM) able to shrink the animated image (IM) acquired by said image sensor (CI) so as to obtain a decimated animated image (IMD) of lower resolution illustrating a background,
- an encoder (ENC) adapted to code said decimated animated image (IMD),
- creation means (M_CREA) for creating a window (F) of fixed and configurable size comprising at least one region of interest (ROI),
- said encoder (ENC) is also adapted to code said window (F) comprising said at least one region of interest (ROI);
- multiplexing means (MUX) for multiplexing said encoded window (FC) and said encoded decimated animated image (IMDC); - a transmitter (TX) adapted to transmit the multiplexed encoded window (FC) and the multiplexed encoded decimated animated image (IMDC).
20. A method (200) for decompressing animated images, said method comprising the following steps:
- reception (201 ) via a remote station (SDT) of an encoded decimated animated image (IMDC) illustrating a background and an encoded window (FC) comprising at least one region of interest (ROI), said encoded decimated animated image (IMDC) and said encoded window (FC) being multiplexed;
- demultiplexing (202) of said encoded decimated animated image (IMDC) and of said encoded window (FC);
- decoding (203) of said encoded window (FC) by means of a decoder (DEC) at said remote station (SDT); - decoding (205) of said encoded decimated animated image (IMDC) by means of said decoder (DEC) ;
- increase (206) of the resolution of said decoded decimated animated image (IMDD);
- superposition (207) of said at least one region of interest (ROI) and of said decoded animated image (IMD).
21 . The method (200) for decompressing images according to claim 20, characterised in that it comprises an additional step (204) of ungrouping at least one region of interest (ROI) in said window
(F) as a function of the metadata.
22. The method (200) for decompressing images according to one of claims 20 or 21 , characterised in that it comprises an additional step of instantaneous reception (208) of an image zone originating from an image sensor (CI).
23. A device (DD) for decompressing images, said device comprising :
- a remote station (SDT) adapted to receive an encoded decimated animated image (IMDC) illustrating a background and an encoded window (FC) comprising at least one region of interest (ROI); said encoded decimated animated image (IMDC) and said encoded window (FC) being multiplexed;
- a demultiplexer (DEMUX) adapted to demultiplex the multiplexed encoded window (FC) and the multiplexed encoded decimated animated image (IMDC),
- a decoder (DEC) adapted to decode said encoded window (FC) and adapted to decode said encoded decimated animated image (IMDC) ;
- means (M_UPS) for increasing the resolution of said decoded decimated animated image (IMDD);
- means (M_SUP) for superposing said at least one region of interest (ROI) and said decoded animated image (IMD).
24. A computer program (PGD) for decompressing images, said program comprising one or more sequences of instructions which can be executed by a data processing unit, the execution of said sequences of instructions enabling implementation of the method (200) for decompressing animated images according to one of claims 20 to 22, when said program is loaded onto a computer.
25. A system (300) for compressing and decompressing images, said system comprising the device (DC) for compressing animated images according to claim 1 9 and the device (DD) for decompressing animated images according to claim 23.
26. A method (400) for compressing and decompressing animated images, said method comprising the following steps:
- acquisition (1 01 ) of an animated image (IM) by means of an image sensor (CI);
- shrinking (1 03) of said acquired animated image (IM) so as to obtain a decimated animated image (IMD) of lower resolution illustrating a background;
- encoding (1 04) of said decimated animated image (IMD) by means of an encoder (ENC);
- creation (105) of a window (F) of fixed and configurable size comprising at least one region of interest (ROI) ;
- encoding (1 09) of said window comprising said at least one region of interest (ROI) by means of said encoder (ENC) ;
- multiplexing (1 1 0) of said encoded decimated animated image (IMDC) and of said encoded window (FC) ;
- transmission (1 1 1 ) of said multiplexed encoded decimated animated image (IMDC) and of the multiplexed encoded window (FC) to a remote station (SDT); - reception (201 ) by said remote station (SDT) of said multiplexed encoded decimated animated image (IMDC) and of said multiplexed encoded window (FC);
- demultiplexing (202) of said encoded decimated animated image (IMDC) and of said encoded window (FC);
- decoding (203) of said encoded window (FC) by means of a decoder (DEC) at said remote station (SDT);
- decoding (205) of said encoded decimated animated image (IMDC) by means of said decoder (DEC);
- increase (206) of the resolution of said decoded decimated animated image (IMDD);
- superposition (207) of said at least one region of interest (ROI) and of said decoded animated image (IMD).
PCT/EP2012/063606 2011-07-29 2012-07-11 Methods for compressing and decompressing animated images Ceased WO2013017387A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1156949A FR2978639B1 (en) 2011-07-29 2011-07-29 METHODS OF COMPRESSING AND DECOMPRESSING IMAGES
FR1156949 2011-07-29

Publications (1)

Publication Number Publication Date
WO2013017387A1 true WO2013017387A1 (en) 2013-02-07

Family

ID=46508353

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2012/063606 Ceased WO2013017387A1 (en) 2011-07-29 2012-07-11 Methods for compressing and decompressing animated images

Country Status (2)

Country Link
FR (1) FR2978639B1 (en)
WO (1) WO2013017387A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014146981A1 (en) * 2013-03-18 2014-09-25 Vega Grieshaber Kg Method for the compressed storage of graphical data
US20190290199A1 (en) * 2015-11-16 2019-09-26 Synapsys System for generating, emitting and interpreting a composite stream, and associated method
FR3131816A1 (en) 2022-01-12 2023-07-14 Sedi-Ati Fibres Optiques Deployable on-board fiber optic link for aerial drones

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060093033A1 (en) * 2002-01-05 2006-05-04 Samsung Electronics Co., Ltd. Image coding and decoding method and apparatus considering human visual characteristics
US20090010328A1 (en) * 2007-07-02 2009-01-08 Feng Pan Pattern detection module, video encoding system and method for use therewith
US20100119157A1 (en) * 2007-07-20 2010-05-13 Fujifilm Corporation Image processing apparatus, image processing method and computer readable medium
US7773670B1 (en) * 2001-06-05 2010-08-10 At+T Intellectual Property Ii, L.P. Method of content adaptive video encoding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7773670B1 (en) * 2001-06-05 2010-08-10 At+T Intellectual Property Ii, L.P. Method of content adaptive video encoding
US20060093033A1 (en) * 2002-01-05 2006-05-04 Samsung Electronics Co., Ltd. Image coding and decoding method and apparatus considering human visual characteristics
US20090010328A1 (en) * 2007-07-02 2009-01-08 Feng Pan Pattern detection module, video encoding system and method for use therewith
US20100119157A1 (en) * 2007-07-20 2010-05-13 Fujifilm Corporation Image processing apparatus, image processing method and computer readable medium

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014146981A1 (en) * 2013-03-18 2014-09-25 Vega Grieshaber Kg Method for the compressed storage of graphical data
EP2790408A1 (en) * 2013-03-18 2014-10-15 VEGA Grieshaber KG Method for the compressed storage of graphic data
CN105191308A (en) * 2013-03-18 2015-12-23 Vega格里沙贝两合公司 Method for the compressed storage of graphical data
US9591314B2 (en) 2013-03-18 2017-03-07 Vega Grieshaber Kg Method for the compressed storage of graphical data
CN105191308B (en) * 2013-03-18 2019-01-22 Vega格里沙贝两合公司 Method for compressing storage graphics data
US20190290199A1 (en) * 2015-11-16 2019-09-26 Synapsys System for generating, emitting and interpreting a composite stream, and associated method
FR3131816A1 (en) 2022-01-12 2023-07-14 Sedi-Ati Fibres Optiques Deployable on-board fiber optic link for aerial drones

Also Published As

Publication number Publication date
FR2978639B1 (en) 2015-09-25
FR2978639A1 (en) 2013-02-01

Similar Documents

Publication Publication Date Title
AU2016209079B2 (en) Video transmission based on independently encoded background updates
US10531094B2 (en) Image processing device and moving image transmission method
CN105120240B (en) The aerial high definition multidimensional of high power zoom unmanned plane investigates transmitting, monitoring device in real time
EP2629520B1 (en) Apparatus and method for shooting moving picture in camera device
US20200236323A1 (en) Method and system for combining multiple area-of-interest video codestreams into a combined video codestream
JP6573673B2 (en) High frame rate-low frame rate transmission technology
US8774539B2 (en) Encoder, decoder, encoder system, decoder system, transmission adapter, encoding method, decoding method, and imaging apparatus
US10812854B2 (en) Image processing device, image processing method, reception device, and transmission device
CN108307202B (en) Real-time video transcoding sending method and device and user terminal
US20170374283A1 (en) Video transmission based on independently encoded background updates
CN105208335B (en) The aerial high definition multidimensional of high power zoom unmanned plane investigates Transmission system in real time
US20160057355A1 (en) Decoder and monitor system
KR101029466B1 (en) Multiple video signal transmission and reception system and its operation method
WO2013017387A1 (en) Methods for compressing and decompressing animated images
JP5274212B2 (en) Digital broadcasting method and system, broadcasting station and receiver
EP3531700B1 (en) Image coding method, transmission method and image coding device
JP6844480B2 (en) Video data streaming monitoring equipment, methods and programs, as well as terminal equipment and video data streaming monitoring systems
US8208555B2 (en) Image capturing and transmission device and image receiving and reconstruction device
US20180270507A1 (en) Method for encoding streams of video data based on groups of pictures (gop)
CN105208336A (en) Transmission method for aerial high-definition multi-dimensional real-time reconnaissance of high-magnification zoom drones
CN201352834Y (en) High-definition conference system based on digital television HDTV platform
KR100540172B1 (en) Object-based Multiview Video Service System and Its Method
Misu et al. Real-time video coding system for up to 4K 120P videos with spatio-temporal format conversion
EP3915090B1 (en) Video generating device and method thereof
CN101521790A (en) Methods for realizing high definition conference system based on digital television

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12735140

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12735140

Country of ref document: EP

Kind code of ref document: A1