WO2021144004A1

WO2021144004A1 - Storing data items

Info

Publication number: WO2021144004A1
Application number: PCT/EP2020/050762
Authority: WO
Inventors: Gianmarco Bruno; Francesco Lazzeri
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2020-01-14
Filing date: 2020-01-14
Publication date: 2021-07-22

Abstract

A method of storing a data item at a data management system comprises storing the data item at a first time, wherein the stored data item has an initial quality. The method also involves storing a predetermined function, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, and wherein the predetermined function comprises a monotonically decreasing function.

Description

STORING DATA ITEMS

FIELD OF THE INVENTION Embodiments described herein relate to methods and apparatus for storing a data item at a data management system.

BACKGROUND

The “Internet of Things” (loT) refers to devices (for example, digital machines, computing devices and mechanical devices) enabled for communication network connectivity, so that these devices may be remotely managed, and data collected or required by the devices may be exchanged between individual devices and between devices and application servers. Such devices, examples of which may include sensors and actuators, are often, although not necessarily, subject to severe limitations on processing power, storage capacity, energy supply, device complexity and/or network connectivity, imposed by their operating environment or situation, and may consequently be referred to as constrained devices.

Closed-circuit TV (CCTV) devices are one example of loT devices. The data (video data, image data, audio data, for example) that is captured by a CCTV device may be analyzed either by individuals, or by a computer, in real time. This real time analysis may allow suspicious activity to be detected and monitored, and may also allow a number of crimes to be prevented. However, it will be appreciated that in a crime investigation scenario, data may be analyzed at a later time to the time at which it was captured by the CCTV device. For example, this analysis may occur weeks or months after the time at which the data was captured.

It will also be appreciated that data captured by a CCTV device may be analyzed, either in real time or at a later date, in order to determine patterns in the captured data (for example, behavioral patterns of individuals in captured video data).

However, there are a number of regulations relating to the capture and storage of data that contains information relating to individuals. These regulations often relate to the privacy of the individuals. For example, an individual may have a right not to be tracked unless there is lawful permission to do so, and in this example, a regulation may exist that only permits data that contains information relating to a specific individual to be captured when lawful permission has been granted. In another example, a regulation may exist that orders that data that contains information relating to a specific individual is destroyed when that data is no longer required, or after a defined amount of time has passed since the data was captured.

In some examples, the data that is captured by a CCTV device may be uploaded to a data management system. The data management system may be a cloud based data management system. In some examples, the data management system may be configured to upload data from a suitable data source (for example, a CCTV device) to a data storage unit. The data that is stored at the data storage unit may then be associated to one or more relevant policies, which may also be stored at the data storage unit. In some examples, the data management system may also be configured to perform data encoding prior to, or following the storage of, the data (for example, the format and/or the resolution of the data may be changed by the data management system). In some examples, the data management system may also be configured to automatically process data (for example, the data management system may be configured to automatically tag data, and/or may be configured to extract certain features from the data). In some examples, the data management system may also be configured to manage clients of the data management system, and/or manage the policies stored within/applied to the data management system.

It will be appreciated that a data management system may be further configured to perform any of the following functions: performance management, fault management, software upgrade, software inventory, hardware inventory and/or security management.

For example, data items for a client C may be streamed to a data management system, and these data items may be stored according to a policy P, comprising a retention criterion. The retention criterion may relate to the data item, the type of data item, the data management system, etc. The policy may correspond to a business requirement of the owner of the data management system, and may also comply with local regulations relating to data storage. For example, a local regulation may order that a stored data item be destroyed after 90 days, if that data item is not in use in a criminal investigation. One example of a policy is that a stored data item is to be stored by the data management system for a minimum of 3 days, regardless of the storage requirements, and following this, the data item is to be destroyed whenever further storage is required by the data management system. Another example of a policy is that a data item is to be stored such that the storage required is at most 1TB, and that data items are to be destroyed such that older items are destroyed whenever further storage is required by the data management system for new data items.

Stored data items will then be persisted at the data management system according to the policy P. When the retention criterion of the policy P is no longer met, the stored data item will be destroyed by the data management system. The storage resource that was being used by this destroyed data item will then become available to the data management system again.

In some examples, a data item may persist for a period of time T, regardless of whether the data item is accessed to be analyzed, or in fact accessed at all. Thus, for the period of time T, the storage will be consumed. Furthermore, if a need to access the data after the time T has passed arises, the data will not be accessible, as the data item will have been destroyed.

SUMMARY

According to a first aspect, there is provided a method of storing a data item at a data management system. The method comprises storing the data item at a first time, wherein the stored data item has an initial quality. The method also comprises storing a predetermined function, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, and wherein the predetermined function comprises a monotonically decreasing function.

The quality may be defined in terms of a quality measure, wherein the quality measure relates to one or more quality-affecting characteristics of the data item.

The quality measure may be defined as a function of the one or more quality-affecting characteristics of the data item.

A value of the function may vary monotonically as the quality of the data item is varied. The function may be such that the minimum permitted quality represented by the predetermined function does not fall below a predetermined minimum acceptable quality.

The method may further comprise replacing the stored data item with a new version of the stored data item in accordance with the predetermined function at at least one time succeeding the first time, such that each new version of the stored data item has a reduced quality compared with each previous version of the stored data item.

The step of replacing the stored data item with a new version of the stored data item in accordance with the predetermined function may comprise replacing the stored data item with a new version of the stored data item that has a quality greater than or equal to the minimum permitted quality that is currently represented by the predetermined function.

A first replacing time may be defined with respect to said first time at which the data item is stored, such that the at least one later time does not precede the first replacing time.

The method may further comprise, at a second time, later than the first time, deleting the current version of the stored data item.

A first deleting time may be defined with respect to said first time at which the data item is stored method, such that the current version of the stored data item is not deleted prior to the first deleting time.

A second deleting time may be defined with respect to said first time at which the data item is stored, such that the second time precedes the second deleting time.

The method may further comprise varying the predetermined function, wherein the varied predetermined function represents a different intended variation with time in a minimum permitted quality of the stored data item, and wherein the varied predetermined function comprises a different monotonically decreasing function. The predetermined function may comprise a monotonically decreasing linear function. The predetermined function may comprise a monotonically decreasing exponential function.

The stored data item may comprise any one of video data, image data, or audio data.

When the stored data item comprises video data, the quality may be defined by at least one of a frame rate and a video resolution.

When the stored data item comprises image data, the quality may be defined by an image resolution.

In the case of video data or image data, the resolution may be expressed in one or more of: a number of pixels, a maximum number of colours, a brightness, and a dynamic range.

When the stored data item comprises audio data, the quality may be defined by at least one of a sample rate and a quantization level.

According to another aspect, there is provided a data management system, comprising a processor, wherein the processor is configured to perform a method of storing a data item. The method comprises: storing the data item in a memory at a first time, wherein the stored data item has an initial quality; and storing a predetermined function in said memory, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, and wherein the predetermined function comprises a monotonically decreasing function.

The data management system may further comprise said memory. As an alternative, the processor may be configured to store the data item in a remote memory.

According to another aspect, there is provided a computer program product, comprising a tangible and/or non-transient computer readable medium, wherein the computer readable medium contains instructions for performing a method of storing a data item, the method comprising: storing the data item in a memory at a first time, wherein the stored data item has an initial quality; and storing a predetermined function in said memory, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, and wherein the predetermined function comprises a monotonically decreasing function.

Thus, according to different aspects, there are provided a method and a system for storing data, where the quality of the stored data can be reduced overtime, but the storing of the predetermined function allows the system operator to ensure that the quality does not fall below a minimum permitted level.

BRIEF DESCRIPTION OF DRAWINGS

For a better understanding of the present invention, and to show how it may be put into effect, reference will now be made, by way of example only, to the accompanying drawings, in which:-

Figure 1 illustrates an example of a network;

Figure 2 illustrates an example of a data management system;

Figure 3 illustrates a method of storing a data item at a data management system; and

Figures 4, 5 and 6 illustrate examples of how a quality measure of a stored data item may change overtime at a data management system.

DETAILED DESCRIPTION

The description below sets forth example embodiments according to this disclosure. Further example embodiments and implementations will be apparent to those having ordinary skill in the art. Further, those having ordinary skill in the art will recognize that various equivalent techniques may be applied in lieu of, or in conjunction with, the embodiments discussed below, and all such equivalents should be deemed as being encompassed by the present disclosure.

The following sets forth specific details, such as particular embodiments for purposes of explanation and not limitation. But it will be appreciated by one skilled in the art that other embodiments may be employed apart from these specific details. In some instances, detailed descriptions of well-known methods, nodes, interfaces, circuits, and devices are omitted so as not obscure the description with unnecessary detail. Those skilled in the art will appreciate that the functions described may be implemented in one or more nodes using hardware circuitry (e.g., analog and/or discrete logic gates interconnected to perform a specialized function, ASICs, PLAs, etc.) and/or using software programs and data in conjunction with one or more digital microprocessors or general purpose computers that are specially adapted to carry out the processing disclosed herein, based on the execution of such programs. Nodes that communicate using the air interface also have suitable radio communications circuitry. Moreover, the technology can additionally be considered to be embodied entirely within any form of computer-readable memory, such as solid-state memory, magnetic disk, or optical disk containing an appropriate set of computer instructions that would cause a processor to carry out the techniques described herein.

Hardware implementation may include or encompass, without limitation, digital signal processor (DSP) hardware, a reduced instruction set processor, hardware (e.g., digital or analog) circuitry including but not limited to application specific integrated circuit(s) (ASIC) and/or field programmable gate array(s) (FPGA(s)), and (where appropriate) state machines capable of performing such functions.

In terms of computer implementation, a computer is generally understood to comprise one or more processors, one or more processing modules or one or more controllers, and the terms computer, processor, processing module and controller may be employed interchangeably. When provided by a computer, processor, or controller, the functions may be provided by a single dedicated computer or processor or controller, by a single shared computer or processor or controller, or by a plurality of individual computers or processors or controllers, some of which may be shared or distributed. Moreover, the term “processor” or “controller” also refers to other hardware capable of performing such functions and/or executing software, such as the example hardware recited above.

It will be appreciated that, in order to effectively analyze and utilize data that has been captured by a suitable device (for example, a CCTV device), the stored data must be of a sufficient quality. For example, if the data comprises video data, a minimum resolution for the stored video data may be required. However, as the quality of stored data increases, so does the memory required to store that data. For example, for video data obtained by a CCTV device, assuming 30% motion- detection, 30 frames per second, and MPEG4 resolution, the storage required for a number of different resolutions for stored video data are listed below:

As the memory required to store a data item increases with the quality of the stored data item, storage availability in a data management system may become limited as a number of data items are stored where the quality of these items are sufficient to allow the items to be analyzed, as described above.

Embodiments described herein provide methods and apparatus for defining at a data management system a required quality of a stored data item as a function of time. In an embodiment, the data management system may define the required quality as a minimum permitted quality of a stored data item as a function of time.

Figure 1 illustrates a network 100 according to some embodiments. The network 100 comprises a data source 102. The data source 102 may comprise any suitable loT device, such as a CCTV camera, a sensor, or an actuator, for example. The data source 102 is communicatively coupled to a communications network 104. The communications network 104 is communicatively coupled to a web front end 106 of a data management system 108. Thus, information may be exchanged between the data source 102 and the data management system 108 via the communications network 104. In one example, the communications network 104 may be a cellular communications network, in which case the data source 102 may include the communications functionality of a User Equipment device in the cellular communications network, and the data management system 108 may be located at one or more server that is accessible through the cellular communications network.

A client 110 is also communicatively coupled to the web front end 106 of the data management system 108 via the communications network 104. Thus, information may be exchanged between the client 110 and the data management system 108 via the communications network 104. In other embodiments, the client 110 may be directly connected to the data management system 108.

The data management system 108 further comprises a data management module 112 that is communicatively coupled to the web front end 106. The data management module 112 may obtain information from the web front end 106, which may have been received from the data source 102 and/or the client 108. For example, the data management module 112 may receive data from the data source 102, and/or may receive policies that dictate how to manage data received from the data source 102 from the client 108. The data management module 112 is further communicatively coupled to the data storage system 114. The data storage system 114 may store data obtained from the data source 102 as data items. The data storage system may also store properties corresponding to the stored data items. The data management module 112 may further obtain policies that dictate how to manage data received from the data source 102 from the data storage system 114. The data management module 112 may also obtain information relating to the amount of storage that is available from the data storage 114. Furthermore, the data management system 112 is also configured to obtain stored data items (and the corresponding stored properties of these data items) from the data storage system 114. These obtained stored data items (and the corresponding properties) may then be communicated to the client 108 via the web front end 106 and the network 104.

The data storage system 114 may include one or more memory devices, and may be co-located with the data management module 112, or may be located remote therefrom.

Thus, the data management module 112 may decide how data received from the data source 102 is stored at the data storage system 114 (as data items). These decisions may be based on policies that are obtained by the data management module 112, and information relating to the amount of storage that is available, that has been obtained from the data storage system 114.

Figure 2 is a block schematic diagram of a data management system 200, for example, the data management system 108 in the architecture of Figure 1. The data management system 200 comprises a communications module 202, and a data processing unit 204.

The communications module 202 is configured for communication using any required communications protocol, for example with other nodes of the architecture shown in Figure 1.

The data processing unit 204 comprises a processor 206 and a memory 208. The memory 208 is configured for storing working data as required, and also for storing program instructions, and the processor 206 is configured for performing data processing and logical operations on the basis of program instructions stored in the memory 208. More specifically, the processor 206 may be programmed to perform the methods described herein. Although shown as respective single units, the processor 206 and the memory 208 may be implemented as multiple units, each carrying out a respective part of the required functionality. As mentioned above, one or more data storage unit of the memory 208 may be located remote from other components of the data management system 200.

Methods are described herein that allow a minimum permitted quality of a stored data item to be defined as a function of time at a data management system.

Figure 3 illustrates a method 300 of storing a data item at a data management system 200.

At step 302, the data item is stored at a first time. In some embodiments, the stored data item comprises any one of video data, image data, or audio data. The data item may have been generated by any suitable data source.

The stored data item has an initial quality.

For example, when the stored data item comprises video data, the quality of the stored data item may be defined in terms of at least one of a frame rate of the video, and a video resolution. The video resolution may for example be expressed as one or more of: a number of pixels per frame; a number of colours, e.g. a maximum number of colours, to be used in the image; a dynamic range, i.e. a maximum colour difference; a brightness of the image; a quantization level; a resolution of colour, e.g. a level of subsampling of colour component values.

In another example, when the stored data item comprises image data, the quality may for example be expressed as one or more of: a number of pixels per frame; a number of colours, e.g. a maximum number of colours, to be used in the image; a dynamic range, i.e. a maximum colour difference; a brightness of the image; a quantization level; a resolution of colour, e.g. a level of sub-sampling of colour component values in the image.

In another example, when the stored data item comprises audio data, the quality may be defined by at least one of a sample rate and a quantization level, i.e. a number of bits per sample.

In some embodiments, the quality of the data item may be defined in terms of a single quality measure, wherein the single quality measure relates to one or more quality- affecting characteristics of the data item. In some embodiments, the quality of the data item may be defined in terms of more than one quality measure, each quality measure relating to one or more quality-affecting characteristics of the data item. Examples of quality-affecting characteristics are provided above. It will be appreciated that the one or more quality-affecting characteristics of the data item may vary depending on the type of the stored data item, as described above.

For example, when the stored data item comprises video data, the quality of the stored data item may be varied by adjusting one of more of the quality-affecting characteristics described above: a frame rate of the video; a video resolution; a number of pixels per frame; a number of colours, e.g. a maximum number of colours, to be used in the image; a dynamic range, i.e. a maximum colour difference; a brightness of the image; a quantization level; a resolution of colour, e.g. a level of sub-sampling of colour component values in the image.

In another example, when the stored data item comprises image data, the quality of the stored data item may be varied by adjusting one of more of the quality-affecting characteristics described above: an image resolution; a number of pixels per frame; a number of colours, e.g. a maximum number of colours, to be used in the image; a dynamic range, i.e. a maximum colour difference; a brightness of the image; a quantization level; a resolution of colour, e.g. a level of sub-sampling of colour component values in the image.

In another example, when the stored data item comprises audio data, the quality of the stored data item may be varied by adjusting one of more of the quality-affecting characteristics described above: a sample rate and a quantization level, i.e. a number of bits per sample.

In some embodiments, the quality measure may be defined as a function of the one or more quality-affecting characteristics of the data item. For example, in some embodiments, a value of the function may vary monotonically as the quality of the data item is varied.

At step 304, a predetermined function is stored in association with the data item. The predetermined function, in this example embodiment, represents an intended variation with time in a minimum permitted quality of the stored data item, and comprises a monotonically decreasing function. For example, the predetermined function may comprise a monotonically decreasing linear function. As another example, the predetermined function may comprise a monotonically decreasing exponential function.

In other words, the predetermined function represents a lower bound for the quality of a stored data item as a function of time. This represents the minimum quality that of a stored data item at the data management system 200 at any particular moment in time following the storing of the data item. It may be considered the quality that will be guaranteed to a user of the data management system 200 who may access the stored data item at any given time following the storing of the data item. The predetermined function may be defined as part of a policy that is stored at the data management system 200. In some examples, the predetermined function may reflect local regulations relating to the quality of stored data items.

In some embodiments, the minimum permitted quality represented by the predetermined function does not fall below a predetermined minimum acceptable quality.

For example, the predetermined minimum acceptable quality may correspond to the minimum quality of a stored data item that is permitted by local regulations. This minimum acceptable quality may be defined such that, should a data item be retrieved from the data management system 200, it will be of a quality that allows the data item to be effectively analyzed by a user (for example, such that the information obtained from the data item may be used in a crime investigation). It will be appreciated that different minimum acceptable qualities may apply to different data types (for example, video data, image data, audio data).

At step 306 of the method shown in Figure 3, the method 300 further comprises replacing the stored data item with a new version of the stored data item in accordance with the predetermined function at at least one time succeeding the first time, such that each new version of the stored data item has a reduced quality compared with each previous version of the stored data item. As described above, the quality of the data item may be defined in terms of a quality measure, wherein the quality measure relates to one or more quality-affecting characteristics of the data item, and, by varying at least one of the one or more quality- affecting characteristics, the overall quality of the data item may also be varied. Thus, the quality of the new version of the stored data item may be reduced by the degradation of one or more quality-affecting characteristics of the data item.

More specifically, the step of replacing the stored data item with a new version of the stored data item in accordance with the predetermined function comprises replacing the stored data item with a new version of the stored data item that has a quality greater than or equal to the minimum permitted quality that is currently represented by the predetermined function.

It will be appreciated that, as the predetermined function may be considered a lower bound for the minimum quality of a stored data item, the data management system 200 will not necessarily replace the stored data item such that the new version of the stored data item has the minimum quality that is given by the predetermined function at that given time. Rather, the data management system 200 will replace the stored data item with a new version of the stored data item in a manner that optimizes usage of the resources of the data management system 200, while also guaranteeing any policies regarding stored data items that have been received at the data management system 200. In some embodiments, the at least one later time does not precede a first replacing time, wherein the first replacing time is defined with respect to said first time at which the data item is stored.

In some embodiments, the method 300 further comprises, at a second time, later than the first time, deleting the current version of the stored data item.

In some embodiments, the current version of the stored data item is not deleted prior to a first deleting time, wherein the first deleting time is defined with respect to said first time at which the data item is stored.

The first deleting time may correspond to a minimum retention time for stored data items. A minimum retention time may be defined by local regulations concerning stored data items (such that the data items will be available for a certain amount of time, should those data items be required for criminal investigations, for example), and/or defined by a policy that has been obtained by the data management system. It will be appreciated that different minimum retention times may apply to different data types (for example, video data, image data, audio data).

In some embodiments, the second time precedes a second deleting time, wherein the second deleting time is defined with respect to said first time at which the data item is stored. The second deleting time may correspond to a maximum retention time for stored data items. A maximum retention time may be defined by local regulations (such that the data items are deleted after a certain amount of time, in order to protect the privacy of any individuals captured in those data items, for example) concerning stored data times, and/or defined by a policy that has been obtained by the data management system. It will be appreciated that different maximum retention times may apply to different data types (for example, video data, image data, audio data).

In some embodiments, the method 300 further comprises the step of varying the stored predetermined function, wherein the varied predetermined function represents a different intended variation with time in a minimum permitted quality of the stored data item, and wherein the varied predetermined function comprises a different monotonically decreasing function. For example, a stored predetermined function may be varied in response to changing available resources at the data management system 200 and/or in response to requests sent by a user of the data management system 200.

Some examples of conventional methods of storing a data item at a data management system 200 are described with reference to Figures 4 and 5.

Figure 4 illustrates a time/quality plane, showing one example of how a quality measure of a stored data item may change over time at a data management system (for example, the data management system 200). It will be appreciated that the quality measure illustrated in Figure 4 may be any suitable quality measure that represents a quality of the data type of the stored data item. For example, in some embodiments, the stored data item comprises any one of video data, image data, or audio data. As described above, the quality of the data item may be defined in terms of a quality measure, wherein the quality measure relates to one or more quality-affecting characteristics of the data item. It will be appreciated that, where the quality of the data item is expressed in terms of multiple quality-affecting characteristics, the multiple quality-affecting characteristics may be combined to form a single quality measure. In this example, one or more of the quality-affecting characteristics of the data item may be adjusted such that the value of the quality measure obtained from the quality- affecting characteristics is greater than or equal to the minimum permitted quality measure that is currently represented by the predetermined function.

In this example, the quality measure Q is shown on the vertical axis, and Q_min represents a predetermined minimum acceptable quality measure for the stored data item (as described above). The predetermined minimum acceptable quality measure may be defined by a policy stored at the data management system 200, for example. The predetermined minimum acceptable quality measure may additionally or alternatively reflect the minimum acceptable quality for the stored data item that is permitted by a local regulation, for example. For example, when the data item is an image, the quality measure for the stored data item may be an image resolution, expressed in terms of a number of pixels in the image, and Q_min represents a predetermined minimum acceptable quality measure for the stored data item, for example a minimum number of pixels in the image, below which the image is useless.

Time is shown on the horizontal axis, starting from a time T₀, at which the data item is received.

Figure 4 also indicates a first replacing time, T_re , which is described in more detail below. Figure 4 also indicates a first deleting time, T_mm, and a second deleting time T_max, as described above.

Thus, the first deleting time, T_mm, represents the minimum time for which the rules or policy dictate that the data item must be kept, while the second deleting time T_max represents the maximum time for which the rules or policy dictate that the data item may be kept. These times may be defined by a policy stored at the data management system 200, for example. These times may additionally or alternatively reflect the minimum acceptable quality for the stored data item that is permitted by a local regulation, for example. In some systems, there may be no such minimum time T_mm, and/or there may be no such maximum time T_max.

Having described the time/quality plane, the specific example or use case illustrated in Figure 4 will now be described.

Specifically, Figure 4 represents a storage limited use case for a data management system 200, where the amount of available data storage is very limited, and so the amount of memory occupied by a data item is kept to a minimum. In this illustrated example, the data item is received at the time T₀, with a quality that is represented by a quality measure Qi that exceeds the predetermined minimum acceptable quality measure GW The data item is modified, so that it has the predetermined minimum acceptable quality measure Q_mm, and the data item is stored with this minimum acceptable quality measure Q_mm until the time of the stored data item is maintained until the time T_mm is reached.

When the time reaches T_mm, the first deleting time, as described above, the stored data item is deleted. This is represented in Figure 4 by the quality measure falling to the value of zero at this time. In other words, in this “storage-limited” example, a data item is stored with the minimum acceptable quality, and the stored data item is deleted at the first available opportunity. As a result of this, the maximum possible storage is freed in the shortest possible amount of time by the data management system 200. This freed storage may then be used to store new data items by the data management system 200. However, in this example, the original quality of the stored data item is irreversibly lost as soon as the item is stored, and the stored data item is completely lost after time T_mm. Figure 5 illustrates another example of how a quality measure of a stored data item may change overtime at a data management system (for example, the data management system 200). It will be appreciated that the quality measure illustrated in Figure 5 may be any suitable quality measure that represents a quality of the data type of the stored data item, as described above.

Figure 5 uses the time/quality plane described above with reference to Figure 4, in which the quality measure Q is shown on the vertical axis, and Q_mm represents a predetermined minimum acceptable quality measure for the stored data item. Time is shown on the horizontal axis, starting from a time T₀, at which the data item is received, with a first replacing time, T_re , a first deleting time, T_mm, and a second deleting time T _max, also being shown.

Figure 5 represents an example of a quality limited use case. In other words, in this example, storage at the data management system 200 is abundant, and data items are stored such that maximum quality is preserved.

In this illustrated example, the data item is received at the time T₀, with a quality that is represented by a quality measure Qi, and is stored with the same quality measure. The quality measure Qi of the stored data item is maintained until the time Ti, which is between T_min, the first deleting time, and T_max, the second deleting time.

At the time Ti, the stored data item is deleted. This is represented in Figure 5 by the quality measure falling to the value of zero at this time. As shown in Figure 5, in this example, the stored data item is deleted in advance of the time T_max, which corresponds to the second deleting time, as described above. This may be as a result of the stored data item no longer being required to be stored by the data management system (for example, it may be determined at this time that the stored data item will not be required for future analysis), rather than because storage is a limited resource in the data management system 200. Alternatively, the time at which the stored data item is to be deleted in this quality-limited example may be based on a policy stored at the data management system 200.

In other words, as storage is not limited in this example, the quality of the stored data item may be maintained at a maximum quality as long as it is required by the data management system 200.

However, in this quality limited example, the stored data item is either maintained at a maximum available quality, or it is removed from the data management system 200 completely. In other words, the stored data item is completely lost after time Ti.

Figure 6 illustrates another example of how a quality of a stored data item may change overtime at a data management system (for example, the data management system 200). Figure 6 again uses the time/quality plane described above with reference to Figure 4, in which the quality measure Q is shown on the vertical axis, and Q_mm represents a predetermined minimum acceptable quality measure for the stored data item. It will be appreciated that the quality measure illustrated in Figure 6 may be any suitable quality measure that represents a quality of the data type of the stored data item, as described above. The predetermined minimum acceptable quality measure Q_min may be defined by a policy stored at the data management system 200, for example. The predetermined minimum acceptable quality measure may reflect the minimum acceptable quality for the stored data item that is permitted by a local regulation, for example. Time is shown on the horizontal axis, starting from a time T₀, at which the data item is received, with a first replacing time, T_re , a first deleting time, T_mm, and a second deleting time T_max, also being shown.

It is now recognized that, in the time/quality plane, there is a region that defines a range of acceptable values for the quality, at each of a range of times. Specifically, at any time between the starting time T₀and the second deleting time T_max, it is acceptable for the quality measure to take any value between the initial quality measure Qi that represents the initial quality of the data item and the predetermined minimum acceptable quality measure G This is shown in Figure 6 as the region 602. Of course, when the quality is reduced, for example by discarding some of the data in the data item, it is probably impossible to increase the quality to what it was previously. Therefore, the path of the quality, and hence the quality measure, over time must be a monotonically decreasing function (that is, a function that never increases over any time period), although it may be either a continuous function or a discontinuous function (i.e. with sudden, effectively instantaneous) changes in the quality measure.

In this example, a reduction in the amount of storage that is used, while ensuring that the minimum acceptable quality is maintained, is enabled by the storing of a predetermined function at the data management system 200. As described above, the predetermined function represents an intended variation with time in a minimum permitted quality measure that represents the quality of the stored data item, and comprises a monotonically decreasing function. In other words, the predetermined function represents a lower bound for the quality of a stored data item as a function of time. It will be appreciated that the replacing of the stored data item with a new version of the stored data item is performed by the data management system 200 in accordance with the predetermined function, and thus the data management system may replace the stored data item any appropriate number of times and/or at any appropriate frequency, as long as the replacing is in accordance with the predetermined function. It will also be appreciated that, as the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item (in other words, a lower bound for the quality measure of a stored data item as a function of time), the stored data item may be replaced with a new version of the stored data item that has a quality that exceeds the minimum permitted quality given by the predetermined function at that particular time.

In the example as illustrated by the path 604, the predetermined function comprises a monotonically decreasing linear function, i.e. a function declining linearly from the initial quality measure Qi at the first deleting time T_minto the predetermined minimum acceptable quality measure Q_mm at the second deleting time T_max.

In the example as illustrated by the path 606, the predetermined function comprises a monotonically decreasing exponential function, i.e. a function declining exponentially from the initial quality measure Qi at the first deleting time T_minto the predetermined minimum acceptable quality measure Q_mm at the second deleting time T_max.

In these examples, the function is such that the quality measure is maintained at the initial quality measure Qi until the first deleting time T_mm. In other examples, the function may be such that the quality, and hence the quality measure, may start to decline immediately, i.e. from the starting time T₀.

In other examples, a first replacing time, T_re , may be defined. Figure 6 shows T_rep occurring before the first deleting time T_mm, but in other examples the first replacing time T_re may occur after the first deleting time T_mm. When a first replacing time T_re is defined, this may represent the earliest time at which the quality may start to decline.

It will be appreciated that exactly how the predetermined function monotonically decreases may depend on the data type of the of the stored data item. This may by dictated by a policy which is stored at the data management system 200, and/or may be specified by a user of the data management system 200. For example, it may be that if the stored data item comprises video data, the predetermined function that is stored may be a monotonically decreasing linear function. It will be appreciated that by storing a predetermined function based on the data type of the stored data item may allow stored data items comprising different data types to decay in different manners. It will be appreciated that a predetermined function may be stored based on any suitable feature of the stored data item.

Additionally or alternatively, exactly how the predetermined function monotonically decreases may depend on the likelihood that the stored data will be accessed and/or analyzed at a later time. This may be specified as part of a policy stored at the data management function. Additionally or alternatively, likelihood that the stored data will be accessed and/or analyzed at a later time and/or the predetermined function may be specified by a user of the data management system 200. Additionally or alternatively, the likelihood that the stored data will be accessed and/or analyzed at a later time and/or the predetermined function may be based on analytics that have been obtained by the data management system 200. For example, the predetermined function may be based on statistics that have been gathered from previously accessed data items. For example, the likelihood that a stored data item will be accessed and/or analyzed at a later time may be 75% within 0 and 10 days of the data item being stored, may be 20% within 10 to 20 days of the data item being stored, and may be 5% within 20 to 30 days of the data item being stored. These percentages may have been specified by a user of the data management system 200, and/or may be based on analytics obtained by the data management system 200. The predetermined function that is stored may then correspond to these likelihood percentages.

As described above, the predetermined function may be varied at any suitable point following the storing of the data item. For example, the predetermined function may be varied at a certain time such that the gradient of the predetermined function is changed from that time onwards. Additionally or alternatively, the predetermined function may be varied at a certain time such that the predetermined function is changed from a monotonically decreasing linear function to a monotonically decreasing exponential function (or vice versa) from that time onwards.

The predetermined function may be varied based on one or more of a user request, a policy stored at the data management system 200, or analytics obtained at the data management system 200. This may allow a more dynamic data management system 200 to be provided, that is able to vary the manner in which a quality of a stored data item decays in response to changing available resources at the data management system 200 and/or in response to requests sent by a user of the data management system 200.

Figure 6 also illustrates an example, where the data item is received at the starting time To, with a quality that is represented by a quality measure Qi. The data item is stored with the quality measure Qi, and a linearly declining function illustrated by the line 604 is also stored.

The data item is stored with the original quality measure Qi until the time T₂, as shown by the bold line 608.

At the time T₂, the quality of the stored data item is degraded, so that the quality measure becomes equal to Q₃. For example, this may take place because available storage resources are becoming limited. Alternatively, the system may be pre- programmed such that the quality is automatically reduced in accordance with the stored function.

As described above, the quality of the stored data item is adjusted via the process of storing a new version of the stored data item. For example, in the case of a data item in the form of an image, and depending on the form of the quality measure, the quality of the stored data item may be degraded by reducing the image resolution. In the case of stored video data, the quality of the stored data item may for example be degraded by reducing the bit depth from 8 to 7, or by reducing the resolution from 4k to Full HD, or form Full HD to HD, etc.

As can be seen from Figure 6, at the time T₂, the quality measure Q₃ indicated by the line 608 is above the line 604 defining the stored function, and so the quality measure representing the degraded quality still exceeds the minimum permitted quality measure defined by the function.

The data item is stored with the reduced quality measure Q₃ until the time T₃. In this example, the time T₃ is equal to the second deleting time T_max, as described above.

At the time T₃, the current version of the stored data item is deleted. This is represented in Figure 6 by the quality measure falling to the value of zero at this time.

In this example, the time T₃ is equivalent to the second deleting time, as described above. In other words, the rules dictate that the current version of the stored data item must be deleted at this time. The second deleting time may be defined at a policy stored at the data management system 200. Additionally or alternatively, the second deleting time may be dictated by local regulations.

Figure 6 only shows one step reduction in the quality between the initial storage at the time To and the deletion of the data item at the time T₃, but it will be appreciated that any desired number of reductions may be made, provided that the quality measure at any time is still greater than or equal to the minimum permitted quality measure defined by the stored function.

Thus, by storing a predetermined function, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, a data management system 200 may allow the quality of a stored data item to vary overtime. This may allow for both storage to be freed within the data management system 200 while simultaneously storing a version of the data item that maintains at least a minimum quality as defined by policy and/or a user of the data management systems 200. Thus, should the stored data item be required to be accessed and/or analyzed following the storage of the data item, the stored data item will always be of at least the minimum quality that allows this access and/or analysis to occur.

Furthermore, the retention time for the stored data item may be maximized, as the storage is freed within the data management system 200 without requiring that the stored data item is deleted. This may allow access and/or analysis of the stored data item to occur as late as the maximum time that is permitted by local regulations.

Furthermore, by varying the quality of a stored data item in this time dependent manner, storage may be preferentially allocated to data items that have been recently stored by the data management system 200. Thus, stored data items that are more likely to be accessed and/or analyzed will be allocated greater storage, and be of a greater quality, allowing higher quality analysis to be performed on the stored data items that are more likely to be accessed.

Messages may be received at the data management system 200 from any suitable network node in the network 100, in order to implement the above described methods. It will be appreciated that these example messages may be serialized in any appropriate manner (for example, the messages may be serialized in XML). It will also be appreciated that these example messages may be applied to any other suitable API endpoint. There is therefore provided a method and apparatus for defining a minimum permitted quality of a stored data item as a function of time.

It should be noted that the above-mentioned embodiments illustrate rather than limit the concepts disclosed herein, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended following statements. The word “comprising” does not exclude the presence of elements or steps other than those listed in a statement, “a” or “an” does not exclude a plurality, and a single processor or other unit may fulfil the functions of several units recited in the statements. Any reference signs in the statements shall not be construed so as to limit their scope.

Claims

1. A method of storing a data item at a data management system, the method comprising: storing the data item at a first time, wherein the stored data item has an initial quality; and storing a predetermined function, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, and wherein the predetermined function comprises a monotonically decreasing function.

2. A method according to claim 1 , wherein the quality is defined in terms of a quality measure, wherein the quality measure relates to one or more quality-affecting characteristics of the data item.

3. A method according to claim 2, wherein the quality measure is defined as a function of the one or more quality-affecting characteristics of the data item.

4. A method according to claim 3, wherein a value of the function varies monotonically as the quality of the data item is varied.

5. A method according to any preceding claim, wherein the minimum permitted quality represented by the predetermined function does not fall below a predetermined minimum acceptable quality.

6. A method according to any preceding claim, wherein the method further comprises: replacing the stored data item with a new version of the stored data item in accordance with the predetermined function at at least one time succeeding the first time, such that each new version of the stored data item has a reduced quality compared with each previous version of the stored data item.

7. A method according to claim 6, wherein the step of replacing the stored data item with a new version of the stored data item in accordance with the predetermined function comprises replacing the stored data item with a new version of the stored data item that has a quality greater than or equal to the minimum permitted quality that is currently represented by the predetermined function.

8. A method according to claim 6 or 7, wherein the at least one later time does not precede a first replacing time, wherein the first replacing time is defined with respect to said first time at which the data item is stored.

9. A method according to any preceding claim, wherein the method further comprises: at a second time, later than the first time, deleting the current version of the stored data item.

10. A method according to claim 3, wherein the current version of the stored data item is not deleted prior to a first deleting time, wherein the first deleting time is defined with respect to said first time at which the data item is stored.

11. A method according to either claim 3 or claim 4, wherein the second time precedes a second deleting time, wherein the second deleting time is defined with respect to said first time at which the data item is stored.

12. A method according to any preceding claim, wherein the method further comprises: varying the predetermined function, wherein the varied predetermined function represents a different intended variation with time in a minimum permitted quality of the stored data item, and wherein the varied predetermined function comprises a different monotonically decreasing function.

13. A method according to any preceding claim, wherein the predetermined function comprises a monotonically decreasing linear function.

14. A method according to any of claims 1 to 2, wherein the predetermined function comprises a monotonically decreasing exponential function.

15. A method according to any preceding claim, wherein the stored data item comprises any one of video data, image data, or audio data.

16. A method according to claim 15, wherein, when the stored data item comprises video data, the quality is defined by at least one of a frame rate and a video resolution.

17. A method according to claim 15, wherein, when the stored data item comprises image data, the quality is defined by an image resolution.

18. A method according to claim 16 or 17, wherein the resolution is expressed in one or more of: a number of pixels, a maximum number of colours, a brightness, and a dynamic range.

19. A method according to claim 15, wherein, when the stored data item comprises audio data, the quality is defined by at least one of a sample rate and a quantization level.

20. A data management system, comprising a processor, wherein the processor is configured to perform a method of storing a data item, the method comprising: storing the data item in a memory at a first time, wherein the stored data item has an initial quality; and storing a predetermined function in said memory, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, and wherein the predetermined function comprises a monotonically decreasing function.

21. A data management system according to claim 20, further comprising said memory.

22. A data management system according to claim 20, wherein the processor is configured to store the data item in a remote memory.

23. A computer program product, comprising a tangible and/or non-transient computer readable medium, wherein the computer readable medium contains instructions for performing a method of storing a data item, the method comprising: storing the data item in a memory at a first time, wherein the stored data item has an initial quality; and storing a predetermined function in said memory, wherein the predetermined function represents an intended variation with time in a minimum permitted quality of the stored data item, and wherein the predetermined function comprises a monotonically decreasing function.