WO2016195545A1 - Authenticating data recording devices - Google Patents

Authenticating data recording devices Download PDF

Info

Publication number
WO2016195545A1
WO2016195545A1 PCT/SE2015/050619 SE2015050619W WO2016195545A1 WO 2016195545 A1 WO2016195545 A1 WO 2016195545A1 SE 2015050619 W SE2015050619 W SE 2015050619W WO 2016195545 A1 WO2016195545 A1 WO 2016195545A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
recording device
features
data recording
time interval
Prior art date
Application number
PCT/SE2015/050619
Other languages
French (fr)
Inventor
Debmalya BISWAS
Lars Andersson
Matthew John LAWRENSON
Original Assignee
Telefonaktiebolaget Lm Ericsson (Publ)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget Lm Ericsson (Publ) filed Critical Telefonaktiebolaget Lm Ericsson (Publ)
Priority to PCT/SE2015/050619 priority Critical patent/WO2016195545A1/en
Publication of WO2016195545A1 publication Critical patent/WO2016195545A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W12/00Security arrangements; Authentication; Protecting privacy or anonymity
    • H04W12/06Authentication
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/10Network architectures or network communication protocols for network security for controlling access to devices or network resources
    • H04L63/107Network architectures or network communication protocols for network security for controlling access to devices or network resources wherein the security policies are location-dependent, e.g. entities privileges depend on current location or allowing specific operations only from locally connected terminals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/04Protocols specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/52Network services specially adapted for the location of the user terminal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W4/00Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02Services making use of location information
    • H04W4/023Services making use of location information using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W12/00Security arrangements; Authentication; Protecting privacy or anonymity
    • H04W12/30Security of mobile devices; Security of mobile applications
    • H04W12/33Security of mobile devices; Security of mobile applications using wearable devices, e.g. using a smartwatch or smart-glasses
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W12/00Security arrangements; Authentication; Protecting privacy or anonymity
    • H04W12/60Context-dependent security
    • H04W12/63Location-dependent; Proximity-dependent

Definitions

  • the invention relates to a method of authenticating a first data recording device and/or a second data recording device, a first data recording device for authenticating a second data recording device, a data collecting device for authenticating a first data recording device and a second data recording device, and corresponding computer programs and computer program products.
  • Lifelogging is a growing trend, where people use phones, wearables, and dedicated lifeloggers, to document aspects of their daily lives. Lifelogging objectives include memory augmentation, reliving old memories in vivid detail, and optimizing one's daily routines.
  • Future lifelogging scenarios are expected to include devices not only worn by the user, but recordings will be sourced from multiple devices, such as devices at the user's home or office, or devices in environments which the user frequently visits, e.g., a friend's home. A majority of these devices is expected to provide wireless connectivity for accessing or retrieving data recorded by the devices. This will allow devices to collaborate, often in an ad- hoc fashion, to capture a more multi-dimensional log of the user's daily life.
  • lifelogging devices are now capable of recording a diverse range of user data, ranging from health attributes, such as physiological data, to location traces by means of position data, to prevailing environmental conditions, e.g., pollution or temperature, to more traditional audio/video recording of the user's activities. It is expected that multiple devices belonging to, or available to the user, including wearables, phones, tablets, Internet-of-Things (loT) devices in the user's home, office, etc., are involved in the recording process. Accordingly, the number of devices recording data is expected to increase manifold.
  • health attributes such as physiological data
  • location traces by means of position data
  • prevailing environmental conditions e.g., pollution or temperature
  • While the devices can independently and autonomously record data representing different aspects of the user's life, there is a need to pair lifelogging devices, or associate lifelogging devices with each other, for the purpose of syncing or sharing recorded data, fusing recorded data, or coordinating devices to optimize the recording process. Paring of lifelogging devices, herein referred to as data recording devices, necessitates the need for such devices to authenticate each other, or authenticate themselves to a third device or party, e.g., a network storage or a cloud (storage) provider, a computer used for collecting and possibly analyzing recorded data, or the like.
  • a third device or party e.g., a network storage or a cloud (storage) provider, a computer used for collecting and possibly analyzing recorded data, or the like.
  • a method of authenticating a first data recording device and/or a second data recording device comprises acquiring first data, acquiring second data, determining an overlap strength between the first data and the second data, and authenticating the first data recording device and/or the second data recording device if the determined overlap strength fulfills an authentication condition.
  • the first data is recorded by the first data recording device during at least one first time interval
  • the second data is recorded by the second data recording device during at least one second time interval.
  • the overlap strength is determined based on one or more features of the first data and one or more features of the second data.
  • a computer program comprises computer-executable instructions
  • a computer program product comprises a computer- readable storage medium which has the computer program according to the second aspect of the invention embodied therein.
  • a first data recording device for authenticating a second data recording device comprises at least one sensor, a data storage, a communication interface, and processing means.
  • the at least one sensor is configured for recording first data during at least one first time interval.
  • the at least one sensor may, e.g., be at least one of a camera, a microphone, a positioning device, a pulse sensor, a heart-rate sensor, a thermometer, or a barometer.
  • the data storage is configured for storing the first data.
  • the communication interface is configured for communicating with the second data recording device.
  • the processing means is operative to acquire first data, acquire second data, determine an overlap strength between the first data and the second data, and authenticate the second data recording device if the determined overlap strength fulfills an authentication condition.
  • the first data is acquired from the data storage.
  • the second data is recorded by the second data recording device during at least one second time interval, and is acquired via the communication interface.
  • the overlap strength is determined based on one or more features of the first data and one or more features of the second data.
  • a data collecting device for authenticating a first data recording device and a second data recording device.
  • the data collecting device comprises a communication interface and processing means.
  • the communication interface is configured for communicating with the first data recording device and the second data recording device.
  • the processing means is operative to acquire first data, acquire second data, determine an overlap strength between the first data and the second data, and authenticate the first data recording device and the second data recording device if the determined overlap strength fulfills an authentication condition.
  • the first data is recorded by the first data recording device during at least one first time interval and acquired via the
  • the second data is recorded by the second data recording device during at least one second time interval and acquired via the communication interface.
  • the overlap strength is determined based on one or more features of the first data and one or more features of the second data.
  • a data recording device herein also referred to as lifelogger or lifelogging device, may, e.g., be a dedicated device configured for capturing audio and/or video, a smartphone, a mobile terminal, digital cameras, webcams, surveillance cameras, a Machine-to-Machine (M2M) or loT-type of device, a smart watch, a wearable device or wearable, and so forth.
  • M2M Machine-to-Machine
  • the proposed authentication scheme is based on the idea that data recorded by two different data recording devices, herein referred to as first data and second data, share common features which represent an event or an object which both data recording devices have captured. Based on a comparison of the two data sets, an assessment can be made as to whether the two data recording devices have a common past, i.e., have both been in proximity to each other and to a captured event or object during the time of recording the data.
  • the at least one first time interval and the at least one second time interval preferably overlap during at least one common time interval, during which both data recording devices have captured an event or an object by recording data.
  • the at least one first time interval and the at least one second time interval may be substantially consecutive. This may be the case for lifeloggers which periodically capture still images, e.g., every 30 seconds, but not simultaneously.
  • the first time interval and the second time interval are the respective shutter times used for capturing the images. If the two time intervals do not overlap, an assessment of whether the two data recording devices share a common past may still be made depending on the time which has lapsed between the non-overlapping first time interval and the second time interval, as well as the nature of the captured event or object. For instance, the second data recording device may capture an image shortly after the first data recording device, such that the same event or object is visible in both captured images.
  • the authentication scheme proposed herein advantageously relies on preferably unique features of the first data and the second data, i.e., subsets of the recorded data which represent an event or object which is identifiable by analyzing the recorded data, such as image processing or audio processing.
  • Embodiments of the invention may utilize any suitable type of recorded data, e.g., video data, image data, audio data, position data, environmental data, or physiological data.
  • two data recording devices recording video or periodically captured images and being located in proximity to each other, such as two people sitting in a restaurant, each having a camera lifelogger.
  • the two data sets recorded by these lifeloggers contain common features, representative of events or objects which both lifeloggers have captured. This may, e.g., be a third person passing through the restaurant and being captured by both lifeloggers.
  • both lifeloggers may capture a certain object, such as a third person sitting in the restaurant, a distinct piece of furniture, or the like.
  • time stamps which facilitate synchronization of the two different data sets, it can be established that the two lifeloggers have been in proximity at the time when a certain event or object was captured. That is, they share a common past.
  • embodiments of the invention may rely on fuzzy rather than exact matches between features of the first data and the second data, with respect to scene characteristics, objects in the scene, appearance or disappearance of an object at the same instant from the scene, and so forth.
  • the two lifeloggers may also capture different types of data, e.g., video and audio, respectively.
  • the common past i.e., a proximity of the two lifeloggers during a time period in the past, can be established based on an event which is
  • the video lifelogger may capture video footage of a closing door, whereas the audio lifelogger captures a distinct sound made by the closing door. That is, corresponding features are common features present in sets of recorded data having different types.
  • an overlap strength is calculated as a measure of a likelihood that the first data recording device and the second data recording device share a common past.
  • the overlap strength increases with a number of features which are common to both data sets, the presence of a common feature which is particularly unique, an increased correlation between the two data sets, or the length of a time period during which the two data sets are in satisfactory agreement with each other.
  • authentication condition may, e.g., be considered fulfilled if the overlap strength is above a threshold value, or if a certain number of common features are identified.
  • the threshold value based on which authentication commences, may be adjusted such that a desired confidence level indicative of a common past is achieved.
  • Embodiments of the invention are advantageous in that authentication of lifelogging data recording device is simplified.
  • the proposed solution does not require passwords, PIN codes, or similar, which are tedious to enter, in particular on lifeloggers of small size which are typically provided with only few buttons and lack a display.
  • the overlap strength is determined by calculating a correlation between the first data and the second data. That is, a mathematical correlation function is used for evaluating the statistical correlation between two random variables, such as the first data and the second data, or variables derived from the first data and the second data, e.g., the time-dependent brightness of a video sequence or the time- dependent audio level of an audio recording.
  • a mathematical correlation function is used for evaluating the statistical correlation between two random variables, such as the first data and the second data, or variables derived from the first data and the second data, e.g., the time-dependent brightness of a video sequence or the time- dependent audio level of an audio recording.
  • Different types of mathematical correlation functions such as cross correlation and auto correlation, are known in the art. In this case, the overlap strength increases with increasing value of the correlation function.
  • the determining an overlap strength comprises identifying one or more corresponding features of the features of the first data and the features of the second data, and determining the overlap strength based on a number of the corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof.
  • lifeloggers recording different types of data may be authenticated based on corresponding features, such as video footage capturing a closing door and an audio recording capturing the sound made by a closing door.
  • the overlap strength increases with the number of identified corresponding features.
  • the determining an overlap strength comprises generating a challenge based on the features of the first data, determining a response to the challenge based on the features of the second data, and verifying whether the response matches the challenge.
  • the authentication condition is fulfilled if the response matches the challenge, i.e., if the compared features are in fact
  • the first lifelogger may pose an authentication challenge to the second lifelogger, requesting the second lifelogger to respond with the time when the third person passes.
  • the second lifelogger analyzes its recorded time-stamped video, finds the third person by means of image recognition, and responds with the time when the third person passes.
  • the first lifelogger may detect that the third person wears a clothing of a distinct color, and challenges the second lifelogger to respond with the color of the clothing, which can be determined by means of image analysis.
  • the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data, wherein the part of the first data and the part of the second data are selected based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data.
  • the part of the first data and the part of the second data may be selected based on a proximity of the first data recording device and the second data recording device during recording of the part of the first data and the part of the second data, respectively, i.e., when the data recording devices were close to each other.
  • the proximity of the two data recording devices may, e.g., be determined based on position information received from a Global Positioning System (GPS) module, or the like, which the data recording devices may optionally be provided with.
  • GPS Global Positioning System
  • their recorded data may be tagged with positioning information.
  • Whether the two data recording devices are considered to be in proximity of each other depends not only on the absolute distance between the devices, but also on the devices' location and the type of data they record. For instance, the two data recording devices may be considered in proximity of each other is they are in the same room.
  • the overlap strength between the first data and the second data is determined based on a part of the features of the first data and/or a part of the features of the second data, wherein the part of the features of the first data and/or the part of the features of the second data are selected based on a uniqueness of the features.
  • the part of the features of the first data and/or the part of the features of the second data may be selected based on a proximity of the first data recording device and the second data recording device during recording of a part of the first data associated with the selected features of the first data and a part of the second data associated with the selected features of the second data, respectively, i.e., when the data recording devices were close to each other, as was discussed hereinbefore.
  • the first data recording devices attempts to authenticate the second data recording device.
  • the first data is acquired from a data storage accessible by the first data recording device, such as an internal or external memory, hard drive, or the like
  • the second data is acquired by receiving the second data from the second data recording device, e.g., via a wired or wireless
  • the second data recording device may subsequently authenticate the first data recording device, resulting in a mutual authentication of the two data recording devices.
  • a data collecting device i.e., a third device which is different from the first data recording device and the second data recording device, attempts to authenticate the first data recording device and the second data recording device.
  • This may, e.g., be a smartphone, a computer, a tablet, a cloud storage, or the like, which a user of the lifeloggers utilizes for collecting, analyzing, or fusing, recorded data.
  • the first data is acquired by receiving the first data from the first data recording device
  • the second data is acquired by receiving the second data from the second data recording device, e.g., via wired or wireless communication links.
  • Fig. 1 illustrates an exemplifying scenario for usage of data recording devices (lifeloggers), in accordance with embodiments of the invention.
  • Fig. 2 illustrates time periods during which data recording devices in accordance with embodiments of the invention capture an event or an object.
  • Fig. 3 shows a flow chart for a method of authenticating a first data recording device and/or a second data recording device, in accordance with an embodiment of the invention.
  • Fig. 4 shows a sequence diagram illustrating authentication of a data recording device, in accordance with an embodiment of the invention.
  • Fig. 5 shows a sequence diagram illustrating authentication of two data recording devices, in accordance with another embodiment of the invention.
  • Fig. 6 shows a data recording device, in accordance with an embodiment of the invention.
  • Fig. 7 shows a data collecting device, in accordance with an embodiment of the invention.
  • Fig. 8 shows a data recording device, in accordance with another embodiment of the invention.
  • Fig. 9 shows a data collecting device, in accordance with another embodiment of the invention.
  • FIG. 1 an exemplifying scenario for usage, in particular
  • a first data recording device 1 10 travels along a route 1 1 1 into a location 101 , such as a room, a restaurant, or the like.
  • First data recording device 1 10 may, e.g., be carried by a user, and may be any one of a lifelogger capturing images or video, a mobile phone or smartphone capturing images, video, or audio.
  • route 1 1 1 is illustrated as having two legs, a first leg 1 1 1 A along which first data recording device 1 10 travels before entering location 101 , and a second leg 1 1 1 B along which first data recording device 1 10 travels after entering location 101 .
  • a second data recording device 120 travelling along a route 121 which is, similar to route 1 1 1 , illustrated as having two legs 121 A and 121 B.
  • Second data recording device 120 may be of the same or similar type as first data recording device 1 10, or of a different type.
  • the two data recording devices 1 10 and 120 are in Fig. 1 illustrated as coming to rest in proximity to each other. This may, e.g., be the case if the two users carrying data recording devices 1 10 and 120, respectively, sit down at a table 102.
  • the users, as well as data recording devices 1 10 and 120 share a common past. For instance, if the two users simultaneously arrive at entrance 104 of location 101 , e.g., because they have agreed to meet outside restaurant 101 , the two data recording devices share a common past starting at entrance 101 and extending along their respective routes 1 1 1 B and 121 B as long as they stay in proximity of each other.
  • the users' common past is reflected by data recorded at each of the data recording devices 1 10 and 120, first data recorded by first data recording device 1 10 and second data recorded by second data recording device 120.
  • the first data and the second data may be of the same type or of different types.
  • Types of first and second data include, but are not limited to, video data, image data, audio data, position data, environmental data, physiological data, and the like.
  • the first data is recorded during at least a first time interval. It may be recorded continuously, during successive time intervals or periodic time intervals, recording may be switched on and off by the user, and so forth. This is illustrated in Fig. 2, where recording the first data 21 1 is exemplified. For instance, recording first data 21 1 may start at time and stop at time fe- It may continue between t c and t D , when recording stops before it is continued at f E and stops at f F - Thus, first data 21 1 is in Fig. 2 illustrated as being recorded during three successive time intervals. The second data 212 is recorded correspondingly, in Fig. 2 illustrated as being recorded during two time intervals.
  • the at least one first time interval and the at least one second time interval at least partially overlap during at least one common time interval, illustrated by means of a diagonal pattern.
  • Authentication of first 1 10 and/or second 120 data recording device in accordance with embodiments of the invention may be performed based on data recorded during the at least one common time interval. This is the case since data recorded simultaneously at both data recording devices 1 10 and 120 may represent a common past, i.e., representing one or more events or objects captured by both data recording devices 1 10 and 120, if in proximity to each other when capturing the event(s) or object(s).
  • authentication of first 1 10 and/or second 120 data recording device in accordance with embodiments of the invention may be performed based on data recorded during respective time intervals which are substantially consecutive, as is illustrated in the lower part of Fig. 2. This may, e.g., be the case if data recording devices 1 10 and 120 record data discontinuously, e.g., periodically capturing still images.
  • the first data and the second data may still be used for the purpose of authentication, if it can be determined that the two data recording devices share a common past. This is the case if the first data and the second data represent an event or object which has been captured by both data recording devices. For instance, with reference to Fig.
  • both data recording devices 1 10 and 120 may capture an event, such as a person 130 passing along route 131 , if second data recording device 120 captures an image shortly after first data recording device 1 10, such that person 130 is visible in both captured images.
  • both data recording devices 1 10 and 120 may capture images of an object, such as a piece of furniture or the like.
  • One of the data recording devices 1 10 or 120 may also capture the other data recording device in an image or a video.
  • Fig. 3 shows a flow chart for an embodiment 300 of a method of the invention.
  • Method 300 is initiated by a request 301 for authenticating first data recording device 1 10, second data recording device 120, or both.
  • Authentication may be initiated by a user, in response to interconnecting or pairing the two data recording devices 1 10 and 120 with each other or with a third device for collecting data, as is described further below, in response to detecting a proximity of the first data recording device 1 10 and the second data recording device 120 to each other or to the third device, or the like.
  • Method 300 comprises acquiring 302 the first data recorded by first data recording device 1 10 during at least one first time interval, acquiring 303 the second data recorded by second data recording device 120 during at least one second time interval, determining 305 an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and authenticating 307 first data recording device 1 10 and/or second data recording device 120 if the determined overlap strength fulfills an authentication condition 306. If authentication condition 306 is not fulfilled, authentication is rejected 308.
  • Determining 305 the overlap strength aims at quantifying the shared knowledge of the common past of the two data recording devices, which shared knowledge is represented by the recorded data. Based on that quantification, authentication 307 is performed if the authentication condition 306 is fulfilled. It will be appreciated that the level of security may be adjusted by selecting the authentication condition accordingly, such that a desired level of confidence indicative of a common past is achieved. That is, if a higher level of security is required, a more stringent authentication condition is selected, whereas a more lenient authentication condition is selected if a lower level of security is required.
  • the relation between quantifying the overlap and the authentication condition is exemplified in further detail below.
  • first data recording device 1 may be performed by first data recording device 1 10, as is further illustrated in Fig. 4.
  • the first data is acquired 302 from a data storage accessible by first data recording device 1 10, e.g., an internal or external storage such as a computer memory, hard drive, memory stick, Random Access Memory (RAM), or Flash memory, where data is stored after being captured by a suitable sensor, such as a camera, a microphone, a positioning module, or the like.
  • a suitable sensor such as a camera, a microphone, a positioning module, or the like.
  • the second data is acquired 303 by receiving the second data from second data recording device 120, e.g., by means of a wired or a wireless communication link such as Universal Serial Bus (USB), FireWire, Thunderbolt, Ethernet, Wireless Local Area Network (WLAN)/WiFi, Bluetooth, a cellular network such as a Global System for Mobile Communications (GSM) network, an Universal Mobile Telecommunications System (UMTS) network, or a Long Term Evolution (GSM) network, an Universal Mobile Telecommunications System (UMTS) network, or a Long Term
  • the second data may be made available to first data recording device 1 10 by transferring 303 the data using a memory stick or the like.
  • first data recording device 1 10 such as a smartphone or tablet
  • second data recording device 120 may also authenticate second data recording device 120 for the purpose of coordinating capturing or recording of data.
  • a master data recording device such as a smartphone
  • Such a master data recording device may, e.g., control other data recording devices to start and stop recording of data depending on the location of the user in relation to the other data recording devices. This is advantageous for data recording devices which are not carried by the user but which are deployed in the user's home, office, or the like.
  • first data recording device 1 10 and second data recording device 120 may prepare or arbitrate 401 authentication.
  • Arbitrating 401 authentication may, e.g., include transmitting a request for authentication or pairing, exchanging information pertaining to the second data recorded by second data recording device 120, e.g., time periods during which the second data has been recorded, or position information indicating at which location or locations the second data has been recorded, and so forth. Based on such information, parts of the first data and parts of the second data which are used for quantifying the shared knowledge of the two data recording devices about their common past may be selected 304, as is described further below. It will be appreciated that such selection 304 may be performed prior to acquiring the first data and/or the second data, i.e., as part of the arbitration 401 indicated in Fig. 4, or after, as is indicated in Fig. 3.
  • first data recording device 1 10 may acquire only parts of the first data which were recorded during the at least one common time interval, and/or when first data recording device 1 10 and second data recording device 120 were in proximity to each other. As a consequence, the amount of data which is transferred between devices is reduced, thereby preserving battery power.
  • first data recording device 1 10 may perform further processing 409 after successful authentication, such as analyzing, storing, or fusing, the first and/or the second data. Furthermore, first data recording device 1 10 may also engage in further
  • communications 410 with second data recording device 120 after successful authentication e.g., for the purpose of coordinating recording of data, as was described hereinbefore.
  • method 300 may be performed by a data collecting device (not shown in Fig. 1 ) which is separate from first data recording device 1 10 and second data recording device 120, as is illustrated in Fig. 5.
  • the data collecting device 140 may, e.g., be a smartphone, a tablet, a computer, a cloud storage, or the like, which is used for collecting, analyzing, or fusing, data recorded by at least two data recording devices, such as first data recording device 1 10 and second data recording
  • the first data is acquired 302 by receiving the first data from first data recording device 1 10, and the second data is
  • the first and the second data may, e.g., be received by means of a wired or a wireless communication link, similar to what is described above.
  • the first data and the second data may be made available to data collecting device 140 by transferring the data using a memory stick or the like.
  • data collecting device 140 may also
  • data collecting device 140 may have the role of a master data recording device, such as a smartphone of the user, for controlling capturing of data by other data recording devices.
  • a master data recording device may, e.g., control other data recording devices to start and stop recording of data depending on the location of the user in relation to the other data recording devices. This is advantageous for data recording devices which are not carried by the user but which are deployed in the user's home, office, or the like.
  • first data recording device 1 10 and second data recording device 120 may prepare or arbitrate 501 authentication.
  • Arbitration 501 authentication may, e.g., include transmitting a request for authentication or pairing, exchanging information pertaining to the first data recorded by first data recording device 1 10 and second data recorded by second data recording device 120, e.g., respective time periods during which the first and the second data have been recorded, or position information indicating at which respective location or locations the first data and the second data have been recorded, and so forth. Based on such information, parts of the first data and parts of the second data which are used for quantifying the shared knowledge of the two data recording device may be selected 304, as is described further below. It will be appreciated that such selection 304 may be performed prior to acquiring the first data and/or the second data, i.e., as part of the arbitration 501 indicated in Fig.
  • data collecting device 140 may acquire only parts of the first data and/or the second data which were recorded during the at least one common time interval, and/or when first data recording device 1 10 and second data recording device 120 were in proximity to each other. As a consequence, the amount of data which is transferred between devices is reduced, thereby preserving battery power.
  • data collecting device 140 may perform further processing 509 after successful authentication, such as analyzing, storing, or fusing, the first and/or the second data. Furthermore, data collecting device 140 may also engage in further communications 510 with first data recording device 1 10 and/or second data recording device 120 after successful authentication, e.g., for the purpose of coordinating recording of data, as was described hereinbefore.
  • the overlap strength between the first data and the second data may be determined 305 based on any method suitable for assessing whether data recording devices 1 10 and 120 share a common past. This is achieved by quantifying the similarity of the first data and the second data, or the amount of common features, i.e., features which are present in the first data and the second data.
  • the overlap strength may be determined 305 by calculating a correlation between the first data and the second data. That is, using a correlation function the statistical correlation between the first data and the second data, which are random variables, is evaluated.
  • a correlation function the statistical correlation between the first data and the second data, which are random variables, is evaluated.
  • Different types of correlation functions are known in the art.
  • an autocorrelation function may be used for evaluating the statistical correlation, i.e., the overlap, between random variables representing the same quantity measured at two different locations. This is the case if the first data and the second data are of the same type, e.g., video footage captured by the two data recording devices, such as video lifeloggers.
  • a cross correlation function may be used for evaluating the statistical correlation, and hence the overlap strength, between the first data and the second data.
  • Correlations functions may be used both on scalar-valued data, such as temperature, brightness of a video sequence, or level of an audio recording, and non-scalar data, such as video. It will also be appreciated that the statistical correlation may be evaluated between variables derived from the first data or the second data, respectively. For instance, one may envisage embodiments of the invention performing authentication based on determining the overlap strength between the time-dependent brightness of a video sequence, the first data, recorded by the first data recording device 1 10 and the time-dependent brightness of a video sequence, the second data, recorded by the second data recording device 120.
  • authenticating the first and/or the second data recording device based on the brightness of recorded video may be sufficient if the two data recording devices have recorded data during a sufficiently long common time interval, and/or if the respective data sets comprise sufficient unique or distinct features. This may, e.g., be the case if the two data recording devices 1 10 and 120 have travelled together for an extended period of time in an environment of varying illumination, resulting in sufficient variations in the derived brightness.
  • the correlation function used for determining 305 the overlap strength may be normalized such that a perfect correlation, i.e., the first data and the second data being identical, yields a value of unity, or any other desired maximum value.
  • the authentication condition 306 may be considered fulfilled if the overlap strength is above a threshold value, e.g., 80% of the maximum value or a value of 0.8 in case unity is used for perfect correlation. It will be appreciated that any suitable value may be used, where an increased threshold value is associated with an increased level security level of the authentication.
  • An alternative method for determining 305 the overlap strength between the first data and the second data comprises identifying one or more corresponding features of the features of the first data and the features of the second data, and determining the overlap strength based on a number of the corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof.
  • data recording devices 1 10 and 120 may record different types of data, such as video and audio, respectively.
  • the overlap strength may be determined 305 based on corresponding features, i.e., common features of the first data and of the second data which represent events or objects captured by both data recording devices, albeit with different types of data.
  • first data recording device 1 10 may be a video lifelogger capturing the closing of door 103 after person 130 has entered location 101
  • second data recording device 120 may be a smartphone recording audio and capturing the sound of closing door 103.
  • the overlap strength increases with the number of identified corresponding features in relation to the number of identified features in one of the first data or the second data, or the total number of features in the first data and the second data.
  • determining 305 the overlap strength between the first data and the second data may comprise generating an authentication challenge based on one or more features of the first data, determining a response to the challenge based on the features of the second data, and verifying whether the response matches the challenge, i.e., whether the respective features of the first data and the second data are in fact common features.
  • the authentication condition is fulfilled if the response matches the challenge.
  • the authentication condition may be
  • first data recording device 1 10 may analyze its recorded first data to identify a suitable feature, i.e., a captured person, object, or event, for posing a challenge.
  • first data recording device 1 10 may determine, by image processing its recorded video data (the first data), that third
  • First data recording device may then determine a color of the clothing of third person 130, by analyzing the pixels of an image capturing third person 130 when passing. Based on this analysis, first data recording device 1 10 may then send a challenge to second data recording device 120, such as a request to determine the color of a person or an object which passes at a certain point in time, or within a short time interval around that point in time.
  • second data recording device 120 analyzes its recorded video data (the second data) at the specified point in time and determines a color of a person or object which passes.
  • the determined color is sent to first data recording device 1 10, using any suitable means of color coding known in the art, such as HTML color codes, RGB color codes, or the like.
  • First data recording device 1 10 compares the color received in the response from second data recording device 120 and authenticates second data recording device 120 if the color determined by second data recording device 120 matches the color determined by first data recording device 1 10.
  • the color of a person, clothing of a person, object, or the like may, e.g., be determined as a dominating color of the person or object under
  • the dominating color is to be understood as the most prominent color value of a set of pixels of an image or a video frame which captures the person or object under consideration.
  • first data recording device 1 10 may also send an image or a video frame capturing person 130 to second data recording device 120, challenging second data recording device 120 to respond with the time when person 130 passes.
  • Second data recording device 120 identifies, by analyzing its recorded second data using image recognition techniques, a video frame of the second data capturing person 130 and responds with the time when person 130 passes, based on a time tag stored with the video data.
  • data collecting device 140 may derive an authentication- challenge based on the first data acquired from first data recording device 1 10, and transmit the challenge to second data recording device 120, as is described hereinbefore.
  • data collecting device 140 may authenticate both first data recording device 1 10 and second data recording device 120.
  • data collecting device 140 may acquire both the first data and the second data and perform the process of generating an authentication-challenge based on the first data, generating a response to the challenge based on the second data, and comparing the response with the challenge locally, rather than transmitting the challenge to second data recording device 120.
  • method 300 may optionally comprise selecting 304 data, i.e., a part of the first data and/or a part of the second data, for the purpose of authenticating first data recording device 1 10 and/or second data recording device 120.
  • the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data.
  • the part of the first data and the part of the second data may, e.g., selected 304 based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data.
  • a feature of data is considered to be unique if a subset of the recorded data corresponding to the feature represents an event or object which is uniquely identifiable by analyzing the recorded data.
  • the part of the first data and the part of the second data may be selected 303 based on a proximity of first data recording device 1 10 and second data recording device 120 during recording of the part of the first data and the part of the second data, respectively. That is, only those parts of the first data and the second data are considered for determining 305 an overlap strength which were recorded when the two data recording devices 1 10 and 120 where in proximity of each other, and accordingly share a common past.
  • the overlap strength between the first data and the second data may be determined 305 based on a part of the features of the first data and/or a part of the features of the second data.
  • the part of the features of the first data and/or the part of the features of the second data may be selected based on a uniqueness of the features.
  • the features of the first data and/or a part of the features of the second data may be selected based on a proximity of first data recording device 1 10 and second data recording device 120 during recording of a part of the first data which is associated with the selected features of the first data and a part of the second data which is associated with the selected features of the second data, respectively. In other words, features are selected which were recorded when the two data recording devices were in proximity of each other.
  • Data recording device 600 comprises at least one sensor 601 configured for recording first data during at least one first time interval, a data storage, such as a memory 602, configured for storing the first data 606, a communication interface 605 configured for communicating with another data recording device, and processing means.
  • Communication interface 605 may utilize any wired or wireless technology known in the art, e.g., Ethernet, USB, FireWire, Thunderbolt, Bluetooth, WLAN/WiFi, or a cellular mobile network.
  • the processing means may, e.g., comprise a processor 604 and computer- executable instructions 603, stored in a memory 602, for causing data recording device 600 to perform an embodiment of the method described hereinbefore, when the computer-executable instructions are executed on processor 604.
  • Data recording device 600 may, e.g., be a lifelogging device, a smartphone, a mobile terminal, a digital camera, a webcam, a surveillance camera, an M2M or loT-type of device, a smart watch, a wearable device, or the like.
  • Sensor 601 may, e.g., be a camera, a microphone, a positioning device, a pulse sensor, a heart-rate sensor, a thermometer, or a barometer, and so forth.
  • the processing means is operative to acquire the first data from data storage 603, acquire, via communication interface 605, second data recorded by the second data recording device during at least one second time interval, determine an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and authenticate the second data recording device if the determined overlap strength fulfills an authentication condition.
  • the second data is acquired by receiving the second data from the second data recording device, via communication
  • the second data may be acquired by transferring the second data via an external memory, such as a USB memory stick which is connected to communication interface 605.
  • the second data may optionally be stored in data recording device 600, e.g., in memory 602.
  • FIG. 7 an embodiment of the data collecting device, such as data collecting device 140, is illustrated.
  • Data collecting device 700 comprises a communication interface 705 configured for communicating with a first data recording device and a second data recording device, such as data recording devices 1 10 and 120, and processing means.
  • Communication interface 705 may utilize any wired or wireless technology known in the art, e.g., Ethernet, USB, FireWire,
  • the processing means may, e.g., comprise a processor 704 and computer- executable instructions 703, stored in a memory 702, for causing data collecting device 700 to perform an embodiment of the method described hereinbefore, when the computer-executable instructions are executed on processor 704.
  • Data collecting device 700 may, e.g., be a smartphone, a computer, a tablet, a network storage, a cloud storage, or the like.
  • the processing means is operative to acquire, via communication interface 705, first data recorded by the first data recording device during at least one first time interval, acquire, via communication interface 705, second data recorded by the second data recording device during at least one second time interval, determine an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and authenticate the first data recording device and the second data recording device if the determined overlap strength fulfills an authentication condition.
  • the first data is acquired by receiving the first data from the first data recording device
  • the second data is acquired by receiving the second data from the second data recording device.
  • the first data and/or the second data may be acquired by transferring the first and/or second data via an external memory, such as a USB memory stick which is connected to communication interface 705.
  • the first data and/or the second data may optionally be stored in data collecting device 700, e.g., in memory 702.
  • data recording device 600 and data collecting device 700 are described.
  • the at least one first time interval and the at least one second time interval may be at least partially overlap during at least one common time interval.
  • the at least one first time interval and the at least one second time interval are substantially consecutive.
  • the processing means may be operative to determine the overlap strength by calculating a correlation between the first data and the second data.
  • the processing means may be operative to determine the overlap strength by identifying one or more corresponding features of the features of the first data and the features of the second data, and determine the overlap strength based on a number of the corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof.
  • the processing means may be operative to determine the overlap strength by generating a challenge based on the features of the first data, determine a response to the challenge based on the features of the second data, and verify whether the response matches the challenge, wherein the authentication condition is fulfilled if the response matches the challenge.
  • the overlap strength between the first data and the second data may be determined based on a part of the first data and a part of the second data.
  • the part of the first data and the part of the second data may be selected based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data.
  • the part of the first data and the part of the second data may be selected based on a proximity of the first data recording device and the second data recording device during recording of the part of the first data and the part of the second data, respectively.
  • data recording device 600 may comprise means for determining a position of data recording device 600, such as a GPS module (not shown in Fig. 6).
  • the position of data recording device 600 may be determined utilizing WLAN/WiFi or cellular technology.
  • positioning information is stored with the first and the second data, e.g., as meta data or as additional data.
  • the overlap strength between the first data and the second data may be determined based on a part of the features of the first data and/or a part of the features of the second data.
  • the part of the features of the first data and/or the part of the features of the second data may be selected based on a uniqueness of the features.
  • the overlap strength between the first data and the second data may be determined based on a part of the features of the first data and/or a part of the features of the second data, wherein the part of the features of the first data and/or the part of the features of the second data is/are selected based on a proximity of the first data recording device and the second data recording device during recording of a part of the first data associated with the selected features of the first data and a part of the second data associated with the selected features of the second data, respectively.
  • Data recording device 600 and data collecting device 700 may comprise additional features and/or may additionally be configured to perform in accordance with what is described hereinbefore, in particular with reference to Figs. 1 to 5.
  • Data recording device 800 comprises a sensor unit 801 comprising at least one sensor and being configured for recording first data during at least one first time interval, a data storage unit 802 configured for storing the first data, a communication interface 803 configured for communicating with a second data recording device, an overlap strength unit 804, and an authentication unit 805.
  • Overlap strength unit 804 is configured for acquiring the first data from data storage unit 802, acquiring, via communication interface 803, second data recorded by the second data recording device during at least one second time interval, and determining an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data.
  • Authentication unit 805 is configured for authenticating the second data recording device if the determined overlap strength fulfills an authentication condition.
  • data storage unit 802 may be configured for string the second data.
  • Data recording device 800 may comprise additional units and/or may additionally be configured to perform in accordance with what is described hereinbefore, in particular with reference to Figs. 1 to 6.
  • Data collecting device 900 comprises a communication interface 803 configured for communicating with a first data recording device and a second data recording device, an overlap strength unit 904 and an authentication unit 905.
  • Overlap strength unit 904 is configured for acquiring, via
  • Authentication unit 905 is configured for authenticating the first data recording device and the second data recording device if the determined overlap strength fulfills an authentication condition.
  • data collecting device 900 may further comprise a storage unit 902 which is configured for storing the first data and/or the second data.
  • Data collecting device 900 may comprise additional units and/or may additionally be configured to perform in accordance with what is described hereinbefore, in particular with reference to Figs. 1 to 5, and Fig. 7.
  • embodiments of data recording device 600 and 800, and data collecting device 700 and 900 may be implemented by means of any suitable combination of software, executed on a processor, or hardware, such as analog or digital electronics, Integrated Circuits (ICs), Application Specific ICs (ASICs), and the like.
  • ICs Integrated Circuits
  • ASICs Application Specific ICs
  • the specific functional units, or modules, illustrated in Figs. 6 to 9 only serve as an example, and embodiments of the data recording device or the data collecting device may use any suitable combination of functional units other than those described herein and illustrated in Figs. 6 to 9.

Abstract

A solution for authenticating a first data recording device and/or a second data recording device, such as a first and a second lifelogger, is provided. The solution is based on acquiring (302) first data recorded by the first data recording device during at least one first time interval, acquiring (303) second data recorded by the second data recording device during at least one second time interval, determining (305) an overlap strength between the first data and the second data, and authenticating (307) the first data recording device and/or the second data recording device if the determined overlap strength fulfills an authentication condition (306). The overlap strength is determined based on one or more features of the first and second data, and quantifies the shared knowledge about a common past of the two data recording devices, as captured by the first data and the second data.

Description

AUTHENTICATING DATA RECORDING DEVICES Technical field The invention relates to a method of authenticating a first data recording device and/or a second data recording device, a first data recording device for authenticating a second data recording device, a data collecting device for authenticating a first data recording device and a second data recording device, and corresponding computer programs and computer program products.
Background
Lifelogging is a growing trend, where people use phones, wearables, and dedicated lifeloggers, to document aspects of their daily lives. Lifelogging objectives include memory augmentation, reliving old memories in vivid detail, and optimizing one's daily routines.
Future lifelogging scenarios are expected to include devices not only worn by the user, but recordings will be sourced from multiple devices, such as devices at the user's home or office, or devices in environments which the user frequently visits, e.g., a friend's home. A majority of these devices is expected to provide wireless connectivity for accessing or retrieving data recorded by the devices. This will allow devices to collaborate, often in an ad- hoc fashion, to capture a more multi-dimensional log of the user's daily life.
Due to the personal nature of the recordings, recorded data can be considered as highly sensitive (private) data of the users, leading to privacy and security concerns. Moreover, the use of multiple recording devices also implies multiple points of attack. Security concerns here include the scenario where a device itself has been (physically) substituted by a malicious device, allowing the substituted device to share any future recordings with an unauthorized party, or report data to the user which is different from the actual recording. As such, there is a need to authenticate recording devices in order to safeguard integrity and/or privacy of recorded data before such data can be considered as an authentic recording of the user's past.
With advances in wearable devices, or wearables, and the range of sensors available in smartphones today, lifelogging devices are now capable of recording a diverse range of user data, ranging from health attributes, such as physiological data, to location traces by means of position data, to prevailing environmental conditions, e.g., pollution or temperature, to more traditional audio/video recording of the user's activities. It is expected that multiple devices belonging to, or available to the user, including wearables, phones, tablets, Internet-of-Things (loT) devices in the user's home, office, etc., are involved in the recording process. Accordingly, the number of devices recording data is expected to increase manifold.
While the devices can independently and autonomously record data representing different aspects of the user's life, there is a need to pair lifelogging devices, or associate lifelogging devices with each other, for the purpose of syncing or sharing recorded data, fusing recorded data, or coordinating devices to optimize the recording process. Paring of lifelogging devices, herein referred to as data recording devices, necessitates the need for such devices to authenticate each other, or authenticate themselves to a third device or party, e.g., a network storage or a cloud (storage) provider, a computer used for collecting and possibly analyzing recorded data, or the like.
A manual solution, based on passwords and other known means of authentication requiring user interaction, is deemed impractical by the anticipated increase in number of data recording devices, and the frequency with which they will need to be paired and consequently authenticated. Summary
It is an object of the invention to provide an improved alternative to the above techniques and prior art.
More specifically, it is an object of the invention to provide an improved authentication of data recording devices such as lifeloggers.
These and other objects of the invention are achieved by means of different aspects of the invention, as defined by the independent claims. Embodiments of the invention are characterized by the dependent claims.
According to a first aspect of the invention, a method of authenticating a first data recording device and/or a second data recording device is provided. The method comprises acquiring first data, acquiring second data, determining an overlap strength between the first data and the second data, and authenticating the first data recording device and/or the second data recording device if the determined overlap strength fulfills an authentication condition. The first data is recorded by the first data recording device during at least one first time interval, and the second data is recorded by the second data recording device during at least one second time interval. The overlap strength is determined based on one or more features of the first data and one or more features of the second data.
According to a second aspect of the invention, a computer program is provided. The computer program comprises computer-executable
instructions for causing a device to perform the method according to an embodiment of the first aspect of the invention, when the computer- executable instructions are executed on a processing unit comprised in the device.
According to a third aspect of the invention, a computer program product is provided. The computer program product comprises a computer- readable storage medium which has the computer program according to the second aspect of the invention embodied therein. According to a fourth aspect of the invention, a first data recording device for authenticating a second data recording device is provided. The first data recording device comprises at least one sensor, a data storage, a communication interface, and processing means. The at least one sensor is configured for recording first data during at least one first time interval. The at least one sensor may, e.g., be at least one of a camera, a microphone, a positioning device, a pulse sensor, a heart-rate sensor, a thermometer, or a barometer. The data storage is configured for storing the first data. The communication interface is configured for communicating with the second data recording device. The processing means is operative to acquire first data, acquire second data, determine an overlap strength between the first data and the second data, and authenticate the second data recording device if the determined overlap strength fulfills an authentication condition. The first data is acquired from the data storage. The second data is recorded by the second data recording device during at least one second time interval, and is acquired via the communication interface. The overlap strength is determined based on one or more features of the first data and one or more features of the second data.
According to a fifth aspect of the invention, a data collecting device for authenticating a first data recording device and a second data recording device is provided. The data collecting device comprises a communication interface and processing means. The communication interface is configured for communicating with the first data recording device and the second data recording device. The processing means is operative to acquire first data, acquire second data, determine an overlap strength between the first data and the second data, and authenticate the first data recording device and the second data recording device if the determined overlap strength fulfills an authentication condition. The first data is recorded by the first data recording device during at least one first time interval and acquired via the
communication interface. The second data is recorded by the second data recording device during at least one second time interval and acquired via the communication interface. The overlap strength is determined based on one or more features of the first data and one or more features of the second data.
The invention makes use of an understanding that an improved authentication of data recording devices which are used for recording data related to the life of a user, or the lives of a group of people, e.g., friends or family of the user, can be based on shared knowledge which is captured by respective data recorded by two or more data recording devices. A data recording device, herein also referred to as lifelogger or lifelogging device, may, e.g., be a dedicated device configured for capturing audio and/or video, a smartphone, a mobile terminal, digital cameras, webcams, surveillance cameras, a Machine-to-Machine (M2M) or loT-type of device, a smart watch, a wearable device or wearable, and so forth.
The proposed authentication scheme is based on the idea that data recorded by two different data recording devices, herein referred to as first data and second data, share common features which represent an event or an object which both data recording devices have captured. Based on a comparison of the two data sets, an assessment can be made as to whether the two data recording devices have a common past, i.e., have both been in proximity to each other and to a captured event or object during the time of recording the data.
In this respect, the at least one first time interval and the at least one second time interval preferably overlap during at least one common time interval, during which both data recording devices have captured an event or an object by recording data. Alternatively, the at least one first time interval and the at least one second time interval may be substantially consecutive. This may be the case for lifeloggers which periodically capture still images, e.g., every 30 seconds, but not simultaneously. In such case, the first time interval and the second time interval are the respective shutter times used for capturing the images. If the two time intervals do not overlap, an assessment of whether the two data recording devices share a common past may still be made depending on the time which has lapsed between the non-overlapping first time interval and the second time interval, as well as the nature of the captured event or object. For instance, the second data recording device may capture an image shortly after the first data recording device, such that the same event or object is visible in both captured images.
The authentication scheme proposed herein advantageously relies on preferably unique features of the first data and the second data, i.e., subsets of the recorded data which represent an event or object which is identifiable by analyzing the recorded data, such as image processing or audio processing. Embodiments of the invention may utilize any suitable type of recorded data, e.g., video data, image data, audio data, position data, environmental data, or physiological data.
As an example, one may consider two data recording devices recording video or periodically captured images and being located in proximity to each other, such as two people sitting in a restaurant, each having a camera lifelogger. The two data sets recorded by these lifeloggers contain common features, representative of events or objects which both lifeloggers have captured. This may, e.g., be a third person passing through the restaurant and being captured by both lifeloggers. Alternatively, both lifeloggers may capture a certain object, such as a third person sitting in the restaurant, a distinct piece of furniture, or the like. Based on the recorded data, and preferably time stamps which facilitate synchronization of the two different data sets, it can be established that the two lifeloggers have been in proximity at the time when a certain event or object was captured. That is, they share a common past.
Even though the two data recording devices were in proximity during the common time period, recording the same scene, their recordings, i.e., the first data and the second data, may differ due to the devices' exact locations and camera angles. Thus, in determining the overlap strength, embodiments of the invention may rely on fuzzy rather than exact matches between features of the first data and the second data, with respect to scene characteristics, objects in the scene, appearance or disappearance of an object at the same instant from the scene, and so forth.
Further with reference to the above example, the two lifeloggers may also capture different types of data, e.g., video and audio, respectively. In this case, the common past, i.e., a proximity of the two lifeloggers during a time period in the past, can be established based on an event which is
represented by corresponding features in the respective recorded data. For instance, the video lifelogger may capture video footage of a closing door, whereas the audio lifelogger captures a distinct sound made by the closing door. That is, corresponding features are common features present in sets of recorded data having different types.
As a result of comparing the first data and the second data, an overlap strength is calculated as a measure of a likelihood that the first data recording device and the second data recording device share a common past. In this respect, it will be appreciated that the overlap strength increases with a number of features which are common to both data sets, the presence of a common feature which is particularly unique, an increased correlation between the two data sets, or the length of a time period during which the two data sets are in satisfactory agreement with each other. The
authentication condition may, e.g., be considered fulfilled if the overlap strength is above a threshold value, or if a certain number of common features are identified. The threshold value, based on which authentication commences, may be adjusted such that a desired confidence level indicative of a common past is achieved.
Embodiments of the invention are advantageous in that authentication of lifelogging data recording device is simplified. The proposed solution does not require passwords, PIN codes, or similar, which are tedious to enter, in particular on lifeloggers of small size which are typically provided with only few buttons and lack a display.
According to an embodiment of the invention, the overlap strength is determined by calculating a correlation between the first data and the second data. That is, a mathematical correlation function is used for evaluating the statistical correlation between two random variables, such as the first data and the second data, or variables derived from the first data and the second data, e.g., the time-dependent brightness of a video sequence or the time- dependent audio level of an audio recording. Different types of mathematical correlation functions, such as cross correlation and auto correlation, are known in the art. In this case, the overlap strength increases with increasing value of the correlation function.
According to another embodiment of the invention, the determining an overlap strength comprises identifying one or more corresponding features of the features of the first data and the features of the second data, and determining the overlap strength based on a number of the corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof. As was described in relation to the example above, lifeloggers recording different types of data may be authenticated based on corresponding features, such as video footage capturing a closing door and an audio recording capturing the sound made by a closing door. In this case, the overlap strength increases with the number of identified corresponding features.
According to a further embodiment of the invention, the determining an overlap strength comprises generating a challenge based on the features of the first data, determining a response to the challenge based on the features of the second data, and verifying whether the response matches the challenge. In this case, the authentication condition is fulfilled if the response matches the challenge, i.e., if the compared features are in fact
corresponding features. Optionally, several challenges may be posed, and the authentication condition is fulfilled if at least a certain number of received responses match. With reference to the example of two video lifeloggers described above, the first lifelogger may pose an authentication challenge to the second lifelogger, requesting the second lifelogger to respond with the time when the third person passes. The second lifelogger analyzes its recorded time-stamped video, finds the third person by means of image recognition, and responds with the time when the third person passes. As a further example, the first lifelogger may detect that the third person wears a clothing of a distinct color, and challenges the second lifelogger to respond with the color of the clothing, which can be determined by means of image analysis.
According to an embodiment of the invention, the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data, wherein the part of the first data and the part of the second data are selected based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data. Alternatively, the part of the first data and the part of the second data may be selected based on a proximity of the first data recording device and the second data recording device during recording of the part of the first data and the part of the second data, respectively, i.e., when the data recording devices were close to each other. The proximity of the two data recording devices may, e.g., be determined based on position information received from a Global Positioning System (GPS) module, or the like, which the data recording devices may optionally be provided with. Advantageously, their recorded data may be tagged with positioning information. Whether the two data recording devices are considered to be in proximity of each other depends not only on the absolute distance between the devices, but also on the devices' location and the type of data they record. For instance, the two data recording devices may be considered in proximity of each other is they are in the same room. According to another embodiment of the invention, the overlap strength between the first data and the second data is determined based on a part of the features of the first data and/or a part of the features of the second data, wherein the part of the features of the first data and/or the part of the features of the second data are selected based on a uniqueness of the features. Alternatively, the part of the features of the first data and/or the part of the features of the second data may be selected based on a proximity of the first data recording device and the second data recording device during recording of a part of the first data associated with the selected features of the first data and a part of the second data associated with the selected features of the second data, respectively, i.e., when the data recording devices were close to each other, as was discussed hereinbefore.
According to an embodiment of the invention, the first data recording devices attempts to authenticate the second data recording device. In this case, the first data is acquired from a data storage accessible by the first data recording device, such as an internal or external memory, hard drive, or the like, and the second data is acquired by receiving the second data from the second data recording device, e.g., via a wired or wireless
communication link. Optionally, the second data recording device may subsequently authenticate the first data recording device, resulting in a mutual authentication of the two data recording devices.
According to another embodiment of the invention, a data collecting device, i.e., a third device which is different from the first data recording device and the second data recording device, attempts to authenticate the first data recording device and the second data recording device. This may, e.g., be a smartphone, a computer, a tablet, a cloud storage, or the like, which a user of the lifeloggers utilizes for collecting, analyzing, or fusing, recorded data. In this case, the first data is acquired by receiving the first data from the first data recording device, and the second data is acquired by receiving the second data from the second data recording device, e.g., via wired or wireless communication links.
Even though advantages of the invention have in some cases been described with reference to embodiments of the first aspect of the invention, corresponding reasoning applies to embodiments of other aspects of the invention.
Further objectives of, features of, and advantages with, the invention will become apparent when studying the following detailed disclosure, the drawings and the appended claims. Those skilled in the art realize that different features of the invention can be combined to create embodiments other than those described in the following.
Brief description of the drawings The above, as well as additional objects, features and advantages of the invention, will be better understood through the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the appended drawings, in which:
Fig. 1 illustrates an exemplifying scenario for usage of data recording devices (lifeloggers), in accordance with embodiments of the invention.
Fig. 2 illustrates time periods during which data recording devices in accordance with embodiments of the invention capture an event or an object.
Fig. 3 shows a flow chart for a method of authenticating a first data recording device and/or a second data recording device, in accordance with an embodiment of the invention.
Fig. 4 shows a sequence diagram illustrating authentication of a data recording device, in accordance with an embodiment of the invention.
Fig. 5 shows a sequence diagram illustrating authentication of two data recording devices, in accordance with another embodiment of the invention. Fig. 6 shows a data recording device, in accordance with an embodiment of the invention.
Fig. 7 shows a data collecting device, in accordance with an embodiment of the invention.
Fig. 8 shows a data recording device, in accordance with another embodiment of the invention.
Fig. 9 shows a data collecting device, in accordance with another embodiment of the invention.
All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested.
Detailed description The invention will now be described more fully herein after with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
In Fig. 1 , an exemplifying scenario for usage, in particular
authentication, of data recording devices, or lifeloggers, in accordance with embodiments of the invention is illustrated.
A first data recording device 1 10 travels along a route 1 1 1 into a location 101 , such as a room, a restaurant, or the like. First data recording device 1 10 may, e.g., be carried by a user, and may be any one of a lifelogger capturing images or video, a mobile phone or smartphone capturing images, video, or audio. In Fig. 1 , route 1 1 1 is illustrated as having two legs, a first leg 1 1 1 A along which first data recording device 1 10 travels before entering location 101 , and a second leg 1 1 1 B along which first data recording device 1 10 travels after entering location 101 .
Also illustrated in Fig. 1 is a second data recording device 120 travelling along a route 121 which is, similar to route 1 1 1 , illustrated as having two legs 121 A and 121 B. Second data recording device 120 may be of the same or similar type as first data recording device 1 10, or of a different type. The two data recording devices 1 10 and 120 are in Fig. 1 illustrated as coming to rest in proximity to each other. This may, e.g., be the case if the two users carrying data recording devices 1 10 and 120, respectively, sit down at a table 102.
Depending on the exact timing of the two users' arrival in location 101 , the users, as well as data recording devices 1 10 and 120, share a common past. For instance, if the two users simultaneously arrive at entrance 104 of location 101 , e.g., because they have agreed to meet outside restaurant 101 , the two data recording devices share a common past starting at entrance 101 and extending along their respective routes 1 1 1 B and 121 B as long as they stay in proximity of each other.
The users' common past is reflected by data recorded at each of the data recording devices 1 10 and 120, first data recorded by first data recording device 1 10 and second data recorded by second data recording device 120. The first data and the second data may be of the same type or of different types. Types of first and second data include, but are not limited to, video data, image data, audio data, position data, environmental data, physiological data, and the like.
The first data is recorded during at least a first time interval. It may be recorded continuously, during successive time intervals or periodic time intervals, recording may be switched on and off by the user, and so forth. This is illustrated in Fig. 2, where recording the first data 21 1 is exemplified. For instance, recording first data 21 1 may start at time and stop at time fe- It may continue between tc and tD, when recording stops before it is continued at fE and stops at fF- Thus, first data 21 1 is in Fig. 2 illustrated as being recorded during three successive time intervals. The second data 212 is recorded correspondingly, in Fig. 2 illustrated as being recorded during two time intervals.
As can be seen in Fig. 2, the at least one first time interval and the at least one second time interval at least partially overlap during at least one common time interval, illustrated by means of a diagonal pattern.
Authentication of first 1 10 and/or second 120 data recording device in accordance with embodiments of the invention may be performed based on data recorded during the at least one common time interval. This is the case since data recorded simultaneously at both data recording devices 1 10 and 120 may represent a common past, i.e., representing one or more events or objects captured by both data recording devices 1 10 and 120, if in proximity to each other when capturing the event(s) or object(s).
Alternatively, authentication of first 1 10 and/or second 120 data recording device in accordance with embodiments of the invention may be performed based on data recorded during respective time intervals which are substantially consecutive, as is illustrated in the lower part of Fig. 2. This may, e.g., be the case if data recording devices 1 10 and 120 record data discontinuously, e.g., periodically capturing still images. Depending on the timing, e.g., the periodicity Δίι with which data is captured, the duration At2 during which data is recorded, such as the shutter time when capturing still images, and the offset Δί3 in time between two images consecutively captured by the two data recording devices 1 10 and 120, the first data and the second data may still be used for the purpose of authentication, if it can be determined that the two data recording devices share a common past. This is the case if the first data and the second data represent an event or object which has been captured by both data recording devices. For instance, with reference to Fig. 1 , both data recording devices 1 10 and 120 may capture an event, such as a person 130 passing along route 131 , if second data recording device 120 captures an image shortly after first data recording device 1 10, such that person 130 is visible in both captured images. Alternatively, both data recording devices 1 10 and 120 may capture images of an object, such as a piece of furniture or the like. One of the data recording devices 1 10 or 120 may also capture the other data recording device in an image or a video.
In the following, authenticating the first data recording device and/or the second data recording device in accordance with embodiments of the invention is described. Reference is made to Fig. 3, which shows a flow chart for an embodiment 300 of a method of the invention.
Method 300 is initiated by a request 301 for authenticating first data recording device 1 10, second data recording device 120, or both.
Authentication may be initiated by a user, in response to interconnecting or pairing the two data recording devices 1 10 and 120 with each other or with a third device for collecting data, as is described further below, in response to detecting a proximity of the first data recording device 1 10 and the second data recording device 120 to each other or to the third device, or the like.
Method 300 comprises acquiring 302 the first data recorded by first data recording device 1 10 during at least one first time interval, acquiring 303 the second data recorded by second data recording device 120 during at least one second time interval, determining 305 an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and authenticating 307 first data recording device 1 10 and/or second data recording device 120 if the determined overlap strength fulfills an authentication condition 306. If authentication condition 306 is not fulfilled, authentication is rejected 308.
Determining 305 the overlap strength aims at quantifying the shared knowledge of the common past of the two data recording devices, which shared knowledge is represented by the recorded data. Based on that quantification, authentication 307 is performed if the authentication condition 306 is fulfilled. It will be appreciated that the level of security may be adjusted by selecting the authentication condition accordingly, such that a desired level of confidence indicative of a common past is achieved. That is, if a higher level of security is required, a more stringent authentication condition is selected, whereas a more lenient authentication condition is selected if a lower level of security is required. The relation between quantifying the overlap and the authentication condition is exemplified in further detail below.
An embodiment of method 300 may be performed by first data recording device 1 10, as is further illustrated in Fig. 4. In this case, the first data is acquired 302 from a data storage accessible by first data recording device 1 10, e.g., an internal or external storage such as a computer memory, hard drive, memory stick, Random Access Memory (RAM), or Flash memory, where data is stored after being captured by a suitable sensor, such as a camera, a microphone, a positioning module, or the like. The second data is acquired 303 by receiving the second data from second data recording device 120, e.g., by means of a wired or a wireless communication link such as Universal Serial Bus (USB), FireWire, Thunderbolt, Ethernet, Wireless Local Area Network (WLAN)/WiFi, Bluetooth, a cellular network such as a Global System for Mobile Communications (GSM) network, an Universal Mobile Telecommunications System (UMTS) network, or a Long Term
Evolution (LTE) network, an ad-hoc network, the Internet, or any combination thereof. Alternatively, the second data may be made available to first data recording device 1 10 by transferring 303 the data using a memory stick or the like.
This may be a typical scenario if one of the data recording devices, e.g., first data recording device 1 10 such as a smartphone or tablet, is used for collecting, analyzing, or fusing, data recorded by one or more separate data recording devices, such as second data recording device 120. As a further example, first data recording device 1 10 may also authenticate second data recording device 120 for the purpose of coordinating capturing or recording of data. For instance, one may envisage embodiments of the invention where a master data recording device is used, such as a
smartphone of the user, for controlling capturing of data by other data recording devices. Such a master data recording device may, e.g., control other data recording devices to start and stop recording of data depending on the location of the user in relation to the other data recording devices. This is advantageous for data recording devices which are not carried by the user but which are deployed in the user's home, office, or the like.
As is illustrated in Fig. 4, first data recording device 1 10 and second data recording device 120 may prepare or arbitrate 401 authentication.
Arbitrating 401 authentication may, e.g., include transmitting a request for authentication or pairing, exchanging information pertaining to the second data recorded by second data recording device 120, e.g., time periods during which the second data has been recorded, or position information indicating at which location or locations the second data has been recorded, and so forth. Based on such information, parts of the first data and parts of the second data which are used for quantifying the shared knowledge of the two data recording devices about their common past may be selected 304, as is described further below. It will be appreciated that such selection 304 may be performed prior to acquiring the first data and/or the second data, i.e., as part of the arbitration 401 indicated in Fig. 4, or after, as is indicated in Fig. 3. Accordingly, first data recording device 1 10 may acquire only parts of the first data which were recorded during the at least one common time interval, and/or when first data recording device 1 10 and second data recording device 120 were in proximity to each other. As a consequence, the amount of data which is transferred between devices is reduced, thereby preserving battery power.
Further with reference to Fig. 4, first data recording device 1 10 may perform further processing 409 after successful authentication, such as analyzing, storing, or fusing, the first and/or the second data. Furthermore, first data recording device 1 10 may also engage in further
communications 410 with second data recording device 120 after successful authentication, e.g., for the purpose of coordinating recording of data, as was described hereinbefore.
In an alternative embodiment, method 300 may be performed by a data collecting device (not shown in Fig. 1 ) which is separate from first data recording device 1 10 and second data recording device 120, as is illustrated in Fig. 5. The data collecting device 140 may, e.g., be a smartphone, a tablet, a computer, a cloud storage, or the like, which is used for collecting, analyzing, or fusing, data recorded by at least two data recording devices, such as first data recording device 1 10 and second data recording
device 120. In this case, the first data is acquired 302 by receiving the first data from first data recording device 1 10, and the second data is
acquired 303 by receiving the second data from second data recording device 120. The first and the second data may, e.g., be received by means of a wired or a wireless communication link, similar to what is described above. Alternatively, the first data and the second data may be made available to data collecting device 140 by transferring the data using a memory stick or the like.
As a further example, data collecting device 140 may also
authenticate first data recording device 1 10 and second data recording device 120 for the purpose of coordinating capturing or recording of data. For instance, data collecting device 140 may have the role of a master data recording device, such as a smartphone of the user, for controlling capturing of data by other data recording devices. Such a master data recording device may, e.g., control other data recording devices to start and stop recording of data depending on the location of the user in relation to the other data recording devices. This is advantageous for data recording devices which are not carried by the user but which are deployed in the user's home, office, or the like.
As is illustrated in Fig. 5, first data recording device 1 10 and second data recording device 120 may prepare or arbitrate 501 authentication.
Arbitration 501 authentication may, e.g., include transmitting a request for authentication or pairing, exchanging information pertaining to the first data recorded by first data recording device 1 10 and second data recorded by second data recording device 120, e.g., respective time periods during which the first and the second data have been recorded, or position information indicating at which respective location or locations the first data and the second data have been recorded, and so forth. Based on such information, parts of the first data and parts of the second data which are used for quantifying the shared knowledge of the two data recording device may be selected 304, as is described further below. It will be appreciated that such selection 304 may be performed prior to acquiring the first data and/or the second data, i.e., as part of the arbitration 501 indicated in Fig. 5, or after, as is indicated in Fig. 3. Accordingly, data collecting device 140 may acquire only parts of the first data and/or the second data which were recorded during the at least one common time interval, and/or when first data recording device 1 10 and second data recording device 120 were in proximity to each other. As a consequence, the amount of data which is transferred between devices is reduced, thereby preserving battery power.
Further with reference to Fig. 5, data collecting device 140 may perform further processing 509 after successful authentication, such as analyzing, storing, or fusing, the first and/or the second data. Furthermore, data collecting device 140 may also engage in further communications 510 with first data recording device 1 10 and/or second data recording device 120 after successful authentication, e.g., for the purpose of coordinating recording of data, as was described hereinbefore. The overlap strength between the first data and the second data may be determined 305 based on any method suitable for assessing whether data recording devices 1 10 and 120 share a common past. This is achieved by quantifying the similarity of the first data and the second data, or the amount of common features, i.e., features which are present in the first data and the second data. For instance, the overlap strength may be determined 305 by calculating a correlation between the first data and the second data. That is, using a correlation function the statistical correlation between the first data and the second data, which are random variables, is evaluated. Different types of correlation functions are known in the art. For example, an autocorrelation function may be used for evaluating the statistical correlation, i.e., the overlap, between random variables representing the same quantity measured at two different locations. This is the case if the first data and the second data are of the same type, e.g., video footage captured by the two data recording devices, such as video lifeloggers. If the common past of two data recording devices recording different types of data is to be assessed, e.g., a video lifelogger capturing video footage and a smartphone recording audio, a cross correlation function may be used for evaluating the statistical correlation, and hence the overlap strength, between the first data and the second data.
Correlations functions may be used both on scalar-valued data, such as temperature, brightness of a video sequence, or level of an audio recording, and non-scalar data, such as video. It will also be appreciated that the statistical correlation may be evaluated between variables derived from the first data or the second data, respectively. For instance, one may envisage embodiments of the invention performing authentication based on determining the overlap strength between the time-dependent brightness of a video sequence, the first data, recorded by the first data recording device 1 10 and the time-dependent brightness of a video sequence, the second data, recorded by the second data recording device 120. Whilst information may be lost by deriving variables from the recorded first and second data, authenticating the first and/or the second data recording device based on the brightness of recorded video may be sufficient if the two data recording devices have recorded data during a sufficiently long common time interval, and/or if the respective data sets comprise sufficient unique or distinct features. This may, e.g., be the case if the two data recording devices 1 10 and 120 have travelled together for an extended period of time in an environment of varying illumination, resulting in sufficient variations in the derived brightness.
The correlation function used for determining 305 the overlap strength may be normalized such that a perfect correlation, i.e., the first data and the second data being identical, yields a value of unity, or any other desired maximum value. In such case, the authentication condition 306 may be considered fulfilled if the overlap strength is above a threshold value, e.g., 80% of the maximum value or a value of 0.8 in case unity is used for perfect correlation. It will be appreciated that any suitable value may be used, where an increased threshold value is associated with an increased level security level of the authentication.
An alternative method for determining 305 the overlap strength between the first data and the second data comprises identifying one or more corresponding features of the features of the first data and the features of the second data, and determining the overlap strength based on a number of the corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof. As was described hereinbefore, data recording devices 1 10 and 120 may record different types of data, such as video and audio, respectively. In such cases, the overlap strength may be determined 305 based on corresponding features, i.e., common features of the first data and of the second data which represent events or objects captured by both data recording devices, albeit with different types of data. For instance, first data recording device 1 10 may be a video lifelogger capturing the closing of door 103 after person 130 has entered location 101 , whilst second data recording device 120 may be a smartphone recording audio and capturing the sound of closing door 103. In this case, the overlap strength increases with the number of identified corresponding features in relation to the number of identified features in one of the first data or the second data, or the total number of features in the first data and the second data.
As yet another alternative for determining 305 the overlap strength between the first data and the second data, a solution based on a challenge- response scheme is described in the following. Accordingly, determining 305 the overlap strength between the first data and the second data may comprise generating an authentication challenge based on one or more features of the first data, determining a response to the challenge based on the features of the second data, and verifying whether the response matches the challenge, i.e., whether the respective features of the first data and the second data are in fact common features. In this case, the authentication condition is fulfilled if the response matches the challenge. Optionally, if several challenges are posed, the authentication condition may be
considered fulfilled if the number of successful challenges exceeds a threshold in relation to the number of challenges posed.
Such a challenge-response scheme may be implemented in a number of ways. For instance, as is illustrated in Fig. 4, if first data recording device 1 10 attempts to authenticate second data recording device 120, first data recording device 1 10 may analyze its recorded first data to identify a suitable feature, i.e., a captured person, object, or event, for posing a challenge. As an example, first data recording device 1 10 may determine, by image processing its recorded video data (the first data), that third
person 130 passes table 102 at a certain time, according to time stamps of the first data. First data recording device may then determine a color of the clothing of third person 130, by analyzing the pixels of an image capturing third person 130 when passing. Based on this analysis, first data recording device 1 10 may then send a challenge to second data recording device 120, such as a request to determine the color of a person or an object which passes at a certain point in time, or within a short time interval around that point in time. In response to receiving the challenge, second data recording device 120 analyzes its recorded video data (the second data) at the specified point in time and determines a color of a person or object which passes. The determined color is sent to first data recording device 1 10, using any suitable means of color coding known in the art, such as HTML color codes, RGB color codes, or the like. First data recording device 1 10 then compares the color received in the response from second data recording device 120 and authenticates second data recording device 120 if the color determined by second data recording device 120 matches the color determined by first data recording device 1 10. In the described example, the color of a person, clothing of a person, object, or the like, may, e.g., be determined as a dominating color of the person or object under
consideration. In this respect, the dominating color is to be understood as the most prominent color value of a set of pixels of an image or a video frame which captures the person or object under consideration.
As a further example of implementing a challenge-response scheme, first data recording device 1 10 may also send an image or a video frame capturing person 130 to second data recording device 120, challenging second data recording device 120 to respond with the time when person 130 passes. Second data recording device 120 then identifies, by analyzing its recorded second data using image recognition techniques, a video frame of the second data capturing person 130 and responds with the time when person 130 passes, based on a time tag stored with the video data.
If authentication is performed at data collecting device 140, as is illustrated in Fig. 5, data collecting device 140 may derive an authentication- challenge based on the first data acquired from first data recording device 1 10, and transmit the challenge to second data recording device 120, as is described hereinbefore. In response to receiving a matching response, data collecting device 140 may authenticate both first data recording device 1 10 and second data recording device 120. As an alternative, data collecting device 140 may acquire both the first data and the second data and perform the process of generating an authentication-challenge based on the first data, generating a response to the challenge based on the second data, and comparing the response with the challenge locally, rather than transmitting the challenge to second data recording device 120.
Further with reference to Fig. 3, method 300 may optionally comprise selecting 304 data, i.e., a part of the first data and/or a part of the second data, for the purpose of authenticating first data recording device 1 10 and/or second data recording device 120. In this case, the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data. The part of the first data and the part of the second data may, e.g., selected 304 based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data. In the present context, a feature of data is considered to be unique if a subset of the recorded data corresponding to the feature represents an event or object which is uniquely identifiable by analyzing the recorded data. As an
alternative, the part of the first data and the part of the second data may be selected 303 based on a proximity of first data recording device 1 10 and second data recording device 120 during recording of the part of the first data and the part of the second data, respectively. That is, only those parts of the first data and the second data are considered for determining 305 an overlap strength which were recorded when the two data recording devices 1 10 and 120 where in proximity of each other, and accordingly share a common past.
In an alternative embodiment, the overlap strength between the first data and the second data may be determined 305 based on a part of the features of the first data and/or a part of the features of the second data. The part of the features of the first data and/or the part of the features of the second data may be selected based on a uniqueness of the features. As an alternative, the features of the first data and/or a part of the features of the second data may be selected based on a proximity of first data recording device 1 10 and second data recording device 120 during recording of a part of the first data which is associated with the selected features of the first data and a part of the second data which is associated with the selected features of the second data, respectively. In other words, features are selected which were recorded when the two data recording devices were in proximity of each other.
In the following, embodiments of the data recording device, such as data recording devices 1 10 and 120, are described with reference to Fig. 6.
Data recording device 600 comprises at least one sensor 601 configured for recording first data during at least one first time interval, a data storage, such as a memory 602, configured for storing the first data 606, a communication interface 605 configured for communicating with another data recording device, and processing means. Communication interface 605 may utilize any wired or wireless technology known in the art, e.g., Ethernet, USB, FireWire, Thunderbolt, Bluetooth, WLAN/WiFi, or a cellular mobile network. The processing means may, e.g., comprise a processor 604 and computer- executable instructions 603, stored in a memory 602, for causing data recording device 600 to perform an embodiment of the method described hereinbefore, when the computer-executable instructions are executed on processor 604. Data recording device 600 may, e.g., be a lifelogging device, a smartphone, a mobile terminal, a digital camera, a webcam, a surveillance camera, an M2M or loT-type of device, a smart watch, a wearable device, or the like. Sensor 601 may, e.g., be a camera, a microphone, a positioning device, a pulse sensor, a heart-rate sensor, a thermometer, or a barometer, and so forth. More specifically, the processing means is operative to acquire the first data from data storage 603, acquire, via communication interface 605, second data recorded by the second data recording device during at least one second time interval, determine an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and authenticate the second data recording device if the determined overlap strength fulfills an authentication condition. Preferably, the second data is acquired by receiving the second data from the second data recording device, via communication
interface 605. Alternatively, the second data may be acquired by transferring the second data via an external memory, such as a USB memory stick which is connected to communication interface 605. The second data may optionally be stored in data recording device 600, e.g., in memory 602.
In Fig. 7, an embodiment of the data collecting device, such as data collecting device 140, is illustrated.
Data collecting device 700 comprises a communication interface 705 configured for communicating with a first data recording device and a second data recording device, such as data recording devices 1 10 and 120, and processing means. Communication interface 705 may utilize any wired or wireless technology known in the art, e.g., Ethernet, USB, FireWire,
Thunderbolt, Bluetooth, WLAN/WiFi, or a cellular mobile network. The processing means may, e.g., comprise a processor 704 and computer- executable instructions 703, stored in a memory 702, for causing data collecting device 700 to perform an embodiment of the method described hereinbefore, when the computer-executable instructions are executed on processor 704. Data collecting device 700 may, e.g., be a smartphone, a computer, a tablet, a network storage, a cloud storage, or the like.
More specifically, the processing means is operative to acquire, via communication interface 705, first data recorded by the first data recording device during at least one first time interval, acquire, via communication interface 705, second data recorded by the second data recording device during at least one second time interval, determine an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and authenticate the first data recording device and the second data recording device if the determined overlap strength fulfills an authentication condition. Preferably, the first data is acquired by receiving the first data from the first data recording device, and the second data is acquired by receiving the second data from the second data recording device. Alternatively, the first data and/or the second data may be acquired by transferring the first and/or second data via an external memory, such as a USB memory stick which is connected to communication interface 705. The first data and/or the second data may optionally be stored in data collecting device 700, e.g., in memory 702.
In the following, further embodiments and optional features of data recording device 600 and data collecting device 700 are described.
The at least one first time interval and the at least one second time interval may be at least partially overlap during at least one common time interval. Alternatively, the at least one first time interval and the at least one second time interval are substantially consecutive.
In an embodiment, the processing means may be operative to determine the overlap strength by calculating a correlation between the first data and the second data.
In another embodiment, the processing means may be operative to determine the overlap strength by identifying one or more corresponding features of the features of the first data and the features of the second data, and determine the overlap strength based on a number of the corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof. As yet a further alternative, the processing means may be operative to determine the overlap strength by generating a challenge based on the features of the first data, determine a response to the challenge based on the features of the second data, and verify whether the response matches the challenge, wherein the authentication condition is fulfilled if the response matches the challenge.
Optionally, the overlap strength between the first data and the second data may be determined based on a part of the first data and a part of the second data. For instance, the part of the first data and the part of the second data may be selected based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data. Alternatively, the part of the first data and the part of the second data may be selected based on a proximity of the first data recording device and the second data recording device during recording of the part of the first data and the part of the second data, respectively. For this purpose, data recording device 600 may comprise means for determining a position of data recording device 600, such as a GPS module (not shown in Fig. 6). Alternatively, the position of data recording device 600 may be determined utilizing WLAN/WiFi or cellular technology. Preferably, positioning information is stored with the first and the second data, e.g., as meta data or as additional data.
As an alternative, the overlap strength between the first data and the second data may be determined based on a part of the features of the first data and/or a part of the features of the second data. For instance, the part of the features of the first data and/or the part of the features of the second data may be selected based on a uniqueness of the features. Alternatively, the overlap strength between the first data and the second data may be determined based on a part of the features of the first data and/or a part of the features of the second data, wherein the part of the features of the first data and/or the part of the features of the second data is/are selected based on a proximity of the first data recording device and the second data recording device during recording of a part of the first data associated with the selected features of the first data and a part of the second data associated with the selected features of the second data, respectively.
Data recording device 600 and data collecting device 700 may comprise additional features and/or may additionally be configured to perform in accordance with what is described hereinbefore, in particular with reference to Figs. 1 to 5.
In Fig. 8, an alternative embodiment of the data recording device is illustrated.
Data recording device 800 comprises a sensor unit 801 comprising at least one sensor and being configured for recording first data during at least one first time interval, a data storage unit 802 configured for storing the first data, a communication interface 803 configured for communicating with a second data recording device, an overlap strength unit 804, and an authentication unit 805. Overlap strength unit 804 is configured for acquiring the first data from data storage unit 802, acquiring, via communication interface 803, second data recorded by the second data recording device during at least one second time interval, and determining an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data. Authentication unit 805 is configured for authenticating the second data recording device if the determined overlap strength fulfills an authentication condition.
Optionally, data storage unit 802 may be configured for string the second data.
Data recording device 800 may comprise additional units and/or may additionally be configured to perform in accordance with what is described hereinbefore, in particular with reference to Figs. 1 to 6.
In Fig. 9, an alternative embodiment of the data collecting device is illustrated. Data collecting device 900 comprises a communication interface 803 configured for communicating with a first data recording device and a second data recording device, an overlap strength unit 904 and an authentication unit 905. Overlap strength unit 904 is configured for acquiring, via
communication interface 903, first data recorded by the first data recording device during at least one first time interval, acquiring, via communication interface 903, second data recorded by the second data recording device during at least one second time interval, and determining an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data. Authentication unit 905 is configured for authenticating the first data recording device and the second data recording device if the determined overlap strength fulfills an authentication condition. Optionally, data collecting device 900 may further comprise a storage unit 902 which is configured for storing the first data and/or the second data.
Data collecting device 900 may comprise additional units and/or may additionally be configured to perform in accordance with what is described hereinbefore, in particular with reference to Figs. 1 to 5, and Fig. 7.
With reference to Figs. 6 to 9, embodiments of data recording device 600 and 800, and data collecting device 700 and 900, may be implemented by means of any suitable combination of software, executed on a processor, or hardware, such as analog or digital electronics, Integrated Circuits (ICs), Application Specific ICs (ASICs), and the like. The specific functional units, or modules, illustrated in Figs. 6 to 9 only serve as an example, and embodiments of the data recording device or the data collecting device may use any suitable combination of functional units other than those described herein and illustrated in Figs. 6 to 9.
The person skilled in the art realizes that the invention by no means is limited to the embodiments described above. On the contrary, many modifications and variations are possible within the scope of this disclosure. In particular, whereas embodiments of the invention have been elucidated by examples involving data recording devices carried by different users, it will be appreciated that embodiments of the invention may also be used for authenticating two or more data recording devices carried by a single person.

Claims

1 . A method (300) of authenticating a first data recording device (1 10) and/or a second data recording device (120), the method comprising:
acquiring (302) first data recorded by the first data recording device during at least one first time interval,
acquiring (303) second data recorded by the second data recording device during at least one second time interval,
determining (305) an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and
authenticating (307) the first data recording device and/or the second data recording device if the determined overlap strength fulfills an
authentication condition (306).
2. The method according to claim 1 , wherein the at least one first time interval and the at least one second time interval at least partially overlap during at least one common time interval.
3. The method according to claim 1 , wherein the at least one first time interval and the at least one second time interval are substantially
consecutive.
4. The method according to any one of claims 1 to 3, wherein the overlap strength is determined by calculating a correlation between the first data and the second data.
5. The method according to any one of claims 1 to 3, wherein the determining an overlap strength comprises: identifying one or more corresponding features of the features of the first data and the features of the second data, and
determining the overlap strength based on a number of the
corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof.
6. The method according to any one of claims 1 to 3, wherein the determining an overlap strength comprises:
generating a challenge based on the features of the first data, determining a response to the challenge based on the features of the second data, and
verifying whether the response matches the challenge,
wherein the authentication condition is fulfilled if the response matches the challenge.
7. The method according to any one of claims 1 to 6, wherein the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data, the part of the first data and the part of the second data being selected (304; 401 ; 501 ) based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data.
8. The method according to any one of claims 1 to 6, wherein the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data, the part of the first data and the part of the second data being selected (304; 401 ; 501 ) based on a proximity of the first data recording device and the second data recording device during recording of the part of the first data and the part of the second data, respectively.
9. The method according to any one of claims 1 to 6, wherein the overlap strength between the first data and the second data is determined based on a part of the features of the first data and/or a part of the features of the second data, the part of the features of the first data and/or the part of the features of the second data being selected (304; 401 ; 501 ) based on a uniqueness of the features.
10. The method according to any one of claims 1 to 6, wherein the overlap strength between the first data and the second data is determined based on a part of the features of the first data and/or a part of the features of the second data, the part of the features of the first data and/or the part of the features of the second data being selected based (304; 401 ; 501 ) on a proximity of the first data recording device and the second data recording device during recording of a part of the first data associated with the selected features of the first data and a part of the second data associated with the selected features of the second data, respectively.
1 1 . The method according to any one of claims 1 to 10, wherein each of the first data and the second data is one of video data, image data, audio data, position data, environmental data, or physiological data.
12. The method according to any one of claims 1 to 1 1 , wherein the authentication condition is fulfilled if the overlap strength is above a threshold value.
13. The method according to any one of claims 1 to 12, wherein a feature of data is a subset of the data representing an event or object captured by the recorded data.
14. The method according to any one of claims 1 to 13, the method being performed by the first data recording device (1 10), wherein the first data is acquired from a data storage accessible by the first data recording device, and the second data is acquired by receiving the second data from the second data recording device (120).
15. The method according to any one of claims 1 to 13, the method being performed by a data collecting device (140), wherein the first data is acquired by receiving the first data from the first data recording device (1 10), and the second data is acquired by receiving the second data from the second data recording device (120).
16. A computer program (603; 703) comprising computer-executable instructions for causing a device to perform the method according to any one of claims 1 to 15, when the computer-executable instructions are executed on a processing unit (604; 704) comprised in the device.
17. A computer program product comprising a computer-readable storage medium (602; 702), the computer-readable storage medium having the computer program (603; 703) according to claim 16 embodied therein.
18. A first data recording device (1 10; 600; 800) for authenticating a second data recording device (120), the first data recording device comprising:
at least one sensor (601 ; 801 ) configured for recording first data during at least one first time interval,
a data storage (602; 802) configured for storing the first data, a communication interface (605; 803) configured for communicating with the second data recording device, and
processing means (604, 603; 804, 805) being operative to: acquire the first data from the data storage (602; 802)), acquire, via the communication interface (605; 803), second data recorded by the second data recording device (120) during at least one second time interval,
determine an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and
authenticate the second data recording device (120) if the determined overlap strength fulfills an authentication condition.
19. The first data recording device according to claim 18, wherein the at least one sensor (601 ; 801 ) is at least one of a camera, a microphone, a positioning device, a pulse sensor, a heart-rate sensor, a thermometer, or a barometer.
20. The first data recording device according to claims 18 or 19, wherein the second data is acquired by receiving the second data from the second data recording device (120).
21 . A data collecting device (140; 700; 900 for authenticating a first data recording device (1 10) and a second data recording device (120), the data collecting device comprising:
a communication interface (706; 903) configured for communicating with the first data recording device (1 10) and the second data recording device (120), and
processing means (704, 703; 904, 905) being operative to:
acquire, via the communication interface (706; 903), first data recorded by the first data recording device (1 10) during at least one first time interval, acquire, via the communication interface (706; 903), second data recorded by the second data recording device (120) during at least one second time interval,
determine an overlap strength between the first data and the second data based on one or more features of the first data and one or more features of the second data, and
authenticate the first data recording device (1 10) and the second data recording device (120) if the determined overlap strength fulfills an authentication condition.
22. The data collecting device according to claim 21 , wherein the first data is acquired by receiving the first data from the first data recording device (1 10), and the second data is acquired by receiving the second data from the second data recording device (120).
23. The device according any one of claims 18 to 22, wherein the at least one first time interval and the at least one second time interval at least partially overlap during at least one common time interval.
24. The device according any one of claims 18 to 22, wherein the at least one first time interval and the at least one second time interval are substantially consecutive.
25. The device according any one of claims 18 to 24, wherein the processing means is operative to determine the overlap strength by calculating a correlation between the first data and the second data.
26. The device according to any one of claims 18 to 24, wherein the processing means is operative to determine the overlap strength by: identifying one or more corresponding features of the features of the first data and the features of the second data, and
determining the overlap strength based on a number of the
corresponding features in relation to a number of the features of the first data, a number of the features of the second data, or a combination thereof.
27. The device according to any one of claims 18 to 24, wherein the processing means is operative to determine the overlap strength by:
generating a challenge based on the features of the first data, determining a response to the challenge based on the features of the second data, and
verifying whether the response matches the challenge,
wherein the authentication condition is fulfilled if the response matches the challenge.
28. The device according to any one of claims 18 to 27, wherein the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data, the part of the first data and the part of the second data being selected based on a uniqueness of the part of the first data and/or a uniqueness of the part of the second data.
29. The device according to any one of claims 18 to 27, wherein the overlap strength between the first data and the second data is determined based on a part of the first data and a part of the second data, the part of the first data and the part of the second data being selected based on a proximity of the first data recording device and the second data recording device during recording of the part of the first data and the part of the second data, respectively.
30. The device according to any one of claims 18 to 27, wherein the overlap strength between the first data and the second data is determined based on a part of the features of the first data and/or a part of the features of the second data, the part of the features of the first data and/or the part of the features of the second data being selected based on a uniqueness of the features.
31 . The device according to any one of claims 18 to 27, wherein the overlap strength between the first data and the second data is determined based on a part of the features of the first data and/or a part of the features of the second data, the part of the features of the first data and/or the part of the features of the second data being selected based on a proximity of the first data recording device and the second data recording device during recording of a part of the first data associated with the selected features of the first data and a part of the second data associated with the selected features of the second data, respectively.
32. The device according to any one of claims 18 to 31 , wherein each of the first data and the second data is of one of video data, image data, audio data, position data, environmental data, or physiological data.
33. The device according to any one of claims 18 to 32, wherein the authentication condition is fulfilled if the overlap strength is above a threshold value.
34. The device according to any one of claims 18 to 33, wherein a feature of data is a subset of the data representing an event or object captured by the recorded data.
PCT/SE2015/050619 2015-05-29 2015-05-29 Authenticating data recording devices WO2016195545A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SE2015/050619 WO2016195545A1 (en) 2015-05-29 2015-05-29 Authenticating data recording devices

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SE2015/050619 WO2016195545A1 (en) 2015-05-29 2015-05-29 Authenticating data recording devices

Publications (1)

Publication Number Publication Date
WO2016195545A1 true WO2016195545A1 (en) 2016-12-08

Family

ID=53488419

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2015/050619 WO2016195545A1 (en) 2015-05-29 2015-05-29 Authenticating data recording devices

Country Status (1)

Country Link
WO (1) WO2016195545A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3477982A1 (en) * 2017-10-30 2019-05-01 Hewlett-Packard Development Company, L.P. Location based authentication
EP3554052A1 (en) * 2018-04-10 2019-10-16 Rolls-Royce plc Machine sensor network management
FR3116982A1 (en) * 2020-12-01 2022-06-03 Orange Event correlation method and system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130127591A1 (en) * 2011-11-20 2013-05-23 International Business Machines Corporation Secure facilities access
EP2683147A1 (en) * 2012-07-03 2014-01-08 Samsung Electronics Co., Ltd Method and apparatus for pairing user devices using voice
EP2731371A2 (en) * 2012-11-07 2014-05-14 Samsung Electronics Co., Ltd Method and device for user terminal pairing
US20140324591A1 (en) * 2013-04-29 2014-10-30 Qualcomm Incorporated Selectively authenticating a group of devices as being in a shared environment based on locally captured ambient sound

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130127591A1 (en) * 2011-11-20 2013-05-23 International Business Machines Corporation Secure facilities access
EP2683147A1 (en) * 2012-07-03 2014-01-08 Samsung Electronics Co., Ltd Method and apparatus for pairing user devices using voice
EP2731371A2 (en) * 2012-11-07 2014-05-14 Samsung Electronics Co., Ltd Method and device for user terminal pairing
US20140324591A1 (en) * 2013-04-29 2014-10-30 Qualcomm Incorporated Selectively authenticating a group of devices as being in a shared environment based on locally captured ambient sound

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
REZA RAWASSIZADEH ET AL: "Securing Shareable Life-logs", SOCIAL COMPUTING (SOCIALCOM), 2010 IEEE SECOND INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 20 August 2010 (2010-08-20), pages 1105 - 1110, XP031766974, ISBN: 978-1-4244-8439-3 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3477982A1 (en) * 2017-10-30 2019-05-01 Hewlett-Packard Development Company, L.P. Location based authentication
EP3554052A1 (en) * 2018-04-10 2019-10-16 Rolls-Royce plc Machine sensor network management
FR3116982A1 (en) * 2020-12-01 2022-06-03 Orange Event correlation method and system
WO2022117946A1 (en) * 2020-12-01 2022-06-09 Orange Method and system for correlating events

Similar Documents

Publication Publication Date Title
US10506406B2 (en) Building network hub with occupancy awareness
US11917037B2 (en) Methods, devices, and systems for determining a subset for autonomous sharing of digital media
US10506054B2 (en) Building occupancy sensor network
AU2019204810A1 (en) Digital fingerprint tracking
US11700071B2 (en) Method, device, system, and storage medium for live broadcast detection and data processing
AU2019295856B2 (en) Object tracking using disparate monitoring systems
CN104537746A (en) Intelligent electronic door control method, system and equipment
US20170262706A1 (en) Smart tracking video recorder
US9792488B2 (en) Adjacent person specifying apparatus, adjacent person specifying method, adjacent person specifying program, and adjacent person specifying system
CN204990444U (en) Intelligent security controlgear
CN104933791A (en) Intelligent security control method and equipment
CN108429745A (en) The authentication method and Verification System of login, webpage login method and system
CN108363939B (en) Characteristic image acquisition method and device and user authentication method
US9594403B2 (en) Embedding biometric data from a wearable computing device in metadata of a recorded image
WO2016195545A1 (en) Authenticating data recording devices
CN106464812A (en) Lifelog camera and method of controlling same according to transitions in activity
JP2015233204A (en) Image recording device and image recording method
CN208013967U (en) A kind of examination hall monitor camera and system based on face recognition technology
CN111126104A (en) Method and device for marking user identity attribute of data
WO2016035632A1 (en) Data processing device, data processing system, data processing method, and program
JP6710817B2 (en) Smile infection determination system, smile infection determination method and program
KR101732379B1 (en) Method for user authentication based on face recognition
JP2020113109A (en) Image capturing system
CN111127704A (en) Data association method and device
Singh et al. Detecting Multiple People with Low-Resolution Thermal Sensors in Smart Spaces

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15731738

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15731738

Country of ref document: EP

Kind code of ref document: A1