GB2465965A

GB2465965A - A computing device for media tagging

Info

Publication number: GB2465965A
Application number: GB0821626A
Authority: GB
Inventors: Ritesh Babla
Original assignee: Nokia Oyj; Symbian Software Ltd
Current assignee: Nokia Oyj
Priority date: 2008-11-26
Filing date: 2008-11-26
Publication date: 2010-06-09
Also published as: GB0821626D0; WO2010061345A1

Abstract

A computing device is arranged to compare a timestamp of a file such as an item of multimedia with a database of calendar entries. In the event of a match between the timestamp and at least one calendar entry data of the calendar entry is copied to the file. The device may comprise a mobile telephone equipped with a camera.

Description

A computing device for media tagging The present invention relates to a computing device which is arranged to apply tags to media. The invention also relates to a corresponding method.

Background to the Invention

Modern media file formats enable information to be stored with the media. For example, an image may have date and time information associated with it. A user may add an image title or image description. This information is generally referred to as metadata and can be read and written to by most modern media reading applications.

Some metadata is recorded automatically when the media is created. For example, most cameras apply time and date information when a photo is created. Applications also exist, for example on modern smart phones, for geographical information to be added automatically when a photo is taken.

Such an application uses inbuilt satellite positioning technology to record, in metadata, the place at which the photo was taken.

Despite the existence of mechanisms for automatic metadata creation, it is generally necessary for a user to manually and retrospectively add detailed information such as a description of what a photo is of or what event it was taken at. For example, if a user attends a friend's birthday party on a particular date, it is necessary for the user to enter the event details at a later time. This is generally done when a user transfers their photos to their personal computer.

Image recognition programs are known which are able to detect objects present in an image. However, the results produced by such programs are not accurate and only relate to the content of the image. They are not able to detect the nature of the event at which the photo was taken.

There is also a problem with erroneous metadata. The existence of large amounts of incorrect or inaccurate metadata can result in difficulties with information management. Individuals who have concerns with the inappropriate use of metadata are referred to as "meta-haters".

There is a need for improved applications which make the task tagging media with useful descriptive details easier.

Summary of the Invention

The present invention provides a computing device which is arranged to compare a timestamp of a file with a database of calendar entries and, in the event of a match between the timestamp and at least one calendar entry, to copy data of said at least one calendar entry to said file.

The present invention also provides a method for providing files with descriptions, comprising the steps of: comparing the timestamp of a file with a database of calendar entries; and in the event of a match between the timestamp and at least one calendar entry, copying data of said at least one calendar entry to said file.

Additionally, the present invention provides a computer program or suite of computer programs arranged such that when executed by a computer they cause the computer to operate in accordance with the aforementioned method.

The present invention also provides a computer readable medium storing the computer program, or at least one of the suites of computer programs, described in the previous paragraph.

Furthermore the present invention provides an operating system for causing a computing device to operate in accordance with the aforementioned method.

Other features of the present invention are defined in the appended claims.

Features and advantages associated with the present invention will be apparent from the following description of the preferred embodiments.

Brief Description of the Drawings

The invention will now be described in more detail, by way of example, with reference to the accompanying drawings in which: Figure 1 shows a front view of a mobile device in accordance with an embodiment of the invention; Figure 2 shows a rear view of the mobile device shown in Figure 1; Figure 3 is a schematic diagram of the components of the mobile device shown in Figure 1; Figure 4 is a flow chart showing a method of operation of the device shown in Figure 1 to 3; Figure 5 is a further flow chart showing a method of operation of the device shown in Figure 1 to 3; Figure 6 is a further flow chart showing a method of operation of the device shown in Figure 1 to 3; Figure 7 is a further flow chart showing a method of operation of the device shown in Figure 1 to 3; and Figure 8 is a further flow chart showing a method of operation of the device shown in Figure 1 to 3.

Detailed Description

A mobile device 101, which is arranged to operate in accordance with a first embodiment of the present invention, is shown in Figure 1. The mobile device 101 comprises an outer casing 102, which includes an earphone 103 and a microphone 104. The mobile device 101 also includes a keypad 105 and a display 106. The keypad 105 enables a user to enter information into the mobile device 101 and instruct the mobile device to perform the various functions which it provides. For example, a user may enter a telephone number, or select another mobile device from a list stored on the mobile device 101, as well as perform functions such as initiating a telephone call.

Figure 2 is a rear view of mobile device 101. The device 101 further includes a camera 107.

Figure 3 is a schematic diagram showing the components of the mobile device 101. The components of the mobile device 101 include the earphone 103, the microphone 104, the keypad 105, the display 106 and the camera 107. The mobile device 101 also includes a system bus 108 to which the components are connected and which allows the components to communicate with each other. Here, the components are shown to communicate via a single system bus 108. However, in practice the mobile device may include several buses to connect the various components. The device also includes an application processor 109, a baseband processor 110, memory 111, an earphone controller 112, a microphone controller 113, a display controller 114, a keyboard controller 115, and a camera controller 116. The device 101 also includes a mobile telephone radio 117 and a storage device controller 118.

The application processor 109 is for running an operating system and user applications. The baseband processor 110 is for controlling a telephony stack. The mobile telephone radio 117 is also connected to an antenna 119.

The mobile device 101 is arranged to communicate, via radio 117, with a base station of a mobile phone network (not shown). The storage device controller 118 is connected to a storage device 120 which may be an internal hard drive or a removable storage device such as a flash memory card. The mobile device 101 also includes an Bluetooth radio 121 which is connected to an antenna 122.

The device 101 includes a camera application (not shown). The application is stored in Random Only Memory (ROM) which forms part of the memory 111.

The camera application controls the camera 107. The application performs the functions generally associated with a camera including control of camera settings, image capture format, image capture and image storage.

The camera application is arranged to store images in JPEG/EXIF file format.

EXIF files include information regarding the image. Such information is referred to as metadata. The EXIF format includes information which is automatically generated by the camera. Examples of such metadata include camera make and model, photo date and time, exposure and aperture settings, etc. In addition, a user may add information to an EXIF file such as a title, keywords and a description. User generated key words are normally applied at a time somewhat after the time when the photo was taken. Such data is referred to as user metadata.

The device 101 also includes an iCalendar compatible calendar application.

Such an application includes a database of event and to-do entries which are stored in accordance with the iCalendar format (RFC 2445). A typical event entry may be stored as follows: BEGIN: VCALENDAR VERSION:2.0 BEGIN: VEVENT DTSTART: 20080416T190000Z DTEND: 200870417T020000Z

SUMMARY:Matthew's Birthday Party

END:VEVENT END: VCALENDAR The format of the data is vCalendar version 2. The entry is an event which starts at 7pm on 16 April 2008 and finishes at 2am on 17 April 2008. The title of the event is "Matthew's Birthday party".

The camera application is arranged to apply information from the calendar to files it creates. This process may be referred to as tagging. The camera application includes a menu system which provides various options for a user in this regard. The camera includes a file tagging master control. This control can be set to "on" or "off'. When the control is switched off, the camera application does not transfer any information to the files it creates. When the control is set to "on", it applies tags to files in accordance with the options described below. This process is described in connection with Figure 4. The process begins when a user takes a picture (step 301).

When the master control is set to "on", the camera application automatically checks whether or not the calendar has any entries which match the date and time of the image which has been captured (step 302). The camera application checks whether or not the timestamp stored in the EXIF file matches the criteria expressed in equation 1.

DTSTART �= timestamp �= DTEND (1) Following this check, the camera application arrives at one of the following three outcomes: 1) no matches (step 303); 2) a single match (step 304); and 3) more than one match (step 305).

At this point, an alert is displayed informing the user of the outcome (steps 306 to 308). The sequence of events which follow depends on various choices which a user has made, or does make, as will be described in the following.

Figure 5 show the process in the event that no calendar entries are located.

The camera application includes menu options relating to what to do in the event that no calendar entry exists. The menu options are: a) take no action without prompting; b) search for matching media without prompting; and c) prompt user.

If the user previously selected option a), the camera application takes no action and stores the image file without tagging it with any calendar information (step 309).

If the user previously selected option b), the camera application searches for image files which have similar time and date information to the newly captured image (step 310). Specifically, the camera application searches for an image file having the closest timestamp to the captured image. Once the application has located a file, it asks the user if they wish to use the description of the image which has been located (step 311). The user is the provided with the options "yes", "no" or "cancel". If the user selects "yes", the description of the matched file is applied to the captured image (step 312). If the user selects "no", the next closest match is selected and the options repeated (step 313).

If the user selects "cancel", the process is ended (314). The camera application is set to search up to a maximum of one day in the past. If no file is located the, the user is informed and the file is saved without any tag (step 315).

If the user previously selected option c), the camera application alerts the user that no calendar information exists, and asks the user whether to take no action or search for similar media (step 316). In the event that the user chooses to take no action, the process returns to step 309. In the event that the chooses to search for similar media, the process returns to step 310.

Figure 6 shows the process which occurs if a single calendar event is located.

The process depends on prior choices made by the user in the camera application menu system. The menu options available are: a) use calendar entry without prompting; b) ignore calendar entry without prompting; and c) prompt user and ask whether to use entry or not.

If the user previously chose option a), the camera application copies the event summary ("SUMMARY", e.g. Matthew's Birthday Party) and enters this information into the description field of the EXIF file (step 317). The user can also choose to be prompted in this scenario. If the user chooses to be prompted, a window is displayed prompting the user that a match has been found and that the summary information is being copied. In this event, the user is not able to cancel the transfer. The process then ends with the image file being stored with the summary information (step 318).

If the user previously selected option b), the entry is ignored and the no information is transferred (step 319). The menu system provides the option for a user to be alerted to the fact that calendar entries are being ignored.

If the user previously selected option C), the camera application prompts the user to choose whether to use the calendar entry or ignore it (step 320). The prompt provides details of the calendar entry which matches the timestamp of the captured image. If the user chooses to transfer the calendar event summary, the process reverts to step 317. If the user chooses to ignore the entry, the process reverts to step 319. In this manner, if a matched event does not in fact relate to a captured image, a user can choose to ignore it.

Figure 7 shows the process which occurs if two or more calendar events are located which match the timestamp of the captured image. The camera application menu system provides a user with options in relation to what the application should do in the event that two or more events are located. The options are: a) use all calendar entries without prompting; b) ignore all calendar entries without prompting; and c) prompt user to select event.

If the user previously chose option a), the camera application copies the event summary for all matching events and enters this information into the description field of the EXIF file (step 321). The user can also choose to be prompted in this scenario. If the user chooses to be prompted, a window is displayed prompting the user that a match has been found and that the summary information is being copied. In this event, the user is not able to cancel the transfer. The details of the event information which is being transferred may also be displayed. The process then ends with the image file being stored with the transferred summary information (step 322).

If the user previously selected option b), the entries are ignored and the no information is transferred (step 323). The menu system provides the option for a user to be alerted to the fact that calendar entries are being ignored.

If the user previously selected option c), the camera application prompts the user to choose one or more of the calendar entries to be ignored (step 324).

The prompt provides details of the calendar entries which match the timestamp of the captured image. The user may choose one or more of the entries to transfer. Additionally the user may choose to ignore all entries, If the user chooses to transfer one or more calendar event summaries, the camera application copies the selected event information and the captured image is saved (step 325). If the user chooses to ignore the entry, the process reverts to step 323. In this manner, if the matched events do not in fact relate to a captured image, a user can choose to ignore them.

In accordance with this embodiment, a camera application is provided which utilises information stored on the device to "tag" newly captured images. This is quicker and more efficient than a user having to individually tag each image.

The information is also specific to the user and uses user information rather than arbitrary location tags or image recognition tags.

In another embodiment, the mobile device 101 includes a file transfer application. This application is arranged to transfer files to and from the device using technologies such as Bluetooth. The file transfer application includes a menu system which provides a user with options relating to file transfer. In particular, the menu system provides a user with options regarding the use of calendar information to "tag" incoming files.

The menu system provides options in relation to the various scenarios which occur when transferring a file to the mobile device 101. The menu system includes a file tagging master control. If master control is switched "off', the file transfer application does not transfer file calendar information. Files are transferred to the device and metadata is not altered by the application. If the master control is switched "on", the file transfer application operates in accordance with selections made in other menus, as will be described below.

The file transfer process will be described with reference to Figure 8. The process begins when a file is transferred to the mobile device 101 (step 400).

When a file is transferred to the device the file transfer application checks whether or not the file has user metadata (in the form of an EXIF file description) associated with it (step 401). The menu system includes a set of options which relate to what the application should do when a file is determined to have user metadata associated with it already. The menu options are: a) keep existing metadata; b) keep existing metadata and use calendar entries; c) delete existing metadata and use calendar entries; and d) delete existing metadata.

If the user previously selected option a), the file transfer application stores the incoming file without altering the user metadata (402). The user can choose to be prompted that this has occurred.

If the user previously selected option b) the file transfer application checks the calendar for entries with matching timestamps (step 403). The file transfer application automatically checks whether or not the calendar has any entries which match the date and time of the file which is being transferred to the mobile device 101. The file transfer application checks whether or not the timestamp stored in the imported files metadata matches the criteria expressed in equation 1, as noted above.

At this point, three options are possible. Either no match is located (step 404), a single match is located (step 405), or two or matches are located (step 406).

The file transfer application then operates in the same manner as the camera application as described above and shown in Figures 5, 6 and 7.

If the user previously selected option c), the file transfer application operates in the same way as for option b) except that the existing metadata is deleted.

This process therefore begins at step 403.

If the user previously selected option d), the existing metadata is deleted but no calendar information is transferred (step 407).

In a separate menu, a user can choose whether or not to be prompted about these actions. In other words, a user a choose for these actions to occur automatically, or for the user to be prompted before action is taken. In particular, the user can select to prompted only if data is going to be overwritten.

If no metadata is associated with the incoming file (step 408), the file transfer application operates in accordance with previously selected user options. The menu options are: a) take no action; and b) use calendar entries.

If a user selects "take no action", then files are transferred to the mobile device without modification (step 409). If a user selects "use calendar entries" the file transfer application compares the timestamp included with a file's metadata with the calendar in accordance with step 403.

In addition to the above, the file transfer application includes further options relating to what the application should do in the event that incoming files includes user metadata, but when the calendar includes no matching events.

The menu system provides the user with an option to copy user meta data to the calendar. If the user copies a single file, the file transfer application will ask the user whether or not to copy the file description to the caIendar as an event. If the user copies several files, each having the same description, the file transfer application creates an event in the calendar with DSTART and DEND information matching the timestamp of the earliest and latest file. If the user transfers a number of files having two or more descriptions, the file transfer application will give the user the option to create events relating to

each description.

The file transfer application may transfer groups of files. The application is arranged to detect when a group of files all match a single event. In this case, the application will give the user the option to "tag" all of the selected files with

the same event description.

The above described applications can be used with a variety of media types, in particular, images and video.

In the above described embodiments, the term "tagging" has been used in a generic sense. Specifically, a "tag" may be any word or phrase associated with a particular file. In the art "tagging" may have a more specific meaning.

In the art a "tag" is a word associated with a particular file in order to facilitate searching. A file which is compatible with "tags" may have a tag field in which a user can enter single words which relate to that file. For example, if a file relates to "Matthew's Birthday Party", a user may choose to tag the file with the term "birthday". Certain applications can use tags to analyse the content of a group of files. For example, a list of tags can be produced indicating how many images contain the tag "birthday".

In a further embodiment, a mechanism is provided for generating new tags for media files. The mobile device 101 is provided with a tagging assist application. The application provides the mechanism by which tags are applied to files. The application works in combination with the applications described above to allow tags to be applied to media files. The application provides a database for storing tags. Initially this database is empty. When the camera application or file transfer application stores a file in device memory, the user is given the option to apply tags. If a file has a description associated with it, the user is invited to apply a tag. The tag must be a word which appears in the file description. For example, if the description is "Matthew's Birthday Party", the tag may be, "matthew" "birthday" or "party".

The user may enter one or more of these tags. Any tags which are entered are stored in the tag database.

When a user next comes to tag a file, the tag application checks the database to see if any of the words in the file description appear there. If so, the tag application suggests the stored tags as possible tags for the new file. The following gives an example of the transfer of four image files to the device: 1) Peer's Birthday -manual tags: peer and birthday; 2) Jen's Birthday -manual tag: jen -assisted tag: birthday; 3) Peer's Anniversary -manual tag: anniversary -assisted tag: peer; 4) Jen's Anniversary -assisted tags: jen and anniversary.

As the user enters more tags, the more dynamic the system becomes at suggesting appropriate tags.

The mechanism can be arranged so that a manually entered tag does not have to relate to description. Such tags are entered in the database and may be used to suggest assisted tags for files at a later time.

The tagging application can also be used when creating calendar entries, For example, when a user enters a new calendar entry, they can also assign manual tags as described above. Subsequent entries can then be automatically tagged using the suggested assisted tags.

The term "timestamp" is used to refer generally to information relating to the time and/or date on which a file was created. It is not limited to any particular format or standard of time/date related information. Nor does it require the presence of information which relates specifically to time. A timestamp may just include data information.

Various modifications, changes, and/or alterations may be made to the above described embodiments to provide further embodiments which use the underlying inventive concept, falling within the spirit and/or scope of the invention. Any such further embodiments are intended to be encompassed by the appended claims.

Claims

Claims 1. A computing device which is arranged to compare a timestamp of a file with a database of calendar entries and, in the event of a match between the timestamp and at least one calendar entry, to copy data of said at least one calendar entry to said file.
2. A device according to claim 1, wherein said data is an eventdescription.
3. A device according to claim 2, wherein said event description is copied to file user metadata.
4. A device according to claims 1, 2 or 3, further arranged to copy said data automatically.
5. A device according to claims 1, 2 or 3, further arranged to prompt a user of said device prior to copying said data.
6. A device according to any preceding claim, wherein said file is generated on the basis of an image captured by a camera and said device is further arranged to automatically compare the timestamp of a said file with the database of calendar entries following generation of said file.
7. A device according to any of claims 1 to 5, wherein said file is transferred to said device from another device and said device is further arranged to automatically compare the timestamp of said file with the database of calendar entries following transfer.
8. A device according to any preceding claim, further arranged to provide a user with an option as to which calendar entry to use in the event that more than one matching calendar entry is located.
9. A device according to claim 7, further arranged to check any files transferred to the device for existing user metadata, and, in the event that exiting user metadata exists a) keep existing data in addition to copying any data of any matching calendar entries; or b) deleting existing data and copying any data of any matching calendar entries.
10. A device according to any preceding claim, further comprising a database of tags, wherein said device is arranged to automatically apply tags to files in the event of a match between a file description and a tag.
11. A device according to any preceding claim, further arranged to create a new calendar entry, using the timestamp of a transferred file and the user metadata of the said file, in the event that the calendar contains no events matching the timestamp of the said file.
12. A method for providing files with descriptions, comprising the steps of: comparing the timestamp of a file with a database of calendar entries; and in the event of a match between the timestamp and at least one calendar entry, copying data of said at least one calendar entry to said file.
13. A method according to claim 12, further comprising copying said data automatically in the event of a match.
14. A method according to claim 12, further comprising prompting a user prior to copying said data.
15. A method according to any of claims 12 to 14, further comprising capturing an image using a camera, generating a file on the basis of the captured image and automatically comparing the timestamp of a said file with the database of calendar entries.
16 A method according to any of claims 12 to 14, further comprising transferring a file to a device from another device and automatically comparing the timestamp of said file with the database of calendar entries.
17. A computer program or suite of computer programs arranged such that when executed by a computer they cause the computer to operate in accordance with the method of any of claims 12 to 16.
18. A computer readable medium storing the computer program, or at least one of the suites of computer programs, according to claim 17.
19. An operating system for causing a computing device to operate in accordance with a method as claimed in any one of claims 12 to 16.
20. A computing device substantially as described hereinbefore and as shown in Figures 1 to 8.