RU2481626C2 - Content annotation by means of context metadata - Google Patents

Abstract

FIELD: information technologies.

SUBSTANCE: method of content annotation with the help of context metadata consists in getting the first list of items of context metadata for a content copy, getting the second list of items of context metadata from the first list, at the same time the second list contains some, but not all items of metadata from the first list, and annotation of the content copy with the help of the second list by generation of an associative connection between items of metadata of the second list and the content copy.

EFFECT: expansion of functional capabilities of data annotation due to content annotation with the help of context metadata.

30 cl, 3 dwg

Description

FIELD OF THE INVENTION

The present invention relates to annotating content using contextual metadata. In particular, it relates to a method, apparatus, and computer program that makes contextual metadata more meaningful and facilitate annotation of content using meaningful metadata.

BACKGROUND

The increase in storage capacity allows users to store very large amounts of content in their mobile phones and other portable electronic devices. While the amount of content that needs to be managed is growing very fast, the physical UI (user interface) does not undergo significant changes. The resolution of the displays is growing, but the physical size of the displays will probably no longer increase, since in existing phones the display occupies most of the surface of the device. Text input speed is unlikely to be significantly increased. Non-keyboard methods of text input (for example, based on a stylus), as it turned out, do not solve the problem of difficulty and slowness of text input, and the general tendency to miniaturize phones, as well as the need to increase the size of the display, leave less and less space for the keyboard.

In the art, there appears to be an unbalanced development of the amount of stored content and the physical telephone user interface (UI).

The solution proposed by Sorvari et al.: “Convenience and ease of use of contextual metadata for managing mobile device content” (NordiCHI 104: Reports of the Third Scandinavian Conference on Human-Computer Interaction, AFM Press: 357-363), suggests that contextual metadata can be used to improve content management.

Contextual metadata is information that describes the context in which content was created (or received). This contextual metadata can be used to annotate an instance of content. They provide effective content management, such as search, and also provide organizational tools.

However, contextual metadata may be clogged with metadata that is not of interest to the user. For example, if contextual metadata includes identifiers of neighboring devices, then some of these devices may be of interest to the user, that is, be meaningful, for example, devices of friends, while other devices are not of interest to the user, that is, insignificant, for example, devices of passers-by. Clogging of meaningful metadata with meaningless metadata can interfere with the efficient use of contextual metadata.

When a user intends to annotate a content instance using contextual metadata, a problem may arise with selecting meaningful contextual metadata from the available contextual metadata. Insignificant contextual metadata clogs significant contextual metadata.

Similarly, when a user accesses metadata that is stored in association with the content, there may be a problem with displaying only meaningful contextual metadata selected from the saved contextual metadata.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a method for using contextual metadata that eliminates a clogging problem.

According to a first embodiment of the present invention, there is provided a method for annotating content using contextual metadata, comprising: a) obtaining a first list of contextual metadata items for a content instance; b) obtaining a second list of contextual metadata items from the first list, the second list containing some, but not all, metadata items from the first list; and c) annotating the content instance using the second list by forming an associative relationship between the metadata points of the second list and the content instance.

According to another implementation of the present invention, there is provided an electronic device for annotating content instances using contextual metadata, comprising: memory to store a first list of contextual metadata items for a content instance and the content instance itself; and a processor, to obtain a second list of contextual metadata items from the first list, the second list containing some, but not all, metadata items from the first list, and to annotate the content instance using the second list by forming associative links between the metadata points of the second list and the instance content.

According to another implementation of the present invention, there is provided a computer program for annotating content instances using contextual metadata, including computer program instructions that, when loaded into a processor, allow the processor to: a) obtain a second list of contextual metadata items from a first list of contextual metadata items for the content instance, moreover, the second list contains some, but not all, metadata items from the first list; and b) annotating the content instance using the second list by forming an associative relationship between the metadata points of the second list and the content instance.

According to another implementation of the present invention, a graphical user interface is provided in which contextual metadata items are displayed for selection by a user, providing a means for highlighting some contextual metadata items among other contextual metadata items to help a user select meaningful metadata items.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present invention, reference will now be made to the accompanying drawings, given here by way of example only, in which:

Figure 1 illustrates a block diagram 1 of devices;

Figure 2 illustrates the relationship between the first list of contextual metadata items and the second list of contextual metadata items that can be used to annotate a content instance; and

FIG. 3 schematically illustrates a procedure for annotating a content instance using metadata points according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The accompanying drawings illustrate a method for annotating an instance of content 43 using paragraphs 50 of contextual metadata, including:

a) obtaining the first list of 41 items (elements) 50 contextual metadata for an instance of 43 content;

b) obtaining a second list of 42 items 50 of contextual metadata from the first list 41, the second list 42 containing some, but not all, of the items 50 metadata from the first list 41; and

c) annotating the content instance using the second list by forming an associative relationship 44 between the metadata items 50 from the second list 42 and the content instance 43.

Information that relates to the location, time, date, and immediate surroundings, when an action is taken on the content instance 43 in the user device, can be used as contextual metadata for this content instance 43. The action may be, for example, creating, receiving, or editing a content instance 43. This metadata makes such information available beyond what is directly present in the content instance. Contextual metadata can be seen as a collection of 50 metadata items that provide a variety of information. Relevant metadata items 50 may include: time, date, location, neighboring devices (and for each neighboring device, its location and / or distance from the user device).

Content instance 43 is annotated using context metadata items 50 collected when an action on content instance 43 was performed on user device 10. This annotation is performed by forming an association 44 between the metadata items 50 from the context metadata and the content instance 43. The abstract provides the organization of an instance of content 43 based on items 50 of contextual metadata and / or search for an instance of content using items 50 of contextual metadata.

The immediate environment of the user device 10 may be determined, for example, by receiving identifiers transmitted by local radio devices 2, 4, 6. One type of identifier may indicate the network segment where local device 10 is located. For example, the identifier may be an SSID for a WLAN network or a combination of a network identifier and a cellular identifier for a cellular telecommunications network. Another type of identifier may indicate the presence of a device in the vicinity. For example, according to the Bluetooth standard, a Bluetooth device (BT) can perform a query procedure that allows it to receive BT device identifiers as well as friendly BT names from local BT devices. Both the 10 BT device and the BT friendly name can be used as an identifier. Some BT devices can be stationary, in which case the device identifier determines the location, while other BT devices can be user portable devices, such as mobile cell phones, which indicate the presence of a particular user.

The contextual metadata points 50 and, in particular, the metadata identifying neighboring devices (device identifiers) can be used to significantly improve the automatic organization and search capabilities provided by the user device 10. In addition, these metadata points 50 can also be made Available on other devices, such as personal computers, to improve content management on other devices.

Automatic detection of contextual metadata means that in some cases, contextual metadata is "clogged" with information that is insignificant for the user. For example, if a user device captures an image in a public place, it is possible that some neighboring devices may not be meaningful to the user, but they will also be included in contextual metadata. These devices may belong to strangers who accidentally fell into the frame when the image was captured.

Even with the limited range of modern Bluetooth radio devices, strangers are more likely to fall within the detection limits of BT radio devices in public places. When using WLAN technology or any other technology with a longer range, the number of insignificant devices increases in proportion to the square of the radius (area of the detection area).

1 illustrates a block diagram 1 of electronic devices 10, 2, 4, 6. The user device 10 is configured to save contextual metadata in the form of a first list 41 when an action, such as creating, receiving, or editing, is performed on the content instance 43. The content instance may be, for example, a video clip, image, sound file, text, multimedia presentation, etc.

The user device 10 includes a processor 20, a user input device 22, for example, a keyboard, a memory 24, a display 26, and context detection devices 30 for receiving context metadata items. The drawing shows only as many elements as necessary to explain the present invention.

The processor 20 is included in the circuit to read from the memory 24 and write to the memory 24, receive input from the user input device 22 and from the context detection devices 30, and provide the output data to the display 26.

The memory 24 stores instructions 25 of a computer program that, when loaded into the processor 20, allow the user device to operate as described below. Computer program instructions provide logic and routines that allow electronic device 10 to implement the methods illustrated in FIGS. 2 and 3.

Computer program instructions may be sent to the electronic device 10 via an electromagnetic carrier signal or from a physical object, such as a computer program product, memory device, or recording medium, such as a CD-ROM or DVD.

The metadata points 50 and the content instances 43 are stored in the database 27 in the memory 24. This database also stores information about the associations between the metadata points 50 and the content instances 43.

Context detection devices 30 may include a time / date sensor 32, such as a real-time clock, a location sensor 34, such as a GPS receiver, and a proximity sensor 36 that detects devices 2, 4, 6 closest to the user device 10. The proximity sensor may be a device Bluetooth that connects to Bluetooth devices 2, 4, 6, or WLAN and WiMAX devices, or a UWB device. Alternatively, any other proximity radio technology may be used to detect neighboring devices. In addition, neighboring devices can also be detected if these devices are included in the same (wireless) network.

Context detection devices 30 provide context metadata points (elements) for processor 20. These metadata may include: time, date, location, neighboring devices (and for each neighboring device, its location and / or distance from the user device).

The procedure according to the invention is schematically illustrated in FIGS. 2 and 3.

Figure 2 illustrates the relationship between the first list of 41 items 50 of contextual metadata that were originally obtained when performing the action on the content instance 43 and the second list of 42 items of contextual metadata that are used to annotate the 44 instance of the content 43. The first list of 41 items 50 of the contextual metadata includes “garbage”, that is, items 50 of contextual metadata that are insignificant for the user of user device 10. The second list of 42 items 50 of metadata contains a subgroup of the first list 41 and includes those nkty metadata 50 from the first list 41, which are significant. The filter automatically filters the first list of 41 metadata points so as to identify metadata points in the first list that are likely to be significant, and / or automatically extract metadata points from the first list that are likely to be significant, and include them in the second list 42. Instance content 43 is annotated by forming an association 44 between each of the metadata items 50 in the second list 42 and the content instance 43.

FIG. 3 schematically illustrates a procedure for annotating a content instance using metadata points according to an embodiment of the present invention.

At step 60, an action is taken on the content instance 43. The action may be, for example, creating, receiving or editing content instances 43. The content instance 43 may be, for example, a video clip, image, sound file, text, etc.

Depending on step 60, in step 62, for the content instance 43, a first list 41 of the context metadata items 50 is obtained. This list is stored in association with this content instance in database 27. If the content instance is received in the user device 10, then the context metadata points of the first list 41 may be context metadata points received with the content instance 43 and / or context metadata points determined by the context detection devices 30 in the user device 10 upon receiving the content instance 43. If a content instance 43 is created, then the context metadata items in the first list 41 represent the context metadata items defined by the context detection devices 30 in the user device 10 when creating the content instance 43.

After step 62, in step 64, a second list of 42 contextual metadata items is obtained from the first list. This is described in more detail below. The second list is stored in association with a content instance in database 27.

Then, at step 66, the content instance 43 is annotated with the second list 42, creating an association between the metadata items 50 from the second list and the content instance 43.

Step 64, where a second list of 42 items 50 of contextual metadata is obtained from the first list 41, includes filtering the first list to improve the significance of the metadata items.

Step 64 may include a manual step, in which case the first list 41 is displayed, a filter is used to highlight some metadata items from others, and the user selects at least some metadata items from the first list 41 for use in the second list 42 The allocation can be carried out using a visual assessment for each metadata item and / or the location of the metadata items in accordance with priority according to rating 51. Manual transfer can be performed when occurrence of action or some time later.

Step 64 may include an automated step, in which case the first list 41 may, but not necessarily, be shown on the display, the filter selects at least some metadata items from the first list 41 for use in the second list 42. The filter may use an estimate when selected 51.

A score of 51 for metadata item 50 may depend on statistics on the use of that item metadata when annotating content instances 43. For example, filtering may associate a metadata subordinate of the first list 41 with content instances that it already annotates. This can be achieved by providing access to the previously stored second metadata lists 42 in the database 27 and identifying these second metadata lists 42 in which this subordinate metadata item from the first list 41 occurs.

Each metadata item 50 may include a rating field 51 that indicates, in this example, the number of content instances that this item 50 metadata is already associated with. This rating field 51 determines the significance of the metadata points 50 by identifying those metadata points that the user statistically prefers to use to annotate the content instances 43. This allows the user to recognize those metadata items that are most likely to be selected from the first list of metadata items for use in the annotation. The metadata items of the first list can be sorted in descending order according to the value of field 51.

The score 51 for the metadata item 50 may depend on the statistics of the appearance of the metadata item in the stored first lists 41, which are associated with different instances of the content. This can be achieved by providing access to the previously saved first metadata lists 41 in the database 27 and identifying these stored first metadata lists 41 in which a subordinate metadata item from the existing first list is found. Each metadata item 50 may include an evaluation field 51, which indicates, in this example, the number of first lists 41 in which this item 50 metadata occurs.

After that, the rating field 51 can be used by the user as a criterion for manually transferring metadata items 50 from the first list 41 to the second list 42. Alternatively or additionally, the rating field 51 can be used as a criterion for automatically transferring metadata items 50 from the first list 41 to the second List 42 without user intervention. The metadata item 50 is automatically added to the second list 42 if the rating field 51 for that metadata item exceeds a threshold value. The threshold value may be different for different types of content instances. That is, the threshold value depends on what a given instance of content is, with which a second list of metadata items should be associated: an audio file, text, image, video, etc. Alternatively or additionally, a combined threshold may be set for all or selected groups of types of content instance types.

The evaluation field 51 is particularly important for metadata items that represent identifiers of neighboring devices 2, 4, 6, since the identifier of such a device is not always meaningful to the user. When transferring the device identifier manually, the user may be given the opportunity to correct or supplement the contents of this metadata item to make it more meaningful, or to associate the device identifier with an existing alias / name / contact or other object that describes the device or its owner. Such an object may be accessible, for example, using a telephone directory, a (instant) messaging system, publishing, or existing applications.

The evaluation field 51 may be calculated in other ways, as a supplement or as an alternative to the method described above. The evaluation field can be made dependent on other features, and not just on the number of associative links / annotations.

The assessment may depend, for example, on any of the following features, or a combination thereof:

a. The number of associative links 44 between the metadata item and other content instances 43. The larger the number of associative links, the greater the score of 51, and the lower the number of associative links, the lower the score of 51. Associative links may be formed as a result of the appearance of a metadata item in the first list group 41. Alternatively, associative links may be formed as a result of the appearance of metadata in the second list group 42.

b. How recently have associations formed 44 between the metadata item and other content instances. More emphasis is placed on more recent associative relationships than older associative relationships. Associations can be formed as a result of the appearance of a metadata item in the group of the first lists 41. Alternatively, associations can be formed as a result of the appearance of a metadata item in the group of the second lists 42.

c. The type of action that is performed on the content instance and / or the type of content, for example, capturing an image / video, recording a sound clip, receiving a message, editing, etc.

d. For a device identifier, the distance to the device and / or direction to the device represented by the device identifier relative to the user device when performing an action on a content instance. If the distance is large or the device is located behind the user, when the action performed is an image / video capture, then the rating value is low. However, if the distance is small and / or the device is located in front of the user, when the action is an image / video capture, the rating value is high. To implement this option, the first list of metadata points includes, for each neighboring device, metadata that provides a device identifier for each neighboring device, as well as metadata that provides distance to this neighboring device and / or direction to that neighboring device.

The local environment of the device from which the content metadata item is obtained can also be used in calculating the score. The environment may include the number of devices, distance and / or direction and rating for neighboring devices.

As an example, for a device identifier, the number of devices, the distance to other neighboring devices, and / or direction to other neighboring devices when an action is taken on an instance of content. These signs can be used to determine whether the device that is equipped with this device identifier is located in the middle of a large crowd, and then the rating value is low, or the device represents one of the few devices, and then the rating value is high. To implement this option, the first list of metadata points includes, for each neighboring device, metadata that provides such a device identifier for each neighboring device, and also metadata that provides the distance to this neighboring device and / or direction to this neighboring device.

As another example for a device identifier, the number of devices, the distance to devices with a high rating and / or direction to devices with a high rating, adjacent to the device represented by this device identifier, when an action is taken on a content instance. Highly rated device means any device whose identifier is already added to the second list or whose identifier contains a highly rated field. If a device is found close to or close to a device with a high rating, then its rating can be significantly increased.

More sophisticated mechanisms for determining grades can be used additionally or alternatively. For example, the rating associated with the device identifier may depend on the rating that the device represented by this device identifier receives in the user device 10. For each device identifier in the first list 41, the user device sends a rating request message to the device represented by this identifier device (target device). This evaluation request message includes a user device identifier. The target device either corrects the rating in connection with the received device identifier, or calculates the rating for that device identifier. It then sends a response message to the user device that contains the calculated score. The user device adjusts its rating for the identifier of this target device depending on the rating received in the response message from the corresponding target device. The higher the accepted rating, the higher the significance increase applied to the evaluation of the identifier of this target device.

In this way, if the user device is recognized as significant by another neighboring device in the first list 41, then another device can be added to the second list 42, which includes the significant devices. In this way, devices that recognize you as significant can be added to your second list of significant devices.

In another embodiment of the present invention, the assessment response message may further or alternatively include names corresponding to the identifiers of the highest rated target devices. The identifiers of devices present in the first list, which are also present in such an assessment response message, can be automatically transferred to the second list, or their rating will increase significantly. Another metadata item, a marker, can also be associated with a device identifier, which is added to the second list in the following way. This marker is also transmitted in the evaluation response message and is incremented by one each time it is received. The device identifier rating received by the user device in the evaluation response message is valid to increase the rating of this device identifier only if the token is less than a predetermined threshold value.

Also, in the second list in step 64, the addition procedure may be performed. This procedure adds a user-recognized value to the seemingly insignificant metadata items in the second list.

For example, instead of or in addition to setting location coordinates, the coordinates can be specified as the name of the area. The name of the area may correspond to the smallest geographical area previously used by the user device, which accurately indicates this location. Alternatively or additionally, the location may be compared with the locations of the addresses stored in the contact (address) database. The location in this case can be specified as, for example, “near John Dow’s house,” if John Dow’s home address represents the closest address in the contact database and is located within a threshold distance.

For example, instead of or in addition to setting the date and time, the date and time can be set as the period relative to the entry in the calendar program. If the date and time correspond to a meeting with John, then the user device may optionally provide the phrase “meeting John at his office” as metadata, or if the date and time follows the meeting, the user device may provide additional metadata in the form of the phrase “after meeting John in his office. ”

Other options for additional metadata may also be presented to the user.

As a result of filtering, a second list of metadata items significant for the user device will be generated. A second list may be presented to the user. The addition procedure can also improve the relevance of the metadata items presented to the user.

The organizer application can be used to organize / manage stored copies of 43 content using the corresponding second lists of 42 items 50 metadata. A search tool may also be generated based on metadata items in second lists 42.

Although implementations of the present invention have been described above with respect to various examples, however, modifications can be made to these examples without departing from the scope of the invention defined in the appended claims.

Although the foregoing description draws attention to those distinguishing features of the present invention that are of particular importance, the applicant claims his rights with respect to any patentable distinctive feature or combination of such features mentioned above and / or shown in the attached drawings, regardless of whether whether or not a specific emphasis is made in the foregoing description.

Claims (30)

1. A method for annotating content using contextual metadata, including
a) obtaining a first list of context metadata items for the content instance, wherein the context metadata items contain information related to devices that were next to the electronic device at the time the content instance was created;
b) obtaining a second list of contextual metadata items, the second list containing some, but not all, of the metadata items from the first list; and
c) annotating the content instance using the second list by forming an associative relationship between the metadata points of the second list and the content instance. 2. The method according to claim 1, wherein obtaining the second list of contextual metadata items from the first list includes processing the first list so as to increase the importance of the metadata items. 3. The method according to claim 2, in which the processing includes the addition of metadata. 4. The method according to claim 3, in which the additional metadata depends on previously stored data. 5. The method according to claim 4, in which the additional metadata includes an assessment. 6. The method according to claim 4 or 5, in which the additional metadata includes user-relevant information based on data obtained from a personal database. 7. The method according to any one of claims 1 to 5, further comprising
display the first list;
highlighting some metadata items in the first list compared to other metadata items in the first list; and
allowing the user to select at least some metadata items from the first list for use in the second list. 8. The method according to claim 7, in which the selection is provided using visual ratings for metadata items. 9. The method according to claim 7, in which the allocation is provided by the arrangement of metadata points in accordance with the priority according to the assessment. 10. The method according to any one of claims 1 to 5, in which the processing automatically selects at least some metadata items from the first list for use in the second list. 11. The method of claim 10, wherein the metadata items in the first list are provided with ratings, and these ratings are used as a selection criterion. 12. The method according to any one of claims 1 to 5, in which obtaining a second list of contextual metadata items from the first list includes providing an assessment of the metadata items in the first list. 13. The method according to claim 5, in which the assessment for the metadata item depends on the statistics of the use of this item metadata for annotating content instances. 14. The method of claim 13, wherein the rating for the metadata item depends on the number of previous uses of the metadata items to annotate content instances. 15. The method according to item 13 or 14, in which the assessment for the metadata item depends on how recently this metadata item was used to annotate content instances. 16. The method according to claim 5 or 14, in which the assessment for the metadata item received from the device depends on the local environment of this device. 17. The method according to claim 5 or 14, in which the rating for the metadata item received from the device depends on the rating that the device has for the metadata item corresponding to another device. 18. The method according to any one of claims 1 to 5, in which the second list may include metadata items not present in the first list. 19. An electronic device for annotating content instances using contextual metadata, including
a memory for storing a content instance and a first list of context metadata items for the content instance, wherein the context metadata items contain information related to devices that were next to the electronic device at the time the content instance was created; and
a processor for obtaining a second list of contextual metadata items, where the second list contains some, but not all, of the metadata items from the first list, and for annotating the content instance using the second list by forming associative links between the metadata points of the second list and the content instance. 20. The electronic device according to claim 19, which is a hand-held portable device. 21. A physical object that implements a computer program for annotating content instances using contextual metadata, wherein the contextual metadata items contain information related to devices that were next to the electronic device at the time the content instance was created, while the computer program includes computer instructions that, when loaded into the processor, allow the processor to
a) obtaining a second list of contextual metadata items by automatically filtering the first list of contextual metadata items for the content instance, the second list containing some, but not all, metadata items from the first list; and b) annotating the content instance using the second list by forming an associative relationship between the metadata points of the second list and the content instance. 22. The physical object according to item 21, in which obtaining a second list of items of contextual metadata from the first list includes processing the first list so as to increase the importance of the items of metadata. 23. The physical object according to item 22, in which the processing includes the addition of metadata. 24. The physical object of claim 23, wherein the additional metadata is dependent on previously stored data. 25. The physical object of claim 24, wherein the additional metadata includes an estimate. 26. The physical object according to paragraph 24 or 25, in which the additional metadata includes user relevant information based on data obtained from a personal database. 27. A physical object according to any one of paragraphs.21-25, also including computer commands for
display the first list;
highlighting some metadata items in the first list compared to other metadata items in the first list; and
allowing the user to select at least some metadata items from the first list for use in the second list. 28. A physical object according to any one of paragraphs.21-25 for automatically selecting at least some metadata items from the first list for use in the second list. 29. A physical object that implements a computer program for implementing the method according to any one of claims 1 to 18. 30. A graphical user interface in which contextual metadata items are displayed for selection by a user, the contextual metadata containing information related to devices that were next to the electronic device at the time the content instance was created, while said user interface provides a means for highlighting some of the contextual metadata items among other contextual metadata items to help the user select meaningful metadata items.

Images