WO2014002004A1 - A method for marking highlights in a multimedia file and an electronic device thereof - Google Patents

A method for marking highlights in a multimedia file and an electronic device thereof Download PDF

Info

Publication number
WO2014002004A1
WO2014002004A1 PCT/IB2013/055202 IB2013055202W WO2014002004A1 WO 2014002004 A1 WO2014002004 A1 WO 2014002004A1 IB 2013055202 W IB2013055202 W IB 2013055202W WO 2014002004 A1 WO2014002004 A1 WO 2014002004A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
index
marks
mark
multimedia file
Prior art date
Application number
PCT/IB2013/055202
Other languages
French (fr)
Inventor
Sumana Krishnaiahsetty BATCHU
Original Assignee
Batchu Sumana Krishnaiahsetty
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from GBGB1211181.1A external-priority patent/GB201211181D0/en
Application filed by Batchu Sumana Krishnaiahsetty filed Critical Batchu Sumana Krishnaiahsetty
Priority to GB1501008.5A priority Critical patent/GB2518791A/en
Publication of WO2014002004A1 publication Critical patent/WO2014002004A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/7867Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/74Browsing; Visualisation therefor

Definitions

  • the present disclosure relates generally to marking highlights in multimedia files. More specifically, the present disclosure relates to marking searchable highlights in multimedia files by associating the highlights with keywords and tags thereby improving the speed and precision of accessing relevant sections in a multimedia file.
  • One way to index the multimedia files is to automate the marking of highlights in the files.
  • automation of marking highlights is desirable, it yields limited results due to the incapability of automated programs to understand the rich semantics of the content, leading to manual method being the ideal choice.
  • automation is extremely processor intensive, making it a difficult choice in the new-age mobile devices which are increasingly becoming the channel of consumption for multimedia files.
  • manual browsing and marking multimedia files for highlights is a very time consuming task, especially due to the abundance of their availability.
  • it would be beneficial to users if they could share the task of identifying highlights and metadata for multimedia files with other users.
  • Such a collaborative effort would be additionally relevant when done by users with similar interests, hobbies, profession, etc., or for example by related members in a social networking environment.
  • the present disclosure presents a system and associated methods to facilitate processor-efficient marking and playing of highlights in multimedia files and also to qualify them with relevant metadata to enable search within the multimedia files,
  • the present disclosure relates to a method and an electronic device to facilitate marking of highlights in multimedia files and to qualify the marked highlights with relevant metadata to enable search within the multimedia files.
  • the present disclosure facilitates marking of highlights in a multimedia file using either manual or automated approaches. Different approaches are suitable under different usage scenarios as detailed in the description below.
  • the present disclosure relates to a method of marking highlights in a multimedia file using an electronic device.
  • the method comprises receiving at least one pair of file-marks comprising a begin-file-mark and an end-file-mark corresponding to predetermined locations in the multimedia file, wherein each pair of file marks denotes a highlight and then, storing the at least one received pair of file -marks in an index-table, in a storage unit, by a processing unit configured in the electronic device.
  • the method comprises receiving one or more data elements by the processing unit for associating with the at least one of stored file-marks.
  • the one or more data elements are selected from at least one of pre-defined keywords with associated values, predefined tags and custom tags.
  • the associated data elements along with references to corresponding file-marks are stored in an information-table in the storage unit by the processing unit.
  • the present disclosure relates to an electronic device for marking highlights on a multimedia file.
  • the electronic device comprises a processing unit coupled with a storage unit.
  • the processing unit is configured to receive at least one pair of file-marks comprising a begin-file-mark and an end-file-mark corresponding to predetermined locations in the multimedia file, wherein each pair of file marks denotes a highlight. Then, the at least one received pair of file-marks are stored in an index-table, in a storage unit, by the processing unit configured in the electronic device.
  • the processing unit receives one or more data elements for associating with the at least one of stored file-marks, wherein the one or more data elements are selected from at least one of pre-defined keywords with associated values, predefines tags and custom tags, and stores the associated data elements along with references to corresponding file -marks in an information-table in the storage unit.
  • the electronic device further comprises an editor unit configured to acquire a multimedia file in at least one of real time mode and non-real time mode and play any highlights in the acquired multimedia file without traversing the non- highlighted areas. The editor unit acquires the multimedia file in real-time from a linked recording unit.
  • Fig, 1 illustrates a networking system in accordance with an embodiment of the present disclosure
  • Fig. 2 illustrates a block diagram of an electronic device in accordance with an embodiment of the present disclosure
  • Fig, 3.4, 3B A ' d 3C illustrate various relationship structures between multimedia files and metadata files in accordance with an embodiment of the present disclosure
  • Fig, 4 illustrates an exemplar ⁇ ? screenshot of editor unit in accordance with an embodiment of the present disclosure
  • Fig. 5 illustrates a flow-chart showing a method for marking highlights in a multimedia file in accordance with an embodiment of the present disclosure
  • Fig, 6A, 6B arad 6C illustrates a flow-chart showing working of the editor unit in accordance with an embodiment of the present disclosure
  • Fig, 7 illustrates a methodology for automated creation of highlights in real-time using an external system in accord ance with an embodiment of the present disclosure.
  • Fig, 1 illustrates a networking system 100 in accordance with an embodiment of the present disclosure.
  • the networking system 100 comprises one or more electronic devices 102 connected to a network 104. Further, the networking system comprises one or more servers 106 connected to the network 104.
  • the server comprises of a processing unit and a storage unit (not shown in Fig.).
  • the networking system 100 can be a linked social networking system, wherein one or more users are connected to each other using the electronic devices 102 and connected to the network 104.
  • the electronic devices can be used as stand-alone devices.
  • the electronic devices 102 may include, but are not limiting to, cameras, video cameras, mobile phones, tablets, Personal Digital Assistants (PDAs), laptops, desktops and any other electronic device capable of processing data and displaying.
  • the network 104 may include, bu t is not limited to Internet, Wi-Fi, Local Area Network (LAN) and other wired and wireless communication networks.
  • Fig, 2 illustrates a block diagram of an electronic device 102 in accordance with an embodiment of the present disclosure.
  • the electronic device 102 comprises an editor unit 200, a recording unit 202, a processing unit ' 204 and a storage unit 206.
  • the electronic device 102 is used to mark and play highlights in a multimedia file.
  • the multimedia files used with the electronic devices 102 of the present disclosure includes, but are not limited to, video files, audio files, image files, eBooks, slide-shows, medical recordings like ECG, Echocardiogram, EEG, MRI-Scan, CT-Scan, PET-Scan, Nuclear-Scan, MEG-Scan, MRS- Scan, ultrasound, etc., and other forms of electronic documents, including simple text files.
  • the editor unit 200 of the electronic device 102 provides an user interface to play a multimedia file and mark highlights in the multimedia file.
  • the editor unit 200 acquires the multimedia file in at least one of real time mode and non-real time mode.
  • real-time mode the editor unit acquires the multimedia file from the recording unit 202, while the multimedia file is being simultaneously captured by the recording unit 202.
  • non real- time mode the editor unit 200 acquires the multimedia files from either one of storage unit 206 of the electronic device 102 or an external storage device of any server 106 through the network 104.
  • the various components of the electronic device could be available on different physical devices connected over the network 104, instead of being on one single physical device. It is also possible, that there can be partial implementations consisting of only processing unit and storage unit with partial editor (for e.g. implementing only search or play features) and optional recorder unit.
  • the storage unit 206 of the electronic device 102 is configured to store the multimedia files, highlights created for the multimedia files and other related information.
  • a highlight of the multimedia file comprises a pair of file-marks.
  • Each pair of the file- marks comprises at least one of a begin-file-mark and an end-file-mark, corresponding to predetermined locations of the multimedia file.
  • the storage unit 206 stores at least one pair of file-marks in an index table configured within the storage unit 206.
  • the begin- file-mark and end-file-mark occurs in pairs.
  • the begin-mark denotes the beginning of a highlight and the end-file-mark denotes the end of a highlight.
  • begin-file-mark When a begin-file-mark occurs without a corresponding end-file-mark, the end-file-mark is assumed to be the end of the multimedia file. Similarly, when an end-file-mark is added without a corresponding begin-file-mark, begin of the multimedia file is assumed to be the begin-file-mark.
  • the storage unit stores one or more data elements related to file-marks that include, but not limited to, predefined keywords with associated values, predefined tags, and custom tags in an information-table.
  • the collection of such data elements is also referred to as metadata.
  • the predefined keywords can be chosen from a list and accepts a user specified value for the chosen keyword.
  • keywords could comprise words like Description, Location, Actors, Occasion, etc., which can be linked to a user specified value.
  • the system can automatically input the keywords.
  • the list of keywords provided by the system can be a function of the category of the multimedia file.
  • keywords metadata is actually a name-value pair of entries.
  • the tags on the other hand are single values. Tags can be of two types, system predefined or custom tags.
  • System predefined tags are available as an enumerated list that a user can choose from.
  • Custom tags on the other hand are user entered values separated by commas.
  • the list of system predefined tags can be a function of the category of the multimedia file.
  • the file -marks need not be associated to any metadata at all. In such a case, there will be no information-tables present.
  • the storage unit also stores relationship between the multimedia file, corresponding index-table(s) and information table(s) in a linkage table.
  • the linkage table can store many kinds of relationships between the multimedia files, the index-tables and the information tables, based on user configurations and system configurations.
  • Fig.3A, 3B and 3C illustrate few exemplary embodiments of the relationship types.
  • one multimedia file can be linked with multiple index-tables owned by different users, where each index table comprises one or more pair of file -marks. Each such index-table is used to store user's personal highlights of the multimedia file.
  • Each index-table in turn is connected to one or more information tables again owned by one or more users.
  • the information table contains information on the one or more file-marks stored in the related index-table.
  • one multimedia file can be linked with multiple index-tables, owned by multiple users, storing one or more file-marks. Each index-table is then linked with a single information-table which contains information associated with the one or more file-marks of the respective index-table linked therewith. In such an embodiment, typically the user owning the information-table would be the same as the owner of the index-table.
  • one multimedia file can be linked with only one index-table which may store one or more pair of file-marks for the linked multimedia file.
  • the single shared index-file may, in turn be connected to multiple information-tables belonging to different users, wherein each user can store personal comments and information regarding the highlights in the single shared index-table.
  • the index- table could be fixed to mark important points, highlights and sections denoting different topics. The students can however be allowed to make their own personal notes at any of these file-marks in their respective information-tables. Thus, each student would have a personal information-table linked to the single shared index-table. Refer to Fig. 3C for such a configuration.
  • one or more students share an information-table, or a single user has multiple information tables.
  • all the index files could belong to one user.
  • one index table could belong to multiple users or each user can have one or more index tables associated with them.
  • the information tables can be shared between users and one user may be associated with one or more information tables.
  • the exact configuration supported by any embodiment is based on access rights provided to the user for accessing the multimedia files, index-tables and the information-tables.
  • Separate privileges can be defined for creation, editing and accessing of index-tables and information-tables for a given multimedia file. Users can be granted one or more of these privileges based on various considerations such as website or system policy, access rights of the multimedia files as defined by publisher or author, role of the user, relation of the user to the author (e.g. in a social networking environment, only author's connections could be granted privilege to edit their index-table), etc.
  • the entries of an index-table and corresponding information-tables could be merged into a single table.
  • each entry is optionally qualified by the user the entry belongs to.
  • the logical relationship between the multimedia files and the related one-or-more index-tables and information-tables are stored in a linkage-table by the processing unit.
  • index-table that is independent of its logical representation
  • the index-table can be stored as a separate physical file (referred to as 'Index-file') or it can be stored as part of the multimedia file itself.
  • information linked to file-marks referred to as 'information-table' could have various physical representations independent of its logical representation.
  • Its physical representation could again be a function of the device it is on, and could be physically a separate file (referred to as 'Information- file') or could reside in the same physical file as its index-table or multimedia file.
  • the linkage-table could have multiple physical representations for the same logical representation.
  • index-tables, information-tables and linkage -tables could take forms such as database- table ⁇ ), xml-file(s), flat text file(s), etc. and this form could be based on the device it is residing on. Additionally, methods to convert the physical representation of the index, information and linkage tables along with their multimedia files, from one form to another, are supported. This is especially relevant, when the files are moved from one type of device to another.
  • the physical representation can also be morphed to suit the formats supported by the multimedia-player on a given device.
  • the processing unit 204 of the electronic device 102 is communicatively coupled to the editor unit 200, recording unit 202, and storage unit 206.
  • the processing unit 204 receives at least one pair of file-marks and marks highlights accordingly by storing the at least one pair of file-marks in the storage unit 206 as index-table along with related data as information-table and linking these index and information tables to one or more users.
  • Fig.4 illustrates an exemplary sereenshot of editor unit 200 in accordance with an embodiment of the present disclosure.
  • the editor unit 200 is an application programming interface which plays multimedia files and also allows users to mark highlights, as well as modify highlights in the multimedia file.
  • the editor unit 200 comprises a display screen 400, an index-panel 402, a progress bar with indicators 404, and one or more options 406.
  • the display 400 is configured to display or play the multimedia file for viewing by the user, in an exemplary embodiment, the one or more options 406 may be configured as one or more of buttons, hyperlinks, tabs or any other User Interface (UI) controls.
  • UI User Interface
  • the one or more options 406 are configured to perform operations including, but not limiting to, rewind to previous begin-file-mark, forward to next begin-file-mark, play/pause the highlights only, play/pause the complete multimedia file, fast-rewind and fast-forward.
  • the one or more options 406 are also configured to perform operations like creating a new index-table, adding new file-marks and editing information-table associated with the index-table displayed in the index-panel 402, etc.
  • the progress bar with indicators displays the current position of the multimedia file.
  • the index-panel 402 is used to view and edit the index-tables and information- tables.
  • the index-panel 402 comprises a select index box 408, search panel 410, and index-table display 412.
  • the select index box 408 displays the list of index-tables linked to the multimedia file and allows selection of a sub-set of index-tables to be opened.
  • a user can view only those index-tables to which the user has access rights, in another embodiment, the user can first select a set of users whose index-tables are of interest, based on profile attributes of the users. The system then displays index-tables belonging to the selected users for opening. The opened index-tables can then be used for playing the multimedia file highlights and can also be edited using features detailed below. Also, the select index box 408 provides an option to add new index-tablets).
  • the index-table display 412 displays the file-marks with metadata relating to the selected index-table(s) and selected informatioii-tabie(s).
  • the display can be facilitated in many different ways (not detailed in the figure).
  • the user can be provided with a choice of sorting options (e.g. sort by user, specific keyword values, chronological order, etc.), display locations and display styles.
  • the index-table display 412 also provides option to delete a file-mark and edit information stored at a file-mark in the information ⁇ table(s). When user selects the 'Edit Information' option against a file-mark, the Edit-information box 414 is displayed to the user for editing information associated with the specific file-mark.
  • the search panel 410 provides an option to search for file-marks.
  • the search can be conducted on keywords and tags associated at multimedia file level and at file-mark level of highlights.
  • the search results retrieved contain matching highlights in the selected index-tables based on access rights of a user.
  • a search can be performed on a single multimedia file for specific file-marks.
  • the search string input is used to match values stored in keywords or tags at various file-marks, and all file-marks containing the search string are retrieved and displayed.
  • the 'View Search' option provides an uninterrupted play/display for all retrieved highlights pertaining to matched file-marks in the search results.
  • an advance search can be provided using the 'Advanced Search' option where user can search for specific file -marks or for values in specific keywords.
  • the values in specific keywords can be provided to the user as a drop-down list. For example, a holiday video of city Rome could be searched for sections on 'Trevi fountain' by using search on keyword 'location' to contain the string 'Trevi fountain'.
  • a single search operation can be used to search multiple multimedia files for matching search criteria.
  • the resulting search results containing a set of multimedia files, where each file in turn can contain matching set of highlights, are made available to the user as a single play-list.
  • the entire multimedia file is assumed to be relevant to the search criteria defined.
  • the user has the option to play/watch the play-list uninterruptedly with a single click operation.
  • transition between videos or highlights of videos can be indicated with video title or a designated keyword-value at the file -mark as the case may be, if available.
  • the search panel also provides an option to share play-lists with other users and store play-lists for playing or watching the play-lists later (not shown in the figure).
  • Fig, 5 illustrates a flow-chart showing a method for marking highlights in a multimedia file in accordance with an embodiment of the present disclosure.
  • Marking of highlights in a multimedia file can be done in single user environment or a multiuser environment.
  • the method comprises receiving at least one pair of file-marks comprising a begin- file-mark and an end- file-mark at step 502.
  • the file-marks can be provided manually or by an automated arrangement.
  • the at least one pair of file-marks corresponds to predetermined locations in the multimedia file.
  • the user uses the editor unit 200 for creating highlights.
  • the user inputs the begin-file- mark and the end-file-mark for the multimedia file being played, by selecting the current position on the progress bar as either begin-file-mark or end-file-mark, using button 'Add File-mark'.
  • the at least one received pair of file-marks is stored in an index-table configured within the storage unit 206 by the processing unit 204 configured in the electronic device 102 at step 504.
  • the user can select to store the file -mark in an existing index-table.
  • the user can create a new index-table and store the created file-mark in the new index-table.
  • one or more data elements are received by the processing unit for associating with the at least one pair of stored file- marks.
  • the one or more data elements are selected from at least one of pre-defined keywords with associated values, predefines tags and custom tags.
  • the associated data elements are stored, along with references to corresponding pair of file-marks, in an information-table in the storage unit 206 by the processing unit 204 at step 508.
  • Typical implementations have data elements attached to a highlight.
  • the data elements can be attached to single file-marks instead of highlights which is a pair of begin and end file -marks.
  • different data elements can be attached to the begin-file-mark and end-file-mark of the same highlight.
  • Fig.6 A, 6B and 6C illustrates flow-chart showing working of the editor unit in accordance with an embodiment of the present disclosure.
  • the user first selects a multimedia file to play at step 6 ⁇ 2.
  • the multimedia file can be selected from the storage unit 206 of the electronic device 102 being used by the user.
  • the multimedia file can be accessed from external storage devices like storage unit of server 106 connected through the network 104.
  • the index-tables are selected by the users at step 604.
  • FIG.6B for illustration on the method of selecting index-table(s). Firstly, the users whose index-tables are to be used are selected at step 616.
  • the selection of users can be made based on their profile attributes such as interests, hobbies, profession, etc., so that more relevant highlights and metadata are chosen.
  • the users can also come from social network connections in a linked social networking website.
  • the available index-tables for the multimedia file are listed at step 618 using the linkage-table, wherein the listing is further based on access privileges of the logged-in user and access rights of index-tables.
  • the index-tables that belong to the users selected in step 616 are flagged for easier identification at step 620, Finally, the list of the selected index-tables is recorded at step 622.
  • the method determines if multiple index-tables are selected at step 606. If multiple index-tables are selected, then all the index-table entries are merged in a predefined order, with entries qualified by the user-id at step 608.
  • the predefined order can be chronological order or based on file-position of begin-file-mark. However, a person skilled in the art would understand that any other predefined or random order can be used with the method of the present disclosure.
  • all the file-marks corresponding to the index-table entries are displayed at step 610. In the alternative, if only one index-table is selected, then file-marks corresponding to the single index-table are displayed directly at step 610.
  • the display of file-marks happens on both the index-panel 402 as well as on the progress bar 404.
  • user can be given an option to select a subset of file-marks from index-panel that should be displayed on the progress bar.
  • the metadata for the file-marks are displayed on the index-panel and/or any other predefined location. Referring to Fig.6 €, the method of displaying metadata for file-marks is illustrated.
  • the selected index-tables are checked to determine if they are linked to multiple information- tables at step 624. If yes, then the user is provided with an option to select one or more information-tables that interests the user at step 626.
  • the process of selecting information-tables is split into multiple steps consisting of a) selecting users whose information-tables are of interest, b) listing information-tables linked to a given index- table, after flagging the ones belonging to the selected users in step a, and c) recording the selection made by the users. Then, the information-tables selected by the user are combined so that the metadata from multiple information-tables is combined and qualified by user at step 628. Referring back to Fig, 6 A, the method waits for input from user to take further action at step 614. Further action could be selected by the user by selecting any of the options in section 406 of the editor unit in Fig.4.
  • the recording unit 202 of the electronic device 102 is configured with capability to receive requests from an external system to read current file position during live recording. This current position is then transmitted to the external system to create a file-mark for the multimedia file being recorded.
  • Fig.7 illustrated in Fig.7 is an embodiment, where the electronic device is represented by a CCTV camera and is used to create automated highlights.
  • the recording-unit of the camera is capable of receiving a request for current file position in the video footage being recorded. In response to such a request, the recording unit responds with current file-position corresponding to the time the request was received.
  • an Automatic Teller Machine ATM
  • ATM Automatic Teller Machine
  • the present disclosure can be used in super-markets, banks, and other such places.
  • Current position request can be triggered by many other kinds of events by external systems. For e.g. automatic door to enter a secure building or a motion-sensor could trigger a current position request to create file-mark in the CCTV footage, every time the door is opened or motion is sensed.
  • the editor unit when a live video is being viewed by a user, the editor unit
  • the 200 of camera can be used as the user interface to create file-marks for current position in the live video.
  • the reference-able data for such a file-mark can be keyed in by the user watching the video. For e.g. a live football match recording could be watched live by a user, who can add file-marks every time there is goal with additional reference-able data such as goal-scorer details to keywords or custom tags linked to the added file-mark(s).
  • a live video editor unit 200 there can be two progress-bars and two current positions. If the viewer is rewinding to watch older footage, one progress bar can indicate the progress of live shoot, and the other progress bar can indicate the current viewing position in the video.
  • the settings can be configurable so as to set which current position is to be used for 'Add file-mark' option.
  • the file-marks can be qualified with a category, and each category of file-marks can be used differently or have different access rights based on user type or user role.
  • specific keywords attached to file-marks can denote file- marks for specific purposes.
  • ⁇ -marks for advertisement-insertion can belong to a special category, made non-editable for normal viewers.
  • File-marks may mark scenes/chapters in a single video file. Such file-marks could include special tags/keywords to store additional searchable information about the video section such as cast, description, location, etc.
  • File-marks for static image extraction may be used as thumb-nail or still images for a given video.
  • the same multimedia file can be marked with highlights and metadata by multiple users and the user playing the multimedia file can be given the choice of combining one or more users' highlights. This facilitates crowd- sourcing of highlights and metadata. Also, more relevant highlights can be accessed by selecting highlights from appropriate users as defined by their profile attributes. For e.g. a holiday video can be marked for highlights and metadata by a photographer and a historian. If the user watching the video is interested in photography, he might find more relevant highlights by choosing the photographer's index and information tables.
  • users are able to play multimedia files based on net- sourced highlights, i.e. highlights and annotations created by their social connections like friends and family members. This can have greater appeal, especially in the entertainment domain.
  • the current disclosure when applied to medical recordings like ECG, echo- cardiograph, EEG, MRI-Scan, CT-Scan, PET-Scan, Nuclear-Scan, MEG-Scan, MRS- Scan, ultrasound, etc. can be used to highlight abnormal sections, or sections that need to be focused on.
  • Such highlighting can be used very effectively in tele-radiology or tele- medicine, where the records are referred to remotely located radiologists and doctors for analysis or diagnosis. It can also be used to save time, when such medical records are being referred to multiple doctors for their respective opinions.
  • remarks and opinions from one or more doctors can be recorded as metadata using index-tables and information-tables, and the recorded information can be shared among a group of doctors observing a specific case.
  • the present disclosure facilitates precision in referencing multimedia files saving time and cost.
  • the present disclosure can be used in business conference by highlighting videos or other forms of multimedia files for topics of discussion. Another use is sports or event videos with highlights. Similarly online video tutorials can be file-marked for chapters, topics and concepts.

Abstract

The present disclosure relates to a method and a system for creating one or more highlights in a multimedia file with the help of file-marks. In an embodiment, the present disclosure provides a method to link searchable keywords and tags to each of the file- marks. Each user of the multimedia file can define an index of file-marks with associated keywords and tags. The user of the multimedia file is provided with an option to play highlights created by a chosen set of users, who could be from the network connections of the user.

Description

A METHOD FOR MARKING HIGHLIGHTS IN A MULTIMEDIA FILE AND AN
ELECTRONIC DEVICE THEREOF
TECHNICAL FIELD
The present disclosure relates generally to marking highlights in multimedia files. More specifically, the present disclosure relates to marking searchable highlights in multimedia files by associating the highlights with keywords and tags thereby improving the speed and precision of accessing relevant sections in a multimedia file.
BACKGROUND
In the recent times, users have access to abundant multimedia content over the internet and through other channels such as television, radio, mobile devices, Digital Versatile Disc (D VDs), etc. This results in the user having to spend a lot of time filtering the content of interest. Also, searching through the abundant multimedia content to find the snippets that interest the user is very difficult and laborious. So, users would be benefited if the multimedia content were to be marked for highlights so as to enable users to precisely access only the relevant sections, instead of having to watch or listen to the complete file contents.
One way to index the multimedia files is to automate the marking of highlights in the files. Although automation of marking highlights is desirable, it yields limited results due to the incapability of automated programs to understand the rich semantics of the content, leading to manual method being the ideal choice. In addition, automation is extremely processor intensive, making it a difficult choice in the new-age mobile devices which are increasingly becoming the channel of consumption for multimedia files. However, manual browsing and marking multimedia files for highlights is a very time consuming task, especially due to the abundance of their availability. Hence it would be beneficial to users, if they could share the task of identifying highlights and metadata for multimedia files with other users. Such a collaborative effort would be additionally relevant when done by users with similar interests, hobbies, profession, etc., or for example by related members in a social networking environment. In order to facilitate effective searching and filtering of multimedia files for relevant content while retaining the benefits of manual identification of highlights and relevant metadata where necessary, the present disclosure presents a system and associated methods to facilitate processor-efficient marking and playing of highlights in multimedia files and also to qualify them with relevant metadata to enable search within the multimedia files,
SUMMARY OF THE DISCLOSURE
The shortcomings of the prior art are overcome and many additional advantages are provided through the present disclosure. Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the disclosure are described in detail herein and are considered a part of the claimed disclosure.
The present disclosure relates to a method and an electronic device to facilitate marking of highlights in multimedia files and to qualify the marked highlights with relevant metadata to enable search within the multimedia files. The present disclosure facilitates marking of highlights in a multimedia file using either manual or automated approaches. Different approaches are suitable under different usage scenarios as detailed in the description below.
The methods described here -in further facilitate, crowd-sourcing of the effort of marking highlights and identifying metadata, and automating the process of combining the crowd-sourced work in a manner that is relevant to the user. In one embodiment, the present disclosure relates to a method of marking highlights in a multimedia file using an electronic device. The method comprises receiving at least one pair of file-marks comprising a begin-file-mark and an end-file-mark corresponding to predetermined locations in the multimedia file, wherein each pair of file marks denotes a highlight and then, storing the at least one received pair of file -marks in an index-table, in a storage unit, by a processing unit configured in the electronic device. Further, the method comprises receiving one or more data elements by the processing unit for associating with the at least one of stored file-marks. The one or more data elements are selected from at least one of pre-defined keywords with associated values, predefined tags and custom tags. Finally, the associated data elements along with references to corresponding file-marks are stored in an information-table in the storage unit by the processing unit.
In one embodiment, the present disclosure relates to an electronic device for marking highlights on a multimedia file. The electronic device comprises a processing unit coupled with a storage unit. The processing unit is configured to receive at least one pair of file-marks comprising a begin-file-mark and an end-file-mark corresponding to predetermined locations in the multimedia file, wherein each pair of file marks denotes a highlight. Then, the at least one received pair of file-marks are stored in an index-table, in a storage unit, by the processing unit configured in the electronic device. Further, the processing unit receives one or more data elements for associating with the at least one of stored file-marks, wherein the one or more data elements are selected from at least one of pre-defined keywords with associated values, predefines tags and custom tags, and stores the associated data elements along with references to corresponding file -marks in an information-table in the storage unit. The electronic device further comprises an editor unit configured to acquire a multimedia file in at least one of real time mode and non-real time mode and play any highlights in the acquired multimedia file without traversing the non- highlighted areas. The editor unit acquires the multimedia file in real-time from a linked recording unit.
The aforementioned and other features and advantages of the disclosure will become further apparent from the following detailed description of the presently preferred embodiments, read in conjunction with the accompanying drawings. The detailed description and drawings are merely illustrative of the disclosure rather than limiting, the scope of the disclosure being defined by the appended claims and equivalents thereof.
BRIEF DESCRIPTION OF THE DRAWINGS
The features of the present disclosure are set forth with particularity in the appended claims. The embodiments of the disclosure itself, together with further features and related advantages, will become apparent from consideration of the following detailed description, taken in conjunction with the accompanying drawings. One or more embodiments of the present disclosure are now described, by way of example only, with reference to the accompanied drawings wherein like reference numerals represent like elements and in which: Fig, 1 illustrates a networking system in accordance with an embodiment of the present disclosure;
Fig. 2 illustrates a block diagram of an electronic device in accordance with an embodiment of the present disclosure;
Fig, 3.4, 3B A ' d 3C illustrate various relationship structures between multimedia files and metadata files in accordance with an embodiment of the present disclosure; Fig, 4 illustrates an exemplar}? screenshot of editor unit in accordance with an embodiment of the present disclosure;
Fig. 5 illustrates a flow-chart showing a method for marking highlights in a multimedia file in accordance with an embodiment of the present disclosure;
Fig, 6A, 6B arad 6C illustrates a flow-chart showing working of the editor unit in accordance with an embodiment of the present disclosure; and
Fig, 7 illustrates a methodology for automated creation of highlights in real-time using an external system in accord ance with an embodiment of the present disclosure.
The figures depict embodiments of the disclosure for purposes of il lustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.
DETAILED DESCRIPTION
The foregoing has broadly outlined the features and technical advantages of the present disclosure in order that the detailed description of the disclosure that follows may be better understood. Additional features and advantages of the disclosure will be described hereinafter which form the subject of the claims of the disclosure. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the disclosure as set forth in the appended claims. The novel features which are believed to be characteristic of the disclosure, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present disclosure.
Fig, 1 illustrates a networking system 100 in accordance with an embodiment of the present disclosure. The networking system 100 comprises one or more electronic devices 102 connected to a network 104. Further, the networking system comprises one or more servers 106 connected to the network 104. The server comprises of a processing unit and a storage unit (not shown in Fig.). In an embodiment, the networking system 100 can be a linked social networking system, wherein one or more users are connected to each other using the electronic devices 102 and connected to the network 104. In another embodiment, the electronic devices can be used as stand-alone devices. The electronic devices 102 may include, but are not limiting to, cameras, video cameras, mobile phones, tablets, Personal Digital Assistants (PDAs), laptops, desktops and any other electronic device capable of processing data and displaying. The network 104 may include, bu t is not limited to Internet, Wi-Fi, Local Area Network (LAN) and other wired and wireless communication networks.
Fig, 2 illustrates a block diagram of an electronic device 102 in accordance with an embodiment of the present disclosure. The electronic device 102 comprises an editor unit 200, a recording unit 202, a processing unit '204 and a storage unit 206. The electronic device 102 is used to mark and play highlights in a multimedia file. The multimedia files used with the electronic devices 102 of the present disclosure includes, but are not limited to, video files, audio files, image files, eBooks, slide-shows, medical recordings like ECG, Echocardiogram, EEG, MRI-Scan, CT-Scan, PET-Scan, Nuclear-Scan, MEG-Scan, MRS- Scan, ultrasound, etc., and other forms of electronic documents, including simple text files. The editor unit 200 of the electronic device 102 provides an user interface to play a multimedia file and mark highlights in the multimedia file. The editor unit 200 acquires the multimedia file in at least one of real time mode and non-real time mode. In real-time mode, the editor unit acquires the multimedia file from the recording unit 202, while the multimedia file is being simultaneously captured by the recording unit 202. In non real- time mode the editor unit 200 acquires the multimedia files from either one of storage unit 206 of the electronic device 102 or an external storage device of any server 106 through the network 104. In an exemplary embodiment, the various components of the electronic device could be available on different physical devices connected over the network 104, instead of being on one single physical device. It is also possible, that there can be partial implementations consisting of only processing unit and storage unit with partial editor (for e.g. implementing only search or play features) and optional recorder unit.
The storage unit 206 of the electronic device 102 is configured to store the multimedia files, highlights created for the multimedia files and other related information. A highlight of the multimedia file comprises a pair of file-marks. Each pair of the file- marks comprises at least one of a begin-file-mark and an end-file-mark, corresponding to predetermined locations of the multimedia file. Thus, the storage unit 206 stores at least one pair of file-marks in an index table configured within the storage unit 206. The begin- file-mark and end-file-mark occurs in pairs. The begin-mark denotes the beginning of a highlight and the end-file-mark denotes the end of a highlight. When a begin-file-mark occurs without a corresponding end-file-mark, the end-file-mark is assumed to be the end of the multimedia file. Similarly, when an end-file-mark is added without a corresponding begin-file-mark, begin of the multimedia file is assumed to be the begin-file-mark.
In addition, the storage unit stores one or more data elements related to file-marks that include, but not limited to, predefined keywords with associated values, predefined tags, and custom tags in an information-table. The collection of such data elements is also referred to as metadata. The predefined keywords can be chosen from a list and accepts a user specified value for the chosen keyword. For example, keywords could comprise words like Description, Location, Actors, Occasion, etc., which can be linked to a user specified value. In one embodiment, the system can automatically input the keywords. The list of keywords provided by the system can be a function of the category of the multimedia file. Thus keywords metadata is actually a name-value pair of entries. The tags on the other hand are single values. Tags can be of two types, system predefined or custom tags. System predefined tags are available as an enumerated list that a user can choose from. Custom tags, on the other hand are user entered values separated by commas. Here again, the list of system predefined tags can be a function of the category of the multimedia file. In one embodiment, the file -marks need not be associated to any metadata at all. In such a case, there will be no information-tables present.
The storage unit also stores relationship between the multimedia file, corresponding index-table(s) and information table(s) in a linkage table. The linkage table can store many kinds of relationships between the multimedia files, the index-tables and the information tables, based on user configurations and system configurations. Fig.3A, 3B and 3C illustrate few exemplary embodiments of the relationship types. In an exemplary embodiment, as illustrated in Fig.3A, one multimedia file can be linked with multiple index-tables owned by different users, where each index table comprises one or more pair of file -marks. Each such index-table is used to store user's personal highlights of the multimedia file. Each index-table in turn is connected to one or more information tables again owned by one or more users. The information table contains information on the one or more file-marks stored in the related index-table. In another exemplary embodiment, as illustrated in Fig.3B, one multimedia file can be linked with multiple index-tables, owned by multiple users, storing one or more file-marks. Each index-table is then linked with a single information-table which contains information associated with the one or more file-marks of the respective index-table linked therewith. In such an embodiment, typically the user owning the information-table would be the same as the owner of the index-table. In yet another exemplary embodiment, as illustrated in Fig.3C, one multimedia file can be linked with only one index-table which may store one or more pair of file-marks for the linked multimedia file. The single shared index-file may, in turn be connected to multiple information-tables belonging to different users, wherein each user can store personal comments and information regarding the highlights in the single shared index-table.
For example, in an embodiment where students can view video tutorials, the index- table could be fixed to mark important points, highlights and sections denoting different topics. The students can however be allowed to make their own personal notes at any of these file-marks in their respective information-tables. Thus, each student would have a personal information-table linked to the single shared index-table. Refer to Fig. 3C for such a configuration. Here, it is also possible that one or more students share an information-table, or a single user has multiple information tables. In the above exemplary embodiments, all the index files could belong to one user. Alternatively, one index table could belong to multiple users or each user can have one or more index tables associated with them. Similarly, in the above exemplary embodiments, the information tables can be shared between users and one user may be associated with one or more information tables. The exact configuration supported by any embodiment is based on access rights provided to the user for accessing the multimedia files, index-tables and the information-tables. Separate privileges can be defined for creation, editing and accessing of index-tables and information-tables for a given multimedia file. Users can be granted one or more of these privileges based on various considerations such as website or system policy, access rights of the multimedia files as defined by publisher or author, role of the user, relation of the user to the author (e.g. in a social networking environment, only author's connections could be granted privilege to edit their index-table), etc.
In another exemplary embodiment, the entries of an index-table and corresponding information-tables could be merged into a single table. In such a table, each entry is optionally qualified by the user the entry belongs to. It is also possible that there are no information-tables attached to the index-tables. In such a case, no information or metadata is attached to the highlights. The logical relationship between the multimedia files and the related one-or-more index-tables and information-tables are stored in a linkage-table by the processing unit.
The physical representation of an index-table (that is independent of its logical representation) could be device dependent, taking different forms on different devices. The index-table can be stored as a separate physical file (referred to as 'Index-file') or it can be stored as part of the multimedia file itself. Similarly, the information linked to file-marks, referred to as 'information-table' could have various physical representations independent of its logical representation. Its physical representation could again be a function of the device it is on, and could be physically a separate file (referred to as 'Information- file') or could reside in the same physical file as its index-table or multimedia file. Similarly, the linkage-table could have multiple physical representations for the same logical representation. When not part of the multimedia file itself, physical representation of index-tables, information-tables and linkage -tables could take forms such as database- table^), xml-file(s), flat text file(s), etc. and this form could be based on the device it is residing on. Additionally, methods to convert the physical representation of the index, information and linkage tables along with their multimedia files, from one form to another, are supported. This is especially relevant, when the files are moved from one type of device to another. The physical representation can also be morphed to suit the formats supported by the multimedia-player on a given device.
Further, the processing unit 204 of the electronic device 102 is communicatively coupled to the editor unit 200, recording unit 202, and storage unit 206. The processing unit 204 receives at least one pair of file-marks and marks highlights accordingly by storing the at least one pair of file-marks in the storage unit 206 as index-table along with related data as information-table and linking these index and information tables to one or more users.
Fig.4 illustrates an exemplary sereenshot of editor unit 200 in accordance with an embodiment of the present disclosure. The editor unit 200 is an application programming interface which plays multimedia files and also allows users to mark highlights, as well as modify highlights in the multimedia file. The editor unit 200 comprises a display screen 400, an index-panel 402, a progress bar with indicators 404, and one or more options 406. The display 400 is configured to display or play the multimedia file for viewing by the user, in an exemplary embodiment, the one or more options 406 may be configured as one or more of buttons, hyperlinks, tabs or any other User Interface (UI) controls. The one or more options 406 are configured to perform operations including, but not limiting to, rewind to previous begin-file-mark, forward to next begin-file-mark, play/pause the highlights only, play/pause the complete multimedia file, fast-rewind and fast-forward. The one or more options 406 are also configured to perform operations like creating a new index-table, adding new file-marks and editing information-table associated with the index-table displayed in the index-panel 402, etc. The progress bar with indicators displays the current position of the multimedia file. Also, there are one or more indicators indicating the begin-file-marks (represented by Bl, B2 and B3 in Fig.4) and end-file- marks (represented by El and E2 in Fig.4) of all the searched or selected highlights from selected index-tables of the multimedia file. The highlights denoted by the file-marks can be represented in many different ways, and can include differently coloured bars or arrows. These indicators can be overlaid on the progress-bar too. Further, the file-marks can be moved and modified by dragging the file-marks on the progress-bar. The index-panel 402 is used to view and edit the index-tables and information- tables. The index-panel 402 comprises a select index box 408, search panel 410, and index-table display 412. The select index box 408 displays the list of index-tables linked to the multimedia file and allows selection of a sub-set of index-tables to be opened. In an embodiment, a user can view only those index-tables to which the user has access rights, in another embodiment, the user can first select a set of users whose index-tables are of interest, based on profile attributes of the users. The system then displays index-tables belonging to the selected users for opening. The opened index-tables can then be used for playing the multimedia file highlights and can also be edited using features detailed below. Also, the select index box 408 provides an option to add new index-tablets).
Once the index-tables are selected, the user is prompted to select information- tables for each index-table (not shown in the figure), if more than one information-tables are linked to an index-table. The index-table display 412 displays the file-marks with metadata relating to the selected index-table(s) and selected informatioii-tabie(s). The display can be facilitated in many different ways (not detailed in the figure). The user can be provided with a choice of sorting options (e.g. sort by user, specific keyword values, chronological order, etc.), display locations and display styles. The index-table display 412 also provides option to delete a file-mark and edit information stored at a file-mark in the information~table(s). When user selects the 'Edit Information' option against a file-mark, the Edit-information box 414 is displayed to the user for editing information associated with the specific file-mark.
The search panel 410, provides an option to search for file-marks. The search can be conducted on keywords and tags associated at multimedia file level and at file-mark level of highlights. The search results retrieved contain matching highlights in the selected index-tables based on access rights of a user. In an embodiment, a search can be performed on a single multimedia file for specific file-marks. The search string input is used to match values stored in keywords or tags at various file-marks, and all file-marks containing the search string are retrieved and displayed. The 'View Search' option provides an uninterrupted play/display for all retrieved highlights pertaining to matched file-marks in the search results. In one embodiment, an advance search can be provided using the 'Advanced Search' option where user can search for specific file -marks or for values in specific keywords. In an example, the values in specific keywords can be provided to the user as a drop-down list. For example, a holiday video of city Rome could be searched for sections on 'Trevi fountain' by using search on keyword 'location' to contain the string 'Trevi fountain'.
In an exemplary embodiment, a single search operation can be used to search multiple multimedia files for matching search criteria. The resulting search results containing a set of multimedia files, where each file in turn can contain matching set of highlights, are made available to the user as a single play-list. In cases, where the search criteria is matched at file level and no matches were found at file-mark level, the entire multimedia file is assumed to be relevant to the search criteria defined. The user has the option to play/watch the play-list uninterruptedly with a single click operation. In case of video files, transition between videos or highlights of videos can be indicated with video title or a designated keyword-value at the file -mark as the case may be, if available. The search panel also provides an option to share play-lists with other users and store play-lists for playing or watching the play-lists later (not shown in the figure).
Fig, 5 illustrates a flow-chart showing a method for marking highlights in a multimedia file in accordance with an embodiment of the present disclosure. Marking of highlights in a multimedia file can be done in single user environment or a multiuser environment. The method comprises receiving at least one pair of file-marks comprising a begin- file-mark and an end- file-mark at step 502. The file-marks can be provided manually or by an automated arrangement. The at least one pair of file-marks corresponds to predetermined locations in the multimedia file. In an embodiment, the user uses the editor unit 200 for creating highlights. As depicted in Fig 4, the user inputs the begin-file- mark and the end-file-mark for the multimedia file being played, by selecting the current position on the progress bar as either begin-file-mark or end-file-mark, using button 'Add File-mark'. Then, the at least one received pair of file-marks is stored in an index-table configured within the storage unit 206 by the processing unit 204 configured in the electronic device 102 at step 504. The user can select to store the file -mark in an existing index-table. In the alternative, the user can create a new index-table and store the created file-mark in the new index-table. Then, at step 506, one or more data elements are received by the processing unit for associating with the at least one pair of stored file- marks. The one or more data elements are selected from at least one of pre-defined keywords with associated values, predefines tags and custom tags. The associated data elements are stored, along with references to corresponding pair of file-marks, in an information-table in the storage unit 206 by the processing unit 204 at step 508. Typical implementations have data elements attached to a highlight. However, in an embodiment, the data elements can be attached to single file-marks instead of highlights which is a pair of begin and end file -marks. In such an embodiment, different data elements can be attached to the begin-file-mark and end-file-mark of the same highlight.
Fig.6 A, 6B and 6C illustrates flow-chart showing working of the editor unit in accordance with an embodiment of the present disclosure. In an embodiment illustrated in Fig.6A, the user first selects a multimedia file to play at step 6Θ2. The multimedia file can be selected from the storage unit 206 of the electronic device 102 being used by the user. In the alternative, the multimedia file can be accessed from external storage devices like storage unit of server 106 connected through the network 104. Then, the index-tables are selected by the users at step 604. Refer to Fig.6B, for illustration on the method of selecting index-table(s). Firstly, the users whose index-tables are to be used are selected at step 616. The selection of users can be made based on their profile attributes such as interests, hobbies, profession, etc., so that more relevant highlights and metadata are chosen. The users can also come from social network connections in a linked social networking website. Then, ail the available index-tables for the multimedia file are listed at step 618 using the linkage-table, wherein the listing is further based on access privileges of the logged-in user and access rights of index-tables. Then, the index-tables that belong to the users selected in step 616 are flagged for easier identification at step 620, Finally, the list of the selected index-tables is recorded at step 622.
Referring back to Fig.6A, once the index-tables are selected, then the method determines if multiple index-tables are selected at step 606. If multiple index-tables are selected, then all the index-table entries are merged in a predefined order, with entries qualified by the user-id at step 608. In an embodiment, the predefined order can be chronological order or based on file-position of begin-file-mark. However, a person skilled in the art would understand that any other predefined or random order can be used with the method of the present disclosure. Then, all the file-marks corresponding to the index-table entries are displayed at step 610. In the alternative, if only one index-table is selected, then file-marks corresponding to the single index-table are displayed directly at step 610. The display of file-marks happens on both the index-panel 402 as well as on the progress bar 404. In one embodiment, user can be given an option to select a subset of file-marks from index-panel that should be displayed on the progress bar. At step 612, the metadata for the file-marks are displayed on the index-panel and/or any other predefined location. Referring to Fig.6€, the method of displaying metadata for file-marks is illustrated. The selected index-tables are checked to determine if they are linked to multiple information- tables at step 624. If yes, then the user is provided with an option to select one or more information-tables that interests the user at step 626. Here again, the process of selecting information-tables is split into multiple steps consisting of a) selecting users whose information-tables are of interest, b) listing information-tables linked to a given index- table, after flagging the ones belonging to the selected users in step a, and c) recording the selection made by the users. Then, the information-tables selected by the user are combined so that the metadata from multiple information-tables is combined and qualified by user at step 628. Referring back to Fig, 6 A, the method waits for input from user to take further action at step 614. Further action could be selected by the user by selecting any of the options in section 406 of the editor unit in Fig.4.
Described below is a methodology for automated creation of highlights in real-time using an external system in accordance with an embodiment of the present disclosure. The recording unit 202 of the electronic device 102 is configured with capability to receive requests from an external system to read current file position during live recording. This current position is then transmitted to the external system to create a file-mark for the multimedia file being recorded. For e.g., illustrated in Fig.7 is an embodiment, where the electronic device is represented by a CCTV camera and is used to create automated highlights. The recording-unit of the camera is capable of receiving a request for current file position in the video footage being recorded. In response to such a request, the recording unit responds with current file-position corresponding to the time the request was received. This can be used by the external system to create a file-mark in its index- table for the file being recorded, and attach the file-mark with relevant data required and store the data in an external storage device. For e.g. an Automatic Teller Machine (ATM) could be the external system that requests for current file position from a CCTV camera in the vicinity, every time cash is drawn from the machine. Then the current file position is saved as a file-mark with transaction-id as linked information in the index-table and information-table for the CCTV footage. This enables quick and accurate reference to CCTV footage relevant to any specific transaction.
The present disclosure can be used in super-markets, banks, and other such places. Current position request can be triggered by many other kinds of events by external systems. For e.g. automatic door to enter a secure building or a motion-sensor could trigger a current position request to create file-mark in the CCTV footage, every time the door is opened or motion is sensed. In another embodiment, when a live video is being viewed by a user, the editor unit
200 of camera can be used as the user interface to create file-marks for current position in the live video. The reference-able data for such a file-mark can be keyed in by the user watching the video. For e.g. a live football match recording could be watched live by a user, who can add file-marks every time there is goal with additional reference-able data such as goal-scorer details to keywords or custom tags linked to the added file-mark(s).
In one embodiment, in a live video editor unit 200, there can be two progress-bars and two current positions. If the viewer is rewinding to watch older footage, one progress bar can indicate the progress of live shoot, and the other progress bar can indicate the current viewing position in the video. The settings can be configurable so as to set which current position is to be used for 'Add file-mark' option.
In an embodiment, the file-marks can be qualified with a category, and each category of file-marks can be used differently or have different access rights based on user type or user role. Alternately, specific keywords attached to file-marks can denote file- marks for specific purposes. For example, ίϊίε-marks for advertisement-insertion can belong to a special category, made non-editable for normal viewers. File-marks may mark scenes/chapters in a single video file. Such file-marks could include special tags/keywords to store additional searchable information about the video section such as cast, description, location, etc. File-marks for static image extraction may be used as thumb-nail or still images for a given video.
In a multi-user environment, the same multimedia file can be marked with highlights and metadata by multiple users and the user playing the multimedia file can be given the choice of combining one or more users' highlights. This facilitates crowd- sourcing of highlights and metadata. Also, more relevant highlights can be accessed by selecting highlights from appropriate users as defined by their profile attributes. For e.g. a holiday video can be marked for highlights and metadata by a photographer and a historian. If the user watching the video is interested in photography, he might find more relevant highlights by choosing the photographer's index and information tables. In a social networking environment, users are able to play multimedia files based on net- sourced highlights, i.e. highlights and annotations created by their social connections like friends and family members. This can have greater appeal, especially in the entertainment domain.
The current disclosure when applied to medical recordings like ECG, echo- cardiograph, EEG, MRI-Scan, CT-Scan, PET-Scan, Nuclear-Scan, MEG-Scan, MRS- Scan, ultrasound, etc., can be used to highlight abnormal sections, or sections that need to be focused on. Such highlighting can be used very effectively in tele-radiology or tele- medicine, where the records are referred to remotely located radiologists and doctors for analysis or diagnosis. It can also be used to save time, when such medical records are being referred to multiple doctors for their respective opinions. In such a case, remarks and opinions from one or more doctors can be recorded as metadata using index-tables and information-tables, and the recorded information can be shared among a group of doctors observing a specific case.
The present disclosure facilitates precision in referencing multimedia files saving time and cost. The present disclosure can be used in business conference by highlighting videos or other forms of multimedia files for topics of discussion. Another use is sports or event videos with highlights. Similarly online video tutorials can be file-marked for chapters, topics and concepts.
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims. With respect to the use of substantially any plural and/or singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity.
Many variations of the invention and embodiments herein described will be apparent to people skilled in the art. For example, features of the different embodiments disclosed herein may be omitted, selected, combined or exchanged in order to form further embodiments. Again, where a preference or particularisation is stated, there is implicit the possibility of its negative, i.e. a case in which that preference or particularisation is absent. The invention is considered to extend to any new and inventive embodiments formed by said variations, further embodiments and cases, without deviation from scope of the invention.
The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
Referral Numerals:
Reference number Description
100 Networking System
102 Electronic Device
104 Network
106 Server
200 Editor Unit
202 Recording Unit
204 Processing Unit
206 Storage Unit
400 Display Screen
402 Index-panel
404 Progress Bar
406 Options

Claims

I claim:
1. A method of marking highlights in a multimedia file in an electronic device, the method comprising:
receiving at least one pair of file-marks comprising a begin-file-mark and an end-file-mark corresponding to predetermined locations in the multimedia file, wherein each pair of file marks denotes a highlight; and
storing the at least one received pair of file-marks in an index-table, in a storage unit, by a processing unit configured in the electronic device.
2. The method as claimed in claim 1, further comprising receiving one or more data elements by the processing unit for associating with the at least one stored file- marks, wherein the one or more data elements are selected from at least one of pre-defined keywords with associated values, predefined tags and custom tags; and storing the associated data elements along with reference to corresponding file-marks in information-table in the storage unit by the processing unit.
3. The method as claimed in claim 1, further comprising an editor unit in the electronic device capable of playing the marked highlights in the multimedia file without traversing non-highlighted sections of the multimedia file.
4. The method as claimed in claim3, wherein the editor unit, while playing the multimedia file, displays the data elements associated with one or more file marks at a preconfigured location on a display screen of the electronic device.
5. The method as claimed in claim 1, wherein one multimedia file is associated with one or more index-tables, and wherein the one or more index-tables belongs to one or more users having access to the multimedia file.
6. The method as claimed in claim 5, wherein one index-table is associated with one or more information-tables, wherein one or more information-tables belongs to one or more users having access to the multimedia file and corresponding index-table.
7. The method as claimed in claim 6, wherein relationship between a multimedia file and corresponding index-tables and information-tables is stored as linkage-tables in the storage unit, by the processing unit.
8. The method as claimed in claim 3, further comprising selecting the highlights to be played on the editor unit by the user, by selecting at least one index-table based on user access privileges and the access rights of index-table, wherein the processing unit displays the file-marks from the selected index-tables.
9. The method as claimed in claim 8, further comprising selecting at least one information-table associated with the index-table by the user based on user access privileges and access rights of information-tables, wherein the processing unit displays the data elements from the selected information-tables.
10. The method as claimed in claim 9, wherein the selection of index-tables and information-tables for a multimedia file is assisted by the processing unit by displaying the index-tables and information-tables belonging to selected users.
11. The method as claimed in claim 10, wherein the users are selected based on profile attributes of the users.
12. The method as claimed in claim 10, wherein the users are selected from connected users in a linked social networking system.
13. The method as claimed in claim 1, wherein upon determining the end- file-mark of a highlight corresponding to a begin-file-mark to be absent, the end-file-mark is marked as the end-of-file of the multimedia file.
14. The method as claimed in claim 1, wherein upon determining the begin-file-mark of a highlight corresponding to an end-file-mark to be absent, the begin-file-mark is marked as the begin-of-file of the multimedia file.
15. The method as claimed in claim 1, wherein the predetermined location in the multimedia file is defined by at least one of elapsed time from the begin of the multimedia file and number of bits from the begin of the multimedia file.
16. The method as claimed in claim 1, further comprising of modifying the index- table, wherein the modification is selected from at least one of addition, deletion, and editing of at least one of file-marks and associated data elements in the related information-table.
17. The method as claimed in claim 1, wherein the highlights are searchable using search strings matching with the data elements associated with at least one file- mark.
18. The method as claimed in claim 1, further comprising:
receiving a request from an external system for current file position of the multimedia file being recorded in real time mode by a recording unit in the electronic device;
transmitting the current file position corresponding to the time the request was received to the external system; and
enabling the external system to store the current file position of the multimedia file as at least one file-mark in the index-table along with relevant data elements as pre-defined keywords and tags associated with the stored at least one file-mark.
19. An electronic device for marking highlights on a multimedia file, comprising
a processing unit coupled with a storage unit, said processing unit is configured to:
receive at least one pair of file-marks comprising a begin-file-mark and an end-file-mark corresponding to predetermined locations in the multimedia file, wherein each pair of file marks denotes a highlight; and
store the at least one received pair of file-marks in an index-table, in the storage unit.
20. The electronic device as claimed in claim 19, wherein the processing unit is configured to:
receive the one or more data elements for associating with the at least one stored file -marks, wherein the one or more data elements are selected from at least one of pre-defined keywords with associated values, predefined tags and custom tags; and
store the associated data elements along with references to corresponding file-marks in information-table in the storage unit.
21. The electronic device as claimed in claim 19, further comprising an editor unit configured to play the multimedia file and mark highlights in the multimedia file, wherein the editor unit acquires the multimedia file in at least one of real time mode and non-real time mode, and wherein the editor unit acquires the multimedia file in real-time from a recording unit.
22. The electronic device as claimed in claim 19, wherein the storage unit is configured to store at least one of the multimedia files, highlights and one or more data elements.
23. The electronic device as claimed in claim 21, wherein the editor unit enables the user to edit, play and traverse through a multimedia file using at least one of file- marks, pre-defined keywords and tags.
24. The electronic device as claimed in claim 21, wherein the editor unit comprises:
a display screen to display the multimedia file being played, with a facility to send the current position of the multimedia file being played as the position for creating a file-mark in the index-table;
an index-panel to enable the users to view and search for specific file- marks and to display, add, edit or delete information-table data associated with the index-table; and
a progress bar with indicators, to indicate at least one of current position of the multimedia file being played and the highlights.
25. The electronic device as claimed in claim 24, wherein the editor unit comprises one or more options for performing operations including at least one of rewind to previous begin-file-mark, forward to next begin-file-mark, play the highlights/pause the highlights, play the multimedia file/pause the multimedia file, fast-rewind and fast-forward.
26. The electronic device as claimed in claim 24, wherein the editor unit comprises one or more options for performing operations including at least one of creating new index-table, adding new file-marks and editing information-table associated with the index-table displayed in the index-panel.
27. The electronic device as claimed in claim 21, wherein the recording unit is configured to:
receive a request from an external system for current file position of a multimedia file being recorded in real time mode;
transmit the current file position corresponding to the time the request was received to the external system; and
enable the external system to store the current file position of the multimedia file as at least one file-mark in the index-table along with relevant data elements as pre-defined keywords and tags associated with the at least one stored file-mark.
28. The electronic device as claimed in any of the preceding claims, wherein at least one of the editor unit, the recording unit and the storage unit are located in at least one of the electronic device and one or more external devices connected over at least one of wired network and wireless network.
PCT/IB2013/055202 2012-06-25 2013-06-25 A method for marking highlights in a multimedia file and an electronic device thereof WO2014002004A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB1501008.5A GB2518791A (en) 2012-06-25 2013-06-25 A method for marking highlights in a multimedia file and an electronic device there-of

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB1211181.1 2012-06-25
GBGB1211181.1A GB201211181D0 (en) 2012-05-14 2012-06-25 Camera device for videos and photos

Publications (1)

Publication Number Publication Date
WO2014002004A1 true WO2014002004A1 (en) 2014-01-03

Family

ID=48917585

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2013/055202 WO2014002004A1 (en) 2012-06-25 2013-06-25 A method for marking highlights in a multimedia file and an electronic device thereof

Country Status (2)

Country Link
GB (1) GB2518791A (en)
WO (1) WO2014002004A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160224672A1 (en) * 2015-01-29 2016-08-04 Facebook, Inc. Multimedia Search Using Reshare Text on Online Social Networks
EP3103263A4 (en) * 2014-02-07 2018-04-18 Music Media Limited Digital media content and associated user pool apparatus and method
CN113949920A (en) * 2021-12-20 2022-01-18 深圳佑驾创新科技有限公司 Video annotation method and device, terminal equipment and storage medium
CN115334354A (en) * 2022-08-15 2022-11-11 北京百度网讯科技有限公司 Video annotation method and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002008948A2 (en) * 2000-07-24 2002-01-31 Vivcom, Inc. System and method for indexing, searching, identifying, and editing portions of electronic multimedia files
US20060282776A1 (en) * 2005-06-10 2006-12-14 Farmer Larry C Multimedia and performance analysis tool
US20100153848A1 (en) * 2008-10-09 2010-06-17 Pinaki Saha Integrated branding, social bookmarking, and aggregation system for media content

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002008948A2 (en) * 2000-07-24 2002-01-31 Vivcom, Inc. System and method for indexing, searching, identifying, and editing portions of electronic multimedia files
US20060282776A1 (en) * 2005-06-10 2006-12-14 Farmer Larry C Multimedia and performance analysis tool
US20100153848A1 (en) * 2008-10-09 2010-06-17 Pinaki Saha Integrated branding, social bookmarking, and aggregation system for media content

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SHAW R ET AL: "Community annotation and remix: A research platform and pilot deployment", 1ST ACM INTERNATIONAL WORKSHOP ON HUMAN-CENTERED MULTIMEDIA, HCM 2006, CO-LOCATED WITH THE 2006 ACM INTERNATIONAL MULTIMEDIA CONFERENCE - 20071027 TO 20071027 - SANTA BARBARA, CA,, 1 January 2006 (2006-01-01), pages 89 - 98, XP009110336 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3103263A4 (en) * 2014-02-07 2018-04-18 Music Media Limited Digital media content and associated user pool apparatus and method
US20160224672A1 (en) * 2015-01-29 2016-08-04 Facebook, Inc. Multimedia Search Using Reshare Text on Online Social Networks
US10061856B2 (en) * 2015-01-29 2018-08-28 Facebook, Inc. Multimedia search using reshare text on online social networks
CN113949920A (en) * 2021-12-20 2022-01-18 深圳佑驾创新科技有限公司 Video annotation method and device, terminal equipment and storage medium
CN115334354A (en) * 2022-08-15 2022-11-11 北京百度网讯科技有限公司 Video annotation method and device
CN115334354B (en) * 2022-08-15 2023-12-29 北京百度网讯科技有限公司 Video labeling method and device

Also Published As

Publication number Publication date
GB201501008D0 (en) 2015-03-04
GB2518791A (en) 2015-04-01

Similar Documents

Publication Publication Date Title
US11294540B2 (en) Categorized and tagged video annotation
US11589137B2 (en) Method for collaborative comments or metadata annotation of video
JP6342951B2 (en) Annotate video interval
US8799300B2 (en) Bookmarking segments of content
US9256632B2 (en) Temporally-correlated activity streams for conferences
US10210253B2 (en) Apparatus of providing comments and statistical information for each section of video contents and the method thereof
US8566315B1 (en) Sequenced video segment mix
US20150139610A1 (en) Computer-assisted collaborative tagging of video content for indexing and table of contents generation
US20120047119A1 (en) System and method for creating and navigating annotated hyperlinks between video segments
US9544544B2 (en) Method and system for sharing information between users in media reproducing system
JP5374758B2 (en) Comment distribution system, comment distribution system operation method, and program
WO2015070807A1 (en) Program recommendation method and device for smart television
WO2014002004A1 (en) A method for marking highlights in a multimedia file and an electronic device thereof
CN108476336A (en) Identify that the audience's of content channel checks characteristic
US20220114204A1 (en) Method for atomically tracking and storing video segments in multi-segment audio-video compositions
WO2008087742A1 (en) Moving picture reproducing system, information terminal device and information display method
CN106605412A (en) Improved interface for accessing television programs
JP2013150221A (en) Information processor, information processing method, and program
CN115563319A (en) Information reply method, device, electronic equipment, computer storage medium and product
JP5757886B2 (en) Television broadcast recording reservation method, apparatus and system
US20230353795A1 (en) Information processing apparatus, information processing method, and program
JP2022162357A (en) Video management system, video management method, and video management program
CN115757870A (en) Information determining method, information providing method, information determining device, information providing device and electronic equipment
JP2021520139A (en) Importing media libraries using graphical interface analysis
CN115134648A (en) Video playing method, device, equipment and computer readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13745479

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 1501008

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20130625

WWE Wipo information: entry into national phase

Ref document number: 1501008.5

Country of ref document: GB

122 Ep: pct application non-entry in european phase

Ref document number: 13745479

Country of ref document: EP

Kind code of ref document: A1