EP3639164A1 - Arrangement and related method for provision of video items - Google Patents
Arrangement and related method for provision of video itemsInfo
- Publication number
- EP3639164A1 EP3639164A1 EP18818244.8A EP18818244A EP3639164A1 EP 3639164 A1 EP3639164 A1 EP 3639164A1 EP 18818244 A EP18818244 A EP 18818244A EP 3639164 A1 EP3639164 A1 EP 3639164A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- video
- arrangement
- item
- user
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000004458 analytical method Methods 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 4
- 238000001931 thermography Methods 0.000 claims 1
- 238000010191 image analysis Methods 0.000 description 10
- 230000002123 temporal effect Effects 0.000 description 10
- 230000008901 benefit Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 8
- 238000007405 data analysis Methods 0.000 description 7
- 239000000463 material Substances 0.000 description 3
- 238000009826 distribution Methods 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- HRANPRDGABOKNQ-ORGXEYTDSA-N (1r,3r,3as,3br,7ar,8as,8bs,8cs,10as)-1-acetyl-5-chloro-3-hydroxy-8b,10a-dimethyl-7-oxo-1,2,3,3a,3b,7,7a,8,8a,8b,8c,9,10,10a-tetradecahydrocyclopenta[a]cyclopropa[g]phenanthren-1-yl acetate Chemical compound C1=C(Cl)C2=CC(=O)[C@@H]3C[C@@H]3[C@]2(C)[C@@H]2[C@@H]1[C@@H]1[C@H](O)C[C@@](C(C)=O)(OC(=O)C)[C@@]1(C)CC2 HRANPRDGABOKNQ-ORGXEYTDSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003703 image analysis method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/102—Programmed access in sequence to addressed parts of tracks of operating record carriers
- G11B27/105—Programmed access in sequence to addressed parts of tracks of operating record carriers of operating discs
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/11—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information not detectable on the record carrier
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
- G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
- H04N21/2353—Processing of additional data, e.g. scrambling of additional data or processing content descriptors specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/482—End-user interface for program selection
- H04N21/4828—End-user interface for program selection for searching program descriptors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
- H04N21/8405—Generation or processing of descriptive data, e.g. content descriptors represented by keywords
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
Definitions
- the present invention is related to provision and consumption of video material. More specifically, however not exclusively, the present invention provides an arrangement and method for searching for digital video items, such as video clips, wherein metadata is provided through utilization of data obtained from or via devices with which said video items have been captured. Search criteria may be related to such metadata.
- Searching for video material taken at a specific event is usually rather difficult, as searches are typically conducted through specifying search words.
- the search words used may be related to the specific event, but video material taken at the event may not have been tagged using these words and is thus not found. Video material may also have been tagged erroneously by the tagger.
- Methods for organizing the search results may thus include relevance according to metadata based tagging, which is problematic due to reasons stated above, including lacking and erroneous (meta)data. Organizing may also be done through popularity based on the number of views or on ratings given by viewers, but this is again subjective and somewhat non-informative in many cases.
- An object of the invention is to alleviate at least some of the problems relating to the known prior art.
- the arrangement of the present invention is characterized by what is stated in attached claim 1.
- the method of the present invention is characterized by what is stated in attached claim 19.
- a computer program is also provided according to claim 21.
- the computer program may be a standalone software program or a mobile application or a portion of a mobile application.
- the computer program may in some embodiments be a plug-in for one or more other software programs such as video editing programs.
- the present invention offers a plethora of advantages over the known prior art, such as providing an intuitive, easy-to-use dynamic arrangement for searching for video items through preferably automatically generated metadata.
- metadata is not specified by an uploader or recorder of the video item, errors, either intentional or accidental, may not be made by the uploader or recorder, and are thus avoided.
- any video item recorded at a specific time or place that has been uploaded may be searched and downloaded from an associated system, such as a server arrangement optionally implemented utilizing a cloud computing environment, by using the time and place (of capture as indicated by supplementary data associated with the video item) as search criteria.
- an associated system such as a server arrangement optionally implemented utilizing a cloud computing environment
- An arrangement may provide many different ways of displaying search results to a user of the arrangement.
- the search results are provided to a user of the arrangement so that video quality may also be indicated through at least one quality indicating parameter, where such quality parameter, or 'indicator', has been advantageously determined as metadata through data obtained via sensors of the device through which the video material has been obtained, such as a smartphone, tablet, or other terminal device, or a camera device provided with or at least functionally connected to a communication device.
- This metadata may preferably be generated automatically by the arrangement.
- video material from an event may be conveniently searched for and found, with the additional benefit of providing search results indicative of video quality, in which case a user of the arrangement may choose not to view video items of poor quality.
- Quality or other metadata, such as tags, associated with a video item may be generated based on data acquired from sensors, such as image (e.g. the image sensor itself used for capturing the video), sound, position and/or acceleration/inertial sensors.
- a video item may comprise one or more, nowadays typically digital, video files, or clips, or at least one or more video frames, which are related to a duration, which may be considered a temporal 'length' or 'extent', of the video item in question, being also indicative of the related recording, or 'shooting', duration.
- a video item may be perceived to be the result of substantially continuous recording having a certain frame rate, or sampling rate, typically but not necessarily falling within a range from at least about 10 or 15 Hz to e.g. about 25, 50, 100 Hz or higher. The rate may be static or dynamic within an item.
- Video quality may be advantageously assessed and preferably also indicated individually for a segment of the video or even separately for each video frame, giving the additional benefit of possibly indicating to a user if only part of a video is of good or bad quality.
- the video quality may then be assessed through analysis of two or more subitems of the video item, where the subitem is related to a subduration. Assessment and e.g. related tagging may take place at a recording device or e.g. the receiving (server) arrangement, or by a third party system. Quality may be assessed e.g. on a predetermined scale, so that a user may be aware of the degree of quality of e.g. segments of a video.
- an average quality of a video item may optionally be communicated to a user.
- the scale or quality in general may be based on e.g. frame rate, resolution, stability, focus, field of view, etc. evaluated utilizing e.g. some more widely adopted standard or a proprietary solution, and/or based on user-selectable settings.
- a quality indicator is assigned to one or more subitems of a video item
- obtained video data may be compared to obtained sensor data for authentication purposes. If there is an inconsistency between the data, it may be determined that either data may be inauthentic. For example, analysis of motion data of a mobile device may be determined from obtained video data, which may be compared to sensor data that is received, and it may be determined that these data match, authenticating the received content.
- data may be received from a plurality of sensors
- some of the received data may be specified or made more precise by the sensor data from other sensors.
- Location data may for instance be made more accurate utilizing the data obtained from a plurality of sensors.
- An external data source for example an RSS feed
- An external source may be utilized, also automatically, to search for video items.
- An external source may provide e.g. an RSS feed providing information relating to events handled by an emergency center, for instance.
- An arrangement may then be configured to be able to suggest video item search results that correspond to such a feed or certain selected instances in such a feed.
- An embodiment of the invention may be used to analyze and/or interpret obtained video content through the obtained sensor data/supplementary data.
- Benefits gained from utilizing embodiments of the invention may also be that it may be possible to supplement or complete content. For instance, it may be determined that two video items have been captured at the same location at the same time by two different mobile user devices. Then, it may be possible to utilize sensor data and/or supplementary data and/or analyzed data from one mobile user device, such as e.g. audio data, to supplement the video content obtained via the other mobile user device.
- one mobile user device such as e.g. audio data
- search criteria may be given that may be related to automatically generated metadata.
- metadata obtained through image analysis may be utilized.
- a user of the arrangement may have a desire to find video material taken at a specific event such as a sports event in which a specific player is visible. This may be specified in the user's search query as for example a player (sur)name or number, and in the search results the arrangement may then provide video items of the event in which the player in question is visible, where the player number has been detected through image analysis of the video footage, essentially involving e.g. pattern recognition, and has been added to the video metadata.
- the arrangement may also, if desired, produce a video compilation of the event in which e.g. a specific player or team is featured exclusively or with positive or negative emphasis, and video items in which the player or team is visible may be compiled, for instance.
- other material for example audio material may be integrated with the video material.
- Video compilations may be generated through utilization of various types of automatically generated metadata. For example time, a direction or angle in which the video has been taken with respect to an event, or certain objects being visible in the video footage may be taken into account. In an embodiment, video quality may have been assessed individually for subitems of a video item. A potential benefit may then be that video compilations with only high or sufficient quality video subitems according to e.g. user- determined criteria are created.
- the benefits explained above may substantially decrease the amount of time that is required to perform video searches. Human labor and computational resources may be saved, as video compilations may be automatically generated even dynamically, in some cases substantially in real-time fashion, responsive to user input such as search queries including criteria given by a user.
- the arrangement may be utilized by private users, whereas in some other embodiments professional users such as reporters or news stations may obtain video material of an event for distribution, publishing or other purposes. Relevant and high-quality footage may be acquired swiftly and with ease through the arrangement. Also, time needed and expenses required may be reduced in the news production process, as reporters may not have to travel to a specific location in order to obtain video footage. A best angle or direction from which video footage is taken may for instance also be found through the arrangement.
- video material obtained even accidentally of an event may be searched for.
- This feature may be utilized for example by law enforcement agencies to investigate possible criminal events or search for individuals suspected of committing a crime.
- fire brigades or other emergency units may exploit the available data in performing their duties.
- Embodiments of the present invention may provide novel methods for news providers or any other private or corporate instance to purchase or generally acquire video content.
- a user may obtain access rights or exclusive rights of video content, and a method of conducting payment to the content uploader may be provided. If exclusive rights to content are purchased or otherwise issued, the content in question may be removed from the database.
- the price of access or exclusive rights may in different embodiments be determined through e.g. auctions, or pricing may be determined automatically by the arrangement based on video content and/or quality analysis. Alternatively or in addition, also other parameters may affect pricing, such as demand or generally popularity of video items in question, which may be estimated based on e.g. search term statistics available.
- Different user rights for an arrangement may also be provided along with different user profile categories. For instance, a user may obtain a user profile free of charge, and may be provided more limited possibilities to search for video content acquired only e.g. in predetermined locations. A fee, for example a monthly fee, may then be charged for user profiles in which available search locations or other search criteria are expanded.
- specific events, locations or instants may also be blocked, so that a group of users or any user may not have access to video material obtained at the event or location, or at a particular instant.
- the blocking may be temporary or permanent and may be determined manually (e.g. by the operator of the arrangement) or automatically based on information or triggers acquired through external sources. For instance, an accident site may be reported through an emergency response center (automatically or manually), whereupon the emergency site and optionally adjacent region may be automatically blocked by the arrangement due to e.g. ethical considerations and/or may, alternatively, be set as a location to be specifically monitored.
- the invention may provide means for uploading video items automatically to a server optionally depending on the settings of the recording terminal or e.g. user account associated therewith. All videos taken using a specific device may, if desired, be uploaded to a server automatically and without unnecessary delay. This way, video items may be available for search briefly, optionally essentially in real-time, and one does not need to remember to manually trigger uploading video material.
- an application for a mobile device may be provided, and all video material captured by the device while the application is running or set in some specific mode or state, e.g. active state, may be automatically uploaded to a server.
- Automatic initial tagging of metadata done through e.g. image analysis methods may also be corrected, adjusted or otherwise altered using external data resources.
- the arrangement may have mistakenly assigned a tag referring to a competitor number "26" to a video footage, when the footage is actually of a competitor with the number "28".
- Timing result records for the marathon may be provided, from which it may be possible in some cases to verify that the competitor "26" could not have been at the location in question at the time that the video footage was captured.
- the arrangement may determine that assigning metadata/tagging has originally been erroneous, and through image analysis, the associated tag may be revised to refer to a competitor "28".
- a user interface may be provided through which video content may be easily captured.
- the user interface may comprise a multifunctional record button which may be manipulated to initiate or terminate video recording and additionally also communicate, in real-time to the content providing user, information related to the video that is being recorded or may be recorded at the given instant, for example.
- the multifunctional record button or similar Ul feature may also be cleverly repositioned by the content providing user or automatically, for instance, to a desired location on a screen of a recording device. This ensures that the record button may at all times be located at an optimal position for the content providing user, where the optimal position may depend, for instance, on the handedness or length of fingers of the content providing user.
- the multifunctional record button may also be resized (enlargened, reduced, and/or otherwise redimensioned) automatically or by the content providing user.
- the multifunctional record button may then be set to a size according to a (user) preference or may be resized to ensure e.g. that the content providing user has desired visibility of the screen of the recording device in which case the size of the button should not exceed e.g. a selected limit of the screen or window size.
- a searching user may, after providing a search query with a time and location (e.g. indication of location- specific target event or more explicit indication of a geographic location, such as address, coordinates, city or region name), view on a map where in the vicinity of the given location videos have been captured at the provided time.
- the quality and/or point of view or direction from which the video has been shot or a field of view of the videos may also be communicated to the searching user.
- the user interface may comprise a timeline which may be navigated by the searching user, where manipulation of the timeline may result in the searching user being able to view, through the aforementioned map, changes with time in the provided information being related to videos that are available from the map region.
- a user interface may additionally show one or more of the videos indicated in the map at the time which is given at the timeline.
- the videos may be played and the timeline may be allowed to proceed, while the map view changes accordingly.
- a searching user may also manipulate the timeline to view video frames and/or map configurations corresponding to a desired time in the vicinity of the time expressed through the search query.
- the time and location may also be chosen or varied by the searching user after a search query has been given.
- the claims and description do not pose limitations to the number of users, either searching or content providing, that may utilize the invention simultaneously or at different times. At a given time, any number of users may exist, or there may be one or no users. From a technical standpoint, the disclosed arrangement may be assigned, optionally dynamically e.g. from a cloud, the necessary hardware in terms of required processing, storage, and communications capacity for serving the users thereof.
- time may herein refer to a date, an exact time (e.g. per second or fraction of a second) on a specific date, or a time interval, such as a time within a predetermined time limit from another time.
- location may refer to a specific location, such as coordinates, or a region such as a city, or a region which may be specified as residing within a predetermined distance from a specific location.
- plurality refers herein to a number of two or more.
- number refers to one or more.
- Figure 1 illustrates an exemplary arrangement according to one embodiment of the invention
- Figure 2 shows an exemplary application view of a first user interface according to an embodiment of the invention
- Figure 3 shows an exemplary application view of a second user interface according to an embodiment of the invention.
- Figure 4 depicts steps that may be executed in a method according to an embodiment of the invention.
- Figure 1 gives an exemplary arrangement 100, optionally comprising one or more server computers and/or other network side elements implementing e.g. a network service for distribution of videos, according to one embodiment of the invention.
- the arrangement 100 comprises a processor 102, which may receive a video item from a first user device 104, which is a content providing user device, where the user device 104 may be a mobile device such as a mobile phone or a tablet.
- the video item may be received from the device 104 as a stream e.g. in substantially real-time fashion upon capturing, whereas in other embodiments, a ready-captured complete item such as a video file, or 'clip', may be received after capturing phase.
- the processor 102 may indeed reside in at least one remote server, and the receiving of a video item may be realized utilizing e.g. at least partially wireless communication between the server or processor 102 and the user device 104.
- the server, or the arrangement 100 may include memory and for communication purposes, a communications interface such as wireless or wired transceiver, e.g. a transceiver operable in a LAN (local area network) network.
- a communications interface such as wireless or wired transceiver, e.g. a transceiver operable in a LAN (local area network) network.
- the arrangement 100 may be coupled via the interface to a communications network, such as the internet, whereto the device 104 also has access via a wireless link of a related wireless, optionally cellular or Wi-Fi type, network.
- the first user device 104 may, in addition to a processor, memory and e.g. communication interface (e.g. a wireless transceiver), comprise a plurality of sensors, at least a camera 106 (image sensor capturing visible and/or invisible such as infrared frequencies; the latter could provide e.g. thermal images for temperature estimation and/or heat-based object/subject locating purposes) and additionally other sensors, such as a magnetometer 108, an accelerometer 1 10, a microphone 1 12, a GPS sensor 1 14, and/or a gyroscope 1 16 or still a further inertial sensor. Further applicable sensors include e.g. temperature, moisture and pressure sensors.
- a camera 106 image sensor capturing visible and/or invisible such as infrared frequencies; the latter could provide e.g. thermal images for temperature estimation and/or heat-based object/subject locating purposes
- sensors such as a magnetometer 108, an accelerometer 1 10, a microphone 1 12, a GPS sensor 1 14, and/or a gy
- Sensor data from sensors in the first user device 104 may be received by the processor 102. Additionally, other data may be received by the processor 102 through the first user device 104, such as temporal data (capture time, duration, etc.) related to the video item.
- An arrangement may additionally or alternatively in some embodiments comprise one or more secondary content providing devices or secondary user devices 105.
- the one or more devices 105 external to the first user device 104 may comprise one or more sensors that acquire and transmit data, e.g. as a stream or ready-captured complete item (e.g. file), to a first user device 104 through wired or wireless communication.
- the first user device 104 may then transmit the data (as such or in adapted form) and possibly other data, e.g., from sensors comprised in the first user device 104, to the processor 102.
- the one or more secondary devices 105 may additionally or alternatively transmit the data from its one or more sensors directly to the processor 102.
- a drone, other unmanned, optionally aerial, vehicle or generally some other type of a sensor/camera carrying (user) device may be used to supply video data and optionally additional data such as sensor data.
- the item may be transmitted to a first user device 104.
- Such data thus originally potentially obtained from a connected video capturing device 105 may then be received by the processor 102 via the first user device 104.
- the arrangement 100 comprises a first user application 1 18 in connection with the first user device 104.
- the first user application 1 18 may provide a first user interface 120, which may enable recording of video content by the first user device 104 and provide information regarding the video content and/or the sensor data that is being acquired.
- the first user device 104 may in some embodiments be a camera, for instance a single-lens reflex or mirrorless interchangeable-lens camera, that is configured to obtain video data and supplementary data and provide such data to the processor 102 e.g. via a memory card.
- the memory card may be configured to send the aforementioned data e.g. via wireless means to the processor 102.
- the arrangement 100 and related invention may be utilized through a first user device 104 that does not have to e.g. intrinsically comprise an application for being included in an arrangement 100.
- a memory card may enable a plurality of first user devices 104 such as cameras to be utilized with the arrangement 100.
- the video item received from the first user device 104 may be stored in a database 122, which the processor 102 may have access to, both of which may be comprised in a server apparatus.
- the processor 102 may be configured to use the data received through a first user device 104 to perform automatic tagging (assigning e.g. a keyword or other descriptor that can be searched) of at least one received video item.
- the data may refer to the data obtained through sensors of the first user device 104 and also additionally to other data that may be received from the first user device 104 in conjunction with the video item, such as data regarding the time that the video item was captured.
- at least the time is added in a metadata file, field, portion or other data element associated with the video content, i.e., the video item is tagged with at least temporal information.
- the processor 102 may comprise or have access to a sensor data analysis module 124, which may be used to analyze the received video item or other data content and additionally may also be utilized in automatic tagging. Alternatively or additionally, some tagging could take place at the user device 104.
- the sensor data analysis module may comprise means for analysis of different types of sensor data or metadata associated with a received video item.
- Information from, e.g., a GPS sensor 1 14 may be used to determine a location in which a video item has been captured.
- the received video material may be automatically tagged with information regarding data received through a GPS sensor 1 12, i.e., the location in which video content has been captured.
- Sensor such as GPS based data may be included in a tag as such, and/or higher-level location data be derived therefrom, e.g. address, landmark, district, building, city, country and/or event data (when combined with temporal/time data and schedule data regarding different events and their locations).
- Sensor data from sensors such as a camera 106, magnetometer 108, an accelerometer 1 10, and/or a gyroscope 1 16, may be analyzed by the sensor data analysis module 124 to create metadata and metadata based tags that are related to e.g. field of view or viewing angle, or 'watching direction', of the captured video content.
- the sensor data may in some embodiments be used to determine metadata that is indicative of quality of the video material.
- the arrangement 102 may through the sensor data analysis module 124 determine, e.g., if the first user device 104 has been shaking during the capturing of the video material. This information may be utilized in tagging. In some embodiments, the aforementioned information may also be used to automatically enhance video quality by a stabilization algorithm, for instance. Additionally or alternatively, in some embodiments shaking may be detected directly from the captured video data based on image analysis.
- quality associated data or similar parameters may also be determined by an arrangement 100 through analysis of the video item and/or sensor data associated therewith.
- the parameters may be further analyzed to determine e.g. a characterizing value on a scale or some other criteria that may be employed to indicate the quality of a video item.
- the video item may then also be tagged with the respective quality information for search and/or compilation purposes.
- one or more quality parameters may be evaluated for a video subitem in addition to or instead of general parameters regarding the item as a whole.
- a video item may be related to a temporal duration, and may be divided into two or more subitems, which may be related to temporal subdurations. These subdurations may be mutually of equal or different length (duration).
- the subitems may be video segments or video frames.
- the device 104 or processor 102 may be configured to divide the video item into subitems based on e.g. recognized video (image) content, quality and/or other parameters of the underlying video or related sensor data.
- the subitems may be created automatically responsive to user input; for example, each recording pause instructed by the user via the Ul may translate into switching over to a new subitem.
- the subitems may be determined as having discrete or overlapping durations relative to the overall item.
- each of the video subitems is assigned at least one quality parameter.
- the sensor data analysis module 124 may comprise means for image analysis, which may utilize various methods of computer vision and pattern recognition.
- the image analysis may be utilized to detect, classify and/or identify e.g. objects appearing in video content, and also be capable of more complicated image analysis, such as facial recognition. Accordingly, related metadata such as tags may be associated with the video item.
- the image analysis may additionally comprise methods where also other sensor data, such as auditory data from a microphone 1 12 is utilized to analyse complex events and provide information on semantic content that may be detected or determined through video material and possible other available sensor data.
- an arrangement 100 may comprise a video compilation module 126, which may combine two or more video items or subitems of one or more video items to create a compiled video.
- the video items may be received from one or several different first user devices 104 in the possession of one or more users.
- the video compilation module 126 may be utilized to create a compiled video of, e.g., an event, where only video items or subitems of desired (user-selected) kind, e.g. video items or subitems that have quality criteria matching or exceeding a certain limit, may be used or at least preferred over other items/subitems.
- Compiled videos where an event or object of interest may be viewed or shown from various different angles may also be created.
- a user requesting the compilation may preferably define characteristics of the compilation to be established based on available video data, said characteristics possibly including quality characteristics, temporal characteristics (e.g. may the compilation have temporally overlapping portions, or should it be temporally more linear or strictly chronological), angle of view/shooting direction/ shooting location related preferences, target object or person preferences, use of slow-motion or other effects (e.g. transitional effects) related preferences, etc.
- characteristics of the compilation possibly including quality characteristics, temporal characteristics (e.g. may the compilation have temporally overlapping portions, or should it be temporally more linear or strictly chronological), angle of view/shooting direction/ shooting location related preferences, target object or person preferences, use of slow-motion or other effects (e.g. transitional effects) related preferences, etc.
- the video compilation module 126 thereby enables compiling videos from an event with various video items or subitems featuring an entity that may be defined through image analysis.
- a compilation video from a certain sports event may feature a certain player, or a compilation video from a certain cultural event such as a concert may feature a certain artist.
- the creation of compilation videos may be further enabled through the use of the sensor data analysis module 124.
- Video subitems such as segments or frames, may be analyzed separately, and a compilation video with high quality featuring an event or an entity of interest may be generated where video subitems appear in chronological order, as temporal information is also utilized in the arrangement 100.
- the video compilation module 126 may incorporate into a compilation video data from a plurality of video (image) sensors comprised in a first user device 104.
- data from some other data source of the device 104 or a further device may be integrated with a compilation video. For instance, if a video compilation from an event such as a concert, other cultural event or a sports event is created, sensor data such as sound from a microphone 1 12 may be added as a sound track to the video.
- the microphone 1 12 or other sensor data may originate from the same first user device 104 that has been used to capture the video content or another first user device 104. Also external data, such as an external sound track, may be added.
- an arrangement 100 may through the video compilation module 126 and image analysis determine semantic content or context of a video and automatically generate a compilation video and optionally also include external data pertinent to the detected context. For instance, an arrangement 100 may identify events or situations happening in a video or videos, for instance that the video content has been captured during a wedding and add an appropriate sound track tagged as wedding type music.
- a processor 102 may receive, possibly through a second user device 128 utilizing a second user application 130 and a second user interface 132, a search query for a video or compilation.
- the search query may cause the processor 102 to conduct a search and identify video items stored in the database 122 that match the search query.
- the processor 102 may then communicate to a user of the second user device 128 the results of the search through providing an output and facilitate access of said user to the video item search results.
- a search query comprises at least a time and/or a location. Any number of other search criteria may also be added.
- at least one of the search criteria is related to the metadata or associated tags that have been received or created by an arrangement 100.
- a search query may also indicate e.g. the name of an event such as a concert or sports event.
- the output provided by the arrangement 100 may be indicative of the characteristics such as the quality of the video item search results.
- the quality may be related to quality parameters obtained through analysis of the at least one video item and/or other sensor data.
- Figure 2 shows two views, 2A and 2B, of an exemplary application view 200 in a depiction of a first user interface 120, which may be provided through a first user application 1 18, the first user interface 120 intended for use with a first user device 104.
- the first user interface 120 may comprise a multifunctional record feature essentially in the form or shape of e.g. button 202, which may comprise an outer portion such as an outer rim 204.
- the multifunctional record button 202 is shown on a screen of the first user device 140 and thereby implemented as a touch- sensitive icon displayed, whereupon pressing the multifunctional record button 202 may result in initiating or terminating recording of video content with the first user device 104.
- initiation or termination of video recording may be done through some other method, such as auditory methods based on the analysis of microphone-captured sound data such as voice commands.
- the outer rim 204 may visually indicate to a content providing user of the first user device 104 information regarding the video content that is being recorded by the first user device 104 or may be recorded at that particular instant in time.
- the information may be related to e.g. video quality.
- a color or pattern may be presented on the outer rim 204 which may be indicative of video quality and/or some other video related characteristics or information.
- the first user application 1 18 may communicate to a user of the first user device 104 information related to video content through haptic feedback initiated through the first user device 104. For instance, if a video is being recorded by the first user device 104 and the first user device 104 is tilted during the recording, haptic feedback via e.g. a vibration element may inform the user of the first device 104 of the tilting. This may be advantageous e.g. in a situation where video content is being recorded by a handheld device such as a mobile phone and a user of the device is not constantly looking at the device or the video content that is being recorded, and may want to focus their attention or gaze on something else, while still obtaining video material of high quality.
- haptic feedback via e.g. a vibration element may inform the user of the first device 104 of the tilting.
- FIG 3 an exemplary depiction of an application view 300 of a second user interface 132 is shown.
- Figure 3 specifically illustrates a possible schematic appearance of an application view that may comprise an output that may be provided by an arrangement 100 in response to a search query.
- the application view 300 may comprise e.g. a map or other location-indicating view 302.
- the view 302 may show an area representative of a geographical location that is specified through the search query or show an area that comprises the location explicitly (e.g. location name, address, coordinates, etc.) or implicitly (e.g. event) specified in the search query.
- the area to be shown may be predetermined as to related one or more parameters such as coverage. For example, it may be specified that the map view 302 initially shows an area comprising of 1 square kilometer.
- the searching user may manipulate the map view 302 via the Ul in order to navigate surrounding areas and/or e.g. zoom in and out.
- the map view 302 may visually indicate geographical points corresponding to locations where at least one first user device 104 has captured video content at a specific time. This may be done by e.g. placing points, circles or other pointers 304 on a map view 302.
- the map view 302 may also visually indicate the quality of the video material.
- a quality parameter relating to the video subitem in question may be indicated to a user through e.g. a color or pattern which may displayed on the map view 302. Also other indicators may be used, such as a percentage or a number, for instance.
- the field of view 306, shooting direction and/or similar characteristics of a video item at a specific time may also be visually and preferably dynamically (i.e. changing in accordance with e.g. shown video 308) indicated on the map view 302, which may also comprise the indication of a quality parameter as discussed above.
- Such an implementation is indeed shown in Fig. 3.
- the application view 300 may in one embodiment also show a preview 308 of one or more video items.
- a preview may be shown for one or more video items indicated in the map view 302.
- the video items for which previews are currently provided may be selected by the arrangement 100 according to some criterion or may be selected by a searching user through, e.g., clicking on the circles 304 or other visual indicators associated with video items.
- An application view 300 may additionally comprise a timeline 310, through which a searching user may navigate the video items that are depicted on a map view 302.
- a searching user may, for instance, manipulate the timeline to directly or gradually proceed to certain time and the map view 302 and preview(s) 308 may change accordingly.
- a timeline may also be allowed to proceed in real time and the preview(s) 308 may play the selected video item(s) accordingly, while the map view 302 may also change accordingly.
- timeline 310 may also be made possible, such as e.g. "fast forwarding" or proceeding on the timeline in a time scale that is faster than real time with the preview(s) 308 and map view 302 changing accordingly.
- the timeline may also be navigated in the opposite direction, i.e., one may move backwards in time on the timeline.
- video items may be provided to a searching user as a live stream.
- the timeline 310 may essentially show a current time and a searching user may observe in the application view 300 video items that are being captured substantially in real time.
- video items are provided as a live stream, they may also be stored in the database. Thus, videos provided as a live stream may be searched for and viewed at a later time such as has been disclosed previously.
- search criteria may yield a number of video item search results that are too large for practical considerations based on e.g. user-defined settings.
- Figure 4 shows steps that may be performed in a method according to an embodiment of the present invention.
- one or more video items are received from a number of first user devices, whereby the video items are stored in a database 122 in 404.
- Supplementary data is received in 406, the data being provided through the device with which said at least one video item has been recorded.
- the data may be provided by one or more sensors comprised in the first user device(s) 104.
- the data may also comprise temporal data (time, timecode, clock, sync, etc.) provided through the first user device(s) 104.
- the one or more video items relating to initial time duration are divided into a plurality of discrete or overlapping subitems related to corresponding subdurations in 408 as contemplated hereinbefore.
- step 410 metadata is provided preferably in connection with video items or even specific subitems (e.g. subitem-specific dedicated quality indicators), the metadata being obtained through utilizing the received supplementary data.
- the metadata may be created by the sensor data analysis module 124.
- Metadata may be assigned to (associated with) one or more video items and/or related subitems as related searchable and preferably also inspectable tags.
- one or more quality indicating parameters may be assigned to each video subitem to enable quality-based searches.
- a search query may be received in 412, and video item search results matching the search query may be determined in 414, after which in step 416, an output indicative of the video item search results may be provided.
- the output may include at least a listing of the video items of the results group, which may contain at least one video item.
- the output may include the actual video item(s).
- a video compilation typically based on a plurality of originally discrete video items, fulfilling the user-defined criteria such as temporospatial criteria (time, location, event, etc.) may be established either fully automatically or in a user-assisted fashion.
- the compilation may be provided as the outputted video item.
- the compilation may comprise a single video stream or a video file constructed from the constituent (source) video items preferably best fulfilling the criteria, for example.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Television Signal Processing For Recording (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FI20175539A FI129291B (en) | 2017-06-12 | 2017-06-12 | Arrangement and related method for provision of video items |
PCT/FI2018/050442 WO2018229332A1 (en) | 2017-06-12 | 2018-06-12 | Arrangement and related method for provision of video items |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3639164A1 true EP3639164A1 (en) | 2020-04-22 |
EP3639164A4 EP3639164A4 (en) | 2021-01-20 |
Family
ID=64660044
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18818244.8A Withdrawn EP3639164A4 (en) | 2017-06-12 | 2018-06-12 | Arrangement and related method for provision of video items |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP3639164A4 (en) |
KR (1) | KR20200017466A (en) |
FI (1) | FI129291B (en) |
WO (1) | WO2018229332A1 (en) |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130097172A1 (en) * | 2011-04-04 | 2013-04-18 | Zachary McIntosh | Method and apparatus for indexing and retrieving multimedia with objective metadata |
US9584834B1 (en) * | 2012-06-25 | 2017-02-28 | Google Inc. | Video broadcasting with geolocation |
WO2014189485A1 (en) * | 2013-05-20 | 2014-11-27 | Intel Corporation | Elastic cloud video editing and multimedia search |
US9685194B2 (en) * | 2014-07-23 | 2017-06-20 | Gopro, Inc. | Voice-based video tagging |
US9734870B2 (en) * | 2015-01-05 | 2017-08-15 | Gopro, Inc. | Media identifier generation for camera-captured media |
KR102246557B1 (en) * | 2015-04-07 | 2021-04-30 | 엘지전자 주식회사 | Mobile terminal and method for controlling the same |
US20170161961A1 (en) * | 2015-12-07 | 2017-06-08 | Paul Salsberg | Parking space control method and system with unmanned paired aerial vehicle (uav) |
-
2017
- 2017-06-12 FI FI20175539A patent/FI129291B/en not_active IP Right Cessation
-
2018
- 2018-06-12 EP EP18818244.8A patent/EP3639164A4/en not_active Withdrawn
- 2018-06-12 WO PCT/FI2018/050442 patent/WO2018229332A1/en unknown
- 2018-06-12 KR KR1020207000782A patent/KR20200017466A/en unknown
Also Published As
Publication number | Publication date |
---|---|
WO2018229332A1 (en) | 2018-12-20 |
FI20175539A1 (en) | 2018-12-13 |
KR20200017466A (en) | 2020-02-18 |
FI129291B (en) | 2021-11-15 |
EP3639164A4 (en) | 2021-01-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20180357316A1 (en) | Arrangement and related method for provision of video items | |
US10084961B2 (en) | Automatic generation of video from spherical content using audio/visual analysis | |
US7594177B2 (en) | System and method for video browsing using a cluster index | |
JP7017175B2 (en) | Information processing equipment, information processing method, program | |
CN105320695B (en) | Picture processing method and device | |
CN107870961B (en) | Method and system for searching and sorting space objects and computer readable storage device | |
US20180132006A1 (en) | Highlight-based movie navigation, editing and sharing | |
US20180103197A1 (en) | Automatic Generation of Video Using Location-Based Metadata Generated from Wireless Beacons | |
JP7372008B2 (en) | Methods, apparatus and systems for time-based and geographical navigation of video content | |
EP3669276B1 (en) | Multimedia focalization | |
WO2021031733A1 (en) | Method for generating video special effect, and terminal | |
CN114520877A (en) | Video recording method and device and electronic equipment | |
CN105049768A (en) | Video playback method of video recording equipment | |
CN108200477B (en) | Method, device and equipment for generating and playing video file | |
JP6617547B2 (en) | Image management system, image management method, and program | |
FI129291B (en) | Arrangement and related method for provision of video items | |
CN112437270B (en) | Monitoring video playing method and device and readable storage medium | |
CN106354808A (en) | Image storage method and device | |
US20180189602A1 (en) | Method of and system for determining and selecting media representing event diversity | |
CN107079144B (en) | Method, device and system for realizing media object display | |
JP5536003B2 (en) | Movie editing apparatus, movie editing method, and program | |
US20220053248A1 (en) | Collaborative event-based multimedia system and method | |
JP6743441B2 (en) | Image collection device, display system, image collection method and program | |
CN117992628A (en) | Image display control method and device and electronic equipment | |
CN117294900A (en) | Video playing method and device, electronic equipment and readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20191231 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20201221 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04N 21/4223 20110101ALI20201215BHEP Ipc: G06F 3/01 20060101ALI20201215BHEP Ipc: B64C 39/02 20060101ALI20201215BHEP Ipc: H04N 21/235 20110101AFI20201215BHEP Ipc: G11B 27/28 20060101ALI20201215BHEP Ipc: H04W 4/02 20180101ALI20201215BHEP Ipc: H04N 21/435 20110101ALI20201215BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20210730 |