KR20200017466A - Apparatus and associated method for providing video items - Google Patents

Abstract

Apparatus 100, optionally comprising one or more network accessible servers for retrieving digital video material, the apparatus comprising: receiving a plurality of video items; Store the video item in a database; Providing metadata, preferably multiple tags, for any or each video item, wherein the metadata is obtained using supplemental data provided by at least one user device 104 from which the video item was obtained. -; Receive a search query, wherein at least one search criterion included in the search query is associated with the metadata; Determine a video item search result that matches a search query among the at least one video item stored in a database; And at least one processor 102, configured to provide an output to the user, the output including a video item search result, wherein the video item is associated with a duration and the provided metadata is associated with one or more subitems of the video item. And at least one quality indicator determined through analysis of the supplemental data, wherein the sub item is associated with a sub duration of the sub item and the at least one quality indicator is assigned to a corresponding sub item. Corresponding methods are presented.

Description

Apparatus and associated method for providing video items

The present invention relates to the provision and consumption of video material. More specifically, but not exclusively, the present invention provides an apparatus and method for retrieving a digital video item, such as a video clip, wherein metadata is the use of data obtained from or through the device from which the video item was captured. Is provided through. Search criteria may be associated with such metadata.

In the past few years, advances in camera technology and mobile devices have greatly increased the amount of video footage recorded on mobile devices such as smartphones and tablets. These images are uploaded to social media sites where users can search for interesting video images using their preferred search criteria. Search criteria are typically used to search for matching metadata associated with a video image, such as another video clip. Related metadata may be associated with the video clip via tagging performed by a recorder or uploader of the video material.

Searching is typically done by specifying a search term, so searching for video footage of a particular event is rather difficult. The search term used may be related to a particular event, but cannot be found because the video material that captured the event may not be tagged using such words. The video material may have been incorrectly tagged by the tag author.

The vast amount of video material available may also result in a large amount of search results that would be practically impossible for the searcher to review them.

Thus, the method of constructing search results may include relevance according to metadata based tagging, which is problematic for the reasons mentioned above including lack of metadata and error (meta) data. You can also build on popularity based on the number of viewers or ratings provided by viewers, but this is again subjective and often lacks information.

Traditional video retrieval methods may also provide information about video quality. However, even within a particular video, video quality can vary greatly, which can be problematic and misleading. Quality itself is a rather subjective measure, and if the user of a video item such as an uploader or downloader can evaluate the quality on their own, the measures used can mean different things to different people, and the published quality ratings can be easily compared with each other. As a result, evaluation of the quality itself is often difficult.

For example, video taken at a particular event can be combined to form a compilation of video taken at different angles or directions. However, tools for easily composing such edits, for example the necessary user input reduced to a minimum or minimum as necessary, are currently unavailable. The tools available require some manual effort and effort, special expertise in video editing, and a jump to other video clips to choose from in editing.

It is an object of the present invention to alleviate at least some of the problems associated with known prior art.

The device of the invention is characterized by what is stated in the attached claim 1. The process of the invention is characterized by what is stated in the appended claims 19.

A computer program is also provided according to claim 21. The computer program may be a standalone software program or part of a mobile application or mobile application. The computer program may in some embodiments be a plug-in for one or more other software programs, such as video modification programs.

The present invention preferably provides even more advantages over known prior art, such as providing an intuitive and easy-to-use dynamic device for retrieving video items via automatically generated metadata. Since metadata is not specified by the uploader or recorder of the video item, an intentional or accidental error may not be generated by the uploader or recorder, so it is avoided.

Using one embodiment of the present invention, any video item recorded at a particular time or place uploaded may be used as a search criterion by using time and place (of capture as indicated by the supplementary data associated with the video item). It may be retrieved and downloaded from an associated system, such as a server device, optionally implemented using a cloud computing environment.

The device may provide many other ways of displaying the search results to the user of the device. In an advantageous embodiment, the search results are provided to the user of the device such that the video quality can also be indicated via at least one quality indication parameter, which quality parameter or 'indicator' is a video such as a smartphone, tablet or other terminal device. The data is advantageously determined as metadata via data obtained at least functionally connected to a sensor or communication device of the device from which it is obtained or via a camera device provided in the communication device. Such metadata may preferably be automatically generated by the device. Using the present invention, with the added benefit of providing search results indicative of video quality, video material from events can be conveniently searched and found, in which case the user of the device chooses not to view poor video items. Can be. Quality or other metadata, such as tags associated with video items, may be applied to data obtained from sensors such as images (e.g., the image sensor itself used to capture video), sound, position, and / or acceleration / parallel sensors. Can be generated based on this.

The video item is currently at least one video that is typically associated with a digital, video file or clip, or a duration that can be considered as the temporal 'length' or 'degree' of the video or related video item indicating the duration of the shooting. It can contain one or more frames. Recognizing that a video item is the result of a substantially continuous recording having a particular frame rate, or sampling rate, but not necessarily, but typically ranging from at least about 10 or 15 Hz to, for example, about 25, 50, 100 HZ or more. Can be. Rate within an item can be static or dynamic. The video quality can be advantageously assessed, and can preferably be indicated individually for each segment of the video or for each video frame, which can indicate to the user if only part of the video is of good or poor quality. That provides additional benefits. Video quality can be assessed through analysis of two or more subitems of a video item, where the subitem is related to the sub duration. The assessment and associated tagging, for example, can be located at the recording device or at the receiving (server) device or by a third party system. The quality can be evaluated, for example on a predetermined scale, so that the user can know the degree of quality, for example the quality of a segment of the video. Alternatively or in addition, the average quality of the video item may optionally be communicated to the user. Scale or quality is generally assessed based on, for example, frame rate, resolution, stability, focus, field of view, and the like, and based on more widely adopted standard or proprietary solutions and / or user selectable settings.

In a preferred embodiment in which a quality indicator is assigned to one or more subitems of a video item, the quality of the video item is convenient by indicating one or more subitems of low quality and then improving the quality of such one or more subitems. It is possible to improve or improve. For example, through quality indicators related to stability, it may be determined that a particular subitem should be modified for stability and that appropriate modifications can be applied to that particular subitem.

Using the present invention, the obtained video data can be compared with sensor data obtained for authentication purposes. If there is a mismatch between the data, either of the two data can be determined to be inaccurate. For example, the analysis of the motion data of the mobile device can be determined from the acquired video data, which can be compared with the received sensor data, and these data can also be determined to match the received content.

In embodiments in which data may be received from a plurality of sensors, some of the received data may be specified or refined by sensor data from other sensors. The position data can be made more precise, for example using data obtained from a plurality of sensors.

In some embodiments, it may be possible to use external data. External data sources, such as RSS feeds, can also be used automatically to retrieve video items. An external source may provide an RSS feed that provides information related to, for example, an event processed by the emergency center. The device may then be configured to suggest video item search results corresponding to that feed or to a particular moment selected from that feed.

Embodiments of the present invention may be used to analyze and / or interpret video content obtained through acquired sensor data / supplementary data.

An advantage obtained by using embodiments of the present invention is that it may also be possible to supplement or complete the content. For example, two video items may be determined to be captured simultaneously at the same location by two different mobile user devices. It may then be possible to use sensor data and / or supplemental data and / or data analyzed from one mobile user device, for example audio data, to supplement video content obtained via another mobile user device.

In some embodiments, other search criteria may be provided that may be associated with automatically generated metadata. In one embodiment, for example, metadata obtained through image analysis may be used. In an example usage scenario, a user of the device may wish to find video material shot at a particular event, such as a sporting event with a particular player visible. This can be specified in the user's search query, for example, the player's last name or number, and in the search results the device can provide a video item of the event in which the player in question is visible, where the player number is essentially For example, it is detected through image analysis of a video image related to pattern recognition and added to the video metadata.

In connection with the example described above, in one embodiment, the device may also create, for example, a video edit of the event, if desired, such video edit being emphasized exclusively or positively or negatively by a particular player or team, for example. It may be a video editing of an event characterized by, for example, a video item in which the player or team is visible may be compiled. Other materials may also be integrated with the video material, for example audio material.

Video editing can be created using various types of automatically generated metadata. For example, in relation to the event, the direction or angle at which the video is captured, or a specific object visible in the video image may be considered. In one embodiment, video quality may be evaluated separately for subitems of video items. A potential advantage thereafter is that video edits can be created having only high quality or sufficient video subitems, for example, according to user-determined criteria.

The above-described advantages can substantially reduce the time required to perform a video search. In response to user input, such as a search query that includes a criterion given by the user, manpower and computational resources can be saved because video editing can be generated automatically, even dynamically, in some cases substantially in real time. .

In some embodiments, the device may be used by an individual user, while in some other embodiments a professional user, such as a reporter or news station, may obtain video material of an event for distribution, publication, or other purposes. . The device makes it fast and easy to obtain relevant, high-quality images. In addition, since the reporter does not need to move to a specific location to acquire a video image during the news production process, the required time and required cost can be reduced. For example, the best angle or direction of taking a video image can also be found through the device.

You can also search for video material, perhaps unexpected, that you accidentally get from the event. This function can be used, for example, to investigate possible criminal cases in law enforcement or to search for individuals suspected of committing a crime. However, for example, fire brigade or other emergency units may utilize available data in performing their duties.

Embodiments of the present invention may provide a novel method for news providers or any other individual or business authorities to purchase or generally obtain video content. Through an embodiment of such an apparatus, a user may obtain access rights or exclusive rights for video content, and a method of performing a payment to a content uploader may be performed. If an exclusive right to the content is purchased or otherwise issued, the content may be removed from the database.

The price of the right to access or the exclusive right may be determined in another embodiment via a price that may be automatically determined by the device, for example, based on auction, or video content and / or quality analysis. Alternatively or additionally, other parameters may also affect the price, such as the general popularity or demand of the video item in question, which can be estimated based on possible query statistics, for example. Can be.

Other user rights for the device may be provided along with other user profile categories. For example, a user may obtain a user profile for free, and more limited possibilities may be provided for retrieving acquired video content, for example only at predetermined locations. You may be charged a fee, for example a monthly fee, for a user profile in which available search locations or other search criteria are expanded.

Through embodiments of the present invention, certain events, locations, or instances may be blocked such that a user group or any user may not have access to video material obtained at the event or location or at a particular instance. Shutdown may be temporary or permanent, and may be determined automatically based on triggers or information obtained manually (eg by the operator of the device) or through an external source. For example, an accident site can be reported (either automatically or manually) through an emergency response center, so that the emergency site and optionally adjacent areas are automatically blocked by the device, for example due to ethical considerations. And / or alternatively, it may be specifically set to a location to monitor.

In another aspect, the present invention may optionally provide a means for automatically uploading a video item to a server according to the settings of the recording terminal or, for example, according to a user account associated therewith. If you want, you can automatically upload all the videos you take using a particular device to the server without unnecessary delay. In this way, video items can be selectively retrieved simply in real time, without having to remember to manually trigger video material upload. In one embodiment, an application for a mobile device may be provided, wherein all video material captured by the device while the application is running or while the device is in a particular mode or state, for example an active state, is automatically provided to the server. Can be uploaded.

For example, automatic initial tagging of metadata may be modified, adjusted or otherwise changed using external data resources, for example, automatic initial tagging performed through an image analysis method. For example, in the case of a marathon, if the picture is actually a competitor with the number " 28 ", the device may incorrectly assign a tag referring to the competitor number " 26 " to the video picture. A recording of timing results for the marathon may be provided, and in some cases it may be possible to confirm that the competitor “26” is not at that location when capturing the video image. In this case, the device may determine that the metadata / tagging assignment was originally wrong, and through the image analysis, the associated tag may be modified to refer to the competitor “28”.

An application having a user interface that includes various advantageous features may be provided to a device of a user of the apparatus. For example, for a content providing user, a user interface (UI) may be provided in which video content can be easily captured. The user interface may include a multifunction recording button that may be manipulated to initiate or end video recording, and further communicates information to the content providing user regarding video that is being recorded in real time or that may be recorded at a given moment.

The multi-function record button or similar UI feature may also be placed cleverly or automatically by the content presenting user, for example at a desired location on the screen of the recording device. This ensures that the record button can always be located at the optimal location for the content presenting user, where the optimal location may depend, for example, on the length or hand of the content presenting user's finger.

In one embodiment, the multifunction record button may also be scaled (zoom in, zoomed out, and / or otherwise expanded) by the content presenting user. The multi-function record button can be set to a size according to the (user) preference or can be resized, for example, to ensure that the content providing user has the desired visibility of the recording device, in which case the size of the button For example, it must not exceed the selected limit of screen or window size.

In one embodiment, using a user interface, a search user provides a search query with time and location (e.g., a representation of a location-specific target event or a more accurate representation of an address, coordinates, city or region name). Then, you can see the map captured by the video at the given time near the given location. The point of view or direction in which the video was taken and / or the quality, or the point of view of the video, may also be communicated to the search user. The user interface may include a timeline that can be adjusted by the search user, the manipulation of the timeline being such that the search user can see the change over time of the information provided in relation to the video available in the map area via the map described above. Can cause results. In one embodiment, the user interface may further show one or more videos directed to the map at a given time in the timeline. While the video can be played and the timeline can be allowed to progress, the map view changes accordingly. The search user may also manipulate the timeline to view the video frame and / or map configuration corresponding to the desired time near the time indicated through the search query. In some embodiments, the time and location may also be selected or changed by the search user after the search query is provided.

The claims and description of the invention do not limit the number of users of search or content provisioning that can use the invention simultaneously or at different times. At a given time, there may be several users, or there may not be one user or one user. From a technical point of view, the disclosed apparatus may optionally be deployed dynamically, eg, from the hardware required in view of the processing capacity, storage, and communication capacity required to provide services to their users.

The term “time” herein may refer to a time interval such as a time within a predetermined time limit from an exact time (eg, per minute or per second) on another date or from another time. The term “location” may refer to a specific location, such as coordinates, or an area, such as an area, which may be specified to exist within a predetermined distance, for example, from a city or a specific location.

The term "plurality" refers to two or more herein. The term "number" refers to one or more.

The illustrative embodiments presented herein are not to be construed as limiting the applicability of the appended claims. The verb “comprises” is used herein as a public limitation that does not exclude the presence of features not cited. The features recited in the claims are freely combinable with one another unless expressly stated otherwise.

The novel features regarded as features of the invention are described in particular in the appended claims. However, with respect to the additional object and advantages of the present invention itself, its configuration and operation method will be best understood from the following description of specific exemplary embodiments when read in conjunction with the accompanying drawings.

The considerations set forth above with respect to various embodiments of the apparatus may be flexibly applied to embodiments of the method, as understood by those skilled in the art, and vice versa.

In the following, the invention will be described in more detail with reference to exemplary embodiments according to the accompanying drawings.
1 illustrates an exemplary configuration in accordance with one embodiment of the present invention,
2 illustrates an example application view of a first user interface in accordance with an embodiment of the present invention;
3 illustrates an example application view of a second user interface in accordance with an embodiment of the present invention;
4 illustrates steps that can be executed in a method according to an embodiment of the invention.

1 illustrates an example apparatus 100 that optionally includes one or more server computers and / or other network-side elements, for example, for implementing network services for the distribution of video in accordance with one embodiment of the present invention. Include.

More specifically, the apparatus 100 includes a processor 102 that receives a video item from a first user device 104 that is a content providing user device, and the user device 104 includes a mobile device such as a mobile phone or tablet. Can be. In other embodiments, ready captured complete items, such as video files, or 'clips' may be received after the capture step, while in some embodiments, the video items are, for example, in a substantially real time manner during capture. May be received from device 104 as a stream. The processor 102 may actually reside on at least one remote server, and the receipt of a video item is realized using, for example, wireless communication at least partially between the server or processor 102 and the user device 104. Can be. For example, it may include a memory for storing data in at least one database 122, server, or device 100 in general, and for communication purposes, wireless or wired transceivers, for example, It may include a communication interface, such as a transceiver operable in a local area network (LAN) network. Indeed, the device 100 may be connected to a communication network, such as the Internet, via an interface, where the device 104 may also be accessible via a wireless link of an associated wireless, optionally cellular or Wi-Fi type network.

In addition to a processor, memory and interface (eg, wireless transceiver), the first user device 104 may also include a plurality of sensors, at least a camera 106 (image sensor that captures visible and / or invisible, such as infrared frequency; The latter may provide thermal images, for example for temperature estimation and / or heat-based object / target positioning purposes) and further other sensors such as magnetometer 108, accelerometer 110, microphone 112, GPS sensor 114, and / or gyroscope 116, or still additional inertial sensors. Further applicable sensors include, for example, temperature, humidity, and pressure sensors. Sensor data from the sensors of the first user device 104 (sensor data obtained when capturing a video item and associated with a capture moment in the video item or associated with a general video item, possibly according to a particular data item) is processed by the processor. May be received by 102. In addition, other data may be received by the processor 102 via the first user device 104, such as temporal data (capture time, duration, etc.) associated with the video item.

In some embodiments, the apparatus may additionally or alternatively include one or more auxiliary content providing apparatus or auxiliary user device 105.

One or more devices 105 external to the first user device 104 may include one or more sensors that acquire data and transmit data to the first user device 104 via wired or wireless communication, The data may be transmitted, for example, as a stream or as a stream or as a pre-captured complete item (eg a file). The first user device 104 then transmits (as such or in adapted form) data and possibly other data, for example, from a sensor included in the first user device 104 to the processor 102. Can be.

In some embodiments, one or more secondary devices 105 may additionally or alternatively directly transmit data from one or more sensors thereof to processor 102.

As an example, drones, other unmanned, optionally aerial, vehicle or generally some other type of sensor / camera carrying (user) devices may be used to supply additional data such as video data and optionally sensor data. The item may be sent to the first user device 104. Thus, such data originally potentially obtained from the connected video capture device 105 may be received by the processor 102 via the first user device 104.

In one embodiment, the apparatus 100 includes a first user application 118 connected with the first user device 104. The first user application 118 can provide a first user interface 120, which enables the recording of video content by the first user device 104, and the video content and / or sensor data obtained. Information can be provided.

The first user device 104 may, in some embodiments, be configured to acquire video data and supplemental data and provide such data to the processor 102 via, for example, a memory card, for example a single lens reflective or mirrorless exchangeable. It may be a lens camera. The memory card may be configured to send the data described above to the processor 102, for example via wireless means. In this embodiment, the apparatus 100 and related invention may be utilized via the first user device 104, which includes, for example, an application that is essentially included in the apparatus 100. The memory card may enable a plurality of first user devices 104, such as a camera, to be used with the apparatus 100.

Video items received from the first user device 104 may be stored in a database 122 that the processor 102 may access, both of which may be included in the server apparatus.

The processor 102 may be configured to perform automatic tagging (eg, searchable keyword or other descriptor assignment) of at least one received video item using data received via the first user device 104. have. Herein, the data may refer to data obtained via a sensor of the first user device 104 and additionally other data that may be received from the first user device 104 in connection with the video item, such as the time when the video item was captured. May refer to data. In an advantageous embodiment, at least time is added to the metadata file, field, portion or other data element associated with the video content, ie the video item is tagged with at least temporary information.

Processor 102 may include or access sensor data analysis module 124, which may be used to analyze received video items or other data content, and may further be used for automatic tagging. Alternatively or additionally, some tagging may occur at the user device 104. The sensor data analysis module may include means for analyzing different types of sensor data or metadata associated with the received video item.

For example, information from the GPS sensor 114 can be used to determine where the video item was captured. In an advantageous embodiment, in addition to the time information, the received video material may be automatically tagged with data received via the GPS sensor 112, ie information about where the video content was captured. Sensor-based data, such as GPS, may be included in the tag as it is, and / or higher level location data may be, for example, addressed when events differ from temporal / time data and schedule data associated with their location. , Landmarks, regions, buildings, cities, countries, and / or event data.

Sensor data from sensors such as camera 106, magnetometer 108, and / or accelerometer 110, and / or gyroscope 116 may be analyzed by sensor data analysis module 124, for example. For example, metadata and metadata-based tags related to the field of view or viewing angle, or the 'viewing direction' of the captured video content can be generated.

In some embodiments, sensor data may be used to determine metadata that indicates the quality of the video material. The apparatus 102 may determine via the sensor data analysis module 124 if, for example, the first user device 104 is shaking while capturing video material. This information can be used for tagging. In some embodiments, the above information may also be used to automatically improve video quality, for example by a stabilization algorithm. Additionally or alternatively, in some embodiments, shake can be detected directly from the captured video data based on image analysis.

Other quality associated data or similar parameters such as video resolution, brightness and / or acuity (clarity) may also be determined by the device 100 through analysis of the video item and / or sensor data associated therewith. The parameter may be further analyzed to determine, for example, a characteristic value of a scale or other criterion that may be employed to indicate the quality of the video item. Video items may also be tagged with respective quality information for search and / or editing purposes.

Preferably, one or more quality parameters may be evaluated for the video sub-item in addition to or instead of the general parameters for the entire item. For example, a video item may be associated with a time duration and may be divided into two or more subitems that may be associated with a temporal duration. These sub durations may be the same or different lengths (durations) from one another. For example, the sub item may be a video segment or a video frame. In some embodiments, device 104 or processor 104 is configured to divide into sub-items based on, for example, recognized video (image) content, quality, and / or other parameters of the base video or related sensor data. Can be. In some other embodiments, the subitems may be automatically generated in response to user input, for example, each recording pause indicated by the user via the UI may be converted into a transition to a new subitem. . The subitems may be determined to have discrete or overlapping durations for the entire item. In one embodiment, each video subitem is assigned at least one quality parameter.

In view of the foregoing, sensor data analysis module 124 may include means for image analysis, which may utilize various methods of computer vision and pattern recognition. Image analysis can be used, for example, for detection, classification, and / or identification, and more complex image analysis is also possible, such as face recognition, for example, objects appearing in video content. Accordingly, relevant metadata such as tags can be associated with the video item. Image analysis also provides information on semantic content that other sensor data, such as auditory data from microphone 112, can be detected or determined through video data and possibly other sensor data while analyzing complex events. The method used may further comprise.

In one embodiment, the device 100 may include a video editing module 126, which combines two or more video items or subitems of one or more video items to produce a compiled video. can do. The video item may be received from one or more different first user devices 104 owned by one or more users. Video editing module 126 may be used to generate, for example, a compiled video of a video item or sub-items of a desired (user-selected) kind, for example, an event, for example, Video items or sub-items with quality criteria that meet or exceed the limit may be used or at least preferred over other items / sub-items. Compiled video may also be generated in which an event or object of interest can be viewed or viewed from various angles. The user requesting the edit can preferably define the properties of the edit to be set based on the available video data, which may possibly be of a quality characteristic, a temporal characteristic (e. G. May have or should be displayed more linearly or strictly in chronological order in time, preferences related to viewing angle / shooting direction / shooting position, target object or person preference, other effects related to slow motion or preferences (e.g. transient effects) ) Use as much as possible.

The video editing module 126 enables the compilation of video from events with various video items or subitems featuring entities that can be defined through image analysis. For example, an editorial video from a particular sporting event may feature a particular player, or an editorial video of a particular cultural event, such as a concert, may feature a particular artist.

In video editing module 126, the creation of edited video may be further enabled through the use of sensor video analysis module 124. Video subitems, such as segments or frames, can be analyzed separately, and high quality features of an event or entity of interest when the video subitems appear in chronological order because temporal information is also used in the device 100. An editing video can be generated.

The video editing module 126 may be integrated into edited video data from a plurality of video (image) sensors included in the first user device 104. Alternatively, data from another data source or additional device of device 104 may be integrated with edited video. For example, if a video edit is generated from an event such as a concert, other cultural event, or sporting event, sensor data such as sound from microphone 112 may be added as a sound track to the video. The microphone 112 or other sensor data may be derived from the same first user device 104 used to capture video content or other first user device 104. In addition, external data such as an external sound track can be added.

In other cases, the device 100 may determine the semantic content or context of the video through video editing module 126 and image analysis, automatically generate an edited video, and optionally external data associated with the detected context. It may also include. For example, device 100 may identify an event or situation that occurs in the video or video, for example, adding video tracks where video content is captured during the wedding and tagged with wedding type music.

In an advantageous embodiment, the processor 102 may receive a search query for video or editing via the second user device 128 using the second user application 130 and the second user interface 132 if possible. . The search query may cause the processor 102 to perform a search and identify video items stored in the database 122 that match the search query. The processor 102 may provide output to communicate the search results with the user of the second user device 128 and to facilitate the user's access to the video item search results.

In one embodiment of the invention, the search query includes at least time and / or location. Any number of other search criteria may be added. Preferably, at least one of the search criteria is associated with metadata or associated tags received or generated by the device 100. The search query may also indicate the name of the event, for example a concert or sporting event.

Through the second user interface 132, the output provided by the device 100 may indicate characteristics such as the quality of the video item search results. The quality may be related to a quality parameter obtained through analysis of at least one of the video item and / or other sensor data.

2 shows two views, 2a and 2b, of an example application view 200 showing a first user interface 120, which may be provided through the first user application 118, which is a first user. The interface 120 is intended to be used with the first user device 104. The first user interface 120 may essentially include a multi-function recording feature, for example in form or shape, for example, the button 202 includes an outer portion such as an outer rim 204. can do. In the embodiment of FIG. 2, the multifunction record button 202 is implemented as a touch-sensitive icon shown on the screen of the first user device 140 and displayed accordingly, such that when the multifunction record button 202 is pressed, Recording of video content with one user device 104 may be initiated or terminated. In some embodiments, initiation or termination of video recording may be performed through other methods, such as an auditory method based on analysis of microphone-captured sound data, such as voice commands.

In one embodiment, the outer rim 204 visually provides the content providing user of the first user device 104 with information about the video content being recorded by the first user device 104 or that may be recorded at a particular moment. Can be directed. The information may be related to quality, for example. For example, a color or pattern may be provided on the outer rim 204 that may indicate video quality and / or some other video related characteristic or information.

In one embodiment, the first user application 118 may communicate information associated with the video content with the user of the first user device 104 via haptic feedback initiated through the first user device 104. have. For example, if video is being recorded by the first user device 104 and the first user device 104 is tilted during recording, the haptic feedback can be generated by the first device 104 via, for example, a vibrating element. You can tell the user to tilt. This may be advantageous, for example, in situations where video content is being recorded by a portable device such as a mobile phone and the user of the device is not constantly watching the device or the recorded video content, they still obtain high quality video material. You may want to focus your attention elsewhere or focus your attention elsewhere.

In FIG. 3, an exemplary depiction of the application view 300 of the second user interface 132 is shown. 3 specifically illustrates a possible schematic appearance of an application view that may include output that may be provided by the device 100 in response to a search query. Application view 300 may include, for example, a map or other location-directed view 302. The view 302 may show an area representing a geographic location specified through a search query, or an explicit (eg location name, address, coordinate, etc.) location or implicit (eg event) specified in a search query. The region including the phosphorus position can be shown. The area to be shown may be predetermined in relation to one or more parameters such as coverage. For example, map view 302 may initially be designated to show an area comprised of one square kilometer. In a preferred embodiment, the search user may manipulate the map view 302 and / or zoom in and out, for example, via the UI to navigate the surrounding area.

The map view 302 may visually indicate a geographic point corresponding to a location where at least one first user device 104 has video content captured at a particular time. This is done, for example, by placing points, circles, or other pointers 304 on the map view 302.

The map view 302 can also visually indicate the quality of the video material. In one embodiment, the quality parameter for the corresponding video subitem may be directed to the user, for example, via a color or pattern that may be displayed on the map view 302, for example. . In addition, other indicators may be used, such as percentages or numbers.

The field of view 306, the shooting direction, and / or similar characteristics of the video item at a particular time are also visually and preferably indicated dynamically on the map view 302, eg, changed according to, for example, the video 308 shown. It may also be an indication of the quality parameters discussed above. This implementation is actually shown in FIG. 3.

Application view 300 may also show a preview 308 of one or more video items in one embodiment. A preview of one or more video items indicated in the map view 302 may be shown. The video item for which the preview is currently provided may be selected by the device 100 according to some criteria, or selected by the search user, for example by clicking on a circle 304 or other visual indicator associated with the video item. Can be.

The application view 300 may further include a timeline 310, through which the search user may manipulate the video items shown in the map view 302. For example, the search user may manipulate the timeline to proceed directly or incrementally to a specific time, and the map view 302 and preview (s) 308 may change accordingly. The timeline is also allowed to proceed in real time, while preview (s) 308 can play the selected video item (s) accordingly, while map view 302 can also be changed accordingly.

Other methods of manipulating the timeline 310 may be run on the timeline at a faster time scale than real time, or with the preview (s) 308 and map view 302 changed accordingly, or for example, "fast forwarding". Make it possible. The timeline can also be steered in the opposite direction, ie move back in time on the timeline.

In one embodiment of the invention, the video item may be provided to the search user as a livestream. Thus, timeline 310 may essentially show the current time, and the search user may observe in the application view 300 the video item being captured substantially in real time.

Video items may be stored in a database if they are provided live. Thus, a video provided as a livestream can later be retrieved and viewed as previously disclosed.

Due to the vast amount of video material available, the search criteria can yield a large number of video item search results that are too large to actually be considered, for example based on user-defined settings. In one embodiment of the invention, it may be possible to filter the search results, for example, via an area density filter. For example, it can be specified that the output can provide only a certain number of video items captured within a particular area or location in essence. For example, you can specify that only one video item per 5 square meters is shown in the output, and the video item shown here meets the search criteria representing the highest quality and / or other criteria, which the user can advantageously adjust. Can be. You can also specify other specifications to filter the search results, such as showing only video items with a specific field of view or shooting direction in the output. In general, filtering may be performed in different embodiments when configuring the second user application 130 and the second user interface 132, or by a search user, for example, a second user interface ( 132).

4 shows the steps that can be performed in a method according to an embodiment of the invention. At 402, one or more video items are received from multiple first user devices, whereby the video items are stored in database 122 at 404. Supplemental data is received at 406, which data is provided through a device in which the at least one video item was recorded. In a preferred embodiment, the data may be provided by one or more sensors included in the first user device (s) 104. The data may also include temporal data (time, timecode, lock, synchronization, etc.) provided through the first user device (s) 104.

In an advantageous embodiment, one or more video items related to the initial duration are divided into a plurality of discrete or overlapping subitems associated with the corresponding sub duration at 408 as previously considered.

At 410, metadata is preferably provided in connection with a video item or even a designated subitem (eg, a subitem-specified dedicated quality indicator), the metadata being obtained using the received supplemental data. The metadata may be generated by the sensor data analysis module 124.

In some embodiments, metadata may be assigned (associated) to one or more video items and / or related subitems as related searchable and preferably also checkable tags. For example, one or more quality indication parameters may be assigned to each video sub-item to enable quality-based retrieval.

A search query may be received at 412, and a video item search result that matches the search query may be determined at 414, and then output may be provided at 416 indicating the video item search result. In some embodiments, the output may include at least a list of video items of the result group that may include at least one video item. In some embodiments, the output may include the actual video item (s).

In some embodiments, video editing typically based on a plurality of original discrete video items that meet user-defined criteria such as time-space criteria (time, location, event, etc.) may be established in a fully automatic or user-assisted manner. Can be. The edit may be provided as an output video item. Compilation can include a single video stream or a video file, preferably consisting of component (source) video items that best meet the criteria.

The present invention has been described above with reference to the above-described embodiments, and various advantages of the present invention have been described. It is apparent that the present invention is not only limited to these embodiments but also includes all possible embodiments within the spirit and scope of the invention and claims.

The features recited in the dependent claims are freely combinable with one another unless explicitly stated otherwise.

Claims (21)

An apparatus 100 for retrieving digital video material,
Optionally includes one or more network accessible servers,
The device,
Receive a number of video items,
Storing the video item in a database,
Providing metadata, preferably a plurality of tags, for any or each video item, wherein the metadata is via using supplemental data provided by at least one user device 104 from which the video item was obtained, respectively. And wherein the user device comprises a user terminal capable of automatically capturing video from a connected video capture device 105, such as a drone, and / or receiving and forwarding video data, for example.
Receive a search query, wherein at least one search criterion included in the search query is related to the metadata;
Determine a video item search result that matches the search query among the at least one video item stored in the database, and
At least one processor (102) configured to provide an output to a user that includes the video item search results,
The video item is associated with a duration, the provided metadata includes at least one quality indicator determined through analysis of the supplemental data associated with one or more sub-items of the video item,
And the sub item is associated with a sub duration of the sub item and the at least one quality indicator is assigned to a corresponding sub item. The method of claim 1,
And the video item is divided into a plurality of sub durations to provide a plurality of sub items, wherein the at least one quality indicator is determined and assigned to each of the sub items. The method according to claim 1 or 2,
And the output indicates the at least one quality indicator. The method according to any one of claims 1 to 3,
At least a portion of the metadata is obtained through using data provided by one or more sensors,
At least one of the sensors comprises a camera, a thermal imaging camera, an image sensor, magnetometer, accelerometer, microphone, position sensor, phase positioning based position sensor, GPS sensor, pressure sensor, temperature sensor, humidity sensor, and gyroscope And 100 for retrieving digital video material. The method according to any one of claims 1 to 4,
And the metadata includes an indication of a time and / or a location obtained via the user device. The method according to any one of claims 1 to 5,
And the search query includes at least time and location as search criteria. The method according to any one of claims 1 to 6,
The output includes a plurality of selected, optionally user-selected plurality of video item search results within a selected, optionally user-selected location range,
And all video item search results within the location range are arranged according to the selected criteria. The method of claim 7, wherein
And the criterion is related to video quality. The method according to any one of claims 1 to 8,
The apparatus (100) for retrieving digital video material for the video item is configured to determine metadata, such as tags, indicating an event associated with the item, optionally a cultural, sporting or party event. The method of claim 9,
The metadata indicative of the event is determined based on video image data of the item, time data associated with the item, location data associated with the item, microphone based sound data, and / or other sensor data. Device 100 for searching. The method according to claim 9 or 10,
And metadata indicative of the event is determined based on event information characterized by the location and time of at least one event. The method according to any one of claims 1 to 11,
The apparatus is configured to provide a map 302 that includes a location indicated in the search query, and a timeline 310 that includes a time indicated in the search query,
And the map indicates a number of points (304) at which a corresponding number of video items were captured at a time corresponding to the time indicated on the timeline. The method of claim 12,
The output 306 is an apparatus 100 for retrieving digital video material that provides information regarding the angle of view, the shooting direction and / or the direction in which the video material of the video item was captured. The method according to any one of claims 1 to 13,
The apparatus is configured to provide an edit composed selectively from two or more video items in view of the search query in selecting the video item and / or determining the contribution of the video item to the edit. Apparatus 100 for retrieving data. The method according to any one of claims 1 to 14,
The apparatus includes the at least one user device provided with a user interface 200,
The user interface preferably includes multifunction record-buttons 202 and 204 as on-screen symbols, the record button indicating the quality of the video material recorded at a particular time. Device 100. The method of claim 15,
And the user interface is configured to enable relocation of the record-button by a user of the user device. The method according to any one of claims 1 to 16,
The apparatus (100) for retrieving digital video material instructing a user of the user device via haptic feedback of a change occurring in captured video content. The method of claim 17,
And the change is related to an angle of view or shooting direction relative to the camera of the capture device, and video content captured by the camera of the capture device. As a way to search for video material,
Receiving 402 a plurality of video items,
Storing the video item in a memory (404), and
Providing 410 metadata for any or each video item, wherein the metadata is obtained through using supplemental data provided by at least one user device from which the video item was obtained 406. ,
The video item is associated with a duration and the provided metadata includes at least one quality indicator determined through analysis of the supplemental data associated with one or more subitems of the video item (408),
The sub item is associated with a sub duration of the sub item and the at least one quality indicator is assigned to a corresponding sub item;
How to retrieve video material. The method of claim 19,
Receiving (412) a search query, wherein at least one search criterion included in the search query is associated with the metadata;
Determining a video item search result that matches the search query (414), and
Providing 416 the user with an output comprising the video item search result,
How to retrieve video material. 21. A computer program comprising program code means adapted to execute a method according to claim 19 or 20 when the program is run on a computer.

Images