WO2020028583A1 - Systèmes et procédés de génération de métadonnées décrivant des objets de données non structurés dans un dispositif périphérique de stockage - Google Patents

Systèmes et procédés de génération de métadonnées décrivant des objets de données non structurés dans un dispositif périphérique de stockage Download PDF

Info

Publication number
WO2020028583A1
WO2020028583A1 PCT/US2019/044534 US2019044534W WO2020028583A1 WO 2020028583 A1 WO2020028583 A1 WO 2020028583A1 US 2019044534 W US2019044534 W US 2019044534W WO 2020028583 A1 WO2020028583 A1 WO 2020028583A1
Authority
WO
WIPO (PCT)
Prior art keywords
metadata
data
storage
media object
object data
Prior art date
Application number
PCT/US2019/044534
Other languages
English (en)
Inventor
Konstantin Kudryavtsev
Noam Mizrahi
Mats Oberg
Nedeljko Varnica
Nitin Nangare
Igor POLIVANYI
Ruwan Ratnayake
Leo JIANG
Quynh CHAU
Wen Lung Chang
Original Assignee
Marvell World Trade Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/263,387 external-priority patent/US11068544B2/en
Application filed by Marvell World Trade Ltd. filed Critical Marvell World Trade Ltd.
Priority to KR1020217004932A priority Critical patent/KR20210037684A/ko
Priority to JP2021504383A priority patent/JP2021533446A/ja
Priority to EP23174241.2A priority patent/EP4220437A1/fr
Priority to CN201980058977.XA priority patent/CN112673368A/zh
Priority to EP19756453.7A priority patent/EP3830714B1/fr
Publication of WO2020028583A1 publication Critical patent/WO2020028583A1/fr
Priority to JP2024000847A priority patent/JP2024038276A/ja

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/41—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G06F12/1027—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
    • G06F12/1045—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB] associated with a data cache
    • G06F12/1054—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB] associated with a data cache the data cache being concurrently physically addressed
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G06F15/17306—Intercommunication techniques
    • G06F15/17331—Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/383—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/387—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
    • G06F16/68—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/683—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/9035—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/9038—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/068—Hybrid storage device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0688—Non-volatile semiconductor memory arrays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/90—Buffering arrangements
    • H04L49/901—Buffering arrangements using storage descriptor, e.g. read or write pointers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/25—Using a specific main memory architecture
    • G06F2212/254—Distributed memory

Definitions

  • This disclosure relates to generation of metadata describing media object data at the storage edge using a computational engine.
  • Typical systems for generating metadata describing unstructured data, such as media data may do so at location at which the data is initially received (ingest point), such as an ingest server or other centralized network processing location.
  • ingest point such as an ingest server or other centralized network processing location.
  • the ingest point may be physically collocated with a user of the system or may be a cloud storage
  • video data may be received from a camera or other video source at a server that is responsible for both transcoding the media and generating metadata describing the media.
  • These systems typically embed the metadata in a media data stream with the received or transcoded media data so as to require retrieval of the entirety of the media data in order to access all metadata describing the media data. This can result in an increased load at the ingest point due to volume of media data to be processed in order to embed or otherwise include the metadata therein.
  • Typical systems generate metadata describing media data at the ingest point or at a
  • Implementations described herein provide an apparatus and method for storing and managing media object data.
  • a storage control device coupled to a storage device and located at a storage edge remotely from a host device receives media object data from the host device.
  • the storage control device identifies a type of the media object data and selects, based on the identified type, a
  • the storage control device accesses the media object data using a data input interface.
  • Each computational model has associated input data interface parameters which identify a data format for which the computational model is configured.
  • the storage control device accesses the media object data using the input data interface parameters and converts the media object data to the identified data format.
  • each computational model has associated hardware interface parameters.
  • the storage control device retrieves the hardware interface parameters and applies the hardware interface parameters to a hardware interface.
  • the storage control device accesses the media object data from a first physical connection using the hardware interface, and stores the media object data and metadata in the storage device through a second physical connection using the hardware interface.
  • each computational model has associated decryption parameters.
  • the storage control device retrieves the decryption parameters and decrypts the media object data using the decryption parameters.
  • the storage control device retrieves media object data from the storage device for which metadata has already been generated using a first computational model.
  • the storage control device identifies the first computational and selects a second computational model to generate second metadata different from the first metadata.
  • the storage control device may store the second metadata so as to be selectively retrievable from the first metadata and from the media object data.
  • the storage control device may combine the first and second metadata to generate combined metadata and store the combined metadata so as to be selectively retrievable from the media object data .
  • the storage control device receives a request from the host device for media data containing objects which meet one or more criteria. In response to this request, in some implementations, the storage control device retrieves, from the storage device, the metadata describing media object data separately from the media object data.
  • the storage control device identifies, based on the retrieved metadata, one or more portions of the media data that contain objects which meet the one or more criteria.
  • the storage control device then retrieves, from the storage device, the identified portions of the media data containing objects which meet the one or more criteria, and responds to the request from the host by sending the retrieved portions of media data over a network connection without sending other portions of the media.
  • the storage control device generates a copy of the media object data and concurrently provides the media object data to the storage device for storage and provides the copy of the media object data to the computational engine for processing to generate metadata .
  • the storage control device stores the media object data in a logically separated manner so as to be selectively retrievable separately from one another.
  • FIG. 1 is a block diagram illustrating components and data flow therebetween for generating metadata
  • FIG. 2 is another block diagram illustrating components and data flow therebetween for generating metadata describing unstructured data objects in accordance with some implementations of the disclosure
  • FIG. 3 is a block diagram illustrating storage of metadata and media data in a non-volatile storage device in accordance with some implementations of the disclosure
  • FIG. 4 is another block diagram illustrating storage of metadata and media data in a non-volatile storage device in accordance with some implementations of the disclosure ;
  • FIG. 5 is a block diagram illustrating components of a computational engine in accordance with some embodiments.
  • FIG. 6 is a block diagram illustrating components of a data structure from which a computational model is extracted in accordance with some implementations of the disclosure ;
  • FIG. 7 is a flowchart representing a process for generating metadata describing an unstructured data object in accordance with some implementations of the disclosure
  • FIG. 8 is a flowchart representing another process for generating metadata describing a data object in
  • FIG. 9 is a flowchart representing a process for retrieving portions of a data object based on separately retrieved metadata describing the data object in accordance with some implementations of the disclosure.
  • Implementations described herein provide an apparatus and method for generating metadata describing unstructured data objects at the storage edge.
  • the storage edge refers to an environment that is local to a storage device such as one or more solid-state storage drives or hard disk drives.
  • a storage control device located at the storage edge is used to control storage and retrieval of data at one or more storage devices.
  • Each storage device may be coupled to, and controlled by, a respective storage control device.
  • several storage devices may be coupled to, and controlled by, a single storage control device, such as a storage aggregator as described in U.S. Patent Application No. 16/264,248, entitled "STORAGE
  • the storage control device receives an unstructured data object, such as a media object for which descriptive metadata has not yet been generated.
  • Control circuitry of the storage control device identifies a type, such as a data type, of the data object. Based on the type, a
  • computational model is selected from among a plurality of computational models.
  • a computational engine for instance a computational engine that is local with respect to the storage edge or the storage device, uses the selected computational model to generate metadata describing the data object.
  • the storage control device provides the metadata to the storage device for storage within or in association with the data object so as to be selectively retrievable
  • the metadata may be embedded into the data object in a header or other discrete portion of the data object which can be retrieved separately from the data object.
  • the data object may include video data and the metadata may be generated as a text file.
  • the storage control device combines the metadata text and video data into a single file having a logically separated text portion and video portion.
  • the metadata can then be separately retrieved by accessing only the text portion of the file.
  • the metadata may be interleaved with the data of the data object.
  • each keyframe contained in the video data includes some metadata describing the portion of the video data to which the respective keyframe applies.
  • the metadata may be stored in a
  • the data object may be stored in a first solid-state drive and the metadata may be stored in a second solid-state drive.
  • the data object may be stored in a first partition of a solid-state drive and the metadata may be stored in a second partition of the same solid-state drive. Any other suitable physical or logical separation of data may also be used.
  • a storage control device includes control circuitry and a computational engine, such as a suitable artificial intelligence (AI) or machine learning (ML) engine.
  • the storage control device is located at the storage edge and receives unstructured data, such as media objects, to be stored in a storage device.
  • the storage control device generates metadata for the media data of unstructured media objects and stores the metadata in the storage device so that it is selectively retrievable separately from the unstructured media data.
  • the term "storage edge” means a module or a component that is local to a non-volatile storage device.
  • a controller that controls the operation of one or more storage devices to store or retrieve data at one or more instances of a non-volatile memory is disposed on storage edge.
  • the storage edge is found for example in dedicated storage devices, or at storage networks, and is separated from a processor that is remotely located, for instance in a host computer or at a data center. Communication between the storage edge and a remote is host is over a computer network connection.
  • the terms “media data objects,” “data objects,” “media objects” or “objects” mean various types of data that is issued by an application running on a host system and can be stored on a storage device.
  • Examples of “media objects” or “objects” can include, but not limited to videos, sound recordings, still images, textual objects such as text messages and e-mails, data obtained from various types of sensors such as automotive sensors and Internet-of-Things (IoT) sensors, database objects, and/or any other suitable objects. In many cases, the media objects are unstructured.
  • IoT Internet-of-Things
  • unstructured object means that the media content (“content media data”) of the object (e.g., textual content, audio content, image content or video content) is provided in raw form and is not organized in advance according to a fixed field format.
  • An unstructured object is not tagged a-priori with metadata that defines any aspects of the content per frame or other content portion.
  • Unstructured data is non transactional, and its format does not readily conform to a relational database schema.
  • metadata refers to a high-level representation of the actual data content of media objects stored in a non-volatile storage device.
  • Metadata can be an abstraction layer of the actual data content, which gives a description or a meaning of data content in a compact format. Metadata can be generated from media objects, which are almost always unstructured, in various ways.
  • Example metadata can include labels, tags, types of data, ob ects/concepts/sentiments detected in data content, spatial/temporal locations of such
  • AI model refers to any suitable AI algorithm, e.g., implemented on a deep neural network or any recurrent neural network or any variation of those.
  • an AI model is suitably any other Supervised learning or Unsupervised Learning or Reinforcement learning algorithms.
  • An AI model is trained using a "training set" - a body of media objects and corresponding metadata that is known to be accurate. The trained AI model is then applied to generate metadata for other media objects.
  • AI engine A software or hardware module that receives a pre-trained AI model and uses it to compute metadata of objects is referred to herein as an "AI engine” or "AI interface engine.”
  • AI engine A software or hardware module that receives a pre-trained AI model and uses it to compute metadata of objects.
  • AI interface engine A software or hardware module that receives a pre-trained AI model and uses it to compute metadata of objects.
  • AI engine A software or hardware module that receives a pre-trained AI model and uses it to compute metadata of objects.
  • AI interface engine A software or hardware module that receives a pre-trained AI model and uses it to compute metadata of objects.
  • several different AI models will be applied to unstructured or partially structured media objects.
  • the storage control device receives the unstructured data from a host device over a network, such as a local area network (LAN) or a wide area network (WAN) using Internet or other suitable networking protocols.
  • unstructured data is received from a capture device (e.g., video recorder, audio recorder, sensor) locally connected to a computer.
  • the storage control device is incorporated into the storage device of the computer and receives data over a bus from other portions of the computer.
  • the storage control device creates a copy of the data object and stores the original data object in the storage device while using the computational engine to generate the metadata using the copy. After generating the metadata, the copy of the data object is no longer required.
  • the metadata is stored in a selectively retrievable manner from the unstructured data, and the copy of the data object is deleted, marked for deletion, overwritten, or otherwise removed from the storage control device.
  • Metadata is stored separately from the
  • metadata and unstructured data may be stored in one or more logically separable portions of a data file or other data structure.
  • the one or more portions containing metadata are structured as self- contained data files which can be accessed and read by the storage control device outside the context of the
  • the one or more portions containing metadata may be tagged or otherwise identified within the data file or data structure to allow the storage control device to easily locate and retrieve them.
  • the metadata identifies or otherwise includes an association with the unstructured data which it describes. This allows the storage control device to retrieve relevant portions of the unstructured data based on a review or analysis of the metadata.
  • retrievable manner from the unstructured data may allow greater efficiency in indexing and searching functions.
  • a search engine or indexer can access or selectively retrieve the metadata describing objects, without having to retrieve the media data. This selective retrieval reduces the amount of data needed to be accessed in order to retrieve metadata or relevant portions
  • unstructured data objects and reduces processor load when performing search or indexing functions, or retrieval of relevant portions of previously unstructured data such as relevant portions of media objects.
  • Generating metadata at the storage edge rather than at the host device also results in less data to be transferred between host devices, processors and the storage edge, which in turn results in faster overall data transfer speeds, either from the ingest point or a central data processing center.
  • metadata is generated at the ingest point, data processing center, or a cloud server (i.e., locations remote to the storage edge)
  • unstructured media data containing data objects must be transferred to the storage edge, and at times from a storage device to a remote processor, for instance at remote data center, to identify and/or tag various objects in the media.
  • the amount of time required to make relevant portions of the unstructured data available for retrieval from the storage edge is reduced by the difference between the amount of time needed to transfer to a data center facility large volumes of content data media needed to generate metadata using computational engines at the data center on the one hand, and the amount of time needed to transfer to the data processing facility relatively small amounts of data containing both metadata generated at the storage edge and those segments of content data that are relevant to address a particular need.
  • metadata is generated for surveillance video to tag the moods of people captured in the surveillance video, or the presence of specific people or activities captured in the surveillance video. Only the relevant frames with the desired persons, moods or activities may need to be sent (e.g. , to a data processing center or to a human operator) for further analysis.
  • Metadata is generated for audio recordings to tag specific speech sequences, such as segments of a conversation made at a robotic service desk.
  • the metadata is searched for specific speech sequences and only the relevant recording portions containing the relevant sequences, instead of an entire recording, are sent to a facility where they are analyzed and used to improve robot response in an automated
  • Second metadata is generated for the retrieved unstructured data using a second computational model that is different from the first computational model.
  • the computational model may be an updated version of the first computational model which more accurately generates the same type of metadata as the first computational model.
  • the second metadata is compared with the first metadata and any metadata of the second metadata that is different from corresponding metadata of the first metadata or any metadata of the second metadata not present in the first metadata are stored.
  • the second computational model may be configured to generate a different type of metadata than the first computational model.
  • the second metadata may also be stored to be selectively retrievable separately from both the previously stored unstructured data as well as
  • the second metadata is merged with the first metadata and the merged metadata is stored to be selectively retrievable separately from the previously stored unstructured media.
  • FIG. 1 is a block diagram illustrating components and data flow therebetween for generating metadata
  • unstructured data object 104 is received at the storage control device from a host device, such as an audio
  • control circuitry 106 is configured to receive, retrieve, or access the unstructured data 104 using an input data interface.
  • the input data interface is configurable to access various types of unstructured data as needed. Configuration of the input data interface can be achieved by applying input data interface parameters to the input data interface. As described below in connection with FIG. 6, input data interface parameters may be stored in association with a particular computational model configured to process unstructured data of a particular format. If the
  • control circuitry 106 is configured to convert the unstructured data 104 to the particular format to facilitate generation of metadata describing the
  • video data may be received in a compressed format, such as H.262, while the
  • the computational model is configured to process H.264 video data.
  • the input data interface parameters identify a set of video compression formats which can be converted to the required H.264 format and may additionally supply a format conversion tool for use by control circuitry 106 to convert the video data to the H.264 format.
  • image data may be received in JPEG format while the
  • the computational model is configured to process images in a bitmap format.
  • the input data interface parameters identify JPEG as a suitable input format for conversion to a bitmap format, and provide a conversion tool for use by control circuitry 106.
  • control circuitry 106 may receive, retrieve, or access unstructured data 104 using a hardware interface.
  • the hardware interface is configurable to access data from various physical or logical ports and/or connections of the storage control device 102. Configuration of the hardware interface can be achieved by applying hardware interface parameters to the hardware interface.
  • hardware interface parameters may be stored in association with a particular computational model.
  • control circuitry 106 applies the hardware interface parameters to the hardware interface.
  • the hardware interface parameters may specify a particular logical port of a network connection, such as TCP port 20 (FTP) , 80 (HTTP) , or 115 (SFTP) .
  • the hardware interface parameters may specify a physical port or
  • connection such as an ethernet port, serial port, USB port, or other data port through which the storage control device 102 may receive unstructured data.
  • the hardware interface listens to the specified port and allows control circuitry 106 access to data received through the specified port .
  • unstructured data 104 is encrypted for transmission to the storage edge. To allow processing of the unstructured data 104, control
  • circuitry 106 decrypts the unstructured data 104 using decryption parameters. As described below in connection with FIG. 6, decryption parameters may be stored in
  • control 118 In response to selection of a particular computational model (e.g., in response to instruction 118, which may be received prior to receipt of the unstructured data) , control
  • circuitry 106 retrieves the decryption parameters.
  • Decryption parameters may include a predefined decryption key or decryption key generation routine such as an RSA algorithm. Control circuitry applies the decryption key or generates and applies a decryption key using the RSA algorithm to access the unstructured data 104.
  • control circuitry 106 of storage control device 102 Upon accessing or receiving unstructured data 104, control circuitry 106 of storage control device 102 creates two identical data streams. A first data stream contains the unstructured data 104 as originally received, and a second data stream contains an identical copy of the unstructured data 104 generated by control circuitry 106. Control circuitry 106 then stores the first data stream 108 in non-volatile storage 110.
  • Non-volatile storage 110 may be any non-volatile data storage device, such as a hard drive, Flash memory array (e.g. , NOR-Flash memory or NAND- Flash memory) , or any other suitable type of data storage medium.
  • the above-described decryption and/or format conversion may be performed on the unstructured data 104 as initially received, or on only the second data stream to be processed by computational engine 116.
  • Control circuitry 106 also determines a data type of the unstructured data, such as video data, image data, audio data, or any other data type. Control circuitry 106 directs the second data stream 114, which is identical to data stream 108, to a computational engine 116 of the storage control device 102. Computational engine 116 generates metadata using a computational model. For example, computational engine 116 may be a machine learning, artificial intelligence or other suitable metadata
  • computational engine 116 may be AI models and may employ matrix multiplication operations, convolutional operations, other operations defined in neural networks, or any other suitable data processing methods or operations.
  • Storage control device 102 selects a computational model to be used by computational engine 116 and transmits an instruction 118 to computational engine 116 to use the selected model.
  • Storage control device 102 selects a computational model corresponding to the determined data type, or a type of metadata that is desired to be extracted from the content. For example, some computational models are suitable for identifying faces in an image frame or voices in a segment of a sound recording, while other computational models are designed, for example, to identify an action or a mood of a person in an image frame. Some computational models identify a location of an object within a frame or a stream of data. Some computational models are configured to identify a plurality of different characteristics of segments of the media data.
  • Computational engine 116 processes the copy of the unstructured data, in an implementation, to segment the unstructured data into frames, generate metadata describing, for instance, content and locations of the content in the unstructured data.
  • the metadata 120 is then stored in non volatile storage 110 to be selectively retrievable
  • the hardware interface parameters may specify a physical or logical address of a particular server or data storage device in which to store the media object data and metadata.
  • the hardware interface parameters may specify a particular port, such as an ethernet port, serial port, USB port, or other data port through which storage control device 102 may access non-volatile storage 110.
  • FIG. 2 is another block diagram illustrating components and data flow therebetween for generating metadata describing unstructured data objects in accordance with some implementations of the disclosure. If no metadata has been generated for an unstructured data object that has already been stored in non-volatile storage 110, or if additional metadata is to be generated for an unstructured data object, the storage control device 102 retrieves the content media data 200 from non-volatile storage 110.
  • Control circuitry 106 identifies a data type of the media data 200, as described above in connection with FIG. 1.
  • Control circuitry 106 provides content media data 202 to computational engine 116 for processing.
  • the content data is unstructured, while in other scenarios the content has been partially structured with partial metadata having been previous generated for the content.
  • Storage control device 102 selects a computational model and transmits an instruction 204 to computational engine 116 to use the selected computational model.
  • storage control device 102 selects a computational model based on the determined data type of media data 200. If metadata describing the unstructured data object already exists, control circuitry 106 determines a first computational model that was used to generate the metadata and selects a second computational model to be used. For example, metadata may have already been generated using a computational model for identifying faces in surveillance video. Control circuitry 106 may select, as a second computational model, a computational model for identifying facial expressions, actions, moods, or voices in the same surveillance video. The metadata 206 generated using the second computational model is then stored in non volatile storage 110 to be selectively retrievable
  • the metadata generated using the first computational model and the metadata generated using the second computational may be stored separately from one another in order to be selectively retrievable.
  • the metadata generated using the second computational model may be combined with the metadata generated using the first computation model to generate a single set of metadata to be stored in a
  • FIG. 3 is a block diagram illustrating storage of metadata and media data in non-volatile storage 110 in accordance with some implementations of the disclosure.
  • Metadata and media data are stored within a single data file 300 in non-volatile storage 110.
  • Data file 300 is of a file format which can host both text-based metadata and unstructured media data in a metadata portion 302 and a media data portion 304, respectively.
  • Data stream 108, containing unstructured media data is received at non volatile storage 110 and data file 300 is created to contain the unstructured media data, for example in media data portion 304.
  • Metadata 120 generated by computational engine 116, is also received at non-volatile storage 110.
  • Non-volatile storage 110 stores the metadata in metadata portion 302 of data file 300.
  • metadata portions 302 are stored separately from the media data portions 304.
  • respective metadata portions 302 and media data portions 304 are stored together, for instance interleaved, but so as to be mutually separable.
  • Data file 300 may be created with both metadata portion 302 and media data portion 304.
  • data file 300 may be created with only media data portion 304 and, upon receipt of metadata 120, metadata portion 302 may be appended to, prepended to, concatenated or interleaved with media portion 304.
  • FIG. 4 is another block diagram illustrating alternative storage of metadata and media data in non volatile storage 110 in accordance with some implementations of the disclosure.
  • Non-volatile storage 110 may include physically separate storage areas (e.g., several hard disk drives or solid-state drives) or logically separate storage areas (e.g., several partitions on a single hard disk or solid-state drive) .
  • Non-volatile storage 110 may designate a first storage area 400 for metadata storage and a second storage area 402 for data object storage.
  • Data stream 108 is received at non-volatile storage 110.
  • Non-volatile storage 110 determines that the received data stream 108 contains unstructured media data and stores the unstructured media data in data object storage 402.
  • Non-volatile storage 110 also receives metadata 120, generated by computational engine 116.
  • Non-volatile storage 110 may include physically separate storage areas (e.g., several hard disk drives or solid-state drives) or logically separate storage areas (e.g., several partitions on a
  • Metadata 120 determines that the data received is text-based metadata and stores the metadata 120 in metadata storage 400.
  • storage of metadata 120 to be separable from media data 108 allows for metadata 120 to be retrievable separately from media data 108, which in turn results in increased efficiency in accessing metadata 120, or specific portions of media data 108 based on metadata 120.
  • computational models are stored in respective data structures in a memory of the storage control device 102.
  • Computational engine 116 or control circuitry 106 extracts the selected computational model from its respective data structure for use by
  • FIG. 5 is a block diagram illustrating components of a computational engine in accordance with some implementations of the disclosure.
  • Computational engine 116 includes a data structure runtime module 500.
  • Data structure runtime module 500 provides an environment in which the contents of a data structure can be executed by or applied to the computational engine 116.
  • data structure runtime module 500 allocates virtual system resources needed for the particular data structure. In other words, data structure runtime
  • module 500 provides an application layer in which the contents of the data structure, as described below in connection with FIG. 6, can be executed using hardware of the computational engine 116, such as CPU 506 and hardware accelerator 512.
  • Computational engine 116 also includes orchestration agent 502. While the data structure runtime module 500 is responsible for executing a data structure (such as data structure 514n as described below in
  • orchestration agent 502 determines which data structure to execute. Orchestration agent 502 receives a command, such as instruction 118, to select a particular data structure, or receives an indication of the determined data type of the unstructured data object to be processed and determines an appropriate data structure.
  • a command such as instruction 118
  • Orchestration agent 502 transmits a request 516 to a memory of the storage control device 102 to retrieve a data structure (e.g., data structure 514b) from among a plurality of data structures (514a, 514b, 514c, ... 514n) .
  • the memory of storage control device 102 transmits response 318 containing the selected data structure 514b, which is then loaded by computational engine 116 into data structure runtime module 500.
  • Data structure runtime module 500 extracts a computational model from data structure 514b.
  • computational model 520 into a volatile memory 510 of the computational engine 116.
  • Data structure runtime module 500 may also extract other data and parameters from the selected data structure needed to process the media data using the extracted computational model 520.
  • Computational engine 116 includes an operating system 504, running on a CPU 506, in an implementation, through which the data structure runtime module 500 and orchestration agent 502 perform the actions described above.
  • computational engine 116 further includes a hardware accelerator 512.
  • Computational models include many thousands of operations which cannot be efficiently executed on CPU 506.
  • Computational engine 116 runs the extracted computational model using the hardware accelerator 512.
  • FIG. 6 is a block diagram illustrating components of a data structure from which a computational model is extracted in accordance with some implementations of the disclosure.
  • data structures such as data structure 514n, include several other items and parameters which may be used by computational engine 116 in processing unstructured data objects.
  • input data interface 602 provides parameters for input format of the unstructured data.
  • the computational model contained within a particular data structure may be configured to process images in a bitmap-type format. If the media object data supplies images in a compressed format, such as JPEG, the computational engine 116 must convert the compressed images to bitmaps.
  • Media object data may be encrypted.
  • the media data may be subject to security concerns or copyright. Data
  • encryption/decryption 604 provides encryption or decryption algorithms and keys needed to access the unstructured data, for example by decrypting the data.
  • Data pre-processing module 606 provides mechanisms for preparing the
  • unstructured data for processing such as conversion of the unstructured data as required by the input data
  • data pre processing module 606 may provide mechanisms for extracting individual video frames or audio samples from the media object data.
  • Data post-processing module 608 provides parameters for formatting of the metadata generated by the computational model 600, in an implementation. For example, data post-processing module provides a particular data structure and/or data fields in which metadata relating to the media object data is to be placed. The data structure or data fields are specific to both the type of data being processed and the configuration of the compiled
  • Hardware interface 610 provides parameters for accessing the data using particular
  • components such as a network interface or a serial data connection .
  • FIG. 7 is a flowchart representing a process 700 for generating metadata describing an unstructured data object in accordance with some implementations of the disclosure.
  • the unstructured data object is a content media data object.
  • storage control device 102 using control circuitry 106, receives the media object from a host device.
  • control circuitry 106 may include a network interface or other data input connection through which data is received from remote sources.
  • Remote sources include any host device or server at which the media object is generated and/or transmitted to the storage edge, such as a video recorder, audio recorder, ingest server, cloud server, and the like.
  • the storage control device 102 may receive, from the host device, a request to store the media object data in non-volatile storage 110.
  • control circuitry 106 creates a copy of the media object data, thereby resulting in two data streams
  • control circuitry containing the same content data.
  • control circuitry 106 provides one data stream to non-volatile storage 110 for storage.
  • control circuitry 106 identifies a data type of the media object data.
  • the media object data may be a video stream.
  • Control circuitry 106 may process, examine, or otherwise inspect packet headers to determine the type of data received from the host device. Control circuitry 106 may further determine a data format, such as MPEG-2, MPEG-4, or other video data formats. As another example, the media object data may be an audio stream. Control circuitry 106 may process, example, or otherwise inspect packet headers to determine the type of data received, and may further determine a data format, such as MP3, WAV, or other suitable audio data format. As a third example, the media object data may be sensor data capture from an imaging sensor such as an infrared sensor. Control circuitry 106 may identify the type of sensor from packet header data, such as a signature of the sensor.
  • control circuitry 106 selects, based on the identified data type, a computational model from among a plurality of computational models for use by the
  • control circuit 116 For example, control
  • circuitry 106 selects a computational model for performing image processing on successive frames of video data to identify objects depicted in each frame.
  • selection of a computational model by control circuitry 106 is further based on the type of metadata desired to be generated. For example, several computational models may be available for processing video data, each generating a different set of metadata. One computational model may identify faces in video data while another computational model identifies actions in video data. Control circuitry 106 may receive input from a user or from a program running at the storage edge to generate a specific type of metadata. Control circuitry 106 therefore selects a computational model configured to process video data and to generate the desired metadata. Additional concepts are described in U.S. Patent Application
  • control circuitry 106 provides the metadata to non-volatile storage 110 for storage so as to be selectively retrievable from non-volatile storage 110 separately from the media object data, using methods described above in connection with FIGS. 3 and 4.
  • the metadata can be retrieved separately from the media object data, thereby enabling the process described above to retrieve only the metadata generated by the first computational model independently of the media object data.
  • FIG. 8 is a flowchart representing another process 800 for generating metadata describing a data object in accordance with some implementations of the disclosure.
  • media object data that has already been processed using a first computational model to generate first metadata is processed again using a second
  • control circuitry 106 retrieves the first metadata describing the media object data previously generated using the first computational model. For example, control circuitry 806 may use query non-volatile storage 110 or metadata storage 400 using an SQL "SELECT" command, in response to which non-volatile storage 110 provides the first metadata. Alternatively, control circuitry 106 may access a file allocation table of non-volatile storage 110 to identify a memory location or memory address range at which the first metadata is stored and retrieve the first metadata from the identified location or address range.
  • control circuitry 106 identifies the first computational model used to generate the first metadata.
  • control circuitry 106 may analyze the type of metadata previously generated, or specific metadata fields contained within the first metadata with respect to one or more computational models, or an index of available computational models to identify a second computational model for processing the identified data type which produces the type of metadata or specific metadata fields contained within the first metadata. For example, if the first metadata contains fields describing faces and facial positions over time, control circuitry 106 determines that the first metadata was produced using a computational model configured to identify faces in video content. [0053] At 806, control circuitry 106 determines a type of metadata to be generated. Control circuitry 106 may determine whether an updated version of the first
  • control circuit For example, control circuit
  • control circuitry 106 identifies a date and time at which the first computational model was compiled or when the data structure containing the first computational model was first made available. If an updated version of the first computational model exists, control circuitry 106 determines that metadata of the same type as the first metadata should be generated. Alternatively, control circuitry 106 may automatically select a type of metadata to supplement the first metadata or may receive input from a user identifying a particular type of metadata to be generated. For example, control circuitry 106 identifies the first computational model as an image processing model for identifying objects, such as faces, depicted in video frames. Control circuitry 106 then determines that metadata identifying additional
  • control circuitry 106 selects a
  • control circuitry 106 determines that the type of metadata to be generated is the same as the type of the first metadata, indicating that an updated version of the first computational model exists, control circuitry 106 selects the updated computational model.
  • control circuitry 106 selects a computational model configured to generate that type of metadata. For example, if metadata describing moods is to be generated, control circuitry 106 selects a computational model re configured to process frames in which faces are identified to identify a mood.
  • control circuitry 106 may select an audio processing computational model to process audio data of the entire video or of segments of the video containing faces, to identify segments containing spoken words within the video, and then using the same or a different computational model textualize the spoken words and/or analyze the content of the speech. Control circuitry 106 may, alternatively or in addition, select a text processing computational model to extract and/or process text, such as closed captioning, in or associated with the content media.
  • control circuitry 106 retrieves at least a portion of the media object data from non-volatile
  • control circuitry 106 uses the first metadata generated by the first computational model to identify portions of the video content which contain faces for processing by the computational engine 116 using the selected computational model.
  • circuitry 106 then retrieves only those identified portions of the video content.
  • computational engine 116 generates second metadata describing content in the media object data using the second computational model.
  • control logic 116 generates second metadata describing content in the media object data using the second computational model.
  • circuitry 106 provides the second metadata to non-volatile storage 110 for storage so as to be selectively retrievable from the storage device separately from the media object data using methods described above in connection with
  • the metadata can be retrieved separately from the media object data, thereby enabling the process described above to retrieve only the metadata generated by the first computational model independently of the media object data.
  • FIG. 9 is a flowchart representing a process 900 for retrieving portions of a data object based on separately retrieved metadata describing the data object in accordance with some implementations of the disclosure.
  • storage control device 102 receives a request from a host device for a media object meeting one or more criteria.
  • the media object is surveillance video and the host device requests video data containing human faces.
  • storage control device 102 retrieves, from non-volatile storage 110, only the metadata describing the media object data.
  • storage control device 102 accesses a logically separable portion of the media object in which the metadata describing the surveillance video is stored and extracts, copies, or otherwise retrieves the metadata contained in the logically separable portion without retrieving any of the surveillance video data.
  • storage control device 102 using control circuitry 106, analyzes the metadata to identify portions of the media object data that meet the one or more criteria.
  • control circuitry 106 analyzes the metadata describing the surveillance video and identifies portions of the surveillance video which meet requested criteria - in the present example portions of the surveillance video that contain human faces.
  • storage control device 102 retrieves only the identified portions of the media object data which contain objects meeting the requested criteria separately from other parts of the media object data that do not include portions containing objects meeting the
  • storage control device 102 retrieves only the portions of the surveillance video that contain human faces as identified based on the metadata.
  • the retrieved one or more portions of the media object data are sent, over a computer network, to the host device.
  • the identified portions containing objects meeting the requested criteria are sent to the host device over a computer network.
  • requested criteria such as those that do contain human faces, need not be sent.
  • control circuitry and hardware accelerators e.g. , control circuitry 106 and hardware accelerator 312
  • control circuitry 106 and hardware accelerator 312 are configured to include a set of
  • Each electronic circuit is configured to include any of, but not limited to logic gates, memory cells, amplifiers, filters, and/or the like.
  • Various implementations and components disclosed herein are configured to be at least partially operated and/or
  • processor-executable instructions stored on one or more transitory or non-transitory processor-readable media .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Library & Information Science (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)

Abstract

La présente invention concerne un dispositif de commande de stockage, couplé à un dispositif de stockage et situé à distance d'un dispositif hôte, qui reçoit des données d'objet multimédias en provenance du dispositif hôte. Le dispositif de commande de stockage identifie un type des données d'objets multimédias et sélectionne, sur la base du type identifié, un modèle de calcul parmi une pluralité de modèles de calcul à utiliser par un moteur de calcul du dispositif de commande de stockage. Le moteur de calcul utilise le modèle de calcul sélectionné pour générer des métadonnées décrivant les données d'objets multimédias. Les métadonnées sont stockées dans le dispositif de stockage afin d'être récupérées de manière sélective dans le dispositif de stockage séparément des données d'objets multimédias.
PCT/US2019/044534 2018-07-31 2019-07-31 Systèmes et procédés de génération de métadonnées décrivant des objets de données non structurés dans un dispositif périphérique de stockage WO2020028583A1 (fr)

Priority Applications (6)

Application Number Priority Date Filing Date Title
KR1020217004932A KR20210037684A (ko) 2018-07-31 2019-07-31 저장소 에지에서 비구조적 데이터 객체를 기술하는 메타 데이터를 생성하기 위한 장치 및 방법
JP2021504383A JP2021533446A (ja) 2018-07-31 2019-07-31 ストレージエッジで非構造化データオブジェクトを記述するメタデータを生成するためのシステムおよび方法
EP23174241.2A EP4220437A1 (fr) 2018-07-31 2019-07-31 Systèmes et procédés de génération de métadonnées décrivant des objets de données non structurés au niveau du bord de stockage
CN201980058977.XA CN112673368A (zh) 2018-07-31 2019-07-31 用于在存储边缘处生成描述非结构化数据对象的元数据的系统和方法
EP19756453.7A EP3830714B1 (fr) 2018-07-31 2019-07-31 Systèmes et procédés de génération de métadonnées décrivant des objets de données non structurés dans un dispositif périphérique de stockage
JP2024000847A JP2024038276A (ja) 2018-07-31 2024-01-05 ストレージエッジで非構造化データオブジェクトを記述するメタデータを生成するためのシステムおよび方法

Applications Claiming Priority (20)

Application Number Priority Date Filing Date Title
US201862128233P 2018-07-31 2018-07-31
US62/712,823 2018-07-31
US201862714563P 2018-08-03 2018-08-03
US62/714,563 2018-08-03
US201862716269P 2018-08-08 2018-08-08
US62/716,269 2018-08-08
US201862726847P 2018-09-04 2018-09-04
US201862726852P 2018-09-04 2018-09-04
US62/726,847 2018-09-04
US62/726,852 2018-09-04
US16/263,387 US11068544B2 (en) 2018-07-31 2019-01-31 Systems and methods for generating metadata describing unstructured data objects at the storage edge
US16/264,473 US11080337B2 (en) 2018-07-31 2019-01-31 Storage edge controller with a metadata computational engine
US16/262,971 US11294965B2 (en) 2018-07-31 2019-01-31 Metadata generation for multiple object types
US16/262,971 2019-01-31
US16/264,248 US11748418B2 (en) 2018-07-31 2019-01-31 Storage aggregator controller with metadata computation control
US16/264,473 2019-01-31
US16/263,387 2019-01-31
US16/262,975 2019-01-31
US16/264,248 2019-01-31
US16/262,975 US11036807B2 (en) 2018-07-31 2019-01-31 Metadata generation at the storage edge

Publications (1)

Publication Number Publication Date
WO2020028583A1 true WO2020028583A1 (fr) 2020-02-06

Family

ID=67704561

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/044534 WO2020028583A1 (fr) 2018-07-31 2019-07-31 Systèmes et procédés de génération de métadonnées décrivant des objets de données non structurés dans un dispositif périphérique de stockage

Country Status (1)

Country Link
WO (1) WO2020028583A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170193362A1 (en) * 2016-01-03 2017-07-06 Gracenote, Inc. Responding to remote media classification queries using classifier models and context parameters
US20170293431A1 (en) * 2016-04-06 2017-10-12 Sandisk Technologies Inc. Data storage device having internal tagging capabilities
US20180189615A1 (en) * 2017-01-03 2018-07-05 Samsung Electronics Co., Ltd. Electronic apparatus and method of operating the same

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170193362A1 (en) * 2016-01-03 2017-07-06 Gracenote, Inc. Responding to remote media classification queries using classifier models and context parameters
US20170293431A1 (en) * 2016-04-06 2017-10-12 Sandisk Technologies Inc. Data storage device having internal tagging capabilities
US20180189615A1 (en) * 2017-01-03 2018-07-05 Samsung Electronics Co., Ltd. Electronic apparatus and method of operating the same

Similar Documents

Publication Publication Date Title
US11068544B2 (en) Systems and methods for generating metadata describing unstructured data objects at the storage edge
US20110022394A1 (en) Visual similarity
WO2019050508A1 (fr) Caviardage de vidéo activé par détection d'émotion
CN104581437A (zh) 一种视频摘要生成及视频回溯的方法及系统
US11823453B2 (en) Semi supervised target recognition in video
US10733454B2 (en) Transformation of video streams
CN112732949A (zh) 一种业务数据的标注方法、装置、计算机设备和存储介质
KR101640317B1 (ko) 오디오 및 비디오 데이터를 포함하는 영상의 저장 및 검색 장치와 저장 및 검색 방법
Alam et al. Intellibvr-intelligent large-scale video retrieval for objects and events utilizing distributed deep-learning and semantic approaches
Sunny et al. Map-Reduce based framework for instrument detection in large-scale surgical videos
WO2020028583A1 (fr) Systèmes et procédés de génération de métadonnées décrivant des objets de données non structurés dans un dispositif périphérique de stockage
US20230056131A1 (en) Server and method for classifying entities of a query
KR20220108668A (ko) 동영상 분석 방법
CN113111354A (zh) 目标检索方法和系统、终端设备、云服务器、介质、设备
KR20200075147A (ko) 자동 분류저장 기술을 이용한 dbms-ai 프레임 워크 및 이를 이용한 자동분류저장 방법
Paul et al. Issues in database management of multimedia information
US11526548B1 (en) Image-based query language system for performing database operations on images and videos
CN115858249B (en) Backup method for massive unstructured data files
Khin et al. Key frame extraction techniques
Bornia et al. Towards a semantic video analysis using deep learning and ontology
Bulysheva et al. Image database management architecture: logical structure and indexing methods
Tran et al. Anomaly Event Retrieval System from TV News and Surveillance Cameras
Schallauer et al. SAM: An interoperable metadata model for multimodal surveillance applications
Perrin et al. An Analysis of Data Curation Techniques throughout the Perception Development Pipeline
CN115858249A (zh) 一种海量非结构化数据文件的备份方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19756453

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021504383

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20217004932

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2019756453

Country of ref document: EP

Effective date: 20210301