KR102422450B1 - Method for managing media data and sensing data - Google Patents

Abstract

Disclosed is a method for efficiently storing and managing media data and sensing data by integrating. The disclosed method for storing media data and sensing data includes a plurality of media files including media data generated by a camera disposed in a monitoring location and sensing data generated by a sensor disposed in the monitoring location together, and a method for the media file. generating an index file including information; and storing the media file and the index file in a storage medium.

Description

How to manage media data and sensing data

The present invention relates to a method for managing media data and sensing data, and more particularly, to a method for integrating, storing, searching, and playing media data and sensing data.

In the past, video data was acquired using CCTV to monitor a specific place, but recently, with the development of communication technology and sensor technology, IoT sensors are being used together. That is, recently, when monitoring a specific place, not only acquire image data using CCTV, but also acquire various additional information about the monitoring place, for example, a gas sensor, a temperature sensor, etc., so that an event occurring in a specific place My ability to analyze situations is improving.

It is common for image data and sensor sensing data to be stored separately in separate formats, but as the types or number of sensors for monitoring increase, the cost and complexity of separately storing and managing image data and sensing data This is increasing.

Accordingly, recently, there is an increasing need for a method for integrating, storing and managing image data and sensing data.

As related prior documents, there are Korean Patent Registration Nos. 10-2058452 and 10-1960530.

An object of the present invention is to provide a method for efficiently storing and managing media data and sensing data by integrating them.

According to an embodiment of the present invention for achieving the above object, a plurality of media files including media data generated by a camera disposed in a monitoring location and sensing data generated by a sensor disposed in the monitoring location together; generating an index file for the media file; and storing the media file and the index file in a storage medium. There is provided a method for storing media data and sensing data.

In addition, according to another embodiment of the present invention for achieving the above object, the method comprising: receiving information about a time interval, event or place requested from a user; and using an index file including information on a plurality of media files, searching for a media file according to a time section, event, or location requested by the user, wherein the media file is located in a preset location. It includes media data and sensing data generated by each of a camera and a sensor, and the index file includes event information about the camera or the sensor and storage location information of the media file, and the index file is stored in the camera. A method for retrieving media data and sensing data is provided, which is a subfolder of a first folder including an assigned ID and is stored in each of the second folders including a date of creation of the media data.

In addition, according to another embodiment of the present invention for achieving the above object, using an index file for a plurality of media files, the method comprising: searching for a media file according to a time interval, event, or location requested by a user; and decoding the media file to reproduce media data and sensing data included in the found media file, wherein the media data and the sensing data are data generated by each of a camera and a sensor disposed in a preset location. wherein the index file includes event information about the camera or the sensor and storage location information of the media file, and the index file is a subfolder of the first folder including an ID assigned to the camera, A method of reproducing media data and sensing data is provided, wherein the media data is stored in each of the second folders including the creation date.

According to an embodiment of the present invention, the media data and the sensing data are stored as one file, so that the media data and the sensing data can be efficiently managed.

In addition, according to an embodiment of the present invention, it is possible to efficiently and quickly search media data and sensing data by using a file system for media data and sensing data.

1 is a view for explaining a monitoring system according to an embodiment of the present invention.
2 is a diagram for explaining a file system according to an embodiment of the present invention.
3 is a diagram for explaining a method of storing media data and sensing data according to an embodiment of the present invention.
4 is a diagram for explaining the format of an MP4 file.
5 is a diagram for explaining a method of reproducing media data and sensing data according to an embodiment of the present invention.
6 is a diagram illustrating an interface through which a media file is played according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

In order to solve the above problems, the present invention proposes a file system capable of storing and managing media data and sensing data in an integrated manner.

Media data and sensing data may be stored in one file through the file system according to an embodiment of the present invention, and according to an embodiment of the present invention, the stored file may be retrieved and reproduced at high speed.

According to an embodiment of the present invention, media data and sensing data generated at the same time are stored as one file, and an index file including various information about the stored file is generated, thereby efficiently storing media data and sensing data. Manage and search.

The present invention may be implemented in a computing device including a processor and a memory, for example, a desktop, a notebook computer, a data storage device, a network video recorder (NVR) server, and the like.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a view for explaining a monitoring system according to an embodiment of the present invention.

Referring to FIG. 1 , a monitoring system according to an embodiment of the present invention includes at least one camera 110 , at least one sensor 120 , a data storage device 130 , and a terminal 140 .

The camera 110 and the sensor 120 are disposed at at least one monitoring location, and an ID may be assigned to each of the camera 110 and the sensor 120 . This ID may be determined according to the monitoring location. The camera 110 and the sensor 120 may transmit the generated data to the data storage device through a wired or wireless communication network, and may transmit the data using the Internet of Things (IoT) as an example.

The camera 110 may be a CCTV camera as an embodiment, and generates media data including images and sounds. The camera 1110 may continuously photograph the monitoring location or detect a preset event to photograph the monitoring location.

The sensor 120 collects various information about the monitoring location. As an embodiment, the sensor 120 may be a gas sensor, a temperature sensor, a humidity sensor, etc., and, like the camera 110 , continuously collects information on a monitoring place, or detects a preset event to collect information. can

The data generated by the camera 110 and the sensor 120 may include IDs and data generation time information for the camera 110 and the sensor 120 . In particular, the media data may include information on when an image was recorded, and the sensing data may include an attribute type and attribute value of the sensing data.

The data storage device 130 includes a data receiver 131 , a file generator 132 , and a file storage 133 .

The data receiver 131 receives media data generated by the camera 110 and sensing data generated by the sensor 120 . The file generator 132 generates a plurality of media files including media data and sensing data together, and an index file including information on the media files. The file generator 132 may generate a media file and an index file according to the file system according to an embodiment of the present invention. The generated media file and index file are stored in the file storage unit 132 . The file storage unit 132 may include a storage medium such as an SSD or HDD, as an embodiment.

The data storage device 130 searches for the stored media file using the index file according to the request of the terminal 140 used by the user, and then provides it to the user. According to an embodiment, the data storage device 130 may decode the searched file and stream the reproduced data to provide it to the terminal 140 or transmit the searched file to the terminal 140 .

The terminal 140 requests the data storage device 130 for a media file for a time section, event, or place input by the user. The terminal 140 may display an interface for receiving an input of a time section, event, or place desired by the user.

The terminal 140 may play the media file transmitted from the data storage device 130 using a decoder, or may display the streamed data played back in the data storage device 130 on the display device.

2 is a diagram for explaining a file system according to an embodiment of the present invention, and is a diagram for explaining a storage folder 210 of an index file and an index file 220 used in the file system.

As described above, the data storage device according to an embodiment of the present invention generates not only a media file including media data and sensing data, but also an index file including information on the media file.

The data storage device according to an embodiment of the present invention stores the index file 220 in a preset index file storage folder 210 . The index file storage folder 210 includes a first folder 211 including an ID assigned to the camera, and a second folder 212 including a media data creation date as a subfolder of the first folder 211 . include The index file 220 is stored in the second folder 212 , and one index file 220 may be stored in each of the second folders created for each date.

The index file 210 includes an ID for the index file, and includes event information about a camera or sensor and storage location information of a media file. The event information may be data generation time information according to an event type, and includes media data generation time information and sensing data generation time information.

Specifically, the index file 220 may include a Type ID field, a Version field, a Flags field, a Size field, a Create Time field, a First Time field, a Last Time field, an Event Timetable field, and a File Timetable field. A unique ID value for the index file 220 is recorded in the Type ID field, and version information of the index file 220 is stored in the Version field. In the Flags field, option information of the index file 220 is recorded, and in the Size field, size information about a Creat Time field to a File Timetable field is recorded. The creation time of the index file 220 is recorded in the Create Time field. The first time when valid recorded data exists is recorded in the First Time field, and the last time when valid recorded data exists is recorded in the Last Time field. The event type and generation time detected by the camera or sensor are recorded in the Event Timetable field, and the creation time of media data included in the media file is recorded in the File Timetable field.

Accordingly, the data storage device may check the index file 220 to easily and quickly search for a media file corresponding to a time section, event, or location requested by a user from among a plurality of media files stored in the storage medium.

For example, if the ID of the camera assigned to the place requested by the user is 001, the requested time interval is a specific time of July 2, 2020, and the requested event is an event in which humidity above a threshold is detected, the data storage device By checking the index file stored in the "/Record/CamID001/20200702" folder, the user can quickly search for the requested media file.

3 is a diagram for explaining a method of storing media data and sensing data according to an embodiment of the present invention, and FIG. 4 is a diagram for explaining a format of an MP4 file.

A data storage device according to an embodiment of the present invention includes a plurality of media files including media data generated by a camera disposed in a monitoring location and sensing data generated by a sensor disposed in a monitoring location together, and information on media files Creates an index file including (S310). Then, the generated media file and index file are stored in a storage medium (S320).

The media file may be, for example, an MP4 file. As shown in FIG. 4 , the MP4 file is largely composed of three boxes.

File Type (ftype) is a file type box that checks the compatibility of files, Media Data (mdat) is a media data box that stores actual media, and Movie Metadata (moov) is a movie box that stores all metadata of media. . The movie box includes a Track box defining a single track in the movie and a User Data box 410, and the sensing data is included in the user data box of the MP4 file, so that the media file and the sensing data are combined into one file. can be integrated and stored.

The sensing data may include, as an embodiment, an ID for the sensor, sensing data generation time information, and an attribute type and attribute value of the sensing data. [Table 1] is a table showing sensing data for the power meter sensor.

Sensing data generation time
(nano timestamp) Sensor ID Sensor property type attribute value 1575858923341393213 PowerMeter Watt 120 1575858923432893458 PowerMeter Watt 60

The data storage device composes the data for each field in [Table 1] as a string, converts it into a hex value, and adds it to the user data box. As an example, when sensing data is generated as shown in [Table 1], the data storage device separates data generated at different times with a semi-colon (;) and separates each value with a colon (,), "1575858923341393213 ,PowerMeter,Watt,120;1575858923432893458,PowerMeter,Watt,60" can be created and converted into a hex value. Returning to FIG. 3 , the data storage device is a time interval in which media data is generated in step S310. Creates a media file that includes the sensed data generated in . That is, the data storage device synchronizes the generation time of the media data and the generation time of the sensing data, and the sensing data generated at the time the media data is generated is included in the media file together with the media data.

Meanwhile, in step S310, the data storage device may generate a plurality of media files in units of the same preset file size, and the file size may be 16 MB as an embodiment. For example, when the media data is 100 MB, the data storage device may divide the media data according to the passage of time and generate a plurality of media files having a total size of the divided media data and the sensing data of 16 MB. Accordingly, the image reproduction time of each media file may be different from each other.

And in step S320, the data storage device overwrites the second media file generated at a second point in time later than the first point in the storage area of the storage medium in which the first media file generated at the first point in time is stored. can

By creating media files of the same size, fragmentation of the hard disk can be prevented, and when data is newly stored in the storage area of the hard disk in which data is stored, a delete operation for already stored data and a write operation for new data By storing new data only through the overwrite operation without going through the

As another embodiment, when the data storage device overwrites the second media file in step S320, the second media file is written according to whether the area of the storage medium in which the first media file is stored is a bad sector. can be overwritten. When the area of the storage medium in which the first media file is stored is a bad sector, the data storage device may overwrite the second media file in another area.

As another embodiment, when generating the index file in step S310, the data storage device may additionally create an index file for backup together with the basic index file. Therefore, if the primary index file is damaged, the media file can be retrieved through the index file for backup.

FIG. 5 is a diagram for explaining a method of reproducing media data and sensing data according to an embodiment of the present invention, and FIG. 6 is a diagram showing an interface according to an embodiment of the present invention through which a media file is played.

The data storage device according to an embodiment of the present invention searches for a media file according to a time section, event, or location requested by a user by using an index file for a plurality of media files ( S510 ). In step S510, the data storage device may receive information about a time section, event, or place requested by the user, and search for a media file according to the received information.

Then, the searched media file is decoded, and media data and sensing data included in the searched media file are reproduced (S520). The reproduced media file may be streamed to the user's terminal.

In step S520, the data storage device displays the generation time of the sensing data generated in the time period requested by the user on the timeline while playing the retrieved media data.

As shown in FIG. 6 , a media file to be played is displayed on the screen 610 of the terminal, and a timeline 620 is also displayed. The length of the time line 620 may correspond to a user's requested time period. The time line 620 may display an event occurrence time, that is, a time point 621 and 622 at which the sensed data is generated, and the event occurrence time may be displayed in synchronization with the current playback time of the media file as an example.

For example, if the event occurrence time is 2 o'clock and 4 o'clock, and the playback time of the current media file is 2 o'clock, the event occurrence time is first displayed at the point corresponding to 2 o'clock on the timeline, and then the playback time of the media file is 4 When the time comes, the event occurrence time may be displayed at a point corresponding to 4 o'clock on the timeline.

In addition, according to another embodiment, the sensing data may be displayed in a graph form on the x-y plane separately from the timeline. Here, the x-axis corresponds to the sensing time of the sensor, and may correspond to the user's requested time interval. And the y-axis corresponds to the sensed data. That is, the data storage device may display the change of the sensed data according to the sensing time in the form of a graph on the screen 610 of the terminal.

The technical contents described above may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are only provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , various modifications and variations are possible from these descriptions by those of ordinary skill in the art to which the present invention pertains. Therefore, the spirit of the present invention should not be limited to the described embodiments, and not only the claims described below, but also all those with equivalent or equivalent modifications to the claims will be said to belong to the scope of the spirit of the present invention. .

Claims (17)

generating a plurality of media files including media data generated by a camera disposed in a monitoring location and sensing data generated by a sensor disposed in the monitoring location together, and an index file including information on the media files; and
Storing the media file and the index file in a storage medium,
The media file is an MP4 file,
The media data is included in a media data box of the MP4 file,
The sensing data is included in the user data box of the MP4 file,
The step of creating the media file is
Creates a plurality of media files with the same preset file size unit,
The step of storing the media file in a storage medium is
overwriting a second media file generated at a second time point later than the first time point in a storage area of the storage medium in which the first media file generated at the first time point is stored;
The storage medium is a hard disk,
How to store media data and sensing data.
delete The method of claim 1,
The step of creating the media file is
generating a media file including the sensing data generated in the time period in which the media data is generated
How to store media data and sensing data.
delete delete The method of claim 1,
The step of storing the media file in a storage medium is
Overwriting the second media file according to whether the storage area of the storage medium is a bad sector
How to store media data and sensing data.
The method of claim 1,
The index file is
ID of the index file, media data generation time information, sensing data generation time information, and storage location information of the media file
A method of storing media data and sensing data comprising a.
The method of claim 1,
The index file is
It is a subfolder of the first folder including the ID assigned to the camera, and is stored in each of the second folders including the date when the media data was created.
How to store media data and sensing data.
The method of claim 1,
The sensing data is
ID for the sensor, sensing data generation time information, including attribute types and attribute values of the sensing data
How to store media data and sensing data.
receiving information about a time interval, event or place requested by a user; and
Using an index file including information on a plurality of media files, searching for a media file according to a time interval, event, or location requested by a user,
The media file includes media data and sensing data generated by each of a camera and a sensor disposed in a preset location,
The index file includes event information about the camera or the sensor and storage location information of the media file,
The index file is a subfolder of the first folder including the ID assigned to the camera, and is stored in each of the second folders including the date when the media data was created,
The media file is an MP4 file,
The media data is included in a media data box of the MP4 file,
The sensing data is included in the user data box of the MP4 file,
Each of the plurality of media files,
It is a media file of the same preset file size unit,
Among the media files stored in a storage medium that is a hard disk, a first media file generated at a first point in time is
The second media file generated at the second point in time is overwritten and stored in the storage area of the storage medium where it is stored,
The first time point is a time later than the second time point,
How to retrieve media data and sensing data.
delete 11. The method of claim 10,
The media file is
Including the media data and sensing data generated in the time period in which the media data is generated
How to retrieve media data and sensing data.
delete using an index file for a plurality of media files, searching for a media file according to a time interval, event, or location requested by a user; and
and decoding the media file to reproduce media data and sensing data included in the searched media file,
The media data and the sensing data are data generated by each of a camera and a sensor disposed in a preset place,
The index file includes event information about the camera or the sensor and storage location information of the media file,
The index file is a subfolder of the first folder including the ID assigned to the camera, and is stored in each of the second folders including the date when the media data was created,
The media file is an MP4 file,
The media data is included in a media data box of the MP4 file,
The sensing data is included in the user data box of the MP4 file,
Each of the plurality of media files,
It is a media file of the same preset file size unit,
Among the media files stored in a storage medium that is a hard disk, a first media file generated at a first point in time is
The second media file generated at the second point in time is overwritten and stored in the storage area of the storage medium where it is stored,
The first time point is a time later than the second time point,
How to play back media data and sensing data.
15. The method of claim 14,
The step of playing the media data and the sensing data is
While playing the media data, the generation time of the sensing data generated in the time section is displayed on the timeline.
How to play back media data and sensing data.
15. The method of claim 14,
The step of playing the media data and the sensing data is
It displays the change of the sensed data according to the sensing time in the form of a graph.
How to play back media data and sensing data.
delete

Images