KR102567011B1 - System and method for event alarm based on metadata and application therefor - Google Patents

Abstract

In order to inform the terminal of an event based on metadata, the event notification system receives image information, checks a surveillance area where the image information is captured, and receives sensing information from at least one sensor installed in the surveillance area. Integrated metadata, which is data representing a plurality of situation information extracted from image information and sensing information in chronological order, is provided to the terminal.

Description

System and method for event alarm based on metadata and application therefor}

The present invention relates to a metadata-based event notification system, method, and application.

In general, video control devices (eg, CCTV) or various detection sensors collect various video information or sensing information as control information. The collected control information is individually transmitted to the terminal of a user who has subscribed to the control service in the form of an app push or Short Message Service (SMS).

In the case of providing control information in this way, since the image information captured by the video control device and the sensing information collected by the sensor are transmitted to the user's terminal, the user can have any organic relationship between the events occurring with only the transmitted sensing information. There is a problem in which it is difficult to easily infer whether there is a relationship. That is, the video information includes a pre-alarm (about 15 seconds) scene and an event scene where a change in motion detection begins, and the sensing information sends a call value determined according to the sensor value whenever the detection sensor detects a change. because it is sent to

Therefore, it is inconvenient for the user to check the content of the video information while directly reproducing the video information that is at least 30 seconds or more in order to check the event information each time. In addition, there is a problem in that it is cumbersome to continuously receive sensing information that is transmitted several times during a proximity time, and it is difficult for a user to recognize a situation by himself as to what image information and sensing information collectively mean.

Accordingly, the present invention provides a meta data-based event notification system, method, and method for informing whether an event has occurred in the form of an emoticon to a user's terminal receiving video control data by using meta data, which is control information collected from a video control device and a sensor, respectively, and provide applications.

One feature of the present invention for achieving the technical problem of the present invention is a method for notifying an event by an event notification system to a terminal,

Receiving image information; checking a surveillance area where the image information is captured; receiving sensing information from at least one sensor installed in the monitoring area; and providing integrated meta data generated based on the video information and the sensing information to the terminal, wherein the integrated meta data indicates a plurality of situation information extracted from the video information and the sensing information in chronological order. It is data.

The receiving of the sensing information may include checking whether the video information collecting device is installed in an integrated monitoring area based on identification information of the video information collecting device that has collected the image information; and receiving sensing information collected by the at least one or more sensors when installed in the integrated monitoring area.

The providing of the integrated metadata to the terminal may include analyzing the image information and generating image analysis data including generated situation information; generating image metadata by arranging the image analysis data in chronological order; and generating sensor meta data by arranging the sensing information in chronological order.

The image meta data includes situation information included in the image information and time information at which the situation occurred, and the sensor meta data includes identification information of a sensor that collected sensing information, time information at which sensing information was collected, and sensing values. Context information corresponding to may be included.

After generating the sensor metadata, matching a pre-stored emoticon with the image metadata in response to context information included in the image metadata; matching a pre-stored emoticon with the sensor meta data in response to context information included in the sensor meta data; and generating the integrated meta data by arranging image meta data and sensor meta data matched with each emoticon in the order of occurrence time.

Another feature of the present invention for achieving the technical problem of the present invention is a system for notifying an event to a terminal based on meta data,

an image information receiving unit for receiving image information captured in the integrated monitoring area; a sensing information receiver configured to receive sensing information from at least one or more sensors installed in the integrated monitoring area; an event arranging unit that generates image metadata by arranging image analysis data generated from the image information in chronological order, and generates sensor metadata by arranging the sensing information in chronological order; and matching event information included in the image metadata with emoticons corresponding to situation information included in the sensor metadata, arranging image metadata matched with emoticons and sensor metadata in chronological order to generate the integrated metadata. and an emoticon matching unit that generates and transmits the emoticon to the terminal.

The system may further include an emoticon storage unit for storing emoticons generated in response to a plurality of context information matching the image metadata and sensor metadata in the emoticon matching unit.

The system stores and manages zone information for an integrated surveillance area set with a video information collection device and at least one sensor, or a single integrated surveillance area set with any one information collection means of a video information collection device and a sensor, , A zone information storage unit providing stored zone information when the sensing information receiving unit checks whether the video information collection device is installed in the integrated monitoring area may be further included.

An application loaded in a terminal, which is another feature of the present invention for achieving the technical problem of the present invention, and executing a programmed operation by combining with hardware including a processor, memory, and display of the terminal,

Receiving integrated metadata from an event notification system; And instructions programmed to perform the step of outputting a plurality of context information included in the received integrated meta data and an emoticon corresponding to the context information to the display in the order of occurrence of the context contains the application.

According to the present invention, since image information and sensing information are provided in the form of emoticons as well as meta data, the user can easily understand the control information transmitted to the terminal and the meaning of the control information.

1 is an exemplary diagram of an environment to which an event notification system according to an embodiment of the present invention is applied.
2 is a structural diagram of an event notification system according to an embodiment of the present invention.
3 is a flowchart of an event notification method according to an embodiment of the present invention.
4 is an exemplary view of event notification according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

In the present specification, a terminal includes a mobile station (MS), a mobile terminal (MT), a subscriber station (SS), a portable subscriber station (PSS), and a user device (User Equipment, UE), Access Terminal (AT), etc., and may include all or some functions of mobile terminals, subscriber stations, mobile subscriber stations, user devices, and the like.

An event notification system and method based on meta data according to an embodiment of the present invention will be described with reference to the drawings below.

1 is an exemplary diagram of an environment to which an event notification system according to an embodiment of the present invention is applied.

As shown in FIG. 1, the event notification system 100 includes at least one image information collection device 10-1 or 10-2 that collects image information and at least one sensor 20-1 that collects sensing information. ∼20-7). Here, the video information collection devices 10-1 and 10-2 are implemented as devices capable of collecting images such as CCTV, and the sensors 20-1 to 20-7 detect heat rays, detect sound, detect door opening, etc. It is implemented with sensors that can detect various situations.

Further, the event notification system 100 generates metadata based on a plurality of image information and sensing information, and transmits an event notification message based on the metadata to a user's terminal (not shown). Here, one image information collecting device creates a monitoring area along with at least one or more sensors installed in a location and a short distance of the image information collecting device.

That is, the first sensor 20-1, the second sensor 20-2, and the third sensor 20-3 are installed around the first image information collection device 10-1. Therefore, the surveillance area monitored by the first image information collection device 10-1, the sensing sensed by the first sensor 20-1, the second sensor 20-2, and the third sensor 20-3, respectively. An area can be set as one monitoring area. An area in which monitoring is performed using image information and sensing information collected by the image information collection device and sensors, respectively, is referred to as an 'integrated monitoring area'.

However, the video information collecting device and the sensor may not necessarily be included in the monitoring area. For example, since the image information collection device is not installed in the area where the fourth sensor 20-4 and the fifth sensor 20-5 are installed, the fourth sensor 20-4 and the fifth sensor ( The area formed by 20-5) is referred to as a 'single monitoring area'.

When information is collected in an integrated monitoring area in which both a video information collection device and a sensor are included, the event notification system 100 may notify the user's terminal of the occurrence of an integrated event. On the other hand, when information is collected in a single surveillance area in which only a video information collection device or sensor is installed, the event notification system 100 may notify the terminal of a single general event.

The terminal provides the integrated meta data or single meta data transmitted from the event notification system 100 to the user in the order of occurrence time of situation information included in the meta data. To this end, the user can check monitoring information through an application installed in the terminal.

Here, since the monitoring area can be set by various methods, such as being set by a user, description is not limited to any one method in the embodiment of the present invention.

A structure of an event notification system 100 that generates an integrated event notification or a single event notification based on information collected from an integrated monitoring area or a single monitoring area and provides it to a user's terminal will be described with reference to FIG. 2 .

2 is a structural diagram of an event notification system according to an embodiment of the present invention.

As shown in FIG. 2, the event notification system 100 includes a zone information storage unit 110, an image information reception unit 120, a sensing information reception unit 130, an event sorting unit 140, and an emoticon storage unit 150. and an emoticon matching unit 160.

The zone information storage unit 110 is an area for an integrated surveillance area set with a video information collection device and at least one sensor or a single integrated surveillance area set with any one information collection means among the video information collection device and sensors. store and manage information Area information includes identification information and location information of information collection means installed in all surveillance areas, and identification information on surveillance areas.

The image information receiving unit 120 receives image information collected by an arbitrary image information collection device. The image information includes identification information about the image information collection device, time information collected by the image information collection device, and a control image. Upon receiving the image information, the image information receiving unit 120 transmits the identification information of the image information collecting device included in the image information to the sensing information receiving unit 130, and delivers the image information to the event sorting unit 140.

The sensing information receiving unit 130 receives sensing information collected by an arbitrary sensor. The sensing information includes identification information about the sensor, time information at which the sensing information was collected by the sensor, and sensing values measured by each sensor. Here, the sensing value may be changed to various values depending on the generated situation. For example, a sensing value when a loud noise occurs and a sensing value when a door is opened have different sensing values.

Then, the sensing information receiving unit 130, when the image information receiving unit 120 receives the image information, collects the location information of the image information collecting device based on the identification information included in the image information in the zone information storage unit 110. Check. When the video information collection device is installed in the integrated monitoring area, sensing information is received from at least one sensor installed in the same integrated monitoring area.

In an embodiment of the present invention, the sensing information receiving unit 130 determines whether the video information collecting device is installed in an integrated surveillance area or a video information collecting device installed in a single surveillance area, based on the identification information of the video information collecting device included in the image information. do. And, when installed in the integrated monitoring area, receiving sensing information from at least one or more sensors installed in the integrated monitoring area will be described as an example.

However, after the event arranging unit 140, which will be described later, analyzes image information to generate image analysis data and generates image meta data, after confirming the type of monitoring area, when it is installed in the integrated monitoring area, the sensors The sensing information collection unit 130 may be requested to collect sensing information.

The event sorting unit 140 receives image information from the image information receiving unit 120 and receives sensing information from the sensing information receiving unit 130 . The event sorting unit 140 analyzes the control video included in the video information to generate video analysis data. Here, since the event arranging unit 140 may analyze the control image through various methods, the embodiment of the present invention is not limited to any one method.

The video analysis data generated by the event arranging unit 140 includes information on a situation (eg, loitering, trespassing, falling, disappearing, etc.) generated by a controlled object in a controlled video and time information at which the corresponding situation occurred. In order to extract situational information generated by the controlled object through analysis of the controlled video, the event arranging unit 140 stores reference information on actions. The event arranging unit 140 sorts the video analysis data in chronological order and generates video meta data.

Then, the event arranging unit 140 sorts the sensing values included in the sensing information in chronological order and generates sensor meta data. The sensor meta data includes identification information of a sensor that collected sensing information, time information at which sensing information was collected, and situation information corresponding to a sensing value.

The emoticon storage unit 150 stores emoticons for various events. Here, the emoticons represent various events, such as situations (eg, roaring noise, door opening, etc.) or actions (eg, invasion of territory, disappearance, wandering around, falling down, etc.) in the form of pictures in advance so that users can easily recognize them As expressed, emoticons for situations or actions are not limited to any one form.

The emoticon matching unit 160 matches the video metadata generated by the event arranging unit 140 and the sensor metadata in chronological order to generate integrated metadata. The integrated meta data includes a plurality of situation information and time information when each situation occurred. Here, the integrated meta data means data including all image meta data and sensor meta data provided to the user's terminal in the order of occurrence of situations, and mapped emoticons corresponding to each situation.

However, if only one of image metadata and sensor metadata is received from the event arranging unit 140, single metadata is generated. The single meta data includes situation information and time information when the corresponding situation occurred.

Also, the emoticon matching unit 160 matches the emoticons stored in the emoticon storage unit 150 to correspond to a plurality of situational information included in integrated meta data or single meta data. Then, an event occurrence notification is transmitted to the user terminal.

Here, the event occurrence notification transmitted to the user's terminal is displayed in the order of occurrence times of a plurality of situations. In addition, in order to inform the user that an event has occurred as an integrated event or a single event, the emoticons may be implemented in a form (eg, GIF, etc.) played back in the terminal within a preset time.

A method of notifying an event based on metadata through the event notification system 100 described above will be described with reference to FIG. 3 .

3 is a flowchart of an event notification method according to an embodiment of the present invention.

As shown in FIG. 3 , the event notification system 100 receives image information collected from the monitoring area in which the image information collection devices 10-1 and 10-2 are respectively installed (S100). The image information includes identification information about the image information collection device, time information collected by the image information collection device, and a control image.

The event notification system 100 checks whether the video information collection device is installed in the integrated surveillance area based on the identification information of the video information collection device that has collected the image information (S110). If it is confirmed that the video information collection device is installed in a single monitoring area, the event notification system 100 performs single event processing using the video information (S160). Single event processing is performed through a general video control procedure, and a detailed description of known single event processing is omitted.

Meanwhile, as a result of checking in step S110, if it is confirmed that the video information collection device is installed in the integrated surveillance area, the event notification system 100 includes at least one sensor installed together in the integrated surveillance area where the video information collection device is installed. Receives sensing information from (S120). Here, the sensing information includes identification information about the sensor, time information when the sensor collected the sensing information, and sensing values measured by each sensor.

The event notification system 100 processes the image information received in step S100 to generate image analysis data. The image analysis data includes information on a situation (eg, loitering, trespassing, falling, disappearing, etc.) generated by a controlled object in a controlled video and time information at which the corresponding situation occurred. Then, the image analysis data is sorted in the order of time when the situation occurred, and the image metadata is generated.

In addition, the event notification system 100 sorts the sensing information in chronological order and generates sensor meta data. Then, when the video metadata and sensor metadata are generated (S130), the event notification system 100 sends pre-stored emoticons corresponding to the plurality of situation information included in the video metadata and sensor metadata, respectively. Data and sensor metadata are matched (S140).

The event notification system 100 generates integrated metadata by matching the emoticon-matched image metadata and sensor metadata in chronological order in step S140. The integrated meta data includes a plurality of situation information and time information when each situation occurred. Then, the generated integrated metadata is transmitted to the user's terminal to notify that an integrated event has occurred (S150).

An example of event notification transmitted to a user's terminal according to the event notification method described above will be described with reference to FIG. 4 .

4 is an exemplary view of event notification according to an embodiment of the present invention.

Assume that a loud noise is generated at 17:58 within the integrated monitoring area and a door opening is detected at 17:59. And it is assumed that the subject invades the monitoring area at 18:01 and immediately disappears.

Then, as shown in Figure 4, the sensor of the event notification system 100 recognizes the roaring sound, detects the door opening and collects each sensing information. Then, the image information collecting device collects, as image information, a situation in which a subject invades and disappears from the integrated monitoring area.

Then, the event notification system 100 detects the opening of the door after a roar is generated based on the sensing information and the image information, and displays on the terminal together with an emoticon that the subject invades the area and then disappears. Emoticons are matched according to each event, and a first emoticon for loud noise, a second emoticon for opening a door, a third emoticon for invasion of a territory, and a fourth emoticon for disappearing are included in the integrated metadata. And each emoticon is played within a preset time in chronological order.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

Claims (13)

A method for an event notification system to inform a terminal of an event,
Receiving image information;
checking a surveillance area where the image information is captured;
receiving sensing information from at least one sensor installed in the monitoring area; and
Providing integrated metadata generated based on the image information and the sensing information and emoticons corresponding to situations occurring in chronological order to the terminal together with the integrated metadata
including,
The integrated metadata is data representing a plurality of situation information extracted from the image information and the sensing information in the chronological order. According to claim 1,
Receiving the sensing information,
checking whether the video information collecting device is installed in an integrated monitoring area based on identification information of the video information collecting device that has collected the image information; and
Receiving sensing information collected by the at least one sensor when installed in the integrated monitoring area
Event notification method comprising a. According to claim 1,
The step of providing the integrated metadata to the terminal,
analyzing the image information to generate image analysis data including generated situation information;
generating image metadata by arranging the image analysis data in chronological order; and
Generating sensor metadata by arranging the sensing information in chronological order
Event notification method comprising a. According to claim 3,
The image metadata includes situation information included in the image information and time information at which the situation occurred,
The sensor meta data includes identification information of a sensor that collected sensing information, time information at which sensing information was collected, and situation information corresponding to a sensing value. According to claim 4,
After generating the sensor metadata,
matching a pre-stored emoticon to the image metadata in response to context information included in the image metadata;
matching a pre-stored emoticon with the sensor meta data in response to context information included in the sensor meta data; and
Generating the integrated metadata by arranging image metadata and sensor metadata matched with emoticons in the order of time when the situation occurred
Event notification method comprising a. According to claim 5,
The terminal,
The event notification method of replaying the integrated metadata received from the event notification system together with the emoticons in the order of occurrence times. In a system for notifying an event to a terminal based on meta data,
an image information receiving unit for receiving image information captured in the integrated monitoring area;
a sensing information receiver configured to receive sensing information from at least one or more sensors installed in the integrated monitoring area;
an event arranging unit that generates image metadata by arranging image analysis data generated from the image information in chronological order, and generates sensor metadata by arranging the sensing information in chronological order;
Event information included in the video metadata and emoticons corresponding to the situation information included in the sensor metadata are matched, and integrated metadata is created by arranging the video metadata and sensor metadata matched with emoticons in chronological order. Emoticon matching unit for transmitting to the terminal; and
An emoticon storage unit for storing emoticons generated in response to a plurality of situation information matching the image metadata and sensor metadata in the emoticon matching unit.
Event notification system that includes.
delete According to claim 7,
Stores and manages zone information for an integrated surveillance area set with a video information collection device and at least one sensor, or a single integrated surveillance area set with any one information collection means among a video information collection device and a sensor, Area information storage unit that provides stored area information when the information receiving unit checks whether the video information collection device is installed in the integrated surveillance area
Event notification system further comprising a. According to claim 9,
The sensing information receiving unit,
When the video information receiving unit receives video information, an event notification for checking whether the video information collection device that has collected the video information is installed in the integrated monitoring area based on the identification information included in the video information, from the zone information. system. According to claim 9,
The image information includes identification information about the image information collecting device, a control image, and time information at which the control image was collected;
The sensing information includes identification information about the sensor, time information when the sensor collects the sensing information, and a sensing value measured by each sensor. According to claim 7,
The emoticon matching unit,
generating video analysis data including the situation information generated by analyzing the controlled video;
Arranging the generated video analysis data in chronological order based on the time information to generate video metadata;
An event notification system for arranging the sensing information in chronological order based on the time information and generating sensor meta data including the situation information corresponding to the sensing information. As an application mounted on a terminal and executing a programmed operation in combination with hardware including a processor, memory, and display of the terminal,
Receiving integrated metadata from an event notification system; and
Outputting a plurality of contextual information included in the received integrated meta data and emoticons mapped to each of the plurality of contextual information to the display in the order of occurrence of situations
An application stored in a storage medium containing instructions programmed to perform.

Images