KR102286418B1 - An apparatus for providing a security surveillance service based on user involvement and a method for operating it - Google Patents

Abstract

According to an embodiment of the present invention, in an operation method of a control service providing device, a method includes the steps of: receiving event occurrence information from a pre-registered security monitoring device; and sharing event notification information with one or more participant terminals for performing a user-participating security monitoring service. Therefore, depending on the service expansion, monitoring and control service is possible without a separate security monitoring agent.

Description

A device for providing a user-participating security monitoring service and an operating method thereof

The present invention relates to a security service providing system and an operating method thereof. More specifically, the present invention relates to an apparatus for providing a security monitoring service capable of providing a user-participating security monitoring service and building a system based thereon to improve monitoring efficiency and expand the service range, and to a method of operating the same.

With the development of current communication and video analysis technology, various security systems that provide security for the area to be monitored through image analysis and object identification of the video received from a remote-located camera have been developed for security in the area to be monitored. Through this, when an object is detected in the area to be monitored, various services are provided, such as notifying the manager whether there is a danger and recording an image according to the detection of the object, thereby enhancing the convenience of managing the area to be monitored.

In general, a security monitoring system detects a monitoring target located in the monitoring target area through a sensor located in the monitoring target area, and when a monitoring target is detected, the monitoring target is captured by a camera in conjunction with the detection of the monitoring target. By providing the image, it is provided so that the monitoring target detected through the sensor can be confirmed through the image.

However, when an intrusion signal is generated according to the detection of a sensor for the monitoring target of the existing security monitoring system, an alarm is provided to the administrator separately from the confirmation through the video. There is a problem that misinformation constantly occurs.

To prevent this, the recent security monitoring system is set to operate so that an alarm is provided to the administrator when a monitoring target object corresponding to the monitoring target is identified in the image captured by the camera when an intrusion signal is generated through the sensor along with the detection of the sensor This is to minimize false information in the security system.

However, the area to be monitored generally has a low-light environment, and for this reason, even in recent security systems, when the sensor detects the object to be monitored, the object does not appear clearly in the image generated by the camera operating in the low-light environment, so the object is not the object to be monitored. Even when an alarm is provided or an object to be monitored appears, the report is frequently missed because the object to be monitored is not identified through the video, so there is a problem in that the misinformation rate cannot be significantly improved.

Due to the above-described problems, there is a problem that reliability and accuracy are lowered according to the intrusion detection report of the surveillance target area of the security monitoring system, and to solve this problem, an image analysis method based on deep learning analysis has been proposed.

However, since a lot of computing resources are consumed for deep learning analysis processing, a deep learning analysis device or server is often located in a separate remote location connected to an external network. Accordingly, object classification and accurate event detection are possible only after the image information is analyzed through the deep learning analysis server, so there is a problem that not only computing cost but also the most important time resource in the security monitoring system is consumed.

As a result, this delays the operation of the monitoring equipment installed in the field, and in reality, only the urgent alarm function based on a simple sensor has to be processed first before the deep learning analysis is completed, so there is a problem that the false alarm rate is still the same.

On the other hand, when an intrusion detection event occurs, the security monitoring system generally requests a pre-set monitoring agent to dispatch, and the monitoring agent directly visits the site to check whether it is an erroneous report, and performs related control processing or reporting.

However, at present, due to the misinformation rate problem, there are a lot of unnecessary dispatches, which not only cause overall system delay, but also cost human time. And, there is a problem in that the problem of reliability degradation such as processing delay increases exponentially.

In addition, security guards must always stand by to be able to be dispatched for a certain period of time for on-site visits. Insufficient cases occur, which lead to delays in dispatch.

The present invention has been devised to solve the above problems, and is a user participation type that can participate in security monitoring and control services by sharing control service processing notification information for an area requiring security monitoring with a plurality of pre-registered users. By providing security monitoring service and inducing participation by providing appropriate compensation for service participation, it reduces repetitive and unnecessary dispatch to false alarm monitoring events, and depending on service expansion, monitoring and control service without separate security monitoring personnel An object of the present invention is to provide an apparatus for providing a user-participating security monitoring service that can reduce system construction and maintenance costs and human time costs, and also minimize service delay, and an operation method thereof.

According to an embodiment of the present invention, there is provided a method for operating a control service providing apparatus, the method comprising: receiving event occurrence information from a pre-registered security monitoring apparatus; and sharing event notification information to one or more participant terminals for performing a user participation type security monitoring service in response to the event occurrence information.

In addition, an apparatus according to an embodiment of the present invention for solving the above problems, in the control service providing apparatus, a communication unit for receiving event occurrence information from a pre-registered security monitoring device; and a service providing unit for sharing event notification information to one or more participant terminals for performing a user participation type security monitoring service in response to the event occurrence information.

In addition, a method according to an embodiment of the present invention for solving the above-described problem, the method of operating a terminal, comprising: receiving an event notification message for participating in a control service from a control service server; outputting a control service participation interface based on the received event notification message; and transmitting a control service participation permission request corresponding to a specific event to the control service server according to a user input to the control service participation interface.

In addition, an apparatus according to an embodiment of the present invention for solving the above problems, in a terminal device, a communication unit for receiving an event notification message for participating in a control service from a control service server; and a control service participation interface processing unit for outputting a control service participation interface based on the received event notification message, wherein the communication unit requests permission to participate in a control service corresponding to a specific event according to a user input to the control service participation interface is transmitted to the control service server.

Meanwhile, the method according to an embodiment of the present invention for solving the above-described problems may be implemented as a program for executing the method in a computer and a recording medium in which the program is recorded.

According to an embodiment of the present invention, it is possible to provide a user participation type security monitoring service that can participate in security monitoring and control service by sharing control service processing notification information for an area requiring security monitoring to a plurality of pre-registered users. there is.

In addition, according to an embodiment of the present invention, by inducing participation by providing appropriate compensation according to service participation, it reduces repetitive and unnecessary dispatch to false alarm monitoring events, and depending on service expansion, monitoring without a separate security monitoring agent And it is possible to provide a method for providing a user-participating security monitoring service that can reduce system construction and maintenance costs and human time costs by enabling the control service, and also minimize service delay, an apparatus supporting the same, and an operation method thereof.

1 is a conceptual diagram schematically illustrating an entire system according to an embodiment of the present invention.
2 is a block diagram illustrating a control service server according to an embodiment of the present invention in more detail.
3 is a block diagram illustrating a service participant terminal in more detail according to an embodiment of the present invention.
4 is a ladder diagram for explaining the overall system operation according to an embodiment of the present invention.
5 to 9 are diagrams illustrating a user interface output from a user terminal and a service participant terminal according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art will be able to devise various devices which, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. Moreover, it is to be understood that all conditional terms and examples listed herein are, in principle, expressly intended solely for the purpose of enabling the concept of the present invention to be understood, and not limited to the specifically enumerated embodiments and states as such. should be

Moreover, it is to be understood that all detailed description reciting specific embodiments, as well as principles, aspects, and embodiments of the present invention, are intended to include structural and functional equivalents of such matters. It is also to be understood that such equivalents include not only currently known equivalents, but also equivalents developed in the future, i.e., all devices invented to perform the same function, regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of illustrative circuitry embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc. may be tangibly embodied on computer-readable media and be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly shown. should be

In addition, clear use of terms presented as processor, control, or similar concepts should not be construed as exclusively referring to hardware having the ability to execute software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM (RAM) and non-volatile memory. Other common hardware may also be included.

The above-described objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram schematically illustrating an entire system according to an embodiment of the present invention.

1 , the entire system according to an embodiment of the present invention includes an intelligent security monitoring device 100, a user terminal 200, a deep learning distributed processing device 300, a relay server 400, a control service server ( 500 ), a service participant terminal 600 , one or more camera devices 750 , and one or more sensor devices 700 .

The camera device 750 and the sensor device 700 may be disposed in a security monitoring area, and may be connected to the intelligent security monitoring device 100 through a local monitoring network. The camera device 750 and the sensor device 700 each photograph a security monitoring area or sense a detection signal generated in or around the security monitoring area to obtain image information and sensor information, and obtain image information and sensor information. The information is transmitted to the intelligent security monitoring device 100 .

The intelligent security monitoring device 100 may collect, store, and manage image information and sensor information received from the camera device 750 and the sensor device 700, and occurs in a security monitoring area based on the image information and sensor information It is possible to detect one or more security monitoring events, and perform monitoring service processing corresponding to the detected event.

Here, the monitoring service processing may include an alarm message transmission service to the user terminal 200 corresponding to the generated event, an alarm notification function operation or cancellation service, a status information transmission service to the control service server 500 , and the like.

For this monitoring service processing, the intelligent security monitoring device 100 may be connected to the relay server 400, the control service server 500, or the user terminal 200 through a wired/wireless network, and the intelligent security monitoring device 100 is One or more wired/wireless communication modules for performing communication through the wired/wireless network may be provided. Various well-known communication methods may be applied to the wired/wireless communication network.

In addition, the intelligent security monitoring apparatus 100 may include one or more information processing modules and information analysis modules for detecting an event by analyzing image information and sensor information.

In addition, the intelligent security monitoring apparatus 100 may detect predetermined first monitoring events based on the initial analysis information according to information processing and analysis, but the second detection and classification of the object requires more accurate analysis. For the monitoring event, a separate deep learning analysis process may be processed in conjunction with one or more deep learning distributed processing devices 300 .

Here, the first monitoring event and the second monitoring event may be grouped according to whether or not deep learning analysis is processed, and the first monitoring event is an intrusion event that is quickly detected according to an initial analysis of sensor information and image information, and an unauthorized camera change event. , fire occurrence events, etc. may be exemplified, and the second monitoring event is a dynamic object tracking event accompanied by accurate object detection and classification information according to deep learning analysis, a trip wire event, a wandering object detection event, a traffic direction violation detection event, Unauthorized object detection event, unauthorized moving object detection event, access counting and statistical event, crowd density detection event, unusual behavior detection event, etc. may be exemplified.

However, since this is an example, it is possible to detect more various events according to the monitoring criteria and object classification settings, and the first monitoring event and the second monitoring event may be complexly detected and used to provide a rapid composite monitoring service. For example, the intelligent security monitoring device 100 transmits the initial analysis information according to the occurrence of the first monitoring event to the control service server 500 or the user terminal 200 through the relay server 400 first, and 1 According to the result of deep learning analysis of video information within a certain time range related to the monitoring event, a second monitoring event is detected as detailed analysis information for the first monitoring event, and a control service server ( 500) or the user terminal 200 may provide the second monitoring event information.

Accordingly, the intelligent security monitoring device 100 can not only quickly process the basic monitoring service processing according to the first monitoring event, but also deep-distributed data for the data initially analyzed and processed in the intelligent security monitoring device 100 . Since only the learning analysis processing is requested and processed by a separate specialized deep learning distributed processing device 300, the occurrence of the second monitoring event, which is provided only when more accurate object classification is possible, can also be quickly detected according to the distributed processing of data. there will be

To this end, the intelligent security monitoring device 100 configures distributed data corresponding to image information requiring deep learning analysis for the second event detection processing, and transmits a deep learning analysis processing request to the deep learning distributed processing device 300 . can

More specifically, the intelligent security monitoring device 100 processes the initial analysis information capable of detecting a first event on the collected image information, and tracks data for tracking an object corresponding to the image information from the initial analysis information. can be obtained

For example, the initial analysis processing may include at least one of image stabilization processing, background modeling processing, shape calculation processing, element connection processing, and object tracking processing, and the tracking data is, for example, from image information to a foreground pixel. may include object tracking data that is separated and filtered, and the connected component is labeled and processed into visual transformation.

The intelligent security monitoring device 100 may generate deep learning distributed processing request data based on the tracking data and the image information, and transmit the deep learning distributed processing request data to the deep learning distributed processing device.

The deep learning distributed processing device 300 may be composed of one or more devices connected through a network to perform edge computing, and according to the deep learning distributed processing request data, the tracking data and the image information Deep learning analysis can be processed, and deep learning analysis result information can be delivered to the intelligent security monitoring device 100 . More specifically, the deep learning analysis result information may include object detection information and object classification information corresponding to the tracked object.

Such object detection information and object classification information may be, for example, dynamic object information identified according to a pattern such as moving or operating in a specific direction from pre-accumulated image information, and the intelligent security monitoring device 100 . may detect a second monitoring event corresponding to the image information based on the object detection information, the object classification information, and the tracking data.

On the other hand, the deep learning distributed processing device 300 processes the deep learning image analysis according to the deep learning distributed processing request data received from the intelligent security monitoring device 100, and transmits the result information to the intelligent security monitoring device 100. It can be delivered, and further, the deep learning image analysis information may be transmitted to the relay server 400 .

The deep learning distributed processing device 300, based on the deep learning distributed processing request data, generally operates in a low-illumination environment in an existing security system, and an image for a video generated when a monitoring target is detected through a sensing signal of a sensor Even when an object is detected during the analysis process, it is difficult to identify the object, so it generates and reports an event for an object other than the object to be monitored. It is possible to process deep learning analysis to improve the problem of increasing the false positive rate, such as when a report is omitted according to the detection of a target.

The deep learning distributed processing device 300 continuously learns the characteristics of the monitoring target object appearing in the environmental characteristics (environmental conditions) of the monitoring target area through deep learning, thereby increasing the identification accuracy of the monitoring target object and the monitoring target object By continuously learning the image characteristics of the identified image to determine the optimal image for easy object identification, and by selecting and providing an image in which the object to be monitored is accurately identified, the object to be monitored is accurately distinguished from other objects and supported to obtain deep learning analysis result information that lowers the false alarm rate and prevents the case of missing a report when a monitoring target is detected, and provides the analysis result information to the intelligent security monitoring device 100 or the relay server 400 there is.

Here, the relay server 400 collects, stores and manages video surveillance analysis data from the intelligent security monitoring devices 100 installed in various places, and the deep learning image analysis result received from the deep learning distributed processing device 300 . and a server that relays the video surveillance analysis data to the user terminal 200 and the control service server 500 .

When the event detection data of the intelligent security monitoring device 100 is received, the relay server 400 indexes the deep learning image analysis information corresponding to the event detection data to correspond to the intelligent security monitoring device 100. It may be a server that transmits the deep learning image analysis information corresponding to the event detection data to the pre-registered user terminal 200 .

In addition, the event detection data and the deep learning image analysis information may be transmitted to the control service server 500 , and the control service server 500 reports an accident occurrence based on the event detection data and the deep learning image analysis information, and a fire occurs. Various control service processing such as reporting, user notification, monitoring device control, and real-time monitoring may be performed, and service performance information may be transmitted to the user terminal 200 .

On the other hand, the control service server 500, a plurality of pre-registered service participant terminals 600, control service processing notification information for a local monitoring network area that requires security monitoring monitored by the intelligent monitoring device 100 By sharing, it is possible to provide a user participation type security monitoring service in which participants of the service participant terminals 600 can participate in security monitoring and control services.

To this end, the control service server 500 analyzes the event detection data and deep learning image of the security monitoring area processed by the intelligent monitoring device 100 and the deep learning distributed processing device 300 and collected through the relay server 400 . Based on the information, control service processing notification information may be generated, and the control service processing notification information may be shared with one or more pre-registered service participant terminals 600 .

The service participant terminal 600 may generate and output a control service participation interface according to the control service processing notification information, and the control service participation interface corresponds to an event detected by the intelligent monitoring devices 100 for each security area. event state information, event level information, event analysis information, and event image information may be output.

In addition, the location information of the service participant terminal 600 is output together on the control service participation interface to display control events that occur within a certain distance and can be accessed. The service participant terminal 600 may transmit a participation permission request to the control service server 500 .

In addition, the control service server 500 may determine whether to allow participation in the selected control event and transmit permission information to the service participant terminal 600 .

According to the participation permission, the service participant terminal 600 user can move to the place where the corresponding control event occurred, and compare and check the current state with the event state information, event level information, event analysis information, or event image information from the moved place and the confirmation result information may be input to the service participant terminal 600 as control processing result reporting information. Here, the confirmation result information may include a confirmation result message and photographed image information for proof.

In addition, the control processing result reporting information is transmitted to the control service server 500, and the control service server 500 generates processing result data based on the control processing result reporting information, and the user terminal 200 that requested monitoring of the security area. ) can be transmitted.

Thereafter, the control service server 500 may receive evaluation information on the control service processing result information from the user terminal 200 , and calculate the service participation level for each service participant terminal 600 based on the evaluation information. In addition, the control service server 500 may determine an appropriate reward according to the event level and service participation, and provide the determined reward information to the service participant terminal 600 .

Here, the reward may be cash, points, virtual currency, cryptocurrency, etc., and the control service server 500 may determine the reward and also vary the reliability level of the service participant terminal 600 .

Accordingly, group monitoring is possible according to the registration of the service participant terminal 600 , and participation in the control service can be induced, the repetitive and unnecessary dispatches to false alarm monitoring events are reduced, and depending on the service expansion, separate A method for providing a user-participating security monitoring service that enables monitoring and control services without security monitoring personnel, reducing system construction and maintenance costs and human time costs, and minimizing service delays, a device supporting it, and an operation method thereof can provide

Furthermore, according to the cumulative expansion of the participation service provision, participants having a high level of reliability by reward and evaluation information can participate through the service participant terminal 600 , so that the user terminal 200 is not a specific security monitoring area, but its own It is possible to set various life monitoring areas, such as around the late night commute, and to request a control service as a protection service to the service participant terminals 600 with a certain level of reliability or higher in real time.

In addition, the participant terminal 600 according to an embodiment of the present invention may be a terminal pre-registered by the user terminal 200 or may be a plurality of terminals designated corresponding to the security monitoring area. For example, in the case of a workplace, a terminal of a business owner and one or more terminals of employees may be exemplified.

In this case, the control service server 500 may transmit a variable event message determined according to the distance from the security monitoring area compared to the current location information among the pre-registered participant terminals 600 when an event occurs. For example, a confirmation request message may be transmitted to the participant terminal 600 of the business owner and the first participant terminal 600 closest to the event occurrence location.

In addition, when the participant terminal 600 of the business owner is more than a certain distance away from the event occurrence location, the control service providing apparatus 500 may transmit a confirmation request message to another participant terminal 600 preset according to priority.

Meanwhile, the control service server 500 may limit the number of participant terminals 600 for collecting reporting data. For example, there may be a maximum of 10 participant terminals 600 capable of transmitting a report. In addition, when the result of the control report according to the reporting data is different for each participant terminal 600 , the control service providing device 500 provides a voting function to the participant terminal 600 to provide control service processing result information using the agreed result data can also create

In addition, the control service server 500 may transmit the control service processing result information to the deep learning distributed processing device 300 through the relay server 400 . The deep learning distributed processing device 300 may perform learning and deep learning analysis processing corresponding to the control service processing result information, and may be utilized to increase the accuracy of the deep learning processing.

2 is a block diagram showing the control service server 500 according to an embodiment of the present invention in more detail.

Referring to FIG. 2 , the control service server 500 according to an embodiment of the present invention includes a control unit 510 , a communication unit 520 , a user registration unit 530 , an event identification unit 540 , and a level determination unit 550 . , a notification processing unit 560 , a performing terminal determining unit 570 , a control data processing unit 580 , a service providing unit 590 , and a database management unit 505 .

The control unit 510 includes one or more microprocessors for generally controlling the components of the control service server 500 .

The communication unit 520 communicates with the control service network including the intelligent monitoring device 100 and the deep learning distributed processing device 300 and the relay server 400 connected thereto, the user terminal 200 and the participant terminal 600 , etc. one or more communication modules for

Here, the communication unit 520 is a type connected through a wireless or wired method through a local area network (LAN) and an Internet network, a type connected through a USB (Universal Serial Bus) port, such as 3G, 4G, 5G A form connected through a mobile communication network, a form connected through a short-range wireless communication method such as Near Field Communication (NFC), Radio Frequency Identification (RFID), Wi-Fi, etc. are possible.

In addition, the user registration unit 530 may receive user registration information for providing a participatory security service from one or more user terminals 200 and one or more participant terminals 600 , and the database management unit 505 may receive user registration It is possible to register user information according to the information, and perform service target area registration corresponding to the registered user information, location information registration, intelligent monitoring device 100 identification information registration, user management processing, and the like.

Here, the service target area may be determined according to the user registration request of the service target user terminal 200 , for example, a specific store, a specific building, or a specific range area requiring security, location-based movement of the user terminal 200 . A region may be exemplified, and location information corresponding to a service target area may be registered in the control service server 500 , and the location information may be in the form of address information, coordinate information, latitude information, longitude information, or GPS information. can be stored and managed.

Then, the event identification unit 540 receives the monitoring image information and the event information received through the communication unit 520 from the intelligent monitoring device 100, and analysis information from the deep learning distributed processing device 300 corresponding thereto. can be received through the communication unit 520, by allocating identification information to the received monitoring image information and event information, storing and managing it through the database management unit 505, and the monitoring image information and event information to which the identification information is assigned. It is transmitted to the level determiner 550 .

Then, the level determining unit 550 determines event level information corresponding to the event information to which the identification information is assigned.

Here, the event level information may correspond to the monitoring difficulty for determining the reward for participation in the control service of the participant terminals 600 . That is, the higher the monitoring difficulty, the higher the event level may be, and when a high event level control service participation is made, the reward compensation ratio may be set higher than the participation level of the participant terminal 600 determined according to the evaluation of the reporting result.

For example, the event level information may include, according to the event content information confirmed from the event state information, the movement of an intruder is detected, the wandering movement of a specific object is detected, a cluster of a plurality of objects is detected, the dumping of a specific object, and When neglect is sensed or a specific person's collapse is detected, etc., the service participants of the participant terminal 600 may be differently set for each event according to the degree or skill level of the security monitoring service in which they can participate.

Meanwhile, when the event level information is determined, the notification processing unit 560 generates event notification information including at least one of the event identification information, event state information, event level information, event analysis information, and event image information, It may be shared by one or more service participant terminals 600 .

The event notification information may be shared, for example, through a social network (SNS) service, a message service, an email service, a mobile communication network service, etc., and the communication unit 520 further includes a separate notification service module for a shared notification service can be

The performing terminal determining unit 570 receives a request for permission to participate in a control service and information on the participant terminal from one or more participant terminals through the communication unit, and according to a preset permission condition, among the terminals that have transmitted the request for permission to participate in the control service It is possible to determine a participant terminal capable of performing a user participation type security monitoring service.

In addition, the performing terminal determining unit 570 may determine by changing the participant terminal capable of performing the user participation type security monitoring service according to the event level.

In addition, the performing terminal determining unit 570 determines whether to allow participation in response to the participation permission request information received from one or more service participant terminals 600 in response to the above-described event notification information, and determines whether to allow the participant terminal for which permission is determined. (600) is determined as the performing terminal. In addition, the performing terminal determining unit 570 may transmit event detail information corresponding to the event notification information to the determined service participant terminal 600 .

Here, various users may be included in the service participant terminal 600 , and it may be difficult to guarantee reliability of participants with little or no history of service participation. In addition, it may be difficult to guarantee the speed of the participant terminal 600 that is too far away.

Accordingly, the performing terminal determining unit 570 may determine whether to allow the participation of the service participant terminal 600 according to the conditions and criteria set in the user terminal 200 requesting the monitoring service.

For example, in the user terminal 200, the reliability rating criterion, the cumulative reward criterion, the average participation evaluation criterion, the authentication information criterion (mobile phone authentication, face authentication, fingerprint authentication, resume authentication, etc.) of the service participant terminal 600, distance Alternatively, at least one event participation condition among the moving time criteria may be set. In addition, when there are a plurality of participant terminals 600 of equal condition, only the participant terminals 600 within a preset number may be permitted on a first-come-first-served basis.

This participation condition setting information may also be transmitted to the notification processing unit 560 , and according to the participation condition setting in the notification processing unit 560 , event notification information may be transmitted only to the participant terminals 600 within a predetermined condition.

And, as described above, when the participant terminal 600 to allow participation is determined by the performing terminal determining unit 570 , the performing terminal determining unit 570 may transmit event detail information to the participant terminal 600 . , event details information may include more clear and detailed address information than event notification information, manager contact information, route guidance information, high-definition video information, and the like so that you can participate in the actual control service.

Therefore, the notification processing unit 560 can anonymize the event notification information so that the control location cannot be specifically specified only with the event notification information, and the actual control service can be performed only with the participant terminal 600 allowed to participate. of detailed event history information may be transmitted.

Meanwhile, the control data processing unit 580 collects the control reporting data processed by the participant terminal 600 to generate control processing result data, and transmits the generated control processing result data to the user terminal 200 . Here, when the control reporting data of one or more participant terminals 600 is different, a plurality of control reporting data selected according to statistics or votes may be generated as the control processing result data.

For example, when an abnormality reporting is received from 7 participant terminals 600 among 10 participant terminals 600 that have processed reporting for the same event, and an abnormal situation reporting is received from three participant terminals 600, The control data processing unit 580 may generate control processing result data indicating that there is no abnormality.

However, this is an example, and the control data processing unit 580 may collect and process the reporting data of all participant terminals as statistical information to generate control processing result data.

Meanwhile, the service providing unit 590 provides control service processing to the user terminal 200 and the participant terminal 600 based on the control processing result data.

The service providing unit 590 may first provide the control processing result information service using the control processing result data to the user terminal 200 . The user terminal 200 may check the processing result according to the control processing result information service, receive evaluation information corresponding to the processing result, and the evaluation information may be transmitted to the service providing unit 590 .

In addition, the service providing unit 590 may calculate the degree of participation of each participant terminal 600 based on the evaluation information and the reporting data. The service providing unit 590 may determine a reward to be provided for each participant of each participant terminal 600 based on the degree of participation and event level information, and may process payment to each participant terminal 600 . As a reward, cash, points, virtual currency, cryptocurrency, etc. may be exemplified, and the reliability rating given to the participant terminal 600 may be improved according to the reward payment.

The database management unit 505 performs registration processing of user information according to user and service participant registration information, service target area registration corresponding to the registered user information, location information registration, identification information registration of the intelligent monitoring device 100, control service, It is possible to perform event information, reward information, and user management processing based thereon.

3 is a block diagram illustrating a service participant terminal in more detail according to an embodiment of the present invention.

Referring to FIG. 3 , a service participant terminal 600 according to an embodiment of the present invention may be a portable terminal such as a smartphone, and may include a controller 610 , a communication unit 620 , an input unit 630 , and an output unit 640 . ), a control service participation interface processing unit 650 , a result reporting unit 660 , a location information processing unit 670 , and a storage unit 680 .

The control unit 610 controls the overall configuration operation of the participant terminal 600 by using the information stored in the storage unit 680, and may include one or more microcontrollers for this purpose.

The communication unit 620 may include one or more modules for performing communication between the participant terminal 600 and an external device such as the control service providing server 500 and the relay server 400 .

Here, the communication unit 6120 is a type of connection through a wireless or wired method through a local area network (LAN) and an internet network, a type connected through a USB (Universal Serial Bus) port, and a mobile communication network such as 3G or 4G. It may include a communication module of a type connected through a short-range wireless communication method such as near field communication (NFC), radio frequency identification (RFID), Wi-Fi, and the like.

The input unit 630 receives a user input for the participant terminal 600 .

Here, the input unit 630 includes various types of buttons, a touch sensor that receives a touch input to the output unit 640, a proximity sensor that receives a motion approaching without being directly connected to the surface of the display unit, and a user's voice input that receives a user's voice input. It may be implemented using at least one of microphones. However, the present invention is not limited thereto, and may be implemented in any form capable of receiving a user input.

In addition, the input unit 630 may further include an audio/video information input unit for inputting camera shooting information for reporting a control service result.

The output unit 640 may include one or more output modules for outputting information processed according to a user input and an application operation.

In particular, the output unit 640 may display an interface screen for processing a control service participation process according to the operation of the participant terminal 600 according to an embodiment of the present invention, a result reporting interface, and the like in video, text, and voice. .

And, the control service participation interface processing unit 650 generates one or more input/output graphical user interfaces (GUIs) for performing the participation process of the control service according to the interworking with the control service server 500 according to an embodiment of the present invention, , the generated interface is output through the output unit 640 , and the user may input input information for the control service participation interface through the input unit 630 .

Here, through the control service participation interface, the user may input participation service request permission information according to the output of the event notification information shared by the control service server 500 . In addition, the control service participation interface may output control service detail information corresponding to the permitted request. In addition, the control service participation interface processing unit 650 may receive control service participation result reporting information to be transmitted to the result reporting unit 660 through the control service participation interface.

The result reporting unit 660 processes the control service result information input through the control service participation interface processing unit 650 into reporting information and transmits it to the control service server 500 .

Here, the reporting information is confirmation information corresponding to the control service processing result, and may further include reporting image information along with processing information such as no abnormality, abnormal situation occurrence, and detailed text text. The reporting image information may include confirmation image information comparable to the previously received event image information. To this end, the input unit 630 may collect shooting reporting image information according to a user input and store it in the storage unit 680 .

Meanwhile, the location information processing unit 670 may include a location information tracking module for identifying location information of the participant terminal 600 and transmitting the location information to the control service server 500 . The location information may be tracked in various ways such as GPS, mobile communication network, etc., and may be used for permission of the participant terminal 600 , route guidance to an event location, and the like.

4 is a ladder diagram for explaining the overall system operation according to an embodiment of the present invention.

Referring to FIG. 4 , the user terminal 200 according to an embodiment of the present invention performs pre-user registration processing in the intelligent security monitoring apparatus 100 and the control service server 500 ( S100 ).

Also, the participant terminal 600 may process participant information registration to the control service server 500 (S101).

User registration processing, for example, user information and mobile phone information registration processing as an administrator using the intelligent security monitoring device 100, authentication information registration of the participant terminal 600, mobile phone information registration processing, etc. may be exemplified, and the user The terminal 200 receives and outputs the processing information of the intelligent security monitoring device 100 and the control service processing information of the control service server 500, or outputs the input information of the user terminal 200 to the intelligent security monitoring device 100 or As an input/output device for transmission to the control service server 500, various electronic devices such as a computer, smart phone, tablet PC, and navigation may be exemplified, and the participant terminal 600 may be a mobile terminal of a service participant. .

Then, the intelligent security monitoring device 100 collects and processes image information collected from one or more camera devices (S103).

For example, the intelligent security monitoring device 100 performs background modeling processing of image information that has been stabilized, and performs shape calculation processing of detecting a shape element region of foreground binary data from which a background image is subtracted according to background modeling processing. and performing element connection processing for labeling by connecting the shape element regions to the object connection regions, and performing object tracking processing for composing an object list corresponding to the object connection regions.

And, the intelligent security monitoring device 100, object tracking information including the object list in the image information obtained according to the initial analysis as described above, and processing the image information into deep learning distributed processing request data, deep learning distributed It is transmitted to the processing device 300 (S105).

Thereafter, the deep learning distributed processing device 300 applies the object tracking information and image information of the deep learning distributed processing request data to the pre-built neural network data, performs deep learning analysis-based object detection processing (S107), and detection A classification process is performed by identifying the pattern of the object that has been changed (S109).

Then, the deep learning distributed processing apparatus 300 responds to the object detection information and classification information according to the deep learning-based analysis to the intelligent security monitoring apparatus 100 (S111).

Here, the object detection information and classification information may include information used for event detection by the above-described event detection unit 180, for example, the type and pattern of the object passing through the trip wire, and the intrusion detection of the object. Types and patterns, types and patterns of wandering objects, types and patterns of objects that violate the direction of passage, types and patterns of unauthorized objects, types and patterns of unauthorized moving objects, types and patterns of objects used for counting access and statistical calculation, crowds It may include various detection information and classification pattern information, such as the type and pattern of the object that is the basis of the detection of the dense state, the type and behavior pattern of the object corresponding to the specific behavior, and the type and pattern of the object indicating the detection of unauthorized camera change. .

Based on the object detection information and classification information, and the object tracking information, the intelligent security monitoring apparatus 100 determines event detection ( S115 ). The sensor information of the above-described sensor device 700 may also be used for event detection determination.

When an event is detected, the intelligent security monitoring apparatus 100 may transmit an event notification message to the user terminal 200 (S121).

First, the user terminal 200 may output a notification interface including an event notification message and corresponding event analysis information based on deep learning ( S123 ). The user may input appropriate processing information of the intelligent security monitoring apparatus 100 such as alert, release, and alarm according to the type of event, and the input processing information may be transmitted to the intelligent security monitoring apparatus 100 (S125) ).

Then, the intelligent security monitoring apparatus 100 performs processing according to the user input information, and transmits the processing result information to the user terminal 200 (S127).

On the other hand, the deep learning distributed processing device 300 may transfer the image analysis information to the relay server 400 for pre-stored and managed processing (S113), and the intelligent monitoring device 100 in the event detection step is performed by the relay server 400 ) to transmit the event detection data (S117).

Here, the image analysis information may be used to provide deep learning image analysis information corresponding to the event detection data to the user terminal 200 .

On the other hand, the relay server 400 receives the control service request information according to the user input of the intelligent security monitoring device 100 or the user terminal 200 (S119), and the control difficulty corresponding to the event included in the control service request information to determine the event level (S129).

Thereafter, the control service server 500 generates control notification information including event level information (S131).

And, the control service server 500 shares the control service participation notification message with one or more participant terminals 600 matching the preset criteria among the pre-registered participant terminals 600 (S133).

Accordingly, the participant terminal 600 may output a control service participation interface (S135).

And, according to the user input from the participant terminal 600, the permission request information in charge of control processing is transmitted to the control service server 500 (S137), and permission information and control according to whether the control service server 500 determines whether permission is granted. The detailed data is transmitted to the participant terminal 600 (S139).

After the control service processing, the participant terminal 600 receives the control processing result reporting information (S141), and the processing result reporting data of the participant is transmitted to the control service server 500 (S143).

Then, the control service server 500 generates the information on the result of the control service processing based on the reporting data (S145), and the information on the result of the processing of the control service is transmitted to the user terminal 200 (S147).

Thereafter, the control service processing evaluation information is input in the user terminal 200 , and the control service processing evaluation information is transmitted to the control service server 500 ( S149 ), and a reward according to the service participation calculation is determined.

Then, the control service server 500 transmits the participation reward payment data according to the calculated reward to the participant terminal 600 (S151).

5 to 9 are diagrams illustrating a user interface output from a user terminal and a service participant terminal according to an embodiment of the present invention.

5 illustrates an event notification setting interface for transmitting setting information from the user terminal 200 to the intelligent security monitoring apparatus 100 and the control service server 500 according to an embodiment of the present invention.

First, as shown in FIG. 5 , the user terminal 200 may provide a setting interface for setting an event to be used in the image analysis process and event conditions to be set for a user participatory security monitoring service, a display, a speaker, etc. Information related to the event setting interface may be output through the same output device. The output device may be included in the user terminal 200 . In addition, the output device may be included in the control service server 500 or the intelligent security monitoring device 100, and the user terminal 200 may output the screen, voice and image of the input/output interface received from the output device. .

As shown in FIG. 5 , event types configurable through the event setting interface may be varied. For example, the event type may include a region-based event, a behavior pattern-based event, an object type-based event, and the like.

In addition, through the event setting interface, various selectable properties may be provided corresponding to each event type and related to the event type.

As an example, the region-based event is for setting a region for determining whether an event occurs in an image, and the user terminal 200 includes a plurality of attributes for classifying an event target region in response to the region-based event. It is provided through the event setting interface, and accordingly, a high level is applied to an area requiring high security so that information on the corresponding level is provided as an event result, thereby supporting different security measures for each level.

In this case, the event setting interface provides graphic information corresponding to each property for displaying the border shape of the event target area according to each grade or the color of the event target area corresponding to the grade selected when specifying the area differently depending on the grade In addition, by applying the graphic information to the event target area selected by the user and supporting it to be displayed on the image, the user can visually and conveniently check which level is set in which area.

On the other hand, the behavior pattern-based event is for detecting an event according to a specific behavior pattern of an object, and the user terminal 200 provides a plurality of selectable properties in response to the behavior pattern-based event through the event setting interface, , As an example of these properties, an object identified in the image stays in an area for more than a specific time, wanders for a specific time or more, an object previously unidentified appears in the image, or an object previously identified in the image disappears. It may include patterns.

In this case, the user terminal 200 provides graphic information corresponding to the properties of each behavior pattern through the event setting interface so that it can be intuitively determined which event function each property provides. As an example of the graphic information, the case of staying for more than a specific time is provided as a clock-shaped icon, and the case of wandering for more than a specific time is provided as a foot-shaped icon so that the user can easily distinguish the event function.

On the other hand, the object-based event is for selecting the type of object, and the user terminal 200 provides a plurality of properties corresponding to the object-based event. It can include the type of object to be generated. At this time, the user terminal 200 may preset a plurality of parameters for distinguishing each property belonging to the object-based event from other properties, and examples of the parameters include the type of object, the size of the object, the color of the object, etc. may include.

Also, the user terminal 200 may provide a virtual avatar as graphic information corresponding to each attribute. For example, when the property of the object is a person, a virtual avatar in the shape of a person may be provided, and when the property of the object is a car, a virtual avatar in the shape of a car may be provided.

Meanwhile, the user terminal 200 may set an event target area through an event setting interface. The event target area may be an event target area corresponding to the security monitoring area. The intelligent security monitoring apparatus 100 may control the camera unit 750 to perform security monitoring on the event target area.

In addition, the user terminal 200 may set an event condition through an event setting interface. The event condition is for setting condition information for detecting an event, and may include properties and detailed conditions of an event to be detected in response to the aforementioned region-based, behavioral pattern-based, object-based, and event target regions.

In addition, the event condition according to an embodiment of the present invention may include an event condition for transmitting a participatory security control service request to the control service server 500 . That is, the event condition may include a first event condition processed by the intelligent security monitoring device 100 , and may include a second event condition for requesting a participatory security control service from the control service server 500 .

In addition, the user terminal 200 may set the event level through the event setting interface. As described above, the event level may be determined by the control service server 500 , but the event level specified by the user terminal 200 may be given according to user settings. Accordingly, the control service server 500 may preferentially determine whether event level information corresponding to the event condition set by the user terminal 200 is preset.

In addition, the user terminal 200 may set the notification target terminal condition through the event setting interface. The notification target terminal condition is an attribute condition of the participant terminal 600 with which the control service server 500 will share the event notification, and may be exemplified by reliability rating, cumulative reward, evaluation information, authentication information, and the like, and the user monitors security The participant terminals 600 accessible to the area may be conditionally set in advance.

Meanwhile, the user terminal 200 may set reward (reward) information for participation in control through the event setting interface. The user can set different rewards for participating in the control by event conditions, and can set different rewards to be given according to the importance of the event. Accordingly, the control service server 500 determines whether there is preset control participation compensation setting information from the user terminal 200 when an event occurs, and may process to provide a reward according to the preset control participation compensation setting information. there is.

In addition, the control service server 500 complexes at least one of the preset control participation compensation setting information, the above-described event level setting information, and the level information determined by the level determination unit 550 of the control service server 500 . By using it, it is possible to more accurately determine the reward to be actually paid.

Meanwhile, FIGS. 6 to 9 are diagrams illustrating a control service participation interface of the participant terminal 600 according to an embodiment of the present invention.

As shown in FIG. 6 , the participant terminal 600 according to an embodiment of the present invention uses the control service participation interface based on the event notification information received from the control service server 500 through the control service participation interface processing unit 650 . can be generated and output as video or audio through the output unit. 6 , the event notification information according to an embodiment of the present invention includes at least one of an intrusion detection event, a fire detection event, an unauthorized dumping event, a wandering detection event, a danger zone approach detection event, a parking and stopping detection event, and a fall detection event. can do.

And, as shown in FIG. 6 , the participant terminal 600 may output one or more accessible event notification information based on current location information, and the event notification information includes event status information, event level information, and events. It may include analysis information and event image information.

In addition, the participant terminal 600 may display each event notification information within a specific radius that can be expanded or reduced at a location point on the screen corresponding to the current location information of the participant terminal 600 . Event level information is also displayed in each event notification information displayed on the participant terminal 600 so that the participant can pre-determine the level of the control service that he/she wants to perform.

And, as shown in FIG. 7 , the participant terminal 600 may select specific control event information according to a user input through the control service participation interface. When event information is selected, the participant terminal 600 may output a message indicating that it is a 'danger zone approach' event as event status information corresponding to the selected event information through the control service participation interface, and as event identification information A message such as 'event #117' can be output.

In addition, when the user selects a specific event in FIG. 7 , a query interface asking whether to participate in the corresponding event may be output, and a menu interface for checking event image information may be further output. Accordingly, the participant may request for participation permission through the control service participation interface to the participant terminal 600 or check an event related video in advance.

On the other hand, FIG. 8 shows the control service participation interface when participation is permitted. When participation permission information is received from the control service server 500 , the participant terminal 600 is a security monitoring area in which the participation permitted event occurs. The route information and route guide information to the destination may be output. The user may move to the selected path, and may perform processing such as checking the security monitoring area for participating in the control service and taking pictures/videos.

And, FIG. 9 shows a participation reporting screen of the control service participation interface according to the completion of the control service processing. The participant terminal 600 may provide a separate reporting screen interface for transmitting the reporting information to the control service server 500 according to a user's input of a participation completion button or the like.

The reporting data may include confirmation history information for the security monitoring area, unusual information, photo/video upload information, etc., and the participant terminal 600 user receives the control service participation reporting data through input through each interface to the control service server It can be processed to be delivered to (500).

In addition, as shown in FIG. 9 , the maximum reward information for the event #117 participation report may be displayed together with a value such as 234 points, and the maximum reward may be determined according to the event level information, and the reward actually granted may be determined through the summation of the conversion value of the evaluation result information input from the user terminal 200 and the maximum reward or weight calculation. For example, as shown in FIG. 9 , when the maximum reward according to the event level is 234 points, the reward actually paid may vary according to evaluation information, such as 200 points.

As such a process is processed, the user terminal 200 may be notified of processing result information according to the participatory security monitoring service processing of the participant terminal 600, and the processing result information includes on-site photos/videos and Relevant reporting messages may be included. For example, in the case of a safe situation, a related dispatch result and a relief message may be notified to the user terminal 200 .

The method according to the present invention described above may be produced as a program to be executed by a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape. , floppy disks, and optical data storage devices.

The computer-readable recording medium is distributed in a network-connected computer system, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the art to which the present invention pertains.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims Various modifications may be made by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (14)

In the operating method of the control service providing device,
Receiving a request for permission to participate in a control service from one or more participant terminals;
determining a participant terminal capable of performing a user participation type security monitoring service from among terminals transmitting a request for permission to participate in a control service according to a preset permission condition;
Receiving event occurrence information from a pre-registered security monitoring device;
In response to the event occurrence information, sharing the event notification information to one or more participant terminals determined as the participant terminal, comprising the step of transmitting control detail data associated with the event occurrence information
The operation method of the control service providing device. According to claim 1,
Sharing of the event notification information,
determining an event level according to the difficulty of participating in a control service corresponding to the event occurrence information; and
Sharing the event notification information including the event level information to the participant terminal
The operation method of the control service providing device. 3. The method of claim 2,
The method further comprising transmitting reward payment data based on the event level to one or more participant terminals that have completed the user participation type security monitoring service
The operation method of the control service providing device. delete According to claim 1,
The preset permission condition includes at least one of a reliability rating condition corresponding to the participant terminal, an authentication condition, an accumulated reward condition, and a location condition
The operation method of the control service providing device. According to claim 1,
collecting processing result reporting data according to the completion of the user participation type security monitoring service from the one or more participant terminals;
generating control service processing result information based on the processing result reporting data; and
The method further comprising the step of providing the control service processing result information to the security monitoring device or a user terminal set in response to the security monitoring device
The operation method of the control service providing device. 7. The method of claim 6,
Further comprising the step of receiving evaluation information corresponding to the control service processing result information from the user terminal,
The evaluation information is an operating method of a control service providing apparatus used for calculating a reward paid to the participant terminal. In the control service providing device,
a communication unit for receiving event occurrence information from a pre-registered security monitoring device;
a service providing unit for sharing event notification information to one or more participant terminals for performing a user participation type security monitoring service in response to the event occurrence information; and
A participant terminal capable of receiving a request for permission to participate in the control service from the one or more participant terminals through the communication unit and performing the user participation type security monitoring service among terminals that have transmitted the request for permission to participate in the control service according to a preset permission condition Including a performing terminal determining unit to determine,
The service providing unit transmits, to the determined participant terminal, control detail data related to the event occurrence information
Control service providing device. 9. The method of claim 8,
Further comprising a level determining unit for determining an event level according to the difficulty of participating in the control service corresponding to the event occurrence information,
The service provider,
Sharing the event notification information including the event level information to the participant terminal
Control service providing device. 10. The method of claim 9,
The service provider,
Transmitting reward payment data based on the event level to one or more participant terminals that have completed the user participation type security monitoring service
Control service providing device. delete 9. The method of claim 8,
The preset permission condition includes at least one of a reliability rating condition corresponding to the participant terminal, an authentication condition, an accumulated reward condition, and a location condition
Control service providing device. 9. The method of claim 8,
Further comprising a control data processing unit for collecting processing result reporting data according to the completion of the user participatory security monitoring service from the one or more participant terminals,
The service provider
generating control service processing result information based on the processing result reporting data, and providing the control service processing result information to the security monitoring device or a user terminal set in response to the security monitoring device
Control service providing device. 14. The method of claim 13,
The communication unit receives the evaluation information corresponding to the control service processing result information from the user terminal,
The evaluation information is a control service providing device used to calculate a reward paid to the participant terminal from the service providing unit.

Images