KR101926510B1 - Wide area surveillance system based on facial recognition using wide angle camera - Google Patents

Abstract

The present invention relates to a wide area surveillance system based on facial recognition using a wide-angle camera and, more specifically, relates to a system capable of obtaining a plurality of images based on a target subject (human) through a plurality of wide-angle cameras, thereby tracking the behavior or movement of the subject while identifying the face of the subject three-dimensionally. In particular, based on the images taken through the plurality of wide-angle cameras, an image for three-dimensionally observing a subject is provided, so that the subject can be efficiently tracked and surveilled based on facial recognition. Moreover, in order to accurately decode a large amount of images taken for a long time, subjects appearing in a space, in which the images are taken, are confirmed, and the subjects are overlapped with the space at the same time, so that the movement or behavior of all the subjects can be accurately confirmed. Therefore, the present invention is capable of improving reliability and competitiveness in security and surveillance system fields, especially visual image-based security and surveillance fields, CCTV-based security and surveillance fields, and IoT-based security and surveillance fields as well as relevant and similar fields.

Description

[0002] Wide area surveillance system based on facial recognition using a wide angle camera [0003]

The present invention relates to a wide area surveillance system based on face recognition using a wide angle camera, and more particularly, to a wide area surveillance system using a wide angle camera to acquire a plurality of images centered on a target object (person) using a plurality of wide angle cameras, It is able to track the movement or behavior of the target object while grasping the face in three dimensions.

In particular, the present invention provides an image capable of observing a target object in a three-dimensional manner based on an image captured using a plurality of wide-angle cameras, thereby providing a wide angle To a wide area surveillance system based on facial recognition using a camera.

Security and surveillance cameras, such as CCTV, are widely installed in residential areas, roads, and various facilities. Security and surveillance systems using such cameras are effective in preventing crime and resolving crime cases more quickly in the event of a crime .

Generally, a security and surveillance system installs a plurality of cameras in a surveillance area to acquire images of the surveillance area in real time, and an administrator monitors surveillance images in a central control room.

However, such a security and surveillance system requires a small number of administrators to visually monitor a large number of areas. Therefore, since visibility is dispersed and only a specific area can not be intensively monitored, an important moment such as the occurrence of a crime is missed frequently And there was a difficulty in re-searching all the video records in order to confirm this later.

In order to solve this problem, techniques for automatically recognizing and tracking objects in a video image through an image recognition technique have been developed for objects to be visually monitored by an administrator.

For example, Korean Patent Registration No. 10-1364046 " Method and apparatus for tracking an object in a moving image " (hereinafter referred to as " prior art " (Size, slope, deformation, etc.) of the object.

More specifically, the prior art extracts feature points of an object in a video and then tracks the feature points to accurately track various objects in a complex background environment.

Using the tracking function through image recognition greatly reduces the burden on the administrator, so that more efficient monitoring can be performed.

However, the moving object (target object, mainly human) detection technique performed in the security and surveillance system through image recognition can not detect errors in the recognition accuracy and processing speed, There is such a limit that there is a problem, which causes problems in the reliability of the system.

As a result, from the viewpoint of operating the security and surveillance system, in order to secure sufficient reliability of the system, the administrator has to manually check the images.

In addition, conventional security and surveillance systems such as the prior art have merely provided only information on the tracking and movement path of the movement of the target object (person) itself.

Korean Patent Registration No. 10-1364046 'Method and apparatus for tracking objects in moving images'

In order to solve the above problems, the present invention provides a method for acquiring a plurality of images centered on a target object (person) using a plurality of wide-angle cameras, And to provide a wide-area surveillance system based on facial recognition using a wide-angle camera capable of tracking a behavior.

In particular, the present invention provides an image capable of observing a target object in a three-dimensional manner based on an image captured using a plurality of wide-angle cameras, thereby providing a wide angle And to provide a wide area surveillance system based on facial recognition using a camera.

In order to accurately read a large number of images photographed for a long time, the present invention confirms objects appearing in a space where the corresponding image is photographed, and simultaneously overlaps a plurality of objects in the space (background image) The object of the present invention is to provide a wide-area surveillance system based on facial recognition using a wide-angle camera which can accurately confirm movement or behavior of all objects for a short time.

In order to achieve the above object, a wide area surveillance system based on facial recognition using a wide angle camera according to the present invention includes: a plurality of wide angle cameras for acquiring omni-directional images; An object identifying unit for identifying a target object and a face portion of the target object in an original image according to a photographing point photographed through the wide angle cameras; And an object image processing unit for generating a facial image at a photographing time point by correcting an original image at each photographing time point so that the facial part of the object is located at the center of the image with respect to the original image of the photographing point at which the facial part of the target object is identified. And a stereoscopic image generating unit for generating a face stereoscopic image for a face portion of the object based on the face image by the shooting time point generated by the object image processing unit.

In addition, the stereoscopic image processing may generate a face stereoscopic image from a plurality of facial images including a facial portion of a target object using a time slicing imaging technique.

An object viewpoint checking module for checking the object viewpoint of the target object by checking the face portion stereoscopic image; And an object view image extracting module for extracting an original image of an object view corresponding to an object view identified by the object view confirmation module among the images photographed through the wide angle camera, The stereoscopic image of the area viewed by the object can be generated and provided based on the original image of the object view point extracted from the viewpoint image extracting module.

In addition, the stereoscopic image processing may include extracting at least one target object included in the original image at the photographing point photographed through the wide angle cameras for a predetermined period of time, overlapping the extracted target object with the same background image A compressed stereoscopic image can be generated and provided.

In addition, the stereoscopic image processing may provide a face stereoscopic image from a time point when the target object is displayed to a time point when the target object disappears when the user selects the specific object object.

In addition, the stereoscopic image processing may overlap the background image after correcting the target object so that the target object is matched with the object display information set for each time period during which the target object is photographed.

In addition, the stereoscopic image processing may analyze the motion of the target object in the process of overlapping the target object with the background image, and may overlap the target object in a warning state when it is determined as a warning situation.

The object tracking unit may further analyze the movement of a specific object from the overlapped image provided from the stereoscopic image system and call a digital map of the corresponding region to confirm a predicted movement path of the corresponding object.

An image collection unit for collecting an original image photographed through the wide angle camera and providing the collected image to the object verification unit; And an IoT camera capable of photographing at least another part of the predicted movement path and providing at least a part of the images taken in cooperation with the image collecting unit on the object Internet basis, The target object can be extracted from the image provided from the wide angle camera and the image provided from the IoT camera through the image collecting unit and overlapped with the background image based on the predicted moving path confirmed by the IoT camera.

In addition, the image preparation may correct an image provided from the IoT camera to a wide angle image using at least a part of the image provided from the wide angle camera, and then overlap the target object with the corresponding image.

According to the present invention, a plurality of images centered on a target object (person) are acquired using a plurality of wide-angle cameras, and the motion of the target object is obtained through three- It has the advantage of tracking behavior.

In particular, the present invention provides an image capable of observing a target object in three dimensions based on an image photographed using a plurality of wide angle cameras, thereby effectively tracking and monitoring a target object based on facial recognition .

In addition, the present invention has an advantage in that it can track the gaze of a target object based on a stereoscopic image of a facial part.

Accordingly, the present invention has the effect of enhancing the crime prevention effect by predicting and preparing the state of the criminal simulation through the sight line of the criminal (object object).

In addition, the present invention has an advantage in that when a crime occurs, a plurality of target objects (people) are gazed at the center of the area, and the association with the crime can be deduced to help solve the case.

Also, the present invention is advantageous in that a large number of images photographed for a long time can be accurately read in a short time by displaying a plurality of objects simultaneously overlapping on one background image.

Accordingly, in the process of visually reading an image acquired through a surveillance camera by a person in charge, a large amount of images can be checked for a short time, and motion or behavior of an object (such as a person) in the image can be tracked There are advantages.

In addition, according to the present invention, when a specific object is selected in the process of displaying the time-compressed image, the normal speed image is displayed based on the object, thereby more accurately determining the behavior of the object (person) There are advantages to be able to.

In other words, the present invention is capable of visually observing a plurality of objects at the same time, and is capable of observing a corresponding object in a normal state when a specific object is suspicious.

In addition, the present invention can interoperate with various cameras based on IoT to enable surveillance over a wider area, thereby greatly reducing the cost of construction and operation of the system, There are advantages.

Accordingly, the present invention can overcome the time and space limitation of the operation of the security and surveillance system, and has the advantage of enabling more efficient operation of the system.

Accordingly, it is possible to improve reliability and competitiveness in the field of security and surveillance system, in particular, security and surveillance based on visual image, security and surveillance using CCTV, security and surveillance based on IoT, and similar or related fields .

FIG. 1 is a block diagram illustrating an embodiment of a wide-area surveillance system based on facial recognition using a wide-angle camera according to the present invention.
FIGS. 2 to 4 are diagrams for explaining embodiments of a method for facial recognition and eye tracking of the stereoscopic image providing unit shown in FIG.
5 to 8 are views for explaining embodiments of a method of providing a time-compressed image shown in FIG.
Fig. 9 is a configuration diagram showing another embodiment of Fig. 1. Fig.

An example of a wide area surveillance system based on facial recognition using the wide angle camera according to the present invention can be variously applied, and a most preferred embodiment will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a wide-area surveillance system based on facial recognition using a wide-angle camera according to the present invention.

1, a wide area surveillance system 100 based on facial recognition using a wide angle camera includes a wide angle camera 110, an object verification unit 120, an object image processing unit 130, and a stereoscopic image processing unit 140 , The image described below can be understood as a comprehensive meaning including a moving image as well as a still image.

As shown in FIG. 1, a plurality of lenses can be constructed, and an image obtained through a plurality of lenses can be converted into a single image And can be integrated into an omnidirectional image. Here, the image processing algorithm for integrating a plurality of images into one omni-directional image can be applied to various methods according to the needs of those skilled in the art.

The object identifying unit 120 identifies a target object and a face portion of the target object at an image-capturing time point (an omnidirectional image) photographed through wide angle cameras, and detects an object included in the original image through image recognition, In particular, it is possible to identify a moving object (for example, a person) and recognize the face of the person as the object. Here, the method of identifying the moving object and the face portion of the moving object from the specific image is not limited to the specific one as described above.

The object image processing unit 130 receives the original image of the photographing point at which the facial part of the target object is identified through the object identifying unit 120 and then determines whether the facial part of the object is located at the center of the image The original image is corrected to generate a facial image at the time of photographing.

In other words, not the method of photographing by adjusting the camera so that the object is positioned at the center of the image, but the image captured by the wide-angle camera 110 acquiring the omnidirectional image is corrected, Position.

The stereoscopic image generation unit 140 generates a face stereoscopic image for the face portion of the target object based on the face image by the shooting time point generated by the object image processing unit 130. [

In addition, the stereoscopic image processing unit 140 can confirm the direction of the line of sight of the target object.

Generally, in order to track a specific person's gaze direction, it is common to analyze the face image based on a close-up image. However, in the system of the present invention, it is extremely rare that a specific person can be photographed close to the subject.

Accordingly, the stereoscopic image processing unit 140 of the present invention analyzes the motion of the target object (the direction of the torso) and the behavior, as well as the face image of the face portion in the face portion as described above, Can be inferred.

In other words, the present invention is not a method of directly tracking a person's eyes to track a person's eyes, but rather a method of inferring in which direction a person is looking based on a direction and a specific behavior of a face and torso of a person, can confirm.

Hereinafter, a method of providing images to prevent or solve crime using the inferred direction of the eyes will be described.

FIGS. 2 to 4 are diagrams for explaining embodiments of a method for facial recognition and eye tracking of the stereoscopic image providing unit shown in FIG.

Referring to FIG. 2, one object may be photographed by a plurality of wide-angle cameras 110.

In this manner, when one object is photographed at various points in time, a face slice stereoscopic image can be generated from a plurality of facial images including a facial portion of a target object using a time slicing imaging technique. Here, the time-divisional shooting technique is a technique for making the camera appear to be turning around the subject in a state where the subject suddenly stops. After a plurality of cameras are positioned around the subject and all the cameras simultaneously shoot the subject at a certain time, It is a technique to create an image centered on a subject using an image.

Accordingly, as shown in FIG. 2, when a plurality of wide-angle cameras 100 are used to photograph a specific object at the same time, an image such as a time-division photographing technique can be acquired based on the photographing.

In particular, since the wide-angle camera constructed in the present invention can acquire images taken in all directions, rather than photographing a specific direction, it is possible to acquire stereoscopic images of a plurality of object objects at the same time in one shot.

The stereoscopic image processing unit 140 may further include an object viewpoint confirmation module 141 and an object viewpoint image extraction module 142 as shown in FIG.

As described above, the object-view-confirmation module 141 analyzes an image captured by a plurality of wide-angle cameras 110 as shown in FIG. 3 by confirming an object viewpoint of a target object by checking a face-based stereoscopic image And the sight line of the target object (box arrow in FIG. 3) can be confirmed.

The object viewpoint image extraction module 142 is for allowing an object T to be viewed by the object object identified by the object viewpoint confirmation module 141 to be visually confirmed. As described above, the wide viewpoint camera 110 may acquire images for all directions. Therefore, as shown in FIG. 3, among the images photographed through the wide-angle camera 110, The image (the image in the direction indicated by the dotted line in Fig. 3) can be extracted.

The stereoscopic image synthesis unit 140 may generate and provide a stereoscopic image of an area viewed by the object based on the original image of the object view extracted from the object view image extraction module 142.

Accordingly, the present invention can check which region or object has been observed by the target object, for example, the criminal, and analyze the image to determine which crime the criminal intends to commit , It is possible to establish countermeasures to prevent such crime.

Also, in the case where a specific crime occurs, the present invention acquires an image of a time and an area where the crime occurred, as shown in FIG. 4, and obtains a crime-related object (T) And that the crime concerned can be resolved based on this.

On the other hand, in the case of a security and surveillance system based on a visual image, it may be difficult for a few administrators to confirm all the images installed in a large area. Hereinafter, a method for solving such a problem will be described.

5 to 8 are views for explaining embodiments of a method of providing a time-compressed image shown in FIG. 1, in which five persons are displayed in a corresponding surveillance area for a predetermined time.

2, the object verification unit 120 analyzes an omnidirectional image photographed for a predetermined period of time through the wide-angle camera 110 and generates five persons 'a' at different times t1 to t5, To 'e') appear in the monitored area.

If the object identifying unit 120 transmits information about the object to the stereoscopic image processing unit 140 through the object image processing unit 130, the stereoscopic image stereoscopic image processing unit 140 extracts each object 'a' 'To' e ') and then overlay the extracted objects' a' to 'e' in one background image as shown in the lower part of FIG.

At this time, the stereoscopic image processing unit 140 may adjust the transparency of the extracted object so as to overlap the background image.

When the administrator intends to observe the motion of a specific object in the process of checking the time compressed image, as shown in FIG. 6 (a), the corresponding object b in the time- ) Can be selected.

When a specific object is selected by the administrator as described above, the stereoscopic image method 140 may be applied to the stereoscopic image processing from the time t2 at which the corresponding object b appears to the time t2 'at which the corresponding object b disappears as shown in FIG. 6 (b) (Fig. 6 (c)) during the exposure time [Delta] t2 of the target object (b)

Accordingly, the administrator can check the image for a certain period of time for a short time through the process as shown in FIG. 5, and if necessary, can select a specific person to more clearly confirm the behavior or movement of the person.

In addition, the present invention can differentiate and display each object so that the order in which objects appearing in one background image can be visually recognized easily.

More specifically, the stereoscopic image processor 140 may correct the target object so that the target object is matched with the object display information set according to the photographed time zone, and then overlap the target objects with the background image. Here, the object display information is set for a method of overlapping the target object on the background screen. The object display information can be displayed by adjusting the transparency of each target object or adjusting the color, contrast, and saturation. Numbers, letters, symbols, and the like.

For example, if the time taken to capture the surveillance region (a certain amount of time) includes morning and afternoon times, the object shown in the morning time will be displayed with a red border line, and the object displayed in the afternoon time will be displayed with a blue border line Can be displayed.

In this way, when an object is differentiated and displayed according to a specific time period, an administrator can more easily grasp the relationship, movement, motion, and the like among a plurality of objects included in one image.

In addition, the stereoscopic image preparation unit 140 can confirm the time at which the target object was photographed as shown in FIGS. 5 and 6 (b) through the object verification unit 120, The target object can be corrected so as to match the object display information.

For example, the stereoscopic image processing unit 140 divides the entire time of the image to be checked into a plurality of time intervals, and then, according to the order of the respective time periods, as shown in FIG. 7A, , And saturation of each of the plurality of pixels.

As another example, the stereoscopic image preparation unit 140 may assign a number according to the order in which the object is displayed in the monitoring area, as shown in FIG. 7 (b).

By overlapping the object with visual differentiation according to the time series in this way, it is possible for the manager to more clearly confirm the relationship between the respective objects.

In addition, the stereoscopic image analysis unit 140 analyzes the motion of the target object in the process of overlapping the target object with the background image, and when the stereoscopic image analysis unit 140 determines that an alert situation (theft, accident, or the like occurs) can do. Here, the warning state may include a state in which the target object is blinking in red, or an exclamation point is set to be given.

Accordingly, the administrator can select the target object indicated with the warning status and clearly understand the behavior of the target object through the process as shown in FIG.

On the other hand, when the target object selected by the administrator is two persons (b and d), the stereoscopic image preparation unit 140 can provide an image at a normal time with respect to each target object.

8 (a), when each object overlaps a part of the time shown in the surveillance area, the stereoscopic image processing unit 140 differentiates and displays the two objects with respect to the overlapping time, The administrator can easily grasp the time-series sequence between two objects.

In other words, the stereoscopic image processing unit 140 can display the object display information by the gradation technique according to the elapsed time as shown in FIG. 8 (b), and can overlap the background image have.

Such a gradation technique is not limited to a case in which an image in which the target object is displayed is reproduced in a normal time, but the present invention can also be applied to a case where two target objects are overlapped and displayed simultaneously on one background image.

Accordingly, the present invention allows an administrator to check an image for a predetermined period of time in a short time, and also can easily grasp the time-series priorities among the target objects, thereby taking into consideration the relationship and the relevance among the target objects So that more clear object tracking is possible.

Fig. 9 is a configuration diagram showing another embodiment of Fig. 1. Fig.

Referring to FIG. 9, the facial recognition-based object tracking system 100 using the wide angle camera of the present invention may further include an object tracking unit 150.

The object tracking unit 150 analyzes a motion of a specific target object from the overlap image provided from the stereoscopic image processing unit 140 and calls a digital map of the corresponding region to confirm a predicted movement path of the corresponding object, The location where the wide-angle camera 110 is installed is checked and compared with the overlapped image, it is possible to predict in which direction the corresponding object moves on the digital map.

At this time, the facial recognition-based object tracking system 100 using the wide angle camera confirms the position of the camera installed in the digital map, identifies the camera monitoring the predicted movement path of the target object, You can make it traceable. Here, the existing camera may be interlocked with the IoT camera 211 capable of transmitting and receiving image data based on IoT.

In other words, the IoT camera 211 can operate in conjunction with the facial recognition based object tracking system 100 using a wide angle camera based on the object Internet, and at least another part of the predicted movement path (excluding the region photographed by the wide- Area), and provide at least some of the photographed images.

For this purpose, the facial recognition-based object tracking system 100 using the wide angle camera is interlocked with the IoT camera 211 on the basis of the object Internet as shown in FIG. 9, And an image acquisition unit 121 for providing the image data to the image processing unit 120.

Based on the predicted movement path confirmed by the object tracking unit 150, the stereoscopic image synthesis unit 140 generates a stereoscopic image based on the image provided from the wide angle camera 110 and the image provided from the IoT camera 211 through the image acquisition unit 121 The target object can be extracted from the background image and overlapped with the background image.

At this time, the stereoscopic image preparation unit 140 may provide the image provided from the IOT camera 211 as a wide-angle image by providing it to the administrator in order to provide the image with a constant quality.

More specifically, the stereoscopic image preparation unit 140 may correct the image provided from the IoT camera 211 to a wide-angle image using at least a part of the image provided from the wide-angle camera 110, have.

Therefore, the object tracking system 100 based on the facial recognition using the wide angle camera of the present invention can perform wide area surveillance using a camera already installed in a target surveillance area, so that an efficient wide area surveillance system can be constructed and operated at a minimum cost. .

The wide area monitoring system based on facial recognition using the wide angle camera according to the present invention has been described above. It will be understood by those skilled in the art that the technical features of the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

It is to be understood, therefore, that the embodiments described above are in all respects illustrative and not restrictive.

100: Wide area monitoring system
110: Wide angle camera 111: IoT camera
120: Object Identification Unit 121:
130: object image processing unit
140: Stereoscopic Imaging
150: Object tracking unit

Claims (10)

A plurality of wide angle cameras for acquiring omnidirectional images;
An object identifying unit for identifying a plurality of target objects and a face portion of the target object in an original image according to a shooting point photographed through the wide angle cameras; And
An object image processing unit for generating a facial image at a photographing time point by correcting an original image at each photographing time point so that a facial part of the object is located at the center of the image with respect to the original image of the photographing point at which the face part of the target object is identified;
A stereoscopic image generating unit for generating a facial image stereoscopic image for a face portion of a target object based on a facial image generated at the imaging time point generated by the object image processing unit;
An object viewpoint checking module for checking the object viewpoint of the target object by checking the face portion stereoscopic image; And
And an object viewpoint image extraction module for extracting an original viewpoint image corresponding to an object viewpoint identified by the object viewpoint confirmation module among the images photographed through the wide angle camera,
In the stereoscopic image processing,
A face slice stereoscopic image is generated from a plurality of facial images including a facial portion of a target object by using a time slicing photography technique and the face stereoscopic image is generated based on the original image of the object view extracted from the object view image extraction module A plurality of target objects included in the original image of the photographing point photographed through the wide angle cameras are extracted for a predetermined period of time, A time compressed stereoscopic image is generated and overlapped with the background image,
The time-compressed stereoscopic image may include:
Wherein the longest exposure time among the exposure times of the extracted object objects is determined as a total exposure time of the time compressed stereoscopic image.
The method according to claim 1,
In the stereoscopic image processing,
When the user selects a specific target object, provides a face stereoscopic image from a point of time when the target object is displayed to a point of time when the target object disappears, based on the facial recognition.
The method according to claim 1,
In the stereoscopic image processing,
Wherein the controller corrects the target object so that the target object is matched with the object display information set for each of the photographed time zones, and then overlaps the background image with the background image.
The method according to claim 1,
In the stereoscopic image processing,
Wherein when the object is determined to be in a warning state, the target object is overlapped with a warning state in a process of overlapping the target object with the background image. system.
The method according to claim 1,
And an object tracking unit for analyzing a movement of a specific object from the overlap image provided from the stereoscopic image system and calling a digital map of the corresponding area to confirm a predicted movement path of the object, Wide Area Monitoring System Based on Facial Recognition Using.
6. The method of claim 5,
An image collecting unit collecting an original image photographed through the wide angle camera and providing the collected original image to the object verifying unit; And
And an IoT camera capable of capturing at least another part of the predicted movement path and providing at least a part of the images taken in cooperation with the image collecting unit based on the object Internet,
In the stereoscopic image processing,
Wherein the target object is extracted from the image provided from the wide angle camera and the image provided from the IoT camera and overlapped with the background image based on the predicted movement path confirmed by the object tracking unit. A recognition - based wide area surveillance system.
The method according to claim 6,
In the stereoscopic image processing,
Wherein the at least one part of the image provided from the wide angle camera is used to correct an image provided from the IoT camera to a wide angle image and then the target object is overlapped with the corresponding image. delete delete delete

Images