KR101464192B1 - Multi-view security camera system and image processing method thereof - Google Patents

Abstract

The present invention relates to an image processing method of a multi-view security camera system. The multi-view security camera system includes: multiple security cameras which are installed in different positions; a wired/wireless network which mutually connects the security cameras; an image processor which receives and processes image information which is obtained from the security cameras through the wired/wireless network; and a database which stores information processed in the image processor. The image processor processes image information by passing: an object determining step of determining whether an attention object exists in an image or not by receiving the image obtained from the security cameras; an object extracting step of extracting a recognizable object if the attention object does not exists; an object converting step of converting multiple of unique information into three-dimensional object information by inserting the multiple of unique information into the extracted object; and an information storing step of extracting only key information from the three-dimensional object information and storing the key information in the database.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a multi-view security camera system and an image processing method,

The present invention relates to an image processing method of a multi-viewpoint security camera system, and more particularly, to a multi-viewpoint security camera system capable of correctly distinguishing information about a desired object from an image obtained through a multi- And a method of processing the same.

In recent years, various types of cameras have been installed in various places such as roads, India, residential areas, and alleys for real-time monitoring of traffic conditions, provision of traffic information, or various security purposes (hereinafter, ).

These security cameras may be integrated with a GPS chip or connected to a network such as a management server by means of wi-fi, Bluetooth or ZigBee depending on the purpose. For ease of management of security cameras, each security camera may have an identifier such as a unique IP address or the like.

In addition, the security camera can capture a multi-view image (a moving image obtained by photographing the same subject and a background with a plurality of cameras) by composing one security camera module by combining one or more cameras for a specific purpose, A zoom-out function and the like may be provided.

The security camera may be used to collect only simple information and use it for a specific purpose (for example, to view real-time traffic conditions), to find a specific object (for example, to find a hit-and-run vehicle in a traffic accident) It is possible.

Especially, there are increasingly examples of using photographed images of security cameras for the purpose of finding a specific object. In addition, there is an advantage that it becomes easier to find a specific object when using three-dimensional image information rather than simple two-dimensional image information, but there is a problem that accuracy is low.

Therefore, there is a need for a new image processing method capable of detecting a specific object more precisely from image information.

In addition, conventionally, the capacity occupied by the image data is so large that it is cumbersome and costly to store and manage. Therefore, it is necessary to reduce the amount of information while maintaining the quality of the image information.

An object of the present invention is to provide an image processing method of a multi-view security camera system that processes image information so that a specific object can be precisely detected from the image information.

According to an aspect of the present invention, there is provided a security camera system including a plurality of security cameras installed at different positions, a wired / wireless network connecting the security cameras, image information obtained from the security camera through the wired / And a database for storing information processed by the image processing apparatus, wherein the image processing apparatus comprises: an image processing unit for receiving an image acquired from the security camera, A target determining step of determining whether a set target object exists in the image; An object extracting step of extracting a recognizable object if the care object does not exist; An object transforming step of inserting a plurality of unique information into the extracted object and transforming the extracted object into three-dimensional object information; And extracting only the main information among the three-dimensional object information and storing the extracted main information in the database, and processes the image information through an information storing step.

An information calling step of calling the three-dimensional object information mapped to the attention object from the database if the object of attention exists in the object determining step; And an information mapping step of mapping the three-dimensional object information to the attention object.

And an information updating step of, after the information calling step, extracting new unique information of the subject of interest from the image and updating the three-dimensional object information.

The attention object is at least one of a human being, an animal, and an object.

The unique information includes at least one of identification information for identifying the object itself, movement route information, posture information, behavior information, biological information including face and body shape information and skeleton information, and meta information including captured time information .

And the face information is used as the identification information if the recognizable object is a person in the object extracting step.

And the main information is an optimum and minimum amount of essential information necessary for conversion into the three-dimensional object information.

In the object extracting step, the recognizable object may be at least one of a person, clothing, shoes, belongings, automobile, and animal.

The present invention also relates to a plurality of security cameras installed at different positions; A wired / wireless network interconnecting the security cameras; An image processing apparatus for receiving and processing image information acquired from the security camera through the wired / wireless network; And a database for storing only important information among the information processed in the image processing apparatus, wherein the plurality of cameras are modularized to form one security camera.

The image processing apparatus integrates and processes the images obtained from the security camera at different directions, distances, and different time points, and determines whether there is a pre-set attention target object from the integrated processed image.

The image processing apparatus extracts a recognizable object and inserts a plurality of unique information into the extracted object to convert the extracted object into three-dimensional object information, and then extracts only important information among the three-dimensional object information And stores it in the database.

Wherein the image processing apparatus maps the three-dimensional object information to the attention object by calling the three-dimensional object information mapped to the attention object from the database if the object of attention exists, Extracting new unique information of the object, and updating the three-dimensional object information.

The unique information includes at least one of identification information for identifying the object itself, movement route information, posture information, behavior information, biological information including face and body shape information and skeleton information, and meta information including captured time information .

The attention object is at least one of a person, an animal, and an object, and the face information is used as the unique information if the object of attention is a person.

And the image processed by the image processing apparatus is an image obtained from the security camera located in the vicinity of or in proximity to each other.

As described above, the image processing method of the multi-viewpoint security camera system according to the present invention efficiently designates, identifies, tracks, and manages a specific object by processing and storing image big data obtained from a plurality of multi-viewpoint security cameras.

In particular, it is possible to reduce the cost and increase the efficiency by minimizing the amount of information to be stored.

In addition, there is an advantage that the result of efficient processing of video big data can be utilized in various industrial fields.

1 is a block diagram illustrating a multi-view security camera system in accordance with an embodiment of the present invention.
2 is a flowchart illustrating an image processing method of a multi-viewpoint security camera system according to an embodiment of the present invention;

Hereinafter, an image processing method of a multi-viewpoint security camera system according to an embodiment of the present invention will be described in detail with reference to the drawings.

As shown in FIG. 1, the present invention includes a plurality of multi-viewpoint security cameras 100, a wired / wireless network 200 connecting them, And a database 400 for backing up only necessary data among the final processed image data may be implemented in a system.

A plurality of security cameras 100 are installed in various places such as a road, a house, a residential area, an alley, etc. For easy management, each security camera has an identifier such as a unique IP address. The security camera 100 may have a plurality of cameras configured in the form of one module, in which case each camera can be used for different shooting ranges and purposes.

For example, the first camera may be configured to capture a large area at a low resolution, and the second camera may be configured to photograph a narrow area in detail with a high-resolution telephoto. That is, when the first camera captures the background, the attention object recognized in this process (which means a specific object temporarily designated for a specific purpose, which means a person or a pet traveling alone, a specific person designated as a dangerous behavior object, Object, etc.), the second camera obtains a high-resolution image by precise tracking. To this end, a high-resolution CCD and a zoom-in / zoom-out function lens may be mounted on the second camera. In addition, the security camera 100 may further include a third camera that captures an image of another spectral range such as an infrared ray and a fourth camera that includes a high-sensitivity CCD that can capture images even in a low light amount environment It is possible.

The security camera 100 may include a GPS chip for acquiring location information and may be connected to the image processing apparatus 300 by a wired / wireless network 200 such as Wi-Fi / Bluetooth / ZigBee.

When the security camera 100 can not shoot a subject on a screen, the subject is photographed by a plurality of modular cameras, and the image processing apparatus 300 can combine the subject into a single high-resolution image. For this, the security camera 100 preferably has a function of rotating up and down and right and left and a function of recognizing a direction vector.

Alternatively, one object of caution may be photographed through a security camera 100 installed at various locations, and then the image processing apparatus 300 may integrate and process the object to convert the object into three-dimensional stereoscopic image data.

In this manner, one or more security cameras 100 can be cooperated to minimize blind spots, and images captured at various angles and distances can be obtained to more accurately identify and track attention targets.

Furthermore, when a security camera of a vehicle black box having a prior consent of a user is linked, mobility can be ensured, so that continuous image capturing data at various viewpoints and distances due to vehicle movement can be acquired and utilized. In the case of a public institution, not only a black box for a vehicle but also a video image of a camera mounted on a network-connectable digital device with prior agreement can be utilized in the same manner.

The image photographed from the above-described security camera 100 includes an identifier such as a unique IP address of the security camera, GPS information indicating the position of the camera, shooting time, vector information, meta information such as a distance / Stored in the security server 100 or in the management server or transmitted to the image processing apparatus 300 through the wired / Even if the image data thus stored or transmitted is stored in different management servers, when the same processing protocol is used, they are connected and integrated so that one big data (data amount, period, format, etc.) The conventional method constitutes a large amount of data which is difficult to collect, store, search, and analyze).

The image processing apparatus 300 processes the above-described image big data (meaning large data obtained by combining various image information obtained from security cameras), extracts information corresponding to a specific purpose from the processed image big data To the database 400 or a specific institution.

The image processing apparatus 300 can divide the image information into body information, face information, posture information, and object information. The image processing apparatus 300 extracts the processed information by dividing the processed information into skeleton information and movement pattern information, do. And inserts the above-described unique information into the object to be objectized and finally stores it in the database 400. [ This will be described in detail with reference to FIG. 2 (hereinafter, the subject of processing various data should be understood as an image processing apparatus, although not mentioned separately).

First, a method of processing body information is realized by integrating photographed images and objectizing a subject (the subject of attention described above).

(Viewpoint) images of the subject corresponding to the attention object in the image acquired from the security camera 100, and recognizes the subject of interest, which is common to these images, as a thing or a person (S100, S200) . If it is determined that the object commonly included in these images is the same object or the same person, the previously stored three-dimensional object information is called from the database 400 (S110), and the called three-dimensional object information is mapped to the attention object ). If there is new information corresponding to the attention object, the 3D object information to be stored in the database 400 is updated (S150).

If the 3D object information to be called does not exist, or if there is no object to be caught in the image, the recognized object is objectified to the 3D model (S300), and the related information is attached and analyzed based on the objectized object to extract the meaning of the object .

The three-dimensional model may be three-dimensional information of a polygonal base (a geometric three-dimensional model represented by a polygon) or an image three-dimensional model that extracts a plurality of images as a set (two phase- (Eg, cognitive intelligence that interprets images as stereoscopic images), or an algorithm that combines multiple images to generate an intermediate image.

Objectification is to extract a semantic element as an object of attention in image information, and to make a concrete object by gradually attaching information about the object as a substantial object of existence. More specifically, when the subject of interest in the video is extracted as a person, an identification ID is assigned to the person, and unique information is added around the ID. The unique information may include information for identifying the subject itself such as clothing and belongings, biological information such as route information, attitude information, behavior information, face and body information, and time information of the photograph.

Since the subject of attention is often a moving subject, the appearance of the subject being exposed to a plurality of security cameras at each time and position is slightly different. That is, even when the security camera 100 is located at the same position, the attention image photographed at different angles and at different angles can be obtained. Or the attention image can be taken via different security cameras 100 installed at different times at the same time or at different times. It is preferable to integrate the image information obtained from the security camera 100 located near each other among the security cameras 100 in order to extract the subject of interest more precisely.

Here, the proximity means a spatial relationship in which all or a part of the space or object included in the photographed image overlaps or overlaps with the image and can complement each other to grasp the meaning of the object recorded in the image, A set can be defined as a neighborhood system. Therefore, in the case of a moving subject, the nearby system, which is a nearby set, may change depending on the motion of the object.

The captured images of the subject of interest are converted into three-dimensional model information through a conventional three-dimensional model conversion method. In the three-dimensional model extracted from the image of the subject, the highest resolution and sharp image Is preferentially mapped.

If the objectized attention object is a person, the best identification method for distinguishing the object is the face information. Accordingly, if the subject is determined as a person, the image processing apparatus 300 processes the face information of the subject.

The face information basically refers to the expressionless face and stores all the facial images of each facial expression having the highest resolution among the facial expressions of the attention object. By multiplying and storing one or more main images of a subject, it is possible to increase the discrimination power of the attention object. In other words, the multiple mappings can be matched to one of the multiple mapped facial expressions even when the subject of attention has a different facial expression by mapping images of different facial expressions among the clearly captured images to one model, You can. The face information thus recognized and mapped can be used as the final ID for distinguishing the subject.

If the objected attention object is a creature including a person, attitude information can be further mapped.

In other words, joints can be inserted by integrating the attitude information read from the motion of the body into the three-dimensional object model converted from the image information. The object information of the subject is corrected while the attitude information is inserted based on the inserted joint. If the subject is a person, the characteristic of the posture such as walking, etc. can be an important clue for distinguishing a person.

The position of the joint can be extracted from the attitude and behavior of the attention object transformed into the 3D model, and the anatomical information of the attention object skeleton can be applied to the 3D model from the extracted joint position information. The posture and the behavior pattern are inserted based on the inserted three-dimensional skeleton information to combine the object information, thereby providing the same effect as compressing the data amount of the information on the attention object at a high ratio.

More specifically, since the spatial phase difference of each security camera 100 can be known in advance from the integrated image obtained by combining the images acquired through the security camera 100 forming the neighborhood, The joint position of the subject of interest can be extracted from the change (time series continuous change). The extracted joint positions are connected to form a skeleton, and a movement pattern (dynamic attitude or attitude, physical characteristics of the body skeleton) can be extracted based on the skeleton information.

Based on the skeleton information, the motion pattern can be extracted from the temporal motional structure of the continuous image. In this case, the most common motion is set as a basic pattern. If the motion classified by the basic pattern lasts for a predetermined time or continues in a certain space section, the motion can be replaced with a basic pattern. The basic pattern is extracted in this manner, and the motion patterns different from the basic pattern can be extracted separately and a plurality of motion patterns can be extracted. These movement patterns are continuously extracted, integrated and modified, converted into final movement patterns, and structurally classified and arranged, so that the movement of the attention object can be classified and extracted.

Based on the combined biological information of the subject, it is possible to simulate behavioral patterns such as footsteps and speed of movement. It is possible to utilize it to predict the next discovery location of the objectized three-dimensional attention object.

As described above, objects that can be converted into three-dimensional object information include people, clothing, shoes, personal belongings, automobiles, animals, and the like. Such objectification and conversion are performed not only by the security camera 100, It is possible to extract information matched with each object by collating all the combined images with the combined image big data.

After the process of integrating the image information acquired from the security camera 100 and converting the attention object into the three-dimensional object from the integrated information is completed, the pre-processing image information of the object recognized as the three- The remaining important information is stored in the database 400 (S400). In this case, the main information is deleted while leaving only the essential information (the optimum and minimum necessary information necessary for conversion into the three-dimensional object).

For example, an object having a narrow range of standards (such as an automobile) having a narrow range of standards (an object having a standard form as compared with other objects and easy to recognize) is a high resolution and sharp image And can extract object information, movement trajectory, and the like of the object through the video big data. After the motion is rendered, motion information is stored in the database 400, such as basic information for recognizing the object, clear images, motion trajectory, and object information.

In the case of objects having a wide range of standards such as personal belongings (objects whose standard shapes are not so different from other objects and whose shapes are so diverse that it is hard to recognize), the security camera 100 obtains the highest resolution and clear images It is preferable to leave a large number of images in order to increase the recognition rate of objects. Also, it extracts the changes of the user's wearing position and wearing position, and the motion locus from the video big data, combines them, and renders the motion. Thereafter, the necessary information is deleted and stored in the database 400. The stored information is more than an object that is easy to recognize, such as an automobile.

In the case of creatures such as dogs, information is processed and stored in the same manner as a person.

In addition to the above-described object information, the database 400 may additionally store spatial information. Here, the spatial information may be generated by combining three-dimensional shape information of a space acquired through the security cameras 100 located near each other and GIS (Geographic Information System) information. Temporal changes in climate and weather, changes in artificial lighting, and other unusual changes can also be stored together with the aforementioned spatial information. Combining the 3D shape information of the actual acquired space with the GIS information can minimize the blind spot of the image acquired through the security camera 100 and can efficiently utilize the position and height of the new camera have.

The storage and reproduction of data is compression-stored and reproduced according to a moving picture standard defined by Moving Picture Experts Group (MPEG), which is a moving picture standard mechanism. Since this moving picture compression technique is a commonly used technical standard, a detailed description thereof will be omitted do.

The image processing apparatus 300 integrates the previous object information and updates the new information when the object of specific attention is recognized from newly acquired image information.

The final data processed by the image processing apparatus 300 is stored in the database 400 and the new object information updated through the image processing apparatus 300 and the previous information of the previous object are also stored.

The final data stored in the database 400 are all three-dimensional information. In the database 400, the image processing apparatus 300 divides the final data into a human being or an object, and inserts position and motion information into the three- This is because only essential information among the information is stored.

When a specific institution designates an object of attention for a specific purpose, the image processing apparatus 300 compares the image obtained through the security camera 100 with the three-dimensional object information stored in the database 400 in real time, Extract the mapped information. Thereafter, when the attention object is recognized in the real time image, the image processing apparatus 300 updates the new information, stores it in the database 400, and can transmit the corresponding information to the above-mentioned specific institution.

The object information stored in the database 400 is not the simple image information obtained from one security camera 100 but the object information obtained from the plurality of security cameras 100 (obtained by integrating and extracting different temporal and spatial information) Because it is information, awareness of attention can be much more precise and accurate.

In the present invention, the image information previously stored in each security camera is extracted through a wired / wireless network to construct one image big data, extracts only a specific object from the configured image big data, And converts it into three-dimensional object information and stores only essential information. In addition, the stored 3D object information includes not only simple object information, but also various information (shape, skeleton, movement pattern, space, etc.) of a multi-point space in time and space, thereby facilitating separation between objects. This has the same effect as compressing and storing a large amount of object information which is difficult to use even if it is considered as the video big data itself, as the usable information.

While the preferred embodiments of the present invention have been described using specific terms, such description is for the purpose of describing the present invention only and is not to be taken in a way that will depart from the spirit and scope of the following claims .

100: Multi-point security camera 200: wired / wireless network
300: image processing apparatus 400: database

Claims (15)

A plurality of security cameras installed at different positions, a wired / wireless network for interconnecting the security cameras, an image processing device for receiving and processing image information acquired from the security camera through the wired / wireless network, And a database for storing information processed in the multi-point security camera system, the method comprising the steps of:
The image processing apparatus comprising:
A target determining step of receiving an image obtained from the security camera and determining whether a target object exists in the image;
An object extracting step of extracting a recognizable object if the care object does not exist;
An object transforming step of inserting a plurality of unique information into the extracted object and transforming the extracted object into three-dimensional object information; And
Extracting only the main information among the three-dimensional object information, and storing the main information in the database,
The information storage step
Wherein the security camera combines trajectories with a high-resolution and sharp-numbered image having the highest resolution among the image big data acquired by the security camera, and stores only essential information. The method according to claim 1,
An information calling step of calling the three-dimensional object information mapped to the attention object from the database if the object of attention exists in the object determining step;
Further comprising an information mapping step of mapping the three-dimensional object information to the attention object. 3. The method of claim 2,
Further comprising an information updating step of, after the information calling step, updating the three-dimensional object information by extracting new unique information of the attention object from the image. The method according to claim 1,
Wherein the attention object is at least one of a human being, an animal, and an object. The method according to claim 1,
The unique information includes at least one of identification information for identifying the object itself, movement route information, posture information, behavior information, biological information including face and body shape information and skeleton information, and meta information including captured time information And the image processing method of the multi-view security camera system. 6. The method of claim 5,
Wherein the face information is used as the identification information if the recognizable object is a person in the object extraction step. delete The method according to claim 1,
Wherein the recognizable object in the object extraction step is at least one of a human, clothing, shoes, belongings, automobile, and animal. delete delete delete delete delete delete delete

Images