KR20200010691A - Character recognition and traffic control system using heterogeneous camera - Google Patents

Abstract

The present invention relates to a face recognition and traffic control system using different kinds of cameras. More specifically, the system uses a thermal image camera, a fixed camera and a pan-tilt zoom (PTZ) camera at the same time in an interworking manner. The thermal image camera obtains a thermal image based on the temperature of a driver in a vehicle and obtains a thermal image based on the temperature of a pedestrian waiting for a traffic light. The fixed camera detects the face of the driver or pedestrian in a monitoring zone. The PTZ camera catches personal characteristics of the face recognized through the fixed camera and provides information for classifying the personal characteristics. Therefore, a specific face can be registered and managed based on skin color subdivision and shape information and face recognition using different kinds of cameras can be enabled and used to control traffic signals and traffic by identifying the existence of a pedestrian and his or her moving line.

Description

Character recognition and traffic control system using heterogeneous camera

The present invention relates to a person recognition and traffic control system using a heterogeneous camera, and more particularly, a thermal image camera, a fixed camera, and a PTZ (Pan-Tilt-Zoom) camera are simultaneously used in conjunction with each other. Acquire a thermal image through the body temperature of a driver, etc. in the interior, obtain a thermal image through the body temperature of a pedestrian waiting for a traffic light, and a fixed camera detects a person such as a pedestrian or driver in the surveillance area, and a PTZ camera It catches the personal characteristics of people recognized by fixed cameras and provides information to distinguish them. As a result, it is possible to register and manage a specific person based on color segmentation and shape information of the person's skin, and to recognize a person using a heterogeneous camera, and to control traffic lights and to control traffic by identifying the presence and movement of pedestrians using it. To provide a system.

Tracking recording, that is, tracking recording, means that one or more cameras having a fixed point and fixed to one point are tilted to correspond to a moving speed and direction of a corresponding subject toward a moving subject, and tracking the corresponding subject.

Also, a PTZ camera refers to a CCTV camera that zooms in and out along a subject from side to side. PTZ dome network cameras can monitor large areas with pan / tilt / zoom and provide detailed images through zoom-in.

PTZ dome cameras are the latest models to support features such as guard tours, tour recording, audio, input / output ports for external alarm devices, gatekeeper functions, AXIS Camera Application Platform for intelligent video applications, and weather resistance with temperature control. It's coming out.

For tracking shooting of a person using a conventional PTZ camera, at least two or more PTZ cameras are used to share the location of a plurality of cameras. And a system that can continue shooting by tracking the movement and the path of the shooting object by using a separate operation logic in consideration of the installation height and the like, but a large number of people and still objects exist simultaneously in a large surveillance area. In this case, there is a problem that it is very difficult to track and photograph the moving line by determining whether the person is a person or a moving object.

In the case of traffic lights installed in areas where traffic is rare, or traffic lights installed in areas where traffic is rare, the walking enable signal or the vehicle traveling signal of the traffic lights is too short for the walking population or the number of vehicles. Unnecessarily long turn-on problem occurs, and in the case of the elderly, there is a significant problem when the accident while waiting for a signal while traversing.

In order to overcome this problem, it is possible to use a thermal imaging camera to determine whether a vehicle is traveling and walking freely, and to distinguish between persons who exist in a large surveillance space from objects and recognize the available cameras and objects. Development of person recognition and traffic control system using more advanced heterogeneous camera that designates each person as an object and transmits location information about it to PTZ camera so that detailed tracking shooting of individual object is progressed through PTZ camera. Is required.

Republic of Korea Patent Application Publication No. 10-2014-0139914

In order to solve the above problems, it is possible to determine the presence of a driving vehicle and a pedestrian using a plurality of cameras including a thermal imaging camera, a fixed camera, a management server, and a PTZ camera, and to have a person recognition and traffic control system in the surveillance area. It is possible to provide a person recognition and traffic control system using heterogeneous cameras using multiple cameras capable of efficient recognition and tracking of people.

Person recognition and traffic control system using a heterogeneous camera according to an embodiment of the present invention for solving the above problems is installed in a fixed position thermal imaging camera or a fixed camera to shoot a surveillance area with a predetermined angle of view, Receiving image information of the thermal imaging camera or the fixed camera, receiving management information for processing, storing, and transmitting the location information of the person in the surveillance area based on the imaging information, and receiving the location information of the person in the surveillance area; Provided as a configuration of a person recognition and traffic control system using a heterogeneous camera including a PTZ camera that performs a pan tilt and zoom control based on the position information of the person to specifically extract, filter and transmit the specific information of the person to the server. It is possible.

A plurality of thermal imaging cameras or fixed cameras may be provided in the surveillance region, and the same or different surveillance regions may be photographed in different directions, and the person detection data may be calculated based on skin color of the persons in the surveillance region. It is preferable to calculate data for person detection based on face size and aspect ratio for the person in the surveillance area.

In addition, the fixed camera may generate a single numerical score by combining the calculated person detection data, and it may be preferable to calculate the color and pattern data of the clothes being worn for the person in the surveillance area.

The fixed camera may assign a serial number to the detected person, calculate a position coordinate of the detected person, and transmit the calculated coordinate to the management server.

The PTZ camera performs camera driving control by using the detected location information of each person received from the management server, extracts detailed information on each detected person through sliding window processing technique, and CNN learning model. It is possible to send it to the management server.

In addition to the system configuration described above, a camera photographing step of photographing a surveillance area using a thermal imaging camera or a fixed camera having a predetermined angle of view installed at a fixed position, and transmitting the imaging information of the thermal imaging camera or the fixed camera to a management server. PTZ server for receiving the location information of the person in the surveillance area from the management server, and panning and zoom control based on the location information of the person to shoot the person to extract and filter specific information specifically A similarity determination step of determining similarity by comparing the individual characteristics of the person according to the photographing step, the image of the thermal camera or the fixed camera with the individual characteristics of the person photographing the PTZ camera, and the result of the similarity determining step, the similarity is recognized. If it is, it is judged as the same object, and the movement of tracking and storing the movement of the object That the camera yijong a person recognizing and traffic control system utilizing the beam including a storage step can be provided.

The camera photographing step includes a moving object determining step of determining a moving object excluding a fixed object in the surveillance area, a person recognizing a face of the person based on the skin color of the person in the surveillance area or the face size and aspect ratio of the person in the surveillance area. As a result of the face recognition step and the face recognition step of the person, when the object is recognized as a face for the person, an object number may be assigned to each person, and an object number assignment and coordinate calculation step of calculating a real time position coordinate may be included.

The moving object determination step may determine whether the moving object in the surveillance area through the frame comparison algorithm.

In the face recognition step (S120), the person detection data may be calculated based on the skin color of the person in the surveillance area, or the person detection data may be calculated based on the face size and the aspect ratio of the person in the surveillance area.

The PTZ camera photographing step (S300) is a PTZ camera driving step (S310), a sliding window processing method, and CNN learning to perform a camera driving control by using the detected location information for each person received from the management server (300). It is possible to include a detailed information extraction and filtering step S320 for extracting and filtering detailed information about each detected person through the model.

It uses a thermal image camera, a fixed camera and a PTZ camera at the same time.In the case of a traffic light installed in an area where human traffic is rare or a traffic light installed in an area where traffic is rare, the traffic signal length is considered in consideration of the presence of a traffic vehicle and a pedestrian. The fixed camera detects the person in the surveillance area from the fixed object as an individual object, and the PTZ camera can catch the personal characteristics of the person and provide detailed information to distinguish them. Can be.

Due to such specific configuration characteristics, it is possible to register and manage a specific person based on color segmentation and shape information such as temperature data, human skin, etc., which analyzes the thermal image. In the surveillance area, it is possible to efficiently recognize a large number of people and to simultaneously perform multiple precise tracking of a large number of individual people.

1 is a schematic diagram illustrating a surveillance state in which a thermal imaging camera, a fixed camera, and a PTZ camera are installed to monitor a surveillance area according to an exemplary embodiment of the present invention.
Figure 2 is a schematic diagram showing the configuration of the thermal imaging camera, the fixed camera and the PTZ camera and the management server according to an embodiment of the present invention.
3 is a flowchart illustrating a person recognition and traffic control system using a heterogeneous camera according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are used to refer to the same components as much as possible, even if displayed on different drawings. In addition, in describing an embodiment of the present invention, if it is determined that the detailed description of the associated projectile configuration or function is to interfere with the understanding of the embodiment of the present invention, the detailed description thereof will be omitted.

In addition, in describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected or connected to that other component, but there is another component between each component. It will be understood that may be "connected", "coupled" or "connected".

1 is a schematic diagram illustrating a surveillance state in which a thermal imaging camera or a fixed camera and a PTZ camera are installed to monitor a surveillance area according to an exemplary embodiment of the present invention. 2 is a schematic diagram illustrating a configuration relationship between a thermal imaging camera or a fixed camera, a PTZ camera, and a management server according to an exemplary embodiment of the present invention.

Referring to Figures 1 and 2, identification of individual components that make up the entire system, including the configuration of multiple thermal imaging cameras or fixed cameras and PTZ cameras for systems capable of individual linkage tracking for multiple people in large surveillance areas. This is possible.

The present invention relates to a person recognition and traffic control system using a heterogeneous camera, and more particularly, a thermal image camera, a fixed camera, and a PTZ (Pan-Tilt-Zoom) camera are simultaneously used in conjunction with each other. Acquire a thermal image through the body temperature of a driver, etc. in the interior, obtain a thermal image through the body temperature of a pedestrian waiting for a traffic light, and a fixed camera detects a person such as a pedestrian or driver in the surveillance area, and a PTZ camera It catches the personal characteristics of people recognized by fixed cameras and provides information to distinguish them. As a result, it is possible to register and manage a specific person based on color segmentation and shape information of the person's skin, and to recognize a person using a heterogeneous camera, and to control traffic lights and to control traffic by identifying the presence and movement of pedestrians using it. To provide a system.

It uses a thermal image camera, a fixed camera and a PTZ camera at the same time.In the case of a traffic light installed in an area where human traffic is rare or a traffic light installed in an area where traffic is rare, the traffic signal length is considered in consideration of the presence of a traffic vehicle and a pedestrian. The fixed camera detects the person in the surveillance area from the fixed object as an individual object, and the PTZ camera can catch the personal characteristics of the person and provide detailed information to distinguish them. Can be.

Due to such specific configuration characteristics, it is possible to register and manage a specific person based on color segmentation and shape information such as temperature data, human skin, etc., which analyzes the thermal image. In the surveillance area, it is possible to efficiently recognize a large number of people and to simultaneously perform multiple precise tracking of a large number of individual people.

In more detail, in order to perform tracking photographing of a plurality of people in the large surveillance area 100, it is necessary to first distinguish the person from the fixed object, and to assign the individual to each person as a separate object. It will be stored, and according to the stored object, individual tracking recording or tracking shooting using a separate PTZ camera will be required.

1 and 2, the thermal imaging camera 150 or the fixed camera 200 configuration of the present invention can be used to detect people in the surveillance area 100, the PTZ camera 400 is a detected person Each characteristic may be searched in detail, and then contrasted with the discrimination information of the thermal imaging camera 150 or the fixed camera 200, followed by tracking and photographing.

In this system, it is possible to provide consistent tracking contents for a specific object (person or object) between PTZ cameras 400, and stable tracking is possible even when a person crosses each other or covers an object.

According to an embodiment of the present invention, as a process for distinguishing a specific person, segmentation of a person (skin) color and a shape telegram of the person may be used to identify, manage, and track a specific person.

In the conventional camera shooting system for tracking people or people in the surveillance area, the fixed camera cannot provide individual high resolution images for clear analysis such as face recognition and expression recognition for the moving object. There was a problem that it could not cope with the blockage caused.

In order to overcome this problem and to efficiently track a number of people in the surveillance area, a technique of tracking and photographing using a thermal camera 150 or a fixed camera 200 and a PTZ camera 400 simultaneously is required.

Specifically, in order to detect a person in the surveillance area, a process of detecting a person is performed based on a face shape and skin color of a moving object (person, etc.) existing in the surveillance area, and the thermal image camera is detected for the detected person. PTZ camera 400 received the position information of each person obtained from the 150 or the fixed camera 200 through the management server to perform tracking shooting for the detected person through the control operation of the PTZ camera 400 A process of acquiring a high-resolution distinguished image, overcoming overlapping and occlusion between an object or a person, and performing tracking photographing on the detected person may be performed.

To this end, the imaging information of the thermal imaging camera 150 or the fixed camera 200, the thermal imaging camera 150, or the fixed camera 200, which is installed at a fixed position and captures the surveillance region 100 with a predetermined angle of view. Receiving the location information of the person in the surveillance area 100 from the management server 300 and the management server 300 for processing, storing and transmitting the location information of the person in the surveillance area based on the captured image information. A person recognition and traffic control system using a heterogeneous camera including a PTZ camera 400 which performs pan tilt and zoom control based on the position information of the person, specifically extracts, filters, and transmits the specific information of the person to the server may be provided. Can be.

In this case, a plurality of the thermal imaging cameras 150 or the fixed cameras 200 are provided in the surveillance region 100, and it is possible to photograph the same or different surveillance regions in the same or different directions.

In addition, the thermal imaging camera 150 or the fixed camera 200 calculates data for detecting a person based on skin color of the person in the surveillance area, or detects data for the person based on the face size and aspect ratio of the person in the surveillance area. It is possible to calculate.

The thermal imaging camera 150 or the fixed camera 200 may be provided in plural, and each of the thermal imaging camera 150 or the fixed camera 200 corresponds to a PTZ camera or the thermal imaging camera 150 or the fixed camera 200. Linked with a greater number of PTZ cameras than the number of cameras 200, the positional coordinates and the characteristics of the person stored as an object through the recording of the thermal camera 150 or the fixed camera 200 to transmit the detailed connection Tracking is possible.

Through the above-described configuration, the entire information about each person is transmitted to the server using only a plurality of PTZ cameras in the related art, and a process of determining whether or not the person is solved solves a significant problem of data transmission load, resulting in rapid data transmission and reception. There is a feature that enables quick response and facilitates the securement of server storage memory.

Specifically, first, skin color segmentation or the like may be used to obtain an initial face set for the characters.

When using this method, the skin color of the person photographed by the fixed camera 200 is concentrated in the normalized color gamut, and the skin color distribution may be characterized as a multivariate normal Gaussian distribution under specific lighting conditions.

In this case, the image pixels of the fixed camera 200 may be projected in a normalized color space defined as red (Gr), green (Gg), and blue (Gb).

Each normalized color represents a component projected onto a pixel that exists in the normalized space, which is the sum of the components of red (Gr), green (Gg), and blue (Gb) plus red (Gr), green (Gg). ) And blue (Gb) can be expressed by the numerical ratio of the corresponding color.

In addition, it is possible to process offline skin tone samples to obtain Gaussian model parameters (mean and variance) and to generate a lookup table (LUT) used to accept or reject pixels based on their proximity to the Gaussian peak. The corresponding Gaussian distribution can be used.

In addition, by using the median-filtering technique, point noise noise can be removed from the captured image frame.

The median filtering is a filter technique capable of removing impulse noise while maintaining the viability of boundaries.

To this end, filtering is performed by aligning pixels with respect to an image in the mask area, taking an intermediate value, and storing the intermediate value.

The image from which the noise is removed is smoothed (I (x, y)), and horizontal and vertical projections of the image are calculated to obtain information about the face size, shape, and area of the person.

Based on information on the size, shape and area of the person's face obtained using the method described above, a fixed object is removed by removing a large number of face candidate areas that are not recorded using heuristics on the size and aspect ratio of the person's face. It is possible to clearly distinguish between and characters.

The fixed camera 200 registers the information of the person between the shooting frames based on the color of the skin, the area of the face, and the aspect ratio of the face, and combines the corresponding data into a single numerical score for the possibility that the person candidate area is a real face. A scoring method can be used to generate.

When the single numerical score is larger than a predetermined threshold, it is desirable to be recognized as a face of a person.

In addition, the fixed camera 200 may calculate the color and pattern data of the clothes being worn for the person in the surveillance area, assign a serial number to the detected person, and calculate the position coordinates for the detected person. It is preferable to transmit to the management server (300).

In this case, the PTZ camera 400 performs PTZ camera driving control by using the detected location information for each person received from the management server 300, and learns sliding window processing technique and CNN (Convolutional Neural Network). Through the model, the detailed information about each detected person can be extracted and transmitted to the management server.

The sliding window processing technique allows a transmission station to transmit an ordered PDU (Protocol Data Unit) in a window as a flow control method. In this case, the window may mean a processing technique of transmitting a PDU by dynamically changing.

3 is a flowchart illustrating a person recognition and traffic control system using a heterogeneous camera according to an embodiment of the present invention.

Referring to Figure 3, to describe the person recognition and traffic control system using a heterogeneous camera according to an embodiment of the present invention.

Specifically, a camera shooting step (S100), the thermal imaging camera 150 to shoot the surveillance area 100 by using a thermal imaging camera 150 or a fixed camera 200 having a predetermined angle of view installed in a fixed position Or a management server transmission step (S200) of transmitting the image capture information of the fixed camera 200 to the management server 300, receiving the location information of the person in the surveillance area 100 from the management server 300, the person According to the PTZ camera shooting step (S300), the thermal imaging camera 150 or the fixed camera 200 to capture and filter specific information by photographing the person by pan and zoom control based on the position information of the When similarity is recognized as a result of the similarity determination step (S400) and the similarity determination step (S400) of comparing the individual characteristics of the person and the individual characteristics of the person according to the shooting of the PTZ camera, the same object is determined.There is one character recognition and traffic control system using two kinds of cameras, including the copper-information storage step (S500) to store and track the circulation of each object can be provided.

In this case, the camera photographing step (S100) is a moving object determination step (S110) for determining a moving object excluding a fixed object in the monitoring area 100, a skin color of a person in the monitoring area 100 or a person in the monitoring area. As a result of the person face recognition step (S120) and the face recognition step (S120) of recognizing a face of a person based on a face size and an aspect ratio for the person, an object number is assigned to each person. And, it may include the object number assignment and coordinate calculation step (S130) for calculating the real-time position coordinates.

In the moving object determination step (S110), it is preferable to determine whether the moving object is in the surveillance area through a frame comparison algorithm. The face recognition step (S120) may include

It is preferable to calculate the data for detecting a person based on the skin color of the person in the surveillance area or to calculate the data for detecting the person based on the face size and the aspect ratio of the person in the surveillance area.

In addition, the PTZ camera photographing step (S300) is a PTZ camera driving step (S310) and a sliding window processing method for performing a camera driving control by using the detected location information for each person received from the management server 300 and the like; It is possible to include a detailed information extraction and filtering step S320 for extracting and filtering detailed information on each detected person through the CNN learning model.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: surveillance area
150: thermal imaging camera
200: fixed camera
300: management server
400: PTZ camera
S100: Camera shooting step
S110: moving object determination step
S120: Face recognition step
S130: object number assignment and coordinate calculation step
S200: management server transmission step
S300: PTZ camera shooting step
S310: PTZ camera driving step
S320: Extracting and filtering details
S400: Similarity judgment step
S500: copper wire information storage step

Claims (14)

A thermal imaging camera 150 or a fixed camera 200 installed at a fixed position and capturing the surveillance region 100 with a predetermined angle of view;
A management server (300) for receiving image information of the thermal imaging camera (150) or the fixed camera (200), and processing, storing, and transmitting position information of a person in a surveillance area based on the image information; And
Receives location information of a person in the surveillance area 100 from the management server 300, and performs pan tilt and zoom control based on the location information of the person to specifically extract, filter, and transmit the specific information of the person to the server. PTZ camera 400;
Portrait recognition and traffic control system using heterogeneous camera including
The method of claim 1,
The thermal imaging camera 150 or fixed camera 200,
A plurality of persons are provided in the surveillance region 100, and the person recognition and traffic control system using a heterogeneous camera, characterized in that to shoot the same or different surveillance region in different directions.
The method of claim 1,
The fixed camera 200,
Person recognition and traffic control system using heterogeneous cameras, which is characterized by calculating person detection data based on skin color of person in surveillance area
The method of claim 1,
The fixed camera 200,
Person recognition and traffic control system using heterogeneous camera, characterized in that to calculate data for person detection based on face size and aspect ratio of person in surveillance area
The method according to claim 3 or 4,
The fixed camera 200,
Person recognition and traffic control system using heterogeneous cameras, characterized by generating a single numerical score by combining the calculated person detection data
The method of claim 1,
The fixed camera 200,
Person recognition and traffic control system using heterogeneous cameras, characterized by calculating color and pattern data of clothes being worn for people in the surveillance area
The method of claim 5,
The fixed camera 200,
Person recognition and traffic control system using a heterogeneous camera, characterized in that the serial number is assigned to the detected person, the position coordinates for the detected person are calculated and transmitted to the management server 300.
The method of claim 1,
The PTZ camera 400,
Using the detected location information for each person received from the management server 300, etc., PTZ camera driving control is performed, and detailed information on each detected person is extracted through a sliding window processing technique and a CNN learning model. Person recognition and traffic control system using heterogeneous camera, characterized in that the transmission to the management server
A camera photographing step (S100) installed at a fixed position to photograph the surveillance region 100 using the thermal imaging camera 150 or the fixed camera 200 having a predetermined angle of view;
A management server transmission step (S200) of transmitting the imaging information of the thermal imaging camera (150) or the fixed camera (200) to the management server (300);
PTZ camera for receiving location information of a person in the surveillance area 100 from the management server 300 and extracting and filtering specific information by photographing a person by performing pan tilt and zoom control based on the location information of the person. Photographing step (S300);
A similarity determination step of determining similarity by comparing the individual characteristics of the person according to the shooting of the thermal camera or the fixed camera with the individual characteristics of the person according to the shooting of the PTZ camera (S400); And
When the similarity determination step (S400), if the similarity is recognized, it is determined as the same entity, and the copper wire information storage step (S500) for tracking and storing the movement of the object;
Portrait recognition and traffic control system using heterogeneous camera including
The method of claim 9,
The camera shooting step (S100),
A moving object determining step of determining a moving object except for the fixed object in the monitoring area 100 (S110);
A face recognition step (S120) of recognizing a face of a person based on a skin color of the person in the surveillance area 100 or a face size and an aspect ratio of the person in the surveillance area; And
As a result of the human face recognition step (S120), when it is recognized as a face for a person, an object number is assigned to each person, and an object number assignment and coordinate calculation step (S130) for calculating real-time position coordinates;
Person recognition and traffic control system using a heterogeneous camera comprising a
The method of claim 10,
The moving object determination step (S110),
Person recognition and traffic control system using heterogeneous camera characterized by judging whether a moving object is in the surveillance area through frame comparison algorithm
The method of claim 10,
The human face recognition step (S120),
Person recognition and traffic control system using heterogeneous cameras, which is characterized by calculating person detection data based on skin color of person in surveillance area
The method of claim 10,
The human face recognition step (S120),
Person recognition and traffic control system using heterogeneous camera, characterized in that to calculate data for person detection based on face size and aspect ratio of person in surveillance area
The method of claim 9,
The PTZ camera shooting step (S300),
A PTZ camera driving step (S310) of performing camera driving control by using the detected location information of each person received from the management server 300; And
Extracting and filtering detailed information on each detected person through a sliding window processing method and a CNN learning model and filtering the extracted information (S320);
Person recognition and traffic control system using a heterogeneous camera comprising a

Images