KR102603692B1 - Moving line tracking system and method for arrival hall using electronic tag of liability to crime travelers - Google Patents

Abstract

In the present invention, the step of comparing the preliminary analysis information on entrants with the traveler database of the traveler information system to search and designate the most criminal travelers among entrants, and facial recognition of entrants using video information captured by a CCTV camera installed at the immigration checkpoint A step of identifying and specifying a high-crime traveler by comparing it with the designated high-crime traveler information, a step of extracting the facial features, color and texture features, and gait features of the identified high-crime traveler, and generating and storing object characteristic information in real time, within the entry hall. A step of generating and storing object characteristic information in real time by extracting facial features, color and texture characteristics, and gait characteristics of entrants using video information captured by installed CCTV cameras. A step of detecting the identity and similarity of object characteristic information, a step of recognizing an entrant whose similarity to at least one of the object characteristic information of the entrant in the entry hall is extracted above a predetermined threshold and designating the person as a high-crime traveler, A real-time tracking system for offenders at the entry hall using electronic tags is disclosed, which includes additionally storing object characteristic information and providing it to the customs situation room and customs agents.

Description

Real-time tracking system and method for frequent travelers at the arrival hall using electronic tags {MOVING LINE TRACKING SYSTEM AND METHOD FOR ARRIVAL HALL USING ELECTRONIC TAG OF LIABILITY TO CRIME TRAVELERS}

The present invention relates to a system and method for attaching an electronic tag to a traveler's personal belongings declaration form and using it to track and monitor in real time travelers who commit criminal offenses within the entry hall of an airport or port, from the entry stage. This is about a system and method for tracking offenders at the entry gate using electronic tags that can efficiently and continuously track and thoroughly observe and monitor an entrant suspected of being a foreign national even if he/she changes clothes or disguises himself/herself to avoid CIQ customs procedures.

Today, as the movement of goods, capital, and services between countries and the number of domestic and foreign nationals entering and leaving the country are increasing explosively, it is realistically impossible to inspect the baggage of all incoming travelers at airports or ports.

Accordingly, customs offices around the world are proactively collecting information on travelers who are at risk of committing crimes, such as travelers with a record of smuggling weapons or foreign currency, smuggling, drug trafficking, or terrorism, or travelers who have not properly completed customs procedures in the past. It is seeking to improve the efficiency of customs inspections by obtaining information and using it as a database to intensively monitor and crack down on travelers who have committed criminal offenses as they pass through the port of entry within the country.

In recent years, as more and more people do not voluntarily report goods exceeding the duty-free range to customs, when travelers purchase expensive goods and bring them in, the Traveler Information System (TIS) is used to track and closely inspect the purchase details and information in advance. : Traveler Information System) has been established and operated.

In other words, customs collects passenger lists from airlines (hereinafter, including shipping companies) before the arrival of an aircraft (hereinafter, including ships) through Passenger Name Record (PNR) and Advance Passenger Information System (APIS). After obtaining the information, the information analyst analyzes the passengers' travel history, destination, use of duty-free shops, etc. in advance, and provides the analysis results for each passenger to the immigration inspector and government agencies (agencies) to screen and inspect the most serious travelers. It is being passed quickly.

At this time, the traveler information system is used to compare the traveler's preliminary analysis information with the high-crime traveler database to designate and determine test subjects.

In addition, the movement of inspection subjects is tracked using CCTV cameras installed along the movement lines of incoming immigrants, customs agents (rovers) receive information on inspection subjects via PDA and monitor their movements, and inspection designated officers (marshals) are on the marshal line. Entrants passing through the marshal line are guided to the customs inspection station.

However, due to recent changes in the entry hall environment, such as unmanned automation and the installation of partitions at the immigration office, there are not only limitations in accurately identifying the face of the test subject from CCTV video information, but also customs and immigration that the test subject must go through. If you change clothes or disguise yourself to avoid the three customs procedures (immigration and quarantine) or to elude customs agents, tracking and surveillance becomes quite difficult.

Meanwhile, the Intelligent Video Surveillance (IVS) system separates the background area from the video information obtained from CCTV and then extracts, recognizes and tracks moving objects, i.e. surveillance targets, and monitors the identified objects or the objects being tracked. Generates an event appropriate for the situation.

Methods for separating moving objects in this intelligent video surveillance system include Gaussian mixture model (GMM) and background subtraction.

Methods for identifying moving objects in an intelligent video surveillance system include contour-based classification and motion-based classification methods. In particular, contour-based classification uses data sets of dots, box-shaped shapes, outlines, and blobs from extracted moving objects. Classify using

Methods for using color information of moving objects in an intelligent video surveillance system include Mean Shift and CAMshift algorithms. These algorithms extract objects at high speed using the color density distribution of the selected search window.

Methods for tracking moving objects, that is, techniques for finding moving objects identified in previous images in the current image include region-based tracking, active contour-based tracking, and feature-based tracking. It can be divided into four categories, including based tracking and model-based tracking.

Here, the area-based tracking algorithm is a method that uses changes in the image corresponding to the moving object or structural characteristics (usually the tracking object is a human and uses the structure of the human head, torso, lower body, etc.), and uses the unique characteristics of the object. Tracking performance can depend on how you model it.

The active contour-based tracking algorithm is a method of tracking objects by expressing the contour using the contour boundary and dynamically updating the contour in consecutive frames. Due to the premise that the contour must be extracted, tracking performance is determined by whether or not the contour is extracted. In addition, it is very sensitive to tracking initialization, making automatic tracking difficult.

The feature-based tracking algorithm is a method of recognizing and tracking objects by matching features between images through feature elements and high-level clustering. It is easy to track when a large number of features of a moving object are secured, but requires a large amount of computation to extract feature points. , there are problems such as limitations (size, color combination, etc.) of moving object images for feature point extraction.

The model-based tracking algorithm is a method that measures the structural characteristics of an object in advance, models it, and then matches it, and is mainly used to recognize object behavior.

Meanwhile, Incheon International Airport Corporation provides AI-based autonomous driving and various advanced information and communications (ICT) services to support the convenience of transportation for the transportation vulnerable within the airport, such as at the arrival and departure halls, and to strengthen various smart services such as temperature checks and provision of information about airport facilities. We have introduced and are operating a guidance robot that incorporates technology.

In addition, it has cleaning capabilities, an autonomous driving function equipped with LiDAR and vision sensors, obstacle avoidance technology, and a database of areas that need cleaning within the airport, allowing it to recognize the surrounding situation and move freely inside the building on its own. We have introduced and are operating an unmanned, self-driving cleaning robot that cleans and a service robot that follows travelers and carries their luggage for them.

Meanwhile, the technique of tracking a person using a mobile robot is a method in which the mobile robot continues to follow a pre-designated person, that is, using the characteristic points of the specific person while maintaining a certain distance from the specific person.

For example, using feature point extraction algorithms such as SURF (Speed Up Robust Features), SIFT (Scale-Invariant Feature Transform), ORB (Oriented FAST and rotated Binary robust independent elementary feature), and RGB histogram, feature point scores are based on recognition of a specific person for each frame. is calculated, feature points for a specific person designated in advance are extracted using an object recognition method, and the specific person is continuously tracked using the extracted feature points.

In this case, not only does the tracking target, a specific person, often fail to be missed due to inaccuracy in calculating feature points, but also the identifier for the specific person is There was a problem that required re-allocation.

Additionally, when the distance between a specific moving person and the mobile robot increases or an obstacle is located between the specific person and the mobile robot, there is a problem in which the mobile robot cannot determine an appropriate driving path to track the specific person.

Meanwhile, most countries require travelers to fill out a customs declaration form when bringing declared goods (alcohol, tobacco products, animals, fresh food, plant materials, seeds, soil, meat and animal products) across their borders.

In other words, travelers must report everything they have acquired abroad upon entering the country, and for this purpose, they fill out and submit the so-called traveler's personal belongings declaration form at the port of entry, including the traveler's personal information and whether they possess items subject to reporting.

Meanwhile, Radio Frequency Identification (RFID) uses radio frequencies to read or record information in a non-contact manner from a tag containing unique identification information, thereby recognizing tagged objects, animals, people, etc. , it is a technology that allows tracking and management.

The RFID system consists of a number of tags (electronic tags or transponders) that carry unique identification information and are attached to objects or animals, and an RFID reader (reader or interrogator) to read or write the information on the tags.

These RFID systems are divided into mutual induction and electromagnetic wave methods depending on the mutual communication method between the reader and tag. They are divided into active and passive types depending on whether the tag operates with its own power. Depending on the frequency used, long wave, medium wave, It is divided into short wave, ultra-short wave and ultra-high wave types.

Meanwhile, object location measurement technology using an RFID system generally measures the location of an object using an RF reader with an RF tag attached to the object. The object location recognition methods used at this time include RSSI (Received Signal Strength Indication), which recognizes according to the signal strength that decreases with distance, and TDOA (Time Difference Of Arrival), which uses the time difference between the signal from the object to the RF reader. , TOA (Time Of Arrival), which uses the arrival time of the signal, and AOA (Angle Of Angle), which measures the object position using the angle.

In particular, the method of measuring distance using RSSI is to measure the signal strength of a tag that receives a signal according to the distance from a fixed reader. Considering the fact that the signal strength decreases with distance due to the physical characteristics of the transmitted signal, the tag This is a method of measuring the distance between a tag and a reader by measuring the strength of the signal's power loss or path loss and the strength of the attenuated received signal when the reader receives the transmitted signal.

The background or prior art described above herein is information possessed by the present inventor or acquired in the process of deriving and completing the present invention, and is helpful in understanding the technical significance of the present invention and useful for prior art search and examination. It is to be clarified that this does not mean a technology that is generally known and widely used in the technical field to which the invention pertains before the application of the present invention.

KR 10-2015-0094074 A(2015.08.19) KR 10-2018-0040255 A(2018.04.20) KR 10-1808587 B1(2017.12.07)

Accordingly, the present inventor comprehensively considered all of the above-mentioned matters and, at the same time, with the idea of solving the technical limitations and problems of the existing traveler tracking system in the entry hall, the inventor sought to allow a serious traveler who passed the immigration checkpoint to avoid the CIQ customs clearance procedure. Even if you disguise yourself, such as wearing heavy makeup, glasses, or a mask, or change your clothes, you can efficiently track your movement path using the electronic tag attached to the traveler's personal belongings declaration form and thoroughly observe and continuously monitor it. This ensures the accuracy of traffic control. Of course, the present invention was created as a result of continuous research with great effort to develop a new system and method for tracking offenders in the entry hall with a new structure that could increase the efficiency of customs work.

Therefore, the technical problem and purpose to be solved by the present invention is a tracking system for travelers with criminal offenses at the entry port that uses electronic tags attached to the traveler's belongings declaration form to continuously track and monitor the movement of criminal travelers within the entry port even if they change clothes or disguise themselves. The purpose is to provide a method.

Here, the technical problems and objectives to be solved by the present invention are not limited to the technical problems and objectives mentioned above, and other technical problems and objectives not mentioned can be understood from the description below as a matter of common knowledge in the technical field to which the present invention pertains. Those who have it will be able to understand it clearly.

Specific means according to the aspect of the present invention for effectively achieving a specific technical purpose while embodying a new idea for solving the technical problem of the present invention as described above include preliminary analysis information on entrants and traveler information. A criminal traveler information input unit that compares the system's criminal traveler databases to receive information on criminal travelers among entrants, a first video information input unit that receives video information from a plurality of first CCTV cameras installed at the immigration checkpoint, and the first video information. A criminal traveler detection unit that identifies the object of the entrant from the image information input through the input unit and detects the criminal traveler by comparing it with the criminal traveler information input through the illegal traveler information input unit, the facial characteristics, color and A first object characteristic information extraction unit that extracts texture characteristics and gait characteristics to generate and store object characteristic information in real time, and a second image information input unit that receives image information from a plurality of second CCTV cameras installed in the arrival hall. , a second object feature information extraction unit that generates and stores object feature information in real time by extracting facial features, color and texture features, and gait features of entrants from the image information input through the second image information input unit, the first The first similarity detection unit, which detects the identity and similarity between the object characteristic information of the serious traveler generated in the object characteristic information extraction unit and the object characteristic information of the entrants in the arrival hall generated in the second object characteristic information extraction unit, among the entrants in the arrival hall. The first traveler identification unit, which recognizes entrants whose similarity value detected by the first similarity detection unit is higher than a predetermined threshold and identifies them as the traveler concerned, a plurality of first CCTV cameras installed at the immigration checkpoint, and a plurality of first CCTV cameras installed in the entry hall A second CCTV camera, each identification information and arrangement information for a plurality of self-driving robots and readers placed in the arrival hall, a storage unit for storing object characteristic information and image information of a specified high-crime traveler, and an object of a specific high-crime traveler A transmission unit that provides characteristic information and image information to the customs situation room, customs agents (rovers), and inspection officers (marshal), a control unit that controls the functions of the autonomous robots and the overall operation of the system, and the autonomous robots. and a data communication unit that communicates data wirelessly, extracting the facial features, color and texture features, and gait features of entrants from the image information received through the data communication unit and captured by the cameras of the self-driving robots, and providing object characteristic information. A third object characteristic information extraction unit that generates and stores in real time, between the object characteristic information of a serious traveler identified in the first criminal traveler specification unit and the object characteristic information of entrants in the arrival hall generated by the third object characteristic information extraction unit. A second similarity detection unit that detects identity and similarity, a second traveler specification unit that recognizes entrants whose similarity value detected by the second similarity detection unit is higher than a predetermined threshold and re-identifies them as high-crime travelers, and receives information from the self-driving robots. A location recognition unit that recognizes each location in real time based on image information and GPS, a GIS display unit that displays the location information of the self-driving robots on the display, and a traveler who is a criminal obtained by taking pictures with the camera of the self-driving robots. An event alarm unit that outputs the included video information to the display unit in real time, is installed in a plurality of units for each zone in the arrival hall, and receives the signal strength value of the electronic tag attached to the traveler's belongings declaration form carried by an entrant located within a certain distance, and A reader receives unique identification information, and the received signal strength value and unique identification information of the electronic tag are collected from the reader to generate location information of the entering person, which is then combined with the information related to the serious traveler specified in the second serious traveler identification unit and the above. If it is within a preset standard range by comparing the location information of the criminal traveler based on the location information of self-driving robots, the unique identification information of the electronic tag attached to the traveler's belongings declaration form carried by the entering person is matched to the criminal traveler. We present a real-time tracking system for offenders at the entry hall using electronic tags, which includes an information combination unit that combines the offenders with the offenders entering the country.

As a result, the present invention not only allows a traveler who has passed through immigration at an airport or port to be tracked along his or her movement line even if he or she disguises himself or changes clothes, such as wearing heavy makeup, glasses, or a mask, within the entry hall to avoid CIQ customs procedures. Even if a serious traveler is hiding or located in a blind spot in the arrival hall outside the shooting range of a fixed CCTV camera, the self-driving robot can be used to collect and analyze object characteristic information taken close up from various angles from a height of about the height of a person, making it more efficient. This allows for thorough observation and surveillance by detecting and tracking erroneous travelers.

In addition, in a preferred aspect of the present invention, the control unit receives image information from the self-driving robots, uses the received image information to confirm the location of the re-specified traveler, and uses the received image information to determine the location of the re-specified traveler. Based on the relative location information corresponding to the distance between the offender and a specific place in the arrival hall, if the relative location information is outside the standard range, the direction of movement of the autonomous robot (R) and the other autonomous robot closest to it is determined. It is possible to control the change to the specific location and re-confirm the location of the re-specified traveler using image information received from the other self-driving robot.

And the specific means according to another preferred aspect of the present invention is the step of comparing the preliminary analysis information on entrants with the high-crime traveler database of the traveler information system to query and designate high-crime travelers among entrants, installed at the immigration checkpoint. A step of recognizing and detecting the face of an entrant passing through the immigration checkpoint using video information captured by a plurality of first CCTV cameras, and comparing this with designated high-crime traveler information to identify and detect the high-crime traveler, facial characteristics of the detected high-crime traveler, A step of extracting color and texture features and gait features to generate and store object feature information in real time; facial features, color and texture features, and gait of entrants using image information captured by a plurality of second CCTV cameras installed in the entry hall Extracting features and generating and storing object characteristic information in real time, detecting the identity and similarity of the created object characteristic information of the criminal traveler and the object characteristic information of entrants in the arrival hall, at least one of the object characteristic information of entrants in the arrival hall A step of recognizing an entrant whose similarity is greater than a predetermined threshold and specifying the person as a criminal traveler; additionally storing the object characteristic information and image information of the identified criminal traveler; and sending information to the customs situation room, customs agent (rover), and inspection sheet. A step of providing to the marshal, a step of generating and storing object feature information in real time by extracting facial features, color and texture features, and gait features of entrants from the image information obtained by cameras taken by self-driving robots in the arrival hall; Detecting the identity and similarity between the created object characteristic information of entrants in the entry hall and the object characteristic information of already identified high-crime travelers, entrants whose similarity to at least one of the object characteristic information of entrants in the arrival hall is detected to be higher than a predetermined threshold A step of recognizing and re-identifying as a serious traveler and recognizing each location in real time based on image information and GPS received from self-driving robots, wherein the information combination unit determines the received signal strength value of the electronic tag from the reader. and unique identification information is collected to generate location information of the entrant, and the location information of the offender and the location of the entrant are based on the information related to the offender specified by the second offender traveler specification department and the location information of the self-driving robots. Comparing the information and if it is within a preset standard range, matching the unique identification information of the electronic tag attached to the traveler's belongings declaration form carried by the entering person to the high-crime traveler and combining the high-crime traveler with the entrant, and the GIS display unit We present a method for tracking criminal travelers in an entry hall, which includes the step of displaying the location information of the criminal travelers and the entrants combined in the information combination unit through a display unit.

Accordingly, the present invention allows for efficient tracking and thorough observation of criminal travelers who have passed through immigration at an airport or port, along their movement lines, even if they disguise themselves or change clothes, such as wearing heavy makeup, glasses, and a mask, within the entry hall to avoid CIQ customs clearance procedures. and can be monitored.

In addition, as a preferred aspect of the present invention, the location of the re-specified high-crime traveler is confirmed using image information received from the autonomous robots during the above step, and the re-specified high-crime traveler and a specific location within the entry hall are confirmed. Based on the relative position information corresponding to the distance between them, if the relative position information is outside the reference range, control the movement direction of the autonomous robot and another autonomous robot closest to the specific location to be changed to the specific location, and The location of a re-identified traveler can be re-confirmed using image information received from another autonomous robot.

In addition, a preferred aspect of the present invention is to compare the facial feature image of the criminal traveler extracted during the above step with the feature point descriptor extracted from the facial feature image of the entrants in the entry hall and perform feature point matching to find similar feature points, thereby ensuring the identity of the facial features. and similarity can be detected.

In addition, a preferred aspect of the present invention is to generate a color histogram from the color feature images of the criminal travelers and the color feature images of the entrants in the entry hall extracted during the above steps, extract the main image features, and compare them with each other. The identity and similarity of color features can be detected only when it exceeds a preset threshold.

In addition, a preferred aspect of the present invention is to perform image segmentation on the texture feature images of the serious traveler extracted during the above step and the texture feature images of the entrants in the entry hall, and then determine the Minimum Boundary Rectangle (MBR) of the object. By configuring and applying wavelet transform to the MBR of the object included in each image, texture features can be generated and their identity and similarity can be detected.

In addition, a preferred aspect of the present invention is to obtain a binary silhouette video sequence from image information captured by a plurality of first CCTV cameras installed at the immigration checkpoint during the above step, and then obtain a silhouette image normalized from the silhouette video sequence. After extracting identifiable gait feature information from the obtained and normalized silhouette image, the extracted gait feature information is compared with the gait feature information of the image information obtained by shooting a plurality of second CCTV cameras installed in the arrival hall to determine the gait feature information. Identity and similarity can be detected.

According to the technical idea and embodiment of the present invention, which is based on a unique solution to solve the above technical problems, a serious traveler who has passed the immigration checkpoint at an airport or port enters the port of entry to avoid CIQ customs procedures. Even if you disguise yourself, such as wearing heavy makeup, glasses, or a mask, or change your clothes, you can thoroughly observe and continuously monitor by tracking your movements more efficiently by using the electronic tag attached to the traveler's carry-on declaration form.

In other words, the location information of the offender travelers based on the information related to the specified travelers and the location information of the autonomous robots (R), and the location information of the entrants collected by the readers installed in each area within the arrival hall are used to identify the offenders and entrants. By monitoring and tracking by combining them in a matching manner, it is possible to increase the accuracy and reliability of customs control as well as improve the efficiency of customs work.

Furthermore, autonomous robots such as guide robots or cleaning robots operated within the arrival hall can be used to check the location and movement of offenders in real time, such as in cases where the density of entrants within the arrival hall is high, inside restrooms or duty-free shops, or behind pillars or sculptures. Even if a criminal traveler is hiding or located in a blind spot within the arrival hall outside the shooting range of a fixed CCTV camera, object characteristic information taken close-up from various angles from a height of about the height of a person can be collected and analyzed, thereby improving the reliability and tracking of criminal traveler detection. Efficiency can be improved.

Here, the effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description of the claims.

Figure 1 is a block diagram schematically showing a real-time tracking system for serious travelers in an entry hall using an electronic tag according to an embodiment of the present invention.
Figure 2 is a flowchart illustrating a method of tracking a serious traveler in an entry hall using an electronic tag according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in more detail with reference to the attached drawings.

Prior to this, the terms described below are defined in consideration of their functions in the present invention, and should be interpreted as concepts consistent with the technical idea of the present invention and meanings commonly used or commonly recognized in the relevant technical field.

Additionally, if it is determined that a detailed description of a known function or configuration related to the present invention may obscure the gist of the present invention, the detailed description will be omitted.

The drawings attached herein may be shown with exaggerated or simplified portions to facilitate explanation and understanding of the composition and operation of the technology, and each component in the drawings does not exactly match the actual size and shape. Reveal.

In addition, in this specification, the term and/or refers to a combination of a plurality of related described items or includes any item among a plurality of related described items, and when a part includes a certain element, this means that it is specifically opposed to Unless otherwise stated, it does not mean excluding other components, but rather including other components.

In other words, terms such as 'include' or 'have' used in this specification mean the existence of features, numbers, steps, processes, operations, components, parts, or a combination thereof. It should be understood that it does not exclude the presence or addition of other features, numbers, steps, processes, operations, components, parts, or combinations thereof.

In addition, each process and step may occur differently from the specified order unless a specific order is clearly stated in the context. That is, each process and step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the opposite order.

Meanwhile, the meaning of “unit” and “unit” used in the present invention refers to a module that plays a role or a unit that processes at least one function or certain operation desired in a device or system, which is hardware, software, or It can be implemented through a combination of hardware and software, or as a device or assembly that can perform independent operations.

And the meaning of "module" used in the present invention may mean a unit containing one or a combination of two or more of hardware, software, or firmware, and may include unit, logic, logic, etc. It can be used interchangeably with terms such as logical block, component, and circuit, and can be the minimum unit of an integrated part or a part thereof, and performs one or more functions. It may be the smallest unit or a part thereof, and may be implemented mechanically or electronically.

In addition, terms such as top, bottom, upper surface, lower surface, or upper, lower, upper, lower, front, back, left and right used in the present invention are used for convenience to distinguish the relative positions of each component. For example, the upper part of the drawing may be referred to as upper and the lower part as lower, the longitudinal direction may be referred to as the front-to-back direction, and the width direction may be referred to as the left-right direction.

Additionally, terms such as first and second used in the present invention can be used to describe various components. That is, terms such as first, second, etc. can be used only for the purpose of distinguishing one component from another component.

As shown in Figure 1, the real-time tracking system for serious travelers in the arrival hall using an electronic tag according to an embodiment of the present invention includes a plurality of first CCTV cameras (C1) installed not overlapping at the immigration checkpoint and Image information obtained by filming by a plurality of installed first CCTV cameras (C2) and video information obtained by filming by a plurality of self-driving robots (R) placed in the arrival hall can be input and displayed on the screen.

In addition, the system for tracking criminal travelers in the entry hall according to an embodiment of the present invention is connected to a plurality of first and second CCTV cameras (C1) (C2) through a wireless or wired network, respectively, and a plurality of autonomous robots (R) Since data communication is possible through a wireless network, images captured from each of them can be transmitted in real time, or data can be transmitted or received.

In addition, various information such as identification information and placement information for the plurality of first and second CCTV cameras (C1) (C2) and the plurality of self-driving robots (R) can be registered and stored.

Here, the information for the first and second CCTV cameras (C1) (C2) and the information for the autonomous robot (R) may include a unique ID and installation location information assigned to each.

Meanwhile, the system for tracking criminal travelers in the entry hall according to an embodiment of the present invention is implemented with a hardware configuration such as a system device and a server, or includes a plurality of first and second CCTV cameras (C1) (C2), a plurality of autonomous robots (R ) Of course, it can also be implemented in the form of software that is mounted on a device or system communicatively connected to the like and performs the processing of the present invention.

Specifically, the main elements that make up the criminal traveler tracking system in the entry hall according to the embodiment of the present invention are the criminal traveler information input unit 11, the first image information input unit 12, the criminal traveler detection unit 13, and the first feature. Extraction unit 14, second image information input unit 15, second feature extraction unit 16, first similarity detection unit 17, first criminal traveler specification unit 18, storage unit 19, transmission unit (20), control unit 21, data communication unit 22, third feature extraction unit 23, second similarity detection unit 24, second criminal traveler identification unit 25, location recognition unit 26, GIS It includes a display unit 27, a display unit 28, an event alarm unit 29, a reader 30, an information combination unit 31, and an administrator input unit 32.

Here, each component that constitutes the system for tracking criminal travelers in the entry hall according to an embodiment of the present invention can be understood as a logically distinct component rather than a physically distinct component.

In other words, each component corresponds to a logical component for realizing the technical idea of the present invention, so if the function performed by the logical configuration of the present invention can be realized even if integrated or separated from each other, it should be interpreted as being within the scope of the present invention. , of course, any component that performs the same or similar function should be construed as being within the scope of the present invention, regardless of consistency in name.

The crime-prone traveler information input unit 11 compares the preliminary analysis information on entrants with the trespasser traveler database of the traveler information system and serves to receive information on trespassers among entrants.

The first image information input unit 12 serves to receive image information captured and transmitted by a plurality of first CCTV cameras C1 installed at the immigration checkpoint.

Here, the first image information input unit 12 can provide a network interface function for receiving image information from a plurality of first CCTV cameras (C1) installed at the immigration counter.

The criminal traveler detection unit 13 processes images captured and transmitted by a plurality of first CCTV cameras C1 installed at the immigration checkpoint to identify the object of the entrant from the video information, and the illegal traveler information input unit 11 It plays a role in detecting anomalous travelers by comparing them with the anomalous traveler information entered through .

The first feature extraction unit 14 extracts the facial features, color and texture features, and gait features of detected criminal travelers and generates and stores object feature information in real time.

In other words, the first feature extraction unit 14 recognizes a criminal traveler from image information obtained by filming by a plurality of first CCTV cameras C1 installed at the immigration checkpoint, and extracts object characteristic information by learning the recognized criminal traveler. do.

The second image information input unit 15 serves to receive image information captured and transmitted by a plurality of second CCTV cameras (C2) installed at the arrival hall.

Here, the second image information input unit 15 can provide a network interface function for receiving image information from a plurality of second CCTV cameras (C2) installed at the arrival hall.

The second feature extraction unit 16 extracts facial features, color and texture features, and gait features of entrants in the arrival hall, and generates and stores object feature information in real time.

That is, the second feature extraction unit 16 extracts object characteristic information of entrants from image information obtained by filming by a plurality of second CCTV cameras (C2) installed at the arrival hall.

The first similarity detection unit 17 serves to detect the identity and similarity between the generated object characteristic information of a serious traveler and the object characteristic information of entrants in the entry hall.

The first criminal traveler identification unit 18 recognizes entrants whose similarity to at least one of the object characteristic information of entrants in the entry hall matches a predetermined threshold or higher and identifies them as a trespassing traveler.

In other words, the first criminal traveler identification unit 18 determines the identity or similarity between the previously detected criminal traveler and the entrant at the entry hall through similarity analysis.

The storage unit 19 contains various information and data related to the operation of the system, as well as a plurality of first CCTV cameras (C1) installed at the immigration checkpoint, a plurality of second CCTV cameras (C2) installed in the arrival hall, and a plurality of second CCTV cameras (C2) arranged in the arrival hall. Each unique number, identification information, angle of view information, placement information, driving path information, mission information, location information, status information, and operation for the autonomous robot (R) and a plurality of readers (30) installed for each zone in the arrival hall. It serves to store information, object characteristic information and image information of a identified serious traveler and provide them under the control of the control unit 21.

And the storage unit 19 includes a plurality of first and second CCTV cameras (C1) (C2) and a specific first and second CCTV camera (C1) (C2) or a self-driving robot (R) among the self-driving robots (R). ), at least one of identification information, location information, angle of view information, status information, arrangement information, placed location information, and image information may exist as matched matching information, which may exist as matching information for specific first and second CCTV cameras (C1). ) (C2) or can be usefully used to specify the space (or place, area) where the autonomous robot (R) is located or to confirm the location of a frequent traveler located in that space.

In addition, the storage unit 19 stores a two-dimensional or three-dimensional map (map or map information) of the arrival hall to determine the current location of the self-driving robot (R) or a serious traveler, or to determine the current location of the self-driving robot (R) or a traveler who is a criminal. You can use it for setting and driving routes.

Here, the 2D or 3D map of the arrival hall consists of data that estimates the location based on the image information to determine the current location of the autonomous robot (R) based on the image information received from the autonomous robot (R). It can be.

Here, the storage unit 19 can be implemented with various storage media such as ROM, RAM, hard disk, flash memory, and solid state disk (SSD).

Meanwhile, the self-driving robot (R) drives by driving multiple wheels, receives information about the scheduled mission performance time through various paths, and can be programmed to perform the mission at that time, and the control unit 21 Through control, various operations and functions such as patrolling, guidance, cleaning, quarantine, and transportation can be performed within the arrival hall.

And the self-driving robot (R) can transmit and receive signals including situation information within the arrival hall with the system control unit 21.

In addition, the self-driving robot (R) can monitor the situation at the arrival hall by receiving and synthesizing video information captured in each area from the plurality of second CCTV cameras (C2) in the arrival hall.

In addition, the self-driving robot (R) can receive commands directly from the manager. For example, commands can be received directly from the manager through input such as touching the screen or voice input, and actions such as patrolling, guiding, and cleaning can be performed according to the received command.

And the self-driving robot (R) includes sensors such as a high-resolution camera, a global navigation system (GPS) that transmits and receives GPS signals for location tracking and location inquiry, a LiDAR sensor for autonomous driving, and a vision sensor. Since it is equipped with a camera, it is possible to recognize the operating environment and make task decisions through images captured by the camera, and can also transmit and receive RF (Radio Frequency) signals.

In addition, the self-driving robot (R) may include a control module to control operations, obtain status information using sensor information obtained from various types of sensors, or detect (recognize) the surrounding environment and objects. Alternatively, it may generate map data, determine a travel route and driving plan, determine a response to an interaction, or determine an action.

Here, the autonomous robot (R) may use sensor information obtained from at least one sensor among lidar, radar, and camera to determine the movement path and driving plan.

In particular, the self-driving robot (R) can recognize the environment or objects in areas where the view is obscured or over a certain distance by receiving sensor information from external devices, or receive recognized information directly from external devices. .

Additionally, the self-driving robot (R) can recognize the surrounding environment and objects using a learning model composed of at least one artificial neural network, and can determine an operation using the recognized surrounding environment information or object information.

In addition, the self-driving robot (R) determines the movement path and driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and controls the driving unit to determine the movement path. You can drive according to your driving plan.

Here, the map data may include object identification information about various objects placed in the space where the autonomous robot (R) moves. For example, map data may include object identification information for fixed objects such as walls and doors and movable objects such as flower pots, chairs, and people. The object identification information includes name, type, distance, location, etc. may be included.

Additionally, the self-driving robot (R) can perform operations or drive by controlling the driving unit based on the manager's control/interaction.

For example, the self-driving robot (R) receives mission information, drives to move to a specific place, receives location information of a high-traffic traveler from the control unit 21, and matches the high-crime traveler to a specific location based on the location information. The distance between them can be calculated.

The transmission unit 20 serves to transmit the object characteristic information and image information of the identified serious traveler to the customs situation room, customs agent (rover), and inspection officer (marshal).

The control unit 21 remotely controls the functions of the self-driving robots (R) and controls the overall operation of the system.

And the control unit 21 image-processes the images transmitted from the plurality of first CCTV cameras (C1) installed at the immigration checkpoint and the plurality of second CCTV cameras (C2) installed in the arrival hall and selectively displays them on the display unit (28). Control to display.

Additionally, the control unit 21 receives image information from the self-driving robots (R) and uses the received image information to confirm the location of the re-specified traveler.

At this time, the control unit 21 uses the relative location information corresponding to the distance between the re-specified high-crime traveler and a specific place such as a duty-free shop, restroom, store, quarantine facility, sculpture, passage, etc. in the arrival hall, and the relative location information is used as a standard. If it is out of range, select another autonomous robot (R) closest to the self-driving robot (R) using GPS signals, etc., and then use it to go to the duty-free shop, restroom, or It controls movement to a specific place (or destination) such as a store, quarantine facility, sculpture, passage, etc., and re-confirms the location of the re-specified high-crime traveler using video information received from the other autonomous robot (R). Flexibility and operational efficiency can be maximized by efficiently and quickly responding to field conditions.

For example, the control unit 21 determines that if the respecified serious traveler falls more than 5M from the predetermined sculpture A, the control unit 21 is outside the assigned service area of the self-driving robot (R), and Move another self-driving robot (R) that has been allocated (allocated) to the current location of the offender A or a service area including sculpture A in advance or is currently located closest to sculpture A toward the respecified offender traveler or to sculpture A. You can control it to do so.

Here, the relative location information refers to a specific place such as a duty-free shop, restroom, store, quarantine facility, sculpture, passage, etc. in the arrival hall, assuming that the re-specified high-crime traveler is on the same floor. It can mean information about how far away is.

In addition, the control unit 21 can receive the main CCTV camera setting among the plurality of first and second CCTV cameras (C1) (C2) from the manager, and can also change the main CCTV camera according to the manager's selection along the way.

In addition, the control unit 21 uses a plurality of CCTV cameras (C1) (C2) capable of operating PTZF (Pan, Tilt, Zoom, Focal length) to capture close-range images (zoom images/high-resolution images) of objects that have stopped moving. Each can be controlled.

And the control unit 21 can extract and specify identification information of the self-driving robot (R) based on the identification mark of the image information received from the self-driving robot (R) and determine its current location. In addition, the control unit 21 uses at least one of the time when the image was captured and the location information matched to the autonomous robot (R) that captured the image, to specify the current location of the autonomous robot (R) that captured the image. can do.

In addition, the control unit 21 can specify coordinate information, such as the 3rd floor, area A (1), for the image acquired from the autonomous robot (R), and the specified coordinate information is the current location of the autonomous robot (R). It can be information.

At this time, the control unit 21 determines the location of the self-driving robot (R) by comparing the image information acquired from the self-driving robot (R) with the map generated by vision (or visual)-based SLAM (Simultaneous Localization and Mapping) technology. (or current location) can be estimated. In this case, the control unit 21 uses image comparison between the image acquired from the self-driving robot R and the images constituting the already created map to specify the image most similar to the image obtained from the self-driving robot R. , the location information of the self-driving robot (R) can be specified by obtaining location information matched to the specified image.

That is, when an image is acquired from the self-driving robot (R), the control unit 21 can specify the location (or current location) of the self-driving robot using the acquired image, and the map already stored in the storage unit 19 From the information, location information (eg, coordinate information) corresponding to the image acquired from the autonomous robot (R) can be extracted.

In addition, the control unit 21 can extract the location information matched with the identification information of the second CCTV camera (C2) or the self-driving robot (R) that captured the video from the storage unit 19 and specify it as the location of the serious traveler.

Meanwhile, the control unit 21 can receive commands related to the operation of the tracking system through the administrator input unit 30, which is connected to an input device such as a mouse, keyboard, touchpad, etc., and inputs or outputs signals, data, information, etc. can be processed and provided.

The data communication unit 22 serves to communicate data wirelessly with each autonomous robot (R).

For example, the data communication unit 22 can receive data such as image information and sensing information from the self-driving robot (R) in real time and transmit control commands to the self-driving robot (R).

Here, the data communication department 22 includes WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), and WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), infrared communication It is possible to communicate with an autonomous robot (R) using a communication method that supports at least one of (Infrared Data Association; IrDA), UWB (Ultra-Wideband), and Wireless USB (Wireless Universal Serial Bus) technologies.

The third object characteristic information extraction unit 23 extracts the facial features, color and texture features, and gait features of entrants from the image information received through the data communication unit 22 and captured by the cameras of the autonomous robots (R). It plays a role in generating and storing object characteristic information in real time.

The second similarity detection unit 24 determines the identity between the object characteristic information of the serious traveler identified in the first criminal traveler specification unit 18 and the object characteristic information of entrants in the entry hall generated by the third object characteristic information extraction unit 23. It plays a role in detecting similarity.

The second criminal traveler identification unit 25 recognizes entrants whose similarity value detected by the second similarity detection unit 24 is higher than a predetermined threshold and re-identifies them as illegal travelers.

In other words, the second criminal traveler identification unit 25 determines the identity or similarity between the illegal traveler detected in advance and the person entering the port of entry through similarity analysis and re-identifies the illegal traveler as the illegal traveler.

The location recognition unit 26 serves to recognize each location in real time based on image information and GPS received from the autonomous robots (R).

The GIS display unit 27 serves to receive real-time location information such as GPS coordinates of self-driving robots (R) and display it on the display unit 28 along with guidance maps for the immigration checkpoint and arrival hall.

That is, the GIS display unit 27 receives signals transmitted from the autonomous robot R, checks the location information, and displays it on the display unit 28 as a GPS electronic map.

The display unit 28 has a function and role of displaying various information and data related to the operation of the system on the screen under the control of the control unit 21.

That is, the display unit 28 can display image information captured and transmitted from a plurality of first and second CCTV cameras C1 and C2 and a plurality of self-driving robots R on a monitor screen.

For example, execution screen information of an application running on the system, or UI (User Interface) and GUI (Graphic User Interface) information according to execution screen information, may be displayed.

Here, the display unit 28 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display ( It may include at least one of a flexible display, a 3D display, and an e-ink display.

The event alarm unit 29 serves to output video information including a serious traveler captured by the cameras of the self-driving robots (R) to the display unit 28 in real time.

For example, the event alarm unit 29 generates an alarm for a criminal traveler object displayed on the display unit 28, and provides the current location coordinates of the criminal traveler and the unique number of the autonomous robot (R) that photographed the criminal traveler. Displays .

A plurality of readers (30) are installed in each area of the arrival hall to receive the received signal strength value (RSSI) and unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form held by an entrant located within a certain distance. .

And the reader 30 detects an electronic tag (T) located within the short-distance wireless communication area and range, obtains the received signal strength value and unique identification information, and based on the received signal strength value according to signal loss and path loss. Location information is generated by calculating the absolute distance between the reader 30 and the corresponding electronic tag (T).

In other words, the reader 30 can calculate the distance to an entrant holding a traveler's belongings declaration form with the corresponding electronic tag (T) attached according to the received signal strength value input from the electronic tag (T) and convert it into location information. .

For example, assuming that the standard received signal strength value received by the reader 30 within a distance of 1 m is '1' and that the received signal strength value is attenuated by 1% per 1 m, the received signal strength at the reader 30 is When the value is input, if the intensity value is measured to be 0.95, it can be determined that the entrant is located at a distance of about 4 m from the reader 30 by calculating the distance change rate by the attenuation ratio of the received signal intensity value. You can.

Here, the reader 30 may be an electronic tag reader that automatically reads and stores the received signal strength value (RSSI) and unique identification number from the RFID or NFC tag of the electronic tag (T) attached to the traveler's carry-on declaration form.

The information combination unit 31 collects the received signal strength value, unique identification information, and distance information of the electronic tag (T) from the reader 30 and generates location information of the entrant.

In addition, the information combination unit 31 compares the location information of the offender traveler and the location information of the entrant based on the information related to the traveler specific to the offender and the location information of the self-driving robots (R) in the second offender identification unit 25, and determines the previously established information. If it is within the standard range, the unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form held by the inbound traveler is matched to the infringing traveler and the entrant is merged.

Meanwhile, an electronic tag (T) is attached to the traveler's belongings declaration form held by the entering person in order to provide unique identification information.

In addition, the electronic tag (T) is capable of radio frequency recognition, reading and writing, and has a memory chip that stores data such as unique identification information, and transmits the unique identification information stored in this memory chip to the reader 30 through short-distance wireless communication. It can be made up of a thinly packaged or assembled electronic device that includes a transceiver, a microcontroller unit with an interface for short-distance wireless communication such as RFID and NFC, and a thin film battery that supplies power required for operation. .

In addition, the transceiver may include an antenna coil unit that generates induced electromotive force by collecting energy transferred from the reader 30, and this antenna coil unit may include one or more condensers for frequency matching required for wireless power transmission and communication. .

Here, short-range wireless communication uses various methods such as RFID (radio frequency identification), NFC (Near Field Communication), Bluetooth, WiFi, Zig Bee, beacon, and LoRa (Long Range). It can be implemented, and in particular, RFID can include concepts such as low-frequency identification (LFID), high-frequency identification (HFID), and ultrahigh-frequency identification (UHFID).

Additionally, the surface of the electronic tag T may be covered with a protective layer made of a flexible material such as polyimide to protect against external physical shock or moisture.

The administrator input unit 32 serves to receive management and control commands from the administrator.

Here, there is no particular limitation on the type of the manager input unit 32, and it may include at least one of a mechanical input means and a touch-type input means.

For example, the administrator input unit 32 consists of a virtual key, soft key, or visual key displayed on the touch screen through software processing, or is placed in a part other than the touch screen. It can be made of a touch key, and when it is made of a touch screen, the display unit 28 can perform both the role of outputting information and the role of receiving information.

Meanwhile, with reference to FIGS. 1 and 2, a tracking method using a tracking system for frequent travelers in an entry hall using an electronic tag according to an embodiment of the present invention will be described as follows.

First, preliminary analysis information on entrants is compared with the traveler database of the traveler information system to search for and designate trespassers among entrants (S101).

Subsequently, facial recognition of entrants passing through the immigration checkpoint is performed in real time using video information captured by a plurality of first CCTV cameras (C1) installed without overlapping at the immigration checkpoint, and this is compared with the designated traveler information. Identify and detect erroneous travelers (S103).

At this time, when image information is received from a plurality of first CCTV cameras (C1) installed at the immigration counter, the image information can be parsed or analyzed to detect objects of interest within the image information.

Here, the object of interest refers to the goal or object of interest and may be a person.

In addition, various methods can be applied to extract objects from image information, such as image difference algorithms including methods such as binarization, labeling processing, and edge detection, optical flow algorithms, and model matching algorithms. Detection results can be expressed using a Minimum Enclosing Rectangle (MER).

In other words, the MER's width, height, center point coordinates, density information, and width/height ratio information are defined according to the characteristics of the object, and the defined MER and characteristics such as the width and height of the object included in the image information are compared to each other. A method is used to detect/classify objects that are of particular interest.

Continuing, the facial features, color and texture features, and gait features of detected criminal travelers are extracted to generate and store object feature information in real time (S105).

At this time, gait recognition extracts features by storing and analyzing the person's gait information obtained through a camera.

Here, the gait feature can be recognized by deleting the background from the acquired image, extracting the feature based on the outline information of the silhouette, and comparing it with the feature in the DB.

Alternatively, it can be recognized using a model-based method using height, length between head and pelvis, maximum length between pelvis and feet, and distance between both feet.

That is, a binary silhouette video sequence is obtained from a video of a walking person with a vision camera, a normalized silhouette image is obtained from the silhouette video sequence, and identifiable gait feature information is extracted from the normalized silhouette image. The object can be identified by comparing it with characteristic information stored in the computer.

Subsequently, the facial features, color and texture features, and gait characteristics of entrants located in the arrival hall are extracted using the image information obtained by filming by a plurality of second CCTV cameras (C2) installed without overlapping in the arrival hall, and object characteristic information is obtained. Create and save in real time (S107).

Next, the identity and similarity between the generated object characteristic information of the serious traveler and the object characteristic information of the entrants in the entry hall are detected (S109).

Here, the identity and similarity of facial features can be detected through feature point matching to find similar feature points by comparing the feature point descriptors extracted from the extracted facial feature images of criminal travelers and the facial feature images of entrants in the entry hall.

At this time, identity and similarity can be determined to be identical and similar when the object characteristic information matches more than a predetermined threshold, such as RGB information matches 70% or more as a result of mutual comparison (S111).

In addition, the identity and similarity of color features are determined by creating a color histogram from the extracted color feature images of criminal travelers and the color feature images of entrants in the entry hall, dividing and extracting main image features, comparing them, and then setting a preset threshold. Detection is possible only in the above cases.

In addition, the identity and similarity of texture features are determined by performing image segmentation on the extracted texture feature images of criminal travelers and texture feature images of entrants in the arrival hall, then constructing the MBR (Minimum Boundary Rectangle) of the object, within each image. Texture features can be generated and detected by applying wavelet transform to the MBR of the included object.

In addition, the identity and similarity of gait characteristics are obtained by obtaining a binary silhouette video sequence from image information obtained by shooting by a plurality of CCTV cameras (C1) installed at the immigration counter, and then obtaining a normalized silhouette image from the silhouette video sequence. After extracting identifiable gait feature information from the normalized silhouette image, the extracted gait feature information is compared with the gait feature information of the image information obtained by filming by a plurality of second CCTV cameras (C2) installed in the arrival hall. It can be detected.

Subsequently, an entrant whose similarity in at least one of the object characteristic information of the entrant within the entry hall matches more than a predetermined threshold is recognized and identified as a high-crime traveler located within the entry hall (S113).

In other words, through mutual comparison and analysis between the facial features, color and texture features, and gait characteristics of the criminal travelers and the entrants in the arrival hall, we analyze the degree of similarity between the previously detected criminal travelers and the entrants in the arrival hall to be compared. Determine the degree of similarity.

For example, if a pre-detected high-crime traveler disguises himself or changes his clothes by wearing heavy makeup, glasses, or a mask within the arrival hall, his gait characteristics are compared to the pre-detected high-crime traveler among the travelers in the arrival hall, a predetermined threshold of 70%. People similar to the above can be judged to be bad travelers.

Afterwards, the object characteristic information and video information of the identified criminal traveler are additionally stored, and the video information is captured and transmitted in real time to the customs situation room, customs agent (rover), and inspection officer (marshal) through a wireless communication network (S115) ).

Next, autonomous robots (R) in the arrival hall extract the facial features, color and texture features, and gait features of entrants from the image information obtained by shooting with cameras, and generate and store object feature information in real time (S117).

Then, the identity and similarity between the object characteristic information of the entrants in the created arrival hall and the object characteristic information of the already identified high-crime traveler are detected (S119).

Here, the identity and similarity of facial features can be detected through feature point matching to find similar feature points by comparing the feature point descriptors extracted from the extracted facial feature images of criminal travelers and the facial feature images of entrants in the entry hall.

At this time, the identity and similarity can be determined to be identical and similar when the object feature information matches more than a predetermined threshold, such as RGB information matches more than 70% as a result of mutual comparison (S121).

In addition, the identity and similarity of color features are determined by creating a color histogram from the extracted color feature images of criminal travelers and the color feature images of entrants in the entry hall, dividing and extracting main image features, comparing them, and then setting a preset threshold. Detection is possible only in the above cases.

In addition, the identity and similarity of texture features are determined by performing image segmentation on the extracted texture feature images of criminal travelers and texture feature images of entrants in the arrival hall, then constructing the MBR (Minimum Boundary Rectangle) of the object, within each image. Texture features can be generated and detected by applying wavelet transform to the MBR of the included object.

In addition, the identity and similarity of gait characteristics are obtained by obtaining a binary silhouette video sequence from image information obtained by shooting by a plurality of CCTV cameras (C1) installed at the immigration counter, and then obtaining a normalized silhouette image from the silhouette video sequence. After extracting identifiable gait feature information from the normalized silhouette image, the extracted gait feature information is detected by comparing it with the gait feature information of the image information obtained by shooting by a plurality of self-driving robots (R) placed in the arrival hall. can do.

Afterwards, the entrants whose similarity to at least one of the object characteristic information of the entrants in the entry hall is detected to be equal to or higher than a predetermined threshold are recognized and re-specified as serious travelers (S123).

In other words, through mutual comparison and analysis between the facial features, color and texture features, and gait characteristics of the already specified criminal travelers and the entrants in the entry hall, what degree of similarity is there between the previously detected criminal travelers and the entrants in the arrival hall to be compared? Analyze to determine the degree of similarity.

For example, if a pre-detected high-crime traveler disguises himself or changes his clothes by wearing heavy makeup, glasses, or a mask within the arrival hall, his gait characteristics are compared to the pre-detected high-crime traveler among the travelers in the arrival hall, a predetermined threshold of 70%. People similar to the above can be judged to be bad travelers.

In this process, the location of the re-specified high-crime traveler is confirmed using the image information received from the autonomous robots (R), and relative location information corresponding to the distance between the re-specified high-crime traveler and a specific place in the arrival hall is provided. In other words, based on the relative location information corresponding to the distance between a specific place such as a duty-free shop, restroom, store, quarantine facility, sculpture, passage, etc. in the arrival hall and a frequent traveler, if the relative location information is outside the standard range, the self-driving robot The direction of movement of the other autonomous robot (R) closest to (R) is determined by the current location of the re-specified traveler or a specific location such as a duty-free shop, restroom, store, quarantine facility, sculpture, passageway, etc. in the nearest arrival hall ( or destination), and the location of the re-specified traveler can be re-confirmed using image information received from the other autonomous robot (R).

For example, if an autonomous robot (R) that is tracking or checking the location of a re-identified criminal traveler within the arrival hall chases or closely follows him even outside the assigned area or predetermined route, the criminal traveler may not notice. Since it can run away or evade, it commands and controls to hand over to the other autonomous robot (R) closest to it, and at the same time, the other autonomous robot (R) checks the location of the re-specified traveler and moves to track it. Command and control is possible.

In other words, if the re-specified high-crime traveler falls more than 5 meters from the predetermined sculpture A, it is judged to be outside the assigned area, set route, or service area of the self-driving robot (R), and the re-specified high-crime traveler Control the current location or other autonomous robot (R) that has previously been assigned a service area containing sculpture A or is currently located closest to sculpture A to move toward the respecified high-crime traveler or toward sculpture A. By doing so, you can efficiently respond to variable on-site situations, such as continuously checking and flexibly tracking re-specified criminal travelers without raising any suspicion from those around you.

Next, each location is recognized in real time based on image information received from autonomous robots (R) and GPS.

Subsequently, the information combination unit 31 collects the received signal strength value and unique identification information of the electronic tag (T) from the reader 30 to generate location information of the entering person, and also collects the received signal strength value and unique identification information of the electronic tag (T) from the reader 30. The location information of the offender traveler and the location information of the entrant are compared based on the information related to the specific traveler and the location information of the self-driving robots (R) (S125).

At this time, if it is within the preset standard range, for example, if the offender and the relevant entrant are located within a 1m radius, the offender is identified by matching the unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form held by the offender to the offender. The traveler and the entrant are combined (S127).

For example, it is determined whether the distance difference between a pre-specified high-crime traveler and the electronic tag (T) attached to the traveler's personal belongings declaration form held by the entering person is within a preset standard distance, and if it is within the standard distance, the high-crime traveler receives the corresponding electronic tag (T). ) can be judged to be in possession of.

In other words, the information combination unit 31 can determine that the person entering the country with location information (electronic tag) within a preset distance from the location of the serious traveler is the same person.

During this process, if a serious traveler gathers in a group of several entrants and then becomes separated or disappears momentarily, the same traveler will be given the same identity based on the information matching the unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form. Entrants holding a traveler's belongings declaration form with an electronic tag (T) with unique identification information are continuously monitored as the same offender.

Next, the GIS display unit 27 displays the location information of serious travelers and entrants combined in the information combination unit 31 on the display unit 28 in real time (S129).

In addition, video information obtained by filming a criminal traveler in the re-identified entry hall is output in real time to the display unit 28.

At this time, the location of the criminal traveler can be confirmed in real time on the electronic map using the unique ID or identifier assigned to the self-driving robot (R) that provides video information containing the re-identified criminal traveler, and the display unit (28 ) can be displayed as an alarm pop-up.

Furthermore, the object characteristic information and video information of re-identified criminal travelers are additionally stored, and the video information is captured and transmitted in real time to the customs situation room, customs agent (rover), and inspection officer (marshal) through a wireless communication network and terminal. A message related to the object characteristic information of the re-specified criminal traveler can be transmitted.

Therefore, even if a serious traveler who has passed the immigration checkpoint at an airport or port disguises himself or changes his clothes by wearing heavy makeup, glasses, or a mask within the arrival hall to avoid CIQ customs procedures, he or she will be more efficiently responded to and tracked along the movement route, and will be thoroughly observed. and can be continuously monitored.

In other words, the location information of the offender traveler based on the information related to the specified traveler and the location information of the self-driving robots (R) is compared with the location information of the entrant collected by the reader 30 installed in each area within the arrival hall. If it is within the set standard range, it can be monitored and tracked by combining the unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form held by the inbound traveler.

In addition, the self-driving robot (R) collects and analyzes the object characteristic information of entrants using image information obtained by shooting with a camera while moving through the target area within the arrival hall, thereby collecting and analyzing object characteristic information of entrants, thereby removing blind spots within the arrival hall beyond the shooting range of the second CCTV camera (C2). Even if a criminal traveler is hidden or located in the zone, the location and movement of the criminal traveler can be confirmed in real time, improving the reliability of detection and tracking efficiency of the criminal traveler.

Meanwhile, the present invention can be implemented as a program that is executed by one or more processes on a computer and can be stored in a computer-readable medium (a recording device that stores data).

That is, the present invention can be implemented as a computer program and recorded on a computer-readable recording medium, and in addition, the program for implementing the operation by the method is recorded and the computer-readable recording medium can be read by the computer. It can include all types of recording devices where data is stored.

Meanwhile, the present invention is not limited to the above-described embodiments and the accompanying drawings, and may be modified and applied in various ways not exemplified without departing from the technical spirit of the present invention. It is clear to those skilled in the art that the present invention can be broadly applied by replacing components and changing it to other equivalent embodiments.

Therefore, contents related to modifying and applying the technical features of the present invention should be construed as being included within the technical idea and scope of the present invention.

11: Criminal traveler information input unit 12: First image information input unit
13: Criminal traveler detection unit 14: First feature extraction unit
15: second image information input unit 16: second feature extraction unit
17: First similarity detection unit 18: First criminal traveler specification unit
19: storage unit 20: transmission unit
21: control unit 22: data communication unit
23: Third feature extraction unit 24: Second similarity detection unit
25: Second criminal traveler specification department 26: Location recognition department
27: GIS display unit 28: display unit
29: Event alarm unit 30: Reader
31: Ministry of Information and Cooperation

Claims (3)

delete A traveler information input unit (11) that receives information on travelers with criminal offenses among entrants by comparing the preliminary analysis information on entrants with the traveler database of the traveler information system;
A first image information input unit (12) that receives image information from a plurality of first CCTV cameras (C1) installed at the immigration checkpoint;
A criminal traveler detection unit that identifies the object of an entrant from the image information input through the first image information input unit 12 and detects a criminal traveler by comparing it with the criminal traveler information input through the illegal traveler information input unit 11. (13);
A first object feature information extraction unit 14 that generates and stores object feature information in real time by extracting facial features, color and texture features, and gait features of detected criminal travelers;
A second image information input unit (15) that receives image information from a plurality of first CCTV cameras (C2) installed in the arrival hall;
A second object feature information extraction unit (16) that extracts the facial features, color and texture features, and gait features of entrants from the image information input through the second image information input unit (15) and generates and stores object feature information in real time. );
A system for detecting the identity and similarity between the object characteristic information of a serious traveler generated by the first object characteristic information extraction unit 14 and the object characteristic information of entrants in the entry hall generated by the second object characteristic information extraction unit 16. 1 similarity detection unit (17);
A first criminal traveler identification unit (18) that recognizes entrants whose similarity value detected by the first similarity detection unit (17) is greater than a predetermined threshold among the entrants in the arrival hall and identifies them as the offenders;
Identification information and arrangement of the plurality of first CCTV cameras (C1) installed at the immigration checkpoint, the plurality of second CCTV cameras (C2) installed in the arrival hall, and the plurality of self-driving robots (R) arranged in the arrival hall. A storage unit (19) that stores information, object characteristic information and image information of a specified criminal traveler;
A transmission unit 20 that transmits object characteristic information and image information of a identified serious traveler to the customs situation room, customs agent (rover), and inspection officer (marshal);
A control unit 21 that controls the functions of the autonomous robots (R) and the overall operation of the system;
A data communication unit 22 that performs wireless data communication with each of the self-driving robots R;
A third device extracts facial features, color and texture features, and gait features of entrants from the image information input through the data communication unit 22 and obtained from the self-driving robots (R), and generates and stores object feature information in real time. Object characteristic information extraction unit (23);
A second device for detecting the identity and similarity between the object characteristic information of a serious traveler specified in the first serious traveler specification unit 18 and the object characteristic information of entrants in the entry hall generated by the third object characteristic information extraction unit 23 Similarity detection unit 24;
a second criminal traveler specification unit (25) that recognizes entrants whose similarity value detected by the second similarity detection unit (24) is equal to or higher than a predetermined threshold and re-identifies them as criminal travelers;
A location recognition unit 26 that recognizes each location information in real time based on image information and GPS received from the autonomous robots (R);
A GIS display unit (27) that displays location information of the autonomous robots (R) on the display unit (28);
A plurality of readers (30) are installed for each zone in the arrival hall and receive the received signal strength value (RSSI) and unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form held by an entrant located within a certain distance; and
The received signal strength value and unique identification information of the electronic tag (T) are collected from the reader (30) to generate location information of the entrant, and this is combined with information related to a specific criminal traveler in the second criminal traveler specification unit (25). If the location information of the self-driving robots (R) is compared with the location information of the serious traveler and is within a preset standard range, the electronic tag (T) attached to the traveler's belongings declaration form possessed by the entering person is given to the serious traveler. an information combination unit 31 that matches unique identification information to combine the serious traveler and the entrant;
In the method of tracking criminal travelers at the port of entry using a tracking system for frequent travelers at the port of entry, including:
(a) the control unit 21 compares the preliminary analysis information on entrants with the traveler database of the traveler information system to search for and designate a high-crime traveler among entrants;
(B) The criminal traveler detection unit 13 uses image information obtained from the plurality of CCTV cameras (C1) installed at the immigration checkpoint to recognize the faces of entrants passing through the immigration checkpoint, and combines this with the designated criminal traveler information Identifying and detecting bad travelers by comparison;
(c) the first object feature information extraction unit 14 extracting the facial features, color and texture features, and gait features of the detected traveler to generate and store object feature information in real time;
(D) The second object feature information extraction unit 16 extracts the facial features, color and texture features, and gait features of entrants from the image information obtained from the plurality of CCTV cameras (C2) installed in the arrival hall, thereby providing object feature information. generating and storing in real time;
(e) the first similarity detection unit 17 detecting the identity and similarity between the generated object characteristic information of a serious traveler and the object characteristic information of entrants in the entry hall;
(F) a step in which the first criminal traveler identification unit 18 recognizes an entrant whose similarity in at least one of the object characteristic information of the entrant within the entry hall is detected to be higher than a predetermined threshold and specifies the entrant as a criminal traveler;
(G) The storage unit 19 additionally stores the object characteristic information and image information of the identified serious traveler, and the transmission unit 20 transmits the information to the customs situation room, customs agent (rover), and inspection officer (marshal). transmitting;
(H) The third object feature information extraction unit 23 extracts the facial features, color and texture features, and gait features of entrants from the image information obtained from the self-driving robots (R) in the arrival hall and provides object feature information in real time. creating and saving steps;
(I) detecting the identity and similarity between the object characteristic information of entrants in the arrival hall created by the second similarity detection unit 24 and the object characteristic information of already specified high-crime travelers;
(J) The second criminal traveler identification unit 25 recognizes the entrant whose similarity in at least one of the object characteristic information of the entrant in the entry hall is greater than a predetermined threshold and re-specifies the entrant as a criminal traveler, and the control unit 21 ) receives image information from the self-driving robots (R), uses the received image information to confirm the location of the re-specified high-crime traveler, and then distances the re-specified high-crime traveler from a specific place in the arrival hall. If the relative position information is outside the reference range based on the corresponding relative position information, control the movement direction of the autonomous robot (R) and the other autonomous robot (R) closest to it to be changed to the specific location, , re-confirming the location of the re-specified traveler using image information received from the other autonomous robot (R);
(k) the location recognition unit 26 recognizing each location information in real time based on image information and GPS received from the autonomous robots (R);
(L) The information combination unit 31 collects the received signal strength value and unique identification information of the electronic tag (T) from the reader 30 to generate location information of the entering person, and the second serious traveler identification unit In (25), the location information of the offender based on information related to a specific traveler and the location information of the self-driving robots (R) is compared with the location information of the entrant, and if it is within a preset standard range, the offender is sent to the traveler. Matching the unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form held by the entering person;
(M) the GIS display unit (27) displaying the location information of the serious traveler and the entrant matched by the information combination unit (31) through the display unit (28);
It is accomplished including,
In the step (other), if the criminal traveler gathers in a group of several entrants and then becomes separated or disappears momentarily, the unique identification information of the electronic tag (T) attached to the traveler's belongings declaration form carried by the entrant is provided to the criminal traveler. A method of tracking criminal travelers at the arrival port, characterized in that, based on the matched information, entrants holding a traveler's belongings declaration form with the same unique identification information attached as the electronic tag (T) are merged into the same criminal traveler and monitored. delete

Images