KR102256271B1 - object tracking and service execution system by setting event zones and execution rule on the camera screen - Google Patents

Abstract

The present invention relates to an object tracking and service execution system through setting an area on a camera screen and an event rule. An object tracking service method according to an aspect of the present invention includes: a first step of setting at least one area, Vid-Fence, in an image acquired by a first camera; A second step of setting a service corresponding to each of the at least one Vid-Fence; A third step of recognizing an entry of an object based on the image acquired by the first camera; A fourth step of obtaining an identifier (ID) of the object based on the image acquired by the second camera; A fifth step of matching the entered object with an identifier (ID) of the object; A sixth step of tracking an object matching the identifier (ID); A seventh step in which an object matching the identifier (ID) is located in a first Vid-Fence among the at least one Vid-Fence; And an eighth step of providing a first service set in the first Vid-Fence to an object matching the identifier (ID).

Description

Object tracking and service execution system by setting event zones and execution rule on the camera screen}

The present invention relates to an object tracking and service execution system through setting an area on a camera screen and an event rule.

Surveillance systems that detect certain actions or events are recognized as the most important among research fields using human tracking.

The reason is that as society develops, the importance of the safety of individuals and facilities is highly recognized not only in public places but also in private spaces.

In particular, as the nature of informationization, unmannedization, automation, and computerization increases in modern society, warnings about the safety of oneself and the workplace as well as the safety and security of property are constantly being raised.

Therefore, efforts to protect and manage the property and safety of themselves and their workplaces are continuing, and the importance and scope of security is expanding from major facilities and public facilities, to schools, businesses, and homes. There is a need for development.

Surveillance systems include video surveillance systems using CCTV cameras or PC cameras, network surveillance systems that can be easily monitored by mobile phones, PDAs, PCs, etc. by quickly transmitting images to a computer in real time using a network camera, and infrared rays suitable for night surveillance. It can be divided into detection systems, etc.

Among them, video surveillance systems using CCTV or PC cameras, unlike other surveillance systems as countermeasures after intrusion or vandalism, can actively identify intruders, detect intrusions or illegal acts in advance, and record possible. It is the most effective monitoring means to provide and store important information.

In this way, a video surveillance system using a CCTV or a PC camera can identify an intruder through object tracking or track the direction, path or movement of an object, such as a person's movement.

A general object tracking method using CCTV tracks an object within a single CCTV area or, when moving to another CCTV, in consideration of the location information such as the visual characteristics or movement direction of the object.

For example, when a user moves from a certain CCTV area to another CCTV area, all CCTVs transmit information about the user's appearance, such as height, head, clothes, movement direction, and speed, to another CCTV, Track continuously.

Republic of Korea Application No. 10-2014-0183200 Korean Patent Registration No. 10-0756318 Korean Patent Registration No. 10-0638829 Republic of Korea Application No. 10-2015-0050779

The present invention aims to provide a user with an object tracking and service execution system through setting an area on a camera screen and an event rule.

Specifically, in the present invention, when the camera image is set as a Vid-Fence by drawing a region of various shapes using a tool on a monitor screen, and setting and executing an event rule and service for each Vid-Fence, a corresponding event will be detected through the camera. At this time, we want to provide a system that runs the service to users.

In addition, the present invention is a system for setting the type of recognizing an object by dividing it into an overall shape or a floor contact surface because the criteria for determining an object entering, staying, or leaving the Vid-Fence are different depending on the camera installation position and the shooting angle. I want to provide to users.

In addition, the present invention creates and determines a criterion by combining one or more overlapping degree, overlapping time, distance, and moving angle in order to increase the recognition rate for entry, stay, and departure. We want to provide a service execution system to users.

In addition, the present invention is to provide a service and system capable of ordering automatic refueling by applying a method of tracking an object and executing a service through setting an area on a camera screen and an event rule to identify a gas station at a gas station.

In addition, the present invention applies an object tracking and service execution method through the setting of an area on the camera screen and an event rule, so that when a control area and a general area on the camera screen exist together, a security notification only when an unauthorized person enters the control area. It is intended to provide a user with a selective security monitoring service and system for an image area that outputs an image.

In addition, the present invention applies an object tracking and service execution method by setting an area on the camera screen and an event rule, providing a work/operation manual according to the movement and location of workers in a production plant, push notification, safety rules, etc. We intend to provide users with Smart Factory services and systems that provide information.

On the other hand, the technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned are clearly to those of ordinary skill in the technical field to which the present invention belongs from the following description. It will be understandable.

An object tracking service method according to an aspect of the present invention for achieving the above technical problem comprises: a first step of setting at least one area, Vid-Fence, in an image acquired by a first camera; A second step of setting an event rule and service corresponding to each of the at least one Vid-Fence; A third step of recognizing an entry of an object based on the image acquired by the first camera; A fourth step of obtaining an identifier (ID) of the object based on an image acquired by a second camera or ID identification data collected from RFID, NFC, BLE beacon, etc.; A fifth step of matching the entered object with an identifier (ID) of the object; A sixth step of tracking an object matching the identifier (ID); A seventh step of checking an event rule where an object matching the identifier (ID) is located in a first Vid-Fence among the at least one Vid-Fence; And an eighth step of providing a first service set in the first Vid-Fence to an object matching the identifier (ID).

In addition, in the second step, a service execution rule (hereinafter referred to as “event rule”) for providing the service may be set together.

In addition, the event rule includes a degree of overlapping and overlapping time of the object with the at least one Vid-Fence, a distance distance from the center of the at least one Vid-Fence, and the object based on the center of the object. It may be set by combining at least one of a moving direction and a moving angle degree between the center of the at least one Vid-Fence.

In addition, in the seventh step, based on the set event rule, it may be determined whether an object matching the identifier (ID) is located within the first Vid-Fence.

In addition, in the seventh step, based on the set event rule, it is determined whether an object matching the identifier (ID) is in an entry state, a stay state, and an exit state with respect to the first Vid-Fence, The eighth step may be performed only when it is determined as the staying state.

In addition, the seventh step may include: a 7-1 step of extracting a bottom surface corresponding to the size of the object matched with the identifier (ID); And based on the set event rule, by comparing the floor surface and the first Vid-Fence, the object matching the identifier (ID) is in the entry state, stay state, and exit state with respect to the first Vid-Fence. It may include a; 7-2 step of determining whether or not.

In addition, the first step may include a step 1-1 of displaying a drawing tool in the image acquired by the first camera; And a 1-2 step of designating a certain point in the image acquired by the first camera, and setting the at least one Vid-Fence of a certain shape at the designated point based on the drawing tool. have.

In addition, in the fourth step, the identifier (ID) of the object may be obtained by additionally or alternatively using at least one from ID card tagging, fingerprint recognition and iris recognition, RFID, NFC, BLE beacon, and the like.

In addition, the at least one Vid-Fence is an area corresponding to a lubricator in a gas station, the object is at least one vehicle, and in the seventh step, when the object is located within the first Vid-Fence, the first The first service provided in step 8 may be an automatic refueling order request service.

In addition, the at least one Vid-Fence is an area corresponding to a control area, the object is a person who has not been previously authorized, and in the seventh step, when the object is located within the first Vid-Fence, the first The first service provided in step 8 may be a service that provides security risk information.

In addition, the at least one Vid-Fence is an area corresponding to a working section, and in the seventh step, when the object is located within the first Vid-Fence, the first service provided in the eighth step, It may include a service for providing a work manual, a service for providing an operation manual, a service for providing safety rules, and a service for providing production status.

Meanwhile, an object tracking service system according to another aspect of the present invention for achieving the above technical problem includes: a first camera for acquiring an image; An input unit for setting a Vid-Fence, which is at least one area in the image acquired by the first camera, and a service corresponding to each of the at least one Vid-Fence; A controller for recognizing an entry of an object based on the image acquired by the first camera; And a second camera that acquires an image related to the object, wherein the controller obtains an identifier (ID) of the object based on the image acquired by the second camera, and Matching the identifier (ID) of the object, tracking the object matching the identifier (ID), and the object matching the identifier (ID) is located in the first Vid-Fence of the at least one Vid-Fence In this case, the control unit may control the first service set in the first Vid-Fence to be provided to the object matching the identifier (ID).

The present invention can provide an object tracking and service execution system to a user through setting an area on a camera screen and an event rule.

Specifically, in the present invention, when the camera image is set as a Vid-Fence by drawing a region of various shapes using a tool on the monitor screen, an event is set and executed for each Vid-Fence, the service is executed when a corresponding event is detected through the camera. It is possible to provide a system to users.

In addition, the present invention is a system for setting the type of recognizing an object by dividing it into an overall shape or a floor contact surface because the criteria for determining an object entering, staying, or leaving the Vid-Fence are different depending on the camera installation position and the shooting angle. Can be provided to the user.

In addition, the present invention creates and determines a criterion by combining one or more overlapping degree, overlapping time, distance, and moving angle in order to increase the recognition rate for entry, stay, and departure. A service execution system can be provided to users.

In addition, the present invention applies an object tracking and service execution method through setting an area on a camera screen and an event rule, thereby identifying a gas station at a gas station and providing a service and a system capable of ordering automatic refueling to a user.

In addition, the present invention applies an object tracking and service execution method through the setting of an area on the camera screen and an event rule, so that when a control area and a general area on the camera screen exist together, a security notification only when an unauthorized person enters the control area. It is possible to provide a selective security monitoring service and system that outputs to a user.

In addition, the present invention applies an object tracking and service execution method by setting an area on the camera screen and an event rule, providing a work/operation manual according to the movement and location of workers in a production plant, push notification, safety rules, etc. Smart Factory services and systems that provide information can be provided to users.

On the other hand, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description. I will be able to.

1 is a block diagram of an object tracking and service execution system through setting an area on a camera screen and an event rule proposed by the present invention.
2 shows an example of setting a Vid-Fence area, setting an event (service) in the Vid-Fence area, and executing a Vid-Fence service according to the present invention.
3 is a flowchart illustrating an object tracking and service execution method through setting an area on a camera screen and an event rule proposed by the present invention.
4 is a diagram illustrating an object recognition standard and method according to a camera photographing angle applied to the present invention.
5 is a diagram illustrating a standard and method for recognizing entry, stay, and exit according to the present invention.
FIG. 6 is a flow chart illustrating a service for ordering automatic refueling by identifying a gas tanker stopped at a gas station as an example of utilizing Vid-Fencing proposed by the present invention.
FIG. 7 shows an example of a service for ordering automatic refueling by identifying a gas tanker that has stopped at the gas station described in FIG. 6.
8 is a diagram illustrating a selective security monitoring method as an example of utilization of Vid-Fencing proposed by the present invention.
9 is an example of utilizing Vid-Fencing proposed by the present invention, showing a Smart Factory that provides information necessary for work, such as providing work/operation manuals, push notifications, and safety rules according to the movement and location of workers in a production factory. will be.

Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. In addition, the embodiments described below do not unreasonably limit the content of the present invention described in the claims, and the entire configuration described in the present embodiment cannot be said to be essential as a solution to the present invention.

Object tracking and service execution system through area on camera screen and event rule setting

1 is a block diagram of an object tracking and service execution system through setting an area on a camera screen and an event rule proposed by the present invention.

Referring to FIG. 1, the object tracking system 100 according to the present invention includes a wireless communication unit 110, an audio/video (A/V) input unit 120, a user input unit 130, a sensing unit 140, and an output unit. 150, a memory 160, an interface unit 170, a control unit 180, a power supply unit 190, and the like.

However, since the components shown in FIG. 1 are not essential, an object tracking system having more components or fewer components may be implemented.

Hereinafter, the above components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the object tracking system and the wireless communication system or between the object tracking system and a network in which the object tracking system is located. For example, the wireless communication unit 110 may include a broadcast reception module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. .

The broadcast receiving module 111 receives a broadcast signal and/or broadcast-related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and/or broadcast-related information, or a server that receives a previously-generated broadcast signal and/or broadcast-related information and transmits it because it is an object tracking system. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a broadcast signal in a form in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 includes, for example, Digital Multimedia Broadcasting-Terrestrial (DMB-T), Digital Multimedia Broadcasting-Satellite (DMB-S), Media Forward Link Only (MediaFLO), and Digital Video Broadcasting (DVB-H). -Handheld), ISDB-T (Integrated Services Digital Broadcast-Terrestrial), and other digital broadcasting systems can be used to receive digital broadcasting signals. Of course, the broadcast receiving module 111 may be configured to be suitable for not only the digital broadcasting system described above, but also other broadcasting systems.

Broadcast signals and/or broadcast related information received through the broadcast reception module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives a radio signal with at least one of a base station, an external object tracking system, and a server on a mobile communication network.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be embedded or external to the object tracking system. As a wireless Internet technology, WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), and the like may be used.

The short-range communication module 114 refers to a module for short-range communication. As short range communication technology, Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra-Wideband (UWB), ZigBee, and the like may be used.

The location information module 115 is a module for obtaining a location of an object tracking system, and a representative example thereof is a Global Position System (GPS) module.

Referring to FIG. 1, an A/V (Audio/Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122.

The camera 121 processes an image frame such as a still image or a moving picture obtained by an image sensor in the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110.

The camera 121 is used in plural according to the use environment.

Meanwhile, as the camera 121, a recognition camera 195 (not shown) for recognizing a vehicle number may be additionally used.

The microphone 122 receives an external sound signal by a microphone in a recording mode, a voice recognition mode, and the like and processes it as electrical voice data. The processed voice data may be converted into a format transmittable to a mobile communication base station through the mobile communication module 112 and then output. Various noise removal algorithms may be implemented in the microphone 122 to remove noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for controlling the operation of the object tracking system by the user. The user input unit 130 may be composed of a key pad, a dome switch, a touch pad (positive pressure/electrostatic), a jog wheel, a jog switch, and the like.

The sensing unit 140 detects the current state of the object tracking system, such as the open/closed state of the object tracking system, the location of the object tracking system, the presence of user contact, the orientation of the object tracking system, and acceleration/deceleration of the object tracking system. It generates a sensing signal to control the operation of. In addition, whether the power supply unit 190 supplies power or whether the interface unit 170 is coupled to an external device may be sensed. Meanwhile, the sensing unit 140 may include a proximity sensor 141.

The output unit 150 is for generating output related to visual, auditory or tactile sense, and includes a display unit 151, an audio output module 152, an alarm unit 153, a haptic module 154, and a projector module ( 155).

The display unit 151 displays (outputs) information processed by the object tracking system.

The display unit 151 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. display) and a 3D display.

Some of these displays may be configured as a transparent type or a light-transmitting type so that the outside can be seen through it. This may be referred to as a transparent display, and a representative example of the transparent display is TOLED (Transparant OLED). The rear structure of the display unit 151 may also be configured as a light-transmitting structure. With this structure, the user can see the object located behind the body of the object tracking system through an area occupied by the display unit 151 of the body as the object tracking system.

Two or more display units 151 may exist depending on the implementation form of the object tracking system.

When the display unit 151 and a sensor for detecting a touch motion (hereinafter, referred to as a'touch sensor') form a mutual layer structure (hereinafter, referred to as a'touch screen'), the display unit 151 is used in addition to the output device. It can also be used as an input device. The touch sensor may have a shape such as, for example, a touch film, a touch sheet, and a touch pad.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 151 or a capacitance generated at a specific portion of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only a touched position and area, but also a pressure at the time of touch.

When there is a touch input to the touch sensor, a signal(s) corresponding thereto is transmitted to the touch controller. The touch controller processes the signal(s) and then transmits the corresponding data to the controller 180. Accordingly, the controller 180 can know which area of the display unit 151 has been touched.

The proximity sensor 141 may be disposed in an inner area of the object tracking system wrapped by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity using the force of an electromagnetic field or infrared rays without mechanical contact. Proximity sensors have a longer lifespan and higher utilization than contact sensors.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, an infrared proximity sensor, and the like. When the touch screen is capacitive, it is configured to detect the proximity of the pointer by a change in an electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a recording mode, a voice recognition mode, a broadcast reception mode, or the like.

The sound output module 152 also outputs sound signals related to functions performed in the object tracking system. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event in the object tracking system. Examples of events occurring in the object tracking system include message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than a video signal or an audio signal, for example, by vibration. The video signal or audio signal may also be output through the display unit 151 or the audio output module 152, so that they 151 and 152 may be classified as part of the alarm unit 153.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or may be sequentially output.

In addition to vibration, the haptic module 154 is designed to respond to stimuli such as an arrangement of pins vertically moving with respect to the contact skin surface, blowing force or suction force of air through the injection or inlet, grazing against the skin surface, contact of an electrode, and electrostatic force. It can generate various tactile effects, such as an effect by the effect and an effect by reproducing the feeling of cooling and warming using an endothermic or exothermic element.

The haptic module 154 may not only deliver a tactile effect through direct contact, but may also be implemented so that a user can feel the tactile effect through a muscle sensation such as a finger or an arm. Two or more haptic modules 154 may be provided depending on the configuration aspect of the portable object tracking system.

The projector module 155 is a component for performing an image project function using an object tracking system, and is the same as or at least as an image displayed on the display unit 151 according to a control signal from the controller 180. Some may display different images on an external screen or wall.

Specifically, the projector module 155 generates an image to be output to the outside by using a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, and light generated by the light source. It may include an image generating means (not shown) for performing and a lens (not shown) for expanding the image to the outside at a predetermined focal length. In addition, the projector module 155 may include a device (not shown) capable of adjusting an image projection direction by mechanically moving a lens or the entire module.

The projector module 155 may be divided into a cathode ray tube (CRT) module, a liquid crystal display (LCD) module, a digital light processing (DLP) module, and the like according to the device type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by magnifying and projecting an image generated by reflecting light generated from a light source onto a digital micromirror device (DMD) chip.

The memory unit 160 may store a program for processing and control of the controller 180, and a function for temporary storage of input/output data (eg, messages, audio, still images, moving pictures, etc.) You can also do The memory unit 160 may also store a frequency of use of each of the data. In addition, the memory unit 160 may store data related to vibrations and sounds of various patterns that are output when a touch input on the touch screen is input.

The memory 160 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), and RAM. (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic It may include at least one type of storage medium among a disk and an optical disk.

The interface unit 170 serves as a path to all external devices connected to the object tracking system. The interface unit 170 receives data from an external device or receives power and transmits it to each component inside the object tracking system, or transmits data inside the object tracking system to an external device. For example, a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port to connect a device equipped with an identification module, an audio input/output (I/O) port, A video input/output (I/O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip that stores various types of information for authenticating the right to use the object tracking system, and includes a user identification module (UIM), a subscriber identification module (SIM), and a universal user authentication module (Universal Subscriber). Identity Module, USIM), etc. may be included. A device equipped with an identification module (hereinafter referred to as'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device can be connected to the object tracking system through the port.

When the interface unit is a moving object tracking system connected to an external cradle, power from the cradle is supplied to the moving object tracking system, or various command signals input from the cradle by a user are transmitted to the moving object. It can be a path to the tracking system. Various command signals or the power input from the cradle may be operated as signals for recognizing that the moving object tracking system is correctly mounted on the cradle.

The controller 180 generally controls the overall operation of the object tracking system.

The controller 180 may also include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180.

The controller 180 draws various shaped areas on the monitor screen using a tool to set the camera image as a Vid-Fence, and when an event is set and executed for each Vid-Fence, the service is executed when a corresponding event is detected through the camera. I can.

In addition, since the criteria for determining whether an object enters, stays, or leaves the Vid-Fence varies according to the camera installation position and the shooting angle, the controller 180 sets the type to recognize the object by dividing it into an overall shape or a floor contact surface. I can.

In addition, the controller 180 may create and determine a criterion by combining one or more of an overlapping degree, an overlapping time, a distance, and a movement angle in order to increase the recognition rate for entry, stay, and departure.

In addition, the controller 180 may apply a service that enables an automatic refueling order by identifying a gas station stopped at a gas station by applying an object tracking and service execution method through setting an area on the camera screen and an event rule.

In addition, the controller 180 applies an object tracking and service execution method through the setting of an area on the camera screen and an event rule, so that when a control area and a general area on the camera screen exist together, only when an unauthorized person enters the control area. Selective security monitoring services that output security notifications can be applied.

In addition, the controller 180 applies an object tracking and service execution method through setting an area on the camera screen and an event rule, and provides work/operation manuals according to the movement and location of workers in the production plant, push notifications, and work such as safety rules. Smart Factory service that provides necessary information can be applied.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for the operation of each component.

Various embodiments described herein may be implemented in a recording medium that can be read by a computer or a similar device using, for example, software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. The described embodiments may be implemented by the controller 180 itself.

According to software implementation, embodiments such as procedures and functions described in the present specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. The software code can be implemented with a software application written in an appropriate programming language. The software code may be stored in the memory 160 and executed by the controller 180.

2 shows an example of setting a Vid-Fence area, setting an event (service) in the Vid-Fence area, and executing a Vid-Fence service based on the configuration of the present invention described above.

2(a) shows an example of setting the Vid-Fence area, (b) shows an example of setting an event (service) in the Vid-Fence area, and (c) runs the Vid-Fence service. One example is shown.

That is, if the camera image according to the present invention is set as a Vid-Fence by drawing various shaped areas using a tool on the monitor screen, and an event is set and executed for each Vid-Fence, the service is provided when the corresponding event is detected through the camera. An example of a running process is shown.

First, referring to FIG. 2A, an example in which various drawing tools are provided to set an area on a camera screen in relation to setting a Vid-Fence area is shown.

In FIG. 2A, the first region 210, the second region 220, and the third region 230 are set as the Vid-Fence region, and the first region 210 and the second region 220 And in setting the third area 230, the setting tool 200 may be used.

Typically, a plurality of polygons are displayed on the setting tool 200, and the user first determines the shape of the area to be set with the setting tool 200, and then designates the corresponding area through a touch, etc., so that the first area 210 , The second area 220 and the third area 230 may be set.

Next, referring to FIG. 2B, an example of setting an event and an execution service in each Vid-Fence area set in FIG. 2A is shown.

As an example of various events that can be set by the user, there may be a rule setting of an object's entry, stay, and exit event rules for Vid-Fence.

In addition, it is possible to set the criteria for determining the object (the whole shape or the part of the floor), and the rules for overlapping objects and Vid-Fence (time, overlapping degree (%), etc.) can be set, and the target person can be selected. (All, customer segment, vehicle segment can be specified) is also possible.

In (b) of FIG. 2, an example of setting a second event 250 for the second area 220 is displayed, and events such as a push message corresponding to a specific event, warning alarm, coupon sending, and payment can be set .

Thereafter, referring to (c) of FIG. 2, an example of execution of the Vid-Fence service is shown.

That is, the vehicle 300 may recognize the number through the recognition camera 195. In (c) of FIG. 2, the vehicle number of the vehicle 300 of 3173 is recognized, and this is tracked through the camera 121.

Other events (services) may be provided depending on whether the vehicle 300 of the 3173 vehicle number enters 211 or stays 212 in the first area 210.

In addition, other events (services) may be provided depending on whether the vehicle 300 with the 3173 vehicle number enters 221, stays 222, or leaves 223 in the second area 220.

In addition, other events (services) may be provided depending on whether the vehicle 300 having the 3173 vehicle number enters 231 or leaves 232 of the third area 230.

In addition, other events (services) may be provided for an event in which the vehicle 300 having the vehicle number 3173 leaves the image captured by the camera 121 (240).

A method of tracking an object by setting an area on a camera screen according to the present invention will be described in more detail below with reference to the drawings.

Object tracking and service execution method by setting the area on the camera screen and event rule

3 is a flowchart illustrating an object tracking and service execution method through setting an area on a camera screen and an event rule proposed by the present invention.

Referring to FIG. 3, the object tracking and service execution method by setting an area on a camera screen and an event rule proposed by the present invention is described in step S1 corresponding to steps (a) and (b) of FIG. 2 and (c) of FIG. ) Can be divided into the S2 step corresponding to the step.

First, referring to step S1, a screen image acquired through the camera 121 is displayed on the display unit 151 (S10).

Thereafter, the user can set the Vid-Fence (S11).

In step S11, as described in (a) of FIG. 2, the first region 210, the second region 220, the third region 230, etc. may be set as a plurality of Vid-Fence regions, and a plurality of In setting the Vid-Fence area, the setting tool 200 may be used.

Typically, a plurality of polygons are displayed on the setting tool 200, and the user first determines the shape of the area to be set with the setting tool 200, and then designates the corresponding area through a touch, etc., so that a plurality of Vid-Fence areas Can be set.

After step S11, a step (S13) of setting a service and an execution rule proceeds.

In step S13, as an example of various events that can be set by the user, the entry, stay, and exit event rules for the Vid-Fence of an object may be set.

In addition, it is possible to set the criteria for determining the object (the whole shape or the part of the floor), and the rules for overlapping objects and Vid-Fence (time, overlapping degree (%), etc.) can be set, and the target person can be selected. (All, customer segment, vehicle segment can be specified) is also possible.

That is, as shown in (b) of FIG. 2 described above, an example of setting the second event 250 for the second area 220 is displayed, and a push message corresponding to a specific event, warning alarm, coupon sending, payment, etc. Events can be set.

After the step S12, the step S14 of activating the Vid-Fence and the service proceeds, and the process for this may be completed (S18).

In the process of step S2, steps S15 in which the Vid-Fence setting in step S12 is applied (S15), and steps in which the service and execution rules set in step S13 are applied (S16, S17) are preliminarily performed.

Regarding step S2, first, a step (S110) in which the moving object 300 enters the screen of the camera 121 is performed.

After step S110, based on the step of applying the Vid-Fence setting of step S12 (S15), and the step of applying the set service and execution rule of step S13 (S16, S17), the controller 180 After the execution rule is inquired, the step of storing it in the memory 169 proceeds.

Thereafter, the ID identification step (S120) proceeds. For example, the vehicle 300 may recognize the number through the recognition camera 195.

In addition to the method of using the recognition camera 195, a method of tagging an ID card, a fingerprint recognition method, an iris recognition method, and the like may be applied.

Thereafter, the step of reading the area of the object, that is, the Vid-Fence, is performed by the controller 180 (S130).

Simultaneously with step S130 or following step S130, a process (S140) of tracking the movement of the identified object under the control of the controller 180 is performed.

In this regard, the controller 180 performs the step S125 of deriving an execution rule based on step S115, and the execution rule derived in step S125 may be applied to the image movement tracking in step S140.

Thereafter, based on the step of applying the Vid-Fence setting preliminarily performed before step S2 (S15), and the step of applying the set service and execution rule of step S13 (S16, S17), execution for each Vid-Fencing service The step of detecting the rule (S150) proceeds.

In addition, step S150 may be performed based on the execution rule derived in step S125.

If the execution rule is met (S160), the controller 180 executes the service (S170).

In addition, if the execution rule is not met or the screen exits while the service is running (S180), the controller 180 terminates the provision of the service (S190).

If the screen does not leave the screen even if it does not meet the screen execution rule, the step (S140) of continuously tracking the object in the image is performed.

Hereinafter, in relation to object recognition and tracking, a method applied for more accurate recognition and tracking by the controller 180 will be described with reference to the drawings.

Object recognition method according to camera shooting angle

4 is a diagram illustrating an object recognition standard and method according to a camera photographing angle applied to the present invention.

Referring to FIG. 4, in the present invention, the object recognizing type is divided into the overall shape or the floor contact surface because the criteria for determining whether the object enters, stays, or leaves the Vid-Fence varies according to the camera installation position and the shooting angle. To set.

First, referring to (a) of FIG. 4, an example in which distortion occurs in a photographing angle of the camera 121 is shown.

That is, even though the vehicle 300 has not entered the set Vid-Fence 260, it is assumed that the vehicle 300 has entered the set Vid-Fence 360 due to the distortion of the shooting angle of the camera 121. It can be misrecognized.

Therefore, in the present invention, as shown in (b) of FIG. 4, the floor surface 400 is first extracted for each vehicle 300.

Thereafter, as shown in (c) of FIG. 4, by determining whether or not the extracted bottom surface 400 has entered the set Vid-Fence 260, a problem in which distortion occurs in the shooting angle of the camera 121 is solved. It can be solved.

How to recognize entry, stay, and exit

Next, in the present invention, a method of creating and determining a criterion by combining at least one degree of overlap, overlapping time, distance, and movement angle may be employed in order to increase the recognition rate for entry, stay, and departure.

5 is a diagram illustrating a standard and method for recognizing entry, stay, and exit according to the present invention.

In (a) of FIG. 5, an example of designating an object and a Vid-Fence overlapping degree% in relation to the setting of basic standards for entry, stay, and departure is shown.

In Fig. 5(a), an example is shown when the entry overlap is set to 50%. As shown in the figure on the right, when the object and Vid-Fence overlap by 50% or more, the controller 180 processes the entry as entering. do.

In addition, with regard to the setting of multiple criteria for entry, stay, and exit. The combination of the degree of overlap + the distance d + the moving angle θ + the time t may be used as a reference.

In (b) of FIG. 5, it is determined whether or not the object exists in the Vid-Fence by using the combination of the degree of overlap and the time t together.

In addition, in (c) of FIG. 5, it is determined whether the object exists in the Vid-Fence by using the combination of the distance d and the moving angle θ.

That is, in the present invention, by creating a standard that combines one or more of d(↑↓), θ(↑↓), %(↑↓), and t(↑↓), whether the object enters, stays or leaves within the Vid-Fence. You can judge whether or not.

Automatic refueling order service

FIG. 6 is a flow chart illustrating a service for ordering automatic refueling by identifying a gas tanker stopped at a gas station as an example of utilizing Vid-Fencing proposed by the present invention.

Referring to FIG. 6, the moving object 300 enters the screen of the camera 121 and is recognized (S22) and the ID identification step (S21) is performed through the recognition camera 195.

In step S22, the controller 180 may divide a stop space within the image acquired by the camera 121 to give a lubricator identification code.

In addition, in step S21, in addition to the method of using the recognition camera 195, a method of tagging an ID card, a fingerprint recognition method, an iris recognition method, and the like may be applied.

In addition, a driver guidance method may be applied at the access road for number recognition.

Thereafter, the controller 180 performs a step (S23) of mapping the recognized vehicle number and the entering vehicle.

After step S23, a process (S24) of tracking the movement of the recognized object while the controller 180 reads the area of the object, that is, the Vid-Fence, is performed.

That is, the moment the vehicle 300 enters the gas station, the image processing server identifies the vehicle number in the image of the recognition camera 195, and the vehicle number identified by converting the entering vehicle 300 into an object in the image of the camera 121 After mapping is performed, the movement of the object 300 to which the number is mapped in the image of the camera 121 is tracked.

At this time, a method of considering network load when installing a preprocessing server for each gas station or transmitting to a real-time platform server is applied.

In addition, various methods of tracking an object while reading a Vid-Fence may be employed as an object tracking algorithm.

Thereafter, the controller 180 determines whether the moving object 300 has stopped and whether it has stopped within the Vid-Fence (S25).

When it is determined that the vehicle stops within the Vid-Fence, various events set by the user may be applied.

That is, as an example of various events that can be set by the user, there is an event rule setting for entering, staying, and leaving the Vid-Fence of an object, and setting an object judgment criterion (total shape or floor surface) Part), object and Vid-Fence overlap rule setting (time, overlapping degree (%), etc.) can be set, and target selection (all, customer segment, vehicle segment can be specified) is also possible.

Representatively, in step S26, a step (S26) of providing a service related to gasoline request and fueling is performed.

That is, if the execution rule is met, the controller 180 executes the fueling service (S26).

Here, when the vehicle number-identified object 300 stops at a stop space to which the identification code is assigned, an automatic refueling order request is made.

If the execution rule is not met or the screen exits while the service is running, the controller 180 may terminate the provision of the service.

In addition, even if the screen execution rules are not met, if the screen does not leave the screen, the object in the image may be continuously tracked.

Thereafter, the step (S27) of finally leaving the vehicle 300 proceeds.

That is, in step S27, when the payment is completed after refueling and the vehicle (object) disappears from the screen, exit processing is performed.

FIG. 7 shows an example of a service for ordering automatic refueling by identifying a gas tanker that has stopped at the gas station described in FIG. 6.

Referring to FIG. 7, the moving targets 300a, 300b, and 300c are recognized by entering the screen of the camera 121, and the vehicle number is identified through the recognition camera 195.

At this time, the control unit 180 may partition the stop space within the image acquired by the camera 121 and give the lubricator identification codes 400a, 400b, 400c, 400d, 400e, 400f, 400g, 400h, 400i. .

For number recognition, a driver guidance method can be applied at the access road, and number recognition guidance and entry for Smart Order & Pay can be applied to the access road.

At this time, the controller 180 maps the recognized vehicle number and the entering vehicle.

That is, the moment the vehicle (300a, 300b, 300c) enters the gas station, the image processing server 500 identifies the vehicle number in the image of the recognition camera 195, and the entry vehicle (300a, 300b) in the image of the camera 121 , 300c) is converted into an object and the identified vehicle number is mapped, and then the movement of the objects 300a, 300b, and 300c to which the number is mapped in the image of the camera 121 is tracked.

In FIG. 7, vehicle numbers of the first vehicle 300a and 3173 are mapped, the second vehicle 300b and the 2468 number are mapped, and the third vehicle 300c and the 1357 number are mapped.

Thereafter, the controller 180 determines whether the moving targets 300a, 300b, and 300c have stopped and whether they have stopped within the Vid-Fence.

When it is determined that the vehicle stops within the Vid-Fence, such as 300b and 300c, various events set by the user may be applied.

That is, a service request and a service related to refueling are provided, and an automatic refueling order request is made when the objects 300b and 300c with the vehicle number identified stop in the stop spaces 400e and 400f to which the identification code is assigned.

If the execution rule is not met or the screen exits while the service is running, the controller 180 may terminate the provision of the service.

In addition, when the payment is completed after refueling and the corresponding vehicle (300a, 300b, 300c) disappears from the screen, exit processing is performed.

Selective security monitoring method

Meanwhile, FIG. 8 is a diagram illustrating a selective security monitoring method as an example of utilization of Vid-Fencing proposed by the present invention.

Referring to FIG. 8, a control area 712 and a general area 711 are displayed together on a screen 151 of a camera 121.

In this case, the Vid-Fence 400 may be set in an area corresponding to the control area 712.

Thereafter, if there is an unauthorized person, the unauthorized person is tracked, and the unauthorized person enters the control area 712 and is located within the Vid-Fence 400 on the screen 151, The controller 180 provides a security notification 720 through the output unit 150.

How to apply Smart Factory

In addition, FIG. 9 is an example of the use of Vid-Fencing proposed by the present invention. A Smart Factory that provides information necessary for work, such as providing work/operation manuals, push notifications, and safety rules according to the movement and location of workers in a production factory. It is shown.

9, on the camera 121 screen 151, the operation manual Vid-Fence (410), the operation manual Vid-Fence (420), safety rules Vid-Fence (430), production status Vid-Fence (440). ) Can be set.

Thereafter, the identified object is tracked, and if the object exists on the manual Vid-Fence (410), the operation manual information (810) is provided, and if it is on the operation manual Vid-Fence (420), it is manipulated. Manual information 820 is provided, and if it exists on the safety rule Vid-Fence 430, it provides safety rule information 830, and if it exists on the production status Vid-Fence 440, the production status information 840 ).

Effects according to the invention

The present invention can provide an object tracking and service execution system to a user through setting an area on a camera screen and an event rule.

Specifically, in the present invention, when the camera image is set as a Vid-Fence by drawing a region of various shapes using a tool on the monitor screen, an event is set and executed for each Vid-Fence, the service is executed when a corresponding event is detected through the camera. It is possible to provide a system to users.

In addition, the present invention is a system for setting the type of recognizing an object by dividing it into an overall shape or a floor contact surface because the criteria for determining an object entering, staying, or leaving the Vid-Fence are different depending on the camera installation position and the shooting angle. Can be provided to the user.

In addition, the present invention creates and determines a criterion by combining one or more overlapping degree, overlapping time, distance, and moving angle in order to increase the recognition rate for entry, stay, and departure. A service execution system can be provided to users.

In addition, the present invention applies an object tracking and service execution method through setting an area on a camera screen and an event rule, thereby identifying a gas station at a gas station and providing a service and a system capable of ordering automatic refueling to a user.

In addition, the present invention applies an object tracking and service execution method through the setting of an area on the camera screen and an event rule, so that when a control area and a general area on the camera screen exist together, a security notification only when an unauthorized person enters the control area. It is possible to provide a selective security monitoring service and system that outputs to a user.

In addition, the present invention applies an object tracking and service execution method by setting an area on the camera screen and an event rule, providing a work/operation manual according to the movement and location of workers in a production plant, push notification, safety rules, etc. Smart Factory services and systems that provide information can be provided to users.

On the other hand, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description. I will be able to.

The embodiments of the present invention described above can be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of implementation by hardware, the method according to embodiments of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), and Programmable Logic Devices (PLDs). , Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory unit and driven by a processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor through various known means.

Detailed description of the preferred embodiments of the present invention disclosed as described above has been provided to enable those skilled in the art to implement and practice the present invention. Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the scope of the present invention. For example, a person skilled in the art may use the components described in the above-described embodiments in a manner that combines them with each other. Accordingly, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The present invention may be embodied in other specific forms without departing from the spirit and essential features of the present invention. Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention. The invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, the embodiments may be configured by combining claims that do not have an explicit citation relationship in the claims, or may be included as new claims by amendment after filing.

Claims (12)

A first step of setting a Vid-Fence that is at least one area in the image acquired by the first camera;
A second step of setting a service corresponding to each of the at least one Vid-Fence;
A third step of recognizing an entry of an object based on the image acquired by the first camera;
A fourth step of obtaining an identifier (ID) of the object;
A fifth step of matching the entered object with an identifier (ID) of the object;
A sixth step of tracking an object matching the identifier (ID);
A seventh step in which an object matching the identifier (ID) is located in a first Vid-Fence among the at least one Vid-Fence; And
An eighth step of providing a first service set in the first Vid-Fence to an object matched with the identifier (ID); Including,

In the second step, an event rule for providing the service is set together,

The event rule,
A degree of overlapping with the at least one Vid-Fence, a distance from a center of the at least one Vid-Fence, a direction in which the object moves based on the center of the object, and a center of the at least one Vid-Fence It is set by combining at least one of a moving angle degree of the liver and a time overlapping with the at least one Vid-Fence,

In the seventh step,
Based on the set event rule, it is determined whether or not the object matching the identifier (ID) is located in the first Vid-Fence,

In the seventh step,
Based on the set event rule, it is determined whether the object matching the identifier (ID) is in an entry state, a stay state, and an exit state with respect to the first Vid-Fence,
The eighth step is performed only when it is determined that the staying state is,

The seventh step,
A 7-1 step of extracting a floor surface corresponding to the size of the object matched with the identifier (ID); And
Based on the set event rule, comparing the floor surface with the first Vid-Fence, and whether the object matching the identifier (ID) is in an entry state, a stay state, or an exit state with respect to the first Vid-Fence Including; a 7-2 step of determining whether or not,

The first step,
Step 1-1 of displaying a drawing tool in the image acquired by the first camera; And
A 1-2 step of designating a certain point in the image acquired by the first camera, and setting the at least one Vid-Fence of a certain shape at the designated point based on the drawing tool; and

The at least one Vid-Fence is at least one of an area corresponding to a gasoline in a gas station, an area corresponding to a control area, and an area corresponding to a working section,

When the at least one Vid-Fence is an area corresponding to a lubricator in a gas station,
The object is at least one vehicle,
In the seventh step, when the object is located within the first Vid-Fence,
The first service provided in the eighth step is an automatic refueling order request service,

When the at least one Vid-Fence is an area corresponding to a control area,
The object is a person who has not been authorized in advance,
In the seventh step, when the object is located within the first Vid-Fence,
The first service provided in the eighth step is a service that provides security risk information,

When the at least one Vid-Fence is an area corresponding to a working section,
In the seventh step, when the object is located within the first Vid-Fence,
The first service provided in the eighth step includes a service for providing a work manual, a service for providing an operation manual, a service for providing safety rules, and a service for providing production status.
delete delete delete delete delete delete The method of claim 1,
In the fourth step,
The identifier (ID) of the object,
It is characterized in that it can be obtained using at least one of a method obtained through a second camera, a card tagging method, a fingerprint recognition method, an iris recognition method, a method using RFID, a method using NFC, and a method using a BLE beacon. Object tracking service method.
delete delete delete ◈ Claim 12 was abandoned upon payment of the set registration fee. A first camera that acquires an image;
An input unit for setting a Vid-Fence, which is at least one area in the image acquired by the first camera, and a service corresponding to each of the at least one Vid-Fence; And
Including; a control unit for recognizing the entry of the object, based on the image acquired by the first camera,

When the identifier (ID) of the object is obtained, the control unit,
Match the entered object and the identifier (ID) of the object,
Track the object matching the identifier (ID),
When the object matching the identifier (ID) is located in the first Vid-Fence of the at least one Vid-Fence,
The control unit,
An object tracking service system, characterized in that controlling so that a first service set in the first Vid-Fence is provided to an object matched with the identifier (ID).

The control unit,
An event rule for providing the service is set together,

The event rule,
A degree of overlapping with the at least one Vid-Fence, a distance from a center of the at least one Vid-Fence, a direction in which the object moves based on the center of the object, and a center of the at least one Vid-Fence It is set by combining at least one of a moving angle degree of the liver and a time overlapping with the at least one Vid-Fence,

The control unit,
Based on the set event rule, it is determined whether or not the object matching the identifier (ID) is located in the first Vid-Fence,

The control unit,
Based on the set event rule, it is determined whether the object matching the identifier (ID) is in an entry state, a stay state, and an exit state with respect to the first Vid-Fence,
Control so that the first service set in the first Vid-Fence is provided to the object matched with the identifier (ID) only when it is determined as the staying state,

The control unit,
The object whose identifier (ID) is matched by extracting the floor surface corresponding to the size of the object matched with the identifier (ID) and comparing the floor surface with the first Vid-Fence based on the set event rule Determines whether the first Vid-Fence is in an entry state, a stay state, and a departure state,

The control unit,
In the image acquired by the first camera, a drawing tool is controlled to be displayed, a certain point is designated in the image acquired by the first camera, and a certain shape at the designated point based on the drawing tool Set the at least one Vid-Fence of,

The at least one Vid-Fence is at least one of an area corresponding to a gasoline in a gas station, an area corresponding to a control area, and an area corresponding to a working section,

When the at least one Vid-Fence is an area corresponding to a lubricator in a gas station,
The object is at least one vehicle,
When the object is located in the first Vid-Fence, the provided first service is an automatic refueling order request service,

When the at least one Vid-Fence is an area corresponding to a control area,
The object is a person who has not been authorized in advance,
When the object is located within the first Vid-Fence,
The provided first service is a service that provides security risk information,

When the at least one Vid-Fence is an area corresponding to a working section,
When the object is located within the first Vid-Fence,
The first service provided includes a service for providing a work manual, a service for providing an operation manual, a service for providing safety rules, and a service for providing production status.

Images