KR101692504B1 - Intelligent image analysis system using beacon and method thereof - Google Patents

Abstract

The present invention relates to an intelligent image analysis system and method, and more particularly, to an intelligent image analysis system and method for analyzing object information included in an image, An identification information obtaining unit that receives identification information transmitted from a user terminal held by a user located in the predetermined area; And a final image information output unit for outputting final image information using the image analysis information and the identification information, and an intelligent image analysis method for the intelligent image analysis system.

Description

[0001] The present invention relates to a beacon-based intelligent image analysis system and method,

The present invention relates to an intelligent image analysis system and method, and more particularly, to a system and method for performing image analysis using a wireless communication device such as an intelligent camera and a beacon.

Conventional general CCTV cameras have only taken a certain area and transmitted the image.

In recent years, intelligent image analysis system has been developed, and it is now possible to analyze objects such as identifying objects and detecting events in photographed images.

1 is a view for explaining a conventional intelligent image analysis system.

The existing object analysis function is a technology that detects the action automatically by using the image information. It detects the event based on the background region separation step, the object identification step, the object tracking step and the predefined rule as shown in the following figure Step

The background region separation step is a process of detecting an active object by separating the foreground region and the background region of interest from the input image. In the initial stage, a method of calculating brightness difference between the current image and the previous image is used .

However, in recent years, there have been many methods of delineating the background information by using a Gaussian or Gaussian Mixture Model (GMM), and then distinguishing the foreground and background regions.

The object identification step is a process of distinguishing whether a detected object is a human or an object among the objects judged as a foreground area in the background area. Currently, the intelligent image analysis technique mainly distinguishes a person (one person, a group), an animal, and an object.

Object tracking is a process of finding the movement path of an identified object in successive images. It is based on various algorithms such as Kalman filter, particle filter, CAMSHIFT (Continuously Adaptive Mean Shift) Algorithm is used.

In the event detection step, an event is detected by determining whether the rule defined by the security manager is violated based on the object identification information and the object movement information, and the detected information is stored in the form of metadata in the form of VMS (Video Management System) To the security management server, and the types of the events are as follows.

The intelligent AP-camera uses intelligent image analysis techniques that support various sensing functions to detect intruding / wandering / wandering / neglecting objects in the virtual boundary area.

The camera shake / damage function is used to prevent theft and damage. When using this function, notification function (SMS, alarm, ..) should be essential for commercialization.

In addition, real-time performance must be ensured in performing these functions.

Since the conventional intelligent image analysis system can perform only the function of distinguishing objects from a photographed image and detecting a simple specific pattern, it is possible to perform only counting of the number of entrances, Such as the ability to identify a person as a person, a function to capture a person, or a person who actually enters or exits through a route.

Korean Patent Laid-Open Publication No. 2015-0034398 (published May 3, 2015) (Method for Detecting Parking Event Based on Object Recognition) Summary, Claim 1, Korean Patent Laid-Open Publication No. 2014-0141239 (published Dec. 10, 2014) (Real Time Object Tracking Method and System Using Average Moving Algorithm) Summary, Claim 1,

In order to solve the above-mentioned problem, according to the present invention, each user passing through a region to be photographed has a near field wireless communication device such as a beacon, and the information obtained using the near field wireless communication device and the information obtained through the intelligent image analysis system And an intelligent image analysis method and an intelligent image analysis method capable of performing more advanced image analysis by combining information.

In order to solve the above problems, the present invention provides a computer-readable recording medium on which a program for realizing the above-described method is recorded.

According to an aspect of the present invention, there is provided an image processing apparatus comprising: an image information analysis unit for outputting image analysis information obtained by analyzing object information included in an image of a predetermined region; An identification information obtaining unit that receives identification information transmitted from a user terminal held by a user located in the predetermined area; And a final image information output unit outputting final image information using the image analysis information and the identification information.

According to another aspect of the present invention, there is provided an image processing method comprising the steps of: outputting image analysis information obtained by analyzing object information included in an image of a predetermined region; Receiving identification information transmitted from a user terminal located in the predetermined area; And outputting the final image information using the image analysis information and the identification information. The present invention also provides an intelligent image analysis method.

The intelligent image analysis system may further include a location information determination unit for determining location information of the user.

Also, the user terminal may be a beacon.

In addition, the final image information may include information for determining whether or not there is an object that does not have a user terminal whose identification information is registered in the image.

In order to solve the above problems, the present invention provides a computer-readable recording medium on which a program for realizing the above-described method is recorded.

According to the present invention, it is possible to provide more improved analysis information beyond the limitation of analyzing only a photographed image.

1 is a block diagram showing the configuration of a conventional intelligent image analysis system
2 is a block diagram illustrating the configuration of an intelligent image analysis system according to the present invention.
3 is a diagram for explaining an event detection function according to a situation in the present invention;
4 is a diagram for explaining the analysis function of the intelligent image analysis system of the present invention;
5 is a functional flowchart for explaining an embodiment of the present invention.
6 is a view for explaining another embodiment of the present invention
7 is a block diagram showing the configuration of the system of the present invention.
Figure 8 is a flow chart illustrating the method of the present invention.

The following merely illustrates the principles of the invention. Thus, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or shown herein, embody the principles of the invention and are included in the concept and scope of the invention. Furthermore, all of the conditional terms and embodiments listed herein are, in principle, intended only for the purpose of enabling understanding of the concepts of the present invention, and are not intended to be limiting in any way to the specifically listed embodiments and conditions . It is also to be understood that the detailed description, as well as the principles, aspects and embodiments of the invention, as well as specific embodiments thereof, are intended to cover structural and functional equivalents thereof. It is also to be understood that such equivalents include all elements contemplated to perform the same function irrespective of currently known equivalents as well as equivalents to be developed in the future.

Thus, the functions of the various elements shown in the drawings, including the functional blocks shown in the figures or similar concepts, may be provided by use of dedicated hardware as well as hardware capable of executing software in connection with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared. Also, the use of terms such as processor, control, or similar concepts should not be construed as exclusive reference to hardware capable of executing software, but may include, without limitation, digital signal processor (DSP) hardware, (ROM), random access memory (RAM), and non-volatile memory. Other hardware may also be included.

The above objects, features and advantages will become more apparent from the following detailed description in conjunction with the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

On the other hand, when an element is referred to as "including " an element, it does not exclude other elements unless specifically stated to the contrary.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating functions of the intelligent image analysis system of the present invention.

According to the present invention, the location information of each beacon is added to the real-time image information for the conventional image analysis, and the identifier is included in the meta information of each image extracted.

As shown in FIG. 2, an identifier and location information of a moving object entering and exiting from a mobile beacon are acquired.

On the other hand, the corresponding region is photographed and the background region and the object are separated from the photographed image.

The identifiers and locations of each object are identified using the identifiers and location information of the separated objects and mobile objects, and the objects are tracked.

The recognized object recognition information such as the identifier of the moving object is included in the meta information for each image.

Meanwhile, it is determined whether an event set in the rule management interface has been detected, and whether an event has occurred or not is also determined.

As shown in FIG. 2, in order to detect an event, a preprocessing process of an image is required, and external input and setting values are required for each actual event function.

According to the event-specific setting, the final event is determined as shown in FIG.

As shown in FIG. 3, when an image is input, it is determined whether the object is recognized, the appearance and disappearance of the object, the direction of the moving object, the counting of the moving object, ID identification of the moving object, camera shake or damage, And judges a separate event.

That is, it is possible to detect whether or not an object appears and disappears in a specific area, whether there is an object moving only in a specific direction, whether or not there is an object entering and leaving the specific area, and how many people are entering and exiting.

FIG. 4 is a view for explaining an example of analysis through the intelligent image analysis system of the present invention.

As shown in FIG. 4, since the location information of each object (identifier) is grasped by using the identification information of the local wireless communication device such as a beacon, which each object has in the object photographed in the image, Unlike analytical information, it is possible to know who appeared and disappeared, which direction the objects moved in and where they went, and the number of movements and total counts for each object.

In addition, there is an advantage that a person who does not have a short-range wireless communication device such as a beacon can easily grasp the intruder.

In other words, it is understood that a person is moved or positioned through a photographed region through a photographed image. If a beacon signal corresponding to the person is not received or an unregistered signal is received, A person who is not allowed to enter, or an intruder.

5 is a functional flowchart for explaining an embodiment of the present invention.

FIG. 5 illustrates an example in which the intelligent image analysis system of the present invention is mainly used for management of entrance / exit of beacon holders and unauthorized intruder management of beacons in a company, a laboratory, a critical facility, or a governmental institution.

In the initial setting, the present invention is implemented by activating the image analysis function-access control function of the AP (1). When the present invention is implemented in this way, the transmission data of the AP (1) and the terminal (4) are transmitted and received via the relay server (3).

1) The beacon ② held by the user periodically transmits the unique identification number and the sensor data. In the AP①, the beacon is confirmed and stored as a signal of the registered beacon, and the data (identifier information) is pushed to the registered terminal.

2) The terminal ④ requests information from the registered AP ①, AP ① sends real-time or stored information received from the beacon ②, and the terminal ④ exposes information such as the number of the entrance counting through the GUI (For example, by applying the applied color).

6 is a diagram illustrating another embodiment of the present invention, and is a diagram for explaining a method of transmitting location information for each identifier.

The operation sequence is as follows.

1) First, a fixed beacon is periodically transmitted for location information.

2) The mobile body carrying the mobile beacon shall have the terminal (smart phone (location calculation app launch) or location calculation device).

3) The terminal periodically transmits the identifier of the mobile beacon and the position information calculated in the terminal.

4) The AP located near the AP receives and stores the terminal information and uses it in the image analysis SW module.

7 is a block diagram showing the configuration of the intelligent image analysis system of the present invention.

7, the intelligent image analysis system of the present invention includes an image information analysis unit 110, an identification information acquisition unit 120, a position information determination unit 130, and a final image information output unit 140 .

The intelligent image analysis system 100 receives a photographed image of a predetermined region of the subject from a device capturing a predetermined region such as CCTV and synthesizes signals received from the user terminal 200 such as a beacon, And the like.

Specifically, we examine each component.

The image information analysis unit 110 outputs image analysis information that is a result of analyzing the received photographed image.

An object to be analyzed by the image information analysis unit 110 is to separate objects such as people from the photographed image, analyze the number of each object, the direction in which each object moves, and output image interpretation information as an interpreted result . A technique of separating an object from an image is well known in the art, and a detailed description thereof will be omitted.

The type of information analyzed by the image information analyzing unit 110 may vary according to conditions preset by the system designer or the operator.

That is, as described above, the object may be simply separated to recognize whether the object is a person or an animal, to determine only the number of the objects, the direction in which the object is moved, Area), whether or not it stays at a specific position for a predetermined time or more, or the like.

As described above, since it is a matter that can be determined by the system designer or the like to which information about the event to be output in the image information analyzing unit 110, in the present invention, the information output from the image information analyzing unit 110 is a specific event And the like.

The identification information obtaining unit 120 obtains the identification information from the user terminal.

Here, the user terminal 200 may be a device capable of performing a wireless communication function, such as a beacon, and does not limit the type of the device.

For example, when a corresponding function is installed in a smart phone or the like, a smart phone may be used as a user terminal.

One or more fixed communication devices (not shown) fixed to a specific position such as an AP are provided in an area to be photographed, and identification information of the user terminal 200 through communication between the fixed communication device and the user terminal 200 Is transmitted to the identification information obtaining unit (120).

The position information determination unit 130 determines a position of a user located in an area to be photographed.

There are two methods for determining the location of the user.

One is that the user terminal 200 calculates the location and transmits the calculated information, and there is a method of determining the location information of the user by using the calculated information. And the position information determination unit 130 directly calculates and determines the received signal.

First, a method of determining a position through calculation in the user terminal 200 will be described.

In order to do this, each fixed communication device provided in an area to be photographed must output its own position or position information which is unique information that can recognize its position.

The user terminal 200 receives the signal output from each fixed communication device and calculates its own position using the received signal.

When the number of signals of the fixed communication device received by the user terminal 200 is only one, the approximate distance is calculated using the strength of the signal, and the distance can be calculated It is possible to calculate the area having the area.

In this case, when only one user is included in the photographed image, the position of the recognized object in the photographed image can be used to calculate the accurate position of the user. The system designer or operator can select the location of the user based on the location information of the object in the photographed image.

Since the method of calculating the distance according to the intensity of the signal may have a large error, when the number of the fixed communication apparatus signals received by the user terminal 200 is one, the apparatus having a function of calculating the distance using the intensity of the signal And as the number of objects included in the photographed image is small or the distance between objects is far away, a more accurate position can be calculated. However, it will not be easy to calculate a relatively accurate position.

In the case of receiving a signal from two fixed communication apparatuses, it is possible to calculate a more accurate position than in the preceding case. In this case, at least the direction in which the user moves can be calculated relatively accurately. However, the error range may still be large for the exact position.

A relatively accurate position can be calculated when a signal is received from three or more fixed communication devices. Also, referring to the position of the object analyzed in the photographed image, it will be possible to calculate a more accurate position.

The technique of detecting the position of the terminal using the triangulation is a well-known technology in the art of calculating the position of the mobile communication terminal by using the position of the base station, and a detailed description thereof will be omitted.

The user terminal 200 receives the signal output from the fixed communication device, calculates its own position, and transmits the calculated information to the position information determination unit 130. The position information determination unit 130 determines the position And the positional information of the object analyzed by the image information analysis unit 110 is used to determine the final positional information.

 When a signal output from the user terminal 200 is received and directly calculated by the position information determination unit 130, the fixed communication apparatus fixed in the target area receives the signal output from the user terminal 200, The information determination unit 130 determines the location of the user terminal 200.

Also in this case, the method of determining the position is the same as the method of calculating the position in the user terminal 200 to determine the final position.

That is, when only one fixed communication device receives a signal transmitted from the user terminal 200, it is assumed that there is a user in the vicinity of the fixed communication device that receives the signal, and furthermore, Estimates the range that the user can exist, and determines the position of the user based on the information and the positional information of the object analyzed by the image information analyzing unit 110.

Even when signals of the user terminal 200 are received from only two fixed communication devices and the signals of the user terminal 200 are received from three or more fixed communication devices, the position of the user is calculated in a manner similar to that described above .

The final image information output unit 140 outputs the image analysis information, the identification information of the users located in the photographed region acquired by the identification information acquisition unit 120, And outputs the final image information combined with the position information of the user obtained by the information determination unit 130.

Choi Choi Young-Sang information may be simply a form of information linked by listing each information, but each information may be included as meta information of the image.

The final image information is not necessarily all the information described above, and may not include any one of the information, and may include other information besides the information. For example, even if the location information is not included, it may be excluded when the operator can achieve the desired purpose.

In addition, all of the generated information may not be included and output, and only the predetermined items requested by the system operator may be included using the information.

For example, if the system operator wants only the identification information of the person who has passed through the target area to be photographed, only the information related thereto can be included.

In addition, the final image information output unit 140 may determine whether a predetermined event has occurred using the generated information, rather than merely combining the information.

For example, when the number of objects on the screen determined by the image information analyzing unit 110 is large in the number of pieces of identification information acquired by the identification information obtaining unit 120, it is determined that a person not registered in the system has entered An alarm message or the like may be output so that the manager can recognize this fact.

In this case, if the object in the image can be accurately identified by the object by the identification information acquisition unit 120 and the position information determination unit 130, the object that can not be identified becomes the intruder, so that the image of the person suspected of being the intruder Or an image emphasizing the corresponding part may be outputted so that the administrator can easily find the intruder.

8 is a flow chart for explaining the method of the present invention.

First, the image information analyzing unit 110 outputs image analysis information as a result of analyzing the received photographic image (301).

The object to be analyzed by the image information analyzing unit 110 may include a person or the like in the captured image, and may include the number of objects, the direction in which the objects move, and the like.

The identification information obtaining unit 120 obtains the identification information from the user terminal (302).

The location information determination unit 130 determines the location of the user located in the area to be photographed (303).

The method of determining the position of the user includes a method of determining the position information of the user using the calculated information when the user terminal 200 calculates the position and transmits the calculated information, A method of receiving the signal and directly calculating it by the position information determination unit 130 may be used as described above. Since the details have been described above, they are omitted here.

The final image information output unit 140 outputs the image analysis information output from the image analysis unit 110, the identification information of users located in the photographed region acquired by the identification information acquisition unit 120, And outputs the generated final image information using the position information of the user obtained in step S304.

As described above, Choi, Joo-young can be a form of linked information by simply listing each information, but each information may be included as meta information of the image.

In addition, the above-described information is not necessarily essential information in the final image information, and any information may not be included in the final image information, and information other than the above information may be further included.

In addition, it is also possible to include not only all of the generated information but also the predetermined information required by the system operator using the above information, as described above.

In addition, the final image information output unit 140 may determine whether a predetermined event such as an intruder has been generated using the generated information, and output the necessary information according to the generated information. As described above.

Each of the steps described above is not necessarily ordered, and the order may be changed other than the step of outputting the final image information.

The method of the present invention can also be embodied as computer readable code on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer-readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, You will understand. Accordingly, the true scope of protection of the present invention should be determined only by the appended claims.

Claims (9)

An image information analyzing unit for outputting image analysis information including appearance and disappearance of a moving object captured through an image photographed in a predetermined area, moving direction of the moving object, or counting of the moving object;
An identification information obtaining unit for receiving identification information transmitted from a user terminal held by a mobile unit located in the predetermined area; And
And a final image information output unit outputting final image information including the identification information as meta information of an image of the moving object. The system of claim 1, wherein the intelligent image analysis system
And a position information determiner for determining the position information of the user. The intelligent image analysis system of claim 1, wherein the user terminal is a beacon. The intelligent image analysis system of claim 1, wherein the final image information includes information for determining whether or not there is an object that does not have a user terminal for which identification information is registered in the image. Outputting image analysis information including appearance and disappearance of a moving object captured through an image photographed in a predetermined area, moving direction of the moving object, or counting of the moving object;
Receiving identification information transmitted from a user terminal of a moving object located in the predetermined area; And
And outputting final image information including the identification information as meta information of an image of the moving object. 6. The method of claim 5, wherein the intelligent image analysis method comprises:
And determining location information of the user based on the location information of the user. 6. The method of claim 5, wherein the user terminal is a beacon. 6. The intelligent image analysis method according to claim 5, wherein the final image information includes information for determining whether there is an object that does not have a user terminal whose identification information is registered in the image. 9. A computer-readable recording medium on which a program for realizing the method according to any one of claims 5 to 8 is recorded.

Images