KR100986918B1 - Apparatus and method for vision sensor automatic placement - Google Patents

Abstract

PURPOSE: An apparatus and a method for automatically placing a vision sensor are provided to supply an intelligent video monitoring system to a vision algorithm and a vision device by providing the basis for a service-centered theorized camera arrangement location decision. CONSTITUTION: A space modeling unit(110) models a predetermined space in two-dimensional map, and a core space extractor(120) indicates each grip space importance degree on a grid map. A fitness level calculator(130) extracts the fitness of the vision sensor based on the performance and the cost of a vision sensor as well as the sum of spatial importance metric of an FOW(Field-Of-View) internal grind. An arrange location selector(140) obtains vision sensor agreement number, and selects vision sensor arrangement location.

Description

Apparatus and Method for Vision Sensor Automatic Placement

The present invention relates to a device and method for automatically arranging a vision sensor, and more particularly, to classify static spatial information into various angles and to derive spatial importance and effectively cover a given space through an agent modeling a user's movement pattern. The present invention also relates to a camera arrangement method and method for automatically arranging a vision sensor for a suitable number of cameras for installation cost in order to properly install a camera having various performances in a specific space.

Ubiquitous has been introduced in various forms as a technical and cultural keyword since its concept was first published in 1993 by Mark Weiser.

The rapid growth of IT technology since the information age has brought the ubiquitous era that can provide networking services to users beyond time and space constraints. Services in the ubiquitous environment should be able to provide timely and autonomous service that meets the user's ideas without user intervention by judging the user's needs and surrounding environment by themselves.

Computing systems that introduce computing functions into the environment and things and connect with each other through a network are essential infrastructure environments for building ubiquitous services.

In order to solve the space constraints and build a ubiquitous infrastructure environment in the space, we used the method of individually building blocky spaces called intelligent spaces.

Intelligent spaces are building a close and vast communication system to build an integrated ubiquitous service on the network, but the various elements that make up the intelligent space are composed of independent forms specific to individual intelligent spaces.

This is because not all intelligent spaces are designed under unified rules, and infrastructure and devices introduced in intelligent spaces are developed by various standards and different levels of technology.

Project planners design the detailed project progress and target outcomes through the planning process before constructing the intelligent space.However, due to the characteristics of the intelligent space, the construction principle of the intelligent space depends on the purpose, size, and cost. As a result, changes in requirements inevitably occur.

It is very difficult to have expertise in various fields, such as knowledge about the development of intelligent objects to be deployed in intelligent spaces with various levels of technology, design principles of space design, and construction of intelligent space services. It is also because the planner does not necessarily have to know the technical knowledge about the infrastructure construction from the point of view of the importance of designing the intelligent space.

In terms of designing functional spaces, vision algorithms related to security services such as tracking and monitoring are more dependent on devices and space than any other service technology. Therefore, they cannot guarantee independent division of labor between space design, services, and device developers. It will have a development structure.

In terms of intelligent space development, this unnecessary connection structure has been pointed out as a major factor to reduce the efficiency of development. Therefore, the research related to the deployment of vision technology and sensor structure is mostly carried out by technology developers or nearsightedness in terms of vision algorithm. It is true that it has not been studied in terms of designers or project planners.

Also, most of the researches on image processing in video surveillance system were the development of stable and efficient hardware, motion and object tracking algorithms.

However, camera placement is a factor that should be taken into consideration when designing the system, in fact, the service of users who are organically provided within the collaboration and its linkages, rather than the performance of hardware and algorithms. There is a problem that can not be verified by experimentation in limited environment such as simple algorithm performance evaluation.

The technical problem to be achieved by the present invention is to design a field-of-view (FOV) of the camera in the space in order to solve the existing problems of the vision sensor arrangement, modeling the behavior patterns of the virtual agents and obtained through the space priority (Space Priority) Based on), we will provide a vision sensor arrangement device and method that can complement the coverage of the space at a minimum cost.

One embodiment of the automatic vision sensor arrangement according to the present invention for solving the above technical problem, the space modeling unit for modeling a predetermined space as a 2D grid (2D Grid) map; An important space extraction unit that numerically represents each grid space importance on the grid map based on the flow amount of the agent using the first area on the grid map and the flow probability of the agent to move from the first area to the second area; The spatial importance of the internal grid of a field-of-view (FOV) defined on the basis of the recognition distance (d), the available angle (a) and the position angle (φ) of the vision sensor at a specific coordinate on the grid map. A suitability extracting unit configured to extract a suitability of the vision sensor based on a sum and performance and cost of the vision sensor; And arranging position lines for obtaining the number of vision sensor arrangements based on the maximum value of the vision sensor placement cost and the minimum satisfying coverage representing the minimum surveillance area ratio of the predetermined space, and selecting the vision sensor arrangement positions in order of high suitability of the vision sensors. Government; characterized by including.

According to an embodiment of the present invention, there is provided a method of automatically arranging a vision sensor, according to an embodiment of the present invention, comprising: a space modeling step of modeling a predetermined space into a 2D grid map; An important space extraction step of numerically representing each grid space importance on the grid map based on the flow amount of the agent using the first area on the grid map and the flow probability of the agent to move from the first area to the second area; The spatial importance of the internal grid of a field-of-view (FOV) defined on the basis of the recognition distance (d), the available angle (a) and the position angle (φ) of the vision sensor at a specific coordinate on the grid map. A suitability extracting step of extracting a goodness of fit of the vision sensor based on a sum and performance and cost of the vision sensor; And arranging the number of vision sensor arrangements based on the maximum value of the vision sensor placement cost and the minimum satisfying coverage representing the minimum surveillance area ratio of the predetermined space, and selecting the placement of the vision sensors in order of high suitability of the vision sensor. It characterized in that it comprises a step.

According to the present invention, the automatic arrangement and method of the vision sensor according to the present invention is to establish a service-oriented theoretical camera positioning that deviates from the spatial design without objectivity limited to the performance and use algorithm of the image sensor in the construction of the existing ubiquitous intelligent video surveillance system. By providing evidence for this, it is effective to provide an intelligent video surveillance system that is independent of vision devices and vision algorithms.

Vision sensor automatic placement apparatus and method of the present invention provides an efficient method for obtaining a suitable number of cameras and placement position in consideration of the installation cost.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view showing the configuration of the automatic vision sensor arrangement according to the invention.

The automatic vision sensor arrangement 100 according to the present invention is for automatic placement position selection in a new town development area or an area requiring installation of a vision sensor such as a camera.

Vision sensor arrangement apparatus according to the present invention comprises a space modeling unit 110, the critical space extraction unit 120, a goodness of fit extraction unit 130 and the arrangement position selection unit 140.

The space modeling unit 110 models a space for arranging the vision sensor as a 2D grid map.

The critical space extracting unit 120 extracts an important spatial region based on the flow rate of the agent using the specific region on the 2D grid map and the probability of the agent moving from the starting point to the destination.

The goodness-of-fit extraction unit 130 divides the maximum monitorable area (maximum spatial coverage) of the extracted critical spatial areas by the cost of the specific location vision sensor by the vision sensor at a specific location on a 2D grid map. Extract vision sensor fit at position.

The placement position selector 140 obtains the number of vision sensor arrangements based on the minimum value of the coverage of the vision sensor placement cost and the minimum coverage area representing the minimum surveillance area ratio of the predetermined space, and arranges the vision sensors in the order of high vision sensor suitability. Select a location.

2 is a view showing the configuration of the layer constituting the spatial modeling unit of the vision sensor arrangement according to the present invention.

The spatial modeling unit 110 of the apparatus for arranging a vision sensor according to the present invention is characterized in that it is specified on a wall / space layer (a) and a two-dimensional grid (2D Grid) map in which the terrain of the space in which the vision sensor is to be arranged is divided. It includes an area layer (b) containing the flow amount of the user using the area and the flow probability information of the user to move from any starting point to the destination.

The wall / space layer is to be applied to an agent's path search algorithm by distinguishing a walkable space from a space where an image sensor is to be disposed and a space that is not.

The area layer includes the flow amount of the user using the specific area on the 2D grid map and the flow probability information of the user to move from the starting point to the destination.

The first region and the second region are represented by a circle of radius r with respect to a specific position on a 2D grid map.

Therefore, the area layer includes attribute information such as a position, a range for extracting an important spatial area, a flow amount of the user, and flow probability information of the user moving from the starting point to the destination.

The flow probability is the probability of selecting another destination when arriving at the second area (target point) from the first area (starting point) on the 2D grid map. The total number of destinations n consists of n-1 arrays excluding itself.

The spatial modeling unit 110 of the vision sensor arrangement further includes a priority layer.

Priority layer is a database (DB) that numerically represents the spatial importance through the agent's path search. Agent updates the importance according to the action and finally it is used as important information for camera positioning.

3 is a view showing a result of modeling the movement pattern of the agent in the automatic vision sensor arrangement according to the present invention.

The present invention simulates an agent as a user, and the agent virtually implements all possible movements of the user using the real space.

Users who use real spaces use various paths that are difficult to predict even if they move to the same starting point and target point.

This is because the priority of route selection depends on various factors such as user's emotion, prior knowledge, and surrounding environment.

The path obtained by the AI-based path search algorithm with the shortest priority as shown in FIG. 3A is significantly different from the general user's path selection (FIG. 3B) as shown in FIG. 3A. It is assumed that all the topographical information of the agent has been learned, no consideration is given to the local field of view, and the cost of the agent's turn is not considered. Appears in the form.

In the present invention, in order to minimize the difference between the actual user's movement path and the agent's movement path selection, various types of paths are further calculated by estimating available paths around the route obtained by the path search algorithm.

Regions with user-selectable possibilities are mostly included in one of the coordinates on the various movement paths shown in these derived routes.

The agent initially examines all the regions that can be moved from all the destinations, and performs route search simulation by repeating inference in the following order.

1. If there is no unselected destination, end the simulation, otherwise choose a destination based on the probability distribution at the current location.

2. The path to the destination is searched through the path search algorithm.

3. In the path of the searched algorithm, the path is extended by using the path scholar algorithm according to the area attribute information.

4. Update the spatial importance of the movable area existing on all the obtained expansion paths to the important spatial area information according to the area probability distribution.

5. Change the destination to your current location and return to step 1.

Finding a path is the process of reasoning that seeks the most efficient sequence of movements from the starting point to the destination. That is, the path search includes a psychological process of recognizing, judging, and reviewing not only the physical action of finding the destination but also the environmental information appearing in the process of performing the action.

The factors considered in the path search are the objective factors that allow the selection of the optimal path according to the topographical environmental factors and the preferred values of the various search paths, and the emotional factors of the subject who finds the route.

There have been many examples of modeling environmental factors into simulations, but few have introduced emotional factors into path search algorithms. The reason is that the emotional part is the area of artificial intelligence related to human's learning ability, reasoning ability, perception ability, natural language comprehension ability, and in most research, it is effective to lead to simulation of this ambiguous and inconclusive area. Because there is no.

The A * algorithm, which is one of the path search algorithms used in the present invention, is a search algorithm that proceeds with inference only for the purpose of shortest path and shortest cost excluding the emotional part.

The A * algorithm is based on the conventional heuristic search and graph search algorithms. The graph search is basically a map or selection response element connected in a tree structure according to the search selection criteria. define.

The A * algorithm is as follows.

1. Put start node A in the list OPEN. Store A in the first search tree T.

2. Define the empty list CLOSED.

3. Check if OPEN is empty and if it is empty return search failure.

4. Select the first node in OPEN, call it An, and put it in CLOSED.

5. If An is the target node, trace and return the path from A to An.

6. Store all nodes connected to node An in search tree T.

7. Put all connected nodes into OPEN.

8. Rearrange the contents of the OPEN according to heuristics and other factors.

9. Return to step 3.

Based on the A * algorithm in the present invention, the direction of the 2D Grid-based map is divided into eight types, and the heuristic evaluation function is defined as follows to utilize the A * algorithm for agent path search.

Here, G is the sum of the moving costs of the current node branch from the starting node, and H is the sum of the moving costs ignoring the obstacles of the arriving node branches from the current node.

Thus, F, the sum of G and H, is the final criterion for prioritizing path traversal in the A * algorithm. In the case of the moving cost, the up, down, left, and right directions were set at 1, and the diagonal direction was set at 1.4.

The Manhattan method was used to calculate the 'movement cost' of the movement between nodes to find H.

4 is a view showing an embodiment of modeling the movement pattern of the agent in the A * algorithm in the automatic automatic vision sensor arrangement according to the present invention.

As mentioned above, there are many differences between the paths found through the A * algorithm and the various paths of real users. In order to compensate for this, the path extension algorithm is additionally applied in the present invention.

The agent path search algorithm updates the information for extracting important spaces by inferring the area where the user is expected to move among the walkable spaces around the path of the A * algorithm through the 'path extension algorithm'.

FIG. 5 is a diagram illustrating an embodiment of modeling a movement pattern of an agent using a path extension algorithm in the automatic vision sensor arrangement according to the present invention.

The reasoning process of the path extension algorithm is as follows.

1. Obtain the area ranges r1 and r2 of the starting point and the arrival point according to the area attribute information.

2. According to the size of the area range r1 of the starting point, the number of extended starting points n is determined, and n extended starting point positions are determined within the area range.

3. The number of extended starting points m is determined according to the size of the area range r2 of the arrival point, and m extended starting point positions are determined within the area range.

4.Explore (n + 1) * (m + 1) additional routes by applying the A * algorithm to n + 1 extended departure points including departure points and m + 1 expansion arrival points including arrival points. .

5. Search for the grids enclosed by the lines in each path and add them to the extension.

6. Return the Grid for all added paths and Grid for # 5 to the extended area.

The algorithm for selecting the position of the vision sensor such as the camera is designed with the algorithm of the Greedy Strategy.

Based on the importance information in the area attributes of the spatial model unit and the route information obtained from the agent's path search algorithm, the critical space region is derived. Finally, the placement position selection selects the placement position through this important space region.

According to the characteristics of the greedy strategy that makes judgments based on the current state information only, the placement position selection examines all coordinate points of the spatial model and determines the most important points through the camera field-of-view (FOV).

6 is a view showing an embodiment modeling a field-of-view (FOV) of the vision sensor of the present invention.

 The camera field-of-view (FOV) has a minimal structure without considering the linkage with the vision system as shown in FIG. 4.

It has a triangular structure based on the camera's position as a parameter of recognition distance and available angle.

One vision camera is positioned as an integer coordinate of X and Y in a 2D based grid map and has a maximum position angle of 360 degrees at a fixed position.

Therefore, there are four factors that determine the FOV of the vision camera in the simulation: the vision camera position (Cx, Cy), the position angle φ, the recognition distance d, and the available angle a.

The optimal position is checked for suitability based on each coordinate point. The suitability depends on various conditions such as the cost and performance of the camera in the spatial importance of the critical space area.

The goodness of fit for all coordinate points and camera types is calculated as:

Goodness-of-fit refers to a number at an angle that can cover the spatial importance within the most FOV for all directions in the corresponding grid coordinates. In other words, it is the maximum space importance coverage that can be obtained in the Grid.

This value divided by the camera cost was defined as goodness of fit.

Among the various types of cameras set in the simulator, the camera with the highest suitability in the grid will be a low cost and high performance camera.

The camera placement position is selected in the order of Gird having the highest goodness of fit among all the grids of the goodness of fit map, and the goodness of fit within the range is reconstructed at the same time as the camera is installed.

7 is a diagram illustrating an embodiment of extracting a goodness of fit of the automatic vision sensor arrangement according to the present invention.

FIG. 7 illustrates three types of cameras as one example of extracting a goodness of fit according to the type of camera.

For the camera FOV modeling, the recognition distance d, the available angle a and the camera cost c are variables.

Camera performance is shown in Table 1 below.

Recognition distance Available angle cost  Camera A  60 Grid  60˚  80  Camera B  90 Grid  450˚  120  Camera C 120 Grid  60˚  200

The placement algorithm using the greedy algorithm calculates the goodness of fit in consideration of the sum of the placement costs and the maximum spatial importance that can be included for all camera types in all Grids on the map.

The goodness-of-fit map of the camera is continuously updated until the optimal number of camera batches is finally set.

The first camera placement suitability map on the finally derived grid is shown in FIG. 5.

FIG. 7A is a graph showing a placement suitability of camera A, FIG. 7B is a graph showing a placement suitability of camera B, and FIG. 7C is a graph showing a placement suitability of camera C. FIG.

If the camera covers all updated spatial importance on the Priority Layer of the input space through the FOV, the placement is complete, but meeting the 100% spatial importance is not the primary camera placement.

Since the optimal placement of the camera is an uncertain standard that may vary depending on the conditions of the environment, resources, etc., in the present invention, for the optimal number of placement of the camera, (1) placement cost limit (maximum number of placements not exceeding the cost limit) and (2) Calculate the set minimum coverage rate (minimum number of batches that meet the set minimum coverage rate) of spatial importance.

The number of batches based on resources or the number of batches according to the level of monitoring of the space can be calculated as needed. The camera placement algorithm is shown below.

1. Find the goodness-of-fit map considering the amount and cost of coverage by camera FOV in all spatial maps by grid strategy.

2. Select the grid coordinate with the highest value in the fitness map.

3. Place the camera and update it in the batch list.

4. When the batch quantity is determined to be optimal, terminate the batch.

5. Initialize the goodness-of-fit map around the deployed camera position.

Go back to 6.2.

8 is a view showing an embodiment of a camera arrangement position that satisfies the limit value of the placement cost of the automatic vision sensor arrangement device according to the present invention.

8A shows a camera arrangement when the threshold of camera placement cost is 5 million won, FIG. 8B shows a camera arrangement when the threshold value of camera placement cost is 12 million won, and FIG. 8C shows a camera arrangement when the threshold value of camera placement cost is 15 million won. An embodiment of the camera arrangement is shown.

9 is a view illustrating an embodiment of a camera arrangement position that satisfies the minimum importance coverage ratio of the automatic vision sensor arrangement according to the present invention.

9A is a camera arrangement when the camera minimum importance coverage ratio is 20%, FIG. 9B is a camera arrangement when the camera minimum importance coverage ratio is 60%, and FIG. 9C is a camera arrangement when the camera minimum importance coverage ratio is 80%. An example is shown.

10 is a flowchart illustrating a method for automatically arranging a vision sensor according to the present invention.

The predetermined space is modeled as a 2D grid map (S1010).

Based on the flow amount of the agent using a specific area on the two-dimensional grid map and the flow probability of the agent to move from the starting point to the destination, the importance of each grid space on the two-dimensional grid map is expressed numerically (S1020).

Spatial importance figures of the grid inside the field-of-view (FOV) defined based on the recognition distance (d), available angle (a) and position angle (φ) of the vision sensor at specific coordinates on a two-dimensional grid map Sum, the suitability of the vision sensor is extracted based on the performance and cost of the vision sensor (S1030).

The number of vision sensor arrangements is obtained based on the maximum value of the vision sensor placement cost and the minimum satisfied coverage representing the minimum surveillance area ratio of the predetermined space, and the vision sensor arrangement positions are selected in the order of high vision sensor suitability (S1040).

Although the present invention has been described with reference to one embodiment shown in the accompanying drawings, it is merely an example, and those skilled in the art may realize various modifications and equivalent other embodiments therefrom. I can understand.

1 is a view showing the configuration of the automatic vision sensor arrangement according to the invention.

2 is a view showing the configuration of the layer constituting the spatial modeling unit of the vision sensor arrangement according to the present invention.

3 is a view showing a result of modeling the movement pattern of the agent in the automatic vision sensor arrangement according to the present invention.

4 is a view showing an embodiment of modeling the movement pattern of the agent in the A * algorithm in the automatic automatic vision sensor arrangement according to the present invention.

FIG. 5 is a diagram illustrating an embodiment of modeling a movement pattern of an agent using a path extension algorithm in the automatic vision sensor arrangement according to the present invention.

6 is a view showing an embodiment modeling a field-of-view (FOV) of the vision sensor of the present invention.

7 is a diagram illustrating an embodiment of extracting a goodness of fit of the automatic vision sensor arrangement according to the present invention.

8 is a view showing an embodiment of a camera arrangement position that satisfies the limit value of the placement cost of the automatic vision sensor arrangement device according to the present invention.

9 is a view illustrating an embodiment of a camera arrangement position that satisfies the minimum importance coverage ratio of the automatic vision sensor arrangement according to the present invention.

10 is a flowchart illustrating a method for automatically arranging a vision sensor according to the present invention.

Claims (10)

A space modeling unit for modeling a predetermined space as a 2D grid map; An important space extraction unit that numerically represents each grid space importance on the grid map based on the flow amount of the agent using the first area on the grid map and the flow probability of the agent to move from the first area to the second area; The spatial importance of the internal grid of a field-of-view (FOV) defined on the basis of the recognition distance (d), the available angle (a) and the position angle (φ) of the vision sensor at a specific coordinate on the grid map. A suitability extracting unit configured to extract a suitability of the vision sensor based on a sum and performance and cost of the vision sensor; And Arrangement position selection unit for obtaining the number of vision sensor arrangement based on the maximum value of the vision sensor placement cost and the minimum satisfied coverage representing the minimum surveillance area ratio of the predetermined space, and selects the vision sensor arrangement position in order of high vision sensor suitability Vision sensor automatic placement device comprising; The method of claim 1, The vision sensor automatic placement device, characterized in that the camera. According to claim 1, The spatial modeling unit A wall / space layer that separates a walkable space from the predetermined space and a space that is not; Area including the flow amount of the user using the first area on the 2D Grid map, the flow probability information of the user to move from the first area to the second area, the center coordinates and radius information of the first area and the second area (Area) layer; And And a priority layer representing a spatial importance value of the grid extracted by the critical space extraction unit on the grid map. The method of claim 1, And the movement of the agent from the first area to the second area is modeled through an A * path search algorithm and a path extension algorithm based on the shortest path and the shortest cost. The method of claim 4, wherein The A * path search algorithm is based on the heuristic search (Heuristic Search) and graph search (Graph Search) algorithm, characterized in that the automatic arrangement of the vision sensor. A space modeling step of modeling a predetermined space as a 2D grid map; An important space extraction step of numerically representing each grid space importance on the grid map based on the flow amount of the agent using the first area on the grid map and the flow probability of the agent to move from the first area to the second area; The spatial importance of the internal grid of a field-of-view (FOV) defined on the basis of the recognition distance (d), the available angle (a) and the position angle (φ) of the vision sensor at a specific coordinate on the grid map. A suitability extracting step of extracting a goodness of fit of the vision sensor based on a sum and performance and cost of the vision sensor; And Arrangement position selection step of obtaining the number of vision sensor arrangement based on the maximum value of the vision sensor placement cost and the minimum satisfied coverage representing the minimum surveillance area ratio of the predetermined space, and selecting the vision sensor arrangement position in order of the high degree of vision sensor suitability Vision sensor automatic placement method comprising; The method of claim 6, The vision sensor automatic placement method, characterized in that the camera. The method of claim 6, wherein the spatial modeling step A wall / space layer that separates a walkable space from the predetermined space and a space that is not; Area including the flow amount of the user using the first area on the 2D Grid map, the flow probability information of the user to move from the first area to the second area, the center coordinates and radius information of the first area and the second area (Area) layer; And And a priority layer representing a spatial importance value of the extracted grating on the grating map. The method of claim 6, And the movement of the agent from the first area to the second area is modeled through an A * path search algorithm and a path extension algorithm based on the shortest path and the shortest cost. The method of claim 9, The A * path search algorithm is based on the heuristic search (Heuristic Search) and graph search (Graph Search) algorithm, characterized in that the automatic positioning method of the vision sensor.

Images