KR101018299B1 - Apparatus and method for detecting a plurality of objects in an image - Google Patents

Abstract

The present invention relates to an image processing technology, and more particularly to a technology for detecting a plurality of objects in the image.

In the method for detecting a plurality of objects in the image input to the image processing apparatus according to the present invention, generating a saliency map using the static and dynamic characteristics of the input image, the reference value set in the saliency map Selecting one or more points having the appearance degree as seed points, and for each of the seed points, evaluating the similarity between the seed points and neighboring points in the input image and determining that the seed points are similar to the seed points. Determining the similar area, and determining the similar area as an object area.

Object detection, image processing, appearance map

Description

A method and apparatus for detecting a plurality of objects in an image {APPARATUS AND METHOD FOR DETECTING A PLURALITY OF OBJECTS IN AN IMAGE}

The present invention relates to an image processing technology, and more particularly to a technology for detecting a plurality of objects in the image.

Object detection can be effectively used for object-based recognition systems, image retrieval, content delivery, and image compression. Object detection is applied in the preprocessing stage of these imaging systems.

Among conventional object detection methods, there is a method based on template matching. This method finds an area where an object exists by determining whether there is a match through comparison with a predefined template in an image. It is mainly used for pedestrian detection, face detection and vehicle detection. However, this method can detect only one predefined object. Therefore, its use is limited.

An object of the present invention is to increase the efficiency of image processing by detecting an object region in an image input to an image processing apparatus and performing image processing only on the detected limited region.

In addition, the object of the present invention is to detect a plurality of objects in the input image to increase the speed of the entire image processing, and to reduce the power consumption of the image processing apparatus.

In order to achieve the above object, a method of detecting a plurality of objects in an image input to an image processing apparatus according to the present invention comprises: generating a saliency map using static and dynamic characteristics of the input image. Selecting one or more points having a degree of appearance greater than or equal to a reference value set in the appearance map as a seed point, and for each of the seed points, evaluating the similarity between the seed point and neighboring points in the input image. Determining a similar area composed of points determined to be similar to and determining the similar area as an object area.

Here, the static characteristic of the input image is a static feature map obtained by using the contrast, color, and linearity of each pixel of the input image, and the dynamic characteristic of the input image is an image of two adjacent frames in time of the input image. It is preferable that it is a dynamic feature map obtained by comparison.

Here, the similarity is preferably determined by using the difference between the contrast, the difference in appearance and the distance between the two points.

Here, it is preferable to calculate the similarity by multiplying each of the contrast difference, the appearance difference, and the distance by a predetermined weight.

According to another embodiment of the present invention, an object recognition method for detecting a plurality of objects from an image input to an image processing apparatus and recognizing the type of the object may include detecting the plurality of objects from the input image. Determining regions of the plurality of objects, extracting a feature for each of the regions of the plurality of objects, and comparing the extracted features with a pre-built object feature information database to determine the type of object; do.

The methods according to the invention can be stored on a record carrier in the form of a program and the program can be executed on a computer.

According to another aspect of the present invention, an image processing apparatus for receiving an image and detecting a plurality of objects in the image includes: a motion estimator for calculating a dynamic feature of the input image, and a static image of the input image. A Visual Attention Engine that calculates a feature and generates an outgoing map using the dynamic and static features, an Object Detection Engine that distinguishes an object area from the outgoing map, and the input And a memory in which an image is stored and outputs of the motion determiner, the visual focusing unit, and the object detector are stored, and a RISC processor for controlling operations of the motion determiner, the visual focusing unit, and the object detector.

According to the present invention, a plurality of object regions may be detected from an image input to the image processing apparatus, and image processing may be performed only on the detected limited region, thereby improving image processing efficiency. According to the present invention, a plurality of objects may be detected in an input image to increase the speed of the entire image processing and to reduce power consumption of the image processing apparatus.

The present invention provides a method for accurately distinguishing an area of each of a plurality of objects from a saliency map in an object detection algorithm, and hardware for quickly performing the object detection algorithm.

In the present invention, a map is obtained through an algorithm that mimics the human's visual concentration phenomenon, and the area is centered around the most prominent area, that is, the area where the degree of manifestation is above a certain value. Region growing separates object regions. Furthermore, the present invention provides a method of efficiently obtaining a manifest map and distinguishing a plurality of object regions from the manifest map. In addition, by sensing a plurality of object areas, subsequent image processing is performed on only a limited area, not an entire area of the image, thereby increasing the speed of the entire image processing of the post-processing stage and reducing power consumption of the image processing apparatus. Can be.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings. The following description is given by way of specific examples to aid in understanding the invention, but it should be noted that the invention is not limited to these embodiments. Like elements throughout the drawings are represented by like reference numerals. In addition, in order not to obscure the gist of the present invention, in the case of conventionally known components, a detailed description thereof will be omitted as long as those skilled in the art can easily understand and reproduce the present invention.

1 is a diagram illustrating a process of representing an object region in a grid form from an input image according to an embodiment of the present invention. The object detection algorithm has four steps.

In a first step, a saliency map 20 is generated using the static and dynamic characteristics from the input image 10 (S100). The definition of the static and dynamic characteristics will be described later with reference to FIG. 2. The appearance map 20 refers to a map in which the input image is divided into a plurality of areas and expresses the degree of concentration in each area by human vision. As an example, the manifest map may represent the degree of concentration of each region by the human eye in gray scale. A method of generating an appearance map will be described later with reference to FIG. 2.

In the second step, the points having the degree of appearance more than the reference value set on the appearance map (seed point) is determined (S200). For example, when the appearance map is represented by the gray scale of each region of the image, the closer to white, the higher the degree of appearance can be set, and a point having a degree of appearance above a predetermined value is selected as the seed point. In addition, the number of seed points to select may be set in advance, and the seed points may be selected in the order of high appearance in the input image. Three seed points are shown in the manifest map 20 of FIG. 1. When the seed point in the appearance map 20 corresponds to the corresponding position of the input image (30), the seed point is located at a part of the object.

In the third step, in the input image, the similarity between the seed point and the neighboring point is evaluated to expand a similar area composed of points determined to be similar to the seed point (S300). That is, for each seed point, the similarity of the seed point is similar to that of the seed point by evaluating the similarity of the points around the seed point, starting from the point immediately adjacent to the seed point, and gradually moving away from each other. Evaluate as included in the area. A method of evaluating the similarity between the seed point and the surrounding point will be described later with reference to FIGS. 3 and 4. 1 shows an image 40 in which a similar region is displayed.

In the fourth step, the object region is finally represented by the grid region, and the grid region is determined as the object region (S400). 1 shows an image 50 in which an object region is represented by a grid region.

2 is a view showing in more detail the step (S100) of generating the appearance map. In the present invention, the static and dynamic characteristics of the input image are used to generate the appearance map. The static characteristic means a static feature map obtained by using the contrast, color, and linearity of each pixel of the input image. A dynamic feature map obtained by comparing images of two adjacent frames in time of an input image.

First of all, the process of obtaining a static feature map is performed by first performing pixel operations on the input image. Each pixel is represented by an 8-bit unsigned value and an 8-bit unsigned value. A generated RG contrast value map and a BY contrast map represented by an 8-bit unsigned value are generated (S101).

Thereafter, the intensity map, the R-G contrast map, and the B-Y contrast map are generated in multi-scale (S102).

 Thereafter, a Gabor filter is applied to the multi-scale intensity map to extract the linearity (S103).

Then, the center-surround absolute difference information is obtained for each of the multi-directional linear information, the multi-scale contrast map, the multi-scale RG contrast map, and the multi-scale BY contrast map. It generates (S104).

Thereafter, the center-peripheral absolute value difference information obtained in the previous step is combined and normalized to obtain direction, contrast, and color information of the input image (S105), and the information is added to generate a static feature map. (S106).

Meanwhile, a contrast map represented by an 8-bit unsigned value for two temporally consecutive frames of the input image is generated (S107), and then converted back to multiscale (S108). Thereafter, gradient-based optical flow information of the intensity map of each scale is extracted (S109). Thereafter, visual flow information of all levels is added to generate a dynamic feature map (S110).

Thereafter, after applying the weights set to each of the static feature map and the dynamic feature map, the sum is generated to generate the appearance map (S111).

The above-described method of obtaining the appearance map is an example, and in addition, the appearance map may be generated using any of the known appearance map generation methods.

3 is a diagram schematically illustrating a step (S300) of expanding a similar region similar to the seed point while evaluating the similarity with the surrounding point starting from the seed point. As described above, with respect to the seed point 301, the similarity of the points around the seed point is evaluated with respect to the points that are gradually moving away from the point immediately adjacent to the seed point, and the point having similarity with the seed point or more than the set reference value is determined. It is evaluated to be included in the similar region of the seed point. Therefore, starting from the seed point, the similar region gradually expands and is surrounded by the points determined to be dissimilar, and the enclosed inner region is determined as the similar region.

4 is a diagram illustrating in more detail an algorithm for extending a similar region based on the similarity. As shown in FIG. 4, the similar region extension algorithm may be represented by the following program. Those skilled in the art will be able to clearly understand the region extension algorithm according to the embodiment of the present invention from the following program code.

Pi a seed point

Create Ri initial region from Pi

Insert the 8 neighbors of R into a list L

While L is not empty do

Remove first point x from L

If x satisfies the membership criteria in Ri then

Add x to ri

Update the membership criteria

Insert all neighbors fo x into L

End if

End while

Looking at the region expansion algorithm from the program code above, first, if Pi is a seed point, a pseudo region Ri consisting of the seed points is generated. Then, eight points adjacent to the seed point are put in the list L, and the similarity with the seed point is evaluated for all the points belonging to L until L is empty, and if they are similar, the point is included in the similar area Ri. Each time a point is added to the similarity region, the similarity determination algorithm is further updated to further improve the accuracy of the similarity determination. On the other hand, whenever a point is included in the similar area, eight adjacent points are added to the list L to determine the similarity with the seed point. When the similarity determination object disappears, that is, when compared to the list L, the expansion of the similar region is stopped.

In the present invention, the intensity, saliency, and distance of the seed point and the surrounding point are compared to determine the similarity. To use these three variables, we use a neuro-fuzzy classifier. The neuro fuzzy classifier determines the similarity of each of the three variables (S401), and applies the weights set to the determined similarities (S402) and adds them (S403), and if the summed value, that is, the similarity is greater than the set reference value, each point is determined. It is determined that it is included in the same object region as the seed point (S404).

In the present invention, a neuro fuzzy classifier can be used to learn to operate correctly using sample images, and the weight of each variable can be adjusted to change connection strength and optimize similarity judgment in a neuro fuzzy network structure. Can be. Finally, the boundary of the object is determined by expanding the area until it is determined that all points adjacent to the object area are not included in the object area.

5 is a block diagram of an image processing apparatus 500 for executing a method of detecting a plurality of objects according to an embodiment of the present invention. Motion Estimator (510) for calculating the dynamic features of the input image, Visual Attention Engine for calculating the static features of the input image and generating the appearance map using the dynamic and static features 520), an Object Detection Engine (530) for distinguishing an object region from the appearance map, a memory (540) in which an input image is stored and an output of other components, and a RISC processor for controlling operations of other components 550.

6 is a diagram illustrating which steps of the object recognition method according to the present invention are performed to each component of the image processing apparatus 500 according to the present invention. The motion determiner 510 extracts a dynamic feature from the entrance image. That is, in FIG. 2, steps S107, S108, S109, and S110 are performed by the motion determiner 510. The visual concentrator 520 extracts the static feature from the input image and generates the appearance map using the dynamic feature extracted by the motion determiner 510. That is, the visual concentrator 520 performs steps S101, S102, S103, S104, S105, S106, and S111 in FIG. 2. The object detector 530 uses the presence map to find the seed point and determine the object area. That is, the object detector 530 performs steps S300 and S400 of FIG. 1.

FIG. 7 is a diagram illustrating an object recognition method of detecting a plurality of objects from an image input to an image processing apparatus and recognizing the types of the objects. When the input image 701 is input to the image processing apparatus, the image processing apparatus recognizes the object in the input image 701 using the object recognition method according to the present invention (S710). When the pre-processing process of recognizing the object is completed, post processing is performed on each recognized object (S720). In FIG. 7, objects 1, 2, and 3 are recognized, and features are extracted for each object region (S721). Thereafter, a descriptor of a predetermined format is formed in the object database for each pixel included in the object region (S722). Thereafter, the extracted feature is compared with an existing database to finally determine the type of the object (S723).

Effects of the Invention

 Using the multi-object detection method and the object recognition method according to the present invention, the performance was evaluated by F-measure combining the detection capability and the accuracy of arbitrary objects using the sample objects of the COIL-100. As shown in the present invention, when the variables of contrast, saturation, and distance are used when the area is expanded, the performance is improved by 27% compared to the results using only two variables, the saturation and distance, and 112% compared to the results using only the contrast and the map.

 The object detection in the present invention is the result of the plurality of object detectors implemented as a preprocessing stage of the object recognition system. By classifying the object areas, the image area processed for object recognition is limited to the area where the object is located, and the object detector separates several object areas from the preprocessing step, thereby eliminating additional operations of the post processing step.

Before applying object detector After applying the object detector Frame rate 10.4 frames / sec 21.7 frames / sec Power consumption 800 mW 496mW

 Applying the object detector to the object recognition system, the frame rate is 2.09 times higher and the power consumption of the entire system is reduced by 38%. The recognition rate of the object was confirmed that there is little difference before and after applying the object detector.

The present invention has been described above by way of example. However, the present invention is not limited to these examples. The scope of the present invention should be specified by the claims, and it should be understood that various modifications to the above-described embodiments also belong to the scope of the present invention.

1 is a diagram illustrating a process of representing an object region in a grid form from an input image according to an embodiment of the present invention.

2 is a view showing in more detail the step (S100) of generating the appearance map.

3 is a diagram schematically illustrating a step (S300) of expanding a similar region similar to the seed point while evaluating the similarity with the surrounding point starting from the seed point.

4 is a diagram illustrating in more detail an algorithm for extending a similar region based on the similarity.

5 is a block diagram of an image processing apparatus 500 for executing a method of detecting a plurality of objects according to an embodiment of the present invention.

6 is a diagram illustrating which steps of the object recognition method according to the present invention are performed to each component of the image processing apparatus 500 according to the present invention.

FIG. 7 is a diagram illustrating an object recognition method of detecting a plurality of objects from an image input to an image processing apparatus and recognizing the types of the objects.

Claims (7)

A method of detecting a plurality of objects in an image input to the image processing apparatus, Generating a saliency map using static and dynamic characteristics of the input image; Selecting at least one point having a degree of appearance or more in the appearance map as a seed point; Determining, for each of the seed points, a similar region composed of points determined to be similar to the seed points by evaluating similarities between the seed points and neighboring points in the input image; And Determining the similar area as an object area; Method for detecting a plurality of objects in the input image, comprising. The method of claim 1 The static characteristic of the input image is a static feature map obtained by using the contrast, color, and linearity of each pixel of the input image. The dynamic characteristic of the input image is a dynamic feature map obtained by comparing images of two adjacent frames in time of the input image, detecting a plurality of objects in the input image. The method of claim 1, The similarity is determined by using the difference between the contrast, the difference in appearance, the distance between the two points, the method of detecting a plurality of objects in the input image. The method of claim 3, And calculating the similarity by multiplying each of the contrast difference, the appearance difference, and the distance by a predetermined weight, and adding the calculated weights to each other. An object recognition method for detecting a plurality of objects in the image input to the image processing apparatus and the type of the object, Determining an area of the plurality of objects in the input image by a method according to any one of claims 1 to 4; Extracting features for each of the regions of the plurality of objects; And And comparing the extracted features with a pre-built object feature information database to determine a type of an object. A recording medium storing a computer program for executing a method according to any one of claims 1 to 4 on a computer. An image processing apparatus for receiving an image and detecting a plurality of objects in the image, A motion estimator for calculating a dynamic feature of the input image; A Visual Attention Engine for calculating a static feature of the input image and generating a manifest map using the dynamic feature and the static feature; An object detector to classify an object region from the present map; A memory in which the input image is stored and outputs of the motion determiner, the visual concentrator, and the object detector; And A RISC processor configured to control operations of the motion determiner, the visual concentrator, and the object detector; Image processing apparatus for detecting an object in the input image, including.

Images