JPH09102032A - Detection of object from picture and object detection device using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection method from an image to which a real time processing can be executed in a simple algorithm and to provide an object detection device using the method. SOLUTION: The device is constituted by an image pickup part 100 which image-picks up an original image and a detection processing part 200. The detection processing part 200 is provided with a quantization part 210 which digitally quantizes the lightness of a picture element, a storage part 220, a processing area setting part 230, an extraction part 240 for the picture element in the detection processing area, a classification part 250 making the respective extracted picture elements based on lightness, an arithmetic part 260 digitizing the feature of the distribution state of the picture element in the histogram obtained in the part 250 by using a statistical method and a processing part 270 executing a learning processing on the numeric value obtained by the arithmetic part by a neural network method. The processing part 200 finally judges whether an object exists in the respective detection areas or not.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting an object from an image and an object detecting device using this method.

[0002]

2. Description of the Related Art A conventional method for detecting an object from an image of this type will be described. First, a processing area is set in an image captured by a camera, and it is detected whether or not there is any object in this processing area. There is a method called edge detection processing as the detection method.

This edge detection process is performed by saying that "characteristics such as density, color and texture are uniform in the part corresponding to the object in the image, and the characteristics change rapidly between different objects or parts of the object. It is based on the assumption. That is, the object is detected by extracting the discontinuous portion of the feature.

Here, as a method for extracting the discontinuous portion, a first differential of an image in which all pixels are differentiated and pixels having a certain differential value are connected to obtain an edge, and the first differential is further differentiated to differentiate. Laplacian (second derivative) that finds an edge by searching the amount of change in value, finds the difference between the lightness values of adjacent pixels, and connects pixels that have a certain difference to obtain an edge. Difference processing of lightness values between pixels. , A method that uses a gradient function that calculates the brightness distribution of all pixels on the image and finds the size and direction of the edge inclination, or if it is a color image, the color information (red,
There is a method based on color information for obtaining an edge by comparing (green, blue) with adjacent pixels.

[0005]

However, the above-mentioned plurality of edge detections have the following problems. That is, in each method, the brightness of all the pixels in the processing area must be extracted, and some calculation with adjacent pixels is required, which makes real-time processing difficult. In particular, the calculation between adjacent pixels requires a huge amount of calculation and requires a great deal of processing time.

Further, when the brightness of pixels is not uniformly distributed, particularly when the pixel distributions at the edges of the background and the object on the background have almost no difference, it becomes difficult to detect the edges. .

As described above, the conventional method of detecting an object from an image based on edge detection has a problem that real-time processing is difficult and a problem that the algorithm is complicated.

The present invention was devised in view of the above circumstances, and an object of the present invention is to provide an object detection method from an image which has a simple algorithm and which can be processed in real time, and an object detection apparatus using the same. I am trying.

[0009]

According to a method of detecting an object from an image according to the present invention, the brightness of each pixel in a detection processing area set in the image is converted into a histogram, and a statistical method is applied to the obtained histogram. Is a method of quantifying the characteristics of the distribution state of pixels using, and determining whether or not any object is present in the detection processing area based on the numerical values obtained by the method, and the determination is obtained by a statistical method. The input data is the obtained numerical value, and the actual data of the detection processing area where this input data was obtained is the teacher data,
The output data obtained by executing a predetermined calculation on the input data is compared with the teacher data, and the learning data is processed by the neural network so that the output data matches the teacher data. .

The statistical method uses at least one of weighted average, standard deviation, and skewness.

An object detecting apparatus according to the present invention detects an image pickup section for picking up an original image and whether or not an object exists in a detection processing area arbitrarily set in the original image picked up by the image pickup section. An object detection device having a detection processing unit, wherein the detection processing unit digitizes the brightness of pixels included in a video signal from the imaging unit, and a quantization unit digitized by the quantization unit. A storage unit that stores the brightness of pixels, a processing region setting unit that can arbitrarily set a detection processing region in an original image, and an extraction that extracts pixels in the detection processing region set by this processing region setting unit Section, a classifying section for converting each pixel extracted by this extracting section into a histogram based on the brightness, and an arithmetic section for converting the characteristics of the distribution state of the pixels in the histogram obtained by this classifying section into numerical values using a statistical method. When, When the numerical value obtained by the statistical method is used as the input data and the actual data of the detection processing area where the input data is obtained is used as the teacher data, it is obtained by executing a predetermined calculation on the input data. A processing unit that learns and processes based on the output data so that the output data matches the teacher data.

[0012]

1 is a block diagram of an object detecting apparatus according to the present invention, and FIG. 2 is each pixel when an object does not exist in a detection processing area obtained by a classification unit of the object detecting apparatus according to the present invention. Brightness histogram, FIG. 3 is a brightness histogram of each pixel when an object exists in the detection processing area obtained by the classification unit of the object detection apparatus according to the present invention, and FIG. 4 shows an object in the detection processing area. If not, set the brightness of each pixel to 1
FIG. 5 is a histogram in the case of dividing into 6 classes, and FIG. 5 is a histogram in the case of dividing the brightness of each pixel into 16 classes when an object exists in the detection processing region. Further, FIG. 6 is an explanatory view showing an outline of a neural network used as a classification unit of the object detection device according to the present invention, and FIG. 7 is used for detecting the presence or absence of a car in a parking lot. It is explanatory drawing of experiment, The figure (A) is a schematic plan view,
FIG. 8B is a schematic side view, and FIG. 8 is an explanatory diagram for explaining the detection processing area.

The object detecting apparatus according to the embodiment of the present invention includes an image pickup section 100 for picking up an original image, and this image pickup section 10.
The detection processing unit 200 detects whether or not an object exists in a detection processing area arbitrarily set in the original image captured at 0.

As shown in FIG. 1, the detection processing unit 200 digitizes the brightness of pixels included in the video signal 110 from the image pickup unit 100, and a quantizer 210. The storage unit 220 that stores the brightness of the selected pixel, the processing region setting unit 230 that can arbitrarily set the detection processing region in the original image, and the detection processing region that is set by the processing region setting unit 230 An extraction unit 240 for extracting pixels, a classification unit 250 for converting each pixel extracted by the extraction unit 240 into a histogram based on lightness, and statistics of the distribution state characteristics of pixels in the histogram obtained by the classification unit 250 An arithmetic unit 260 that digitizes using a method, and a numerical value obtained by the statistical method as input data, and the realization of the detection processing area where this input data is obtained. If the data was a teacher data,
And a processing unit 270 that learns and processes based on the output data obtained by executing a predetermined calculation on the input data so as to match the output data.

The image pickup section 100 has a CCD camera capable of picking up an image including all detection processing areas. When the image pickup unit 100 is installed outdoors, a wiper for removing rainwater adhering to the lens of the CCD camera is required, and if all the detection processing areas cannot be stored in one shot. A turning unit for turning the CCD camera is required. However, multiple CCs
If the D camera can be used to cover the entire detection processing area, the revolving unit becomes unnecessary. Further, an illumination unit that illuminates the detection processing area is required when capturing an image at night or when the light tends to be insufficient even in the daytime. The CCD camera has a VTR 120 that records the original image.
Is connected.

On the other hand, the quantizer 210 of the detection processor 200
Includes a video signal 11 from the CCD camera of the image pickup unit 100.
0 is input via the VTR 120. The quantizer 210 digitally quantizes the brightness of all pixels included in the video signal 110. That is, in this object detection device, since an 8-bit A / D conversion element is used, each pixel is digitally quantized into 256 (= 2 ⁸ ) gradations.

All the pixels digitally quantized into 256 gradations are stored in the storage section 220 which is a memory.

On the other hand, the processing area setting unit 230 can arbitrarily set one or more detection processing areas in the original image. That is, in the original image, the position where it is desired to determine whether or not an object exists can be set as the detection processing area. Therefore, after that, the predetermined processing is performed only on the portion set as the detection processing area.

The extraction unit 240 extracts only the brightness of the pixels in the detection processing area set by the processing area setting unit 230, that is, the pixels in the detection processing area.

The brightness of the pixel extracted by the extraction unit 240 is input to the classification unit 250 and converted into a histogram.
Here, it is not preferable to directly handle the brightness of a pixel digitally quantized into 256 gradations in terms of data variation, accuracy, or processing speed. Therefore, the brightness of each pixel will be divided into classes (classes) having a certain width. Here, it is easy to classify the brightness of pixels.
Divided into ranks. It should be noted that if too many classes are used, the frequency of each class will decrease, and the overall visibility will deteriorate.If the number of classes will be reduced, the brightness of pixels with a small frequency will be included in the brightness of pixels with a large frequency. Not preferable.

Here, FIGS. 2 and 3 show histograms of the case where an object (vehicle) exists and the case where it does not exist in a parking frame (corresponding to a detection processing area) in a parking lot described later. It can be seen that the characteristics of the histogram are completely different between when there is no object (FIG. 2) and when there is an object (FIG. 3). The difference is that the brightness of the road surface (asphalt) in the background parking lot is almost the same regardless of the pixel, but the car that is the object to be parked clearly has the same brightness as the road surface. This is because even if the car has the same brightness as the road surface, there are irregularities, so that the brightness of the pixels tends to vary.

However, the histograms shown in FIGS. 2 and 3 are not desirable in terms of processing speed and accuracy due to the large number of data as described above. Therefore, the classifying unit 250 divides the brightness of each pixel into 16 classes to form a histogram, as shown in FIGS. 4 and 5. FIG.
Shows the case where the object (car) does not exist, and FIG. 5 shows the case where the object (car) exists.

The computing unit 260 digitizes the features of the pixel distribution state in the histogram obtained by the classifying unit 250 by using a statistical method. Specifically, three statistical methods of weighted average, standard deviation, and skewness are used. That is, the arithmetic unit 260 can obtain three numerical values for the brightness distribution of one detection processing area.

The weighted average value x _AVE is a place where the histogram (frequency distribution chart) can be balanced when it is likened to a balance, and is expressed by the following equation. However, x ₁ ~x _n is the median ranks, the f ₁ ~f _n is the number of distribution in its class, n represents represents the sample size, i.e. processing a number of pixels, respectively.

The standard deviation s represents the width from the weighted average value x _AVE of the frequency distribution to the inflection point, and is represented by the following equation. If the value of this standard deviation is large, the frequency distribution is spread over a wide range, and if it is small, the frequency distribution is concentrated in a narrow range. The square of the standard deviation s is called variance. Also,
m ₂ is a second-order product moment.

Further, the skewness a ₃ is a value showing how much the frequency distribution is widened as compared with the normal distribution, and the cubic product factor m ₃ is expressed by the following equation. If, the skewness a ₃ is Becomes Here, if the skewness a ₃ is 0, the frequency distribution is bilaterally symmetric with respect to a vertical line passing through the weighted average value, and if positive, the frequency distribution has a long tail on the right side, and if negative, The frequency distribution has a long tail on the left side. Further, the larger the value of the strain degree a _3, the larger the strain.

The characteristic of the shape of the histogram can be expressed as a number by the above-mentioned three numerical values.

These three numerical values are sent to the processing unit 270, and it is determined whether or not an object exists in each detection processing area based on the result of the calculation in the processing unit 270.

The processing section 270 is operated by a so-called neural network. As shown in FIG. 6, this neural network includes three input units 271A to 271C and the input units 271A to 271A to 271A to 271C.
Calculation based on data input from 271C 3
The hidden layers 272 to 274 and the hidden layers 272 to 27
4 and the output unit 275 which outputs the result of the calculation.

This neural network presents a result that is a correct answer to the input data, that is, the teacher data to the output, and changes the coupling coefficient of each hidden layer 272 to 274 so that the error between the input data and the teacher data is reduced. The coupling coefficient is manipulated to minimize it.

Each of the three hidden layers 272 to 274 has, for example, six units 272A to 272F and 273A, respectively.
.About.273F, 274A to 274F. The input data from the input units 271A to 271C are all the units 272A to 272 that form the hidden layer 272 of the first layer.
Input to F, each unit 27 of the hidden layer 272 of the first layer
The outputs from 2A to 272F are input to all the units 273A to 273F of the hidden layer 273 of the second layer. Similarly, the output from each of the units 273A to 273F of the second hidden layer 273 is input to all the units 274A to 274F of the third hidden layer 274. And the hidden layer 274 of the third layer
The output from each of the units 274A to 274F configuring the above is output from the output unit 275 as a determination result, that is, a calculation result.

Preliminary learning is required for the neural network configured as described above. That is, it is necessary to determine the coupling coefficient through learning in advance.

For example, six sets of input data (weighted average value, standard deviation, and skewness) as shown in Table 1 and six sets of teacher data corresponding to them are given to the processing unit 270 in advance so that each hidden layer The coupling coefficients of 272 to 274 are calculated.

[0034]

[Table 1]

The processing section 270 continues the learning until the error of the output data with respect to the teacher data becomes the minimum, and the decrease of the error reaches the saturation state, or the learning is repeated a predetermined number of times. The error in Table 1 is expressed by the formula: error = (teaching data-output data) ² . In Table 1, data No. 3 was an error.

Here, when determining the presence or absence of an object in the detection processing area from the output data, for example, there is no object when the output data is less than 0.500, and there is an object when the output data is 0.500 or more. Is set. Then, new 6 sets of input data as shown in Table 2 are input to the processing unit 270 in which the coupling coefficients of the hidden layers 272 to 274 are determined based on the 6 sets of input data and the teacher data shown in Table 1. Try. Then, the output data shown in Table 2 was obtained. As a result, an error occurred in the case of data NO6.

[0037]

[Table 2]

Here, an actual experiment will be described. The actual experiment was carried out by detecting how many cars are parked in the parking lot in the black circle parking area on the up line of the Meishin Expressway. The imaging unit 100 uses an existing illumination lamp as an illumination unit, and a CCD camera is installed on the pole of the illumination lamp. The relationship between the imaging unit 100 and the parking lot is shown in FIG.
This is as shown in (A) and (B). Here, it is assumed that it is detected whether or not a vehicle as an object is parked in the small vehicle parking frames 1 to 42 (where 11 is a missing number).

The detection processing area is set to a total of 41 areas of the parking frames 1 to 42. However, each detection processing area can be set only to a quadrangle that is parallel to the left side or the upper side of the original image due to the function of the processing area setting unit 230, and therefore the diagonal parking frame should cover the maximum area. I set it so that I could do it. Further, a parking frame that may be hidden by a car parked in front, that is, the parking frames 12 to 2 on the back side.
For No. 7, the detection processing area was set to be small in anticipation of a portion that might be hidden. As shown in FIG. 8, each detection area is represented by the X coordinate and the Y coordinate which are the coordinates of the pixel in the original image. Each detection processing area is set as shown in Table 3 below. However, this Table 3 shows only 1 to 3 and 40 to 42 of the detection processing regions 1 to 42 as an example.

[0040]

[Table 3]

In this experiment, actual data from 8:30 to 19:00 from March 2 to 16, 1995 was used as teacher data. This input data is the VTR1 connected to the CCD camera of the image pickup unit 100.
Learning was performed by applying the video signal 110 every 10 minutes among the video signals 110 recorded in 20 to the detection processing unit 200.

Since this teacher data includes all differences in weather (fine, cloudy, rain) in each time zone, differences in road surface (dry, wet) in all weather conditions in all time zones are included. Has been.

As a result of such an experiment, with respect to the processing unit 270 in which the coupling coefficient of each hidden layer 272 to 274 has been determined by the above-mentioned teaching data, the processing unit 270 has a time from 9:30 to 17:00 on March 7, 1995. Actual input data up to 40 minutes (clear input data), actual input data from 9:20 to 17:20 on March 8, 1995 (input data on cloudy), and March 10, 1995 The actual input data (rain input data) from 9:10 to 17:30 on the day was given. Table 4 shows the result of the calculation performed by the processing unit 270 based on these input data. The presence / absence of an object (car) is determined according to a threshold value indicating that there is no object when the output data is less than 0.500 and that there is an object when the output data is 0.500 or more.

[0044]

[Table 4]

Total detection rate = 100− (number of errors /
Total number of data) x 100 (%), the number of errors is the total number of errors every 10 minutes in each weather, and the total number of data indicates the total number of detection processing areas (that is, parking frames) in each weather. ing. Further, the error count is the sum of the number determined that the car is not parked despite the fact that the car is actually parked and the opposite number.

In this experiment, it took about 1 second to judge 41 detection processing areas per screen. When making this determination, the Pentium (90MH
Z) (registered trademark) was used.

In the above description, the presence or absence of a vehicle in the parking frame of the parking lot is detected, but the present invention is not limited to this. For example, it can be applied to a security system for detecting an intruder, an unmanned monitoring system for abnormal situations, a pattern recognition system for handwritten characters, and the like.

For example, in the case of a security system,
An intruder can be detected by inputting input data at short intervals. Further, in the unmanned monitoring system, it is possible to detect an abnormal situation by taking an image of the scales of the meters with the imaging unit and detecting the abnormal movement of the needle of the meter or detecting the flashing of the lamp. Furthermore, if it is character pattern recognition,
By setting many detection processing areas in close contact with each other, it is possible to detect the line forming the character as being equivalent to the above-mentioned object.

[0049]

According to the method of detecting an object from an image according to the present invention, the brightness of each pixel in the detection processing area set in the image is converted into a histogram, and a pixel is obtained by using a statistical method for the histogram thus obtained. Is a method of quantifying the characteristics of the distribution state of, and determining whether or not there is any object in the detection processing region based on the value obtained by the determination, wherein the determination is the value obtained by a statistical method. As input data, the actual data of the detection processing area where this input data was obtained is used as teacher data, and the output data obtained by performing a predetermined calculation on the input data and the teacher data are compared, A neural network that learns and processes based on the learned data is used so that the output data matches the teacher data.

Compared with the conventional method of detecting an object from an image, in the conventional method of edge detection, a calculation based on the brightness of adjacent pixels is required. Since the brightness is made into a histogram and the characteristics of the distribution state of the pixels in the histogram are digitized, the amount of calculation is much smaller than in the conventional method. Therefore, the object detection method from this image can detect the object at a higher speed, and a system with higher accuracy than the conventional method can be configured at low cost.

Further, the advantage of the neural network, that is, multi-value input is possible, ambiguity can be dealt with flexibly, unknown new data can be dealt with only by learning the representative value. It is possible to enjoy the advantage that the configuration can be changed.

Since the statistical method uses at least one of the weighted average, the standard deviation, and the skewness, more accurate detection can be performed.

On the other hand, the object detecting apparatus according to the present invention determines whether an object exists in an image pickup section for picking up an original image and a detection processing area arbitrarily set in the original image picked up by this image pickup section. An object detection apparatus having a detection processing unit for detecting, wherein the detection processing unit digitizes the brightness of pixels included in a video signal from the image capturing unit into a digital amount, and the quantization unit digitizes the brightness. A storage unit that stores the brightness of the selected pixel, a processing region setting unit that can arbitrarily set the detection processing region in the original image, and a pixel in the detection processing region set by this processing region setting unit An extracting unit, a classifying unit that makes each pixel extracted by this extracting unit into a histogram based on lightness, and a feature of the distribution state of pixels in the histogram obtained by this classifying unit is digitized using a statistical method. Calculation When the numerical value obtained by the statistical method is used as input data and the actual data of the detection processing area where this input data is obtained is used as teacher data, the input data is obtained by executing a predetermined calculation. A processing unit that learns and processes based on the learned output data so that the output data matches the teacher data.

Therefore, the amount of calculation is much smaller than that of the device using the conventional edge detection method. Therefore, in this object detection device, an object can be detected at a higher speed, and a system with higher accuracy than the conventional device can be configured at low cost. Further, since the neural network is used for the processing unit, the advantages of the apparatus and the above-mentioned neural network can be enjoyed.

Further, in this object detecting apparatus, since at least one of weighted average, standard deviation, and skewness is used as the statistical method, it is possible to perform detection with higher accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram of an object detection device according to the present invention.

FIG. 2 is a histogram of the brightness of each pixel when an object does not exist in the detection processing area obtained by the classification unit of the object detection apparatus according to the present invention.

FIG. 3 is a histogram of the brightness of each pixel when an object exists in the detection processing area obtained by the classification unit of the object detection device according to the present invention.

FIG. 4 is a histogram when the brightness of each pixel is divided into 16 classes when an object does not exist in the detection processing area.

FIG. 5 is a histogram when the brightness of each pixel is divided into 16 classes when an object exists in the detection processing area.

FIG. 6 is an explanatory diagram showing an outline of a neural network used as a classification unit of the object detection device according to the present invention.

FIG. 7 is an explanatory diagram of an experiment in which the object detection device according to the present invention is used to detect the presence or absence of a car in a parking lot, and FIG.
Is a schematic plan view, and FIG. 6B is a schematic side view.

FIG. 8 is an explanatory diagram illustrating a detection processing area.

[Explanation of symbols]

 100 Image pickup unit 110 Video signal 200 Detection processing unit 210 Quantization unit 220 Storage unit 230 Processing area setting unit 240 Extraction unit 250 Classification unit 260 Calculation unit 270 Processing unit

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G08G 1/04 H04N 7/18 K 1/14 G01V 9/04 S H04N 7/18 G06F 15/70 465A

Claims (4)

[Claims] 1. A histogram of the brightness of each pixel in a detection processing area set in an image, a statistical method is applied to the histogram thus obtained, and the features of the distribution state of the pixel are digitized. In the object detection method from the image to determine whether or not there is any object in the detection processing area based on the numerical value obtained, the determination, the input value is a numerical value obtained by a statistical method,
The actual data in the detection processing area where this input data was obtained is used as teacher data, and the output data obtained by executing a predetermined operation on the input data is compared with the teacher data. An object detection method from an image, which is performed by a neural network that learns and processes based on the data so as to match the data. 2. The statistical method comprises weighted average, standard deviation,
The method for detecting an object from an image according to claim 1, wherein at least one of the distortion degrees is used. 3. An object having an image pickup section for picking up an original image and a detection processing section for detecting whether or not the object exists in a detection processing area arbitrarily set in the original image picked up by the image pickup section. In the detection device, the detection processing unit includes a quantizer that digitizes the brightness of pixels included in the video signal from the imaging unit, and a storage unit that stores the brightness of pixels digitized by the quantizer. , A processing area setting unit that can arbitrarily set a detection processing area in the original image, an extraction unit that extracts pixels in the detection processing area set by the processing area setting unit, and an extraction unit that extracts the pixels in the detection processing area. A classification unit that histograms each pixel based on the brightness, a calculation unit that digitizes the features of the distribution state of pixels in the histogram obtained by this classification unit using a statistical method, and the statistical method When the input data is a numerical value and the actual data in the detection processing area where the input data is obtained is the teacher data, the output data obtained by executing a predetermined calculation on the input data is the teacher data. An object detection apparatus comprising: a processing unit that learns and processes based on the learning so that they match each other. 4. The statistical method comprises weighted average, standard deviation,
The object detecting device according to claim 3, wherein at least one of the distortion degrees is used.