JPH0614358B2 - Method and apparatus for extracting moving object image - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image process, and more particularly to a moving process suitable for a process of extracting a moving object image from a moving image continuously input in real time. The present invention relates to a method for extracting an object image.

[Background of the Invention]

A conventional image processing apparatus for moving object images performs processing by using a value obtained by time differential or similar calculation for each pixel of continuously input moving images.
Execution of time differentiation or similar operation by an electronic circuit simplifies the circuit configuration and can be performed at extremely high speed. However, extracting the image of a moving object from only the values obtained by time differentiation or similar operation is
Have difficulty. For example, when a circular area moves in front of a plain background, three-month shaped areas with different signs are output at the front and rear of the traveling direction, but this area can be used to extract the circular area. It is difficult.

As another method, conventionally, a moving object image is obtained by temporarily storing an image obtained from an image sensor in a storage device, then taking out images one by one from the storage device, and calculating corresponding points between the images. Is used. When the corresponding point is found by this method, the moving amount of the corresponding point is accurately calculated, so that the moving object image can be accurately extracted, but the time required for the processing becomes long. Further, when the object is rotated or deformed, it becomes very difficult to detect the corresponding points.

In addition, an image of only the background that does not include the image of the moving object is stored in advance in the device, and the moving object image is extracted by comparing the continuously input image and the background image. It is known how to do it. This method cannot be used at all when the background changes. JP-A-59-80
87 also describes a mechanism for updating the background image.
This is to capture the images at the moment when the image of the moving object is not included in the device one after another and store it as a background image. This is designed so that a gradual change in the background with time is not mistakenly taken as a moving object image. However, in this method, for example, if the moving object is always within the imaging range, the background is not updated at all, and it is not possible to deal with a case where the background changes momentarily. Also, as a related publicly known example, “Information Processing” (Vol. 21, No. 6, Ju
There is ne 1980).

[Object of the Invention]

An object of the present invention is to provide an image processing method capable of accurately extracting a moving object image in real time from a moving image obtained from an image sensor.

[Outline of Invention]

The present invention captures an invariant part of a moving image obtained from an image sensor, or a characteristic component that changes slowly, and stores it as a characteristic amount of a background, and the stored data of the background and an image input momentarily. By comparing the features,
The information about the moving object is extracted.

Example of Invention

 An embodiment of the present invention will be described in detail below.

FIG. 1 shows a block diagram when the present invention is realized by an electronic circuit. The arrows shown in the figure show only the flow of data,
Control lines are omitted. First, the meanings of all the constituent elements will be described, and then the detailed functions thereof will be described.

Reference numeral 101 is an image pickup device for capturing a predetermined image pickup range including a moving object in the background, and 102 is an image storage for storing image data converted into a digital signal by an A / D converter (not shown) for one frame. Is a device, 103 is 10
2 is a mechanism for extracting the features of the image stored in FIG. 2, and in particular, is a mechanism for outputting the edge-enhanced image of the image stored in 102, and 104 is an image memory for storing the edge-enhanced image output from 103 for one frame. A device 105 is an image storage device that stores an edge-enhanced image of a background image for one frame, and a device 106 compares corresponding pixels of two images stored in 104 and 105 to obtain information about an image of a moving object. A mechanism for updating the stored data in 105 at the same time as outputting, and a mechanism for extracting only the image of the moving object from 102 based on the information outputted by 106.

In FIG. 1, 1a is an output from the image pickup device, which is a normal video signal, 1b is a digital signal of an input image, which is data sequentially transferred pixel by pixel to the feature extraction mechanism 103, and 1c is input image data 1b. Of the edge-enhanced image of 1), which is sequentially transferred pixel by pixel. 1d indicates the edge-enhanced image stored in the image storage device 104.
The data 1e is sequentially transferred pixel by pixel, and 1e indicates 1 in relation to the edge enhanced image of the background stored in the image storage device 105.
Data is sequentially transferred pixel by pixel, 1f is data that is sequentially transferred pixel by pixel related to the edge-enhanced image of the background updated by 106, and 1g is information related to a moving object image, and is sequentially transferred pixel by pixel. On-off signal, 1h is data sequentially transferred pixel by pixel for the input image, and 1i is 1 for the moving object image.
It is a digital signal that is sequentially transferred pixel by pixel.

When the image sensor 101 captures a scene as shown in 201, 202, 203 in FIG. 2, processing is performed and each edge-enhanced image is stored in 104. A background edge emphasizing tank 204 is stored in 105. Comparison mechanism 106
However, by comparing 201, 202, 203 and 204 and detecting the difference, 201, 202, 203
At, information about the location occupied by the moving object is output. 107, based on the information,
02, 203 output moving object images 205, 206, 207.

Next, the detailed structure and operation will be described. Image sensor 101
Captures an image and outputs a normal video signal to 102.
The captured image contains a moving object in a stationary background.

The image storage device 102 is a normal storage device that stores a video signal input from 101 through an A / D converter (not shown) into a digital value and stores the digital value for one screen.

The feature extraction mechanism 103 sequentially fetches pixel data from 102, emphasizes edges, and sequentially stores the result in 104. It is generally known in the field of psychology that the contour line plays a very large role when a human recognizes an object, and 103 is an object installed based on this knowledge, and a moving object. It is effective in preventing the contour line from falling off when extracting the image. In the field of image processing, a spatial filter as shown in FIG. 3 (b) has been conventionally used in order to emphasize the contour (the portion where the brightness changes abruptly spatially). Each numerical value in FIG. 3 represents a coefficient of convolution integration. Generally, the brightness of the boundary of the object changes rapidly, and when this filter is applied in the vicinity, the value of the convolution integral becomes a large positive or negative value. In addition, since the brightness inside the object changes slowly, the output of the convolution integral becomes a value close to zero. By performing this convolution operation around all the pixels of the image data stored in 102, the contour line can be emphasized. However, in the case of handling a moving image as handled by the present invention, the spatial filter shown in FIG. 3 (b) has the following problems.

(1) A shift of about 1 pixel often occurs in an image due to a shift of a synchronization signal between frames. Therefore, in the contour image processed in Fig. 3 (b), the position change due to this inter-frame synchronization shift occurs, and it is erroneous as if the contour line of the background image, which is supposed to be still, fluctuates. It will be recognized.

(2) Noise is superimposed on an isolated pixel in the image due to various factors, but the filter in FIG. 3 (b) acts to emphasize this noise.

(3) The outline of the moving object is included in the outline-emphasized image obtained by applying the spatial filter of FIG. 3 (b) to the image data of 102, but information other than the outline is lost.

The present invention is to extract the image of the moving object based on the information obtained from the contour line enhanced image which is the output of the filter, but if the information other than the contour line is lost, the image of the whole moving object is extracted. The purpose of the present invention to extract the (grayscale image) itself is not suitable. If a spatial filter with a large convolution coefficient spread is used as shown in Fig. 3 (c), the effects of image shift and isolated noise can be reduced, but in (3) the internal information of the contour of the moving object is used. Dropout cannot be improved. Therefore, in the present invention, a spatial filter as shown in FIG. 3 (a) is applied as a filter suitable for extracting a moving body image from a moving image. In the filter of FIG. 3 (a), first, as in FIG. 3 (c), the spread of the weighting coefficient is expanded as a countermeasure against the inter-frame synchronization of images and the isolated noise.
Larger than (b). Further, the coefficient of the central portion is increased so that the information other than the contour line, that is, the information about the gray value of the image is not lost. The filter shown in FIG. 3 (a) outputs a value that reflects the grayscale value of a place other than the contour line. The arithmetic unit of the spatial filter is provided with 16 multipliers and executes the product-sum operation of 16 neighborhoods on the image at high speed.

An image storage device 104 in FIG. 1 is a storage device that stores the edge-enhanced image 102, which is output from 103, for one screen.

Similarly, the image storage device 105 in FIG. 1 is a normal storage device that stores an edge-enhanced image for one screen. 104
The edge-enhanced image of the input image composed of the background and the moving object is stored in, and the edge-enhanced image of the image composed of only the background is stored in 105.

Comparison and background edge-enhanced image updating mechanism 106 in FIG.
Is a mechanism for comparing each pixel of 104 with the corresponding pixel of the background image stored in 105 and outputting information about the moving object image, and a function of updating the background image stored in 105 based on the image of 104. Have. 106 sequentially inputs corresponding pixels from 104 and 105, calculates a difference for each pixel, and when the difference is a certain value θ or more, the pixel forms a moving object image at 104, and the difference is If the value θ is less than a certain value θ, it is determined that the pixel forms a background image at 104, and the determination result is output to 107. Also, 106 newly forms the weighted average value for each of the pixels 104 and 105 as the value of 105, thereby forming or updating the edge enhanced image of the background. FIG. 4 shows how the edge-enhanced image of the background is formed. 40
1, 402 and 403 represent edge enhanced images of the input image,
Reference numerals 404, 405, 406, and 407 indicate how an edge-enhanced image of the background is formed. First, it is assumed that the image storage device that stores the background edge enhanced image is cleared. When an input image 401 including a cat and a house is input,
Its image is memorized as a background image a little like 404. Next, when the input image 402 including the bird and the house is input,
In the background image, the image of the house that still exists, such as 405, is highlighted, the image of the bird is slightly remembered, and the image of the cat, which does not exist this time, is weakened. Similarly, when the input image 403 including a person and a house is input, the house image is further emphasized. Finally, as the background edge enhanced image 407
Is formed. A detailed view of FIG. 1 106 is shown in FIG.
The difference calculation 501 is a mechanism for calculating the difference between the pixel data 1d and 1e obtained from 104 and 105 and outputting (5a) to 502. The comparator 502 compares the output 5a from the difference calculator 501 with a preset value θ, and outputs 0 when θ is larger and outputs 1 when θ is equal or smaller. . The multiplier 503 is 10
The pixel data obtained from 5 is multiplied by a preset value (2 ⁿ −1), and the result is output to 504 (5d). The adder 504 adds the pixel data 1d obtained from 104 and the output 5d from the multiplier 503, and outputs the result to 505 (5e). The n-bit shifter 505 shifts the output from the adder 504 to the right by a preset value of n bits, and outputs the result to 105 (1f). The storage device control mechanism 506 includes 104,
This is a mechanism for controlling the address line and the control line so that reading and writing from and to 105 can be performed correctly. As an operation method, first, 506 outputs the address of the pixel to be processed to 104 and 105 through 5g and 5h, and puts 104 and 105 in a read state. The data of the corresponding pixels are input through 1d and 1e, respectively, and after being processed by 501 and 502, 0 or 1 is output through 1g to 107, and at the same time, the pixel data is processed by 503, 504 and 505. It 506 outputs the above address,
At an appropriate time, 105 is put in the writing state.
As a result, 105 can be updated through 1f as shown in the following equation.

Here, BK _new (x, y): pixel data on the updated background edge-enhanced image at coordinates (x, y) BK _old (x, y): background edge enhancement before updating at coordinates (x, y) Pixel data for image I (x, y): Pixel data for edge-enhanced image of input image at coordinates (x, y) n: Pre-set value Also, which pixel the data output through 1g corresponds to Address information indicating whether or not is output through 5f. The above operation is repeated for one screen.

FIG. 1 shows a mechanism for outputting only the pixel data forming the image of the moving object at 102 to the outside based on the information 1g output from 106. Details are shown in FIG. An information selection mechanism 601 is input from 106. This is a mechanism for determining whether to output the input image 1h to the outside or 0 to the outside based on the information 1g on the moving body. 5f includes address information of a pixel being processed and a signal for controlling the image storage mechanism 102.

106 outputs address information regarding the pixel being processed through 5f and puts 102 in a read state. 102
Outputs the data of the designated pixel through 1h.
106 outputs a signal of 1 or 0 through 1g, 601
Outputs pixel data from 102 when 1 is input through 1g and outputs 0 when 0 is input through 1g. Finally, it is possible to output the address information regarding the pixel being processed, that is, the pixel currently output, and the image data including only the moving object image.

FIG. 6 shows a configuration capable of shortening the overall processing time. The arrows indicate only the data flow, and the control lines are omitted.

The analog value relating to the brightness output from 101 is converted into a digital value through an A / D converter (not shown), and then sequentially stored in 102, the data is simultaneously output to 103, and edge enhancement is performed in parallel. Perform processing. Further, the pixel data sequentially output from 103 is directly input to 106, and the pixel data and the corresponding pixel data of 105 are sequentially input to calculate the difference for each pixel, and at the same time calculate the weighted average for each pixel. As a result, 105 can be updated at the same time that the information regarding the moving object image is output.

As described above, it is possible to configure an image processing apparatus that extracts a moving object image that has a shorter processing time than the object shown in FIG.

According to this embodiment, it is possible to configure an image processing device that can extract only a moving object image using an electric circuit.

〔The invention's effect〕

As described above, according to the present invention, since the processing content is simple, it is possible to extract the image of the moving object at high speed without using a large-scale device such as a large-scale computer. It can be used as image processing relating to, or pre-processing such as recognition of a moving object.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention, and FIG.
FIG. 3 is a screen view showing how a moving object image is extracted from an input image, FIG. 3 is a coefficient of a spatial filter used for edge enhancement, and FIG. 4 is a screen view showing how a background image is formed. FIG. 5 is a detailed view of FIG. 1 106, FIG. 7 is a detailed view of 107, and FIG. 6 is a block diagram when the processing speed of the embodiment of FIG. 101 ... Image sensor, 102 ... Input image storage device, 1
03 ... edge enhancement mechanism, 104 ... edge enhanced image storage device, 105 ... background edge enhanced image storage device, 106
... Comparison mechanism and background edge enhanced image updating mechanism, 107
…… Moving object image extraction mechanism.

Claims (5)

[Claims] 1. A moving object image extracting method for extracting a moving object image using a background image stored in advance from image data that is continuously input, wherein a difference between the image data and the background image is obtained, and A moving object image characterized in that the difference from the background image is multiplied by α (α <1) and added to the background image, and the added background image is updated to be a background image corresponding to the next input image data. Extraction method. 2. A method for extracting a moving object image from image data that is continuously input, wherein a feature amount of a background image is stored in advance for the image data, and a difference between the image data and the background image is obtained. While extracting a moving object image from the difference signal, the background image is updated by multiplying the difference signal by α times (α <1) and adding the background signal to the background image for the next input image data. A method for extracting a moving object image, which is characterized in that 3. A moving object image extracting apparatus for extracting a moving object image from continuously input image data by using a background image stored in advance, wherein a characteristic image is extracted from the image data by using a spatial filter. Feature extraction means, a difference calculator that subtracts the background image from the feature image, a comparator that compares a difference value calculated from the difference calculator with a set value, the image data based on the result of the comparator From the information selecting means for selecting the moving object image from, the difference between the image data and the background image is obtained, and the difference between the background image and the background image is multiplied by α (α <1) and added to the background image. A moving object image extracting device comprising: an updating unit that updates a background image after that to be a background image for the next input image data. 4. The updating device according to claim 3, wherein the updating means adds a multiplier for multiplying the background image by (2 ⁿ −1), and an adder for adding the output of the multiplier and the characteristic image. And an n-bit shift register for shifting the output result of the adder to the right by n bits, and the output of the n-bit shift register is used as a background image for the next input image data. . 5. The moving object image extracting device according to claim 3, wherein the spatial filter is a filter shown in the following figure.