JP2003168117A - Device and method of image processing, computer program and computer readable storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable discrimination of an object entering an image pickup range of an image pickup means from background with high accuracy while focusing on the object. <P>SOLUTION: An object detection part 104 acquires an image focused on the background from an image pickup part 102 by controlling a focus lens control part 103. In addition, a plurality of background images the focal points of which are gradually shifted in front are also acquired and stored by associating them with positions of the focal points. When the focal point is changed due to a fact that an object is positioned in a photographic visual field, a background image corresponding to a position of the changed focal point is selected. Then, pixels in which difference between the selected background image and an image being photographed exceeds a threshold are determined as pixels of the object. then, this decision result and the image being photographed are outputted to an image encoding part 105 and encoded there. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method and apparatus and a storage medium, and more particularly to an image processing method and apparatus for detecting an object in a moving image, a computer program for executing the method, and a computer program storing the computer program. The present invention relates to a computer-readable storage medium.

[0002]

2. Description of the Related Art As a method of detecting an object in moving image processing, a method called a background subtraction method is known. this is,
The change point is detected by comparing the background image captured in advance with the actual input image. The principle will be briefly described below. The pixel value of the input image of the point at the coordinates (x, y) on the image plane is Pc (x, y), and the pixel value of the background image is Pb (x, y). At this time, Pc
The difference between (x, y) and Pb (x, y) is calculated, and the absolute value is compared with a certain threshold Th.

An example of the judgment formula is as follows. | Pc (x, y) −Pb (x, y) | ≦ Th (1) If the absolute difference value is equal to or smaller than the threshold value Th in the above formula (1),
Since it means that the difference between Pc and Pb is small, Pc is determined to be the background. If the absolute difference value exceeds the threshold Th, it is regarded as a detection target. By making the above judgment at all points on the screen, 1
Detection of frames is completed.

This will be further described with an example of a display image. Figure 1
1 (a) is an example of a background image, and FIG. 11 (b) shows a captured image being monitored. Some point P1b on the background image
And the value of the point P1c on the monitoring screen at the same position is compared, the absolute difference value becomes 0 or a value close to 0, so that the above equation (1) is satisfied, and the position of P1c is determined to be the background. To be done.

On the other hand, when the points P2b and P2c are compared with each other, it is determined that they are not the background because the difference absolute value is large, that is, they are objects.

A pixel determined to be in the object is defined as "1", a pixel determined to be in the background is defined as "0", and the result obtained for all points on the photographed image is shown in FIG.
1 (c). In the figure, the black portion represents the background, and the white portion represents the target object.

[0007]

However, the background subtraction method requires a constraint condition for each of the background scene and the shooting camera. The basic condition of the background scene must not include moving objects.
That is, Pb (x, y) must be constant regardless of time. Actually, even when there is no moving object in the background area, a change in lighting conditions such as the weather causes a change in the pixel value of the background. Also, as the condition on the camera side,
The shooting direction must be fixed with a tripod or the like so as not to shift the position of the corresponding points. Furthermore, changes in camera parameters such as focus, aperture, and white balance also tend to increase the difference between corresponding points. for that reason,
Camera parameters at the time of background shooting and at the time of object detection must be the same.

[0008] The above conditions may be severe constraints that cannot be put to practical use, depending on the actual usage. For example, when considering a surveillance camera application, when the surveillance target area is indoors, it is not so difficult to set the background so as not to include a moving object and to keep the illumination condition constant. Further, fixing the camera position does not become a practical obstacle. However, among the camera parameters, the focus causes a practical problem when fixed. Hereinafter, a detailed description will be given with reference to FIG.

Currently, many cameras have an autofocus function. This is for automatically focusing according to the distance between the camera and the object. When the background image is generated, the focus point is naturally aligned with the background, but if an intruding object is detected during monitoring, the focus point is aligned with the detected object, so the background is out of focus. FIG. 12A shows the situation at this time, in which the person is in focus and the background represents a blurred image. As described above, the actual image is different from that described in FIG. 11B, and even if the absolute value of the difference between the background portions is calculated, a large difference occurs, and it may be erroneously determined not to be the background. is there.

Conversely, it is also possible to calculate the difference by locking the focus point while keeping the background image in focus without using the autofocus function.
At this time, the background portions have almost the same value, and erroneous determination can be prevented. However, since the object is out of focus, another problem occurs, for example, the face of a person is blurred. FIG. 12B shows the image at this time. If the object to be monitored is out of focus, it deviates from the original purpose of monitoring.

The present invention has been made in view of the above problems, and provides an image processing method and apparatus capable of discriminating a background with high accuracy while focusing on an object entering an imaging range, and a method thereof. It is intended to provide a computer program to be executed and a computer-readable storage medium storing the computer program.

[0012]

In order to solve this problem, for example, an image processing apparatus of the present invention has the following configuration. That is, an image processing device for extracting information indicating an object within an imaging field of view of an imaging means having a function of focusing on a target object, wherein the imaging means stores a background image with a plurality of focus points in advance. Means, selecting means for selecting one of the background images stored in the storing means, and detecting a difference between the image obtained by the imaging means and the background image selected by the selecting means. Then, there is provided an extracting means for extracting information indicating a position where the target object other than the background exists within the field of view of the image capturing means.

[0013]

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a block diagram of the image processing apparatus according to the embodiment. In the figure, an imaging unit 102 is a unit that converts an optical image of a subject into an electric signal and outputs it as a video signal of a predetermined format. The focus lens control unit 103 controls the focus lens 101 so that the captured image is in focus (focus lens 101
And the image pickup unit 102 constitutes an image pickup means). There are several types of lens groups required for imaging other than the focus lens, but description thereof is omitted here. The video signal output from the imaging unit becomes an input signal of the object detection unit 104. The object detection unit 104 has functions such as generation and selection of a background image and difference processing. The object detection unit 104 also has a communication function with the focus lens control unit 103, receives the current shooting condition (parameter data indicating the focus in the embodiment), and uses it also for the detection process. The output of the object detection unit 104 is captured image data and binary detection result data (data indicating whether it is a background or an object).
Details of the object detection unit 104 will be described later.

These output data are output to the image coding unit 105.
By encoding with, communication on a line with a narrow band,
Recording on a storage medium with a small capacity is possible. As a method that can obtain high coding efficiency with moving image coding technology,
There is an international standard MPEG-4 visual coding.

FIG. 10 shows a block diagram of this encoding system. Two types of image signals of luminance and color difference and a shape signal representing the shape of an object to be encoded are input, and a compressed bit stream is output. The details of the encoding process are known (for example, "All about MPEG-4", edited by Shinichi Miki 1
998), detailed description thereof is omitted here. By the combination of the detection technology of the present invention and the highly efficient coding technology,
It is possible to realize a monitoring system with high accuracy and high image quality.

Next, details of the processing in the object detection unit 104 will be described below in three embodiments.

<First Embodiment> A first embodiment of the present invention will be described with reference to FIGS. 2 is a block diagram of the object detection processing according to the first embodiment, and FIG.
Is a flowchart of background image generation, FIG. 4 is an example of the generated background image, and FIG. 5 is a flowchart of the background image selection method.

First, the background generation part will be described. In step S301 of FIG. 3, the focus is set to auto,
The focus point of the background image coming from the image pickup unit is set to the background portion. When the focus is achieved, the focus lens is fixed at that position (step S302).
Then, the position (imaging parameter) of the focus lens at this time is recorded in a memory or the like (step S303), and the image at that time is input (step S304). Next, in step S305, a background image for one frame is generated from the input image. The input image at this time may be only one frame, or may be averaged from a plurality of frames to generate one frame. In either case, one focus point and one background image are combined to generate data.

Then, some of these data sets (sets of different focus points and respective background images) are prepared. Therefore, loop processing is performed in step S306. If the number of frames is less than the required number of frames (the number of background images), in step S307, the lock of the autofocus is released, and the focus point is intentionally fixed in the state of being shifted toward you. Returning to step S303,
The focus lens position at this time is recorded, and step S
In 304 and S305, a new background image is generated. In this way, while shifting the focus point little by little, the lens position and the background image data at that time are recorded.
When the required number of frames is reached, a series of processes for background image generation is finished.

FIG. 4 shows an example of the background image generated by the above method. (A) is an image in which the focus point is in the background, (b) is an image in which the focus point is slightly in front of the background, and (c) is
The image in the case where the focus point is set further in front of FIG.

In the block diagram of FIG. 2, the control unit 201
Is a circuit for controlling each block. In addition, it has a communication part with the focus lens control unit 103, and transmits and receives commands relating to focus. Specifically, it is on / off of autofocus, and setting and acquisition of a focus lens position (focus lens position). The input signal is image data output from the imaging unit, and is subjected to format conversion, noise removal processing, and the like as necessary. When the background image is generated, the input data is input to the background image generation unit 202 and stored in a memory (not shown). Also, the position information of the focus lens at this time is
It is recorded (stored) in association with the image data through the control unit 201. The recording medium may be of any type, such as a memory or a hard disk, as long as the information can be stored. It should be noted that the correspondence relationship between the background image and the position information of the focus lens only needs to be clear, and thus the focus lens position may be used as a file name when the background image is stored.

During the background image recording process, the background image selection processing unit 203 and the background difference processing unit 204 stop the output operation by the signal from the control unit 201. Also, when you need to transmit or record a background image,
The switch circuit 205 on the output side is set to the A side for output. Since the output image is one frame of image data, still image coding such as JPEG coding can be used instead of the MPEG-4 coding described above. If only the detected object is to be recorded and transmitted, it is not necessary to send the background image to the image encoding unit.

As described above, when the background image generation unit 202 completes the storage process of the background images corresponding to the plurality of focus lens positions, the object detection process is actually performed.

First, the process of selecting a background image when detecting an object will be described with reference to the flowchart of FIG. First, in step S501,
The focus is set to auto and the change in focus is continuously detected in step S502. Until the object is detected, the focus point remains in the background, so no change is detected. When an object enters the monitoring area, the focus point is set there, so that a change in focus is detected. At this time, a background image having the same focus point as the new focus point detected in step S503 is selected. When the background image of the same focus point is not found, the background image having the closest value is selected and the selected background image is used as the background difference processing unit 20.
Output to 4.

Next, another method of selecting a background image will be described with reference to FIG.
This will be explained using the flowchart of. The first step S511 in which the focus is set to auto is the same as the previous example. Next, in step S512, difference detection is performed between the input image and the background image. The background image used at this time is an image in which the focus point is initially generated. Therefore, the difference value hardly occurs until the object is detected. When an object enters the monitoring area, the object is in focus and a large difference value is detected. At this time, the focus point is detected in step S513, and the background image having the same focus point as this focus point is selected in step S514. If no background image with the same focus point is found, select the background image with the closest value. Since this method detects the change of the focus point by the image processing, it is possible to reduce the communication time required to acquire the focus information, and it is expected to improve the performance of the entire system.

This will be described with an example of a display image. If a plurality of background images described in FIG. 4 are prepared and the image when the object enters is FIG. 12A, the difference between the focus point values at this time is shown in FIG. 4C, for example. It is selected as a background image for comparison. Figure 12 (a)
And the pixel value of FIG. 4C is compared with that of FIG.
Since the relationship is the same as the comparison between (a) and FIG. 11 (b), the result of FIG. 11 (c) can be obtained.

Description will be given using the block diagram of FIG. Since the background image has already been generated, the background image generation unit 20
The output operation of No. 2 is stopped by the control unit 201. The background image selection processing unit 203 obtains focus point information via the control unit 201, and if there is a change,
A background image having the same focus point is selected from the background image generation unit 202 and output to the background difference processing unit 204. If the input image is not used to detect the change in the focus point, it is not necessary to input the monitoring image to the background image selection processing unit 203. This input is
It is necessary in the case described in the flowchart of (b). The background difference processing unit 204 performs comparison and determination processing between the newly selected background image and the input monitoring image, and outputs the result as shape data. At this time, the switch circuit 205 is set to the B side, and the set of image data and shape data is sent to the image encoding unit 105. In the MPEG-4 visual encoding, only the extracted portion of the image data is encoded, so that it is possible to significantly compress the data as compared with the case of encoding the entire image frame.

As described above, according to the first embodiment, the background image having the same focus point as that at the time of object detection is selected from the plurality of background images, and the difference processing is performed. Therefore, it is possible to eliminate the determination error in the background portion and perform highly accurate detection processing in which the entering object is also in focus. In addition, a highly efficient monitoring system can be realized by using the highly accurate detection processing result and encoding the detection target.

<Second Embodiment> A second embodiment of the present invention will be described. In the first embodiment, the information on the position of the focus lens is held as a set with the background image data, but in the second embodiment, when the information on the position of the focus lens cannot be obtained. An example will be described. Hereinafter, a detailed description will be given with reference to FIGS. 4 and 6 to 8. FIG. 6 is a block diagram of the object detection processing in the second embodiment, FIG. 7 is a flowchart of background image generation, and FIG. 8 is a flowchart of a background image selection method.

First, the background generation part will be described. In step S701 of FIG. 7, the focus is set to auto,
The focus point of the background image coming from the image pickup unit is set to the background portion. When the focus is achieved, the frame image is input (step S702). Next, in step S703, a background image for one frame is generated from the input image. The input image at this time is only one frame,
One frame may be generated by averaging a plurality of frames. This completes the generation of the first background image. In step S704, loop processing is performed until the required number of frames (the number of background images) is reached. The second and subsequent background images are subjected to the filtering process in step S705. An image with a shifted focus point can be generated by dropping the frequency component on the high frequency side using a low-pass filter. By gradually narrowing the frequency band passed through the filter, it is possible to generate an image in which the shift of the focus point is gradually increased. In this way, if you can generate the background image of the required number of frames,
The process exits the loop of step S704 and ends the series of processes for generating the background image.

The background image generated by the above method is the same as that explained in FIG. 4, that is, in the first embodiment.
FIG. 4A shows an image in which the focus point is in the background, FIG. 4B is an image in which the focus point is slightly in front of the background, and FIG. 4C is in which the focus point is further in front of B. The image is shown.

In the block diagram of FIG. 6, the control unit 601.
Is a circuit for controlling each block. It also has a communication part with the focus lens control part, and transmits and receives commands relating to focus. Specifically, it is on / off of autofocus.

Unlike the case of FIG. 2, data such as setting and acquisition of the focus lens position cannot be obtained. The input signal is image data output from the imaging unit, and is subjected to format conversion, noise removal processing, and the like as necessary. When generating the background image, the input data is input to the reference background image generating unit 602. Reference background image generation unit 602
Is a focused one-frame background image, and the detection background image generation unit 603 generates a plurality of background images from this reference background image. The generated background images are stored in the detection background image selection processing unit 604. The storage medium may be a memory or a hard disk. At the time of generating the background image, the detection background image selection processing unit 604 and the background difference processing unit 605 stop the output operation by the signal from the control unit 601. When the background image needs to be transmitted or recorded, the output side switch circuit 606 is set to the A side and output. Background image is 1
Since it is frame image data, the MPEG-
It is also possible to use still picture coding such as JPEG coding instead of the coding of 4. If only the detected object is to be recorded and transmitted, it is not necessary to send the background image to the image encoding unit.

Next, with reference to the flow chart of FIG. 8A, a process of selecting a background image when detecting an object in the state where these plural background images are prepared will be described. .

First, in step S801, the focus is set to auto, and the difference is detected in step S802. The background image used at this time is an image in which the focus point is initially generated. Therefore, the difference value hardly occurs until the object is detected. When an object enters the surveillance area, a large difference value is detected because the focus point is placed there. At this time, in step S803, the background image and the monitor image are compared to calculate the degree of coincidence. The method of calculating the degree of coincidence will be described later. In step S805, the next background image is selected, and the process returns to step S803 to calculate the degree of coincidence with the input monitoring image. In the loop processing of step S804, it is determined whether or not the degree of coincidence with all the background images prepared in advance has been calculated,
Upon completion of all calculations, the next step S806
Proceed to. In step S806, one background having the highest matching score is selected from all the background images for which the matching score has been calculated. A series of processes is completed by using this background image as an image for difference processing. This method produces a large difference value on the entire screen in the first frame when entering an object.
Even if the position information of the focus lens cannot be obtained, reliable detection of changes can be expected.

Next, another method of selecting a background image will be described with reference to FIG.
This will be explained using the flowchart of.

First, in step S811, the focus is fixed while being in the background. Next, in step S812, difference detection is performed between the input image and the background image. Since the background image is an image in which the focus points match, a difference value hardly occurs until an object is detected. When an object enters the monitoring area, a large difference value is detected in the part where the object exists. At this time, the focus is set to auto in step S813, and the focus point of the input monitoring image is set to the entering object. In step S814, the background image and the monitor image at this time are compared to calculate the degree of coincidence. Step S8
In step 16, the next background image is selected, and the process returns to step S814 to calculate the degree of coincidence with the input monitoring image. Step S8
In the loop process of 15, it is determined whether or not the degree of coincidence with all the prepared background images has been calculated, and when all the calculations are completed, the process proceeds to the next step S817. In step S817, the background having the highest degree of coincidence is selected from all the background images for which the degree of coincidence has been calculated, and the series of processes is ended. According to this method, since the background image is in focus even in the first frame when the object enters, highly accurate shape data can be output.

This will be described with an example of a display image. The following description is the same as the description given in the first embodiment. If a plurality of background images described in FIG. 4 are prepared and the image when an object enters is FIG. 12A,
Based on the degree of coincidence of the backgrounds at this time, FIG. 4C is selected as the background image for difference comparison. FIG. 12A and FIG.
Since the comparison of the pixel values of (c) has the same relationship as the comparison of FIGS. 11 (a) and 11 (b) described above, FIG.
The result of (c) can be obtained.

The processing up to this point will be described with reference to the block diagram of FIG. Since the background image has already been generated, the output operations of the reference background image generation unit 602 and the detection background image generation unit 603 are stopped by the control unit 601. The detection background image selection processing unit 604 measures the degree of coincidence between the input monitoring image and the plurality of background images, selects the background image with the highest degree of coincidence, and outputs it to the background difference processing unit 605. The background difference processing unit 605 performs comparison and determination processing between the selected background image and the input monitoring image, and outputs the result as shape data. At this time, the switch circuit 606 is B
Then, the set of the image data and the shape data is sent to the image coding unit 105. In the MPEG-4 visual encoding, only the extracted portion of the image data is encoded, so that it is possible to significantly compress the data as compared with the case of encoding the entire image frame.

Next, the method of detecting the degree of coincidence will be described in detail. The easiest way to find a match is to
The sum of differences between images is calculated. The following formula (2) is an example. ki = Σ | Pc (x, y) -Pbi (x, y) | (2) i = 1, ..., N (N is the number of background images) where Pc (x, y) is input. Pbi (x, y) is the pixel value of the monitoring image, and Pbi (x, y) is the pixel value of the i-th background image. ki is the sum of absolute differences between the pixels. The smallest one among k1 to kN is the background image for detection.

Further, when the calculation is performed for all the pixels, the pixels of the entering matter are also included in the image, so that the degree of coincidence between the background portions may not be the highest. At this time, the degree of coincidence detection using a part of the image is also possible. If it is known in advance that a region in which the entering matter is not displayed is known in the monitoring region, the matching degree detection may be performed in the region. FIG. 9A shows this example. Since the hatched area 901 is an area in which an entering matter is not displayed, this area is targeted for coincidence detection. Further, when the area in which the entering matter is not displayed cannot be specified in advance, it is possible to set a plurality of small areas and obtain the relationship from the degree of coincidence.

FIG. 9B shows A1 to A5 on the display screen.
This is an example in which five small areas are set. When the degree of coincidence at each of these five points is obtained, the areas A1, A2, A4, and A5 including only the pixels of the background portion have values close to each other, and the areas A3 including the pixels of the entering matter have greatly different values. Therefore, if the target area for the matching degree determination is other than A3, the matching degree can be detected easily (contributes to the speedup) with high accuracy.

As described above, according to the second embodiment, 1
Multiple background images are generated from one background image by filtering, and when selecting a background image, the background image is selected because the one with the highest degree of coincidence between the monitoring image and the multiple background images is selected, and then the difference processing is performed. It is possible to eliminate a determination error in a portion and perform highly accurate detection processing in which an entering object is in focus. In addition, a highly efficient monitoring system can be realized by using the highly accurate detection processing result and encoding the detection target.

<Third Embodiment> A third embodiment will be described. In the first embodiment, the information on the position of the focus lens is used when the background image is generated and when the background image is selected. In the second embodiment, the background image generation and the background image selection are performed by image processing. It was something to do. However, although the camera parameters cannot be obtained when the background image is selected, it may be possible to control the camera when the background image is generated. At this time, the combination of using the flowchart of FIG. 3 described in the first embodiment for generating the background image and using the flowchart of FIG. 8 described in the second embodiment for selecting the background image is realized. It is possible.

As described above, according to the third embodiment, a plurality of background images are generated while changing the focus point when the background is generated, and when the background image is selected, the degree of coincidence between the monitor image and the plurality of background images is high. After choosing the highest of
Since the difference process is performed, it is possible to eliminate the determination error in the background portion and perform the highly accurate detection process in which the entering object is also in focus. In addition, a highly efficient monitoring system can be realized by using the highly accurate detection processing result and encoding the detection target.

<Other Embodiments> The present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a scanner, a video camera, etc.), but also an apparatus including one device. May be applied to.

Further, an object of the present invention is to supply a storage medium (or recording medium) recording a program code of software for realizing the functions of the above-described embodiments to a system or apparatus, and to supply a computer of the system or apparatus ( Alternatively, by the CPU or MPU) reading and executing the program code stored in the storage medium,
It goes without saying that it will be achieved. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.
Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instruction of the program code,
An operating system (OS) running on the computer does some or all of the actual processing,
It goes without saying that the processing includes the case where the functions of the above-described embodiments are realized. Here, as the storage medium for storing the program code, for example, a floppy (registered trademark) disk, a hard disk, a ROM,
RAM, magnetic tape, non-volatile memory card, CD-
ROM, CD-R, DVD, optical disk, magneto-optical disk, MO, etc. are considered.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, based on the instruction of the program code. , The CPU provided in the function expansion card or the function expansion unit performs some or all of the actual processing,
It goes without saying that the processing includes the case where the functions of the above-described embodiments are realized.

When the present invention is applied to the above storage medium, the storage medium stores the program code corresponding to the flowcharts shown in FIGS. 3, 5, 7, and 8 described above.

As described above, according to the present invention, a plurality of background images are generated in advance, the one having the highest degree of coincidence in the background portion between the input monitor image and the plurality of background images is selected at the time of monitoring, and then the difference is calculated. Since the processing is performed, it is possible to eliminate the determination error in the background portion, and to present the highly accurate image processing method and device and the storage medium with the accurate focus point even for the entering object.

[0052]

As described above, according to the present invention, it is possible to discriminate from the background with high accuracy while focusing on an object that enters the image pickup range of the image pickup means.

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an image processing apparatus according to an embodiment.

FIG. 2 is a block configuration diagram of an object detection unit in the embodiment.

FIG. 3 is a flowchart showing a background image generation processing procedure according to the embodiment.

FIG. 4 is a diagram showing a plurality of background images in the embodiment.

FIG. 5 is a flowchart showing a background image selection processing procedure according to the first embodiment.

FIG. 6 is a block configuration diagram of an object detection unit in the second embodiment.

FIG. 7 is a flowchart illustrating a background image generation processing procedure according to the second embodiment.

FIG. 8 is a flowchart showing a background image selection processing procedure according to the second embodiment.

FIG. 9 is a diagram for explaining a background image selection process in the second and third embodiments.

FIG. 10 is a block diagram of MPEG-4 visual encoding.

FIG. 11 is a diagram illustrating object detection by background subtraction processing.

FIG. 12 is a diagram illustrating a problem of object detection by background subtraction processing.

Claims (11)

[Claims] 1. An image processing apparatus for extracting information indicating an object within an imaging field of view of an imaging means having a function of focusing on an object, wherein the imaging means preliminarily displays a background image with a plurality of focus points. Storing means for storing; selecting means for selecting one of the background images stored in the storing means; and a difference between the image obtained by the image pickup means and the background image selected by the selecting means. An image processing apparatus comprising: an detecting unit that detects and detects information indicating a position where a target object other than the background exists in the photographing field of view of the image capturing unit. 2. The image processing apparatus according to claim 1, wherein each focus point and a background image are stored in the storage unit in association with each other. 3. The selecting means includes input means for inputting information about a focus point of the image pickup means,
The image processing apparatus according to claim 2, wherein a background image having a corresponding focus point is selected based on the information input by the input unit. 4. The selection means, in an initial state, comprises a first selection means for selecting a background image focused on a background image, an image obtained by the imaging means, and a background image previously selected. When the difference becomes large, a calculating unit that sequentially calculates the difference from the other background images stored in the storing unit, and a second selection that selects the background image having the smallest difference calculated by the calculating unit. A first means comprising:
The image processing device according to item. 5. The image processing apparatus according to claim 4, wherein the calculation unit calculates a difference between a plurality of predetermined partial areas set in advance in the field of view of the image pickup unit. 6. The plurality of background images are generated by subjecting a background image obtained by focusing to a reference background image and performing a filtering process on the reference background image while gradually changing filter coefficients. Claim 4
The image processing device according to item. 7. The extraction means extracts binary information for identifying, for each pixel, whether the pixel is a background pixel or a target object other than the background. The image processing apparatus according to any one of items. 8. The apparatus according to claim 1, further comprising means for outputting the image information obtained by the image pickup means and the information extracted by the extraction means to an MPEG-4 encoding means.
The image processing apparatus according to any one of items 1 to 7. 9. An image processing method for extracting information indicating an object within an imaging field of view of an imaging means having a function of focusing on a target object, wherein the imaging means preliminarily creates a background image with a plurality of focus points. A storing step of storing in a predetermined storage medium, a selecting step of selecting one of the background images stored in the storing step, an image obtained by the image capturing means, and a background selected in the selecting step. An image processing method, comprising: detecting a difference from an image, and extracting information indicating a position where a target object other than the background exists in the photographing field of view of the image pickup unit. 10. A computer program functioning as an image processing device for extracting information indicating an object within an imaging field of view of an imaging means having a function of focusing on a target object, wherein the imaging means pre-selects a plurality of focus points. Storage means for storing a background image based on points, selection means for selecting one of the background images stored in the storage means, image obtained by the image pickup means, and background selected by the selection means A computer program that functions as an extracting unit that detects information indicating the position of a target object other than the background in the photographing field of the image pickup unit by detecting a difference from the image. 11. A computer-readable storage medium storing the computer program according to claim 10.