JPH09186931A - Image input device and image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract an object at a high speed and to control an image pickup condition required to make the quality of an extracted image high. SOLUTION: This image input processing unit is composed of an image pickup means D to pick up an image for object extraction, an image pickup mode setting means A to set the image pickup mode for the image pickup means D to extract an object, an image pickup parameter measurement control means B to conduct measurement control of the image pickup parameter based on the image pickup mode set by the image pickup mode setting means A, a recording means C to record the image pickup parameter and an image. At the time of image pickup by the image pickup means D, the optimum image pickup parameter is set or controlled and the image pickup condition is controlled to optimize the image quality of the extracted image so as to extract an object image at a high speed and to improve the image quality.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image input device such as a camera and an image processing device, and is particularly suitable for use in an image pickup system having an image cutting function.

[0002]

2. Description of the Related Art Conventionally, in image pickup means, as the degree of freedom in processing image information and the degree of freedom in processing increase with the digitization of signal processing, internal processing is performed for brightness level, color tone conversion, and white balance. Processing, relatively easy processing such as quantization size conversion, edge extraction function and image extraction function using sequential growth method of color components (Technical direction of Television Society, vol.18.pp.13-18.1994) Have been realized.

Further, in a method of extracting an image based on a difference from a background image, Japanese Patent Laid-Open No. 6-253197 discloses
The aperture control means is set so that the average brightness is appropriate when the background image is captured, and the current image is captured using the same set value as the background image. Then, the object image is extracted based on the difference data of those images.

[0004]

However, in the above-mentioned conventional example, the command communication control means for controlling the image pickup mode from the outside when the subject is extracted in the image pickup means, or the subject extraction image in the image pickup means. Since the control function of the imaging parameter necessary for imaging is not provided, there is a problem that the optimal imaging condition for image extraction cannot be set.

Therefore, in the case of the conventional image pickup means,
For the purpose of image extraction, it was not possible to optimally set the imaging conditions according to the imaging situation, such as changes in illumination conditions, presence / absence of subject movement, and presence / absence of movement of the imaging means itself.

In view of the above-mentioned problems, the first aspect of the present invention is to make it possible to perform subject extraction at high speed, and to control the imaging conditions necessary for improving the image quality of an extracted image. The purpose of.

A second object of the present invention is to allow the characteristics of the image pickup operation for subject extraction to be externally controlled and to allow the extracted image to be transferred to a remote location.

A third object of the present invention is to make it possible to externally control imaging parameters for subject extraction and to transfer the subject image to be extracted to a remote location.

A fourth object of the present invention is to enable automatic setting of imaging conditions and control of parameters for subject extraction.

A fifth object of the present invention is to improve the operability of the image input device when extracting a subject.

A sixth object of the present invention is to enable pre-processing of an image necessary for accurately extracting a subject to be performed inside the image input device, and from the imaging operation to the output of the extracted image. Is to be able to do it at high speed.

A seventh object of the present invention is to enable stable image cutting of a specific subject from an arbitrary background regardless of illumination conditions.

An eighth object of the present invention is to perform subject extraction with high accuracy without being affected by illumination conditions, and to control the exposure amount for improving the image quality at the time of extraction and image cutting based on it. Is to be able to do.

A ninth object of the present invention is to make it possible to selectively control appropriate imaging parameters in order to improve the image quality of a clipped image and speed up the processing in accordance with the illumination condition at the time of imaging and the subject distance. That is.

A tenth object of the present invention is to enable stable clipping of an image at high speed in an image pickup operation under a plurality of image pickup conditions.

An eleventh object of the present invention is to make it possible to automatically select an appropriate image compression depending on whether the clipped image is a still image or a moving image.

A twelfth object of the present invention is to perform stable image clipping regardless of whether or not the subject is a moving body, or whether or not there is relative motion between the imaging means and the subject. Is to be able to do it.

A thirteenth object of the present invention is to make it possible to appropriately perform an image pickup operation for subject extraction without performing control via communication means from an external terminal or the like or complicated manual operation. is there.

[0019]

An image input device of the present invention comprises an image pickup means for picking up an image for extracting a subject, and an image pickup mode setting means for setting an image pickup mode for the subject to pick up an object. An image pickup parameter measurement control unit for performing measurement control of the image pickup parameter based on the image pickup mode set by the image pickup mode setting unit, and a recording unit for recording the image pickup parameter and the image. .

Another feature of the present invention is that
Imaging means for performing imaging for subject extraction, imaging parameter measurement control means for performing measurement control of imaging parameters based on a preset imaging mode, and recording means for recording the imaging parameters and images A control for performing communication regarding a cutout image processing unit that performs a predetermined process on an image generated by the image pickup unit under a plurality of image pickup conditions to cut out the image, and a control command for controlling the image pickup operation of the image pickup unit It is characterized by comprising a command communication means and an image transfer means for transferring the image data generated by the image pickup means to the outside.

Another feature of the present invention is that the image pickup parameter measurement control means controls the image pickup parameter based on a control command given from the control command communication means.

Another feature of the present invention is that the image capturing mode setting means has an image capturing mode setting button, and the image capturing mode setting operation is started by operating the image capturing mode setting button. The image pickup parameter measurement control means sets the control amount of the image pickup parameter based on the image pickup parameter measurement value.

Another feature of the present invention is that the image pickup mode setting means automatically sets the image pickup mode to capture the necessary image and then automatically returns to the standard image pickup mode.

Another feature of the present invention is that the cut-out image processing means includes comparison data generating means for generating comparison data between a plurality of images having different imaging parameters, and the comparison data generating means. The cutout image is processed based on the comparison data generated by the control command and the control command given from the control command communication means.

Another feature of the present invention is that the image pickup parameter measurement control means controls the lens arrangement of the image forming optical system based on a signal from the image pickup mode setting means. A focusing degree detecting means for detecting a focusing degree, and a capturing signal generating means for generating a signal for capturing a plurality of images having different focusing degrees based on an output from the focusing degree detecting means are provided. There is.

Another feature of the present invention is that the image pickup parameter measurement control means, based on the signal from the image pickup mode setting means, the shutter speed, the aperture diameter of the optical diaphragm, or the characteristic value of the image sensor. The present invention is characterized by comprising a capture signal generating means for controlling exposure conditions by controlling the above, and generating a signal for capturing a plurality of images having different exposure conditions.

Another feature of the present invention is that the image capturing mode setting means can set the priority of the image capturing parameters.

Another feature of the present invention is that the image recording means stores a video signal which is not subjected to signal processing, and has a plurality of high-speed image recording means and image transfer means. It has a feature.

Another feature of the present invention is that the image recording means has image compression encoding means for moving images or still images, and in the imaging mode for subject extraction, the cut-out image is obtained. The compression method is variable according to the type.

Another feature of the present invention is that the cut-out image processing means uses an image cut-out processing unit for cutting out an image from a time-series image and a cut-out image obtained by the image cut-out processing unit as initial data. And an image tracking means for tracking time-series images.

Another feature of the present invention is that the image pickup parameter measurement control means is operated by a basic control program written in a storage means built in the main body of the image input apparatus or a removable recording medium. It is characterized by doing.

[0032]

Since the present invention comprises the above technical means, according to the first invention of the present application, when the image pickup means performs the image pickup for extracting the specific subject image, the optimum image pickup parameter is set or controlled. It is also possible to optimize the image quality of the extracted image by controlling the imaging conditions.

According to the second invention of the present application, since it is possible to externally control the feature amount of the image which is a trigger for inputting the image pickup start command, mode locking, and image capturing, it is possible to remotely control the specific subject. Images can be extracted, and the extracted images can be transferred to a remote location.

According to the third invention of the present application, it becomes possible to control the imaging parameter by supplying a control command from the outside through the control command communication means.

According to the fourth invention of the present application, it becomes possible to automatically set the image pickup mode for subject extraction without manually setting the image pickup parameter.

According to the fifth invention of the present application, an operation for returning to the normal image pickup mode after the image pickup operation (image taking operation) for extracting a specific object from a plurality of images having different image pickup conditions is unnecessary. , The operability can be improved.

According to the sixth aspect of the present application, since the cutout image is processed based on the comparison data generated by the comparison data generating means and the control command given from the control command communication means, the image pickup condition is set. The initial data required for controlling and cutting out the specific subject image can be calculated at high speed.

According to the seventh invention of the present application, a specific subject image can be extracted at high speed from a plurality of images whose resolution (focus) is appropriately controlled, and is affected by changes in the environment and lighting conditions. It is possible to perform stable subject extraction with little occurrence.

According to the eighth invention of the present application, the exposure condition in the entire image frame or in the specific region is appropriately controlled so that the specific subject image can be extracted at high speed and stably. Become.

According to the ninth invention of the present application, since the priority order of the imaging parameters can be set, it is possible to specify, for example, focus priority control or exposure amount priority control, and an appropriate imaging according to the imaging situation. Conditional control can be set.

According to the tenth invention of the present application, since the image recording means stores the video signal which is not subjected to the signal processing, it is stable because it is less affected by the change of the imaging environment condition and the motion of the subject. It is possible to cut out the image.

According to the eleventh invention of the present application, the image compression encoding means for moving images or still images is provided, and in the image pickup mode for subject extraction, the image compression encoding means is selected according to the type of the cut out image. Since the compression method is variable, it is possible to cut out an image in a still image or a moving image, and to select an appropriate compression encoding method according to whether the extraction mode is a still image or a moving image.

According to the twelfth invention of the present application, it is possible to stably cut out a specific subject in a moving image.

According to the thirteenth invention of the present application, it becomes possible to appropriately perform the basic image pickup operation required for subject extraction without online operation from an external terminal.

[0045]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of an image input device and an image processing device of the present invention will be described below with reference to the drawings. FIG.
FIG. 3 is a functional configuration diagram showing a main configuration of an image input device of the present invention. In FIG. 1, A is an imaging mode setting unit, B is an imaging parameter measurement control unit, C is a recording unit, D is an imaging unit, E is a cutout image processing unit, F is a control command communication unit, and G is an image transfer unit. .

The image pickup mode setting means A sets an image pickup mode for the image pickup means D to extract a subject. The image capturing mode setting means A has an image capturing mode setting button (not shown), and when the image capturing mode setting button is operated, the image capturing mode setting operation is started. Further, the image capturing mode setting means A is adapted to automatically return to the standard image capturing mode after setting the image capturing mode and capturing a necessary image.

The image pickup parameter measurement control means B controls the measurement of image pickup parameters based on the image pickup mode set by the image pickup mode setting means A. In addition, the measurement control of the imaging parameter may be performed based on the imaging mode set in advance. Further, the imaging parameter may be controlled based on the control command given from the control command communication means F. Further, it is possible to set the control amount of the imaging parameter based on the imaging parameter measurement value and set the priority order of the imaging parameter.

The image pickup parameter measurement control means B is operated by a basic control program written in a storage means built in the image input apparatus main body or a removable recording medium. Further, the image pickup parameter measurement control means B is a lens arrangement control means for controlling the lens arrangement of the imaging optical system based on a signal from the image pickup mode setting means A, and a focus degree detection for detecting a focus degree. And means for generating a signal for capturing a plurality of images having different degrees of focus on the basis of the output from the degree-of-focus detection means (none of which is shown).

The capture signal generating means controls the exposure condition by controlling the shutter speed, the aperture diameter of the optical diaphragm, or the characteristic value of the image sensor based on the signal from the image pickup mode setting means A. A signal for capturing a plurality of images with different exposure conditions is generated.

The recording means C is provided for recording the image pickup parameters and the image.
The image recording means C stores a video signal that is not subjected to signal processing, and has a plurality of high-speed image recording means C and image transfer means G. The image recording means C has an image compression coding means (not shown) for moving images or still images. In the image pickup mode for subject extraction, the compression method is set according to the type of the cut out image. It can be changed.

The image pickup means D is provided for picking up an image for extracting a subject.

The cut-out image processing means E cuts out an image by subjecting the image obtained by the image pickup means D under a plurality of image pickup conditions to a predetermined process. Further, a comparison data generating means (not shown) for generating comparison data between a plurality of images having different imaging parameters is provided, and the comparison data generated by the comparison data generating means and the control command communication means F are provided. The cut-out image is processed based on the control command given from.

Further, the cut-out image processing means E is
An image cropping processing unit (not shown) that crops an image from the time series image, and an image tracking unit (not shown) that tracks the time series image using the cropped image obtained by the image cropping processing unit as initial data. Have

The control command communication means F communicates with respect to a control command for controlling the image pickup operation of the image pickup means D. The image transfer means G is for transferring the image data generated by the image pickup means D to the outside.

According to the image input device configured as described above, when the image pickup means D performs image pickup for extracting the specific subject image, it is possible to set or control optimum image pickup parameters. Moreover, since the image quality of the extracted image can be optimized by controlling the imaging conditions, the subject image can be extracted at high speed and with high image quality.

According to another feature of the present invention, since the feature amount of the image can be controlled from the outside, the image pickup operation for subject extraction and the image processing performed inside the image input device can be remotely controlled. And the extracted images can be transferred online to a remote location.

Further, according to the other features of the present invention, it is possible to externally supply the control command via the control command communication means F to control the imaging parameter, and therefore the imaging parameter for subject extraction can be externally controlled. It is possible to control, and it is possible to realize the high quality of the extracted image and the high speed of the extraction process.

According to another feature of the present invention, the image pickup mode setting means A is provided with an image pickup mode setting button,
When the imaging mode setting button is operated, the imaging mode setting operation is started, and the control amount of the imaging parameter is set based on the imaging parameter measurement value. Thereby, the imaging parameter in the subject extraction mode can be automatically set, and it is not necessary to manually set the control amount of the imaging parameter.

Further, according to the other features of the present invention, the operation for returning to the normal image pickup mode after the image pickup operation (image capturing operation) for extracting the specific subject from the plurality of images having different image pickup conditions is unnecessary. It is possible to improve the operability by eliminating the need to set the return from the imaging in the subject extraction mode to the normal imaging mode.

Further, according to the other features of the present invention, the cutout image is processed based on the comparison data generated by the comparison data generating means and the control command given from the control command communication means F. It is possible to perform the preprocessing necessary for cutting out the specific subject image by controlling the imaging condition at high speed in the image input device.

According to the other features of the present invention, the lens arrangement control means for controlling the lens arrangement of the imaging optical system based on the signal from the image pickup mode setting means A, and the combination for detecting the focus degree. Since the focus degree detecting means and the capture signal generating means for generating a signal for capturing a plurality of images having different focus degrees based on the output from the focus degree detecting means are provided, it can be obtained by resolution (focus) control. A specific subject image can be extracted from a plurality of images at high speed, and stable image extraction can be performed regardless of illumination conditions.

According to another feature of the present invention, the exposure condition is controlled by controlling the shutter speed, the aperture diameter of the optical diaphragm, or the characteristic value of the image sensor based on the signal from the image pickup mode setting means A. Since a capture signal generating means (not shown) for generating a signal for capturing a plurality of images having different exposure conditions is provided, the exposure conditions in the entire image frame or a specific area are appropriately controlled to achieve high speed and stability. It is possible to extract the specific subject image.

Further, according to the other features of the present invention, since the priority order of the imaging parameters can be set, it is possible to specify, for example, focus priority control or exposure amount priority control, which is suitable for the photographing situation. It is possible to improve the image quality of the cutout image by setting various image pickup condition controls and speed up the process.

Further, according to the other features of the present invention, since the video signal which is not subjected to the signal processing is stored in the image recording means C, it is less likely to be affected by the change of the imaging environment condition and the motion of the subject. Stable image clipping can be performed, and high-speed imaging operation can be performed under the control of imaging conditions. Due to this, changes in imaging environment conditions,
It is possible to improve the accuracy of the cut-out image and stabilize the image quality without being affected by the subject motion.

Further, according to the other features of the present invention, it is possible to cut out an image in a still image or a moving image, and automatically select an appropriate compression encoding method depending on whether the extraction mode is the still image or the moving image. You can choose.

Further, according to the other features of the present invention, stable image cutting can be performed regardless of the presence or absence of the motion of the subject and the presence or absence of camera work (panning, scanning, etc.). Selective clipping can be performed.

According to another aspect of the present invention, the basic control program written in the storage means built in the image input apparatus main body or the removable recording medium,
Since the imaging parameter measurement control means B is operated, it is not necessary to perform an online operation from an external terminal or a complicated manual operation. Also,
It is possible to incorporate a control program that can appropriately perform the basic imaging operation necessary for subject extraction, or to supply the control program by a removable memory.

Next, a more specific structure and operation of the image input apparatus of the present invention will be described in detail based on the embodiments. First, the first embodiment will be described. In this embodiment, the focus state is controlled to perform the specific subject extraction processing from images under different imaging conditions stably and at high speed regardless of the illumination conditions.

FIG. 2 is a block diagram of the essential parts of an image processing camera having an image pickup mode for extracting a specific subject and an extraction processing mechanism. The image pickup mode for subject extraction refers to an image pickup operation mode in which image pickup parameters (lens arrangement, shutter speed, aperture size, etc.) are controlled and a plurality of image data are output from an image sensor to an image recording (storing) means. It represents.

In FIG. 2, reference numeral 1 is the whole image pickup means (camera header) which is an image input means, 2 is an image forming optical system, 3 is a lens motor drive control section, a focusing lens drive means (focus motor), and a zoom. It is composed of a lens driving means and a lens information reading (measuring) means such as the type of mounted lens.

Reference numeral 4 is an image sensor, typically a solid-state image pickup device such as a CCD, 5 is an image pickup parameter measurement control means, a zoom detection control means 51, a focus signal (focusing state).
The detection control unit 52 (focusing state detection unit 52a, focus control unit 52b), shutter speed detection control unit 53, aperture measurement control unit 54, and the like are provided.

The shutter speed control means 53 controls a mechanical shutter (not shown). When an electronic shutter is used, the sensor drive circuit 18 described later controls a substantial shutter speed (charge accumulation time). Do it. The detection of the in-focus state may be either a detection method using a video signal or a distance measurement method using infrared rays.

An image recording / reproducing means 6 digitally records in a predetermined format on a recording medium (tape medium, optical (magnetic) disk medium, IC memory medium such as IC memory card) at the time of image pickup.

The recording medium or image recording / reproducing means 6
May be detachable from the camera body. In addition, an imaging operation procedure program in the subject extraction mode, necessary data, and the like may be written in the recording medium, and the image may be controlled based on the information.

During image reproduction, the viewfinder (EV
Image data is output to the data transfer means 12 described later or the like. 8 is an imaging parameter recording means,
Information at the time of image pickup including lens information from the lens motor driving means 3, image pickup parameters, image signal characteristic parameters and presence / absence of stroboscopic light emission, intentional camera operation (motion) such as scanning, presence / absence of camera shake, etc. is recorded.

Especially, camera operation (scan, pan, etc.)
May be detected by video signal processing, or may be determined from the output data by incorporating an unillustrated acceleration sensor, a gyro or the like in the image pickup means.

Reference numeral 9 denotes a switch as a subject extraction mode setting means for subject extraction. When this switch is set, an image pickup start operation signal from a shutter button (not shown) or the like following the turning on of the switch causes an image pickup for subject extraction. The image is captured according to the mode.

The image pickup start switch may be configured to also serve as the subject extraction mode setting switch. 10
Is a command data communication control means from the outside, 11 is an output image generation means, and generates an analog / digital image signal according to an output destination.

For example, the TV receiver generates an NTSC signal or a PAL signal. Further, the image signal which has been subjected to the gradation correction of light and shade according to the output device, the error diffusion processing, and the color component correction processing may be output to a display, a printer or the like.

It is assumed that such an output signal generation and processing instruction is given externally via the command / data communication control means 10 or by using a switch (not shown) on the operation panel of the camera.

Reference numeral 12 denotes a data transfer means, which includes a communication interface means (not shown), an adapter terminal, etc.
Used when outputting image data to an external display device or computer. Reference numeral 13 denotes an image processing unit for subject cutout, which cuts out an image obtained by processing an image obtained under a plurality of imaging conditions by a method described later.

Reference numeral 14 is an image capturing signal generating means,
In order to properly capture an image in consideration of the exposure condition, the control state of the image capturing condition, the charge accumulation / transfer time in the image sensor (CCD), the timing of the image writing control means in the memory, and the like. It is a thing.

Reference numeral 15 denotes a system control section as a means for setting a plurality of image pickup conditions, and the command data communication control means 1
0, the signal from the subject extraction mode setting switch 9 or the information of the control program written in the recording medium of the image recording / reproducing means 6 is read to generate a series of imaging operation control signals in the subject extraction mode.

Reference numeral 16 is a diaphragm, and 17 is a video signal processing section, which includes a preprocessing circuit (17a: sample and hold circuit, auto gain control circuit, A / D conversion circuit, etc.), gamma correction control circuit 17b, and other video signals. The processing circuit (17c: includes a white balance correction circuit, a camera shake correction circuit, etc.). Note that these circuit configurations are not limited to those schematically shown in FIG.

Reference numeral 18 denotes a sensor drive circuit, which drives the image sensor 4 to control the charge storage time, transfer timing and the like. Note that, in FIG. 2, the flow of control signals particularly related to the present embodiment is shown by an arrow.

In the present embodiment, in the image pickup mode for subject extraction, based on the output from the focus level detecting means,
The focusing lens motor (focus motor) is driven to capture a high-resolution image from the image sensor at the optimum focusing level, that is, in the state where the subject is most focused.

After that, the focusing lens motor is driven so as to be out of focus in the same direction as the driving direction of the focusing lens motor immediately before, and the low-resolution images are continuously captured at an appropriate non-focus level. Here, the appropriate non-focus level is a value set in advance, a value recorded in the storage means, or a value set by a command signal from an external terminal or the like,
For example, it is given as a ratio from the focus level (a numerical value such as 90% of the focus value). Further, the image may be picked up at a different out-of-focus level and used for initial data extraction (roughly cut image data) for image cutting.

When the continuous image pickup operation as described above is completed,
The subject extraction mode is automatically canceled, or the subject extraction mode setting switch 9 is set to return to the standard mode. The above-described operations such as setting and canceling the imaging mode may be performed by command input from a terminal (not shown).

FIG. 3 shows a case of basically operating a switch on a camera without receiving a control command from an external device.
9 is a basic processing flow for capturing a plurality of images with different focus states and performing image cutting processing.

As shown in FIG. 3, when the processing is started,
First, the power is turned on and the camera is set to the standby state (step S5a). Next, the subject extraction imaging mode is set and the imaging condition data is read (step S5b). This is a standard parameter value (for example, the range of the out-of-focus level, the number of images to be taken out for the clipping process) for controlling the imaging conditions built in the plural imaging condition setting control means 15, or a basic control program. Is a process for reading out.

Next, in step S5c, processing for detecting lens information and recording it in the imaging parameter storage means is performed. This is a process in which the lens motor driving means 3 detects the specifications and types of the mounted imaging optical system such as lens information. This information enables speedup of setting according to the type of the optical system when appropriately controlling the drive amount of the focus motor during focus control or when setting the same as the previously used imaging conditions. Used for the purpose.

After that, the magnification is set as in the normal image pickup mode (step S5d), and the photometry and (automatic) exposure conditions are set (step S5e). Next, the focus motor is driven until the optimum focus level for the subject is reached, and the focus signal is evaluated (step S5f). After that, the process proceeds to step S5g to determine the focus based on the mountain climbing method,
An image in which the subject is in focus is captured (step S5).
h).

Next, the focus motor is driven in the same direction for a predetermined width to evaluate the focus signal level (step S5).
i) It is determined whether or not the non-focus level is within a preset allowable range (step S5j). If the non-focus level is within the above-mentioned allowable range, the process proceeds to step S5k to perform imaging.

After that, the flow advances to step S51 to check the number of images to be picked up, and the focus control and the image pickup operation are repeated until the preset maximum number is reached. If the result of determination in step S5j is not within the allowable range, step S5
Then, the process proceeds to i ', the lens drive direction is reversed, and the drive width is reduced. Then, the process returns to step S5i to repeat the above-described operation. After that, the difference image data is extracted (step S5m) between the plurality of images having a small difference in focus signal level, and the image cutout process (step S5n) is performed.

FIG. 4A is a diagram schematically showing the relationship between the focus signal and the image pickup position of the focus lens in the case of performing image pickup under a plurality of image pickup conditions under focus control. FIG.
As is clear from (a), after the focus lens is driven from the initial position to detect the focus level and the image is taken,
The figure shows that the focus level is driven by a predetermined amount to capture an out-of-focus image within an allowable range.

FIG. 5 is a flowchart showing the basic control procedure of the camera body. As described above, the method of setting the image capturing mode for subject extraction is set by operating the camera body (step S6a) and setting by loading a recording medium having a program for subject extraction built therein (step S6).
6b), and setting by receiving command data from an external terminal or the like (step S6c).

In the former two cases, the basic control program is operated (step S6d), and when the command data is received, the imaging parameters and the imaging condition control amount to be further controlled,
The number of images to be picked up, a processing range inside the camera (for example, preprocessing such as difference data calculation for cutout processing, and only shooting under a plurality of image pickup conditions are designated) are set (step S6e).

It is assumed that these conditions are preset as standard in the basic control program. During image pickup under a plurality of image pickup conditions (step S6f), control of image pickup conditions based on set parameters (focus control, gamma correction control) and image pickup parameters at that time (lens data, focal length, magnification, focus signal, focus) The lens arrangement, gamma control value, aperture diameter, charge storage time, etc.) are recorded.

The subsequent operation procedure branches into three control flows according to the processing range inside the camera. That is, the first operation is to automatically return to the standard mode (step S6j). The second operation is to receive a cutout condition input (step S6g) after the image pickup operation in the subject extraction mode, and the third operation is to perform a cutout process under a preset standard condition (step S6h: (To be described later).

After the processing is completed, the cut-out image is recorded in the storage means in the camera body, transferred to an external memory or a display device (step S6i), or displayed on a monitor such as a finder or a display (step S6i '). You may. After the above processing is completed, an automatic or manual return to the standard imaging mode (step S6j) is performed.

The process of processing each image captured by the image sensor 4 will be described in detail below. The video signal processing means 1 is provided for each of the video signals from at least two sensors obtained by controlling the in-focus state (focus).
In 7c, gamma correction and white balance correction are performed.

Then, in the cut-out image processing means 13 inside the image pickup means, the difference between the high resolution image (focused image) and the low resolution image (non-focused image) or between two different low resolution images. Difference data is obtained, and smoothing, binarization, and thinning processing is performed on these.

The processing other than the difference may be performed by inputting an appropriate processing parameter from the external terminal (not shown) or the like via the command / data communication control means 10. When using multiple low-resolution images with different out-of-focus degrees, components with high spatial frequencies are cut, and when extracting the contour line of the specific target from their difference data, extra isolated feature points or isolated regions are extracted. The influence can be suppressed.

Therefore, stable initial data (contour line) can be obtained without being affected by noise, high spatial frequency components of the background, or illumination conditions. Note that the output value of the pixel that becomes negative due to the difference is set to zero,
Alternatively, the absolute value of the difference in each pixel may be output.

Next, the subsequent initial data (contour line) extraction processing will be described. This processing can be performed by the terminal connected to the camera, but the processing by the computing means inside the camera is registered as the standard processing mode or the command data communication control means 1
It can be specified from a terminal or the like via 0.

Typically, standard processing within the camera is set in the basic control program such as difference calculation and binarization processing between images under a plurality of image pickup conditions or difference calculation and smoothing processing.

Smoothing processing after calculation of comparison (difference) data between images obtained by image pickup condition control includes removal of an isolated feature (a local area having the same brightness or color of an irregular shape, a line segment, etc.) and a Gaussian filter or the like. Is performed, or general low-pass filter processing is performed.

The threshold value for the subsequent binarization processing is set by Otsu's method (Journal of the Institute of Electronics, Information and Communication Engineers, vol.J63, pp.349-356, 1).
980), or after classifying the image into blocks of appropriate size, setting based on local statistical processing of image data (luminance, color, etc.) (mean, variance, hypothesis test from histogram, etc.), etc. Can be used.

FIG. 6 shows the processing progress of image data to which the processing method described above is applied. In FIG. 6, (1) is an image in focus, and (2) is a diagram in which an image in the case of defocus is schematically emphasized.

Further, (3) shows a local area set when the user designates one point in the extracted subject while confirming the image of (1) on the display means. (4) shows an image that has been smoothed, binarized, and thinned in the local region from the difference image data of (1) and (2).
Further, (5) is a diagram showing an example of an image in an initial contour line obtained as a result of performing contour line tracking processing.

Although not shown, when obtaining the difference image data, it is possible to use two images having different degrees of focus as described above and both of which are out of focus. In the present embodiment, the image processing camera 1 using the configuration shown in FIG.
The image capturing and image processing operations of the above may be controlled from the terminal 21.

Hereinafter, with reference to FIG. 7, a computer terminal as the calculation means 21, a display means 22 (first display device 22a, second display device 22b), instruction means (mouse)
The image extraction processing procedure in the system including the data communication network unit 23 and the data communication network unit 24 (including the bus control unit) will be described.

Here, a binarization threshold value is set based on statistical processing for a local area of a certain size centered on a specified point set by the mouse 23 or the like (or an area set in an arbitrary shape by the mouse 23 or the like). May be set, and binarization, thinning, and contour tracking processing may be performed within the area. After the processing is completed, a local area centering on the end point is automatically set and the same processing is repeated. Among these, thinning is a process for obtaining a representative point in a thick initial contour line, and is not limited to a method performed by conventional image processing.

For example, an end point may be searched for in any of the left, right, up, and down directions from any one point in the rough initial contour, and then the end point adjacent to that point may be searched for as the representative point. If there are multiple edges or contours in the neighboring area due to thinning, they are displayed on the display superimposed on the input image, and which edge is taken is selected using a means such as a mouse. It may be possible to give an instruction.

If only a single edge exists in the neighborhood area, or if the user selects an edge, contour line tracing processing is performed in a preset direction, and each point of the thinned image connected to the edge is traced. The respective positions are sequentially recorded by using as a sampling point of the initial contour line.

When the thinned image has a branching structure, the method of selecting the branching direction is as follows: image data attributes (color, brightness or dispersion thereof) of the contour line in the direction orthogonal to the tracing direction of the point before branching. Priority is given to the continuous branching direction and the direction with little change before and after the branching.

In this case, the image attribute of a micro area centered on each point after the branch is detected. As another method, the user may be allowed to select which branch to take by interrupting the contour tracking and blinking the area around the branch point.

If the contour line is divided in the middle by the binarization and thinning processing, the edge connection processing is performed. In this case, the connection is automatically performed by a predetermined algorithm (see the 23rd Conference on Image Engineering Conference, pp.67-70, 1992, etc.), or the end point remaining as a result of contour tracing is displayed on the display means. Blinks with.

Alternatively, a display is made so as to clearly indicate that it is an end point, such as by displaying a color different from other contour lines, and the user confirms it and indicates the end point to be connected by the instruction selecting means 23.
You may connect by instructing using.

As the contour line connecting the connection points, a straight line or a representative point on the contour line may be used to generate a spline interpolation curve or the like. In particular, if the end point after contour tracing is on the image frame, it may be connected to the upper end points of other frames.

As described above, after the closed contour line and the image data inside thereof are extracted, the background region removal processing by the statistical processing, the area growing method, or the active contour processing method is applied. More precise image extraction may be performed.

Next, a second embodiment of the image input device of the present invention will be described. In the present embodiment, 3 is obtained by continuously controlling the in-focus state in the cut-out image processing means 13.
The difference data (there are a plurality) between two or more images is averaged, and an initial contour extraction process (smoothing, binarization, thinning process, etc.) of the image to be cut out is performed on the result.

For this purpose, the level motor is driven at high speed on the basis of the output from the focus state detecting means to include the first half (front pin) or the second half (rear pin) including the optimum focus value, or the front pin and the rear pin. At least three still images including both pins are continuously captured at a predetermined time interval and stored in the image storage unit 6. Typically, after capturing images by focusing detection, a plurality of non-focused images are continuously photographed.

FIG. 8 shows the internal structure of the camera when a relatively large number of images are captured by changing the image capturing conditions as in this embodiment. Here, the signal from the image sensor 4 is transferred and recorded by the high-speed switching transfer means 30 directly to the temporary storage means 31 such as a cache memory without passing through the video signal processing section.

After image pickup under a plurality of image pickup conditions, that is, after the image writing to the temporary storage unit 31 is completed, the image data is sequentially transferred to perform the video signal processing and the preprocessing necessary for the image cutout. At this time, the switching transfer means 30 may directly transfer to the cut-out image processing means 13.

As described above, the non-linear processing which is usually performed on the video signal from the sensor 4 is not performed, so that the influence of the change of the illumination condition at the time of image pickup, the movement of the object, etc. is suppressed and the processing accuracy is stabilized. Can be supported. Further, by performing the video signal processing after capturing a series of images, it is possible to speed up the imaging operation, stabilize the clipping processing, and improve flexibility.

It should be noted that when the images are taken in time series, only the immediately preceding image data and the difference data updated by the averaging process may be temporarily held. Further, if the number of images used for the initial contour line extraction processing is designated as N in advance, if N or more images are captured in the focusing process, the latest N images are held (erasing in the oldest order), and then, Difference data may be calculated and averaged.

As in the present embodiment, the influence of noise can be suppressed by averaging the difference data between a plurality of images having slightly different in-focus states, and image extraction with high accuracy and resolution can be realized. .

Next, a third embodiment of the present invention will be described. In this embodiment, extraction processing from an image under a plurality of imaging conditions is performed with exposure condition control or sensor sensitivity characteristic control.

FIG. 9 shows a block diagram of the image processing camera.
Imaging parameters controlled in this case include aperture diameter, image sensor gain, gamma characteristic, strobe emission intensity, strobe color temperature (emission spectrum), sensor (CCD) charge accumulation time (or shutter speed), and the like. Can be mentioned. The gamma characteristic is performed by the gamma correction control circuit 17b provided in the video signal processing circuit 17.

In the present embodiment, one of them is different by sending a control signal to the video signal processing section 17, the stroboscopic light emitting means 19, or the imaging parameter measurement control means (shutter speed control section 53 or aperture measurement control section 54). Imaging under the imaging conditions, that is, an image obtained by controlling a minute amount from the optimal value in the normal imaging mode and an image under the optimal condition are captured.

Further, in the present embodiment, a switch for setting which imaging parameter is preferentially controlled, that is, the priority imaging parameter setting means 2 for setting the priority order.
Has zero. For example, setting switches for focus priority, exposure condition priority, etc. may be provided in order to relatively weight and control the (non) focus degree, exposure condition, sensor sensitivity characteristic, etc. of the above-described embodiment.

In the system configuration shown in FIG. 7, FIG.
When a point on the image displayed on the display device 22 is designated by the mouse 23 or the like, the plural imaging condition setting control section 15 of the image signal processing section in the area of a preset size centering on the point or the entire screen Send a control signal to 17.

Then, the image with the gamma characteristic value set in the normal image pickup mode and its video signal characteristic parameter are controlled in each pixel or in the local region (changed with a constant width).
In this case, the designation of the shape and size of the area for capturing a plurality of images may be directly performed by the instruction unit 23.

FIG. 4B shows the luminance level (I ₁ > I ₂ ).
FIG. 6 is a diagram schematically showing the relationship between the amount of change in the gamma coefficient from the optimum value (varies according to the sensor signal level) and the amount of change in the brightness level according to the above.

When the difference data between images obtained by modulating in this manner is binarized and thinned with an appropriate threshold value, it corresponds to the contour line (edge component) in the actual image due to the non-linearity of the gamma characteristic. A line image is obtained. From this, the initial contour line is obtained by the method described in the first embodiment.

In the control of the aperture diameter or the charge storage time, when the background, the object, and the brightness level are greatly different, the contour line extraction by the above method may be particularly effective due to the nonlinear saturation characteristic of the stored charge. is there.

Further, when the stroboscopic light emission characteristic (intensity, spectrum) is controlled and the image is picked up, it is effective when the reflection intensity of the stroboscopic light from the subject is sufficient (in indoor photography, when the subject distance is sufficiently small, etc.). .

Next explained is the fourth embodiment of the invention. The image processing camera according to the present embodiment automatically records a plurality of moving images as in the previous embodiment when the subject extraction imaging mode switch is pressed while recording a moving image (video signal processing for camera shake correction is performed). Record and shoot continuous still images under the shooting conditions.

FIG. 10 is a block diagram showing the main parts of the image processing camera of this embodiment. After the power is turned on, when the recording / recording mode is set, the moving image is captured and recorded. At the time of recording, the image compression coding unit 40 may code a moving image by MPEG or the like and a clipped image by still image coding by JPEG or the like.

When the moving image of the subject is cut out by the cut-out image processing means 13, the cut-out moving image may be coded by the image compression coding unit 40 in correspondence with the moving image. The switching of such processing is performed by the plural imaging condition setting control unit 15 sending a switching control signal to the image compression encoding unit 40 in response to an ON / OFF signal of the subject extraction mode setting switch.

The image compression / encoding unit 40 generates an index of a moving image or a continuous still image as a header portion or header file of image data. In the present embodiment, when the subject extraction mode setting switch 9 for subject extraction is pressed, high-speed image capture is performed in the same manner as in the previous embodiment while controlling the imaging conditions, and the video output from the video signal processing unit 17 is performed. The signal data may be processed by the cutout image processing unit 13, and the cutout image may be output to the image compression encoding unit 40 to be further compression encoded.

An extracted image obtained as a result of cutting out a continuous still image obtained in the subject extraction mode is registered, and this image is used as a template image to track the subject image by the image tracking means 41 to generate a moving image. You may cut out by. In this way, it is possible to realize high image quality of a still image from the viewpoint position to be cut out while changing the angle of the moving image for shooting.

[0144]

As described above, according to the first aspect of the present invention, the optimum image pickup parameters are set or controlled when the image pickup means performs the image pickup for extracting the specific subject image. can do. Moreover, since the image quality of the extracted image can be optimized by controlling the imaging conditions, the subject image can be extracted at high speed and with high image quality.

According to the second invention of the present application, since the feature amount of an image can be controlled from the outside, the image pickup operation for subject extraction and the image processing performed inside the image input device are remotely controlled. And the extracted images can be transferred online to a remote location.

According to the third invention of the present application, since it is possible to control the imaging parameter by supplying a control command from the outside through the control command communication means, the imaging parameter for subject extraction can be controlled from the outside. It is possible to control, and it is possible to realize the high quality of the extracted image and the high speed of the extraction process.

According to the fourth invention of the present application, the image capturing mode setting means is provided with an image capturing mode setting button, and when the image capturing mode setting button is operated, the image capturing mode setting operation is started, and the image capturing parameter measurement value is set. Since the control amount of the imaging parameter is set on the basis of, it is possible to automatically set the imaging parameter in the subject extraction mode, and it is not necessary to manually set the control amount of the imaging parameter. be able to.

According to the fifth aspect of the present application, there is no need to perform an operation for returning to the normal image capturing mode after the image capturing operation (image capturing operation) for extracting a specific subject from a plurality of images under different image capturing conditions. Therefore, it is not necessary to set the image capturing in the subject extraction mode to the normal image capturing mode, and the operability can be greatly improved.

According to the sixth invention of the present application, the cutout image is processed based on the comparison data generated by the comparison data generating means and the control command given from the control command communication means. It is possible to perform the preprocessing necessary for cutting out the specific subject image by controlling the imaging condition at high speed in the image input device.

According to the seventh invention of the present application, the lens arrangement control means for controlling the lens arrangement of the imaging optical system based on the signal from the image pickup mode setting means, the focus for detecting the focus degree. Since the degree detection means and the capture signal generation means for generating a signal for capturing a plurality of images having different degrees of focus based on the output from the degree of focus detection means are provided, a plurality of pixels obtained by resolution control can be obtained. The specific subject image can be extracted from the image at high speed, and stable image extraction can be performed regardless of the illumination conditions.

According to the eighth invention of the present application, the exposure condition is controlled by controlling the shutter speed, the aperture diameter of the optical diaphragm, or the characteristic value of the image sensor based on the signal from the imaging mode setting means. Since the capturing signal generating means for generating a signal for capturing a plurality of images having different exposure conditions is provided, the exposure condition in the entire image frame or in the specific region is appropriately controlled, and the specific subject image of the specific subject image can be stably and quickly. Extraction can be done.

According to the ninth invention of the present application, since it is possible to set the priority order of the imaging parameters, it is possible to specify, for example, focus priority control or exposure amount priority control. As a result, it is possible to set the appropriate imaging condition control according to the shooting condition, improve the image quality of the clipped image, and speed up the process.

According to the tenth invention of the present application, since the video signal which is not subjected to the signal processing is stored in the image recording means, a stable image which is hardly influenced by the change of the imaging environment condition or the motion of the subject is obtained. It is possible to perform clipping, perform high-speed imaging operation under the control of imaging conditions, and improve the accuracy of a clipped image without being affected by changes in imaging environment conditions or subject motion. In addition, the image quality can be stabilized.

According to the eleventh invention of the present application, still images or moving images can be cut out, and an appropriate compression encoding method is automatically selected according to whether the extraction mode is still images or moving images. Can be selected.

According to the twelfth invention of the present application, stable image cutting can be performed regardless of the presence / absence of motion of a subject and the presence / absence of camerawork (panning, scanning, etc.), and further, a moving image or a still image can be obtained. Can be selectively cut out.

According to the thirteenth invention of the present application, the imaging parameter measurement control means is operated by the basic control program written in the storage means incorporated in the main body of the image input apparatus or the removable recording medium. This eliminates the need for online operation from an external terminal or complicated manual operation, and has a built-in control program that can perform basic imaging operations suitable for subject extraction, and a removable memory. A control program can be supplied.

[Brief description of the drawings]

FIG. 1 is a functional configuration diagram showing a configuration of a main part of an image input apparatus of the present invention.

FIG. 2 is a main part configuration diagram of an image processing camera that performs subject extraction by focus control.

FIG. 3 is a flowchart showing a basic process inside the image input device in a subject extraction mode.

FIG. 4 is a characteristic diagram showing a relationship between changes in imaging parameters and image characteristics.

FIG. 5 is a flowchart showing a basic control procedure of the image input apparatus main body from setting of a subject extraction mode to output of a cutout image.

FIG. 6 is a diagram showing a process of performing image cutting processing on a plurality of images subjected to focus control.

FIG. 7 is a configuration diagram showing an example of an image extraction system.

FIG. 8 is a configuration diagram of a main part of an image processing camera that captures a high-speed image.

FIG. 9 is a configuration diagram of a main part of an image processing camera that performs subject extraction by exposure amount control.

FIG. 10 is a configuration diagram of a main part of a camera when an image is captured in a subject extraction mode in a specific scene while capturing a moving image.

[Explanation of symbols]

 1 Cylindrical mirror of image pickup means 2 Imaging optical system 3 Lens motor drive control section 4 Image sensor typically CCD 5 Image pickup parameter measurement control means 6 Image recording / reproducing means 7 Viewfinder (EVF etc.) 8 Imaging parameter recording means 9 Subject extraction mode setting means 10 Command data communication control means 11 Output image generation means 12 Data transfer means 13 Cutout image processing means 14 Image capture signal generation means 15 Plural imaging condition setting control section 16 Aperture 17 Video signal processing section 18 Sensor drive circuit 19 Strobe light emitting means 20 Priority imaging parameter setting means A Imaging mode setting means B Imaging parameter measurement control means C Recording means D Imaging means E Cutout image processing means F Control command communication means G Image transfer means

Claims (16)

[Claims] 1. An image pickup means for picking up an image for extracting a subject, an image pickup mode setting means for setting an image pickup mode for the image pickup means to extract a subject, and an image pickup mode set by the image pickup mode setting means. An image input device comprising: an imaging parameter measurement control unit for performing measurement control of an imaging parameter based on the image pickup parameter; and a recording unit for recording the imaging parameter and the image. 2. An image pickup means for picking up an image for extracting a subject, an image pickup parameter measurement control means for measuring and controlling an image pickup parameter based on a preset image pickup mode, and the image pickup parameter and the image are recorded. Recording means, a cutout image processing means for performing predetermined processing on an image generated by the image pickup means under a plurality of image pickup conditions to cut out the image, and a control command for controlling the image pickup operation of the image pickup means. An image processing apparatus, comprising: a control command communication unit that performs communication relating to the above; and an image transfer unit that transfers the image data generated by the image pickup unit to the outside. 3. The image processing apparatus according to claim 2, wherein the image pickup parameter measurement control unit controls the image pickup parameter based on a control command given from the control command communication unit. 4. The image pickup mode setting means has an image pickup mode setting button, and when the image pickup mode setting button is operated, an image pickup mode setting operation is started, and the image pickup parameter measurement control means sets an image pickup parameter measurement value. The image input device according to claim 1, wherein the control amount of the imaging parameter is set based on the image input parameter. 5. The image input device according to claim 1, wherein the image capturing mode setting means automatically returns to the standard image capturing mode after a predetermined image capturing mode is set and necessary images are captured. 6. The image input device according to claim 4, wherein the image capturing mode setting means automatically returns to the standard image capturing mode after setting a predetermined image capturing mode and capturing a necessary image. 7. The cut-out image processing means includes comparison data generation means for generating comparison data between a plurality of images having different imaging parameters, the comparison data generated by the comparison data generation means, and the control command. The image processing apparatus according to claim 2, wherein the cutout image is processed based on a control command given from the communication means. 8. The image pickup parameter measurement control means, the lens arrangement control means for controlling the lens arrangement of the imaging optical system based on the signal from the image pickup mode setting means, and the focus degree for detecting the focus degree. The image according to claim 1, further comprising: a detection unit and a capture signal generation unit that generates a signal for capturing a plurality of images having different focus levels based on the output from the focus level detection unit. Input device. 9. The image pickup parameter measurement control means, the lens arrangement control means for controlling the lens arrangement of the imaging optical system based on the signal from the image pickup mode setting means, and the focus degree for detecting the focus degree. The image according to claim 4, further comprising: a detection unit, and a capture signal generation unit that generates a signal for capturing a plurality of images having different focus levels based on the output from the focus level detection unit. Input device. 10. The imaging parameter measurement control means comprises:
Lens arrangement control means for controlling the lens arrangement of the imaging optical system based on the signal from the imaging mode setting means, focus degree detecting means for detecting the focus degree, and outputs from the focus degree detecting means 6. The image input device according to claim 5, further comprising: capture signal generating means for generating a signal for capturing a plurality of images having different degrees of focus based on. 11. The imaging parameter measurement control means comprises:
The exposure condition is controlled by controlling the shutter speed, the aperture of the optical diaphragm, or the characteristic value of the image sensor based on the signal from the imaging mode setting means, and a signal for capturing a plurality of images with different exposure conditions is generated. The image input device according to claim 1, further comprising: a capture signal generating means for performing the above. 12. The image input device according to claim 1, wherein the image capturing mode setting means is capable of setting a priority order of image capturing parameters. 13. The image input according to claim 1, wherein said image recording means stores a video signal which is not subjected to signal processing, and has a plurality of high speed image recording means and a plurality of image transfer means. apparatus. 14. The image recording means has image compression encoding means for moving images or still images, and in the imaging mode for subject extraction, the compression method is variable according to the type of the cut out image. The image input device according to claim 1, wherein: 15. The cut-out image processing means includes an image cut-out processing section for cutting out an image from a time-series image, and an image tracking means for tracking the time-series image using the cut-out image obtained by the image cut-out processing section as initial data. The image processing apparatus according to claim 2, further comprising: 16. The imaging parameter measurement control means comprises:
The image input device according to claim 1, wherein the image input device is operated by a storage means built in the main body of the image input device or a basic control program written in a removable recording medium.