JP2004304822A - Image processing system and its image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing system that can control a series of processings from the setting of image picking-up conditions to the generation of extracted images on a terminal processing side. <P>SOLUTION: In an image pickup system in which a camera 1 and a terminal device 21 are connected to each other through a data communication bus 24, the camera 1 picks up the image of the same object by controlling an image pickup parameter so as to photograph the object under a plurality of image pickup conditions in response to a command for setting an object extracting photographing mode when the terminal device 21 transmits the command to the camera 1. The terminal device 21 receives the plurality of image data photographed by means of the camera 1 and displays one image data of the received image data on a display device 26. Then the terminal device 21 designates an arbitrary point on the displayed image data and segments the image data of an objective area for extracting the object from the plurality of image data based on the plurality of image data and the designated point. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing system that processes an image obtained by controlling an imaging parameter and an image processing method of the image processing system.

  Conventionally, in the image pickup unit of this type of image processing system, with the digitization of signal processing, processing of video information and internal processing according to the improvement of the degree of freedom of processing include brightness level, color tone conversion, white balance. In addition to relatively simple processing such as processing and quantization size conversion, there are processing having an edge extraction function and an image extraction function using a sequential growth method of color components (Non-Patent Document 1).

In the method of extracting an image based on the difference from the background image, the aperture control unit is set so that the average luminance is appropriate when the background image is captured, and the current image is captured using the same setting values as the background image. It is disclosed that an object image is extracted based on these difference data (Patent Document 1).
In JP-A-6-253197, Technical Report of the Institute of Television Engineers of Japan, vol. 18, pp. 13-18, 1994

  However, in a conventional image processing system, a command communication control unit for appropriately controlling an imaging mode from the outside when a subject is extracted in the imaging unit, or an imaging parameter necessary for capturing an image for subject extraction in the imaging unit. The lack of a control function makes it impossible to set the optimal imaging conditions for image extraction. Therefore, for the purpose of image extraction, it has not been possible to optimally set the imaging conditions according to the imaging conditions, such as changes in lighting conditions, the presence or absence of subject movement, and the presence or absence of movement of the imaging unit itself.

  Further, when a camera is operated from a remote location to extract a specific subject, communication control means, a communication method, a control command, and the like have not been established. In particular, it has not been possible to automatically or interactively set optimal imaging conditions such as setting of an angle of view, focusing, and lighting conditions (for example, whether or not to use a strobe) for a specified subject.

  For example, the optimal setting of the angle of view is important to eliminate unnecessary background areas as much as possible and to perform the image extraction process efficiently, but the camera and the terminal device for performing this automatically or interactively Since the communication control and the imaging control method between the two have not been established, the functions cannot be realized.

  Accordingly, an object of the present invention is to provide an image processing system and an image processing method of the image processing system, in which a series of processes from setting of imaging conditions to generation of an extracted image can be controlled on the terminal processing side.

  In order to achieve the above object, an image processing system of the present invention is an image processing system for performing data communication between an imaging device and a terminal device via a communication medium, wherein the imaging device captures an image of a subject. A terminal for transmitting an image capturing command for extracting a subject to the image capturing apparatus, comprising: an image capturing unit that obtains image data by using the image capturing unit; and an image capturing parameter control unit that controls an image capturing parameter of the image capturing unit. Receiving means for receiving the plurality of image data captured by the imaging means by controlling the imaging parameters so that the imaging parameter means captures the same subject under a plurality of imaging conditions in response to the imaging command Display means for displaying one image data of a plurality of image data received from the imaging device; and display means for displaying image data displayed by the display means. Designating means for designating a desired point; area discriminating processing means for discriminating a target area for subject extraction based on the plurality of image data and the designated point; and a target area determined by the area discriminating processing means. Cutting means for cutting out image data from the plurality of image data.

  An image processing method for an image processing system according to the present invention is an image processing method for an image processing system that performs data communication between an imaging device and a terminal device via a communication medium. In response to the imaging command, the imaging device transmits an imaging command for subject extraction, and controls the imaging parameters so that the same subject is imaged under a plurality of imaging conditions in response to the imaging command, and causes the imaging unit to perform imaging. In the terminal device, receiving a plurality of image data captured by the imaging unit, displaying one image data of the plurality of image data received from the imaging device on a display unit, and displaying an image displayed by the display unit Specifying an arbitrary point on the data, determining a target area for subject extraction based on the plurality of image data and the specified point, and determining the determined target area Wherein the cut out image data from said plurality of image data.

  According to the present invention, a series of processes from setting of imaging conditions to generation of an extracted image can be controlled on the terminal processing side.

  An embodiment of an image processing system and an image processing method of the image processing system according to the present invention will be described. The image processing system according to the present embodiment is applied to an imaging system.

[First Embodiment]
The imaging system according to the first embodiment is characterized in that the focus state is controlled to perform a specific subject extraction process from images under different imaging conditions stably and at high speed regardless of lighting conditions and the like. The specific subject extraction process is a process of extracting a subject image from a background image, but this process may be applied to processes such as image classification, recognition, and tracking.

  FIG. 1 is a block diagram illustrating a configuration of an imaging system according to the first embodiment. The imaging system has a configuration in which at least one or more cameras (imaging devices) 1, a plurality of terminal devices 21, a display device 22, and a printer 25 are connected to a data communication bus 24. The terminal device 21 is provided with a mouse 23 and a display device 26 for selecting an instruction. The instruction may be selected by pen input instead of the mouse 23. The data communication bus 24 includes a bus control.

  In the present imaging system, when a plurality of terminal devices 21 and cameras 1 are connected, control of an imaging mode of an arbitrary camera 1 from an arbitrary terminal device 21 and control of image processing inside the camera are performed. Is possible, and one of the terminal devices 21 functions as a server.

  In addition, it is also possible to process the image data captured in a predetermined mode and output from the camera 1 by the terminal device 21 to perform a cutout process of a specific subject. Further, it is also possible to output the extracted image to an arbitrary display device (the display device 22, the display device 26 or a finder display inside the camera not shown) or to the printer 25.

  FIG. 2 is a block diagram showing an internal configuration of the camera 1. The camera 1 has an extraction processing mechanism for extracting a specific image from an image captured in a specific subject extraction imaging mode. However, the processing necessary for extracting the image may not be performed by the camera 1 but may be performed by the terminal device 21. Here, the subject extraction imaging mode refers to an imaging operation mode until a plurality of image data is output from the image sensor to the image recording unit by controlling imaging parameters (lens arrangement, shutter speed, aperture diameter, and the like).

  In the camera 1 shown in FIG. 2, reference numeral 2 denotes an imaging optical system. It is desirable that the imaging optical system 2 has a small depth of focus. Reference numeral 3 denotes a lens motor drive control unit, which includes a focusing lens drive unit (focus motor), a zoom lens drive unit, a lens information readout (measurement) unit that reads the type of the mounted lens, and the like.

  Reference numeral 4 denotes an image sensor, which uses a typical solid-state imaging device such as a CCD. Reference numeral 5 denotes an imaging parameter measurement control unit, which includes a zoom control unit 51, a focus state detection unit 52a, a focus control unit 52b, a shutter speed control unit 53, an aperture measurement control unit 54, and image signal characteristic parameters (gamma, knee, white balance). A characteristic amount (for example, a correction coefficient for gamma) of a correction, an accumulation time of a CCD or the like, and a detection unit 55 are included.

  The shutter speed control unit 53 controls a mechanical shutter (not shown). When an electronic shutter is used, a substantial shutter speed (charge accumulation time) is controlled by a sensor drive circuit 18 described later. The in-focus state can be detected by any of a detection method using a video signal and a distance measurement method using infrared rays. Reference numeral 6 denotes an image recording / reproducing unit which performs digital recording in a predetermined format on a recording medium (a disk medium such as a tape medium, an optical (magnetic) disk, or an IC memory medium such as an IC memory card) at the time of photographing. Note that the recording medium or the image recording / reproducing unit 6 may be provided detachably from the camera body. In addition, an imaging operation procedure program in the subject extraction mode, necessary data, and the like may be written in the recording medium, and control may be performed such that imaging is performed based on the information.

  At the time of image reproduction, the image data is output to a viewfinder (EVF) 7 or a data transfer unit 12 described later. Reference numeral 8 denotes an imaging parameter storage unit, which includes lens information from the lens motor driving unit 3, imaging parameters, image signal characteristic parameters, presence / absence of strobe light emission, intentional camera operation (movement) such as scanning, presence / absence of camera shake, and the like. Records information at the time of imaging. Such information is recorded in any of external control, manual operation, and autonomous control by a built-in program, and is used later to reproduce the same imaging conditions.

  Note that camera operation (scanning, panning, etc.) information may be detected by video signal processing, or an imaging device may include an acceleration sensor (not shown) or a gyro, and may be determined from output data.

  Reference numeral 9 denotes an imaging mode setting switch for subject extraction, tracking, identification, and the like. When this switch is set, an image capturing operation in the same imaging mode is performed by an imaging start operation signal from a shutter button (not shown) subsequent to turning on the switch. Is performed. Note that the imaging start switch may be configured to also serve as an imaging mode setting switch.

  Reference numeral 10 denotes an external command data communication control unit. An output image generation unit 11 generates an analog / digital image signal according to an output destination. For example, an NTSC signal or a PAL signal is generated for a TV receiver. Further, an image signal that has been subjected to gradation correction of light and shade, error diffusion processing, and color component correction processing according to the output device may be output to a display, a printer, or the like. Such a signal generation and processing instruction can be performed externally via the command data communication control unit 10, or can be performed using a switch on an operation panel of a camera (not shown).

  A data transfer unit 12 includes a communication interface unit, an adapter terminal, and the like (not shown), and outputs image data and the like to an external display device or computer. Reference numeral 13 denotes a subject extraction area discrimination processing unit (cutout image processing unit), which discriminates a background area and a subject area from an image obtained under a plurality of imaging conditions by a method described later, cuts out an image, and the like.

  Reference numeral 14 denotes an image capture signal generation unit which checks the control state of exposure conditions and imaging conditions, charge accumulation / transfer time in an image sensor (CCD), and the timing of an image writing control signal in the image recording / reproducing unit 6. Is automatically taken into account and appropriate image capture is performed.

  Reference numeral 15 denotes a system control unit as a setting unit for a plurality of imaging conditions, which is a command data communication control unit 10, a signal from the imaging mode setting switch 9, or information of a control program written on a recording medium of the image recording / reproducing unit 6. Is read, a series of control signals for the imaging operation in the subject extraction mode are generated.

  Reference numeral 16 denotes an aperture. Reference numeral 17 denotes a video signal processing unit, which includes a preprocessing circuit (a sample and hold circuit, an auto gain control circuit, an A / D conversion circuit, etc.) 17a, a gamma correction processing circuit 17b, and other video signal processing circuits (white balance correction circuit, 17c (including a camera shake correction circuit). Note that these circuit configurations are not limited to those schematically shown in FIG.

  Reference numeral 18 denotes a sensor driving circuit that drives the image sensor 4, and controls a charge accumulation time, a transfer timing, and the like. Also, in the figure, the flow of control signals particularly related to the present embodiment is indicated by thick arrows.

  In the present embodiment, in the subject extraction imaging mode, the focusing lens motor (focus motor) is driven based on the output from the focusing state detection unit 52a, and the optimal focusing level, that is, the focusing on the subject is achieved. After the high-resolution image is captured from the image sensor 4 in the state where the focus lens motor is driven, the focus lens motor is driven so that the focus shifts in the same direction as the driving direction of the immediately preceding focusing lens motor. Capture resolution continuously.

  Here, the appropriate out-of-focus level is a value that is set in advance and stored in a storage unit such as the image recording / reproducing unit 6 or a value set by a command signal from the external terminal device 21 or the like. It is set to a ratio from the focus level (a numerical value such as 90% of the focus value).

  Alternatively, an image may be taken at a different out-of-focus level and used for initial data extraction (roughly extracted image data) for image extraction.

  When the continuous imaging operation is completed, the subject extraction mode is automatically canceled, or the return to the standard mode is set by the imaging mode setting switch 9. Note that operations such as setting and canceling of the above-described imaging mode may be performed by inputting a command from the terminal device 21.

  FIG. 3 is a graph schematically showing a relationship between a focus signal and an imaging position of a focus lens when performing imaging under a plurality of imaging conditions by focus control. It is shown that a focus lens is driven from an initial position, a focus level is once detected and an image is taken, and then the focus lens is driven by a predetermined amount to capture an out-of-focus image within an allowable range.

  FIG. 4 is an explanatory diagram showing respective functions in a basic imaging system to which a camera and a terminal device are connected. In the figure, functions indicated by thick frames (steps S51, S52, S54, S58, S60, S61) belong to the terminal device 21, and functions indicated by broken lines (steps S57, S59, S62) correspond to the camera or the terminal. Is one of the devices. Others are functions in the camera (steps S53, S55, S56). These functions will be described below in order.

  First, the terminal device 21 establishes a communication session with the camera 1 (step S51). A command for setting the imaging mode for subject extraction is transmitted to the camera 1 (step S52).

  At this stage, the basic control program is activated (step S53), or the imaging parameters to be controlled, the control amount of the imaging conditions, the number of shots, and the processing range inside the camera (for example, calculation of difference data for clipping processing) And the like are designated from the terminal device 21 and set (step S54). This may be selected from a menu or the like displayed on the display device 26 of the terminal device 21.

  These conditions are standardly set in advance in the basic control program. At the time of imaging by setting a plurality of imaging conditions (step S56), control of imaging conditions based on the set parameters and imaging parameters (lens data, focal length, magnification, focus signal, focus lens arrangement, gamma control value, Recording of the aperture diameter, charge accumulation time, etc.) is performed.

  After that, there are three control processing procedures according to the processing range inside the camera. It automatically returns to the standard mode (step S57), receives a cutout condition after the end of the imaging operation in the subject extraction mode (step S58), or performs a cutout process under a preset standard condition (step S59).

  After the processing is completed, the cut-out image may be recorded in the image recording / reproducing unit 6 in the camera body, transferred to an external memory or the display device 22 (step S60), or displayed on a finder display (step S61). It is assumed that the output form of the cut-out image is specified in advance when inputting the control data or in the basic program. After the above processing, finally, the automatic or manual return to the standard imaging mode is performed (step S62).

  Next, a process of processing each image captured from the image sensor 4 will be described in detail. First, gamma correction, white balance correction, and the like are performed in the video signal processing unit 17 on video signals from at least two sensors obtained by controlling the in-focus state (focus).

  The area discrimination processing unit 13 inside the imaging apparatus calculates a difference between a high-resolution image (focused image) and a low-resolution image (unfocused image) or difference data between two different low-resolution images. To perform smoothing, binarization, and thinning processing.

  The processing other than the difference may be performed by inputting appropriate processing parameters from the terminal device 21 via the command data communication control unit 10.

  When using low-resolution multiple images with different degrees of out-of-focus, components with high spatial frequencies are cut, and when extracting a contour line of a specific target from their difference data, the effects of extra isolated feature points or isolated regions may be reduced. Can be suppressed. Therefore, stable initial data (contour lines) can be obtained without being influenced by noise, high spatial frequency components in the background portion, or lighting conditions. The output value of a pixel that becomes negative due to the difference may be set to the value 0, or the absolute value of the difference at each pixel may be output.

  Next, the subsequent initial data (contour line) extraction processing will be described. This processing can also be performed in the terminal device 21 connected to the camera 1. However, calculation processing inside the camera is registered as a standard processing mode, or the terminal device 21 is controlled via the command data communication control unit 10. It can also be specified from such as. Typically, it is set in a basic control program as a standard process inside the camera, such as a difference calculation between images under a plurality of imaging conditions and a binarization process or a difference calculation and a smoothing process.

  The smoothing process after calculating comparison (difference) data between images obtained by controlling the imaging conditions includes removing isolated features (local regions having irregular shapes or colors having the same brightness or color, line segments, and the like), and performing processing with a Gaussian filter or the like. This is performed by a convolution operation or general low-pass filter processing. After that, the threshold value of the binarization process is set by the method of Otsu (Transactions of the Institute of Electronics, Information and Communication Engineers, vol. J63, pp. 349-356, 1980), or after dividing an image into blocks of an appropriate size, A method of setting based on statistical processing (average, variance, hypothesis test from histogram, etc.) of local image data (luminance, color, etc.) may be used.

  Further, in the present embodiment, the region discrimination processing unit 13 performs averaging processing of difference data (a plurality of data) between three or more images obtained by continuously controlling the in-focus state. On the other hand, initial contour extraction processing (smoothing, binarization, thinning processing, etc.) of the image to be cut out may be performed.

  For this purpose, the lens motor is driven at a high speed based on the output from the focus state detection unit 52a to include the first half (front focus) or the second half (back focus) including the optimum focus value, or includes both the front focus and the rear focus. At least three still images are continuously captured at predetermined time intervals, and are stored in the image recording / reproducing unit 6. Typically, after capturing an image by focusing detection, a predetermined number of unfocused images are continuously photographed.

  In this case, a signal from the image sensor 4 is directly transferred and recorded in a temporary storage unit (not shown) such as a cache memory without passing through the video signal processing unit 17 by a high-speed switching transfer unit (not shown). After completion of image capturing under a plurality of image capturing conditions, that is, image writing to the temporary storage unit, the image data is sequentially transferred to perform video signal processing and preprocessing necessary for image clipping.

  At this time, the high-speed switching transfer unit may directly transfer the data to the area determination processing unit 13. As described above, by not performing the non-linear processing that is normally performed on the video signal from the image sensor, it is possible to suppress the influence of a change in the lighting condition at the time of imaging, the movement of the subject, and the like, thereby stabilizing the processing accuracy. .

  Further, by performing the video signal processing after a series of image capturing, it is possible to increase the speed of the imaging operation, stabilize the clipping process, and improve flexibility. When capturing images in time series, only the immediately preceding image data and the difference data updated by the averaging process may be temporarily stored. Also, if the number of images used for the initial contour line extraction processing is designated in advance as N, if more than N images are taken in the focusing process, the latest N images are held (deleted in the oldest order). The calculation and the averaging process of the difference data may be performed. By averaging the difference data between a plurality of slightly different images in the in-focus state as in the present embodiment, it is possible to suppress the influence of noise and realize an image extraction with high accuracy and high resolution.

  FIG. 5 is an explanatory diagram showing a process of processing image data. FIG. 2A is a focused image, FIG. 2B is a diagram schematically highlighting an image in a case where the image is out of focus, and FIG. 2C is a diagram in which the user views the image shown in FIG. This shows a local region set when one point in the extracted subject is designated while checking on the display device 26.

  FIG. 3D shows an initial contour obtained as a result of performing smoothing, binarization, and thinning in the local region from the difference image data of FIGS. The image inside is shown. Although not shown, when obtaining the difference image data, two images having different degrees of focus as described above and both of which are out of focus may be used. Further, only the contour line tracking processing may be performed without performing the smoothing and binarization.

  6 and 7 are flowcharts showing the procedure of the image extraction processing. First, the imaging system is initialized (step S1), and the devices connected to the network are set in a standby state. This initialization is usually performed on the terminal device 21 side.

  Next, a device that performs control from the imaging operation to the output operation is selected (step S2). It is determined whether or not the control device is the terminal device 21 (step S3). If the control device is the terminal device 21, the presence or absence of received data or a command is first checked in a standby state (step S4). If there is no, the imaging device 1 to be operated is specified (step S5).

  When the user selects a subject extraction mode setting from the initial menu panel displayed on the display device 22, a subject extraction imaging mode setting command is transmitted to the imaging device 1 (step S6). Upon receiving this command, the imaging device 1 reads a basic control program including a standard data set for extracting a subject from a memory (such as a ROM) built in the system control unit 15.

  In the present embodiment, the standard data set includes a control imaging parameter (focus), the number of images to be captured, an allowable range of the imaging parameter to be controlled (in the present embodiment, a ratio to a focus level, etc.), and image data output. (E.g., provided by the terminal device or the memory of the camera body). This basic program may be read from a recording medium in the image recording / reproducing unit 6.

  At this time, a message indicating the contents of the standard data and prompting the user to change the standard data is displayed, and these changes are performed here if necessary. Then, preprocessing (such as comparison data between a plurality of images) for setting control of a plurality of imaging conditions and cutting out is performed by the method described above (step S7).

  Next, an image from the imaging device 1 is displayed on the screen of the display device 26, and the user selects a subject to be extracted (step S8). That is, a point near the boundary between the inside of the subject and the background is indicated (clicked) by the pointing device 23 such as a mouse. In this case, it is desirable to appropriately set the angle of view exposure condition at the time of imaging. Therefore, appropriate determination of the imaging conditions at this stage (for example, that the subject is imaged under appropriate exposure conditions, that the entire area to be extracted exists, that the size is sufficiently large, etc.) is performed (step S9). If there is no problem, the process proceeds to area determination and cutout processing (step S11).

  If it is not appropriate, the zooming, the change of the attitude of the imaging device 1, and the change of other imaging parameters are automatically performed based on the image data (step S10). For example, by performing photometry and exposure condition setting in a predetermined area centered on the designated point, calculating an area area having a similar image attribute, and performing zooming control such that the ratio of the area in the screen becomes a certain value or more. Is also good.

  As an interactive designation method, the user designates a plurality of points to designate a rough region of the subject, or designates the shape and size of a region surrounding the subject, and for example, designates a rectangular or elliptical region on the subject image by the user. The mouse may be used to designate a point at that position, or may be scanned in one direction to semi-automatically set a predetermined shape area having a size corresponding to the length.

  Thereafter, zooming and focusing operations are performed centering on the region, the focus level is determined, and the above-described imaging at the out-of-focus level and calculation of difference data, and the above-described process for extracting a subject region are performed. Up to the contour tracing process on the determined boundary.

  After the cut-out processing, it is selected whether or not to record the extracted data (step S12), and when recording, specify the location and write the data (the storage unit inside the imaging device 1, the storage unit inside the terminal device, Step S13). At this stage, the editing of the extracted image (such as synthesis with another image) is performed using the mouse 23 of the terminal device 1 including the case where no image is recorded (step S14). Further, the image data may be converted (Step S15) according to the characteristics of the output device (such as a printer or a display) and then displayed (Step S16).

  A description will be given of a case where data is transmitted from the imaging device 1 at the time of the presence / absence check of received data (step S4), or a case where a recording medium having a built-in imaging control program in the subject extraction mode is loaded in the terminal device 1.

  The transmission device is determined (step S17), and the imaging mode is read out together with the image data (standard, subject extraction mode, etc.) (step S18). Here, it is determined whether or not to perform imaging and clipping processing in subject extraction under the same imaging mode condition (step S19). 1 is specified, and the processing after step 7 is executed.

  If not, the user is made to select whether or not to perform the cutout processing in the terminal device 1 (step S20), and step S11 or step S12 and subsequent steps are executed accordingly. Further, when command data is received from another terminal device in step S4, it goes without saying that processing corresponding to each request (step S21) is performed.

  On the other hand, when the imaging device 1 is selected at the time of selecting the control device (step S2), the operation on the imaging device 1 is executed with the first priority. Basically (step S22) of a basic process (step S22) from capturing a plurality of images having different focus states and performing pre-processing of image clipping when basically performing a switch operation on the camera without receiving a control command from the outside. Is as follows.

  First, after the power is turned on, when the subject extraction mode is set, the standard parameter values (for example, the range of the out-of-focus level, the number ) Or a basic control program.

  The specifications and type of the mounted imaging optical system such as lens information are detected by the lens motor drive unit 3. This information is used to properly control the drive amount of the focus motor during focus control, or to speed up the setting according to the type of optical system when setting the same as the previously used imaging condition. Used in

  After that, the magnification setting, photometry and (automatic) exposure conditions are set in the same manner as in the normal imaging mode. Then, the focus motor is driven until the optimum focusing level is reached for the subject, and the focusing is performed based on the hill-climbing method or the like. A focus is determined, and an image in which the subject is in focus is captured.

  The focus motor is driven in the same direction by a predetermined width to evaluate the focus signal level, and if the level is out of focus within a preset allowable range, imaging is performed. At this time, the number of captured images is checked, and focus control and the imaging operation are repeated until the number reaches the preset maximum number. Hereinafter, difference image data is extracted between a plurality of images having a small difference in focus signal level, and an image clipping process is performed.

  The image cutout processing (contour line extraction processing) will be described. Up to this point, a binarization threshold has been set based on statistical processing of a local area of a fixed size centered on a specified point set with a mouse or the like (or an area set in an arbitrary shape with a mouse or the like). The binarization, thinning, and contour tracking processing may be performed within the area.

  After the processing is completed, a local region centered on the end point is automatically set, and the same processing is repeated. Of these, thinning is a process for obtaining a representative point in a thick processing contour line, and is not limited to a method performed by conventional image processing. For example, an end point may be searched from any one point in the coarse initial contour in any of the left, right, up, and down directions, and then an end point adjacent to the point may be searched for as a representative point.

  If there are a plurality of edges or outlines in the vicinity area due to thinning, these are superimposed on the input image and displayed on the display, and the edge to be taken is selected and selected using a mouse or the like. It may be possible to do that.

  If only a single edge exists in the neighboring area, or if an edge is selected by the user, a contour tracing process is performed in a preset direction, and each point of the thinned image connected to the edge is defined as an initial contour. Each position is sequentially recorded as a line sampling point.

  When the thinned image has a branching structure, the branching direction may be selected such that the image data attributes (color, luminance, or their variance, etc.) of the contour in the direction orthogonal to the tracking direction of the point before the branching are continuous. The direction in which the direction and the direction change before and after branching are small is preferentially selected. In this case, the detection is performed by detecting the image attribute of the minute area centered on each point after the branch.

  As another method, the user may select which branch to take, for example, by suspending contour tracing and blinking an area around the branch point. However, when an imaging optical system having a shallow depth of focus is used, when the distance between the extraction target and the background is sufficiently larger than the subject distance, a branch structure may appear at the boundary between the background and the subject in the differential image data. Needless to say, this can be suppressed.

  When the outline is cut off in the middle by the binarization and thinning processing, an edge connection processing is performed. In this case, connection is automatically performed by a predetermined algorithm (see the 23rd Collection of Image Engineering Conference Papers, pp. 67-70, 1992, etc.), or an end point remaining as a result of contour tracking is displayed on a display device. Flashing or displaying a different color from the other outlines to indicate that it is an end point, and the user confirms it and specifies the end point to be connected using the instruction selection unit. The connection may be made by doing so. As a contour line connecting the connection points, a spline interpolation curve or the like may be generated using a straight line or a representative point on the contour line. In particular, when the end point after the contour tracking is on the image frame, it may be connected to the end point on another frame.

  As described above, after extracting the closed contour and the image data inside the closed contour, the background area is removed by statistical processing, the area growing method, or the dynamic contour processing method is applied to further refine the image. An extraction may be performed.

  In the present embodiment, a plurality of images having different focus states are processed by the focus lens control. However, other imaging parameters (aperture, shutter speed, etc.), imaging conditions (exposure conditions, sensor signal characteristics, etc.), or those A plurality of images having different combinations may be processed to extract or identify or track a specific subject.

[Second embodiment]
In the second embodiment, a case will be described in which the imaging method and the principle of extracting a specific subject image shown in the first embodiment are applied to a digital copying machine connected to a computer terminal or the like.

  FIG. 8 is a block diagram illustrating a configuration of an imaging system according to the second embodiment. In the imaging system, a case is shown in which a copying machine 27 as an image input unit is connected to a communication network.

  FIG. 9 is an explanatory diagram showing a schematic configuration of a digital copying machine. The image input unit 200 includes an optical system 201 having zoom and focus adjustment functions, an image sensor 202, an illumination unit 203 such as a halogen lamp, a focus zoom lens motor 204, a visible light spectral filter 205, a control unit 206, and an infrared cut / spectrum. It is composed of a correction filter (not shown) and the like.

  The printer unit 220 includes an image processor 210, a semiconductor laser 221, a photosensitive drum 222, a transfer drum 223, a rotary developing machine 224, and the like. The image forming apparatus includes a fixing device 230, a paper supply unit 240, a toner supply unit (not shown), a control processor 250, a document table 260, an instruction selection unit 270, a data communication control unit 280, and an operation panel.

  Further, an optical system (not shown) of the printer unit 220 typically includes a reflecting mirror, a toric lens (f-θ lens), a spherical lens, a polygon mirror, a cylindrical lens, and the like in the order close to the photosensitive drum.

  The operation panel 290 has a switch 291 for setting a copy mode of the copying machine. The setting of the same mode can also be performed from the keyboard of the terminal device 21 or an instruction selecting unit such as a pen or a mouse. In the present embodiment, after the same mode is set, once image capturing for creating area mask data to be copied is performed, then only the mask area is shaded with image data previously read from the semiconductor laser 221. Light of the corresponding output is emitted, and a latent image is formed on the photosensitive drum 222.

  An image obtained by imaging in the normal copy mode is displayed on the display unit of the terminal device 21, and the user confirms the image and designates a point on the contour line with the background of the subject to be extracted or its vicinity.

  Next, the control processor 250 controls the imaging parameters, captures images under different imaging conditions (for example, focus), and calculates difference data between the images by the image processor 210. Thereafter, by performing the same extraction region cutout processing (mask data generation processing) as in the first embodiment, only the image to be extracted is copied. Further, the extracted image may be transferred to another terminal device 21 or the display device 22 on the network and output.

  The present invention can also be applied to an electrophotographic facsimile apparatus having a part of the above configuration and having an imaging optical system. The reading unit of the facsimile apparatus has an illuminating unit, a mirror scanning mechanism, an optical system capable of controlling imaging parameters as in the first embodiment, and an image sensor.

  The recording unit may use any of an electrophotographic system, a laser beam printer system, and a thermal transfer recording system. The communication controller has, for example, a G4 protocol control function for a digital line and a G3 / G2 communication control function for a subscribed line. The control unit has an encoding / decoding unit.

  The facsimile device 27 is connected to the external terminal device 21 and can be controlled from the terminal device 21. That is, after shooting in the normal mode, the image data is displayed on the display unit of the terminal device 21. The user extracts an image from the image displayed on the display unit and extracts the image by the above-described method by designating a specific subject to be transferred.

  Note that the above-described imaging by the focus control and the extraction and copying of the specific subject can be performed with high accuracy and efficiency by applying to a three-dimensional object. In this case, the copying machine is configured by components of a normal copying machine such as a three-dimensional object installation table, an imaging device, an image processor, a printer unit, and a fixing device. In particular, the image input unit 200 may be separated from other components of the copying machine and installed above the three-dimensional object installation table. The process of transferring the image area extracted by the image processor 210 to the photosensitive drum and finally copying only the specific subject is the same as the above-described method.

  In addition, as a suitable imaging control method for extracting a specific target image from a planar area such as a paper surface, a filter control unit 206 controls a characteristic of a visible light spectral filter 205 in an image input unit 200 to perform a plurality of imaging operations. An image may be input under a condition, and a specific subject area may be extracted and copied based on the difference data.

  For example, when extracting an object whose color tone characteristic is unevenly distributed in a specific spectrum (for example, a red object as a whole), an image obtained by controlling a spectral filter so as to increase the transmittance with respect to that spectrum is usually used as an image. An image obtained in the mode is input, and extraction based on their difference data is performed.

  As another imaging control method, a plurality of image signals obtained by controlling the image signal characteristics (gamma or the like) of the image sensor 202 or the exposure conditions (illumination light intensity, the diameter of the diaphragm if there is a diaphragm, the scanning speed when performing scanning input, etc.) An extraction process may be performed on an image.

  In this case, in particular, after performing normalization processing (for example, scaling conversion of the luminance value so that the maximum luminance level becomes the same) between a plurality of image data, difference data between the two images is calculated, and the same as the method described above. It is desirable to perform a clipping process.

[Third Embodiment]
In the third embodiment, a case is described in which the above-described principle of extracting a specific subject is applied to an image input device (image scanner, facsimile device, or the like) having another contact image sensor (CCD device or the like). Here, a case where the present invention is applied to a facsimile machine is shown for simplicity.

  FIG. 10 is an explanatory diagram showing a schematic configuration of the image input device. The main body 300 is connected to the terminal device 21, and the original 350 is photoelectrically converted by the contact image sensor 310 to read image information.

  In the present embodiment, there is provided a drive control mechanism 320 for finely moving the image sensor 310 in a direction substantially orthogonal to the sensor surface, thereby causing a defocus and a document image input in a normal mode (high-resolution image input mode). Perform image input.

  A piezoelectric element, a voice coil motor, or the like is typically used as a drive unit of the fine movement control mechanism 320. The image data transferred via the modem 370 is transferred to a recording paper 360 by a recording unit (such as a thermal head) 330 as in other facsimile machines. The image processing unit 340 divides the image area into a character area and an image area based on the output from the image sensor 310 and, based on the method described in the first embodiment, specifies a specific graphic element specified in advance in the image area. Alternatively, a specific subject area is extracted.

  Specifically, the image is scanned and read at a normal sensor surface position, the first image data is captured, and then the sensor surface is slightly moved to scan and read the same image region in a defocused state. 2 is obtained. Further, the contour extraction and cutout processing of the subject are performed on the difference data between the first and second images as described in the first embodiment.

  The control processor 380 controls image reading, subject extraction, drive control of a contact sensor position, thermal head recording, paper feed, and the like, but also enables control by a control command from the terminal device 21.

[Fourth Embodiment]
In the fourth embodiment, the imaging apparatus includes an inexpensive and small wireless communication control unit having an infrared communication function. Other components inside the camera are the same as those in the first embodiment.

  In addition, by incorporating a communication control unit in the terminal device, transmission / reception of a setting signal of the imaging mode for subject extraction of the imaging device, imaging parameter control data, cutout image data, and the like are performed. The communication system defined by the modulation system, coding system, data transmission speed, data format, communication protocol, etc. is not particularly limited, but typically the system defined by IrDA is used. In this case, it is assumed that one-to-one two-way communication is performed within a distance of several meters (typically 1 m) from each other, but in principle, one-to-many communication (one imaging device and many Communication between terminal devices, and communication between one terminal device and a large number of imaging devices) are possible.

  Normally, a session is first established at a speed of 9600 bits / second at the start of communication with the other party, and at this time, the maximum transmission speed that both can support is checked, and thereafter, communication is performed at that speed.

  Also, half-duplex communication is performed by exchanging the data transmission right every 5 ms to 500 ms. When a communication session with a specific imaging device is established from the terminal device, the type (imaging device or terminal device), the ID number of the user, and the like are mutually transmitted and received, so that the imaging device enters an imaging standby state by remote operation. .

  In this case, a display indicating the communication control mode may be made on the operation panel display of the camera. Thereafter, the process from the setting of the subject extraction imaging mode to the transfer or recording of the captured image data is the same as in the first embodiment. Note that control data or appropriately compressed and encoded image data can be connected to a line such as a public network or ISDN and transferred to a remote location through communication equipment employing the same communication method. Further, an operation panel similar to the operation panel of the imaging device may be displayed on the display panel of the terminal device, and camera operation information may be transmitted to the imaging device by a touch or a hand or a pen.

FIG. 1 is a block diagram illustrating a configuration of an imaging system according to a first embodiment. FIG. 2 is a block diagram illustrating an internal configuration of the camera 1. 5 is a graph schematically showing a relationship between a focus signal and an imaging position of a focus lens when performing imaging under a plurality of imaging conditions by focus control. It is an explanatory view showing each function in a basic imaging system to which a camera and a terminal device were connected. FIG. 4 is an explanatory diagram illustrating a process of processing image data. It is a flowchart which shows an image extraction processing procedure. FIG. 7 is a flowchart illustrating an image extraction processing procedure following FIG. 6; It is a block diagram showing the composition of the imaging system in a 2nd embodiment. FIG. 2 is an explanatory diagram illustrating a schematic configuration of a digital copying machine. FIG. 3 is an explanatory diagram illustrating a schematic configuration of an image input device.

Explanation of reference numerals

Reference Signs List 1 camera 9 imaging mode setting switch 15 system control unit 17 video signal processing unit 21 terminal device 26 display device 52b focus control unit

Claims (7)

An image processing system for performing data communication between an imaging device and a terminal device via a communication medium,
The imaging device,
Imaging means for imaging a subject to obtain image data;
Imaging parameter control means for controlling imaging parameters of the imaging means,
The terminal device,
Transmission means for transmitting an imaging command for subject extraction to the imaging device,
A receiving unit that receives the plurality of image data captured by the imaging unit by controlling the imaging parameter so that the imaging parameter unit captures the same subject under a plurality of imaging conditions in response to the imaging command;
Display means for displaying one image data of the plurality of image data received from the imaging device;
Designation means for designating an arbitrary point on the image data displayed by the display means,
Area determination processing means for determining a target area for subject extraction based on the plurality of image data and the designated point;
A cutout unit that cuts out the image data of the target region determined by the region determination processing unit from the plurality of image data;
An image processing system comprising: The imaging apparatus further includes a storage unit that stores in advance standard data corresponding to the imaging command including at least imaging parameters for imaging under the plurality of imaging conditions,
2. The image processing system according to claim 1, wherein the imaging parameter control unit reads the imaging parameters stored in the storage unit when receiving the imaging command.   3. The image processing system according to claim 2, wherein the terminal device displays the standard data on the display unit, and changes the displayed standard data. 4.   One of the plurality of imaging conditions is a condition for obtaining the focused image data by focusing on the subject and obtaining the focused image data, and one is a condition for capturing the subject without being focused and obtaining the out-of-focus image data. 4. The image processing system according to claim 1, wherein the following condition is satisfied.   5. The area determination unit according to claim 4, wherein the target area for subject extraction is determined based on difference data between the in-focus image data and the out-of-focus image data and the designated point. 6. Image processing system.   The image processing system according to claim 1, wherein the display unit displays the image data clipped by the clipping unit. An image processing method of an image processing system that performs data communication between a capturing device and a terminal device via a communication medium,
In the terminal device,
Transmitting an imaging command for subject extraction to the imaging device;
In the imaging device,
In response to the imaging command, the imaging unit controls the imaging parameters so that the same subject is imaged under a plurality of imaging conditions, and the imaging unit performs imaging,
In the terminal device,
Receiving a plurality of image data imaged by the imaging means,
Displaying one image data of the plurality of image data received from the imaging device on a display unit;
Designating an arbitrary point on the image data displayed by the display means,
Based on the plurality of image data and the designated points, determine a target area for subject extraction,
An image processing method for an image processing system, wherein image data of a determined target area is cut out from the plurality of image data.

Images