EP0366136A1 - Bildaufnahme- und Verarbeitungseinrichtung - Google Patents

Bildaufnahme- und Verarbeitungseinrichtung Download PDF

Info

Publication number
EP0366136A1
EP0366136A1 EP89119917A EP89119917A EP0366136A1 EP 0366136 A1 EP0366136 A1 EP 0366136A1 EP 89119917 A EP89119917 A EP 89119917A EP 89119917 A EP89119917 A EP 89119917A EP 0366136 A1 EP0366136 A1 EP 0366136A1
Authority
EP
European Patent Office
Prior art keywords
image
area
movement
detecting
focus
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP89119917A
Other languages
English (en)
French (fr)
Other versions
EP0366136B1 (de
Inventor
Masayoshi Sekine
Masamichi Toyama
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP63269557A external-priority patent/JP2832960B2/ja
Priority claimed from JP63269554A external-priority patent/JP2956056B2/ja
Priority claimed from JP1027038A external-priority patent/JP2756293B2/ja
Application filed by Canon Inc filed Critical Canon Inc
Publication of EP0366136A1 publication Critical patent/EP0366136A1/de
Application granted granted Critical
Publication of EP0366136B1 publication Critical patent/EP0366136B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/78—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using electromagnetic waves other than radio waves
    • G01S3/782—Systems for determining direction or deviation from predetermined direction
    • G01S3/785—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system
    • G01S3/786—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system the desired condition being maintained automatically
    • G01S3/7864—T.V. type tracking systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/223—Analysis of motion using block-matching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H04N23/673—Focus control based on electronic image sensor signals based on contrast or high frequency components of image signals, e.g. hill climbing method
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H04N23/675—Focus control based on electronic image sensor signals comprising setting of focusing regions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6811—Motion detection based on the image signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6815—Motion detection by distinguishing pan or tilt from motion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation
    • H04N23/686—Vibration or motion blur correction performed by mechanical compensation with a variable apex prism
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation
    • H04N23/687—Vibration or motion blur correction performed by mechanical compensation by shifting the lens or sensor position
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B2217/00—Details of cameras or camera bodies; Accessories therefor
    • G03B2217/005—Blur detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • This invention relates to an image sensing and processing device which is highly suited for a video camera or the like and is arranged to detect a moving part of an image plane, to set an object tracing area and a distance measuring area and to perform automatic focusing and automatic image stabilization control.
  • image shake detecting devices adopting varied methods.
  • the movement of the camera is physically detected by means of an acceleration sensor (an angular velocity sensor) and an optical system is compensated for the movement according to the direction and the degree of the movement.
  • an acceleration sensor an angular velocity sensor
  • the parallel moving extent of the whole image plane is detected through a video signal and is expressed in a movement vector. Then, the optical system is compensated on the basis of the vector.
  • the size of the device becomes larger thus requiring increases in space and weight. Besides, it results in a complex structural arrangement.
  • This method is therefore hardly suited for a home video camera such as a camera-incorporating type video tape recorder which must be compact in size and light in weight.
  • the above-stated known image shake detecting devices include, for example, an image stabilizing camera which is disclosed in Japanese Laid-Open Patent Application No. SHO 61-248681.
  • the camera of this kind is arranged as follows: an optical image is converted into an electrical signal by means of an image sensing system which consists of a lens system and a photo-electric converting element.
  • a TV image signal is obtained through a signal processing operation performed in a given manner on the electrical signal by a signal processing circuit.
  • the image signal thus obtained is supplied to an image shake detecting circuit as well as to a monitor.
  • a correlation between two image planes obtained at a given time interval is detected from the image signal to find the degree and the direction of any image shake.
  • a driving control circuit and a motor are operated to control and move the lens system to offset the image shaking on the basis of the result of detection.
  • the camera is thus arranged to be capable of obtaining a stable image even when the camera shakes.
  • image stabilization means a camera shake correcting function.
  • tracing means a moving object tracing function of the camera. The processing and control operations are performed for these functions fundamentally in the same manner as mentioned in the foregoing.
  • the image shake detecting device which is arranged in this manner is incapable of making a discrimination between the movement of an object occurring only in a part of the image plane and a shake of the whole image plane.
  • the image shake detecting sensitivity of the device must be arranged to vary for the different areas of the image plane.
  • the adoption of the method of detecting the quan­tity of shake with uniform sensitivity for the whole image plane does not permit a discrimination between a deformation or movement of the object and tilting of the camera by hand vibrations.
  • the camera might trace a wrong object to take a picture of it against the intention of the photographer.
  • an image shake detecting device has been proposed as disclosed in an article entitled “About an Image Plane Shake Compensating Device", appeared in "The Technical Report” of the Television Society, Vol. 11, No. 3, p 43 to 48, PPOE, '87-12 (May, 1987).
  • the whole image plane is divided into 140 blocks of areas.
  • a shake detection switch is arbitrarily turned on or off for each of these areas and an image shake is detected only from the areas for which the shake detection switch is turned on in accordance with a representing point matching method.
  • an image to be used as reference must be temporarily stored at a frame memory with its varied density values kept intact.
  • the device necessitates the use of an analog-to-digital (hereinafter referred to as A/D) converter and a memory of a relatively large capacity.
  • A/D analog-to-digital
  • the device is arranged to have one image plane superposed on another by staggering them to a certain degree of vector and to find out a vector that gives the highest degree of coincidence. Therefore, the operation of the device includes a large amount of computation. The device thus necessitates a circuit arrangement on a large scale and requires a long computing time.
  • An image shake detecting area has been deter strictlymined by the following methods: (1) A method of using a difference in luminance between the background and the object and (2) a method of detecting a movement occurring area from a difference occurred between the image of one frame and that of another obtained during a given period of time.
  • (1) A method of using a difference in luminance between the background and the object and (2) a method of detecting a movement occurring area from a difference occurred between the image of one frame and that of another obtained during a given period of time.
  • each of the conventional methods has a demerit as well as a merit. Therefore, with the conventional image shake detecting device incorporated in a compact video camera, it does not enable the camera to accurately cope with every one of varied image conditions.
  • an image sensor serves also as a sensor for automatic focusing.
  • the lens might be focused on one of them that has a stronger signal than the other even though the lens is intended to be focused on the other object. This is known as a "far-near competing condi­tion".
  • This invention is directed to the solution of the above-stated problems of the prior art. It is a first object of the invention to provide an image shake detecting device which is capable of accurately and stably detecting shaking of an image plane without mistaking a movement of an image resulting from shaking of the camera for an image movement resulting from a movement of a photographing object or from a panning movement of the camera or the like.
  • an image processing device arranged according to this invention as a preferred embodiment thereof comprises: movement vector detecting means for detecting a movement vector of an image in each of detection blocks formed by dividing an image sensing plane; computing means for detecting a distributed state of an image movement within the image sensing plane by statistically processing the movement vector obtained from each of the detection blocks; and area discriminating means for detecting an area having an image of a main photographed object on the basis of an output of the computing means.
  • a distance measuring area setting device arranged according to this invention as a preferred embodiment thereof comprises: movement vector detecting means for detecting a movement vector of an image in each of detection blocks formed by dividing an image sensing plane; computing means for detecting a distributed state of an image movement within the image sensing plane by statistically processing the movement vector obtained from each of the detection blocks; area discriminating means for detecting an area having an image of a main photographed object on the basis of an output of the computing means; and setting means for setting a focus detecting area on the image sensing plane on the basis of the area detected by said area discriminating means.
  • Figs. 1 to 3(c) show a first embodiment of this invention.
  • the image shake detecting device arranged according to the invention is applied to an automatic tracing (tracing) type focusing device, automatic tracing type exposure control device, etc. for the purpose of setting, within an image plane, a tracing area for tracing a movement of an object occurring within the image plane.
  • tracing automatic tracing
  • the first embodiment is arranged to obtain movement vectors from a plurality of areas set within the image plane.
  • An optical flow is statistically processed to determine an object tracing area and an image shake detection area.
  • Fig. 1 is a block diagram showing the arrangement of a video camera which is provided with the image shake compensating device arranged according to this invention.
  • the illustration includes an object O B; a variable apex-angle prism 12 which has a variable optical axis; a photo-taking lens 14; an image sensor 16 which is, for example, composed of a two dimensional CCD; a signal processing circuit 18 which performs a signal processing operation on an image signal output from the image sensor 16, including processes of gamma correction, blanking and addition of synchronizing signals, so that a TV signal of, for example, the NTSC system can be output from an output terminal 20; a luminance signal Y; a horizontal synchronizing signal H.SYNC; a vertical synchronizing signal V.SYNC; a delay circuit 22 which is arranged to delay the luminance signal Y for a given period of time and is composed of, for example, a field memory of the FIFO (first-in/first-out) type; a block dividing pulse
  • the dividing circuits 26 and 28 are arranged to output the input luminance signal in units of the blocks set on the image plane. More specifically, each of them consists of a gate circuit which is arranged to be opened and closed by the pulses output from the block dividing pulse generating circuit 24 and a memory which is arranged to store signal parts allowed to pass through the gate circuit.
  • a movement vector detecting circuit 30 is arranged to compare the signal of the currently obtained image plane with the signal output from the delay circuit 22 thus representing a previous image plane which precedes the current image plane by a given period of time; and to obtain a movement vector by detecting any change occurred in each of the divided blocks.
  • a memory 32 is arranged to store the movement vector information for each part of the image plane.
  • a statistical processing circuit 34 is arranged to prepare histogram indicating the size and frequency of each movement vector.
  • a threshold determining circuit 36 is arranged to recognize the shape of the histogram and to determine a threshold value which will be described later.
  • An area determining circuit 38 is arranged to look up and find from the histogram the blocks which are within the threshold determined by the threshold determining circuit 36.
  • An image shake quantity detecting circuit 40 is arranged to detect the quantity of an image shake from within the luminance signal.
  • the circuit 40 is composed of a correlation computing circuit which is arranged to perform a representing pint matching action.
  • An actuator 42 is arranged to change the apex angle of the variable apex-angle prism 12.
  • a driving control circuit 44 is arranged to drive the actuator 42 in accordance with the output of the image shake quantity detecting circuit 40.
  • the deflection angle of the exit optical axis relative to the entrance optical axis of the prism 12 is controllable by adjusting the apex angle of the variable apex-angle prism 12.
  • the video camera of Fig. 1 operates as follows: As object's image which passes the variable apex-angle prism 12 and the photo-taking lens 14 comes to the image sensor 16. The image sensor 16 then produces an image signal.
  • the signal processing circuit 18 performs the above-stated signal processing operation on the output of the image sensor 16.
  • a luminance signal Y output from the signal processing circuit 18 is supplied directly to the dividing circuit 28 and also indirectly to the dividing circuit 26 through the delay circuit 22.
  • the delay circuit 22 delays the luminance signal for the period of one field (about 16.7 msec) before it is supplied to the dividing circuit 26.
  • the dividing circuits 26 and 28 are arranged to divide one whole image plane into m X n blocks in accordance with the pulses output from the blocks in accordance with the pulses output from the block dividing pulse generating circuit 24.
  • m is assumed to be 20 and n to be 14 to give a total of 280 blocks.
  • the movement vector detecting circuit 30 obtains the movement vectors for every block by a method called a time-space gradient method. This method is discussed by B. K. P. Horn, et al. in "Artificial Intelligence", 17, p 185 - 203 (1981). It permits real time processing by a special purpose hardware.
  • the movement vector obtained by this method for the whole image plane showing any movement taking place at every one of the blocks is called an optical flow.
  • the actions of the circuit elements 30 to 38 are described with reference to Figs. 2(a) to 2(d) as follows: Fig. 2(a) shows by way of example the image plane of a currently photographed field. Fig.
  • FIG. 2(b) shows an optical flow obtained by accumulating for a given period of time a difference between the current field and an immediately preceding field.
  • Fig. 2(c) shows histograms representing the optical flow in sizes obtained in the directions X and Y respectively.
  • Fig. 2(d) shows the sections of areas recognized by this embodiment.
  • a motion picture of a moving object is taken.
  • the movement vectors detected by the movement vector detecting circuit 30 are accumulated for a given period (one sec, for example) by the memory 32. After that, the accumulated vectors are supplied to the statistical processing circuit 34.
  • the statistical processing circuit 34 prepares the histograms as shown in Fig. 2(c) by ranking these vectors according to the sizes of the X and Y components of each vector. In Fig. 2(c), the upper half of the drawing shows the vector histogram for the direction X and the lower half the vector histogram for the other direction Y.
  • each of axes in the directions X and Y indicates positive and negative sizes, which are separated by an origin O set, for example, in the center of the image plane.
  • the frequency values of the vectors are indicated in the vertical direction of the drawing.
  • the threshold determining circuit 36 determines threshold values from the shape of these two histograms. In each of the directions X and Y, a very small value near a distribution part having a peak value closest to zero in each of the directions X and Y is found and the position of this value is determined to be the threshold value. Threshold values are obtained in this manner on both the positive and negative sides. In Fig. 2(c), reference symbols Thx1, Thx2, Thy1 and Thy2 respectively denote these threshold values.
  • the threshold values thus determined are supplied to the area determining circuit 38.
  • the area determining circuit 38 looks up the blocks which are within the range of the threshold values among the movement vectors stored in the memory 32. For example, assuming that a block located in the i-th place in the direction of m and in the j-th place in the direction of n is expressed as a block Bij and that, within the block Bij, the moving quantity in the direction X is expressed as Uij and the moving quantity in the direction Y as Vij, the circuit 38 sets the following conditions: Thx1 ⁇ Uij ⁇ Thx2, and Thy1 ⁇ Vij ⁇ Thy2
  • the image shake quantity detecting circuit 40 computes and obtains a moving quantity from within the "on" area.
  • the detecting method to be employed by the image shake quantity detecting circuit 40 is not limited to the representing point matching method.
  • the movement can be detected at a high speed by image processing.
  • the method may be replaced with any method that permits control over execution ("on") and non execution ("off") of computation of correlation for every one of blocks.
  • the quantity of shake obtained by the image shake quantity detecting circuit 40 is supplied to the driving control circuit 44.
  • the actuator 42 then drives the variable vertex-angle prism 12 in such a way as to lessen the output (quantity of shake) of the image shake quantity detecting circuit 40, i.e., to lessen the movement of the image plane due to shake of the camera.
  • the movement vector detecting circuit 30 and the image shake quantity detecting circuit 40 are preferably arranged to perform a computing operation for every field.
  • the processing operation of the circuit elements 32 to 38 may be performed once in several hundred msec to several sec, because:
  • the camera shake frequency is generally within a range from 1 to 3 Hz and the frequency of mechanical vibrations of vehicles or the like higher than that. Therefore, any inadvertent object tracing action can be prevented by setting the cycle of the statistical computation of these circuit elements at a value longer than these periods.
  • the histograms have been described as to be prepared both in the directions X and Y. However, this may be changed to prepare a histogram for a two-dimensional space X-Y.
  • the computing cycle of the circuit elements 32 to 38 must be changed accordingly.
  • the embodiment operates as follows: Let us assume that a person B which is located in the central part of an image plane as shown in Fig. 2(a) is to be traced. In this instance, the optical flow becomes as shown in Fig. 3(a). Thus, the photographer tries to have the person B always in a given position within the image plane. As a result, the movement vectors become small at the image of the person B. Meanwhile, the movement vectors in the rest of the image plane including the background become large.
  • Fig. 3(b) shows histograms similar to hose of Fig. 3(c).
  • Fig. 3(c) shows an "on" area (hatched part) determined by the area determining circuit 38.
  • the threshold values Thx1, Thx2, Thy1 and Thy2 and the areas are determined by the same process as mentioned in the foregoing.
  • the vector dispersion can be expressed as follows:
  • the values Sxij and Syij of each block are compared with those of another. It is also possible to make this comparison in the square roots (standard deviations) of the values Sxij and Syij.
  • the object tracing area is determined by statistically processing the optical flow. Therefore, the embodiment is capable of automatically discriminating an object which is desired to be stably positioned within the image plane from other objects, even if these objects are hardly distinguishable by luminance from each other or even if there are plurality of moving objects within the image plane.
  • the invented area determining method is not limited to a TV camera. It will find a widely ranged applications including an industrial TV camera and a monitoring camera.
  • the embodiment of this invention has salient advantages for practical applications.
  • Fig. 4 is a block diagram showing the distance measuring area setting device of a video camera which is arranged as a second embodiment of the invention.
  • the same parts which are arranged in the same manner as in the case of Fig. 1 are indicated by the same reference numerals and the details of them are omitted from the following description.
  • the second embodiment differs from the arrange­ment of Fig. 1 in the following point:
  • the variable angle prism 12, the prism driving actuator 42 and the prism driving control circuit 44 are omitted.
  • the second embodiment is provided with, instead of them, a focusing actuator 60 for adjustment of the focus of the photo-taking lens 14 and a driving control circuit 58 for driving the actuator 60.
  • the second embodiment is further provided with automatic focus detecting blocks 48 to 56 in combination with a distance measuring area setting circuit 46.
  • the circuit 46 is arranged to set a distance measuring area on the image sensing plane for measuring a distance to the object on the basis of information on the movement vectors obtained from the movement vector detecting circuit 30.
  • the actuator 60 is arranged to drive the focusing lens of the photo-taking lens 14 for the purpose of automatic focusing.
  • the driving control circuit 58 is arranged to control the actuator 60.
  • a high-pass filter (HPF) 48 is arranged to extract a high frequency component from a luminance signal.
  • a detection circuit 50 is arranged to convert the high frequency component extracted by the HPF 48 into a DC signal.
  • An integrating circuit 52 is arranged to produce a signal which hereinafter will be called AF signal.
  • a delay circuit 54 is arranged to delay the AF signal produced from the integrating circuit 52 for a given period of time which is, for example, one field period.
  • a comparison circuit 56 is arranged to compare the delayed AF signal with the current AF signal.
  • the driving control circuit 58 drives the actuator 60 in accordance with the output of the comparison circuit 56 for focusing.
  • the operation of the second embodiment shown in Fig. 4 is as follows: The object's image which has passed through the photo-taking lens 14 comes to the image sensor 16. The image sensor 16 then produces an image signal. The signal processing circuit 18 performs a signal processing action on the output of the image sensor 16 in the manner as described in the foregoing. The luminance signal Y output from the signal processing circuit 18 is applied directly to the dividing circuit 28. The luminance signal Y is also applied to another dividing circuit 26 after it is delayed for one field period by the delay circuit 22. Each of the dividing circuits 26 and 28 is arranged to divide the whole image plane into m X n blocks (m represents 20 and n 14, to have a total of 280 divided blocks).
  • the movement vector detecting circuit 30 obtains the movement vector of every divided block by the above-stated time space inclination method. Histograms showing magnitudes obtained in the directions X and Y as shown in Fig. 2(c) are obtained from the optical flow shown in Figs. 2(a) and 2(b). Then, the states of movement of areas obtained on the image sensing plane are classified as shown in Fig. 2(d) on the basis of the histograms. This process is the same as in the case of the first embodiment and, therefore, its details are omitted from the following description.
  • a hatched area consisting of "off" blocks which satisfy the condition Thx1 ⁇ Uij ⁇ Thx2 and Thy1 ⁇ Vij ⁇ Thy2 is discriminated from the void area consisting of "on" blocks other than the "off” blocks as shown in Fig. 2(d).
  • the hatched area of the "off” blocks approximately coincides with the background area while the void areas of the "on” blocks coincide with the object areas.
  • the distance measuring area setting circuit 46 allows only the luminance signal parts for the areas A, B and C of Fig. 2(d) to pass there.
  • the luminance signal thus passed is supplied to the HPF 48 to have the high frequency component extracted.
  • the output of the HPF 48 is subjected to a detection process at the detection circuit 50.
  • the output of the circuit 50 is integrated by the integrating circuit 52 to be output from the integrating circuit 52 as the AF signal.
  • the comparison circuit 56 compares the AF signal of the current image plane with the AF signal of the preceding image plane. The result of comparison is applied to the driving control circuit 58.
  • the driving control circuit 58 drives the actuator 60 to operate in the direction of increasing the AF signal.
  • the AF signal reaches a maximum value and just begins to decrease, the position of the actuator 60 is brought back to the maximum value point and the focusing action comes to an end.
  • the integrated value of the AF signal also differs. Simple comparison under that condition would mislead magnitude determination. In that event, therefore, the area ratio must be corrected.
  • integration sensitivity changeover information is sent to the integrating circuit 52 from the distance measuring area setting circuit 46 in accordance with the changes occurred in the area of the distance measuring area. The area ratio is thus corrected by virtue of this arrangement.
  • the distance measuring area is assumed in the description above to be the same as the area set by the area setting circuit 38, the area B of Fig. 2(d) which is located in the central part of the image plane may be selected solely as the distance measuring area.
  • other areas A and C may be either completely cut by a logic circuit arrangement or variously weighted by arranging a window function in such a manner that the central part of the image plane is weighted 100% and the four diagonal corners of the image plane weighted 0% with the weight attaching rate continuously varied between the central part and each of the corner parts.
  • Fig. 5 shows the arrangement of another embodi­ment.
  • a camera shake compensating device which is arranged like that of Fig. 1 is added to the second embodiment described above.
  • the lens is continuously focused on the main object while the camera shake is automatically compensated for by the arrangement which is as shown in the block diagram of Fig. 5.
  • Fig. 5 the same component parts as those of Figs. 1 and 4 are indicated by the same reference numerals and the details of them are omitted from the following description.
  • a variable angle prism 68 is arranged in front of the photo-taking lens 14 to have its optical axis variable.
  • the prism 68 is composed of two glass plates and a transparent silicone rubber, liquid or the like sealed in between the glass plates.
  • An inverting gate circuit 66 is arranged to obtain a D area signal for the area D which is other than the distance measuring object area selected by the area determining circuit 38 and is indicated by the hatching in Fig. 2(d) representing a background.
  • a shake detecting circuit 62 is arranged to extract from the movement vector detecting circuit 30 the size and direction of the movement vectors obtained within the background area D of Fig. 2(d) as set by the inverting gate circuit 66.
  • a driving circuit 64 is arranged to cause the variable angle prism 68 to be driven in the direction of offsetting a shaking degree by controlling an actuator 70 on the basis of the output of the shake detecting circuit 62.
  • the shake detecting circuit 62 detects the degree and direction of the camera shake and then the variable angle prism 68 is driven in the direction of compensating for the shake. This enables the lens to be continuously focused without having the problem of the "far-near competing condition" by setting the distance measuring area for an object moving within the image sensing plane while shaking of the camera is being compensated for in the above-stated manner.
  • the embodiment operates as follows: Assuming that the camera is following the middle person B as shown in Figs. 2(a) to 2(d), the optical flow then becomes as shown in Fig. 3(a). Since the photographer is attempting to have the person or object B always in a fixed part of the image plane, the movement vector of the position of the person B becomes small. Meanwhile, the movement vectors of other parts including the background becomes large. Therefore, the distance measuring area is set at the area B.
  • Fig. 3(b) shows histograms in the same manner as Fig. 2(c). Fig.
  • 3(c) shows the relation to other parts of the "on" area which is indicated by the hatching and is determined by the area determining circuit 38.
  • the thresholds Thx1, Thx2, Thy1 and Thy2 and the area are determined in the same manner as described in the foregoing.
  • the camera is moved to follow a moving object.
  • the distance measuring area must be set at the area B of Fig. 3(c) which has the small movement vector. Therefore, when the panning of the camera is detected, the above-stated area setting output signal of the area determining circuit 38 should be inverted before it is supplied to the distance measuring area setting circuit 46.
  • FIG. 6 shows in a block diagram one example of arrangement for meeting the above-stated require­ment.
  • a panning detecting circuit 72 is arranged to detect the panning movement of the camera. For example, when the movement of the camera in one direction continues for a short period of time, the circuit 72 determines it as the panning and produces a control signal. More specifically, the frequencies of camera shaking are distributed within a band from 0.5 to 10 Hz or thereabout. Therefore, the degree of movement is detected from the output of a band pass filter which is arranged to extract a frequency band lower than this frequency band. When the movement comes to exceed a given value, the circuit 72 determines it as the panning.
  • the panning of the camera can be detected by other different methods. However, the other methods are omitted from thc following description.
  • the distance measuring area setting circuit 46 is instructed to adjust the distance measuring area to the object B which is as shown in Fig. 3(c) and has no movement vector between the thresholds Thx1, Thx2, Thy1 and Thy2 mentioned in the foregoing.
  • the distance measuring area is set at an object part having a large movement in cases where the camera is fixed with an object part showing some movement while there is not much movement in the background part.
  • the distance measuring area is set at the object part showing not much movement.
  • the distance measuring area may be set in an area having about the same vector as the vector of the area located near the center of the image plane.
  • Fig. 7 is a block diagram showing a further embodiment of the invention.
  • a vector threshold setting circuit 74 is arranged to set the threshold value of vector by extracting the movement vector of a predetermined central part of the image plane from the content of a memory 32 which stores the movement vectors of the whole image plane, by determining the size of the extracted movement vector and by carrying out a predetermined statistical process on the size determined.
  • the area determining circuit 38 is arranged to look up, on the memory 32, any area that has the movement vector output from the vector threshold setting circuit 74 and to produce and supply an area setting gate signal to the distance measuring area setting circuit 46.
  • the movement vector values of the central part of the image sensing plane are always detected; and the distance measuring area having movement vectors which are within about the same range as the range of the central part vector values.
  • a distance measuring area can be automatically set without making any discrimination between the case of Figs. 2(a) to 2(d) where the camera is fixed and the case of Figs. 3(a) to 3(c) where the camera is moved to follow a moving object.
  • the embodiment is capable of setting a distance measuring area in a shape and a position nearly in conformity to the intention of the photographer as the embodiment is arranged to determine the distance measuring area by statistically processing the optical flow in accordance with this invention.
  • the arrangement of the embodiment described precludes any possibility of having the lens caused to be focused on the background by the "far-near competing condition" or having the focus shifted from the main object to the background with the main object coming outside the distance measuring area.
  • the embodiment is also capable of following the object when its image moves within the image plane.
  • the arrangement of the embodiment will find a wide range of applications including not only photographing TV cameras but also industrial TV cameras, monitor cameras, etc.. The embodiment thus has a great advantage in terms of practical applications.
  • Figs. 8 to 11 show a sixth embodiment of the invention.
  • an image movement detecting device which utilizes the optical flow of the movement vectors mentioned in the foregoing is applied to an automatic focusing device.
  • the background of the sixth embodiment is first described as follows:
  • the video camera has recently come to use an automatic focusing device which uses the video signal of the camera and is capable of detecting an in-focus state by using information on the image obtained within an image plane irrespectively of distance to the object to be photographed. Therefore, the automatic focusing device of this kind has become popular in place of the focus detecting method of performing focus detection by projecting infrared rays or ultrasonic waves and detecting the reflection waves of them.
  • the focus detection method of the kind using the video signal can be roughly divided into two methods.
  • the video signal is modulated by forcedly bringing about an out-of-focus state by slightly vibrating either a component lens member of a photo-taking optical system or an image sensor in the direction of the optical axis with an actuator such as a dimorph element, a voice coil or the like (hereinafter this will be called a modulation method).
  • the photo-taking optical system is driven in such a way as to cause, for example, a high-frequency component of the video signal to be at a maximum value (hereinafter called a trial method).
  • an in-focus degree little varies and the modulated signal remains almost at zero after an in-focus state is attained even when the photo-taking optical system is vibrated.
  • a modulated signal is generated.
  • the phase of the modulated signal relative to the modulating signal is inverted according to whether it is in a near focus state or in a far focus state. Therefore, the focus adjusting actuator (which is a DC motor in general) is restarted according to the presence or absence of this modulated signal.
  • the restarting direction is determined according to the phase of the modulated signal. This enables the actuator to be accurately restarted.
  • the modulation method necessitates the modulating actuator to be of a high degree of precision and to be capable of performing a highly accurate action. Therefore, the structural arrangement of the camera becomes complex and large. As a result, it not only requires complex adjustment but also is disadvantageous in terms of electric energy consumption.
  • the lens or the image sensor is moved in such a way as to bring a signal component corresponding to a focusing degree to its maximum value. After attainment of an in-focus state, a focus signal obtained then is stored. After that, a current focus signal is compared with the stored signal. When the result of comparison indicates occurrence of any change, the focus adjusting actuator is restarted.
  • This method does not require any complex driving system for focus adjustment to permit simplification and reduction in size of the camera.
  • a drawback of the trial method resides in that: In restarting the actuator after it has been brought to a stop with an in-focus state attained, information on the direction in which the lens or the image sensor is to be driven is often unobtainable.
  • the invented shake detecting method of using the optical flow of movement vectors is applied to the simple arrangement of the above-stated trial method in such a way as to make available the information on the lens driving direction.
  • the automatic focusing device which is arranged as the sixth embodiment comprises: focus detecting means which is arranged to perform focus adjustment by detecting a focusing degree on an object appearing within the image plane; movement vector detecting means for obtaining the movement vectors of an image in each of a plurality of divided areas of the image plane; and control means arranged to compute the moving degree of the image on the basis of movement vector information output from the movement vector detecting means and to actuate the focus detecting means when the moving degree comes outside a predetermined range.
  • the automatic focusing device can be restarted according to the information.
  • the embodiment is capable of stably, promptly and highly accurately carrying out an automatic focusing action with a simple structural arrangement which is fundamentally based on the trial method.
  • Fig. 8 is a block diagram, which differs from the arrangement of Fig. 1 in the following points:
  • the variable apex-angle prism 12, the prism driving actuator 42 and the driving control circuit 44 are excluded similarly to the second embodiment shown in Fig. 4.
  • a focus adjusting actuator 60 for focus adjustment of the photo-taking lens 14
  • a driving control circuit 58 which controls the actuator 60.
  • automatic focus detection blocks 46 to 56 which are arranged to detect a movement of the object on the basis of movement vector information obtained from the movement vector detecting circuit 30.
  • a restart determining circuit 78 is provided for restarting the automatic focusing device.
  • the restart determining circuit 78 is arranged to receive the output of the movement vector detecting circuit 30, the outputs of a distance measuring area setting circuit 46 and the output of a comparison circuit 56 and to determine according to these outputs whether the automatic focusing device is to be restarted.
  • the distance measuring area setting circuit 46 is composed of a gate circuit which is arranged to cause only a luminance signal which is included in a luminance signal Y output from the signal processing circuit 18 and which corresponds to a focus detection area (distance measuring area) set on the image sensing plane to pass.
  • the circuit 46 sets the distance measuring area on the image sensing plane on the basis of the output of the area determining circuit 38.
  • a high-pass filter 48 is arranged to extract a high-frequency component from the luminance signal which corresponds to the distance measuring area set by the distance measuring area setting circuit 46.
  • a detection circuit 50 is arranged to convert into a DC level signal the high-frequency component extracted by the high-pass filter 48.
  • An integrating circuit 52 is arranged to integrate for a given period the DC level signal output from the detection circuit 50. Further, the integrating circuit 52 is arranged to have its integrating sensitivity controlled by a control signal 46a output from the distance measuring area setting circuit 46 accordingly as the distance measuring area changes, so that area correction is performed.
  • a delay circuit 54 is arranged to delay the output signal (hereinafter referred to as a focus signal) of the integrating circuit 52 for a predetermined period of time which, for example, corresponds to one field.
  • the comparison circuit 56 is arranged to compare a currently obtained focus signal with a focus signal obtained one field period before and delayed by the delay circuit 50.
  • a driving control circuit 58 is arranged to drive and control an actuator 60 (a DC servo motor, for example) which is arranged to control the focusing state of the photo-taking lens 14.
  • the automatic focusing device which is arranged according to this invention as described above operates in the following manner:
  • the area determining circuit 38 is arranged to produce information on the patterns of the object parts A, B and C and the pattern of the background area D, as shown in Fig. 2(d).
  • the distance measuring area setting circuit 46 performs a gating action on the basis of this information in such a way as to cause only the luminance signal of a main object part to pass and supplies it to a high-pass filter 48.
  • a high-pass filter 48 the high-frequency component is extracted from the luminance signal supplied there.
  • the extracted high-frequency component is converted into a DC level signal by the detection circuit 50 and is integrated for a predetermined period of time by the integrating circuit 52.
  • the integrating circuit 52 then produces a focus signal.
  • the focus signal is supplied to the comparison circuit 56 along with a focus signal which relates to a previous image plane obtained one field period before and which has been delayed for one field period by the delay circuit 54.
  • These focus signals of the current image plane and the previous image plane are compared with each other by the comparison circuit 56.
  • the result of comparison is supplied via the restart determining circuit 78 to the driving control circuit 58.
  • the driving control circuit 60 then controls and causes the actuator 60 to drive the photo-taking lens 14 in the direction of increasing the level of the focus signal.
  • the focus signal level reaches a maximum value and then decreases, the position of the lens obtained when the focus signal level has reached the maximum value is determined to be an in-focus position.
  • the photo-taking lens 14 is brought back to that position and the automatic focusing action comes to an end.
  • the integration outputs of the integrating circuit 52 are compared by the comparison circuit 56 during the process of the automatic focusing action, if the size of the distance measuring area previously set one field period before is found to differ from that of the distance measuring area of the current field, the integration signals differ in level although the focused state remains unchanged. In such a case, the difference in level might cause erroneous determination as the result of level comparison does not correspond to the focused degree.
  • a control signal 46a which represents the size of the distance measuring area is supplied to the integrating circuit 52. Any area difference between the distance measuring areas of the two field image planes is compensated for by virtue of this control signal.
  • the integration signal output level of the integrating circuit 52 is normalized by the area value of the distance measuring area. In the case of this embodiment, the above-stated compensation is accomplished by changing the integrating sensitivity of the integrating circuit 52 from one value over to another.
  • the distance measuring area setting means is as described above.
  • the histograms are prepared both for the directions X and Y of the image sensing plane. However, this may be changed to prepare a histogram for an X-Y two-dimensional space.
  • the operation cycle of the group of circuit elements from the memory 32 to the area determining circuit 38 must be changed according to the image accumulating period of time.
  • the distance measuring area is set according to the output of the area determining circuit 38.
  • a main object is located in the central part of the image plane.
  • an area in and around the central part of the image plane alone may be selected as the distance measuring area.
  • the areas A and C can be completely disregarded.
  • a window function in such a way as to have the central part of the image plane weighted 100% and the four diagonal corners 0% with the weight of the area between these parts arranged to continuously vary.
  • the distance measuring area is assumed to be set in the central part of the image plane and around it, for example, as represented by the area B in Fig. 2(d).
  • Fig. 9 shows an optical flow obtained in the area B under this condition.
  • the object corresponding to the area B is moving a little toward the upper right-hand side of the image plane while approaching to the camera.
  • the movement vectors in the divided blocks are as a whole trending to the right having the directions and sizes divergent toward the peripheral area. This indicates that the object is approaching the camera and, at the same time, is moving to the right-hand side thereof.
  • Fig. 10 shows the nature of an optical flow of this type.
  • a photo-taking lens 14 is identical with the photo-taking lens 14 of Fig. 8.
  • a reference numeral 80 denotes the object which corresponds to the area B of Fig. 2(d).
  • a numeral 84 denotes an image formed on the image sensing plane representing the object.
  • a numeral 82 denotes a state of the object obtained when it moves upward as viewed on Fig. 9 while it is approaching the photo-taking lens 14.
  • a numeral 86 denotes the image of the moved object.
  • the vectorial size of the optical flow which represents the upward movement as viewed on the drawing is unvarying irrespectively of their positions within the image plane.
  • the optical flow resulting from the approach of the object to the lens is brought about by the expansion of the object's image and the vectorial size thereof changes accordingly as its position varies within the image plane.
  • the size of the vector is zero on the optical axis of the photo-taking lens 14, i.e., in the center of the image plane and increases accordingly as its distance from the central part of the image plane increases.
  • the optical flow of Fig. 9 is a composite optical flow consisting of the vectors resulting from these two different factors.
  • the images 84 and 86 on the image sensing plane are illustrated as in different positions on the optical axis, they are in actuality formed on one and the same plane.
  • the displacement of the optical flow vector due to the expansion or contraction of the image means the divergence of vectors on the image plane.
  • the divergence "divA" of the vector A can be expressed as follows:
  • This divergence represents a change in distance between the object and the camera.
  • the restart determining circuit 78 obtains control information on the distance measuring area from the distance measuring area setting circuit 46 and also obtains information on movement vectors of each block of the image sensing plane from the movement vector detecting circuit 30.
  • the sum of the divergence "divA" within the areas B changes according to the expansion or contraction of the object's image on the image sensing plane.
  • the focus adjusting direction can be determined by the positive or negative value of the sum "d". More specifically, the object can be considered to have come closer to the photo-taking lens 14 if the sum "d" is larger than zero. In this instance, the focal point of the photo-taking lens 14 is in the rear of the object and there obtains image the so-called far-focus state. If the sum "d" is less than zero, the object has moved farther away from the lens. In that instance, there obtains a near-focus state of the lens.
  • a change in the object distance can be reliably detected by carrying out the statistical process on the movement vectors to compute the divergence of the optical flow vectors even in the event of a plurality of objects within the image sensing plane, a complex background or a hardly extractably image feature because of a slight difference in luminance, etc..
  • the sensitivity and accuracy of the embodiment are thus arranged to be much higher than those of the conventional device.
  • the optical flow of the image plane is obtained by detecting the movement vectors for each of the plurality of blocks on the image sensing plane; the occurrence or nonoccurrence of any movement of the object is detected from the displacement of the optical flow; a discrimination is accurately made between an approaching movement and a parting movement; and the necessity for restart of an automatic focus adjusting action is determined on the basis of the information thus obtained. Therefore, unlike the conventional modulation method, the embodiment obviates the necessity of having recourse to a highly precise and complex mechanism for positively modulating the focus signal by creating an in-focus or out-of-focus state in obtaining information on a near-focus or far-focus state.
  • the automatic focusing device according to this invention is thus arranged to be capable of accurately and precisely determining the necessity for a restart of the focus adjusting action from the image signal through an efficient and stable operation.
  • a movement vector of an image is detected in each of detection blocks formed by dividing an image sensing plane, and a distributed state of the image movement within the image sensing plane is detected by statistically processing the movement vector detected in each of the detection blocks.
  • a distributed state of the image movement On the basis of the distributed state of the image movement, an area having an image of a main photographed object is detected so as to set a focus detecting area on the image sensing plane or to control a focusing circuit.
EP89119917A 1988-10-27 1989-10-26 Bildaufnahme- und Verarbeitungseinrichtung Expired - Lifetime EP0366136B1 (de)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
JP269557/88 1988-10-27
JP269554/88 1988-10-27
JP63269557A JP2832960B2 (ja) 1988-10-27 1988-10-27 測距領域設定装置
JP63269554A JP2956056B2 (ja) 1988-10-27 1988-10-27 追尾制御方法及び装置並びにぶれ補正方法及び装置
JP27038/89 1989-02-06
JP1027038A JP2756293B2 (ja) 1989-02-06 1989-02-06 自動焦点調節装置

Publications (2)

Publication Number Publication Date
EP0366136A1 true EP0366136A1 (de) 1990-05-02
EP0366136B1 EP0366136B1 (de) 1995-03-22

Family

ID=27285638

Family Applications (1)

Application Number Title Priority Date Filing Date
EP89119917A Expired - Lifetime EP0366136B1 (de) 1988-10-27 1989-10-26 Bildaufnahme- und Verarbeitungseinrichtung

Country Status (3)

Country Link
US (1) US5949481A (de)
EP (1) EP0366136B1 (de)
DE (1) DE68921840T2 (de)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0455444A1 (de) * 1990-04-29 1991-11-06 Canon Kabushiki Kaisha Vorrichtung zum Erfassen von Bewegungen und Fokusdetektor, der eine solche Vorrichtung benutzt
EP0458239A2 (de) * 1990-05-21 1991-11-27 Matsushita Electric Industrial Co., Ltd. Gerät zur Detektion eines Bewegungsvektors und damit ausgestatteter Bildstabilisator
EP0496635A2 (de) * 1991-01-25 1992-07-29 Victor Company Of Japan, Ltd. Bewegungsvektorerkennungsvorrichtung zur Erkennung der Bewegung eines Bildes, um dessen Störung zu verhindern
WO1994005118A1 (en) * 1992-08-12 1994-03-03 British Broadcasting Corporation Derivation of studio camera position and motion from the camera image
US5657080A (en) * 1989-07-26 1997-08-12 Canon Kabushiki Kaisha Photographing system for detecting and correcting camera shake
US6014169A (en) * 1990-06-19 2000-01-11 Canon Kabushiki Kaisha Pickup device apparatus including vibration correction means
DE10017333A1 (de) * 2000-04-07 2001-10-18 Pilz Gmbh & Co Schutzvorrichtung zum Absichern eines Gefahrenbereichs sowie Verfahren zum Überprüfen der Funktionssicherheit einer solchen
EP1255146A2 (de) * 2001-02-14 2002-11-06 Leica Microsystems AG Verfahren und Vorrichtung zum automatischen fokussieren eines optischen Gerätes
USRE38420E1 (en) * 1992-08-12 2004-02-10 British Broadcasting Corporation Derivation of studio camera position and motion from the camera image
US7310109B2 (en) 2000-04-07 2007-12-18 Pilz Gmbh & Co. Protective device for safeguarding a hazardous area and method of checking the functional reliability of such a device
EP2247097A3 (de) * 2009-04-28 2014-09-17 FUJIFILM Corporation Bildumwandlungsvorrichtung und Verfahren zu dessen Betrieb

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5012270A (en) 1988-03-10 1991-04-30 Canon Kabushiki Kaisha Image shake detecting device
DE19701186C2 (de) * 1996-01-12 2001-10-18 Fraunhofer Ges Forschung Interaktives Überwachungssystem
JPH10178619A (ja) 1996-12-19 1998-06-30 Nikon Corp 画像再生装置および電子カメラ
JP3630905B2 (ja) * 1997-02-28 2005-03-23 キヤノン株式会社 撮像方法及び撮像装置
JP2000050151A (ja) * 1998-07-28 2000-02-18 Olympus Optical Co Ltd 撮像装置
DE19909627A1 (de) * 1999-03-05 2000-09-07 Bosch Gmbh Robert Verfahren und Vorrichtung zur Bestimmung einer Verschiebung von Bildern einer Bildsequenz
US7742073B1 (en) 2000-11-01 2010-06-22 Koninklijke Philips Electronics N.V. Method and apparatus for tracking an object of interest using a camera associated with a hand-held processing device
US7522209B2 (en) * 2002-12-26 2009-04-21 Hoya Corporation Automatic focusing apparatus including optical flow device calculation
DE60234649D1 (de) * 2002-12-26 2010-01-14 Mitsubishi Electric Corp Bildprozessor
JP4478422B2 (ja) * 2003-09-18 2010-06-09 キヤノン株式会社 像振れ補正装置、交換レンズおよび撮影装置
US8326084B1 (en) * 2003-11-05 2012-12-04 Cognex Technology And Investment Corporation System and method of auto-exposure control for image acquisition hardware using three dimensional information
JP4345651B2 (ja) * 2004-11-29 2009-10-14 セイコーエプソン株式会社 画像情報の評価方法、画像情報の評価プログラム及び画像情報評価装置
CN100423551C (zh) * 2005-01-28 2008-10-01 广达电脑股份有限公司 数字图像稳定方法与装置
EP1913557B1 (de) 2005-06-23 2016-06-01 Israel Aerospace Industries Ltd. System und verfahren zum verfolgen beweglicher objekte
KR100843093B1 (ko) * 2006-11-28 2008-07-02 삼성전자주식회사 움직임에 따라 컨텐츠를 디스플레이하는 장치 및 방법
EP2136317B1 (de) 2008-06-19 2013-09-04 Samsung Electronics Co., Ltd. Verfahren und Vorrichtung zur Erkennung von Zeichen
JP5214476B2 (ja) * 2009-01-15 2013-06-19 富士フイルム株式会社 撮像装置及び画像処理方法並びにプログラム
KR101710624B1 (ko) * 2010-07-27 2017-02-27 삼성전자주식회사 객체의 모션 벡터를 이용하여 자동 촬영 기능을 수행하는 디지털 영상 촬영 방법, 디지털 영상 촬영 장치 및 상기 방법을 기록한 기록 매체
KR101298023B1 (ko) * 2010-09-17 2013-08-26 엘지디스플레이 주식회사 사용자 동적 기관 제스처 인식 방법 및 인터페이스와, 이를 사용하는 전기 사용 장치
US20130258167A1 (en) * 2012-03-28 2013-10-03 Qualcomm Incorporated Method and apparatus for autofocusing an imaging device
IL219639A (en) 2012-05-08 2016-04-21 Israel Aerospace Ind Ltd Remote object tracking
WO2014111923A1 (en) 2013-01-15 2014-07-24 Israel Aerospace Industries Ltd Remote tracking of objects
IL224273B (en) 2013-01-17 2018-05-31 Cohen Yossi Delay compensation during remote sensor control
JP6203077B2 (ja) * 2014-02-21 2017-09-27 株式会社東芝 学習装置、密度計測装置、学習方法、学習プログラム、及び密度計測システム
TWI692969B (zh) * 2019-01-15 2020-05-01 沅聖科技股份有限公司 攝像頭自動調焦檢測方法及裝置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3443558A1 (de) * 1983-12-02 1985-06-13 Canon K.K., Tokio/Tokyo Automatische scharfeinstelleinrichtung
US4531158A (en) * 1982-01-27 1985-07-23 Hitachi, Ltd. Automatic focusing system for video cameras
EP0180446A2 (de) * 1984-10-27 1986-05-07 Sony Corporation Verfahren zur Feststellung von Bewegung in Fernsehbildern
EP0181215A1 (de) * 1984-11-07 1986-05-14 Sony Corporation Anordnung zur Feststellung der Bewegung von Fernsehbildern
EP0279424A2 (de) * 1987-02-18 1988-08-24 Sanyo Electric Co., Ltd. Automatische Fokussierungsschaltung für automatische Fokusanpassung in Abhängigkeit von Videosignalen

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4405940A (en) * 1977-05-31 1983-09-20 Westinghouse Electric Corp. Apparatus and method for preprocessing video frame signals
US4264161A (en) * 1977-10-12 1981-04-28 Canon Kabushiki Kaisha Motion detecting device in exposure control system for optical instruments
FR2502875B1 (fr) * 1981-03-30 1985-11-29 Tugaye Jean Procede et dispositif de cadrage et de stabilisation d'image de camera electro-optique
JPS60168276A (ja) * 1984-02-13 1985-08-31 Kokusai Denshin Denwa Co Ltd <Kdd> 画面上の移動体の動き方向検出方式
EP0188269B1 (de) * 1985-01-16 1994-04-20 Mitsubishi Denki Kabushiki Kaisha Videokodierungsvorrichtung
JP2502282B2 (ja) * 1985-03-29 1996-05-29 キヤノン株式会社 防振光学系
US4788596A (en) * 1985-04-26 1988-11-29 Canon Kabushiki Kaisha Image stabilizing device
US4780739A (en) * 1985-09-06 1988-10-25 Canon Kabushiki Kaisha Anti-vibration imaging device
JP2535845B2 (ja) * 1986-10-08 1996-09-18 キヤノン株式会社 自動合焦装置
DE3853555T2 (de) * 1987-06-09 1995-08-17 Sony Corp Verarbeitung des Bewegungsvektors in digitalen Fernsehbildern.
US5012270A (en) * 1988-03-10 1991-04-30 Canon Kabushiki Kaisha Image shake detecting device
DE68929448T2 (de) * 1988-09-09 2003-10-02 Canon Kk Automatische Bildstabilisierungsvorrichtung
JP2863188B2 (ja) * 1989-03-22 1999-03-03 松下電器産業株式会社 撮像装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4531158A (en) * 1982-01-27 1985-07-23 Hitachi, Ltd. Automatic focusing system for video cameras
DE3443558A1 (de) * 1983-12-02 1985-06-13 Canon K.K., Tokio/Tokyo Automatische scharfeinstelleinrichtung
EP0180446A2 (de) * 1984-10-27 1986-05-07 Sony Corporation Verfahren zur Feststellung von Bewegung in Fernsehbildern
EP0181215A1 (de) * 1984-11-07 1986-05-14 Sony Corporation Anordnung zur Feststellung der Bewegung von Fernsehbildern
EP0279424A2 (de) * 1987-02-18 1988-08-24 Sanyo Electric Co., Ltd. Automatische Fokussierungsschaltung für automatische Fokusanpassung in Abhängigkeit von Videosignalen

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5657080A (en) * 1989-07-26 1997-08-12 Canon Kabushiki Kaisha Photographing system for detecting and correcting camera shake
EP0455444A1 (de) * 1990-04-29 1991-11-06 Canon Kabushiki Kaisha Vorrichtung zum Erfassen von Bewegungen und Fokusdetektor, der eine solche Vorrichtung benutzt
US5729290A (en) * 1990-04-29 1998-03-17 Canon Kabushiki Kaisha Movement detection device and focus detection apparatus using such device
US5526044A (en) * 1990-04-29 1996-06-11 Canon Kabushiki Kaisha Movement detection device and focus detection apparatus using such device
US5317685A (en) * 1990-05-21 1994-05-31 Matsushita Electric Industrial Co., Ltd. Motion vector detecting apparatus and image stabilizer including the same
EP0458239A2 (de) * 1990-05-21 1991-11-27 Matsushita Electric Industrial Co., Ltd. Gerät zur Detektion eines Bewegungsvektors und damit ausgestatteter Bildstabilisator
EP0458239B1 (de) * 1990-05-21 1996-01-10 Matsushita Electric Industrial Co., Ltd. Gerät zur Detektion eines Bewegungsvektors und damit ausgestatteter Bildstabilisator
US6014169A (en) * 1990-06-19 2000-01-11 Canon Kabushiki Kaisha Pickup device apparatus including vibration correction means
EP0496635A2 (de) * 1991-01-25 1992-07-29 Victor Company Of Japan, Ltd. Bewegungsvektorerkennungsvorrichtung zur Erkennung der Bewegung eines Bildes, um dessen Störung zu verhindern
EP0496635A3 (de) * 1991-01-25 1994-04-20 Victor Company Of Japan
WO1994005118A1 (en) * 1992-08-12 1994-03-03 British Broadcasting Corporation Derivation of studio camera position and motion from the camera image
EP0909100A2 (de) * 1992-08-12 1999-04-14 British Broadcasting Corporation Bestimmung der Position und der Bewegung einer Studiokamera unter Benutzung des Kamerabildes
EP0909100A3 (de) * 1992-08-12 1999-04-21 British Broadcasting Corporation Bestimmung der Position und der Bewegung einer Studiokamera unter Benutzung des Kamerabildes
US5502482A (en) * 1992-08-12 1996-03-26 British Broadcasting Corporation Derivation of studio camera position and motion from the camera image
USRE38420E1 (en) * 1992-08-12 2004-02-10 British Broadcasting Corporation Derivation of studio camera position and motion from the camera image
DE10017333A1 (de) * 2000-04-07 2001-10-18 Pilz Gmbh & Co Schutzvorrichtung zum Absichern eines Gefahrenbereichs sowie Verfahren zum Überprüfen der Funktionssicherheit einer solchen
DE10017333C2 (de) * 2000-04-07 2003-02-13 Pilz Gmbh & Co Schutzvorrichtung zum Absichern eines Gefahrenbereichs sowie Verfahren zum Überprüfen der Funktionssicherheit einer solchen
US7310109B2 (en) 2000-04-07 2007-12-18 Pilz Gmbh & Co. Protective device for safeguarding a hazardous area and method of checking the functional reliability of such a device
EP1255146A2 (de) * 2001-02-14 2002-11-06 Leica Microsystems AG Verfahren und Vorrichtung zum automatischen fokussieren eines optischen Gerätes
EP1255146A3 (de) * 2001-02-14 2003-11-19 Leica Microsystems AG Verfahren und Vorrichtung zum automatischen fokussieren eines optischen Gerätes
EP2247097A3 (de) * 2009-04-28 2014-09-17 FUJIFILM Corporation Bildumwandlungsvorrichtung und Verfahren zu dessen Betrieb

Also Published As

Publication number Publication date
US5949481A (en) 1999-09-07
EP0366136B1 (de) 1995-03-22
DE68921840D1 (de) 1995-04-27
DE68921840T2 (de) 1995-09-07

Similar Documents

Publication Publication Date Title
US5949481A (en) Image sensing and processing device
EP0458373B1 (de) Einrichtung zur Feststellung von Bildzittern
KR100276681B1 (ko) 비디오 카메라 시스템
US5629735A (en) Image sensing device having a selectable detecting area
US5355163A (en) Video camera that automatically maintains size and location of an image within a frame
US5467127A (en) Automatic objects tracing device of camcorder
US6078751A (en) Image-shake correcting device
US6094222A (en) Image pickup apparatus using eye point dispersion in focus control
US5442397A (en) Image sensing apparatus
JP2756293B2 (ja) 自動焦点調節装置
JP2956056B2 (ja) 追尾制御方法及び装置並びにぶれ補正方法及び装置
JPH04280175A (ja) 自動焦点調節装置
JP2832960B2 (ja) 測距領域設定装置
JPH0638090A (ja) ビデオカメラ
JP2925172B2 (ja) 自動追尾装置
JP6071173B2 (ja) 撮像装置、その制御方法及びプログラム
JPS60126976A (ja) 自動焦点整合装置
JPS6139009A (ja) カメラ
JPS60143068A (ja) 自動焦点整合装置
US11669975B2 (en) Subject tracking device, subject tracking method, and imaging apparatus
JP2675807B2 (ja) 撮像装置
JP2611237B2 (ja) 自動合焦装置
Nelson et al. Zoom techniques for achieving scale invariant object tracking in real-time active vision systems
JPH0446472A (ja) オートフォーカス装置
JPH05130487A (ja) 自動焦点調節装置およびビデオカメラ

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB NL

17P Request for examination filed

Effective date: 19900918

17Q First examination report despatched

Effective date: 19920818

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB NL

REF Corresponds to:

Ref document number: 68921840

Country of ref document: DE

Date of ref document: 19950427

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed
REG Reference to a national code

Ref country code: GB

Ref legal event code: IF02

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20071015

Year of fee payment: 19

Ref country code: DE

Payment date: 20071018

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20071024

Year of fee payment: 19

Ref country code: FR

Payment date: 20071009

Year of fee payment: 19

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20081026

NLV4 Nl: lapsed or anulled due to non-payment of the annual fee

Effective date: 20090501

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20090630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090501

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090501

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20081031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20081026