JP2005101905A - Image pickup device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that an optical component prevents the realization of the miniaturization, the power-saving, and the cost reduction of a device, since the exclusive optical component is conventionally incorporated into a scanning image pickup device capable of changing the photographing direction by rotating a part of an optical system in order to cancel the rotation of an image by reversely rotating the image, because the image formed on the surface of a photoelectric transducer is rotated due to a secondary effect caused when a part of the optical system is rotated for changing the photographing direction. <P>SOLUTION: A two-dimensional element with pixels arranged in plane is used as the photoelectric transducer. The image rotation amount on the photoelectric transducer is obtained by calculation after detecting the rotation amount of the optical system. After that, the image subjected to the image rotation is reversely rotated by signal processing. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus, and more particularly, to an imaging apparatus that can shoot a wide area by changing an imaging direction by operating an optical system and has a calibration function for defective pixels of a photoelectric conversion element and pixels having characteristic variations. Is.

There are various types of surveillance systems, but some of them are imaged over a wide area by periodically moving the shooting direction of the camera, and targets with preset characteristics are automatically detected from these images. , There is something like an alarm.
Some cameras (imaging devices) of such a monitoring system move the imaging direction by moving the entire imaging device, but only a part of the optical system is exposed to the outside due to circumstances such as making the imaging device unnoticeable. In some cases, the imaging direction is changed by rotating this portion (see, for example, Patent Document 1).

JP-A-6-261240 (Example 1, FIG. 1)

  As a photoelectric conversion element of such a wide-area image pickup device, a so-called line sensor in which pixels are arranged in a line may be used. In other words, it takes advantage of the fact that the image formed on the sensor by the movement of the shooting direction (visual axis) flows, and images a wide horizontally wide area at a stretch in the same way as a copier.

Here, the image must flow in a direction orthogonal to the line sensor, but the direction in which the image flows varies depending on the direction of movement of the visual axis, and there is no guarantee that the image flow will be exactly orthogonal to the line sensor. In addition, in the case of a structure that changes the shooting direction by rotating a part of the optical system, the image formed as a secondary effect due to the movement of the visual axis rotates, and the movement of the image is a translational movement. Rotational motion becomes a complex motion synthesized.
Therefore, it is necessary to control the image so that the image always flows orthogonally to the line sensor by intentionally reversely rotating the image. This is done with components such as the corrected rotating frame of Patent Document 1, FIG.

On the other hand, in addition to the line sensor, there is a photoelectric conversion element called a so-called area sensor in which pixels are arranged in a plane, and some imaging devices of a monitoring system also use this.
In this case, the unit of the obtained image is an image having an aspect ratio of 1: 1 or 3: 4 corresponding to the angle of view of the area sensor, and the target detection process is generally performed in units of this image.
When continuously moving the shooting direction while shooting such an image, it is necessary to shorten the exposure time in order to suppress the image flow due to the movement. However, if the imaging target is dark, the sensitivity cannot be gained, which is disadvantageous. .

  Also, as such an imaging apparatus, a camera that outputs an NTSC or PAL standard video signal is often used, but target detection processing is performed in real time on an image having a frame rate of 30 to 60 Hz. Then, it is necessary to simplify the processing contents with a small amount of calculation or increase the hardware scale.

In view of this, there is also a method in which a wide monitoring area is divided into tiles and shot by moving the shooting direction stepwise with the angle of view and stopping the movement during shooting. In this way, the frame rate can be lowered, thereby making it possible to use precise target detection processing, and by performing frame integration or the like, the substantial photocharge accumulation time can be increased, so that the sensitivity is also improved.
However, the movement time of the visual axis becomes a blind time in monitoring, and the possibility of missing a target that moves at a timing that just crosses the boundary of the imaging area in the form of facing the scanning direction increases, and the visual axis repeats acceleration and deceleration. Since it takes a considerable amount of time to travel, the efficiency is poor and the search cycle tends to be long.

In any case, if all or part of the optical system is rotated, the image will be rotated. However, if an area sensor is used, there is no particular problem in obtaining an image, and an image as a rotated state can be obtained. Can do.
However, for example, when the monitoring area is photographed in a tile shape, image rotation compensation is necessary to continuously connect adjacent images, and a mechanism for optically compensating for the image rotation is also provided.

For the reasons already mentioned, it is more convenient to image the monitoring area at once by using a line sensor in some wide-area monitoring imaging devices, but in conventional imaging devices, optics for compensating for image rotation The parts had the following problems.
First, since the optical image rotation compensation component is a relatively large component composed of a prism or a mirror, it becomes an obstacle to downsizing of the entire apparatus, and power consumption is also increased to drive the component. Large prisms are expensive and occupy a large part of the cost of the apparatus.
Second, in order to make the optical image rotation compensation component as small as possible, it is necessary to focus the light beam in the vicinity of the component, which restricts the degree of freedom in designing the optical system. This is not only a factor for inhibiting downsizing, but also a factor for deteriorating sensitivity by limiting the aperture diameter.
Thirdly, in order to reduce the size of the optical image rotation compensation component, it is effective to form the optical image rotation compensation component with a prism, but the transmittance decreases with the prism. This is especially true in the infrared.

In addition, when a defect occurs in some pixels of the line sensor, a linear insensitive area is formed on the captured image, which causes the target to be overlooked when the monitoring target is a minute target, which is a fatal problem as a monitoring device. It becomes.
Such a defective pixel is also a problem in the area sensor. Especially, in the area sensor, in addition to the problem that the defective pixel becomes an insensitive area, the defect may be recognized as an incorrect target, for example, as a bright spot. is there.

Furthermore, calibration is important because infrared photoelectric conversion elements have many uneven sensitivity and defective pixels, and it is necessary to calibrate so that a uniform surface such as a shutter can be inserted in the middle of the optical system.
However, the installation of the shutter inside the apparatus is an obstacle to downsizing, and there is a problem that the sensitivity is deteriorated if a calibration means such as a shutter is not provided.
In particular, in infrared imaging devices, the photoelectric conversion element characteristics fluctuate significantly, and calibration may be required frequently during operation. However, since shooting is interrupted during calibration, there is a problem of blind time in monitoring. there were.

  The present invention has been made to solve such a problem, and an optical image rotation compensation component in an imaging apparatus for searching a wide area by changing a shooting direction by moving a part of an optical system. The purpose is to obtain an image pickup device that does not interrupt shooting by calibration because it achieves miniaturization, power saving, and high sensitivity, and does not require dedicated parts for calibrating photoelectric conversion elements. And

  An imaging apparatus according to the present invention includes a two-dimensional photoelectric conversion element (area sensor) that performs photoelectric conversion, an optical apparatus that forms an image on the photoelectric conversion element, and a part or the whole of the optical apparatus by moving the optical apparatus. A visual axis driving device for changing the photographing direction, a visual axis direction sensor for detecting a visual axis driving amount by the visual axis driving device, and a visual axis detected by the visual axis direction sensor while controlling the visual axis driving device. A visual axis control device that calculates the amount of rotation or movement of the image formed by the optical device from the axis drive amount, and an image rotation amount calculated by the visual axis control device for an image digitized by the photoelectric conversion element Frame integration while moving the image based on the amount of movement of the image calculated by the visual axis control device. A frame integration processor that electronically corrects the rotation of the image formed on the photoelectric conversion element surface caused by movement in the shooting direction based on the detected visual axis drive amount. is there.

  According to the present invention, there is no need for an optical component for rotating an image, which is effective for downsizing, power saving, and cost reduction of the apparatus. In addition, calibration can be performed without the need for dedicated parts for calibration and without interruption of photographing.

Embodiment 1 FIG.
Embodiment 1 according to the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing Embodiment 1 of the present invention. As an example, an imaging device portion of a search device mounted on an aircraft is simply described.

Incident light 1 enters from the opening of the optical device 2, is bent by a mirror 3 that is a component of the optical device 2, and forms an image on the photoelectric conversion element 4.
The element output of the photoelectric conversion element 4 is input to the element output correction processor 5, and the gain and DC component variation of each pixel are corrected according to the preset element correction data storage 6, and the output signal of the defective pixel is also corrected. Is subjected to correction processing such as replacement with a signal of an adjacent normal pixel, and becomes an original image signal.
Note that such correction processing may be unnecessary depending on the characteristics of the photoelectric conversion element.

The EL frame 7 is an operation unit for swinging (driving) the imaging direction (visual axis) up and down, and a part of the optical device 2 (all examples are shown in FIG. 1) around the EL rotation center 8. Can be rotated.
The visual axis drive device (EL) 9 drives this, and the visual axis drive amount is measured by the visual axis direction sensor (EL) 10 as drive angle information, for example.
Here, if the mirror 3 is adjusted so as to be driven by a half of the driving angle of the EL frame 7, the incident light 1 always forms an image on the photoelectric conversion element 4 even if the EL frame 7 rotates up and down.
The AZ frame 11 is an operation unit for swinging the visual axis to the left and right, and can be rotated around the AZ rotation center 12.
The visual axis driving device (AZ) 13 drives this, and the visual axis driving amount is measured by the visual axis direction sensor (AZ) 14 as driving angle information, for example.
The EL frame 7, the visual axis drive device (EL) 9, and the visual axis direction sensor (EL) 10 are all included in the AZ frame 11.

  In Embodiment 1, since the imaging apparatus is mounted on an aircraft, the attitude of the imaging apparatus also varies with respect to the inertial space due to a change in the attitude of the aircraft. Therefore, the visual axis control device 15 is based on the posture / azimuth angle of the imaging device obtained by the posture sensor 16 so that the visual axis is scanned at a predetermined angular velocity with respect to the inertial space. (EL) 9, the visual axis drive device (AZ) 13 is controlled.

  In the case of the configuration as shown in FIG. 1, although the image is rotated with the left-right movement of the shooting direction in the AZ rotation, the image rotation does not occur even if the shooting direction is moved up and down in the EL rotation. This is because in FIG. 1 the optical device 2 has a simple configuration consisting of a single lens and a mirror in order to make the drawing easier to understand, but the actual optical device has a complicated structure in which a plurality of mirrors and lenses are mixed. Therefore, the image is rotated even by EL rotation.

  FIG. 2 illustrates an image formed on the photoelectric conversion element 4 by the above mechanism. Here, in order to make the vertical and horizontal directions easy to understand, the photoelectric conversion element 4 is a two-dimensional element having a larger number of horizontal pixels than the vertical number of pixels, and is drawn as a horizontally long rectangle.

An image 17 is an image formed on the photoelectric conversion element 4, and illustrates a situation in which scanning is performed while performing AZ rotation of the visual axis.
As the time progresses to t0, t1, and t2, the center of the imaging area moves (that is, scans) in the AZ rotation direction, but at the same time, the image rotates around the center of the imaging area, and the image translates and rotates. Will be a complex movement. Therefore, the obtained image is rotationally converted by signal processing, and the rotational movement that occurs secondary by the movement of the visual axis is canceled.

Specifically, in the configuration of FIG. 1, when the visual axis is rotated 180 ° around AZ, the scene behind the scene appears upside down, which means that the image rotates as much as the AZ rotation angle. Means. Therefore, if the AZ rotation amount is measured by the visual axis direction sensor (AZ), and the image is rotationally converted by an angle measured in the direction opposite to the image rotation direction by the AZ rotation, the obtained image has the vertical direction. It will remain.
Even if the direction of the visual axis is not changed, the image is rotated when the body on which the imaging device is mounted rotates about the visual axis direction. For example, if the aircraft rolls upside down while shooting in front of the aircraft, the resulting image will also be upside down. Also in this case, if the image is reversely converted by the roll rotation amount measured by the attitude sensor 16, the top and bottom of the obtained image are always maintained.

In this operation, an image can be obtained within the viewing angle indicated by the circle 18 regardless of the direction of the visual axis. Therefore, for the rectangular area A19 which is a rectangle inscribed in the circle 18, the rectangular area B20 and the rectangular area C21 obtained at the following time are shifted and added by the scanning movement amount 22 corresponding to the parallel movement by scanning. , An integrated image 23 is obtained.
However, since it is often difficult to reduce distortion sufficiently in a practical optical device, even if an image 17 taken at each time is added only by rotational conversion and parallel movement for scanning, they do not overlap correctly. . For this reason, distortion correction is performed before rotational conversion.

Returning to FIG. 1 again, the above process will be described.
The original image signal is input to the distortion correction processor 24, and the distortion due to distortion at each point on the image is corrected by the distortion data storage 25 for storing the distortion data measured in advance.
Subsequently, the visual axis control device 15 is obtained by the attitude sensor 16 and the EL angle information obtained by the visual axis direction sensor (EL) 10 and the AZ angle information obtained by the visual axis direction sensor (AZ) 14. The rotation angle from the reference of the image formed on the photoelectric conversion element 4 is calculated from the attitude information of itself, and a value obtained by multiplying the rotation angle by (-1) is used as the required rotation amount, and this is rotated. This is given to the processor 26.

Here, the reference is such that the direction in which the image flows by the movement of the visual axis after the reverse rotation is the side direction of the photoelectric conversion element 4 (long side direction in the first embodiment). It is a direction and can be obtained by calculation from the above information and the direction in which the visual axis is moved (that is, the scanning direction).
The image rotation processor 26 performs signal processing on the image signal obtained as the output of the photoelectric conversion element 4 and performs rotation conversion by an angle given from the visual axis control device 15.
The image signal thus obtained is input to the frame integration processor 27, and the signals are added while shifting the image by the scanning movement amount calculated by the visual axis control device 15 (scanning movement amount 22 in FIG. 2). Thus, frame integration is performed.

According to this embodiment, it is not necessary to use an optical component for compensating for the rotation of the image, which leads to downsizing, power saving, and cost reduction of the apparatus.
In addition, such an optical component is a major factor that lowers the transmittance, particularly in an imaging device using infrared rays. However, when this is eliminated, the transmittance is improved and the sensitivity of the imaging device is improved.
Further, since it is not necessary to focus the light beam in the vicinity of such optical components, the design restriction conditions are reduced, and the optimum design of the optical device becomes possible. This is advantageous in improving sensitivity and downsizing.
In addition, a wide area can be generated as a single image while using the area sensor, and the total photocharge accumulation time for obtaining a signal for one pixel can be very long as an effect of frame integration. Higher sensitivity can be obtained.
Further, in addition to the above effect, in this embodiment, since the signal of one pixel in the finally obtained image is obtained as an integrated value of a plurality of pixel signals on the photoelectric conversion element, it is generated by a defective pixel or the like. The influence of abnormal output is distributed to a plurality of pixels in the final image, and as a result, the influence of defective pixels can be reduced.

FIG. 3 illustrates this effect.
As an example, a cloud image 29 is shown in the image 28 before the image rotation processing, and an abnormal signal 30 due to abnormal pixels also appears on the image.
As time advances to t0, t1, and t2, the cloud image 29 moves in a complicated manner due to scanning of the visual axis and image rotation that occurs along with it, but the abnormal signal 30 due to abnormal pixels always appears at the same position. This means that, in the image 31 after the image rotation process, the abnormal signal 30 due to the abnormal pixel moves around more complicatedly.
Therefore, in the output image 32 of the frame integration processor that has been processed so that the scenes are correctly superimposed, the image of the abnormal signal 30 due to abnormal pixels is blurred and becomes a blurred abnormal signal 33.
That is, the dot-like pattern due to abnormal pixels is not noticeable.

Embodiment 2. FIG.
The second embodiment according to the present invention will be described below with reference to the drawings.
FIG. 4 is a block diagram showing Embodiment 2 of the present invention, which is obtained by adding processing to Embodiment 1. In FIG.

  The output of the element output correction processor 5 is input to the distortion correction processor 24, and thereafter proceeds to the same processing as in the first embodiment, but simultaneously the image signal is also input to the simple frame integration processor 34. Here, the simple frame integration processor 34 directly performs frame integration on the image signal without performing positional conversion such as rotation and movement.

FIG. 5 illustrates an image signal obtained by the above mechanism, and is obtained by adding a diagram to FIG. As shown in FIG. 3, in the output image 32 of the frame integration processor, the image of the object to be photographed is clearly shown, and the dot-like image generated by the abnormal pixels becomes thin and inconspicuous.
However, if the frame integration is simply performed without performing the positional conversion, the cloud image 29 to be imaged is moved by scanning, as shown in the output image 35 of the simple frame integration processor. However, since the image signal generated by the image rotation is blurred as in the blurred cloud image 36, the abnormal signal 30 due to the abnormal pixel is always generated in the same pixel, so that it is emphasized and clearly displayed as the integrated abnormal signal 37. The resulting image (point).

Returning to FIG. 4 again, processing for the image signal will be described.
The abnormal pixel detection processor 38 detects an isolated bright spot / dark spot of about 1 to 2 pixels from the correction image signal passed from the simple frame integration processor 34. This can be done by comparing the pixel signal to be inspected with the average value of a plurality of pixel signals around it, and judging that it is an isolated bright spot / dark spot when there is a difference of a predetermined value or more, There are a plurality of methods such as making a determination based on a value obtained by obtaining “fixed value” as a fixed multiple of the standard deviation of a plurality of pixel signals in the vicinity without setting a fixed value.

The abnormal pixel detection processor 38 determines that the pixel thus detected is an abnormal pixel, and corrects the element correction data storage 6 for the pixel position. Replace with a signal. Here, instead of modifying the element correction data storage 6, a dedicated database may be provided.
By doing so, it is possible to detect and correct abnormal pixels.

Normally, detection of such abnormal pixels has been performed by inserting a uniform surface such as a shutter into the optical path. However, according to this embodiment, since the shutter is unnecessary, the size of the apparatus can be reduced. Power saving and cost reduction are possible.
In addition, in the abnormal pixel detection using the shutter, the imaging target cannot be taken at the time when the shutter is inserted, and this is a blind time. However, in this embodiment, abnormal pixels can be detected at the same time while taking an image. There is a merit that time does not occur.

Embodiment 3 FIG.
The third embodiment according to the present invention will be described below with reference to the drawings.
FIG. 6 is a block diagram showing Embodiment 3 of the present invention. A part of FIG. 4 is taken out and modified in accordance with the embodiment of the present invention. The photoelectric conversion element 4, the element output correction processor 5, and the image rotation processor 26 are exactly the same as those in FIG.
The offset detection processor 39 receives the correction image signal from the simple frame integration processor 34 and acquires the signal intensity at the position of the abnormal pixel obtained as the output of the abnormal pixel detection processor 38.
At the same time, the signal strength of the surrounding pixels of the abnormal pixel is also obtained from the original image signal, and the difference between the average signal strength of the surrounding pixels and the signal strength of the abnormal pixel is calculated.

Normally, a newly generated abnormal pixel is a defective pixel that is completely useless if its output is near the upper limit (saturation level) or lower limit (0 level) of the signal, but its output has an intermediate value. When outputting, it can be said that it is a temporary abnormal state in which some offset component is added to the DC component of the signal.
Such an offset component may be suddenly generated due to a structure caused by impurities in the photoelectric conversion element, or may gradually appear due to low-frequency noise called 1 / f fluctuation. In any case, there is sensitivity, and the clogging is only put on the surrounding signals.

The offset detection processor 39 checks the signal intensity of the abnormal pixel, and if it is at a level at which it is determined to be a defective pixel, the pixel is regarded as a defect as described in the second embodiment, and the element correction data storage 6 By correcting the data of the defective pixel, it is replaced with another signal thereafter.
Conversely, if it is determined that the abnormal pixel can be corrected, the offset amount is obtained by dividing the difference between the average signal intensity of the surrounding pixels and the signal intensity of the abnormal pixel by the number of frame integrations, and thereby the element correction data storage 6 The offset correction data of the pixel is corrected, and thereafter, the offset value of the pixel is appropriately extracted.

  By doing in this way, in addition to the effect in Embodiment 1 or Embodiment 2, the pixel which can be used can be utilized effectively.

Embodiment 4 FIG.
The fourth embodiment according to the present invention will be described below with reference to the drawings.
FIG. 7 is a configuration diagram showing the fourth embodiment of the present invention, and a part of the first embodiment is changed.
In Embodiment 1, since the imaging apparatus is mounted on an aircraft, the imaging apparatus also changes its attitude with respect to the inertial space due to a change in the attitude of the aircraft. Therefore, the visual axis control device 15 receives the posture information of the imaging device itself from the posture sensor 16 and drives the visual axis so that the visual axis direction moves at a constant angular velocity with respect to the inertial space without being influenced by the posture variation of the device itself. Although a drive command is sent to the device (EL) 9 and the visual axis drive device (AZ) 13, the time lag between them is at least about several tens to several hundreds msec in a practical device. For this reason, if the angular velocity of posture variation is large, the angle variation within the time lag is large, and the shift in the photographing direction due to the posture variation cannot be suppressed to a level sufficiently smaller than the pixel size.
That is, in the case of an imaging device mounted on an aircraft that moves violently, the image is shifted due to this time lag, and even if the frame integration as shown in the first embodiment is performed, the images that should be overlapped do not overlap, The S / N of the image is rather lowered.

Therefore, in the fourth embodiment, posture information output by the posture sensor 16 is first received by the posture variation compensation device 40, and the posture variation compensation device 40 records posture information (this is referred to as “posture information A”). At the same time, the posture information is sent to the visual axis control device 15 as it is.
The visual axis control device 15 appropriately drives the visual axis drive device (EL) 9 and the visual axis drive device (AZ) 13 and is measured by the visual axis direction sensor (EL) 10 and the visual axis direction sensor (AZ) 14. The actual visual axis drive amount and the required image rotation amount for returning the image that has been rotated in accordance with the change in the visual axis direction are sent back to the posture variation compensator 40, and the posture variation compensator 40 uses this value. Record. (This will be referred to as “visual axis drive amount B” and “required image rotation amount C”)

  Here, the posture information measured by the posture sensor 16 at the same time (this time is referred to as “time D”) when the actual visual axis driving amount is measured is also sent to the posture variation compensating device 40. Sent and recorded. The synchronicity at time D (which will be referred to as “attitude information E”) can be guaranteed in the design by synchronizing the movement of each part of the system with a single clock.

An image photographed in the visual axis direction at time D (hereinafter referred to as “image F”) is output through the element output correction processor 5, but this image was truly photographed. The direction is shifted by the posture change corresponding to the time lag than the direction. This deviation corresponds to the difference between the posture information A and the posture information E.
Therefore, the posture variation compensation device 40 calculates a required movement amount for correcting the photographing direction deviation of the image F from the posture information A and the posture information E (hereinafter referred to as “required movement amount G”), and further Using the required image rotation amount C, a new required image rotation amount (hereinafter referred to as “required image rotation amount H”) is calculated and transmitted to the image rotation processor 26 and the image translation processor 41, respectively. And correct the image.

At this time, the time until the required movement amount G and the required rotation amount H are aligned is often longer than the time until the image E comes out of the element output correction processor 5, and therefore the image E is temporarily stored. It is necessary to wait. Therefore, the delay processor 42 is placed after the element output correction processor 5, and the image information is temporarily stored in the memory block configured in a FIFO form, and then output after a predetermined time to output the image E. Delay for a certain time.
Depending on the configuration, the image E may come out later in time, but in this case, the required movement amount G and the required rotation amount H may be delayed, and this can be configured on a small scale. The memory in the attitude variation compensation device 40 may be configured as a FIFO buffer.

  By adopting the above-described configuration, it is possible to suppress the tracking direction tracking error to be negligible even in an environment where the posture change is severe, and the frame integration can be effectively operated.

Note that the image rotation process and the image translation process can be performed separately. However, when position correction is performed with a width of one pixel or less in each process, for example, the position correction of 0.5 pixel is a signal of one pixel. Dividing into two pixels causes blurring. Since this blur occurs every time the position correction processing is performed once, blurring can be reduced by combining the position correction processing at one time.
Therefore, the image rotation processor 26 and the image translation processor 41 of the fourth embodiment are combined into one, and the position movement amount by both the rotation process and the translation process is calculated simultaneously for each pixel, and the position correction process is performed at once. If performed, an imaging device with less blur can be obtained.

  Further, the unification of the above processing is extended to the distortion correction processing, and the processing of the distortion correction processing unit 24 of the fourth embodiment is added to be integrated into one position correction processing, thereby further reducing blur. An imaging device can be obtained.

It is a block diagram which shows the structure of the imaging device by Embodiment 1 of this invention. It is a figure which shows the image generation procedure in the imaging device by Embodiment 1 of this invention. It is a figure explaining the characteristic of the acquired image in the imaging device by Embodiment 1 of this invention. It is a block diagram which shows the structure of the imaging device by Embodiment 2 of this invention. It is a figure explaining the output image of the simple frame integration processor in the imaging device by Embodiment 2 of this invention. It is a block diagram which shows the structure of the imaging device by Embodiment 3 of this invention. It is a block diagram which shows the structure of the imaging device by Embodiment 4 of this invention.

Explanation of symbols

  1 incident light, 2 optical device, 3 mirror, 4 photoelectric conversion element, 5 element output correction processor, 6 element correction data storage, 7 EL frame, 8 EL rotation center, 9 visual axis drive device (EL), 10 view Axis direction sensor (EL), 11 AZ frame, 12 AZ rotation center, 13 visual axis drive device (AZ), 14 visual axis direction sensor (AZ), 15 visual axis control device, 16 posture sensor, 17 image, 18 circle, 19 rectangular area A, 20 rectangular area B, 21 rectangular area C, 22 scanning movement amount, 23 integrated image, 24 distortion correction processing unit, 25 distortion data storage unit, 26 image rotation processing unit, 27 frame integration processing unit, 28 Image before image rotation processing, 29 Cloud image, 30 Abnormal signal due to abnormal pixels, 31 Image after image rotation processing, 32 The output image of the Lame integration processor, 33 The blurred abnormal signal, 34 The simple frame integration processor, 35 The output image of the simple frame integration processor, 36 The blurred cloud image, 37 The integrated abnormal signal, 38 The abnormal pixel detection processing 39, an offset detection processor, 40 an attitude variation compensator, 41 an image translation processor, 42 a delay processor.

Claims (6)

A two-dimensional photoelectric conversion element for performing photoelectric conversion;
An optical device for forming an image on the photoelectric conversion element;
A visual axis drive device that changes the shooting direction by moving part or all of the optical device; and
A visual axis direction sensor for detecting a visual axis drive amount by the visual axis drive device;
A visual axis control device that controls the visual axis drive device and calculates a rotation amount or a movement amount of an image formed by the optical device from a visual axis drive amount detected by the visual axis direction sensor;
An image rotation processor that reversely rotates the image by the amount of image rotation calculated by the visual axis control device with respect to the image digitized by the photoelectric conversion element;
A frame integration processor that integrates the image obtained by the image rotation processor while moving the image based on the amount of movement of the image calculated by the visual axis control device;
With
An image pickup apparatus that electronically corrects rotation of an image formed on the surface of the photoelectric conversion element caused by movement in the photographing direction based on the detected visual axis driving amount. An element correction data storage for storing element correction data for correcting characteristic variations and abnormal values such as gain and DC component of each pixel in the photoelectric conversion element;
An element output correction processor for correcting the output signal of the photoelectric conversion element using the element correction data storage;
A simple frame integration processor for frame integrating the original image signal obtained from the element output correction processor;
From the image signal for correction output by the simple frame integration processor, an abnormal pixel that outputs a difference in intensity greater than or equal to a predetermined intensity of the output signal of the surrounding pixels is extracted, or a predetermined standard deviation of the output signal of the surrounding pixels is determined. An abnormal pixel detection processor that extracts an abnormal pixel that outputs a multiple or more and corrects the element correction data of the element correction data storage for the abnormal pixel;
With
The imaging apparatus according to claim 1, wherein an abnormal pixel in the photoelectric conversion element is detected and the element correction data is automatically corrected. For the abnormal pixel output by the abnormal pixel detection processor, a difference between the abnormal pixel signal and the corresponding surrounding pixel signal is detected from the correction image signal output by the simple frame integration processor, and thereby the abnormal pixel is detected. An offset detection processor for correcting the element correction data of the element correction data storage by obtaining the offset amount of the DC component in the output signal;
The imaging apparatus according to claim 2, wherein a pixel in which an offset amount of a DC component in the photoelectric conversion element is abnormal is detected, and the offset amount is automatically estimated or calibrated. . An attitude sensor for detecting the attitude of the imaging device itself;
An image translation processor for translating the captured image;
A delay processor that delays the image signal before rotating the captured image or translating the captured image;
The output information of the posture sensor is relayed and given to the visual axis control device, and the rotation amount of the image and the parallel movement amount of the image are calculated from the output information of the posture sensor and the output information of the visual axis control device. Attitude variation compensation device for supplying the image rotation processor and the image translation processor with
With
In an environment where the posture of the photographing apparatus itself fluctuates, the mechanism that detects the posture and adjusts the photographing direction so as to cancel the variation in the posture follows the detected posture information. 4. The imaging apparatus according to claim 1, wherein a tracking error caused by a time difference until the photographing direction is adjusted is compensated by image processing. 5. The imaging apparatus according to claim 4, wherein an image rotation process by the image rotation processor and an image translation process by the image translation processor are collectively processed. A distortion correction processor for correcting distortion of an image based on distortion data set in advance for an original image signal obtained from the element output correction processor;
5. The imaging apparatus according to claim 4, wherein an image rotation process by the image rotation processor and an image translation process by the image translation processor are collectively processed.

Images