JP3089229B2 - Imaging device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus used for picking up an image in a wide visual field at a high resolution.

[0002]

2. Description of the Related Art There is known a vehicle on which a flying object is mounted and which launches the flying object toward the target when a predetermined target exists in a predetermined ground area. When the operator operating this vehicle recognizes the presence of the target in the ground area, in other words, identifies the object existing in the ground area and recognizes that the object is the target, He is firing a flying object at an object. In this vehicle,
An imaging device used by an operator to identify an object is mounted, and the operator looks at an image captured by the imaging device, and obtains the shape, size, brightness, and the like of the object obtained from the image. Based on the information, it is determined whether or not the object being imaged is a target.

[0003] Such an image pickup apparatus is widely used so that an operator can quickly identify an object over a wide ground area and can judge with high accuracy whether or not it is a true target. It is desirable to have a viewing angle and high resolution. There are two types of imaging devices having a wide viewing angle and high resolution: (1) a device using a large-diameter lens, and (2) a panoramic imaging device.

In the former (1) imaging apparatus 1 using a large-diameter lens, as shown in FIG. 9, a received optical image is formed on a light-receiving portion of a detector 4 by an imaging optical system 3, and this detector Imaging means 2 for converting an optical image into an image signal by means of an imaging device 2
And an afocal optical system 5 for guiding an optical image to the imaging means 2. A large-diameter lens 6 is used as a lens constituting the afocal optical system 5. The imaging apparatus 1 has a wide viewing angle by increasing the amount of an optical image guided to the imaging unit 2 and guiding a highly reliable optical image to the imaging unit 2 by the afocal optical system 5 having the large-diameter lens 6. Image with high resolution over the entire imaging range.

In the latter (2) panoramic imaging apparatus, a wide ground area is divided into a plurality of areas using the same imaging means as the imaging means 2 of the imaging apparatus 1 shown in FIG. For each region, change the imaging direction and sequentially capture images,
By combining a plurality of images obtained in this way as one image, an image similar to an image captured at high resolution over a wide viewing angle can be obtained.

[0006]

However, in the apparatus 1 using the large-aperture lens 6 as described above, there is a problem that the configuration of the apparatus 1 is enlarged by the large-aperture lens 6. Also, (2) the panoramic imaging apparatus has a large-diameter lens 6 compared to the apparatus 1 of (1).
Is not required, the configuration does not increase in size. However, since the imaging direction must be changed, there is a problem that the update cycle of the obtained image is lengthened. Also, regardless of whether the imaging direction needs to be changed or not, the time required to convert an optical image from an imaging target into an image signal is longer than that of the apparatus 1, and as a result, the update cycle is lengthened. Will do.

An object of the present invention is to provide an imaging apparatus having a wide viewing angle and a high resolution while preventing a large-sized configuration and a long update cycle.

[0008]

According to a first aspect of the present invention, there is provided an afocal optical system for reducing an optical image from an object to be imaged, and having a predetermined resolution and converting a received optical image into an image signal. A first imaging unit, a second imaging unit having a resolution higher than the predetermined resolution and converting the received optical image into an image signal, and an optical path of the optical image reduced by the afocal optical system, Switching means for selectively allocating and switching time between the first and second imaging means;
An image signal output from the image pickup means, a target image extraction means for extracting an image region including a predetermined target image, and an afocal optical system interposed between the switching means and the second image pickup means Changing the optical path of the optical image that has passed through to the imaging direction corresponding to the image area extracted by the target image extracting unit, and guiding a part of the optical image that has passed through the afocal optical system to the second imaging unit. Means, and an area corresponding to the image obtained by the second image pickup means in the image obtained by the first image pickup means is replaced with an image obtained by the second image pickup means based on each image signal output from the first and second image pickup means. And an image synthesizing unit that outputs a synthesized image signal representing an image obtained by synthesizing the images obtained by the first and second imaging units. When the light path changing means changes the light path, an area corresponding to the image stored in the frame memory of the image by the first image pickup means and the area corresponding to the image by the second image pickup means is stored. An image pickup apparatus characterized in that an image before an optical path change stored in a frame memory is displayed in a frozen state instead of an image stored in a frame memory.

According to the present invention, the optical image of the object to be imaged is reduced by the afocal optical system, and the reduced optical image is selectively switched to the first and second imaging units by the switching unit and guided. . The first imaging unit converts the received optical image into an image signal with a predetermined resolution, and the image signal is output to the target image extraction unit and the image control unit, respectively. The target image extracting unit performs image processing on an image signal from the first imaging unit, and extracts an image region including a predetermined target image from the image obtained by the first imaging unit. As a method of extracting this image region, for example, there is a contrast box method, and this method is a method of extracting a region having a large difference in luminance between the central portion and the outer peripheral portion as an image region including the target image. This method is useful when, for example, using an infrared optical image to extract an image area including an image of a relatively high temperature portion of the imaging target as an image area including the target.

The second imaging means converts the received optical image into an image signal with higher resolution than the first imaging means. An optical path changing unit is interposed between the second imaging unit and the switching unit, and the optical path changing unit changes an optical path of an optical image guided to the second imaging unit. Accordingly, the second imaging unit can perform imaging without changing the direction of the second imaging unit itself, in a state where the imaging direction is changed due to the change of the optical path by the optical path changing unit.

The light path changing means changes the light path of the optical image so that the image pickup direction of the second image pickup means is the image pickup direction corresponding to the image area extracted by the target image extraction means, and changes the light path to the second image pickup means. Lead. As described above, the optical path of the optical image guided to the second imaging unit is changed, and thereby, an area corresponding to the extracted image area to be imaged is imaged by the second imaging unit. As an example, the target image extracting unit sets a coordinate axis on the image obtained by the first imaging unit,
A signal representing the center coordinates of the extracted image area of the image captured by the imaging unit is output to the optical path changing unit. The light path changing means changes the light path of the optical image guided to the second image pickup means based on the signal so as to be the center of the extracted image area.

The image signal of the image picked up by the second image pickup means is output to the image control means. The image control unit combines the images from the first and second imaging units such that a portion of the image obtained by the first imaging unit that corresponds to the image obtained by the second imaging unit is replaced with an image obtained by the second imaging unit. Can be. The composite image signal representing the composite image can be output to a display unit such as a monitor, and the composite image can be displayed by the display unit.

The first imaging means having a relatively low resolution has a relatively short time required for converting an optical image received from an imaging object having a predetermined imaging range into an image signal, and has a short image update cycle. On the other hand, the second, which has relatively high resolution,
The image capturing means increases the time required to convert an optical image received from an image capturing target having a predetermined image capturing range into an image signal, and obtains an image with relatively high reliability, although the image update cycle becomes longer. Can be. The imaging apparatus according to the present invention makes use of the advantages of the first and second imaging means, respectively, and synthesizes an image as described above.
It is possible to image an imaging target in an imaging range with a wide viewing angle with high resolution, and furthermore, it is possible to suppress an increase in the size of the apparatus and to suppress a prolongation of an image update cycle.

More specifically, the first imaging means comprises:
Although the resolution is relatively low, the reliability of the image is low, but the time required to convert an optical image into an image signal is short, so that an image can be captured over a wide viewing angle imaging range, for example, a large ground area. However, imaging can be performed in a short cycle, for example, in almost real time. The image control means can replace such a partial area of the image by the first imaging means, that is, the extracted image area extracted by the second imaging means. It is possible to obtain an image with high reliability taken at an extremely high resolution. That is, the image finally obtained is obtained by replacing the image of the extracted image area in the obtained image with the image obtained by the second imaging unit, which is taken at a high resolution, by taking a picture with a wide viewing angle by the first taking means. An image captured at a higher resolution than an image captured only by the first image capturing means can be obtained, and an update cycle of the finally obtained image is longer than an image captured only by the second image capturing means. It can be prevented from becoming too much. That is, it is possible to image a wide-field imaging target with high resolution. Further, since a large-diameter lens as in the conventional case is not required, the size of the apparatus is not increased.

As described above, there is no problem that the configuration becomes large or the image update cycle becomes long.
It is possible to quickly image an imaging target in a wide field of view, for example, a large ground area. In addition, an area which needs to be imaged with high resolution, such as a ground area, can be extracted by the target image extracting means based on predetermined extraction conditions, and can be imaged with high resolution. Also, of the imaging target, an area that needs to be imaged with high resolution is
Since the image is automatically extracted by the target image extracting means, the operator visually observes the image from the first image capturing means and selects an image area satisfying a predetermined condition as an area to be imaged with high resolution. There is no need to work and the workability is excellent.

[0016]

In such an image pickup apparatus, the image control means stores the image picked up by the second image pickup means in a frame memory, and stores the image in the frame memory when the light path changing means changes the light path. Before the image signal, that is, the light path changing means changes the light path,
The image captured by the second imaging unit is displayed in a frozen state.

Accordingly, when there are a plurality of image areas extracted by the target image extracting means in the image obtained by the first image pickup means, for example, the optical path is periodically changed by the optical path changing means to correspond to each extracted image area. The captured ground area is individually imaged by the second imaging means, and for one of the plurality of extracted image areas, a substantially real-time detailed continuous image imaged by the second imaging means is displayed, and the remaining As for the extracted image area, the image of the image signal stored in the frame memory can be displayed in a frozen state.

With this arrangement, when a plurality of extracted image areas exist, the operator can freeze the extracted image area even if the second imaging means is imaging another extracted image area. Although it is a past image, a detailed image by the second imaging means can be visually checked. Further, since the image stored in the frame memory is displayed in an area other than the area where the real-time image is displayed by the second imaging means, even when a plurality of extracted image areas overlap each other, The image from the second imaging means can be displayed with priority. Therefore, almost real-time new image information obtained from the second imaging means can be displayed as much as possible.

Further, since the image represented by the image signal stored in the frame memory can be displayed in a frozen state, for example, in order to determine whether an object existing in a predetermined ground area is a target or not. When an imaging device is used, a detailed image of an object can be displayed in a frozen state. In this case, the operator can easily determine whether or not the object is a target by visually checking the frozen image of the object.

[0021]

FIG. 1 is a system diagram showing a schematic configuration of an image pickup apparatus 11 according to an embodiment of the present invention, and FIG. 2 is a perspective view showing a vehicle 13 on which the image pickup apparatus 11 is mounted. It is. FIG. 1 shows, in principle, a configuration for guiding an optical image in an imaging device 11, and FIG.
It is a figure showing the state where 2 was fired. The imaging device 11 basically includes a first imaging unit 15, a second imaging unit 16, a switching unit 17, a target image extracting unit 18, an optical path changing unit 19, and an image combining unit 20, Further, an afocal optical system 35 is included.

The image pickup device 11 has a low resolution, but is capable of picking up images over a wide field of view.
And the second imaging unit 16 capable of capturing images at a high resolution although the visual field that can be captured is narrow, thereby capturing, for example, a wide ground area as a wide-field imaging target at a high resolution. It is a device for performing. In the present embodiment, the imaging device 11 is provided, for example, on a vehicle 13 on which a flying object 12 is mounted and the operator launches the flying object 12 toward the targets A1 to A3. The image captured by the imaging device 11 is such that the operator checks whether or not an object is present in the ground area 38, and determines that the confirmed object is the target A1.
It is used for identifying whether or not it is A3.

The first image pickup means 15 includes a first image forming optical system 21.
And a first detector 22. First imaging optical system 21
Is configured to have a predetermined imaging magnification by one or a plurality of lenses, and the ground area 3 to be imaged is
The first detector 22 forms an optical image based on the infrared rays from the first detector 8. The first detector 22 is realized by a scanning type photodetector, converts the infrared optical image into an analog electric signal, further converts the analog electric signal into a digital electric signal, and converts the digital electric signal into an image signal. Are output to the target image extracting means 18 and the image control means 20.

The second imaging means 16 includes a second imaging optical system 23
And a second detector 24. Second imaging optical system 23
Is configured to have an imaging magnification higher than that of the first imaging optical system 21 by one or a plurality of lenses.
To form an image. The second detector 24 has the same configuration as the first detector 22, and an image signal converted from the infrared optical image received by the second detector 24 is output to the image control unit 20.

As described above, the second image pickup means 16 has the second detector 24 similar to the first detector 22 and the second detector 24.
The imaging optical system 23 is configured to have a higher imaging magnification than the first imaging optical system 21. As a result, the second imaging unit 16 can generate an infrared optical image received from an area having the same viewing angle with a high light receiving element density, that is, a high pixel density, although the imaging range is smaller than that of the first imaging unit 15. It can be converted to an image signal. In other words, the second imaging unit 16 can obtain a detailed image captured with high resolution by reducing the viewing angle per pixel, that is, the instantaneous viewing angle, as compared with the first imaging unit 15. it can. That is, the first imaging unit 15 converts almost the entirety of the infrared optical image received by the imaging device 11 into an image signal with a predetermined number of pixels, and the second imaging unit 16 converts the infrared optical image received by the imaging device 11 into an image signal. The image signal is converted into an image signal with the same number of pixels as the first imaging unit in a partially enlarged state, so that the second imaging unit 16 can obtain a more detailed image than the first imaging unit 15.

The switching means 17 is substantially disk-shaped, and has a fan-shaped notch 2 cut out by two radial lines at one location in the circumferential direction.
7 on which the reflection mirror 7 is formed, and the reflection mirror 25
And a driving unit 26 that rotationally drives. The imaging device 11 is provided with an infrared optical image from the imaging target, for example, 10 upstream of the switching means 17 in the direction in which the optical image is guided.
An afocal optical system 35 composed of a plurality of lenses 33 and 34 is provided to reduce the size by a factor of one. Is led.

The afocal optical system 35 and the switching means 17 provide an optical image reduced by the afocal optical system,
The reflection mirror 25 is disposed so as to be guided to a position shifted radially outward from the center of a circle including the outer periphery of the reflection mirror 25. The reflection mirror 25 is rotationally driven by a drive unit 26 around a rotation axis O perpendicular to a flat reflection surface 25 a facing the afocal optical system 35. This rotation axis O is the optical axis C of the optical image immediately after passing through the afocal optical system 35.
With respect to the angle θ. The infrared optical image from the afocal optical system 35 is reflected by the reflecting surface 25 a or passes through the notch 27 according to the angular position of the reflecting mirror 25.

When the infrared optical image is applied to the reflecting surface 25a of the reflecting mirror 25, it is reflected by the reflecting surface 25a and guided to the first coupling optical system 21 of the first image pickup means 15. Further, the infrared optical image is formed by the notch 27 of the reflection mirror 25.
Is passed through the optical path changing means 19
It is guided to the imaging means 16.

The driving section 26 is realized by, for example, a stepping motor, and connects the reflecting mirror 25 to, for example,
It is rotationally driven at ３０30 Hz, preferably 30 Hz. Therefore, when the reflection mirror 25 is rotationally driven by the driving unit 26, the optical path of the infrared optical image that has passed through the afocal optical system 35 is changed to the first and second imaging units 15, 16.
Is selectively switched at a period of 10 to 30 Hz. The size of the notch 27 is appropriately selected according to the time distribution for guiding the optical images to the first and second imaging units 15 and 16.

The target image extracting means 18 is a first image pickup means 1
5, the image signal is subjected to image processing, an image area including a target image satisfying a predetermined extraction condition in the image by the first imaging unit 15 is extracted, and a signal representing the coordinates of the image area is extracted. Occur. The signal representing the coordinates of the extracted image area is provided to the light path changing unit 19.

The light path changing means 19 includes a switching means 17 and
It is interposed between the second imaging means 16. As described above, almost the entire optical image that has passed through the afocal optical system 35 is guided to the first imaging unit 15, while the second
Of the optical images that have passed through the afocal optical system 35, only the optical images that have passed through the optical path in some directions are guided to the imaging unit 16. The light path changing means 19 is a means for changing the direction of the light path of the optical image guided to the second imaging means 16, whereby the optical image from a partial area in any direction of the ground area 38 is changed. Whether or not to lead to the second imaging means 16 is changed.

The light path changing means 19 includes a pitch direction changing section 70, a yaw direction changing section 71, and an optical path control section 32. The pitch direction changing unit 70 includes a pitch direction changing mirror 28 and a driving unit 30 that drives the pitch direction changing mirror 28 by angular displacement.
Is a flat rectangular mirror, which is provided at a position where an optical image that has passed through the notch 27 of the reflecting mirror 25 is irradiated onto the reflecting surface 28a, and reflects the irradiated optical image. The drive unit 30 drives the pitch direction changing mirror 28 with high precision angular displacement about a horizontal angular displacement axis. The yaw direction changing unit 71 includes a yaw direction changing mirror 29 and a driving unit 31 that drives the yaw direction changing mirror 28 to be angularly displaced. The yaw direction changing mirror 29 is a flat rectangular mirror. The surface 29a is provided at a position where the optical image reflected by the pitch direction changing mirror 28 is irradiated, and reflects the irradiated optical image. The drive unit 31 drives the yaw direction changing mirror 29 with high precision angular displacement about a vertical angular displacement axis.

The pitch direction changing section 70 selects the direction of the optical path of the optical image guided to the second image pickup means 16 with respect to the pitch direction, and the yaw direction change section 71 guides the optical image guided to the second image pickup means 17. Is selected with respect to the yaw direction. By combining these pitch and yaw direction changing units 70 and 71, the second imaging unit 16
The direction of the optical path of the optical image guided to is selected.

The light path control unit 32 includes the target image extracting unit 1
8, a signal representing the coordinates of the extracted image area is provided. Based on this signal, an optical image from a partial area corresponding to the extracted image area of the ground area 38 is guided to the second imaging means 16. A command signal is output to the pitch direction changing unit 70 and the yaw direction changing unit 71. That is, the optical path control unit 3
2 in response to the command signal from
Drives the pitch direction changing mirror 28 and the yaw direction changing mirror 29, respectively, so that the imaging direction of the second imaging means 16 is changed.

The image control means 20 is provided with image signals from the first and second image pickup means 15 and 16, and based on these image signals, a second image of the image by the first image pickup means 15 is provided.
Image signals from the first and second imaging units 15 and 16 are combined so that an area corresponding to the image by the imaging unit 16 is replaced with an image by the second imaging unit 16. A composite image signal representing the composite image obtained by the composite processing is provided to the display means 52, and the composite image can be displayed by the display means 52.

FIG. 3 is a block diagram showing an electrical configuration of the image pickup device 11. In FIG. 3, the optical path of the optical image is
Shown by broken lines. The infrared optical image from the ground area 38 to be imaged is guided to the switching means 17 via the afocal optical system 35 as described above, for example.
As a result, the optical path of the optical image is selectively switched between the first imaging unit 15 and the second imaging unit 16. Switching means 1
7, the optical image is directly guided to the first imaging unit 15,
The light is guided to the second imaging unit 16 via the optical path changing unit 19.

The target image extracting means 18 includes the first image pickup means 1
Based on the image signal output from 5, a contrast box image processing is performed. In this contrast box image processing, first, a contrast box processed image 43 is generated according to the procedure shown in FIG. Specifically, FIG. 4 (1)
The image 39 from the first imaging unit 15 shown in FIG.
A plurality of brightness comparison areas 42a1 and 42a2 having a first comparison area 40 and a second comparison area 41 surrounding the first comparison area 40 as shown in (2) are moved from left to right in the uppermost row. Then, when the setting in that line is completed, the line is sequentially moved so as to move to the lower line and move from left to right and set in a matrix. At this time, each brightness comparison area 42
a1, 42a2, 42a3 ...; 42b1, 42b2, 4
2b3 ...; 42c1, 42c2, 42c3 ...; 42d1
(Hereinafter, when referring to an arbitrary brightness comparison area or collectively, it may be described as “brightness comparison area 42”), the average brightness of the first comparison area 40 and the second comparison area 41 The local contrast values, which are the differences from the average luminance, are calculated and obtained, and a contrast box processed image 43 shown in FIG. 4C is generated. That is, the contrast box processed image 43 is a signal in which the respective brightness comparison areas 42 are adjacent to each other without any gap, and represent the local contrast value of each of the brightness comparison areas 42,
The signal representing the contrast box processed image 43 is
It is stored in a memory (not shown).

After the contrast box processing image 43 is generated in this manner, the 3 × image shown in FIG.
3, 9 areas SA, SB, SC, SD, SE, SF,
The filter 44 divided into SG, SH and SL is shown in FIG.
As shown in (2), each area SA to SH of the filter 44
And SL and each luminance comparison area 42 are applied so as to correspond to each other. This filter 44 is provided from the upper left part to the lower right part in the same manner as the setting of the luminance comparison area 42 described above.
The brightness comparison area 42 that is adapted while moving one by one brightness comparison area 42 and satisfies the following conditional expression (1) is extracted. Further, XY coordinates are set for the image obtained by the first imaging means 15, and the center coordinate value of the extracted luminance comparison area 42 in XY coordinates is obtained.

(L> A) and (L> B) and (L> C) and (L> D) and (L ≧ E) and (L ≧ F) and (L ≧ G) and (L ≧ H) (1) Here, the local contrast values of the brightness comparison area 42 corresponding to the areas SA to SH and SL of the filter 44 shown in FIG.

As shown in FIG. 5B, the target candidate determination area SL located at the center of the filter 44 is changed to the upper leftmost luminance comparison area 42 of the contrast box processed image.
To the lower right luminance comparison area 42 in order,
Fit it. At this time, it is determined whether conditional expression (1) is satisfied. That is, the local contrast value of the target candidate determination area SL is larger than the local contrast values of the first comparison determination areas SA to SD adjacent to the target candidate determination area SL on the upstream side in the scanning direction. Second comparison determination area SE adjacent on the downstream side in the scanning direction
ＳSH is equal to or greater than each local contrast value, and the target candidate determination SL when both conditions are satisfied
Is extracted as an image area including the target image.

By performing the contrast box image processing on the image signal output from the first image pickup means 15 in this way, the brightness comparison area 42 having a locally large brightness difference is obtained.
Exists, the center coordinates of the brightness comparison area 42 can be extracted as the center coordinates of the extracted image area. In this way, the center coordinates of the luminance comparison area 42 having the largest luminance difference in the contrast box processing image 42 corresponding to the image obtained by the first imaging unit 15 are not used as the center coordinates of the extracted image area, Since the central coordinates of the luminance comparison area 42 having a locally large luminance difference are set as the central coordinates of the extracted image area, it is possible to extract a plurality of extracted image areas from the image obtained by the first imaging unit 15, and In the present embodiment, if there is an object such as a target object or an object that satisfies the same condition as the target object, the object has a higher luminance difference than the surroundings, the object or the like is raised by the second imaging unit 15. Images can be taken at a resolution.

The center coordinates of the brightness comparison area 42 where the local contrast value is equal to or greater than a predetermined value are not used as the center coordinates of the extracted image area, but the center coordinates of the brightness comparison area 42 having a locally large brightness difference are extracted. Since the center coordinate of the image area is set and the determination is made in accordance with the conditional expression (1), the brightness comparison area 42 having a relatively large difference in brightness exists adjacently.
Target candidate determination area SL and target candidate determination area SL
, Even if there is a plurality of comparison determination areas consisting of the first and second comparison areas SA to SH, that is, the area to which the filter 44 is applied, the brightness comparison areas having the same local contrast value are adjacent to each other. One extracted image region is extracted using the center coordinates of the luminance comparison region 42 having the largest luminance difference as the center coordinates of the extracted image region without causing a problem such as being unable to be extracted because of being extracted or being extracted as a plurality of extracted image regions. can do.

With this configuration, as in the present embodiment, in a configuration in which an extracted image region is extracted based on a difference in luminance of infrared rays, for example, in a configuration in which an image of an object whose temperature is higher than the surroundings is set as a target image, the extracted image region is extracted. For example, when the object is the vehicle A1 as shown in FIG. 2 and the image of the vehicle A1 by the first imaging unit 15 is almost the same size as the comparison determination area, the engine, the muffler, and the Although there are a plurality of comparison determination areas 42 having a relatively large luminance difference in the comparison determination area based on a plurality of heat sources such as the occupants of the vehicle, the vehicle A1 is extracted as one extracted image area. , Ground area 3
Despite the presence of a plurality of objects A1 to A3 as targets in FIG. 8, it is possible to eliminate the problem of intensive imaging of one object, for example, the object A1 by the second imaging means 16.

The target image extracting means 18 sets the coordinates obtained by the contrast box processing as the center coordinates of the image area including the target image, and outputs the coordinates to the light path control unit 32 as a target candidate coordinate signal representing the center coordinates. I do.

The light path control unit 32 of the light path changing means 17
Means that the center coordinate value (X1, Y1) in the XY coordinates of the image obtained by the second imaging means 16 is the target image extraction means 18
Command signal for driving the pitch direction changing mirror 28 and the yaw direction changing mirror 29 to be angularly displaced so as to match the center coordinates (X2, Y2) of the extracted image area represented by the target candidate coordinate signal output from To each drive unit 30,
31, respectively. More specifically, the angular positions of the pitch direction changing mirror 28 and the yaw direction changing mirror 29 are detected by an angle position detector (not shown),
Based on this, the XY of the image by the second
A visual field center coordinate signal representing the center coordinate value (X1, Y1) in the coordinates is given to the optical path control unit 32. In response to the field-of-view center coordinate signal and the target candidate coordinate signal from the target image extracting means 18, the required change amount ΔX in the yaw direction and the required change amount ΔY in the pitch direction are calculated using the coordinate values represented by the respective coordinate signals. ΔX = X1−X2 ΔY = Y1−Y2 (2), and the respective change mirrors are set so that the required change amounts ΔX and ΔY become 0 as in the following equation: ΔX = 0 ΔY = 0 (3) Command signals for driving the driving units 28 and 29
Output to 0,31.

When a plurality of target image areas are extracted by the target image extracting means 18 described above, the optical path control unit 32 issues a command signal so that the field-of-view center coordinates periodically coincide with the target candidate coordinates. Is output. Accordingly, when a plurality of extracted image areas are present in the image obtained by the first imaging unit 15, a part of the ground area 38 corresponding to each of the extracted image areas is converted by the second imaging unit 16.
Images can be taken sequentially and periodically.

The optical path control unit 32 is supplied with a detected angular position signal indicating the angular position of the reflecting mirror 25 detected by a detector for detecting the angular position of the reflecting mirror 25. The optical path control unit 32, in response to the detected angular position signal, avoids when the reflecting mirror 25 is at an angular position where the optical image passing through the afocal optical system 35 passes through the notch 27 of the reflecting mirror 25. When the optical image that has passed through the afocal optical system 35 is reflected by the reflection mirror 25 and guided to the first imaging unit 15, the driving units 30 and 31
A command signal for driving each of the change mirrors 28 and 29 is output.

Thus, each of the change mirrors 28, 29
Is angularly displaced using only the time during which the infrared optical image from the ground area 38 is not guided, and when the second imaging means 16 is capable of imaging, the imaging direction is not changed, and the second The imaging unit 16 can capture a clear optical image without blurring. That is,
Although the infrared optical image from the ground area 38 is guided to the second imaging unit 16, the optical path changing unit 17 is changing the optical path, and the image obtained by the second imaging unit 16 is blurred. It is possible to prevent the image from becoming a useless image.

The image control means 20 includes an image correction calculating section 46
And an image synthesizing unit 47. Image correction calculator 46
Is supplied with a field-of-view center coordinate signal representing the current center coordinate of the field of view of the second imaging unit 16 from the light direction control unit 32, and based on this signal, for example, the pitch during the imaging by the second imaging unit 16 When the direction change mirror 28 and / or the yaw direction change mirror 29 undesirably swing, so-called image flow occurs, the swing is detected by the angle position detector, and the detection signal from the angle position detector is detected. Then, a correction signal for correcting the image deletion is output to the image synthesizing unit 47.

The correction of the image flow caused by the fluctuation of each of the change mirrors 28 and 29 is performed, for example, by detecting the fluctuation of each of the mirrors 28 and 29 by the angle detector. Correcting the image distortion due to the image flow by subtracting the video signal representing the unnecessary luminance and color from the video signal representing the luminance and the color based on the video signal of the pixel on the upstream side in the rocking direction. be able to.

The image correction calculating section 46 corrects the image distortion caused by the distortion of the infrared optical image caused by the aberration between the central portion and the peripheral portion of each lens 33, 34 of the afocal optical system 35, that is, the so-called peripheral aberration. The calculation formula is given to the image synthesizing unit 47. In particular, the distortion due to the peripheral aberration of each of the lenses 33 and 34 is caused by an image by the second imaging unit 16 for obtaining a detailed image.
Its importance increases. In the present embodiment, the correction of the peripheral aberration is performed on the images by the first and second imaging units 15 and 16 respectively, but only the image by the second imaging unit 16 having high importance is corrected.
The correction processing efficiency may be improved. The arithmetic expression for correcting the peripheral aberration is an expression for performing inverse transformation of distortion on an image based on the characteristics of the afocal optical system 35. This arithmetic expression may be stored in, for example, a storage unit (not shown) and called at the time of correction. In the case where only the image obtained by the second imaging unit 16 is corrected,
Only the characteristics of the afocal optical system 35 may be stored in the storage unit, and the characteristics may be called up at the time of correction, and an arithmetic expression may be created only for a portion corresponding to the imaging direction of the second imaging unit 16.

A signal representing information for correcting the image flow due to the fluctuation of each of the change mirrors 28 and 29 and correcting the image distortion due to the peripheral aberration of the afocal optical system 35 is given to the image synthesizing section 47. Can be When reconstructing an image, the image synthesizing unit 47 corrects the image flow and the image distortion.

The image synthesizing section 47 has a frame memory 50. The frame memory 50 appropriately updates and stores the synthesized image signal, and appropriately updates and stores the image signal from the first imaging means 15. A first storage area and a second storage area
A second storage area for appropriately updating and storing the image signal from the imaging means 16. The image synthesizing unit 47 stores the image signals from the first and second image pickup units 15 and 16 in the first and second storage areas of the frame memory 50 and stores the image signals from the first and second image pickup units 15 and 16. Based on the signal synchronization signal, the image signals stored in the first and second storage areas are appropriately called and combined, and stored in the combined signal storage area as an image combined signal. The stored composite image signal is
It is called up as appropriate and output to the display means 52.

The image signals from the first and second imaging means 15 and 16 are synthesized, for example, as follows. First, based on the detection signals output from the angular position detectors of the respective change mirrors 28 and 29, the center coordinates of an area of the image corresponding to the imaging range of the second imaging unit 16 by the first imaging unit 15 are calculated. Next, based on the address information of the image signal of the image by the first imaging unit 15 stored in the first area of the frame memory 50, the second imaging of the image information of the image signal of the first imaging unit 15 is performed. The video information of the area corresponding to the imaging range of the means 16 is deleted, and the second video information stored in the second recording area is replaced with the deleted video information.
The image signal of the image from the imaging means 16 is called, and the video information is inserted. The synthesized image signal obtained by the synthesis in this way is stored in the synthesized signal storage area, called up as appropriate, and output to the display means 52.

In the image control section 47, when the light path changing means 19 changes the light path based on the detection signal from each angular position detector, the image data is stored in the second frame memory 50.
Image signal stored in the area, that is, light path changing means 1
9 stores the image signal of the image captured by the second imaging unit 16 before changing the optical path, in a third storage area different from the first and second storage areas and the combined signal storage area. In the third storage area, a plurality of image signals of the image obtained by the second imaging means 16 can be stored, and when there are a plurality of extracted image areas, each extracted image picked up by the second imaging means 16 can be stored. A predetermined number of image signals of the image corresponding to the area can be stored.

When it is necessary to store a new image signal in a state where the capacity of the third storage area is full, the image data is updated and stored so that the oldest image signal is deleted and stored.

When the image pickup direction of the second image pickup means 16 is changed as described above, a part of the image obtained by the first and second image pickup means 15 and 16 is replaced with the image signal of the image signal stored in the third storage area. The image is synthesized so as to replace the image. In synthesizing the image signals representing the images from the first and second imaging means 15 and 16 and the image signals stored in the third storage area of the frame memory 50, the second signal is generated based on the address information of each image. When the image by the imaging means 16 and the image stored in the third storage area overlap each other in close proximity, the image signal stored in the second storage area of the image signal called from the third storage area In a state where the video signal of the portion overlapping with the image is deleted, a new image signal captured by the second imaging unit 16 and stored in the second storage area is preferentially combined. Further, when the images of the image signals stored in the third storage area of the frame memory 50 overlap with each other, the overlapping image is synthesized by giving priority to the newer image signal.

For example, when there are a plurality of extracted image areas in the image obtained by the first imaging means 15, the light path is periodically changed by the light path changing means 19 as described above, and the ground corresponding to each extracted image area is changed. When the areas are sequentially imaged by the second imaging means 16, the image synthesis section 47 synthesizes a substantially real-time image of one of the plurality of extracted image areas currently being imaged by the second imaging means 16. The remaining extracted image area combines the image of the image signal stored in the third storage area of the frame memory 50. In other words, when a plurality of extracted image areas are present and a combined image is displayed by the display unit 52, one extracted image area displays a substantially real-time continuous image and the remaining extracted image area Indicates that the stored image is frozen. This allows the operator to obtain the second image pickup unit 16 for one extracted image region even when the second image pickup unit 16 is imaging another extracted image region.
Although the detailed image taken by the user is in a frozen state, it can be viewed, and the image of the image signal called out from the third area of the frame memory 50 is displayed as a substantially real-time image from the second imaging means 16. Since the image is displayed in an area excluding the area, even if a plurality of extraction areas overlap each other, a new real-time image by the second imaging unit 16 can be displayed with priority. As a result, an image as new as possible can be displayed by the display means 52.

The display means 52 has a display capability capable of displaying a detailed image taken at a high resolution by the second imaging means 16 in detail without impairing the information amount. The display 52 receives the composite image signal as described above, and displays the composite image represented by the signal.

Referring again to FIG. 2, a vehicle 13 equipped with such an image pickup device 11 has a telescopic columnar body 13a.
And a launch pad 13b, 1 for launching the flying object 12
3c. The imaging device 11 is provided at the tip of the column 13a so as to be angularly displaceable about the axis of the column 13a. The imaging device 11 can be driven to be angularly displaced around the columnar body 13a by driving means such as a motor (not shown). The operator can give a command to the driving unit to drive the imaging unit 11 in angular displacement, and change the imaging direction by angularly displacing the imaging device 11 without moving the vehicle 13.

The columnar body 13a is extendable and contractable, and there is a shielded area between the vehicle 13 and the ground area 38 to be imaged, such as a raised area 37 and / or a building that blocks the view. Can extend the columnar body 13a and move only the imaging device to a position where the shield does not obstruct, so that the ground area 38 can be imaged.

FIG. 6 is a flowchart showing an example of the operation of the imaging device 11. The vehicle 13 is arranged at a predetermined position, the rotational driving of the reflection mirror 25 is started, and the angular positions of the mirrors of the change mirrors 28 and 29 are set to the initial positions, for example, the coordinates of the center of the visual field of the first imaging means 15, that is, the first position. The central position of the image by the imaging unit is set to an angular position where the center coordinate of the field of view by the second imaging unit 16 is coincident, and power is supplied to the first and second imaging units 15 and 16 so that the first and second imaging units The two detectors 22 and 24 are driven, and the preparation for imaging is completed.

In this state, the image pickup operation is started. In step a1, the first image pickup means 15 picks up an image of the ground area 38. That is, when the switching unit 17 guides the infrared optical image from the imaging target 38 to the first imaging unit 15, the first detector 22 converts the infrared optical image into an image signal.

Next, at step a2, the first image pickup means 15
Based on the image signal from, the image area including the target image is extracted by setting the image satisfying the predetermined condition as the target image by, for example, the above-described contrast box processing. Specifically, when the objects A1 to A3 exist in the ground area 38 as shown in FIG. 2, an image area including the images of the objects A1 to A3 is extracted.

Next, at step a3, the first image pickup means 15
It is determined whether or not the extracted image area exists in the image captured by the above. If the determination in step a3 is affirmative, that is, the image of the ground area 38 includes a target and an image satisfying the same conditions as the target, for example, the object A1
If it is determined that there is an image of A3 to A3, the process proceeds to step a4. If the determination is negative, the process proceeds to step a10.

In step a4, it is determined whether there is one extracted image area. If the determination is affirmative, the process proceeds to step a5. If the determination is negative, the process proceeds to step a5. Shift to a11. In step a5,
The calculation of the center coordinates of the extracted image area in the image from the first imaging unit 15 is performed. Next, in step a6, the optical path guided to the second imaging means 16 based on the calculated center coordinates is set to be the center of the extracted image area, that is, the center coordinates of the extracted image area and the second Imaging means 1
The light path changing means 19 changes the coordinates so that the coordinates of the center of the field of view 6 coincide with each other. In this state, in step a7, the second imaging means 16 images a part of the ground area 38. That is, when the switching unit 17 is the imaging target 3
When the infrared optical image from 8 is led to the second imaging means 16, the second detector 22 converts the infrared optical image into an image signal.

Next, at step a8, a signal for correcting the image flow and the image distortion of the image signals from the first and second image pickup means 15 and 16 is converted to an image correction operation section 46.
And is provided to the image synthesis unit 48. In step a9, the image synthesizing unit 48 synthesizes the image signals from the first and second imaging units 15 and 16 and the image signal stored in the third storage area of the frame memory 50, and sets the image correction calculation unit. Based on the signal for image correction calculated by 46, the composite image is reconstructed and corrected, and stored in the composite signal storage area of the frame memory 50. In addition, when an image signal is stored in the third storage area of the frame memory 50 when combining images, an area excluding the image of the image signal stored in the second storage area of the frame memory 50 is deleted. (3) An image is synthesized so as to replace the image of the image signal stored in the storage area. In step a10, the image synthesis signal stored in the frame memory 50 is output to the display means 52.

When the determination is negative in step a3 and the process proceeds to step a10, only the image by the first image pickup means 15 is corrected only by the image due to the peripheral aberration of the afocal optical system 35, and the image signal is corrected. The information is stored in the synthesized signal storage area of the frame memory 50, retrieved as appropriate, and output to the display means 52.

When the determination in step a4 is negative and the process proceeds to step a11, the calculation of the center coordinates of each extracted image area in the image from the first image pickup means 15 is performed. Next, in step a12, it is determined whether or not the calculated center coordinates of each extracted image area include coordinates downstream of the field-of-view center coordinates of the second imaging unit 16 in the scanning direction. If the determination in step a12 is affirmative, the flow shifts to step a13 where the coordinates of the center of the visual field and the center coordinates of the extracted image area on the downstream side in the scanning direction and the most upstream in the scanning direction of the central coordinates of the visual field are determined. The optical path of the light guided to the second imaging means 16 is changed so as to match. If the determination in step a12 is negative, the process proceeds to step a14, where the second imaging means 16 is caused to make the center coordinates of the visual field coincide with the center coordinates of the extracted image area most upstream in the scanning direction. Change the optical path of the guided optics. After the step a12 or the step a13 or the step a14 has been performed as described above, the process proceeds to a step a7, and the image is captured by the second imaging unit 16.

Thus, the composite image signal is output to the display means 52, and the one-cycle operation of the image pickup means 11 is completed. Such an operation is repeated at the same cycle as the switching of the optical path by the switching unit 17, and the operator operates the display unit 52.
Confirm the display image by and fly to the display image 1
Identification is made as to whether or not there is an object to be guided to 2.

FIG. 7 is a diagram showing transition of a display image of an image picked up by the image pickup device 11. FIG. 7 (1)
The image of the ground area 38 captured by the first imaging unit 15 at a predetermined resolution is displayed by the display unit 52. In the first cycle, in the image from the first imaging means 15, there are a target and an image having the same condition as the target, that is, an image having a difference in luminance from the surroundings. Regions C1 to C3 are extracted. The center coordinates of the extracted image areas C1 to C3 are obtained by the target image extracting means 18 described above.

In the next cycle, one of the center coordinates of each extracted image area, for example, the center coordinate of the extracted image area C1 located on the upstream side in the scanning direction is made coincident with the center of the visual field. The area corresponding to the area is
The image is taken by the imaging means 16, and this image is given priority,
The images of the first and second imaging units 15 and 16 are combined and displayed. As a result, as shown in FIG. 7B, in the extracted image area C1, a real-time image having a higher resolution than the image obtained by the first imaging unit 15 is obtained.

After displaying such an image for a predetermined number of cycles, the center coordinates of the extracted image area C2 whose center coordinates exist on the downstream side in the scanning direction from the current center of the field of view of the second imaging means 16 are set to the center of the field of view. According to the present embodiment, the area corresponding to the extracted image area is imaged by the second imaging means 16, and this image is given the highest priority, and the image from the second imaging means 12 imaged in the previous cycle is given. The signal is stored in the third storage area of the frame memory 50, and the image of the image signal stored in the third storage area is given priority over the image of the first imaging means 15, and the images of the first and second imaging means are stored. In addition, the image stored in the third storage area of the frame memory 50 is synthesized and displayed. As a result, as shown in FIG. 7C, in the extracted image area C2, a real-time image having a higher resolution than the image obtained by the first imaging means 15 is obtained, and the extracted image area C1 is obtained by the first imaging means 15 A frozen image having a higher resolution than the image obtained by the above is obtained.

After displaying such an image for a predetermined number of cycles from this point, the center coordinates of the extracted image area C3 whose center coordinates exist on the downstream side in the scanning direction from the current center of the field of view of the second imaging means 16 are determined. In this embodiment, an area corresponding to the extracted image area is imaged by the second imaging means 16 so as to coincide with the center of the visual field, and this image is given top priority, and the second imaging means 12 which has been imaged in the previous cycle is taken. Is stored in the third storage area of the frame memory 50 together with the currently stored image signal of the extracted image area C1, and the image of the image signal stored in the third storage area is stored in the first imaging unit. The image of the first and second imaging units 15 and 16 and the image stored in the third storage area of the frame memory 50 are combined and displayed with priority over the image of 15. As a result, as shown in FIG. 7D, a real-time image having a higher resolution than the image obtained by the first imaging means 15 is obtained in the extracted image area C3, and the extracted image areas C1 and C2 are obtained by the first imaging A frozen image having a higher resolution than the image obtained by the means 15 is obtained.

At this time, the extracted image area C3 being imaged by the second imaging means 16 partially overlaps the extracted image area C2 in which the image signal is stored in the third storage area. The image of the extracted image area C3, which is the image information, is displayed with priority over the image of the image signal stored in the third storage area of the frame memory 50.

In this way, the first image pickup means 15 picks up an image at a wide viewing angle and replaces a part of the extracted image area extracted from the obtained image with the image of the second image pickup means 16 picked up with high resolution. Thus, an image finally obtained can be an image captured with high resolution as compared with an image obtained only by the first imaging unit 15. As described above, since the images C1 to C3 satisfying the predetermined conditions can be captured with high resolution, the operator can visually check the display image on the display means 52, and the objects A1 to A3 can fly the flying object 12 with high accuracy. Can be identified as a target to be guided.

Further, since the image of the entire large ground area 38 can be obtained almost in real time by the first image pickup means 15, the update cycle of the image finally synthesized can be changed only by the second image pickup means 16 Can be suppressed from becoming longer with respect to the update cycle of the data. As a result, it is not necessary to provide an afocal optical system using a large-aperture lens as in the above-described prior art.
Can be miniaturized.

In general, in an imaging apparatus having a wide viewing angle and a high resolution, the afocal optical system is a component that is most difficult to miniaturize.
In a configuration in which an afocal optical system is separately provided for the imaging means 16, the imaging apparatus becomes large, but in the present invention, the afocal optical system is made one and the switching means 17 is used. A configuration in which an image is selectively changed and guided to the first and second imaging units 15 and 16, and a mirror 25 is rotationally driven by a drive unit 26, and an optical path is changed between the notches 27 until the optical path is guided. The switching means 17 can be configured to be smaller than the afocal optical system, and the size of the imaging device 11 can be reduced by using one afocal optical system. Since such an imaging device 11 can be downsized, the imaging device 11 does not stand out,
1 can be undesirably found as little as possible.

The image extraction areas C1 to C3 corresponding to the areas to be imaged by the second imaging means 16 are automatically extracted, so to speak, by the target image extraction means 18.
The operator does not need to select the extracted image area by looking at the image obtained by the first imaging unit 15 and can concentrate on identifying the object using the obtained image. In addition, compared to a configuration in which the selection work is performed by the operator, there is no difference in the extracted image region or the time required for selecting the extracted image region due to the experience and reaction speed of the operator.

FIG. 8 is a diagram for explaining an image processing method of the target image extracting means according to another embodiment of the present invention.
In FIG. 8A, it is assumed that an image of the object AA exists in the image 60 from the first imaging unit 15. For example, the first
During the imaging by the imaging means 15, the luminance of the horizontal scanning line indicated by SS changes as shown in FIG. That is, the distribution of the luminance 62 represented by the voltage value has a portion exceeding the preset threshold level 63. An image in which a high-luminance portion is detected for each horizontal scanning line SS and changed into a two-dimensional binary image by image processing is, for example, as shown in FIG. The center of the screen is the center 64 of the second imaging unit 16 in the imaging direction,
Are displaced by ΔX1 in the horizontal direction and ΔY1 in the vertical direction. When the imaging direction of the second imaging means 16 is changed so that this displacement becomes 0, the state shown in FIG. In this way, from the luminance distribution corresponding to the binarized image when the second imaging unit 16 changes the imaging direction to the high luminance area,
The region where the object AA in FIG. 8A exists can be imaged with high resolution.

In this embodiment, an area exceeding a predetermined threshold level 63 can be extracted using the distribution of the luminance 62. Therefore, as compared with the embodiment shown in FIGS. 1 to 7, by setting an appropriate threshold level, it is possible to reliably extract an image region including the target image.

In each of the above-described embodiments, infrared rays are used to image the ground area 38. However, the present invention is not limited to this, and visible rays may be used. The displayed image is the same as the image information that ordinary people usually see with the naked eye,
The operator can easily see the display means 52 and understand the information of the image.

In each of the embodiments described above, the imaging device 1
1 describes a configuration mounted on a vehicle 13 for launching a flying object 12, but is not limited to this, and is mounted on, for example, a helicopter and used to observe the ground area 13 from above. The configuration and a configuration used as a monitoring camera for monitoring a port or the like may be used.

[0084]

[0085]

[0086]

[0087]

According to the first aspect of the present invention, the image picked up by the second image pickup means is stored in the frame memory, and is stored in the frame memory when the light path changing means changes the light path. Since the image signal obtained, that is, the image captured by the second imaging unit can be displayed in a frozen state before the optical path changing unit changes the optical path, the target image is included in the image by the first imaging unit. When there are a plurality of image areas extracted by the image extracting means,
For example, the light path is periodically changed by the light path changing means, and the ground area corresponding to each extracted image area is individually imaged by the second imaging means. For one of the plurality of extracted image areas, It is possible to display a substantially continuous real-time detailed continuous image picked up by the second image pickup means, and display the image of the image signal stored in the frame memory in a frozen state in the remaining extracted image area.

Thus, when a plurality of extracted image areas are present, the operator is in a frozen state with respect to one extracted image area even when the second imaging means is imaging another extracted image area. Although it is a past image, a detailed image by the second imaging means can be visually checked. Further, since the image stored in the frame memory is displayed in an area other than the area where the real-time image is displayed by the second imaging means, even when a plurality of extracted image areas overlap each other, The image from the second imaging means can be displayed with priority. Therefore, almost real-time new image information obtained from the second imaging means can be displayed as much as possible.

Further, since the image represented by the image signal stored in the frame memory can be displayed in a frozen state, for example, in order to determine whether an object existing in a predetermined ground area is a target or not. When an imaging device is used, a detailed image of an object can be displayed in a frozen state. In this case, the operator can easily determine whether or not the object is a target by visually checking the frozen image of the object.

[Brief description of the drawings]

FIG. 1 is a system diagram showing a schematic configuration of an imaging device 11 according to an embodiment of the present invention.

FIG. 2 is a perspective view showing a vehicle 13 on which the imaging device 11 is mounted.

FIG. 3 is a block diagram showing an electrical configuration of the imaging device 11;

FIG. 4 is a diagram for explaining an operation of generating a contrast box processed image in a target image extracting unit 18.

FIG. 5 is a diagram for explaining an operation of extracting a local maximum contrast value on a contrast box processed image by the target image extracting means 18;

FIG. 6 is a flowchart illustrating an operation of the first imaging unit 15;

FIG. 7 is a schematic diagram showing a progress state of image processing corresponding to the operation of FIG. 6;

FIG. 8 is a schematic diagram illustrating a progress state of image processing according to another embodiment of the present invention.

FIG. 9 is a system diagram showing a schematic configuration of a typical prior art.

[Explanation of symbols]

 Reference Signs List 15 first imaging means 16 second imaging means 17 switching means 19 optical path changing means 20 image control means A1 to A3 object 38 ground area

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-7-231442 (JP, A) JP-A-6-178292 (JP, A) JP-A-6-217185 (JP, A) JP-A 61-231 141434 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 7/18 G01B 11/00 H04N 5/225 H04N 5/265

Claims (1)

(57) [Claims] 1. An afocal optical system for reducing an optical image from an object to be imaged, a first imager having a predetermined resolution and converting a received optical image into an image signal, and higher than the predetermined resolution. A second imaging unit having resolution and converting a received optical image into an image signal; and selectively allocating an optical path of the optical image reduced by the afocal optical system to the first and second imaging units. Switching means for performing switching and image processing of an image signal output from the first imaging means;
A target image extracting means for extracting an image area including a predetermined target image; and a light path of the optical image passing through the afocal optical system, interposed between the switching means and the second imaging means. An optical path changing unit that changes the imaging direction corresponding to the extracted image area to guide a part of the optical image that has passed through the afocal optical system to the second imaging unit; and an optical path changing unit that is output from the first and second imaging units. Based on the respective image signals, the images obtained by the first and second imaging units are combined so that the area corresponding to the image obtained by the second imaging unit in the image obtained by the first imaging unit is replaced with the image obtained by the second imaging unit. Image synthesizing means for outputting a synthetic image signal representing an image, wherein the image synthesizing means stores the image signal output from the second imaging means in a frame memory, and the optical path changing means changes the optical path. While the image is being updated, the area corresponding to the image stored in the frame memory of the image captured by the first image capturing means and the area corresponding to the image captured by the second image capturing means is replaced with the image stored in the frame memory. An image pickup apparatus characterized by displaying an image before an optical path change stored in a memory in a frozen state.