JPH11285015A - Image pickup device, image processing method, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output a high-resolution image pickup signal directly to the input terminal of a projector not through a computer by reading digital video data stored in a buffer memory and converting the data into an analog signal through color spatial conversion. SOLUTION: A monitor discriminating circuit 28 discriminates the kind of a monitor and sends the discrimination result to a system controller 20. A changeover switch 24 determines whether data are read out of the buffer memory 17 in the timing of an interface controller 19 or monitor display timing controller 23. A D/A converter 25 converts the digital RGB signal outputted from an RGB matrix circuit 18 into an analog RGB signal. When the data of the buffer memory 17 are displayed on the monitor, the changeover switch 24 is set to the side of a monitor display timing controller 23 and the data read out of the buffer memory 17 are outputted to the monitor from the RGB matrix circuit 18 through the D/A converter 25.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus having a built-in signal processing section for directly displaying a picked-up image on a monitor such as a VGA or SVGA, an image processing method thereof, and a storage medium storing the method. It is.

[0002]

2. Description of the Related Art A projector used for a presentation or the like is provided with a computer signal input terminal and a video signal input terminal. Computer signals have resolutions such as VGA, XGA, and SVGA, and can display fine characters. That is, various kinds of document data created by a computer in advance can be displayed.

On the other hand, with a video signal, it is possible to display an image photographed beforehand, or to immediately display a three-dimensional object or an object to be shown on the spot, but because of the low resolution, it is necessary to display a fine sentence. Is not suitable.

For this reason, when displaying a document with fine characters or a document on which an image requiring resolution is displayed, the document is read once by a scanner and then displayed on a projector from a computer, or an overhead projector (OH) is used.
P) A sheet is made and OHP is used.

That is, when making a presentation using a projector, even if a conventional video camera is connected to the video input terminal of the projector, it is not possible to capture and display a document with fine characters as it is with the video camera.

Accordingly, the applicant of the present application has filed Japanese Patent Application No. Hei 9-172.
No. 108 proposes an imaging device capable of capturing a high-resolution still image using a pixel shifting technique. In this high-resolution imaging device, an interface (U
High-resolution data can be transferred to a computer using an SB or IEEE 1394, and a high-resolution still image can be displayed on a projector via a display interface (VGA, XGA, SVGA, or the like) of the computer.

FIG. 3 is a diagram showing a configuration of a high-resolution image pickup device in the above-mentioned Japanese Patent Application No. 9-172108.

In FIG. 3, reference numeral 1 denotes an image pickup lens, and reference numeral 2 denotes a pixel shift mechanism provided at a rear end of the image pickup lens, which realizes a pixel shift unit. The pixel shifting mechanism shifts the incident position of the incident light beam on the imaging screen, for example, by changing the inclination angle of the parallel plate inserted into the optical path of the imaging lens optical system with respect to the optical axis, so that the image sensor has no pixels. This enables imaging of information to be input between pixels, and achieves high resolution.

Reference numeral 3 denotes an image sensor, such as a CCD, which is attached to the rear end of the pixel shift mechanism and has an on-chip color filter (complementary color filter and a checkerboard arrangement or an RGB primary color Bayer arrangement as described later). Double sampling (CDS) circuit and automatic gain control (AGC)
A circuit (hereinafter referred to as a CDS / AGC circuit), 5 is an image sensor driving circuit (Vdrive) for driving the image sensor, and 6 is a timing generator (TG) for generating various timing pulse signals for driving the image sensor. .

Reference numeral 7 denotes a microcomputer (hereinafter, referred to as a microcomputer) for controlling the image pickup unit system which controls the entire image pickup unit. Reference numeral 8 denotes an electric circuit for driving a pixel shifting mechanism, which is controlled by the microcomputer 7.

Reference numeral 9 denotes a camera head, which constitutes an image pickup unit.

Reference numeral 10 denotes an image pickup unit and a subsequent image processing unit (described later).
A reference numeral 11 denotes an A / D converter for converting an image signal output from the image sensor 3 into digital data;
Is an image memory that stores digital high-resolution still image data,
Reference numeral 13 denotes a memory controller for controlling writing and reading addresses of the image memory 12, reference numeral 14 denotes an addition circuit of read data of upper and lower two lines from the image memory 12, and reference numeral 15 denotes luminance from a color signal corresponding to a pixel pattern of a complementary color checkerboard of a color filter. Signal (Y) and chroma signal (RY, B-
Y), a camera process circuit for generating the image data, 16 a horizontal scaler for converting the aspect ratio of the screen of the image sensor into a square pixel pattern 1: 1, 17 a buffer memory for transferring image data to a personal computer, 18 Is an RGB matrix circuit for generating an RGB signal from a luminance signal and a chrominance signal.

An interface (I / F) controller 19 controls data transfer with a personal computer. The image transfer interface includes a USB or IE capable of high-speed transfer.
EE1394 or the like is used.

Reference numeral 20 denotes a system control microcomputer (system controller) for controlling the entire system including the image processing unit 22 and the image pickup unit 9 of the apparatus, and 21 denotes a switch for switching the output mode, that is, a luminance and chroma signal and an RGB signal. , 22 indicate blocks of the image processing unit. A series of processes related to pixel shifting, imaging, storage of pixel data read from the image sensor in an image memory, address conversion, and transfer are performed by a program stored in a storage medium such as a ROM provided in the system controller 20,
Alternatively, the program is executed by a program stored in a storage medium of an externally connected personal computer.

The operation will be described below.

An optical image of a subject (not shown) incident through the imaging lens 1 is incident on a pixel shift mechanism 2.
The pixel shifting mechanism 2 is provided between the imaging lens 1 and the imaging element 3. Therefore, even if the imaging lens 1 is used to change the zoom ratio by using a zoom lens, the pixel shifting mechanism is required. This mechanical operation is not related to the imaging lens 1 but is related only to the pixel pitch of the image sensor 3.

The pixel shifting mechanism 2 shifts the optical path by 2/3 pixel in both the plus and minus directions, that is, shifts the pixels independently in the horizontal and vertical directions.

The image sensor 3 is driven by the TG 6 and the drive circuit 5 in a frame all pixel read mode. When the filter is a complementary color checkerboard pattern, only the cyan C and yellow Y color signals are output from the first field. From the second field, only green G and magenta M color signals are read and output.

The output signal of the image sensor 3 is CDS / AGC
After being supplied to the circuit 4 and correlated double samples, it is amplified to a predetermined level. The output of the CDS / AGC circuit 4 is supplied to an A / D converter 11 via a cable 10 and is converted into a digital color signal.

This digital color signal is shifted as a result of pixel shift.
The data is written to the image memory 12. This write control is
This is performed by controlling the memory controller 13 by the system controller 20.

The image data written in the image memory 12 has a pixel density three times as high in the vertical and horizontal directions (X, Y), and has the same framing as the normal case (no pixel shift), that is, the pixel arrangement, and the pixel density is three times as large. It has increased.

After the writing is completed, the image data in the image memory 12 is read by the memory controller 13 after being divided into nine frames.

When reading out each frame,
The upper and lower two lines are read simultaneously, and the addition circuit 14
The upper and lower two lines are added and mixed, and converted into field image data.

The field image data output from the adder circuit 14 is supplied to a camera process circuit 15, and a luminance signal and a chroma signal are output from a normal complementary color checkerboard field read signal.

The output digital signal is supplied to a horizontal scaler 16 where the square pixel is converted into an aspect ratio (aspect ratio) of 1: 1. This is the conversion required for display on the PC display.

If an image sensor having a pixel size of an aspect ratio of 1: 1 is used from the beginning, the scaler can be omitted.

The scaled image data is supplied to a transfer buffer memory 17, where one frame of data is accumulated.

This buffer memory reads out a predetermined amount of data in accordance with the transfer rate under the control of the interface controller 19, and reads out the RG data as necessary.
After being converted into RGB signals via the B matrix circuit 18, the signals are supplied to the interface controller 19 in the form of a luminance signal and a chroma signal as they are.

The interface controller 19 communicates with a personal computer according to a predetermined transfer method to transfer image data. In this example, an interface capable of high-speed transfer such as IEEE1394 is used.
Etc., and is not particularly limited, and may be determined in consideration of the personal computer side.

When the transfer of one frame is completed and the buffer memory 17 becomes empty, the second frame is read from the image memory 12, and after performing the same processing, is stored in the buffer memory 17, and Controller 1
9 to the personal computer.

The same operation is performed for all nine frames, and the transfer of one high-quality still image to the personal computer is completed.

The system controller 20 controls and executes the above operations in a predetermined sequence for each component.

According to the above-described pixel shifting system, all pixels are read from the image pickup device and stored in the memory in the same spatial arrangement as when the image is picked up by the image pickup device in which the number of pixels is substantially increased, and read out from the memory. Since the upper and lower two lines are added when the image data is supplied to the camera process circuit, the resolution does not decrease before the image data is stored in the memory.

For example, as is known as a field reading method for an image sensor, a method of adding and reading two odd lines in a first field and adding and reading two even lines in a second field is called a pixel reading method. Even if the displacement is performed, the two overlapping lines are alternately read, so that the virtual pixel size increases (3
Pixels), and the resolving power does not improve so much. This is because the pixel has already been shifted while adding two lines sandwiching one line when reading from the image sensor.

On the other hand, here, all the pixels are read from the image pickup device and stored in the memory. At this stage, a high image quality is obtained by shifting the pixels, and when reading from the memory (after the high-quality pixel space arrangement). Since two adjacent lines are added, image quality degradation can be minimized.

As described above, since a high-resolution still image cannot be displayed with a video signal, in the above-described conventional imaging apparatus capable of high-resolution imaging, high-resolution data uses an i / f for high-speed data transfer. Then, it is processed through a computer.

[0037]

However, in the above-mentioned conventional high-resolution imaging apparatus, it is necessary to always use a computer when displaying high-resolution imaging data on a projector. In other words, even if you do not normally use a computer for presentations, you need to use a computer in order to display high-resolution image data, so portability, installation space, startup complexity, and maintainability are required. It is extremely inconvenient in such situations.

Further, even if it is only necessary to monitor high-resolution image data, the computer must be connected to a computer, and a high-resolution still image can be monitored as easily as watching a video image on a conventional video monitor. Can not. In such a situation, if an imaging device capable of high-resolution imaging can be displayed on a projector as it is, a computer is not required in many cases, and use in a state completely replacing the conventional overhead projector (OHP) is possible. In addition to this, it is possible to immediately display an object on the spot, which is an advantage of a video imaging device. Further, as an incidental matter, the OHP sheet, which has been discarded separately from paper, is no longer necessary, and is therefore useful from an ecological point of view.

Accordingly, the present invention has been made in view of the above-described problems, and an object of the present invention is to provide an imaging apparatus and an imaging method capable of directly outputting a high-resolution imaging signal to a computer input terminal of a projector, and a storage storing the method. Is to provide a medium.

It is another object of the present invention to provide an image pickup apparatus and an image pickup method capable of easily monitoring a high-resolution image pickup signal with a monitor or the like without using a computer, and a storage medium storing the method. .

Still another object of the present invention is to easily display the entire image data or a desired size on the monitor even when the image data is larger than the monitor display size when displaying the image data on the monitor. It is an object of the present invention to provide an imaging apparatus, an imaging method, and a storage medium storing the method.

[0042]

Means for Solving the Problems The above-mentioned problems are solved,
In order to achieve the object, an imaging device according to the present invention includes an imaging device capable of reading out all pixels, a pixel shifting unit that shifts a light beam incident on an imaging surface of the imaging device by a predetermined number of pixels, An image memory capable of storing pixel data captured at each position shifted by the pixel shifting means, and storing the image data in the image memory
(N is an integer) a memory control unit that stores in a spatial array of pixel data equivalent to an image pickup unit having a multiple of the number of pixels, and that reads out the stored pixel data one frame at a time and performs reading multiple times; Adding means for performing field reading of upper and lower line mixing by performing arithmetic processing of upper and lower line mixing on pixel data read from the image memory; and field reading pixels of upper and lower line mixing output from the adding means. A camera processing unit for processing data, a buffer for temporarily storing digital video signals output from the camera processing unit, a space conversion unit for performing color space conversion, a synchronization signal generation unit, and control of the entire imaging apparatus. A digital video data stored in the buffer by VGA, XGA,
Monitor display control means for performing control for display on a monitor such as SVGA; and D / A conversion means for converting digital video data read from the buffer and color space converted into an analog signal. Features.

Further, in the imaging apparatus according to the present invention, a scaling means for changing the data size of the digital video data stored in the buffer so as to match the resolution of a monitor such as VGA, XGA, SVGA is further provided. It is characterized by having.

Further, in the image pickup apparatus according to the present invention, the scaling means is arranged at a stage preceding the buffer.

[0045] In the imaging apparatus according to the present invention, the scaling means may include the buffer and the D / D.
It is characterized by being arranged between the A conversion means.

The imaging apparatus according to the present invention can be connected to different displays, and is an imaging apparatus for supplying an image to the display and displaying the image. Control means for controlling an information amount of an image signal supplied from the image pickup apparatus to the display in accordance with an output of the detection means.

Further, in the image pickup apparatus according to the present invention, the control means switches an image pickup mode in the image pickup means according to the output of the detection means.

Further, in the image pickup apparatus according to the present invention, the control means thins out image signals obtained from the image pickup means according to the output of the detection means.

The image processing method according to the present invention is an image processing method in an image pickup apparatus which can be connected to different displays and supplies an image to the display for display. And an adjusting step of adjusting an information amount of an image signal supplied from the imaging device to the display in accordance with a detection result in the detection step.

Further, in the image processing method according to the present invention, in the adjusting step, the imaging mode in the imaging means is switched according to the detection result in the detection step.

Further, in the image processing method according to the present invention, in the adjusting step, the image signal obtained from the imaging means is thinned out according to the detection result in the detecting step.

The storage medium according to the present invention is a storage medium which can be connected to a different display and stores a control program for image processing in an image pickup apparatus for supplying and displaying an image on the display. A code for a detection step in which the control program detects the display capability of the display, and a code for an adjustment step for adjusting an information amount of an image signal supplied from the imaging device to the display according to a detection result in the detection step. And the following.

Further, in the storage medium according to the present invention, the code of the adjusting step switches an imaging mode in the imaging means according to a detection result in the detecting step.

Further, in the storage medium according to the present invention, the code of the adjusting step thins out an image signal obtained from the imaging means in accordance with a detection result in the detecting step.

[0055]

Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

(First Embodiment) FIG. 1 shows a first embodiment of the present invention.
FIG. 3 is a diagram illustrating a block configuration of an imaging device capable of performing high-resolution imaging according to the embodiment.

FIG. 1 differs from the conventional example shown in FIG. 3 in that a monitor display timing controller 23 (hereinafter referred to as a timing controller) and a changeover switch 2
4, a D / A converter 25 and a monitor identification circuit 28 for identifying the type of the connected monitor.

The timing controller 23 controls to read out the data in the buffer memory 17 at the timing when the data is displayed on a monitor such as VGA or SVGA under the control of the system controller 20. At that time, the monitor identification circuit 28 identifies the type of monitor such as VGA or SVGA, and the system controller 20 performs control according to the type of monitor according to the identification result. The changeover switch 24 is a switch that switches between reading the data in the buffer memory 17 at the timing of the interface controller 19 and reading at the timing of the timing controller 23. The D / A converter 25 converts a digital RGB signal output from the RGB matrix circuit 18 into an analog RGB signal.

The operation will be described focusing on the parts different from the conventional example.

A predetermined area of the video data stored in the image memory 12 is read out according to an instruction from the memory controller 13, and a luminance signal and a color signal are generated by the camera process circuit 15. These are stored in the buffer memory 17 as digital data. Here, when data is passed to the system of the interface controller 19 as in the conventional example, the changeover switch 24 is set to the interface controller 19 side by the system controller 20, and the signal is read at the read timing of the interface controller 19. The data is output from the interface controller 19 via the switch 21.
At this time, the output system to the monitor is turned off.

On the other hand, when displaying on the monitor of VGA or SVGA, the changeover switch 24 is set to the timing controller 23 side by the system controller 20 and read out at the readout timing of the monitor display timing controller 23. The read data is converted from the RGB matrix circuit 18 through the D / A converter 25 into an analog signal and output to the monitor. At this time, the output from the interface controller 19 is turned off. Of course, in a circuit configuration in which RGB signals are already output at the time of output from the camera process circuit 15, the RGB matrix circuit 18 is unnecessary, and the D / A converter 25
Enter it in

According to this embodiment, high-resolution image data can be displayed directly on a projector or a monitor without connecting a computer.

(Second Embodiment) FIG. 2 shows a second embodiment of the present invention.
FIG. 3 is a diagram illustrating a block configuration of an imaging device capable of performing high-resolution imaging according to the embodiment. The difference from the first embodiment in FIG. 2 is that the horizontal scaler 16 is deleted and the scaler 26 is arranged at the subsequent stage of the buffer memory 17.

By removing the horizontal scaler, the pixel configuration data (720 × 240) of the video field output from the camera process circuit 15 is stored in the buffer memory 17 without being scaled in the horizontal direction. The data read from the buffer memory 17 is a scaler 2
In step 6, the pixel size is scaled according to the instruction from the system controller. The data scaled by the scaler 26 is supplied to the RGB matrix circuit 18 and the D / A converter 25.
Is output to the monitor.

The means for scaling is the buffer memory 1
In the stage preceding the step 7, since only the data after scaling is stored in the memory, the memory can be saved. In addition, since the data is not read through the scaling means at the time of reading, the output speed is increased accordingly.

On the other hand, in the present embodiment, the scaling means is provided after the buffer memory 17. This is effective when the buffer memory 17 has as much capacity as possible. For example, if the image memory 12 has a data capacity of the same number of pixels (that is, nine frames), all the pixel data of the image memory 12 captured by shifting the pixels can be buffered. Monitor is VG
The A monitor can display only 640 × 480 pixels, and cannot display nine frames at a time. S
Even a VGA monitor (1024 × 768) cannot display all images at once. However, in the present embodiment, since the scaler 26 is arranged after the buffer memory 17, an arbitrary area in the buffer memory 17 specified by the system controller 20 can be displayed by making full use of the zooming function. That is, when displaying an area larger than the display size of the monitor, the scaler 26 performs various pixel thinning-out or line thinning-out controls, and conversely, when displaying an area smaller than the monitor display size, various scales of pixel interpolation are performed. Alternatively, inter-line interpolation control is performed.

According to this embodiment, even when image data larger than the monitor display size is displayed on the monitor, the entire image data or a desired size can be simply displayed on the monitor.

[0068]

[Other Embodiments] Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus (for example, a copying machine) Machine, facsimile machine, etc.).

Further, an object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (or CPU) of the system or apparatus.
And MPU) read and execute the program code stored in the storage medium.

In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD
-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the OS (Operating System) running on the computer based on the instruction of the program code. ) May perform some or all of the actual processing, and the processing may realize the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instructions of the program code, It goes without saying that the CPU included in the function expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0074]

As described above, according to the present invention,
The high-resolution imaged data can be displayed directly on the projector or monitor without using a computer, so portability, installation space, start-up simplicity,
The maintainability is improved as compared with the case where a computer is required.

Further, since the OHP sheet can be completely replaced with OHP, the labor, time and cost for preparing and sorting and disposing of the OHP sheet can be reduced, which is useful from the viewpoint of cost reduction and ecology.

Further, when displaying on the monitor, even if the image data is larger than the monitor display size, the entire image data or a desired size can be simply displayed on the monitor. In doing so, it becomes extremely easy to confirm the state.

[0077]

[Brief description of the drawings]

FIG. 1 is a block diagram of an imaging apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram of an imaging device according to a second embodiment of the present invention.

FIG. 3 is a block diagram of a conventional imaging device capable of high-resolution imaging.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image pickup lens 2 Pixel shift mechanism 3 Image sensor 4 CDS / AGC circuit 5 Image sensor drive circuit 6 Timing generator 7 Microcomputer 8 Pixel shift drive circuit 9 Image pickup unit 10 Cable 11 A / D converter 12 Image memory 13 Memory controller 14 Addition Device 15 Camera process circuit 16 Horizontal scaler 17 Buffer memory 18 RGB matrix circuit 19 Interface controller 20 System controller 21 Signal changeover switch 22 Image processing section 23 Monitor display timing controller 24 Changeover switch 25 D / A converter 26 Scaler 28 Monitor identification circuit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁶ Identification code FI H04N 5/335 H04N 5/66 B 5/66 9/64 F 5/765 G06F 15/64 450E 5/781 H04N 5/781 510D 9/64

Claims (13)

[Claims] An image pickup device capable of reading out all pixels; a pixel shift unit for shifting a light beam incident on an image pickup surface of the image pickup unit by a predetermined number of pixels; and at each position shifted by the pixel shift unit. An image memory capable of storing imaged pixel data, and the image memory stores the image data in a spatial arrangement of pixel data equivalent to N (integer) times the number of pixels in the vertical and horizontal directions. A memory control unit that divides the stored pixel data one frame at a time and performs a plurality of times by dividing the pixel data read from the image memory; And a camera for processing the field read pixel data of mixed upper and lower lines output from the adder. Processing means, a buffer for temporarily storing a digital video signal output from the camera processing means, a space conversion means for performing color space conversion, a synchronization signal generation means, and a system control means for controlling the entire imaging apparatus. The digital video data stored in the buffer is converted to VG
Monitor display control means for controlling display on a monitor such as A, XGA, SVGA, etc., and D / A conversion means for converting digital video data read from the buffer and color space converted into an analog signal. An imaging device, comprising: 2. The apparatus according to claim 1, further comprising scaling means for changing a data size of the digital video data stored in the buffer so as to be adapted to a resolution of a monitor such as VGA, XGA, SVGA. Item 2. The imaging device according to Item 1. 3. The imaging device according to claim 2, wherein the scaling unit is arranged at a stage before the buffer. 4. The imaging apparatus according to claim 2, wherein the scaling unit is disposed between the buffer and the D / A conversion unit. 5. An image pickup apparatus connectable to different displays, for supplying an image to the display and displaying the image, wherein: a detecting means for detecting a display capability of the display; Control means for controlling the amount of information of an imaging signal supplied from the imaging device to the display. 6. The imaging apparatus according to claim 5, wherein the control unit switches an imaging mode of the imaging unit according to an output of the detection unit. 7. The imaging apparatus according to claim 5, wherein the control unit thins out an image signal obtained from the imaging unit according to an output of the detection unit. 8. An image processing method in an image pickup apparatus connectable to different displays and for supplying an image to the display and displaying the image, a detecting step of detecting a display capability of the display; Adjusting an information amount of an imaging signal supplied from the imaging device to the display according to a detection result in the step. 9. The image processing method according to claim 8, wherein, in the adjusting step, an imaging mode in an imaging unit is switched according to a detection result in the detection step. 10. The image processing method according to claim 8, wherein in the adjusting step, an image signal obtained from an imaging unit is thinned according to a detection result in the detecting step. 11. A storage medium which is connectable to different displays and stores a control program for image processing in an imaging device for supplying an image to the display and displaying the image, wherein the control program comprises: And a code for an adjusting step of adjusting the information amount of an imaging signal supplied from the imaging device to the display according to the detection result in the detecting step. Storage medium. 12. The storage medium according to claim 11, wherein the code of the adjusting step switches an imaging mode in an imaging unit according to a detection result in the detecting step. 13. The storage medium according to claim 11, wherein the code of the adjusting step thins out an image signal obtained from an imaging unit according to a detection result in the detecting step.