JP2017011504A - Imaging device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable images captured by two imaging means to be combined with each other at low cost and with high resolution.SOLUTION: An imaging device 1 includes a second imaging unit 17, a first imaging unit 16, an adjustment processor 52, and an image composition unit 54. The second imaging unit 17 acquires a thermal image IMG2. The first imaging unit 16 captures an imaging range which contains the imaging range of the second imaging unit 17 and is wider than that of the second imaging unit 17, thereby acquiring a visible light image IMG1. The adjustment processor 52 adjusts the size of either one or both of the thermal image IMG 2 and the visible light image IMG1 such that the sizes of the same subject contained in the respective images are equal to each other, thereby generating either one or both of a reduced image IMG21 and an enlarged image. The image composition unit 54 combines the reduced image IMG21 with the visible light image IMG1 or combines the thermal image IMG2 with the enlarged image, or combines the reduced image IMG21 with the enlarged image to generate a composite image IMG3.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an imaging apparatus, an image processing method, and a program.

Conventionally, a technique for acquiring a temperature distribution of a subject by imaging the subject with infrared light has been used.
In such a technique, the temperature distribution alone makes it difficult to understand the positional relationship with the subject, so the temperature distribution is extracted by extracting the contour from an image of the same subject captured with visible light and combining it with the image captured with infrared light. A technique for making it easy to understand the positional relationship between a subject and a subject is disclosed (see Patent Document 1).

Japanese Patent Laid-Open No. 9-218100

  However, the technique described in Patent Document 1 described above has a problem that the cost of imaging means using high-resolution infrared light is high.

  The present invention has been made in view of such a situation, and an object of the present invention is to synthesize images captured by two imaging units at a lower cost and with higher resolution.

In order to achieve the above object, an imaging device of one embodiment of the present invention includes:
First imaging means for acquiring a first captured image;
A second imaging unit that includes an imaging range of the first imaging unit, captures a wider imaging range than the first imaging unit, and acquires a second captured image;
Adjusting the size of one or both of the first captured image and the second captured image so that the same subject included in each image has the same size, the first adjusted image and An adjusted image generating means for generating one or both of the second adjusted images;
The first adjusted image is combined with the second captured image, or the first captured image is combined with the second adjusted image, or the first adjusted image is combined with the second adjusted image. Image compositing means for generating a composite image by compositing with an image;
It is characterized by providing.

  According to the present invention, it is possible to synthesize images picked up by two image pickup means at a lower cost and with higher resolution.

It is a block diagram which shows the structure of the hardware of the imaging device which concerns on one Embodiment of this invention. It is a schematic diagram for demonstrating the production | generation of the synthesized image in this embodiment. It is a schematic diagram which shows the form of a synthesized image. It is a functional block diagram which shows the functional structure for performing an imaging | photography process among the functional structures of the imaging device of FIG. 5 is a flowchart for describing a flow of imaging processing executed by the imaging apparatus of FIG. 1 having the functional configuration of FIG. 4.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram illustrating a hardware configuration of an imaging apparatus according to an embodiment of the detection apparatus of the present invention.
The imaging device 1 is configured as a digital camera, for example.

  The imaging device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, a first imaging unit 16, 2 includes an imaging unit 17, an input unit 18, an output unit 19, a storage unit 20, a communication unit 21, and a drive 22.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 20 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. The input / output interface 15 is connected to the first imaging unit 16, the second imaging unit 17, the input unit 18, the output unit 19, the storage unit 20, the communication unit 21, and the drive 22.

  Although not shown, the first imaging unit 16 includes an optical lens unit and an image sensor.

The optical lens unit includes a lens that collects light, such as a focus lens and a zoom lens, in order to capture a subject image.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. The zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

The image sensor includes a photoelectric conversion element, AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. Through various signal processing, a digital signal is generated and output as an output signal of the first imaging unit 16.
The first imaging unit 16 uses visible light as an imaging target, and the output signal of the first imaging unit 16 is hereinafter referred to as a “visible light image”. Visible light image data is appropriately supplied to the CPU 11 and an image processing unit (not shown).

The second imaging unit 17 can acquire a thermal image obtained by imaging far-infrared light using far-infrared light as an imaging target, while the first imaging unit 16 uses visible light as an imaging target. Configured as a simple infrared image sensor. The second imaging unit 17 is disposed in the vicinity of the first imaging unit 16, and the focal length of the optical lens unit that constitutes the second imaging unit 17 is the same as that of the optical lens unit that constitutes the first imaging unit 16. It is shorter than the focal length and is configured to be able to image a part of the angle of view of the first imaging unit 16.
Note that the light that can be imaged by the first imaging unit 16 or the second imaging unit 17 is not limited to visible light or far-infrared light, but is at least electromagnetic waves, mainly visible light, up to infrared rays and ultraviolet rays. Includes X-rays, gamma rays, etc. with shorter wavelengths, or microwaves with longer wavelengths.

The input unit 18 is configured with various buttons and the like, and inputs various types of information according to user instruction operations.
The output unit 19 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 20 is configured by a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 21 controls communication with other devices (not shown) via a network including the Internet.

  A removable medium 31 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 22. The program read from the removable medium 31 by the drive 22 is installed in the storage unit 20 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 20 in the same manner as the storage unit 20.

In the imaging device 1 configured as described above, a thermal image is acquired by imaging far-infrared light with a telephoto lens, and visible light is captured by imaging a visible light with a wide-angle lens having a focal length shorter than that of the telephoto lens. It has a function of acquiring an image, reducing the thermal image, increasing the resolution, and synthesizing it with a visible light image.
Generally, an image sensor that captures a thermal image has a lower resolution than an image sensor that captures a visible light image, so a thermal image captured at the same angle of view as a visible light image is simply combined into a visible light image. The resolution of the temperature distribution represented by the thermal image is relatively low, and it is difficult to grasp the detailed temperature distribution.
On the other hand, in the imaging apparatus 1 according to the present embodiment, a thermal image is acquired by imaging a subject whose temperature distribution is to be grasped with a telephoto lens, and the thermal image is reduced to relatively increase the resolution. Then, it is combined with the same subject portion of the visible light image captured by the wide-angle lens including the thermal image capturing range. As a result, since a higher-resolution thermal image can be acquired from the subject whose temperature distribution is to be grasped, it is possible to synthesize an image captured by the thermal image capturing unit and the visible light image capturing unit at a higher resolution. it can. In addition, since it is not necessary to use a high-resolution and high-cost image sensor that captures a thermal image, a composite image can be generated at a lower cost.

FIG. 2 is a schematic diagram for explaining generation of a composite image in the present embodiment.
When a composite image is generated, first, a visible light image IMG1 output from the first imaging unit 16 and a thermal image IMG2 output from the second imaging unit 17 are acquired as illustrated in FIG.

Then, an image (hereinafter referred to as “reduced image IMG21”) obtained by reducing the thermal image IMG2 according to the ratio of the angle of view between the acquired visible light image IMG1 and the thermal image IMG2 is generated.
In the present embodiment, contour extraction is performed on the visible light image IMG1, and an image representing the contour (hereinafter referred to as “contour image IMG11”) is generated.
Further, by combining the reduced image IMG21 with the position of the subject in the contour image IMG11, a composite image IMG3 is generated in which the subject represented as the thermal image IMG2 and the contour image IMG11 are superimposed.
However, the display can be sequentially switched between a state in which the thermal image IMG2 is combined and a state in which the thermal image IMG2 is not combined. Thereby, the user can confirm the temperature and appearance of the subject in real time.

Note that in the composite image IMG3, the periphery (margin area) of the subject (that is, the reduced image IMG21) represented as the thermal image IMG2 has various forms depending on the purpose of obtaining the composite image IMG3 and ease of viewing. can do.
FIG. 3 is a schematic diagram showing the form of the composite image IMG3.
As shown in FIG. 3A, the contour image IMG11 is superimposed on the portion of the subject (reduced image IMG21) represented as the thermal image IMG2, and the visible light image IMG1 can be synthesized for the blank area.
Further, as shown in FIG. 3B, the contour image IMG11 can be superimposed on the portion of the subject (reduced image IMG21) represented as the thermal image IMG2, and the contour image IMG11 can also be synthesized for the blank area. .
Further, as shown in FIG. 3C, the visible light image IMG1 is also synthesized with respect to the blank area without superimposing the contour image IMG11 on the portion of the subject (reduced image IMG21) represented as the thermal image IMG2. It is good as well.
Further, as shown in FIG. 3D, the contour image IMG11 may be superimposed on the portion of the subject represented as the thermal image IMG2 (reduced image IMG21), and the blank area may be deleted. In this case, the composite image IMG3 has the same size as the reduced image IMG21.

  In this embodiment, when the synthesized image IMG3 is recorded, an image in a state where the thermal image IMG2 is not synthesized is recorded together, or an image obtained by performing contour extraction on the visible light image IMG1 and an image not performed are combined. It is possible to record or record various images (see FIGS. 3A to 3D) in which processing for the blank area is switched.

FIG. 4 is a functional block diagram showing a functional configuration for executing the photographing process among the functional configurations of the imaging apparatus 1 as described above.
The imaging process is to shoot the thermal image IMG2 with a telescopic angle of view, generate a visible light image IMG1 with a wide angle of view, and reduce the thermal image IMG2 according to the focal length ratio to reduce the visible light image IMG1. Are combined to generate a composite image IMG3 including a high-resolution thermal image IMG2 of the subject.

When the photographing process is executed, as shown in FIG. 4, in the CPU 11, the imaging control unit 51, the adjustment processing unit 52, the contour extraction unit 53, the image composition unit 54, and the recording control unit 55 are included. Function.
An image data storage unit 71 is set in one area of the storage unit 20.
The image data storage unit 71 stores captured thermal image IMG2 data, visible light image IMG1 data, and various synthesized composite image IMG3 data.

The imaging control unit 51 controls the imaging of the first imaging unit 16 and the second imaging unit 17, and outputs a visible light image IMG1 and a thermal image IMG2.
Specifically, the imaging control unit 51 sets the angle of view when the first imaging unit 16 and the second imaging unit 17 capture the visible light image IMG1 and the thermal image IMG2. In addition, the imaging control unit 51 instructs the first imaging unit 16 and the second imaging unit 17 to perform imaging.

The adjustment processing unit 52 adjusts the size of the thermal image IMG2 according to the focal length ratio between the visible light image IMG1 and the thermal image IMG2 (in this case, reduces), and generates a reduced image IMG21.
The contour extraction unit 53 performs contour extraction on the visible light image IMG1 and generates a contour image IMG11.
Note that the process of extracting the contour from the visible light image can be performed by using a known technique, and thus detailed description thereof is omitted.

As shown in FIG. 2, the image composition unit 54 composes the reduced image IMG21 with the position of the subject in the contour image IMG11. As a result, a composite image IMG3 is generated in which the subject represented as the thermal image IMG2 and the contour image IMG11 are superimposed.
The image composition unit 54 selects whether the reduced image IMG21 is to be combined with the contour image IMG11 or the visible light image IMG1 when a selection instruction is made by the user via the input unit 18, and the reduced image It is selected whether or not the contour image IMG11 is superimposed on the IMG 21. Note that the image composition unit 54 deletes the blank area around the reduced image IMG 21 when an instruction to delete the blank area is given by the user via the input unit 18. As a result, a composite image IMG3 (see FIGS. 3A to 3D) having a form designated by the user is generated.
Furthermore, when the user gives an instruction to switch between the synthesized state and the unsynthesized state of the thermal image IMG2, the reduced image IMG21 is displayed on the contour image IMG11 (or the visible light image IMG1). The synthesized state and the unsynthesized state are switched and displayed on the output unit 19.

  The recording control unit 55 determines whether or not a recording instruction is made by the user via the input unit 18, and when the recording instruction is made, the image data storage unit records each image in response to the instruction. 71 is controlled.

[Operation]
Next, the operation will be described.
FIG. 5 is a flowchart for explaining the flow of imaging processing executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.
The photographing process is started by an operation for starting the photographing process on the input unit 18 by the user.

In step S <b> 11, the imaging control unit 51 sets the angle of view when capturing the visible light image IMG <b> 1 and the thermal image IMG <b> 2 with respect to the first imaging unit 16 and the second imaging unit 17.
In step S <b> 12, the imaging control unit 51 instructs the first imaging unit 16 and the second imaging unit 17 to execute imaging. As a result, the visible light image IMG1 and the thermal image IMG2 shown in FIG. 2 are acquired.

In step S13, the adjustment processing unit 52 reduces the thermal image IMG2 according to the ratio of the focal lengths of the visible light image IMG1 and the thermal image IMG2. As a result, a reduced image IMG21 shown in FIG. 2 is generated.
In step S14, the contour extraction unit 53 performs contour extraction on the visible light image IMG1. As a result, the contour image IMG11 shown in FIG. 2 is generated.

In step S15, the image composition unit 54 composites the reduced image IMG21 with the position of the subject in the contour image IMG11. As a result, any composite image IMG3 shown in FIGS. 3A to 3D is acquired.
In step S <b> 16, the image composition unit 54 displays the composite image IMG <b> 3 acquired in step S <b> 15 on the output unit 19.

In step S <b> 17, the image composition unit 54 determines whether or not an instruction to switch the display of the composite image IMG <b> 3 has been given by the user.
If the user gives an instruction to switch the display of the composite image IMG3, YES is determined in step S17, and the process proceeds to step S18.
On the other hand, when the instruction to switch the display of the composite image IMG3 is not made by the user, NO is determined in step S17, and the process proceeds to step S19.
In step S18, the image composition unit 54 displays any one of the visible light image IMG1, the contour image IMG11, the thermal image IMG2, the reduced image IMG21, and the composite image IMG3 on the output unit 19 as the instructed image.
After step S18, the process proceeds to step S17.

In step S <b> 19, the recording control unit 55 determines whether there is a recording instruction via the input unit 18.
If there is a recording instruction, YES is determined in step S19, and the process proceeds to step S20.
On the other hand, if there is no recording instruction, NO is determined in step S19, and the process proceeds to step S11.

In step S <b> 20, the recording control unit 55 records each image in the image data storage unit 71.
After step S20, the process proceeds to step S11.

The imaging device 1 configured as described above includes the second imaging unit 17, the first imaging unit 16, the adjustment processing unit 52, and the image composition unit 54.
The second imaging unit 17 acquires a thermal image IMG2 that is a first captured image.
The first imaging unit 16 includes the imaging range of the second imaging unit 17, captures an imaging range wider than the second imaging unit 17, and obtains a visible light image IMG1 that is a second captured image.
The adjustment processing unit 52 determines that one or both of the thermal image IMG2 that is the first captured image and the visible light image IMG1 that is the second captured image have the same size of the same subject included in each image. The size is adjusted so that one or both of the reduced image IMG21 that is the first adjustment image and the enlarged image that is the second adjustment image is generated.
The image synthesis unit 54 synthesizes the reduced image IMG21 that is the first adjustment image with the visible light image IMG1 that is the second captured image, or the thermal image IMG2 that is the first captured image as the second adjustment image. Is combined with the enlarged image, or the reduced image IMG21, which is the first adjustment image, is combined with the enlarged image, which is the second adjustment image, to generate a combined image IMG3.
Thereby, in the imaging device 1, when the images captured by the two imaging units are combined, the size of the image captured by one or both imaging units is adjusted to increase the resolution.
Therefore, it is possible to synthesize images picked up by the two image pickup means with higher resolution at a lower cost.

The adjustment processing unit 52 uses the first captured image so that the size of the same subject included in each of the thermal image IMG2 that is the first captured image and the visible light image IMG1 that is the second captured image is the same. The size of a certain thermal image IMG2 is reduced to generate a reduced image IMG21 that is the first adjustment image.
The image composition unit 54 synthesizes the reduced image IMG21 that is the first adjustment image with the visible light image IMG1 that is the second captured image or the enlarged image that is the second adjustment image, and generates a composite image IMG3.
Thereby, the resolution of the thermal image IMG2 which is the first captured image can be increased to generate the composite image IMG3.

The resolution of the second imaging unit 17 is lower than the resolution of the first imaging unit 16.
Thereby, in the imaging device 1, by reducing the captured image of the second imaging unit 17 having a relatively low resolution, the resolution can be increased and combined with the captured image of the first imaging unit 16.

The focal length of the lens provided in the second imaging unit 17 is longer than the focal length of the lens provided in the first imaging unit 16.
Thereby, a captured image can be acquired at a telescopic angle of view by the second imaging unit 17, and a captured image can be acquired at a wider angle of view by the first imaging unit 16.

The second imaging unit 17 and the first imaging unit 16 image light in different wavelength ranges.
Thereby, in the imaging device 1, images representing different characteristics of the subject can be synthesized with higher resolution.

The second imaging unit 17 images infrared light, and the first imaging unit 16 images visible light.
Thereby, in the imaging device 1, the image representing the temperature distribution of the subject and the image representing the appearance of the subject can be synthesized with higher resolution.

In addition, the imaging apparatus 1 performs a predetermined image process on the visible light image IMG1 that is the second captured image or the enlarged image that is the second adjustment image, and generates the contour image IMG11 that is the processed image. Is further provided.
The image composition unit 54 synthesizes the reduced image IMG21 that is the first adjustment image with the contour image IMG11 that is the processed image, and generates a composite image IMG3.
Thereby, in the imaging device 1, it is possible to generate the composite image IMG3 in which the outline of the subject represented as the reduced image IMG21 is easy to understand.

The image synthesizing unit 54 performs a predetermined process on the blank area of the contour image IMG11 that is a processed image generated when the reduced image IMG21 that is the first adjustment image and the contour image IMG11 that is the processed image are combined, A composite image IMG3 is generated.
Thereby, in the imaging device 1, the blank area of the reduced image IMG21 can be configured according to the purpose.

The image composition unit 54 replaces the blank area of the contour image IMG11 that is the processed image with the visible light image IMG1 that is the second captured image or the enlarged image that is the second adjustment image, and generates the composite image IMG3.
Thereby, in the imaging device 1, a blank area around the reduced image IMG21 can be represented as a visible light image IMG1 or an enlarged image.

The image composition unit 54 deletes the blank area of the contour image IMG11 that is the processed image so that the image size of the contour image IMG11 that is the processed image has the same image size as the reduced image IMG21 that is the first adjusted image. To generate a composite image IMG3.
Thereby, in the imaging device 1, it is possible to generate the composite image IMG3 including only a necessary part including the subject.

The image composition unit 54 converts the reduced image IMG21 that is the first adjustment image into any one of the visible light image IMG1 that is the second captured image, the enlarged image that is the second adjustment image, and the contour image IMG11 that is the processed image. Select whether to compose.
The image compositing unit 54 generates a composite image IMG3 by compositing the selected one of the visible light image IMG1 that is the second captured image, the enlarged image that is the second adjustment image, and the contour image IMG11 that is the processed image. To do.
Thereby, in the imaging device 1, the reduced image IMG21 can be combined with any one of the visible light image IMG1, the enlarged image, and the contour image IMG11 according to the purpose.

The second imaging unit 17 images infrared light, and the first imaging unit 16 images visible light.
The predetermined image processing is processing for generating a contour image IMG11 by extracting the contour of the subject from the visible light image IMG1 that is the second captured image or the enlarged image that is the second adjustment image.
Thereby, in the imaging device 1, an image representing the temperature distribution and an image representing the contour of the subject can be synthesized.

The adjustment processing unit 52 includes the thermal image IMG2 and the first captured image according to the ratio of the focal length of the lens included in the second imaging unit 17 and the focal length of the lens included in the first imaging unit 16. Any one or both of the visible light images IMG1 that are the two captured images are adjusted so that the size of the same subject included in each image is the same, and the reduced image IMG21 that is the first adjustment image and Either one or both of the enlarged images that are the second adjusted images are generated.
Thereby, in the imaging device 1, the visible light image IMG1 and the thermal image IMG2 captured at different angles of view can be adjusted to appropriate sizes.

Further, the image composition unit 54 performs control to select the composite image IMG3 or the visible light image IMG1 that is the second captured image and display it on the output unit 19 in accordance with a user operation.
Thereby, the user can switch and display the visible light image IMG1 and the composite image IMG3 of the subject as appropriate.

In addition, the imaging apparatus 1 further includes a recording control unit 55 that selects the composite image IMG3 and / or the visible light image IMG1 that is the second captured image and generates a captured image for recording and storage in response to the imaging instruction. .
Thereby, in the imaging device 1, the composite image IMG3 and the visible light image IMG1 can be appropriately recorded and stored.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  In the above-described embodiment, the light used as infrared light may be not only far infrared light but also near infrared light. In the above-described embodiment, in addition to imaging visible light and infrared light, a combination of visible light and ultraviolet light, a combination of infrared light and ultraviolet light, or the like may be captured.

In the above-described embodiment, the adjustment processing unit 52 reduces the thermal image so that the size of the same subject included in each of the thermal image and the visible light image becomes the same, and the image synthesis unit 54 synthesizes the visible image. Like to do. On the other hand, the adjustment processing unit 52 enlarges the visible light image to generate an enlarged image, and the image synthesis unit 54 cuts out and synthesizes the enlarged image to have the same size as the thermal image, or the adjustment processing unit 52 A reduced image IMG21 in which the thermal image reduction rate is suppressed may be generated, a visible light image may be enlarged by the pressed amount to generate an enlarged image, and the enlarged image may be cut out and combined by the image composition unit 54. . Although the resolution is reduced by enlarging the visible light image, recent image sensors that capture visible light are notable for high resolution even at low cost. Sufficient resolution can be secured if the magnification is the same.
Further, when enlarging the visible light image, the contour extracting unit 53 performs contour extraction from the visible light image magnified by the adjustment processing unit 52, and the contour image and thermal image acquired by the image synthesizing unit 54 are imaged. do it.

In the above-described embodiment, the imaging apparatus 1 to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto.
For example, the present invention can be applied to general electronic devices having a photographing processing function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a smartphone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 4 is merely an example, and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional block is used to realize this function is not particularly limited to the example of FIG.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in this. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disc is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disc), a Blu-ray (registered trademark) Disc (Blu-ray Disc), and the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 20 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
First imaging means for acquiring a first captured image;
A second imaging unit that includes an imaging range of the first imaging unit, captures a wider imaging range than the first imaging unit, and acquires a second captured image;
Adjusting the size of one or both of the first captured image and the second captured image so that the same subject included in each image has the same size, the first adjusted image and An adjusted image generating means for generating one or both of the second adjusted images;
The first adjusted image is combined with the second captured image, or the first captured image is combined with the second adjusted image, or the first adjusted image is combined with the second adjusted image. Image compositing means for generating a composite image by compositing with an image;
An imaging apparatus comprising:
[Appendix 2]
The adjusted image generation means reduces the size of the first captured image so that the size of the same subject included in each of the first captured image and the second captured image is the same. 1 adjustment image is generated,
The imaging apparatus according to appendix 1, wherein the image synthesizing unit generates the synthesized image by synthesizing the first adjusted image with the second captured image or the second adjusted image.
[Appendix 3]
The imaging apparatus according to appendix 1 or 2, wherein the resolution of the first imaging unit is lower than the resolution of the second imaging unit.
[Appendix 4]
The imaging apparatus according to any one of appendices 1 to 3, wherein a focal length of a lens included in the first imaging unit is longer than a focal length of a lens included in the second imaging unit.
[Appendix 5]
The imaging apparatus according to any one of appendices 1 to 4, wherein the first imaging unit and the second imaging unit capture light in different wavelength ranges.
[Appendix 6]
The imaging apparatus according to appendix 5, wherein the first imaging unit images infrared light, and the second imaging unit images visible light.
[Appendix 7]
Processing image generation means for generating a processed image by performing predetermined image processing on the second captured image or the second adjusted image;
The imaging apparatus according to any one of appendices 1 to 5, wherein the image synthesis unit generates the synthesized image by synthesizing the first adjustment image with the processed image.
[Appendix 8]
The image synthesizing unit performs predetermined processing on a margin area of the processed image generated when the first adjusted image and the processed image are combined to generate the combined image. 8. The imaging device according to 7.
[Appendix 9]
The imaging apparatus according to appendix 8, wherein the image synthesizing unit generates the synthesized image by replacing a blank area of the processed image with the second captured image or the second adjusted image.
[Appendix 10]
The image synthesizing unit generates the synthesized image by deleting a blank area of the processed image so that an image size of the processed image is the same as the first adjusted image. The imaging apparatus according to appendix 8.
[Appendix 11]
And further comprising selection means for selecting which of the second captured image, the second adjusted image, and the processed image is to be combined with the first adjusted image.
The image synthesizing unit generates the synthesized image by synthesizing the image with any of the second captured image, the second adjustment image, and the processed image selected by the selection unit. The imaging device according to any one of 1 to 10.
[Appendix 12]
The first imaging means images infrared light, the second imaging means images visible light,
The predetermined image processing is processing for extracting a contour of a subject from the second captured image or the second adjusted image to generate a contour image, and any one of appendices 7 to 11 The imaging device described in 1.
[Appendix 13]
The adjustment image generation unit is configured to control the first captured image and the second image according to a ratio between a focal length of a lens included in the first imaging unit and a focal length of a lens included in the second imaging unit. Either one or both of the captured images of the first image and the second adjusted image are adjusted by adjusting the size of the same subject included in each image to be the same. 13. The imaging device according to any one of appendices 1 to 12, wherein the imaging device is generated.
[Appendix 14]
14. The display device according to any one of appendices 1 to 13, further comprising display control means for performing control to select and display the composite image or the second captured image on a display means in response to a user operation. Imaging device.
[Appendix 15]
Any one of Supplementary notes 1 to 14, further comprising photographing means for selecting the composite image and / or the second picked-up image and generating a picked-up image for recording and saving in response to a photographing instruction. The imaging device described in 1.
[Appendix 16]
A first imaging process for acquiring a first captured image;
A second imaging process that includes an imaging range of the first imaging process and captures a wider imaging range than the first imaging process to obtain a second captured image;
Adjusting the size of one or both of the first captured image and the second captured image so that the same subject included in each image has the same size, the first adjusted image and An adjusted image generation process for generating one or both of the second adjusted images;
The first adjusted image is combined with the second captured image, or the first captured image is combined with the second adjusted image, or the first adjusted image is combined with the second adjusted image. Image compositing processing to generate a composite image by compositing with an image;
An image processing method comprising:
[Appendix 17]
On the computer,
A first imaging function for acquiring a first captured image;
A second imaging function including an imaging range of the first imaging function, capturing a second imaging image by imaging an imaging range wider than the first imaging function;
Adjusting the size of one or both of the first captured image and the second captured image so that the same subject included in each image has the same size, the first adjusted image and An adjusted image generation function for generating one or both of the second adjusted images;
The first adjusted image is combined with the second captured image, or the first captured image is combined with the second adjusted image, or the first adjusted image is combined with the second adjusted image. An image compositing function that generates a composite image by compositing with an image;
A program characterized by realizing.

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input-output interface, 16 ... 1st imaging part, 17. Second imaging unit 18 ... Input unit 19 ... Output unit 20 ... Storage unit 21 ... Communication unit 22 ... Drive 31 ... Removable media 51 ..Image pickup control unit, 52... Adjustment processing unit, 53... Contour extraction unit, 54... Image synthesis unit, 55.

Claims (17)

First imaging means for acquiring a first captured image;
A second imaging unit that includes an imaging range of the first imaging unit, captures a wider imaging range than the first imaging unit, and acquires a second captured image;
Adjusting the size of one or both of the first captured image and the second captured image so that the same subject included in each image has the same size, the first adjusted image and An adjusted image generating means for generating one or both of the second adjusted images;
The first adjusted image is combined with the second captured image, or the first captured image is combined with the second adjusted image, or the first adjusted image is combined with the second adjusted image. Image compositing means for generating a composite image by compositing with an image;
An imaging apparatus comprising: The adjusted image generation means reduces the size of the first captured image so that the size of the same subject included in each of the first captured image and the second captured image is the same. 1 adjustment image is generated,
The imaging apparatus according to claim 1, wherein the image synthesizing unit generates the synthesized image by synthesizing the first adjusted image with the second captured image or the second adjusted image.   The imaging apparatus according to claim 1, wherein a resolution of the first imaging unit is lower than a resolution of the second imaging unit.   4. The imaging apparatus according to claim 1, wherein a focal length of a lens included in the first imaging unit is longer than a focal length of a lens included in the second imaging unit. 5.   5. The imaging apparatus according to claim 1, wherein the first imaging unit and the second imaging unit capture light in different wavelength ranges. 6.   The imaging apparatus according to claim 5, wherein the first imaging unit images infrared light, and the second imaging unit images visible light. Processing image generation means for generating a processed image by performing predetermined image processing on the second captured image or the second adjusted image;
6. The imaging apparatus according to claim 1, wherein the image synthesizing unit generates the synthesized image by synthesizing the first adjusted image with the processed image. 7.   The image synthesizing unit generates a synthesized image by performing a predetermined process on a margin area of the processed image generated when the first adjusted image and the processed image are synthesized. Item 8. The imaging device according to Item 7.   The imaging apparatus according to claim 8, wherein the image synthesis unit generates the synthesized image by replacing a blank area of the processed image with the second captured image or the second adjusted image.   The image synthesizing unit generates the synthesized image by deleting a blank area of the processed image so that an image size of the processed image is the same as the first adjusted image. The imaging device according to claim 8. And further comprising selection means for selecting which of the second captured image, the second adjusted image, and the processed image is to be combined with the first adjusted image.
The image synthesizing unit generates the synthesized image by synthesizing the image with any of the second captured image, the second adjusted image, and the processed image selected by the selecting unit. The imaging apparatus according to any one of 7 to 10. The first imaging means images infrared light, the second imaging means images visible light,
12. The process according to claim 7, wherein the predetermined image processing is processing for extracting a contour of a subject from the second captured image or the second adjustment image to generate a contour image. The imaging device according to item.   The adjustment image generation unit is configured to control the first captured image and the second image according to a ratio between a focal length of a lens included in the first imaging unit and a focal length of a lens included in the second imaging unit. Either one or both of the captured images of the first image and the second adjusted image are adjusted by adjusting the size of the same subject included in each image to be the same. The imaging device according to any one of claims 1 to 12, wherein:   14. The display control unit according to claim 1, further comprising a display control unit that performs control to select and display the composite image or the second captured image on a display unit according to a user operation. The imaging device described.   15. The apparatus according to claim 1, further comprising a photographing unit that selects the composite image and / or the second captured image in accordance with a photographing instruction and generates a recorded image for recording and storage. The imaging device according to item. A first imaging process for acquiring a first captured image;
A second imaging process that includes an imaging range of the first imaging process and captures a wider imaging range than the first imaging process to obtain a second captured image;
Adjusting the size of one or both of the first captured image and the second captured image so that the same subject included in each image has the same size, the first adjusted image and An adjusted image generation process for generating one or both of the second adjusted images;
The first adjusted image is combined with the second captured image, or the first captured image is combined with the second adjusted image, or the first adjusted image is combined with the second adjusted image. Image compositing processing to generate a composite image by compositing with an image;
An image processing method comprising: On the computer,
A first imaging function for acquiring a first captured image;
A second imaging function including an imaging range of the first imaging function, capturing a second imaging image by imaging an imaging range wider than the first imaging function;
Adjusting the size of one or both of the first captured image and the second captured image so that the same subject included in each image has the same size, the first adjusted image and An adjusted image generation function for generating one or both of the second adjusted images;
The first adjusted image is combined with the second captured image, or the first captured image is combined with the second adjusted image, or the first adjusted image is combined with the second adjusted image. An image compositing function that generates a composite image by compositing with an image;
A program characterized by realizing.

Images