JP2013197894A - Image processing device and image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a user to grasp a size of a subject intuitively.SOLUTION: An image acquisition unit 51 acquires raw image data including the subject. On the basis of the raw image data acquired by the image acquisition unit 51, an information acquisition unit 42 acquires, as information on the main subject image included in the raw image, data on the length (exact size) thereof, for example. An indicator image selection unit 53 searches for an indicator image indicating an object having a length within a predetermined range of the length of the main subject to select an indicator image to be combined with the raw image. An indicator image conversion unit 54 converts the indicator image data into image data constituting information in which the indicator image is expressed in the same scale as the raw image. An image synthesis unit 55 generates combined image data in which the indicator image is shown on the top of the raw image.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program capable of intuitively grasping subject information.

Conventionally, when an object is photographed, an object that serves as an index of the size is placed next to the subject in order to convey the actual size of the subject image to a viewer who appreciates the photographed subject image. Has been done.
For example, when shooting a subject with a length of several centimeters, place a coin with a known diameter (for example, a 10-yen coin with a diameter of 2.5 cm issued in Japan) beside the subject, Shoot with. In the original image obtained, an image of a coin that allows the viewer to grasp the size is copied, so that the viewer can compare the image of the coin in the image with the image of the subject. The actual size can be recognized.

However, there are situations in which it is difficult to shoot with an object that serves as a size index beside the subject. For example, when a large object such as a building or a bridge is a subject, it is difficult to actually place an object serving as a size index beside the subject. Even when a small object is a subject, the user can accurately recognize the size of the subject if he / she does not have an appropriate object as an index of size at the time of shooting. Can't get.
Therefore, Patent Document 1 discloses a technique for displaying a scale for accurately grasping the size of a subject on an image including the subject image.

Japanese Patent No. 3059773

  However, in the technique described in Patent Document 1, the viewer tries to grasp the size of the subject from the displayed scale, but the information that the viewer can obtain is a numerical value corresponding to the length of the subject. Is an expression. Even if the viewer can obtain accurate numerical information such as the length of the subject, it may be difficult to intuitively grasp the subject information.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an image processing apparatus, an image processing method, and a program capable of intuitively grasping subject information.

  To achieve the above object, an image processing apparatus according to an aspect of the present invention includes an acquisition unit that acquires an image, an information acquisition unit that acquires information about a subject image included in the image acquired by the acquisition unit, Based on information acquired by the information acquisition means, a selection means for selecting an index image corresponding to the subject image, information acquired by the information acquisition means, and information on the index image selected by the selection means The image processing apparatus includes: conversion means for converting the index image, and combining means for combining the image with the index image converted by the conversion means.

  According to the present invention, it is possible to intuitively grasp subject information.

1 is a block diagram illustrating a hardware configuration of an image processing apparatus according to an embodiment of the present invention. It is a functional block diagram which shows the functional structure for performing an index image display process among the functional structures of the image processing apparatus of FIG. It is a schematic diagram which shows the method of measuring the information of a main subject. It is a schematic diagram which shows the specific example of the index image used for an index image display process. It is a schematic diagram which shows each image as a specific example of the result of an index image display process. It is a flowchart which shows the flow of the index image display process which the image processing apparatus of FIG. 1 which has the functional structure of FIG. 2 performs. It is a block diagram which shows the hardware constitutions of the image processing apparatus which concerns on 2nd Embodiment of this invention. It is a functional block diagram which shows the functional structure for performing an index image display process among the functional structures of the image processing apparatus of FIG. It is a schematic diagram which shows each image as a specific example of the result of the index image display process which concerns on 2nd Embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram showing a hardware configuration of an image processing apparatus according to an embodiment of the present invention.
The image processing apparatus 1 can be configured by a digital camera, for example. In the present embodiment, the image processing apparatus 1 is described as having an imaging function. However, the imaging function and the image processing function of the image processing apparatus 1 are separated, and image data captured by another imaging apparatus. It is also possible for the image processing apparatus 1 to execute various processes for the above.

  The image processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a DSP (Digital Signal Processor) 14, a bus 15, and an input / output interface 16. An imaging unit 17, an operation unit 18, a display unit 19, a storage unit 20, a communication unit 21, and a drive 22.

The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 20 to the RAM 13.
The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The DSP 14 performs various image processing on the image data in accordance with instructions from the CPU 11.

  The CPU 11, ROM 12, RAM 13, and DSP 14 are connected to each other via a bus 15. An input / output interface 16 is also connected to the bus 15. An imaging unit 17, an operation unit 18, a display unit 19, a storage unit 20, a communication unit 21, and a drive 22 are connected to the input / output interface 16.

  The imaging unit 17 includes an optical lens unit 31, an image sensor 32, and a lens driving unit 33.

The optical lens unit 31 includes various lenses such as a focus lens (a focus lens 31F in FIG. 3 described later) and a zoom lens (a zoom lens 31Z in FIG. 3 described later) in order to photograph a subject.
The focus lens is a lens that is driven by a lens driving unit 33 described later, and forms an image of a subject on a light receiving surface of an image sensor 32 described later. The zoom lens is a lens that freely changes the focal length within a certain range.

The image sensor 32 includes a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit 31. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing and output as an output signal of the imaging unit 17.
Hereinafter, the output signal of the imaging unit 17 is referred to as “captured image data”. Data of the captured image is output from the imaging unit 17 and is appropriately supplied to the CPU 11, DSP 14, and the like.

  The lens driving unit 33 drives the focus lens of the optical lens unit 31 based on the result of AF (Auto Focus) processing by the CPU 11 and changes the lens position to focus on a predetermined subject.

The operation unit 18 includes various buttons and the like, and accepts user instruction operations. Moreover, the operation part 18 is comprised so that the touch panel laminated | stacked on the display part 19 may be included as needed.
The display unit 19 is composed of a liquid crystal display or the like, and displays various images (for example, a composite image described in detail later).

  The storage unit 20 includes a DRAM (Dynamic Random Access Memory) or the like, and stores image data output from the DSP 14 or the like. The storage unit 20 also stores various data necessary for processing of the DSP 14 and the like.

  The communication unit 21 controls communication with other devices (not shown) via a network including the Internet.

  A removable medium 34 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 22. The program read from the removable medium 34 by the drive 22 is installed in the storage unit 20 as necessary. The removable medium 34 can also store various data such as image data stored in the storage unit 20 in the same manner as the storage unit 20. In the present embodiment, the removable medium 34 stores data such as a composite image described later.

  FIG. 2 is a functional block diagram showing a functional configuration for executing the index image display process among the functional configurations of the image processing apparatus of FIG.

  The index image display process is an image (an index image to be described later) that acquires data of an original image such as a captured image and makes a user grasp information indicating the size of a subject included in the original image. ) Is a series of processing until it is displayed together with the original image.

When the index image display process is executed, the main subject detection unit 41, the information acquisition unit 42, the display control unit 43, and the storage control unit 44 function in the CPU 11.
Note that at least a part of the functions of the main subject detection unit 41 to the storage control unit 44 may be transferred to other components (such as the DSP 14) than the CPU 11.

When the index image display process is executed, in the DSP 14, the image acquisition unit 51, the index image acquisition unit 52, the index image selection unit 53, the index image conversion unit 54, the image composition unit 55, the index The image generation unit 56 functions.
Of course, it is possible to transfer at least part of the functions of the image acquisition unit 51 to the index image generation unit 56 to other components (CPU 11 or the like) other than the DSP 14.

The image acquisition unit 51 acquires captured image data output from the imaging unit 17 as original image data, and acquires imaging condition data. Here, the imaging condition data refers to data such as a zoom value, a focus value, a focus area (a focusing area of a focus lens in a captured image), and the like.
In the present embodiment, it is assumed that the original image data is in an Exif (Exchange Image File Format for Digital Still Camera) format. Here, since Exif is a standard that allows tag information to be added to image data and stored, data such as imaging conditions is added to the original image data as tag information.

When the original image data is supplied from the image acquisition unit 51, the main subject detection unit 41 detects the main subject image included in the original image, and supplies the original image data to the information acquisition unit 42 together with the detection result. To do.
Here, the main subject refers to a subject on which the focus lens is focused. For this reason, at least a part of the image data in the focus area is data of the main subject image.
The detection method of the main subject image is not particularly limited. For example, in the present embodiment, in the focus area or in the focus area based on the focus area data included in the tag information added to the original image data. A detection method is adopted in which a main subject image is detected from the contrast gradient of an image within a predetermined range including.

After detecting the main subject image, the main subject detection unit 41 determines a minimum rectangular frame surrounding the main subject image (hereinafter referred to as a “main subject image frame”). In the present embodiment, the frame of the main subject image includes the horizontal width obtained from the difference between the maximum and minimum coordinates in the X-axis direction (lateral direction) of the image area of the main subject image, and the area of the main subject image. It is an outer edge of a rectangular area having a vertical width obtained from the difference between the maximum value and the minimum value of coordinates in the Y-axis direction (vertical direction).
The frame shape of the main subject image can be determined if the four vertices of the frame are specified. Therefore, in the present embodiment, it is assumed that the frame data of the main subject image is configured by positional information of the four vertices of the frame.
The main subject detection unit 41 supplies the data of the frame of the main subject image as a detection result to the information acquisition unit 42 together with the original image data.

When the main subject image frame data and the original image data are supplied from the main subject detection unit 41, the information acquisition unit 42 is information indicating the size of the main subject in the real world (for example, data of the actual length). ) To get. In the present embodiment, as will be described later, information on the size of the main subject in the real world is measured from information such as the position of the focus lens obtained at the time of imaging. Note that this size information can be acquired from another apparatus together with the captured image data.
The method of measuring information (for example, length data) of the main subject by the information acquisition unit 42 is not particularly limited. For example, in the present embodiment, the ratio of the distance to the subject and the length of the subject image in the imaging range is used. A measurement technique is employed in which the length of the main subject is measured based on the information.

Hereinafter, a specific measuring method of the size information (for example, length information) of the main subject employed in the present embodiment will be described with reference to FIG.
FIG. 3 is a schematic diagram showing a method for measuring the size (for example, length) of the main subject.

  FIG. 3A shows an original image 71 including images of subjects such as a fisherman OB1, a rock OB2, a forest OB3, and a mountain OB4. Here, in the real world (imaging space of the image processing apparatus 1 configured as a digital camera), the distance from the image processing apparatus 1 is as follows: angler OB1, rock OB2, forest OB3, and mountain OB4. .

式（１）において、Ｌは、フォーカスレンズ３１Ｆと被写体との距離を、ｂは、フォーカスレンズ３１Ｆとイメージセンサ３２との距離を、ｆは、焦点距離を、示している。
レンズの焦点距離ｆは予め定められているので、被写体距離Ｌと、フォーカスレンズ３１Ｆとイメージセンサ３２との距離ｂは、一方が大きくなれば他方が小さくなる関係にある。
図３（Ｂ）に示すように、フォーカスレンズ３１Ｆと釣り人ＯＢ１、岩ＯＢ２、林ＯＢ３、山ＯＢ４との距離は、それぞれ１ｍ、５ｍ、１５ｍ、無限大であるものとみなされている。この場合、上記のそれぞれの被写体に合焦するフォーカスレンズ３１Ｆのレンズアドレスは、釣り人ＯＢ１、岩ＯＢ２、林ＯＢ３、山ＯＢ４の順にイメージセンサ３２に近くなる。 FIG. 3B is a diagram for explaining the relationship between the position of the focus lens 31F (hereinafter referred to as “lens address”) of the image processing apparatus 1 and the distance to the subject on which the focus lens 31F is focused.
As shown in FIG. 3B, the lens address is located between the zoom lens 31L and the image sensor 32. The farther the lens address is from the image sensor 32, the closer the in-focus position in the real world is to the image processing apparatus 1.
The distance from the focus lens 31F to the subject (hereinafter referred to as “subject distance”) and the distance from the focus lens 31F to the image sensor 32 are related by the following formula (1) which is a lens formula.

In Expression (1), L represents the distance between the focus lens 31F and the subject, b represents the distance between the focus lens 31F and the image sensor 32, and f represents the focal length.
Since the focal length f of the lens is determined in advance, the subject distance L and the distance b between the focus lens 31F and the image sensor 32 have a relationship in which the larger the one, the smaller the other.
As shown in FIG. 3B, the distances between the focus lens 31F and the angler OB1, rock OB2, forest OB3, and mountain OB4 are considered to be 1 m, 5 m, 15 m, and infinity, respectively. In this case, the lens address of the focus lens 31F that focuses on each of the subjects is close to the image sensor 32 in the order of angler OB1, rock OB2, forest OB3, and mountain OB4.

式（２）において、δは、許容錯乱円の直径を、Ｌは、被写体距離を、Ｆは、Ｆ値（絞り値）を、ｆは、焦点距離を、示している。
ここで、被写体距離Ｌがｆ・ｆ／（δ・Ｆ）の値をとる場合、分母が０となるので、後方被写界深度Ｌｒは無限大となる。したがって、被写体距離Ｌ＝ｆ・ｆ／（δ・Ｆ）に位置する被写体に、フォーカスレンズ３１Ｆを合焦させたときは、被写体だけでなく無限遠までの物体にピントがあっているように見えることになる。このため、フォーカスレンズ３１Ｆとイメージセンサ３２との距離ｂが一定値以下に達したときは、実用上、被写体距離Ｌが無限大までの被写体に合焦したものとして被写体距離は無限大として扱われる。 Here, in the above description, the distance between the focus lens 31F and the mountain OB4 is assumed to be infinite, but the fact that the focal length is infinite will be further described.
Generally, when a subject is focused, there is a range in which the subject can be regarded as being in focus before and after the subject. This range is called depth of field. Then, the depth of field on the rear side of the subject as viewed from the image processing apparatus 1, that is, the rear depth of field Lr is expressed by the following equation (2).

In Expression (2), δ represents the diameter of the allowable circle of confusion, L represents the subject distance, F represents the F value (aperture value), and f represents the focal length.
Here, when the subject distance L takes a value of f · f / (δ · F), the denominator is 0, and the rear depth of field Lr is infinite. Therefore, when the focus lens 31F is focused on a subject located at a subject distance L = f · f / (δ · F), it seems that not only the subject but also an object up to infinity is in focus. It will be. For this reason, when the distance b between the focus lens 31F and the image sensor 32 reaches a certain value or less, the subject distance is treated as infinite as being practically focused on the subject up to the subject distance L being infinite. .

Returning to Equation (1), description will be made. In Expression (1), since the focal length f of the lens is determined in advance, if one of the object distance L and the distance b between the focus lens 31F and the image sensor 32 is determined, the other value is obtained. It can be seen that there is a relationship.
In the present embodiment, the lens address is output from the imaging unit 17 as imaging condition data. Since the image sensor 32 is normally in a fixed position, when the lens address is determined, the distance b between the focus lens 31F and the image sensor 32 is calculated. That is, when the lens address is determined, the subject distance L is calculated.
That is, the subject distance L can be calculated by the equation (1) based on the lens address information when the focus lens 31F is focused on the subject.
Therefore, the information acquisition unit 42 calculates the subject distance based on the lens address and focal length information recorded in the tag information of the original image data. Then, the information acquisition unit 42 records the calculated subject distance in the tag information of the original image data.

式（３）において、Ｌ０は、水平画角に相当する撮像範囲の幅（実寸）を、Ｗ０は、原画像の水平幅を、Ｗ１は、原画像に含まれる被写体像の水平幅を、示している。
また、主要被写体像の水平幅Ｗ１は、主要被写体像の枠の水平幅を測定することにより求められる。 FIG. 3C shows the horizontal width W0 of the original image (the horizontal width of the original image in the shooting range corresponding to the horizontal angle of view) and the horizontal width W1 of the angler's image.
Here, a plane in the real world having the normal of the optical axis of the focus lens of the image processing apparatus 1 and a plane including the angler who is the subject is considered. The horizontal width of the imaging range projected by the angle formed by the horizontal angle of view on the plane, that is, the width (actual size) L0 of the imaging range can be obtained geometrically from the subject distance and the focal length of the focus lens 31F.
Since the imaging plane or the captured image and the plane are similar, the ratio of the imaging range width (actual size) L0 to the original image horizontal width W0 is the subject horizontal width (actual size) L1 and the subject. Is equal to the ratio to the horizontal width W1 of the angler's image.
From the above, the horizontal width (actual size) L1 of the subject can be obtained by the following equation (3).

In Expression (3), L0 indicates the width (actual size) of the imaging range corresponding to the horizontal angle of view, W0 indicates the horizontal width of the original image, and W1 indicates the horizontal width of the subject image included in the original image. ing.
Further, the horizontal width W1 of the main subject image can be obtained by measuring the horizontal width of the frame of the main subject image.

The measurement method for measuring the width (actual size) of the subject has been described above. When measuring the length of the subject in the vertical direction, the vertical angle of view and the vertical width may be used instead of the horizontal angle of view and the horizontal width in the measurement method described above.
As described with reference to FIGS. 3A to 3C, when the lens address of the focus lens and the ratio of the length of the subject image to the horizontal width of the original image are determined, the length of the subject (actual size) ) Can be measured.

Hereinafter, returning to FIG. 2, each function of the index image display processing will be described.
In the present embodiment, the information acquisition unit 42 measures the length of the main subject in the longitudinal direction as the length of the main subject. The horizontal length of the main subject image can be obtained by measuring the horizontal width of the main subject image frame. Further, the vertical length of the main subject image can be obtained by measuring the vertical width of the main subject image frame.
The information on the measurement target is not particularly limited to the length, and may be any information as long as the information indicates the size, such as the area of the main subject.

  The information acquisition unit 42 measures main subject information (for example, length data), and then supplies the information to the index image acquisition unit 52. Then, as will be described later, when generating index image data from original image data, the information acquisition unit 42 supplies the information and main subject frame data to the index image generation unit 56.

  Further, the information acquisition unit 42 calculates the ratio of the horizontal width (actual size) L1 of the subject and the horizontal width W1 of the subject image in the process of obtaining the length (actual size) of the subject. That is, the information acquisition unit 42 has a ratio of the length of the subject image displayed on the display unit 19 to the length (actual size) of the subject in the real world (hereinafter referred to as “scale information”), that is, W1 / L1. Is calculated. Then, the information acquisition unit 42 supplies the calculated scale information of the original image to the index image conversion unit 54.

The index image acquisition unit 52 acquires information (for example, length data) of the main subject measured by the information acquisition unit 42 from the information acquisition unit 42.
Further, the index image acquisition unit 52 acquires data of an image (hereinafter referred to as “index image”) serving as an index that allows the user to intuitively grasp the information of the main subject from the storage unit 20.
In the present embodiment, the index image acquisition unit 52 acquires the index image data from the storage unit 20, but the present invention is not limited to this. The index image acquisition unit 52 may acquire index image data from a website on the Internet via the communication unit 21. In this case, the communication unit 21 controls communication performed with the Web server via a network including the Internet. Further, the index image acquisition unit 52 may acquire index image data stored in the removable medium 34.

The object represented by the index image does not necessarily have to be recognized by the viewer, such as the length of the object itself, and may be any object that the viewer can intuitively grasp the length of the object. . For example, the object represented by the index image is preferably an object that is regularly used by a viewer, an object that has been seen empirically, a building, or the like.
FIG. 4 is a schematic diagram illustrating a specific example of the index image used for the index image display processing.
FIG. 4 includes (A) an index image C1 indicating a cigarette box, (B) an index image C2 indicating a 350 ml can for beverages, (C) an index image C3 indicating a beverage bottle, and an index image C4 indicating a Buddha. And the index image C5 showing the T tower is shown.
The form of the index image data is not particularly limited, but in the present embodiment, it is assumed to be an Exif format image file. In the present embodiment, the tag information of the Exif format image file includes ID (Identification) information of the index image and information (for example, length) indicating the size of the object represented by the index image in the real world. It shall be.

Hereinafter, returning to FIG. 2, each function of the functional block when the index image display process is executed will be described.
The index image acquisition unit 52 supplies information (for example, length data) of the main subject image and index image data measured by the information acquisition unit 42 to the index image selection unit 53.

  The index image selection unit 53 selects index image data corresponding to the main subject image based on the information acquired by the information acquisition unit 42.

Specifically, the index image selection unit 53 acquires information on the main subject image (for example, length data) and index image data from the index image acquisition unit 52.
The index image selection unit 53 selects index image data indicating an object (an object having a length within a predetermined range of the length of the main subject) having information approximate to the information of the main subject image. Here, the predetermined range may be a range of a ratio to the length of the main subject (for example, 0.7 to 1.3 times the length of the main subject), or a range of difference (for example, the length of the main subject). The difference may be within ± 5 cm). Further, these ratios and differences may be changed according to the length of the main subject.

When the index image selection unit 53 selects an index image (an index image having a length within a predetermined range of the length of the main subject) having information approximate to the information of the main subject image, the index image data is displayed. It supplies to the control part 43. The display control unit 43 controls the display unit 19 to display the index image and an image from which the user selects the index image.
Then, the user operates the operation unit 18 from the index images displayed on the display unit 19 to select and specify a desired index image. ID information of the index image designated by the user (hereinafter referred to as “designated index image”) is notified to the index image selection unit 53.
Next, the user operates the operation unit 18 to designate a position (hereinafter referred to as “designated position”) where the designated index image is displayed superimposed on the original image. At this time, the user may determine the designated position using a touch panel or the like on which the original image is displayed.
The index image selection unit 53 supplies the ID information of the designated index image to the index image conversion unit 54.
In addition, the index image selection unit 53 supplies the information on the designated position to the image composition unit 55.

The index image conversion unit 54 acquires index image data corresponding to the ID information of the index image notified from the index image selection unit 53 from the storage unit 20, and acquires scale information of the original image from the information acquisition unit 42. . Then, the index image conversion unit 54 has the same ratio of the length of the object represented by the index image in the real world and the length of the displayed index image as the ratio indicated by the scale information of the original image. Is converted to
The index image data converted by the index image conversion unit 54 is hereinafter referred to as “converted index image data”.
The index image conversion unit 54 supplies the converted index image data to the image synthesis unit 55.

The image composition unit 55 acquires the original image data from the image acquisition unit 51.
Further, the image composition unit 55 acquires the data of the conversion index image from the index image conversion unit 54.
Further, the image composition unit 55 acquires information on the designated position from the index image selection unit 53.
Then, the image composition unit 55 synthesizes the data of the conversion index image with the data of the original image based on the information on the designated position, thereby superimposing the conversion index image on the designated position in the original image. (Hereinafter referred to as “composite image”).

In some cases, the user desires to use the main subject image included in the original image as an index image in the next and subsequent imaging. In such a case, the user operates the operation unit 18 to instruct to generate an index image from the main subject image.
Upon receiving the index image generation instruction, the index image generation unit 56 receives the original image data from the image acquisition unit 51, the main subject information (for example, length data) from the information acquisition unit 42, and the main subject image frame. Receive data. Then, the index image generation unit 56 generates data of the image area within the frame of the main subject image as index image data. At this time, the storage control unit 44 controls to store the generated index image data in the storage unit 20. At the same time, the storage control unit 44 controls to store the information on the main subject (length (actual size) data) in the tag information added to the index image data.

  The display control unit 43 executes control for causing the display unit 19 to display an original image, a composite image, and the like.

  The storage control unit 44 records the composite image data generated by the image composition unit 55 on the removable medium 34 as necessary, for example, when there is an instruction from the user.

  The functional configuration of the image processing apparatus 1 according to the present embodiment has been described above with reference to FIGS.

Next, details of each function of the main subject detection unit 41 to storage control unit 44 configured by the CPU 11 and the image acquisition unit 51 to index image generation unit 56 configured by the DSP 14, that is, index image display processing. Details of the execution function will be specifically described with reference to FIG.
FIG. 5 is a schematic diagram showing each image as a specific example of the result of the index image display process.
FIG. 5A shows an original image 81 including subject images of anglers 91 and fish 92.
In FIG. 5A, a frame FW5 indicating a focus area is shown near the center of the fish 92 image. That is, the focus lens is positioned at a lens address that focuses on the vicinity of the center of the fish as the subject.

Then, the main subject detection unit 41 detects the main subject image, that is, the image of the fish 92, from the contrast gradient of the original image within a predetermined range including the focus area within the frame FW5 indicating the focus area.
In this case, the display control unit 43 causes the display unit 19 to display a symbol or the like that can identify the detected main subject superimposed on the original image 81. Specifically, in the example of FIG. 5A, for example, the fish 92 is specified as the main subject by the frame CW5 of the main subject image.

Next, the information acquisition unit 42 acquires the length of the main subject, that is, the length of the fish 92 in the longitudinal direction (vertical direction) as information on the main subject image. In the present embodiment, it is assumed that the length of the fish 92 is measured by the information acquisition unit 42 to be 22 cm. The information acquisition unit 42 supplies the data of the length of the fish 92 measured to the index image acquisition unit 52 as information on the main subject image.
The index image acquisition unit 52 acquires information (length data) of the main subject image supplied from the information acquisition unit 42 and acquires index image data from the storage unit 20.
The index image acquisition unit 52 supplies index image data and main subject image information (length data) to the index image selection unit 53.

The index image selection unit 53 has a length within a predetermined range with respect to the length of the main subject based on the index image data and the main subject image information (length data) supplied from the index image acquisition unit 52. The index image data indicating the object is selected.
In this embodiment, when the length of the main subject is 30 cm or less, the index image selection unit 53 selects an object having a length within a range of 0.4 to 1.2 times with respect to the length of the main subject. Assume that an index image is selected. That is, the index image acquisition unit 52 stores index image data of an object having a length in the range of 8.8 cm to 26 cm corresponding to 0.4 times to 1.2 times the length 22 cm of the main subject fish 92. Select. As a result, the index image selection unit 53 displays an index image C1 indicating a cigarette box having a length of 9 cm, an index image C2 indicating a 350 ml can for beverage having a length of 12 cm, and an index image C3 indicating a plastic bottle for beverage having a length of 20 cm. Are determined as candidates for the designated index image.

  The index image selection unit 53 notifies the display control unit 43 to display the index image C1, C2, C3 and a display screen for selecting the index image (hereinafter referred to as “index image selection screen”). To do.

Then, the display control unit 43 causes the display unit 19 to display the index image selection screen KW1. As shown in FIG. 5B, in the index image selection screen KW1, candidates for the designated index image determined by the index image selection unit 53 are indexed from the top to the bottom in the index image selection screen KW1. Index images C3, C2, and C1 are displayed in descending order of the length of the object represented by the image.
Therefore, the user can easily grasp the magnitude relationship between the lengths of the objects indicated by the index images in the index image selection screen KW1.

Further, a scale SL1 is displayed beside the candidate image for the designated index image. The instruction lines CL1, CL2, and CL3 indicate scales corresponding to the lengths of the objects indicated by the index images C1, C2, and C3, respectively. That is, the instruction lines CL1, CL2, and CL3 indicate that the objects indicated by the index images C1, C2, and C3 are 9 cm, 12 cm, and 20 cm, respectively. The user can grasp the length of the object indicated by the index image as a numerical value by reading the scales indicated by these instruction lines.
Further, in the index image selection screen KW1, the arrow AR1 indicates the scale SL1 corresponding to the length of 22 cm of the main subject fish 92. The user can grasp the length of the main subject as a numerical value by reading the scale indicated by the arrow.

Next, the user operates the operation unit 18 from among the index images C1, C2, and C3 displayed in the index image selection screen KW1, and shows the index image C1 indicating a cigarette box and a 350 ml can for beverages. Assume that the index image C2 is designated as the designated index image. In this case, the index image selection unit 53 is notified of the ID information of the index image designated by the user.
Furthermore, it is assumed that the user operates the operation unit 18 and designates the designated index images, that is, the index images C1 and C2 to be arranged on the right side and the left side of the image of the fish 92, respectively.
The index image selection unit 53 supplies the ID information of the data of the index images C1 and C2 to the index image conversion unit 54.

The index image conversion unit 54 converts the index image data C1 and C2 corresponding to the ID information of the index image data supplied from the index image selection unit 53 to the same converted index image D1 and D2 data as the scale information of the original image. Convert to
The index image conversion unit 54 supplies the data of the conversion index images D1 and D2 to the image synthesis unit 55.

The image composition unit 55 includes data of the original image supplied from the image acquisition unit 51, information on the designated position supplied from the index image selection unit 53, and conversion index images D2 and D3 supplied from the index image conversion unit 54. Data and information on the specified position. Then, the image composition unit 55 generates data of the composite image 101 that is displayed by superimposing the conversion index image on the designated position in the original image.
Then, the display control unit 43 causes the display unit 19 to display the composite image 101 as shown in FIG.
In FIGS. 5A and 5B, a frame FW5 indicating a focus area is shown for convenience of explanation, but such a display of the frame of the focus area is not particularly essential, and actually May be omitted.

The index image generation unit 56 generates screen data that allows the user to select whether or not to generate data of an image area within the frame of the main subject image as new index image data. The display control unit 43 causes the display unit 19 to display the screen.
When the index image generation unit 56 receives a notification to generate an index image by the operation of the operation unit 18 by the user, the index image generation unit 56 receives the original image data from the image acquisition unit 51 and the information on the main subject image from the information acquisition unit 42 Then, the data of the length of the fish 92) and the data of the frame CW5 of the main subject image are received.
Then, the index image generation unit 56 generates data of an image area including the image of the fish 92 within the frame CW5 of the main subject image as new index image data. At this time, the storage control unit 44 records the length data of the fish 92 as the main subject in the tag information added to the generated index image data.

  The storage control unit 44 stores the data of the composite image 101 in the removable medium 34 as necessary.

The index image data only needs to be stored in the storage unit 20 at the stage of being acquired by the index image acquisition unit 52.
When the user operates the operation unit 18 and the index image generation unit 56 generates new index image data as described above, the index image data is stored in the storage unit 20 afterwards. You can also.
Furthermore, index image data obtained via the communication unit 21, the drive 22, and the like can be stored in the storage unit 20.

  In this embodiment, object information (for example, data such as length) indicated by the index image and imaging condition data are stored as tag information added to the index image data. However, the present invention is not particularly limited thereto. For example, a table in which information (for example, length data) of the index image is associated with each of the plurality of index image data may be referred to.

Next, the flow of the index image display process among the processes executed by the image processing apparatus 1 having such a functional configuration will be described.
FIG. 6 is a flowchart showing the flow of the index image display process executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.

  In this embodiment, the index image display process is started when the user operates the operation unit 18 to instruct the index image display process and then the imaging operation by the imaging unit 17 is performed. .

  In step S <b> 1, the image acquisition unit 51 acquires captured image data output from the imaging unit 17 as original image data. For example, it is assumed that the data of the original image 81 in FIG.

  In step S2, the main subject detection unit 41 detects the main subject image in the original image based on the original image data acquired in the process of step S1. For example, as shown in FIG. 5A, the main subject detection unit 41 detects a fish 92 image surrounded by a frame of the main subject image as the main subject image.

  In step S3, the information acquisition unit 42 acquires data on the length of the main subject detected in the process of step S2 as information on the main subject image. For example, the information acquisition unit 42 acquires data on the length (actual size) in the longitudinal direction (vertical direction) of the fish 92 detected in the process of step S2 as information on the main subject image. For example, it is assumed that a length of 22 cm of the fish 92 as the main subject is measured, and data of the measurement result is acquired as information on the main subject image.

In step S4, the index image acquisition unit 52 selects index image data within a predetermined range of information of the main subject image. Specifically, the index image acquisition unit 52 acquires information (length data) of the main subject image from the information acquisition unit 42 and acquires index image data from the storage unit 20. Then, when the index image data and the length data of the main subject are supplied from the index image acquisition unit 52, the index image selection unit 53 indicates an object having a length within a predetermined range of the length of the main subject. Select image data.
For example, here, the length of the main subject fish 92 is measured to be 22 cm, and the predetermined range is set to 0.4 to 1.2 times the length of the main subject. Therefore, an index image in which the length of the object is in the range of 8.8 cm to 26 cm, that is, an index image C1 indicating a cigarette box having a length of 9 cm, and an index indicating a 350 ml can for beverage having a length of 12 cm. Each data of the image C2 and the index image C3 indicating a beverage plastic bottle having a length of 20 cm is selected.

In step S5, the display control unit 43 causes the display unit 19 to display an index image selection screen.
For example, in the present embodiment, an index image selection screen KW1 is displayed on the display unit 19 as shown in FIG. In the index image selection screen KW1, index images C3, C2, and C1 are displayed in descending order from top to bottom.

  In step S6, the index image selection unit 53 notifies the user to select the specified index image from the index images in the index image selection screen.

  In step S7, the index image selection unit 53 determines whether an index image has been selected.

  When the index image is not selected, it is determined as NO in Step S7, and the process returns to Step S6. That is, the index image selection unit 53 notifies the user again so as to select the designated index image from among the index images in the index image selection screen. Until the index image is selected, the index image display process enters a standby state by repeatedly executing the loop processing of steps S6 and S7. Although not shown, the index image display process may be terminated if the index image is not yet selected even after a predetermined time has elapsed.

  Thereafter, when the index image is selected, it is determined as YES in Step S7, and the process proceeds to Step S8.

  In step S8, the index image selection unit 53 determines the designated position in the original image of the designated index image by the operation of the operation unit 18 by the user. In response to this, the index image conversion unit 54 converts the data of the specified index image into the data of the converted index image so that the scale information is the same as the scale information of the original image.

  In step S <b> 9, the image composition unit 55 generates composite image data by combining the original image data acquired in step S <b> 1 and the conversion index image data generated in step S <b> 8. To do.

  In step S <b> 10, the display control unit 43 causes the display unit 19 to display the composite image data generated in the process of step S <b> 9.

In step S11, the index image conversion unit 54 determines whether or not all index image selections have been completed. If all the processes have not been completed, it is determined as NO in Step S11, and the process returns to Step S7. In other words, when the user desires to display a plurality of index images superimposed on the original image, the process returns to step S7 to select the index image again, thereby superimposing the plurality of index images on the original image. Can be displayed.
In this embodiment, as shown in FIG. 5C, data of the composite image 101 in which the index image C1 of the cigarette box and the index image C2 of the 350 ml can for beverage are superimposed next to the fish 92 of the original image 81. Is generated.
If all the index images have been selected, it is determined as YES in Step S11, and the process proceeds to Step S12.

In step S12, the storage control unit 44 determines whether or not there is a save instruction.
That is, the user can instruct to save whether or not to save the composite image data displayed on the display unit 19 by operating the operation unit 18.
If there is such a storage instruction, it is determined as YES in step S12, and the process proceeds to step S13.
If there is no save instruction, it is determined as NO in step S12, and the process proceeds to step S14.

  In step S <b> 13, the storage control unit 44 causes the removable media 34 to record (save) the composite image data.

In step S <b> 14, the index image generation unit 56 determines whether or not there is a generation instruction. That is, the user can instruct generation of whether or not to generate index image data by operating the operation unit 18.
If there is such a generation instruction, it is determined as YES in step S14, and the process proceeds to step S15.
On the other hand, if there is no generation instruction, it is determined as NO in step S14, and the index image display process is performed without executing the process of step S15, that is, without generating the index image data. Ends.

  In step S <b> 15, the storage control unit 44 records (saves) the index image data in the storage unit 20. Thereby, the index image display process is ended.

The image processing apparatus 1 configured as described above includes an image acquisition unit 51, an index image acquisition unit 52, an index image selection unit 53, an index image conversion unit 54, an image composition unit 55, and an index image generation unit. 56, a main subject detection unit 41, an information acquisition unit 42, a display control unit 43, a storage control unit 44, and a display unit 19.
The image acquisition unit 51 acquires image data including a subject as original image data.
The main subject detection unit 41 detects a main subject image in the original image.
The information acquisition unit 42 acquires data such as the length of the main subject included in the original image as information on the main subject image based on the data of the original image.
The index image acquisition unit 52 acquires index image data of an object having information (length) within a predetermined range of main subject image information (main subject length data).
The index image selection unit 53 selects index image data desired by the user from the index image data.
The index image conversion unit 54 converts the data of the selected designated index image into data of a converted index image having the same scale information as the original image.
The image composition unit 55 generates a composite image in which the conversion index image is superimposed on the original image.
The display control unit 43 performs control to display various images on the display unit 19.
The storage control unit 44 controls the index image data to be stored in the storage unit 20 and controls the composite image data to be recorded on the removable medium 34.
The display unit 19 displays the index image together with the original image based on the length of the main subject.

In short, the image processing apparatus 1 according to the first embodiment automatically measures the information (length) of the main subject based on the original image and the imaging conditions, and the predetermined range of the measured length of the main subject. An index image of an object having a length within is selected. Then, the image processing apparatus 1 further superimposes the index image data selected by the user from the selected index image on the original image data so as to have the same scale information as the original image. Generate composite image data.
Accordingly, an index image of an object whose length can be intuitively grasped can be displayed beside the main subject image. As a result, the viewer of the composite image can intuitively grasp the length of the main subject.
Then, the image processing apparatus 1 measures the length of the main subject based on the focus lens position and the angle of view, and measures the length of the main subject within a predetermined range of the length of the main subject. It is possible to search for an index image of an object having. Therefore, since the candidate images for the designated index image can be narrowed down, the user can efficiently designate the index image.

Further, the index image generation unit 56 generates image data including the main subject image included in the original image as index image data.
As a result, the user can generate a new index image, so that the index image newly generated this time at the next and subsequent imaging can be used as an index of size.

[Second Embodiment]
In the first embodiment, the information acquisition unit 42 measures the length (actual size) of the main subject based on the information on the subject distance and the length of the main subject image calculated from the lens address of the focus lens. Was acquired as data of the main subject image. However, the main subject image information acquisition method of the information acquisition unit 42, in other words, the main subject length (actual size) measurement method is not particularly limited to the example of the first embodiment.
As described in the first embodiment, when the subject distance L takes the value of f · f / (δ · F), the denominator is 0, so that the rear depth of field Lr is infinite (formula ( 2)). Thus, generally, when the distance b between the focus lens 31F and the image sensor 32 reaches a certain value or less, the subject distance is assumed to be infinite as practically focusing on the subject up to the subject distance L being infinite. Be treated. Therefore, if the subject distance is greater than a certain distance, an accurate value cannot be measured from the lens address of the focus lens.
Therefore, in the second embodiment, a method that can calculate information (for example, height data) indicating the size of the main subject even when the subject distance of the main subject cannot be calculated will be described. To do.

In the following, differences from the hardware configuration and functional configuration in the first embodiment will be mainly described.
FIG. 7 is a block diagram illustrating a hardware configuration of the image processing apparatus according to the second embodiment. The image processing apparatus 1 can be configured by a digital camera, for example.
In FIG. 7, the image processing apparatus 1 in the present embodiment includes a GPS receiver 23 in addition to the configuration in FIG. 1 in the first embodiment.
The GPS receiver 23 can obtain position information of the image processing apparatus 1 by receiving radio waves from GPS satellites.

FIG. 8 is a functional block diagram showing a functional configuration for executing the index image display process among the functional configurations of the image processing apparatus of FIG.
A position information acquisition unit 61 realized by the GPS receiver 23 acquires position information of the image processing apparatus 1. Then, the position information acquisition unit 61 supplies position information to the information acquisition unit 42.
The information acquisition unit 42 acquires position information from the position information acquisition unit 61.
In addition, the information acquisition unit 42 acquires information (hereinafter referred to as “building information”) in which information on the image, height, and position of the building is associated with each other from the storage unit 20.

Hereinafter, the index image display process executed by the image processing apparatus 1 according to the second embodiment will be described with reference to FIG.
FIG. 9 is a schematic diagram showing each image as a specific example of the result of the index image display process according to the second embodiment.
FIG. 9A shows an original image 121 including subject images of the building 131 and the S tower 132. In FIG. 9A, a frame FW6 indicating a focus area is shown near the center of the image of the S tower 132. That is, the focus lens is focused near the center of the S tower 132.

The main subject detection unit 41 detects the main subject image, that is, the image of the S tower 132, from the contrast gradient of the original image within the frame FW6 indicating the focus area or within a predetermined range including the focus area.
In this case, the display control unit 43 causes the display unit 19 to display a symbol or the like that can identify the detected main subject superimposed on the original image 121. Specifically, in the example of FIG. 9A, for example, it is specified that the S tower 132 is the main subject by the frame CW6 of the main subject image.

  In FIG. 9A, even if the distance between the focus lens and the S tower 132 is obtained by the subject distance calculation method based on the formula (2), only information that the subject distance is infinite can be obtained, and The subject distance cannot be obtained.

In the present embodiment, it is assumed that building information is stored in the storage unit 20.
Then, the information acquisition unit 42 acquires the position information of the image processing apparatus 1 from the position information acquisition unit 61. Next, the building information acquisition unit 421 in the information acquisition unit 42 acquires building information in a predetermined range centered on the position. For example, in the present embodiment, building information within a radius of 500 m centering on the position of the image processing apparatus 1 is searched. As a result, it is assumed that building information corresponding to the building 131, the S tower 132, and the like is retrieved.

  Next, the information acquisition unit 42 calculates image similarity using an image recognition technique or the like based on the building image and the main subject image included in the searched building information. As a result, the information acquisition unit 42 determines that the image of the building that matches or is most similar to the main subject image is the image of the main subject. For example, when the information acquisition unit 42 acquires the position information of the image processing apparatus 1 from the position information acquisition unit 61, the image of the building information within the radius of 500 m around the position of the position information and the S tower that is the main subject 132 images are compared. When it is determined that the main subject image and the image of the S tower 132 included in the building information match or have the highest similarity, the building information is specified as the building information for the S tower 132. .

  Next, the information acquisition unit 42 acquires the height data of the specified building information, that is, the height data of the S tower 132 as information on the main subject image. Then, the information acquisition unit 42 supplies the acquired building height data to the index image acquisition unit 52.

The index image acquisition unit 52 acquires information on the main subject image (building data) from the information acquisition unit 42 and acquires index image data from the storage unit 20.
The index image acquisition unit 52 supplies the main subject image information (building data) and the index image data to the index image selection unit 53.

  The index image selection unit 53 selects the index image data of the building having information (height within a predetermined range of the height of the main subject) that approximates the information of the main subject image. For example, in this embodiment, the index image selection unit 53 selects index image data of a building having a height within a range of 1.2 times or less the height of the main subject. Therefore, the index image selection unit 53 selects each data of the index image C4 indicating the Great Buddha having a height of 12 m and the index image C5 indicating the T tower having a height of 333 m as data of a candidate image of the designated index image. To do.

Next, the index image selection unit 53 causes the display unit 19 to display an image in which the index image selection screen is superimposed on the original image 121 via the display control unit 43.
FIG. 9B shows that candidate images for designated index images are displayed in the index image selection screen KW2. In the index image selection screen KW2, candidate images of designated index images determined by the index image selection unit 53, that is, index images C4 and C5, indicate the height of the object indicated by the index image from the top to the bottom. Displayed in descending order.
Therefore, the user can easily grasp the magnitude relationship of the height of the building indicated by the index image in the index image selection screen KW2.

Further, a scale SL2 is displayed beside the candidate image for the designated index image. The instruction lines CL4 and CL5 indicate the scale corresponding to the height of the building indicated by the index images C4 and C5, respectively. That is, the instruction lines CL4 and CL5 indicate that the buildings indicated by the index images C4 and C5 are 12 m and 333 m, respectively. The user can grasp the height of the building indicated by the index image as a numerical value by reading the scales indicated by these instruction lines.
Further, in the index image selection screen KW2, the arrow AR2 points to the scale SL2 corresponding to the height 634m of the S tower 132.
The user designates at least one index image from among the candidate images of the designated index image in the index image selection screen KW2 by operating the operation unit 18. In response to this, the image composition unit 55 converts the data of the designated index image into the data of the conversion index image having the same scale information as the original image.

FIG. 9C shows a composite image 141 in which an image of the conversion index image D5 indicating the T tower is combined with the original image. When the user designates the image of the conversion index image D5 indicating the T tower and determines the side of the S tower 132 as the designated position, the conversion index image D5 indicating the T tower is superimposed and displayed next to the image of the S tower 132. The
In the example shown in FIG. 9C, the viewer of the composite image compares the image of the S tower 132 of the composite image 141 with the conversion index image D5 of the T tower, and the S tower 132 is nearly twice the T tower. Intuitively grasp what is expensive.

In short, the image processing apparatus 1 according to the second embodiment is based on the position information received by the GPS receiver 23 on the main subject image information (main subject height data). And the position information (building information). Then, the image processing apparatus 1 selects index image data based on data of information (height of the main subject) that approximates the acquired information of the main subject image. The selected index image is displayed on the selection display screen as a candidate for the specified index image. The user designates an index image by operating the operation unit 18. The designated index image data is converted into converted index image data having the same scale information as the original image, and as a result, the display unit 19 displays the converted index image superimposed on the original image.
Therefore, even if the main subject is located at a distance where the focal length is considered to be infinite and an accurate subject distance cannot be obtained, the image processing apparatus 1 according to the second embodiment can measure the size. It is possible to generate a composite image in which the index image to be displayed is superimposed on the original image.
Therefore, it is possible to display an index image of a building whose length can be intuitively grasped beside the main subject image. As a result, the viewer of the composite image can intuitively grasp the height of the main subject.
The image processing apparatus 1 can search for an index image of a building having a height within a predetermined range with respect to the height of the main subject, and thus can narrow down candidate images for designated index images. . For this reason, the user can designate an index image efficiently.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

That is, the image processing apparatus to which the present invention is applied is not particularly limited to the above-described embodiment, and various various embodiments can be adopted so as to have the following configuration.
That is, the image processing apparatus
An acquisition means for acquiring an image;
Information acquisition means for acquiring information of a subject image included in the image acquired by the acquisition means;
Selection means for selecting an index image corresponding to the subject image based on the information acquired by the information acquisition means;
Conversion means for converting the index image based on the information acquired by the information acquisition means and the information of the index image selected by the selection means;
Combining means for combining the image with the index image converted by the converting means;
It can comprise.
In the image processing apparatus having such a configuration, it is possible to display a composite image in which an index image converted based on information on a subject is combined with an image including the subject.
The user can intuitively grasp subject information by viewing the composite image.

Furthermore, the image processing apparatus
Further comprising display control means for displaying a plurality of candidate images corresponding to the subject image based on the information acquired by the information acquisition means;
The selection unit selects the index image by a user operation from a plurality of candidate images displayed by the display control unit.
It can also be configured as follows.
Thereby, the user can easily perform an operation of freely selecting a desired index image. That is, since the user can narrow down the index image, the user can efficiently specify a desired index image.

Furthermore, in the image processing apparatus,
The information acquisition means acquires large information of the subject image based on the image,
The display control means displays a plurality of candidate images based on the information on the size of the subject image,
The conversion means converts the size of the index image according to the ratio of the size information of the subject image and the size of the index image,
The synthesizing unit synthesizes the image and the index image whose size is converted by the converting unit.
It can also be configured as follows.
Thereby, the user can intuitively grasp the size of the subject image information in particular.

Furthermore, the image processing apparatus
A focusing unit for focusing on the subject image using a focus lens;
The information acquisition means acquires information on the size of the subject image based on information on the position and angle of view of the focus lens when the subject image is focused by the focusing means.
It can be constituted as follows.
As a result, the information on the size of the subject image can be acquired by utilizing the existing AF processing, and thus the image processing apparatus can be easily realized with a simple configuration.

Furthermore, in the image processing apparatus,
The display control means displays the candidate images in an order based on the magnitude relationship of the plurality of candidate images,
The selection means selects an index image by a user operation from a plurality of candidate images displayed in an order based on a magnitude relationship by the display control means.
It can be constituted as follows.
Thereby, the user can easily grasp the magnitude relationship between the heights of the objects (buildings, etc.) indicated by the plurality of index images, and as a result, the desired index image can be easily and efficiently selected. it can.

Furthermore, the image processing apparatus
Index image generation means for generating data of the index image based on the data of the image including the subject image;
It can comprise.
Thereby, index image data appropriate for the subject image included in the image is generated. The user can grasp the subject information more intuitively by looking at the composite image including such an index image.
Further, the index image data generated based on the image data including the subject image can be used as index image data at the next imaging.

Furthermore, the image processing apparatus
Position information acquisition means for acquiring position information;
Building information acquisition means for acquiring building information in which the image, height and position data of the building are associated with each other;
Further comprising
The information acquisition means relates to the subject image acquired by the acquisition means based on the position information acquired by the position information acquisition means and the building information acquired by the building information acquisition means. Obtaining information representing the size of the subject;
It can be constituted as follows.
As a result, in particular, when a subject image that can be compared with a building is included, the user can intuitively grasp the size especially as the subject image information.

  In the above embodiment, the height of the main subject information represented by the index image is used, but the present invention is not limited to this. For example, weight may be used as information on the main subject image.

  Moreover, in the said embodiment, the data of the height of the building in the predetermined range centering on the positional information obtained from the GPS receiver 23 were searched. At this time, for example, it is also possible to obtain information on the imaging direction from a gyro sensor or a geomagnetic sensor, and to retrieve data on the height of the main subject within a range limited to the direction.

  Furthermore, in the said embodiment, although building information is memorize | stored in the memory | storage part 20, you may acquire building information from the network containing the internet via the communication part 21. FIG. In this case, when imaging a building having building information that is not stored in the storage unit 20, it is possible to acquire building information from a website or the like via the Internet.

  In the first embodiment and the second embodiment described above, the electronic apparatus to which the present invention is applied has been described by taking the image processing apparatus 1 such as a digital camera as an example, but is not particularly limited thereto. The present invention can be applied to general electronic devices capable of executing the index image display process described above. Specifically, for example, the present invention is applicable to a video camera, a mobile phone device with an imaging device, and the like.

  The series of processes described above can be executed by hardware or can be executed by software. For example, the series of processes described above can be executed by hardware or can be executed by software. In other words, the functional configurations of FIGS. 2 and 8 are merely examples, and are not particularly limited. That is, it is sufficient that the image processing apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is particularly the example shown in FIGS. It is not limited to. In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

  When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  A recording medium including such a program is provided not only to an external memory distributed separately from the apparatus main body in order to provide the program to the user, but also provided to the user in a state of being preinstalled in the apparatus main body. It consists of a recording medium. The external memory is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. The recording medium provided to the user in a state of being pre-installed in the apparatus main body is, for example, the hard disk included in the ROM 12 in FIGS. 1 and 7 in which the program is recorded or the storage unit 20 in FIGS. 1 and 7. Etc.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
An acquisition means for acquiring an image;
Information acquisition means for acquiring information of a subject image included in the image acquired by the acquisition means;
Selection means for selecting an index image corresponding to the subject image based on the information acquired by the information acquisition means;
Conversion means for converting the index image based on the information acquired by the information acquisition means and the information of the index image selected by the selection means;
Combining means for combining the image with the index image converted by the converting means;
An image processing apparatus comprising:
[Appendix 2]
Further comprising display control means for displaying a plurality of candidate images corresponding to the subject image based on the information acquired by the information acquisition means;
The selection unit selects the index image by a user operation from a plurality of candidate images displayed by the display control unit.
The image processing apparatus according to appendix 1, wherein:
[Appendix 3]
The information acquisition means acquires large information of the subject image based on the image,
The display control means displays a plurality of candidate images based on the information on the size of the subject image,
The conversion means converts the size of the index image according to the ratio of the size information of the subject image and the size of the index image,
The image processing apparatus according to appendix 2, wherein the synthesizing unit synthesizes the image and an index image whose size has been converted by the converting unit.
[Appendix 4]
A focusing unit for focusing on the subject image using a focus lens;
The information acquisition means acquires information on the size of the subject image based on information on the position and angle of view of the focus lens when the subject image is focused by the focusing means.
The image processing apparatus according to Supplementary Note 3, wherein
[Appendix 5]
The display control means displays the candidate images in an order based on the magnitude relationship of the plurality of candidate images,
The selection means selects an index image by a user operation from a plurality of candidate images displayed in an order based on a magnitude relationship by the display control means.
The image processing apparatus according to appendix 3 or 4, characterized by the above.
[Appendix 6]
Index image generation means for generating data of the index image based on the data of the image including the subject image;
The image processing apparatus according to any one of appendices 1 to 4, further comprising:
[Appendix 7]
Position information acquisition means for acquiring position information;
Building information acquisition means for acquiring building information in which the image, height and position data of the building are associated with each other;
Further comprising
The information acquisition means relates to the subject image acquired by the acquisition means based on the position information acquired by the position information acquisition means and the building information acquired by the building information acquisition means. Obtain information about the size of the subject,
The image processing apparatus according to appendix 1, wherein
[Appendix 8]
In the image processing method executed by the image processing apparatus,
An acquisition step of acquiring an image;
An information acquisition step of acquiring information on a subject image included in the image acquired by the processing of the acquisition step;
A selection step of selecting an index image corresponding to the subject image based on the information acquired by the processing of the information acquisition step;
A conversion step of converting the index image based on the information of the subject image acquired by the process of the information acquisition step and the information of the index image selected by the process of the selection step;
A synthesis step of synthesizing the image and the index image converted by the process of the conversion step;
An image processing method including:
[Appendix 9]
Computer
Acquisition means for acquiring images,
Information acquisition means for acquiring information on a subject image included in the image acquired by the acquisition means;
Selection means for selecting an index image corresponding to the subject image based on the information acquired by the information acquisition means;
Conversion means for converting the index image based on the information acquired by the information acquisition means and the information of the index image selected by the selection means;
Synthesizing means for synthesizing the image and the index image converted by the converting means;
Program to function as.

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... DSP, storage control part, 15 ... Bus, 16 ... Input / output interface, DESCRIPTION OF SYMBOLS 17 ... Imaging part, 18 ... Operation part, 19 ... Display part, 20 ... Memory | storage part, 21 ... Communication part, 22 ... Drive, 23 ... GPS receiver, 31 ... Lens optical part, 32 ... Image sensor, 33 ... Lens drive part, 34 ... Removable media, 41 ... Main subject detection part, 42 ... Information acquisition part, 43 ... Display control unit 44 ... Storage control unit 51 ... Image acquisition unit 52 ... Index image acquisition unit 53 ... Index image selection unit 54 ... Index image conversion unit 55 ... Image compositing unit, 56... Index image generating unit, 61. 1 ... building information acquisition unit

Claims (9)

An acquisition means for acquiring an image;
Information acquisition means for acquiring information of a subject image included in the image acquired by the acquisition means;
Selection means for selecting an index image corresponding to the subject image based on the information acquired by the information acquisition means;
Conversion means for converting the index image based on the information acquired by the information acquisition means and the information of the index image selected by the selection means;
Combining means for combining the image with the index image converted by the converting means;
An image processing apparatus comprising: Further comprising display control means for displaying a plurality of candidate images corresponding to the subject image based on the information acquired by the information acquisition means;
The selection unit selects the index image by a user operation from a plurality of candidate images displayed by the display control unit.
The image processing apparatus according to claim 1. The information acquisition means acquires large information of the subject image based on the image,
The display control means displays a plurality of candidate images based on the information on the size of the subject image,
The conversion means converts the size of the index image according to the ratio of the size information of the subject image and the size of the index image,
The image processing apparatus according to claim 2, wherein the synthesizing unit synthesizes the image and an index image whose size is converted by the converting unit. A focusing unit for focusing on the subject image using a focus lens;
The information acquisition means acquires information on the size of the subject image based on information on the position and angle of view of the focus lens when the subject image is focused by the focusing means.
The image processing apparatus according to claim 3. The display control means displays the candidate images in an order based on the magnitude relationship of the plurality of candidate images,
The selection means selects an index image by a user operation from a plurality of candidate images displayed in an order based on a magnitude relationship by the display control means.
The image processing apparatus according to claim 3 or 4, Index image generation means for generating data of the index image based on the data of the image including the subject image;
The image processing apparatus according to claim 1, further comprising: Position information acquisition means for acquiring position information;
Building information acquisition means for acquiring building information in which the image, height and position data of the building are associated with each other;
Further comprising
The information acquisition means relates to the subject image acquired by the acquisition means based on the position information acquired by the position information acquisition means and the building information acquired by the building information acquisition means. Obtain information about the size of the subject,
The image processing apparatus according to claim 1. In the image processing method executed by the image processing apparatus,
An acquisition step of acquiring an image;
An information acquisition step of acquiring information on a subject image included in the image acquired by the processing of the acquisition step;
A selection step of selecting an index image corresponding to the subject image based on the information acquired by the processing of the information acquisition step;
A conversion step of converting the index image based on the information of the subject image acquired by the process of the information acquisition step and the information of the index image selected by the process of the selection step;
A synthesis step of synthesizing the image and the index image converted by the process of the conversion step;
An image processing method including: Computer
Acquisition means for acquiring images,
Information acquisition means for acquiring information on a subject image included in the image acquired by the acquisition means;
Selection means for selecting an index image corresponding to the subject image based on the information acquired by the information acquisition means;
Conversion means for converting the index image based on the information acquired by the information acquisition means and the information of the index image selected by the selection means;
Synthesizing means for synthesizing the image and the index image converted by the converting means;
Program to function as.

Images