JP2009159358A - Imaging apparatus and method for generating background image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of displaying a photographed image as the background image of a display device installed in an external apparatus such as a portable telephone by a simple procedure. <P>SOLUTION: The imaging apparatus 100 for generating a background image for performing background-display on the display device provided to the other external apparatus from a photographed image includes: a display device information acquisition part 101 for acquiring display device information about the display capacity of the display device and added information display information about added information including characters or images to be displayed on the background image of the display device; an object recognition part 102 for recognizing an object included in the photographed image; an object size acquisition part 103 for acquiring information indicating the size of the object for the background image; and a background image generation part 104 for extracting all or a part of the photographed image and generating a background image from the extracted image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a background image generation method.

  In general, an image pickup apparatus such as a digital camera can take a higher quality image than an image pickup function provided in a mobile phone. Therefore, there is a need to set an image taken with a digital camera as a wallpaper (standby screen) of a display such as a mobile phone.

  However, transferring images taken with a digital camera to a mobile phone is time consuming, and there are cases where the format and size of image data that can be handled by a mobile phone are limited. It cannot always be displayed as it is. In such a case, a technique has been devised for appropriately processing an image taken by an imaging device and inputting the image to a mobile phone or the like.

  For example, Patent Document 1 discloses an image based on image conditions (file format, image size, etc.) requested by a destination service (print service, wallpaper conversion service, etc.) for image data captured by a digital camera or the like. A technique for performing processing and transmitting to a transmission destination is disclosed.

  Patent Document 2 discloses a digital camera that receives information related to the image display capability of an image display device, converts the image data into image data corresponding to the received display capability information, and outputs the image data to the image display device. Has been.

JP 2004-40601 A Japanese Patent No. 3925051

  As described above, a technique has been proposed in which an image captured by an imaging device such as a digital camera is transmitted after being edited according to the image display capability of a transmission destination device. However, when assuming an application in which an image taken with a digital camera or the like is set as wallpaper for a mobile phone or the like, various problems that cannot be solved by the above-described prior art occur.

  For example, in general, a display provided in a digital camera is a horizontally long screen, whereas many displays of a mobile phone are vertically long screens. Therefore, if an image captured in landscape orientation is reduced in size in accordance with the lateral length of the display of the mobile phone and displayed on the portrait orientation screen, a margin will be generated in the portrait direction. Conversely, when the image is reduced in accordance with the length in the vertical direction, an area that is not displayed in the horizontal direction of the image is generated. It takes a lot of trouble to adjust this so that the main subject included in the image is displayed in the center of the display.

  Furthermore, even if an image suitable for displaying on a mobile phone display is created by performing these complicated operations, when it is actually displayed, it is not possible to use a clock or calendar that is already displayed on the display. The subject may be hidden. In addition, when the display color of the clock or calendar and the wallpaper image of the portion displayed on the background are similar, there is a problem that the display of the clock or calendar becomes difficult to identify.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to transfer an image taken by an imaging device such as a digital camera to an external device such as a mobile phone by a simple procedure. It is an object of the present invention to provide a new and improved imaging device and background image generation method capable of being displayed as a background image (wallpaper, standby image) of a display device provided.

  In order to solve the above-described problem, according to an aspect of the present invention, there is provided an imaging device that generates a background image for displaying a background on a display device provided in another external device from a captured image. A display device information acquisition unit for acquiring display device information relating to display capability of the display device and additional information display information relating to additional information including characters or images displayed on a background image of the display device; and a captured image. A subject recognition unit that recognizes a subject, a subject size acquisition unit that acquires information indicating the size of the subject with respect to the background image, and all or part of the captured image are extracted, and a background image is generated from the extracted image An image pickup apparatus including a background image generation unit is provided. The background image generation unit includes an extraction position calculation unit that calculates the size and position of the extracted image based on information output from the display device information acquisition unit, the subject recognition unit, and the subject size acquisition unit. Features.

  With this configuration, it is possible to easily use an image photographed by the imaging device as a background image on the display of the external device. At this time, the main subject included in the photographed image is recognized, and the background image is generated in consideration of the position, size, etc., so it is possible to easily obtain the background image that meets the user's intention. Can do.

  Further, the position of the extracted image is changed while the position of the extracted image calculated by the extracted position calculating unit is displayed on the captured image and the position of the extracted image is displayed on the display unit. The background image generation unit extracts the extracted image from the photographed image based on the position of the extracted image changed by the input from the input unit, and uses the extracted image. A background image may be generated. Thereby, the user can confirm which part of the captured image is extracted as the background image before the background image is generated. If the extracted position is different from the user's intention, the desired background image can be generated by changing the position.

  The additional information display information includes information about the display position and display color of the additional information on the display device, and the background image generation unit converts the extracted image into the background image based on the display position of the additional information on the display device. When the display position of the additional information is calculated, and the color of the extracted image at the display position is similar to the display color of the additional information, the image color change for changing the image color of the extracted image at the display position A part may be further included. At this time, the image color changing unit may change the luminance value of the extracted image at the display position.

  Thus, when the generated background image is displayed on the display device, it is possible to prevent the clock or calendar displayed on the display device from being difficult to see. It is possible to save the trouble of changing the setting of the display color and the like.

  Further, when the position of the extracted image is displayed on the display unit, the color of the extracted image changed by the image color changing unit may be changed and displayed. Further, a character or an image representing the additional information may be superimposed and displayed at a position where the additional information is displayed. Thereby, the user can confirm in advance the image when displayed on the display device as the background image, and can thereby determine whether or not to change the position of the image.

  The background image generation unit extracts a portion that is outside the region of the photographed image when there is a portion that is outside the region of the photographed image in the region of the extracted image determined by the position of the extraction image calculated by the extraction position calculation unit. A supplementary image generation unit that generates a supplementary image for supplementing the image may be further included. At this time, the supplemental image generation unit may generate the supplemental image based on the color of the pixel of the extracted image located at the boundary with the portion outside the region of the background image. Thereby, for example, even when the main subject is reflected at the end of the captured image, it is possible to generate an image by supplementing the surrounding area.

  In addition, when the position of the extracted image is displayed on the display unit, the supplementary image generated by the supplemental image generation unit may be displayed at the same time. Thereby, it becomes possible to confirm in advance the image of the final background image including the supplementary image.

  In order to solve the above-described problem, according to another aspect of the present invention, a background image for displaying a background on a display device provided in another external device is generated from a photographed image photographed by an imaging device. A display device information acquisition method for acquiring display device information relating to display capability of a display device and additional information display information relating to additional information including characters or images displayed on the background image of the display device A step, a subject recognition step for recognizing a subject included in the photographed image, a subject size acquisition step for obtaining information indicating the size of the subject relative to the background image, and all or a part of the photographed image is extracted and extracted And a background image generation step of generating a background image from the extracted image. A background image generation method is provided. The background image generation step includes an extraction position calculation step for calculating the size and position of the extracted image based on information output from the display device information acquisition unit, the subject recognition unit, and the subject size acquisition unit. Features.

  As described above, according to the present invention, it is possible to display an image taken by an imaging device such as a digital camera as a background image of a display device provided in an external device such as a mobile phone by a simple procedure. Become.

  Exemplary embodiments of an imaging apparatus and a background image generation method according to the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  First, an imaging apparatus according to an embodiment of the present invention will be described. The imaging apparatus according to the present embodiment has a background image generation function for easily displaying a captured image as a background image (wallpaper, standby image) of a display device provided in an external device such as a mobile phone. It is characterized by having. Hereinafter, a digital still camera will be described as an example of an imaging apparatus, but the present invention is not limited to such an example. The imaging apparatus according to the present invention may have a moving image shooting function such as a digital video camera.

(Hardware configuration of digital still camera 100)
FIG. 1 is a diagram showing an outline of a hardware configuration of a digital still camera 100 as an imaging apparatus according to an embodiment of the present invention. As illustrated in FIG. 1, the digital still camera 100 according to the present embodiment includes an imaging processing unit 110, an image processing unit 120, and a control unit 130. Details of each part will be described below.

(Imaging processing unit 110)
The imaging processing unit 110 includes an optical block 111, a driving unit 112, an imaging element 113, and an analog processing circuit 114. The optical block 111 includes a lens group for imaging a subject, an aperture adjustment mechanism, a focus adjustment mechanism, a zoom mechanism, a shutter mechanism, a flash mechanism, and the like. The drive unit 112 is controlled in response to a control signal from the control unit 130, and electrically drives the aperture adjustment mechanism, the zoom mechanism, the shutter mechanism, and the like of the optical block 111.

  The image sensor 113 is configured by, for example, a solid-state image sensor such as a charge coupled device (CCD), and an image through the optical block 111 is formed on an image formation surface thereof. Further, the image sensor 113 receives the timing signal generated by the analog processing circuit 114, converts the subject image formed on the imaging surface into an imaging signal, and supplies the image signal to the analog processing circuit 114.

  The analog processing circuit 114 performs processing for converting an analog imaging signal from the imaging element 113 into a digital signal. The analog processing circuit 114 is an AGC (Auto Gain Control) that automatically adjusts the gain of an image pickup signal from the image sensor 113, an AD (Analog Digital) converter that converts an analog signal into a digital signal, and a control unit according to a shutter operation. A timing generator or the like that generates an image capture timing signal to the image sensor 113 based on a control signal supplied from 130 is provided internally. The digital imaging signal output by the analog processing circuit 114 is stored in a storage medium such as the RAM 134 as RAW image data.

(Image processing unit 120)
The image processing unit 120 reads out the RAW image data output from the analog processing circuit 114 and stored in the RAM 134 and the like, performs digital image processing, and image formats such as JPEG (Joint Photographic Experts Group) and TIFF (Tagged Image File Format). And recorded in a storage medium such as the memory card 143. The image processing unit 120 performs various types of image processing such as pixel interpolation processing, gamma correction processing, and white balance processing on the RAW image data. Further, the image processing unit 120 changes the size (resizing, changing the number of pixels) or trimming (Trimming, Crop: Crop) on the captured image stored in the memory card 143 or the like based on the control signal from the control unit 130. ) Etc. are performed.

(Control unit 130)
As shown in FIG. 1, the control unit 130 is an operation input for connecting an MPU (Micro Processing Unit) 131, a ROM (Read Only Memory) 133, a RAM (Random Access Memory) 134, and an operation input unit 141. An interface 135, a display control unit 136 for connecting the display unit 142, a memory card interface 137 for loading the memory card 143, a communication interface 138, and an external input / output interface 139 are connected via the system bus 132. It is configured by being connected.

  The control unit 130 controls the imaging processing unit 110 and the image processing unit 120 to control the entire digital still camera 100 so that the imaging process is executed. In particular, the control unit 130 has a background image generation function for generating a background image to be displayed as a background image on the display device of the external device from the image captured by the imaging processing unit 110. The control unit 130 causes the MPU 131 to execute the program stored in the ROM 133 or the like on the image data read from the memory card 143 or the like or the JPEG image data output from the imaging processing unit 110 and stored in the RAM 134 or the like. Executes the background image generation function. Hereinafter, the configuration of the control unit 130 will be described.

  The MPU 131 supplies various control signals generated by executing the control program recorded in the ROM 133 to each unit, and controls the entire digital still camera 100. The ROM 133 stores programs for controlling the imaging processing unit 110, the image processing unit 120, the display control unit 136, and the like, and the memory card interface 137. The RAM 134 is a storage medium that stores RAW image data output from the analog processing circuit 114 of the imaging processing unit 110 described above and JPEG data read from the memory card 143 and decoded by the image processing unit 120. . The RAM 134 is also used as a work area for the MPU 131.

  The operation input unit 141 connected to the operation input interface 135 includes a plurality of modes such as a mode switching key for switching a shooting mode, a zoom adjustment key, a key for exposure adjustment, a shutter key, and a display adjustment key for the display unit 142. A key is provided. The operation input interface 135 transmits an operation signal from the operation input unit 141 to the MPU 131. The MPU 131 determines the operation content based on the operation signal input from the operation input interface 135, and performs control processing according to the determination result.

  The display unit 142 connected to the display control unit 136 is configured by a liquid crystal display including a TFT (Thin Film Transistor) and the like. The display control unit 136 is controlled by the MPU 131 and causes the display unit 142 to display an image processed by the image processing unit 120 and an image read from the memory card 143.

  The memory card interface 137 writes the image data output from the image processing unit 120 to the memory card 143. The memory card interface 137 reads image data from the memory card 143 and supplies it to the image processing unit 120 and the like. The memory card 143 is a storage medium that is read and written by the memory card interface 137, and mainly records captured image data, attribute information associated with the image data, and the like.

The communication interface 138 is an interface for performing communication with an external device, and receives data transmitted from the external device via the network or transmits data via the network. The external input / output interface 139 is an interface for connecting an external device and the digital still camera 100 to input / output data, and outputs the data stored in the memory card 143 of the digital still camera 100 to the external device. Or an interface for inputting data input from an external device to the digital still camera 100.
Heretofore, the hardware configuration of the digital still camera 100 according to the present embodiment has been described.

(Configuration of background image generation function)
Next, a background image generation function executed by the control unit 130 of the digital still camera 100 will be described with reference to FIG. FIG. 2 is a functional block diagram showing a schematic configuration for realizing the background image generation function of the digital still camera 100.

  As illustrated in FIG. 2, the background image generation function of the present embodiment includes a display device information acquisition unit 101, a subject recognition unit 102, a subject size acquisition unit 103, and a background image generation unit 104.

  2 may be realized by a software program that executes the function of each unit, which is stored in the ROM 133 or the like and can be executed by the MPU 131, or a dedicated image generation function shown in FIG. It may be realized by hardware. The software program may be executed by reading out a program stored in a computer-readable storage medium such as the ROM 133 as described above, or may be provided to the digital still camera 100 via a network or the like. It may be a thing.

(Display device information acquisition unit 101)
The display device information acquisition unit 101 acquires information about a display device of an external device that displays a background image generated by the image editing function and inputs the information to the background image generation unit 104. The display device information includes, for example, information (displayable image size, image format, etc.) regarding the display capability of the display device provided in the target external device, and a clock displayed on the background image of the display device. And setting information (additional information display information) related to display of icons, calendars, etc. (hereinafter referred to as additional information). The additional information display information includes, for example, information on the display position of the additional information, the size of the displayed area, and the display color.

  The display device information acquisition unit 101 acquires the display device information from the operation input unit 141, the communication interface 138, the external input / output interface 139, and the like. For example, the display device information acquisition unit 101 acquires a value of display device information input from the operation input unit 141 (such as a numerical value of an image size that can be displayed on the display device) via the operation input interface 135. Also good. Alternatively, a value selected from options stored in a storage medium such as the ROM 133 may be input from the operation input unit 141.

  The display device information acquisition unit 101 acquires display device information transmitted from an external device via a network from the communication interface 138 or information input from an external device connected via the external input / output interface 139. May be obtained.

  In addition, the display device information acquisition unit 101 may acquire display device information from an image captured by the imaging processing unit 110. For example, the display device of the target external device is photographed by the digital still camera 100, and information such as the display position of the additional information, the size of the display area, and the display color is acquired from the image of the photographed display device. Also good. In this case, the information regarding the display capability of the display device is acquired by another method, for example, input from the operation input unit 141 or the like. Note that the display device information acquisition method described above may be used in combination of two or more methods.

(Subject recognition unit 102)
The subject recognition unit 102 automatically recognizes main subjects included in a captured image (hereinafter referred to as a target image) that is a target for generating a background image. The number of recognized subjects may be one or plural. In the following description, it is assumed that one of the subjects included in the target image is recognized as the main subject. When the main subject is recognized from the target image, the image size of the area including the main subject and the position in the target image (for example, the upper left pixel coordinates of the area including the main subject) are obtained and input to the background image generation unit 104 To do.

  The main subject is recognized by detecting a face image or detecting an object by edge extraction or the like. For example, the subject recognition unit 102 may detect a face image by a method such as pattern matching and recognize the detected face image region as the main subject region. When a plurality of face images are detected, the face image closest to the center of the image may be determined as the main subject, or the face image having the largest area may be determined as the main subject.

  When no face image is detected, the subject recognition unit 102 performs object detection by performing edge extraction or the like on the target image. Again, when a plurality of objects are detected, the main subject may be determined based on the position and area of the object. In the face image detection process and the object detection process described above, conventionally known image processing techniques such as pattern matching and edge extraction can be used.

(Subject size acquisition unit 103)
The subject size acquisition unit 103 acquires information related to the size of the main subject (hereinafter referred to as subject size) in the generated background image and inputs the information to the background image generation unit 104. The subject size can be specified by, for example, the ratio of the main subject to the background image (for example, 50% of the background image). The subject size information may be a value recorded in advance in a storage medium such as the ROM 133, or may be a value input from the operation input unit 141. Alternatively, it may be a value selected from a plurality of values recorded in advance in the ROM 133 or the like.

(Background image generation unit 104)
The background image generation unit 104 edits the target image based on information input from the display device information acquisition unit 101, the subject recognition unit 102, and the subject size acquisition unit 103, and displays the target image as a background image on the display device of the external device. Generate an image of As shown in FIG. 2, the background image generation unit 104 includes an extraction position calculation unit 104a, a compensation image generation unit 104b, an image color change unit 104c, a trimming processing unit 104d, and a size change unit 104e. The

(Extraction position calculation unit 104a)
The extraction position calculation unit 104 a is acquired by the display device information acquisition unit 101, the displayable image size of the display device, the size of the main subject area recognized by the subject recognition unit 102, and the subject size acquisition unit 103. Based on the subject size, the size and position of the image (extracted image) of the portion extracted as the background image from the captured image is calculated. Specifically, the size of the extracted image is calculated so that the size of the area of the main subject recognized by the subject recognition unit 102 becomes the size of the subject in the background image determined by the subject size.

  For example, the displayable image size of the display device given by the display device information acquisition unit 101 is 460 × 800 pixels, the main subject area recognized by the subject recognition unit 102 is 600 × 800 pixels, and subject size acquisition is performed. Assume that the subject size acquired by the unit 103 is 50% of the background image. In this case, the total number of pixels of the extracted image is 600 × 800 × 1 / 0.5 = 960000 pixels. When this is allocated to the aspect ratio of the displayable image size of the display device, the size of the extracted image (x is the size in the vertical direction and y is the size in the horizontal direction) is obtained by the following formula 1.

従って、抽出画像サイズは、７４２×１２９２ピクセルとなる。

Therefore, the extracted image size is 742 × 1292 pixels.

If the original target image is smaller than the extracted image size obtained by the above formula, the extracted image size is further corrected according to the size of the target image. For example, in the above example, when the vertical size (y) of the target image is 1000 pixels, the size in the y direction is 1000 <1292, and the vertical length of the target image is less than the extracted image size. Correction is necessary. In this case, for example, the vertical length y of the extracted image is set to 1000 pixels, and the horizontal length x is corrected as follows in accordance with it.
x = 1000 × 742 / 1292≈574
Therefore, the extracted image size is 574 × 1000 pixels.

  The extraction position calculation unit 104a determines the position of the extracted image after calculating the extracted image size. At this time, the extraction position calculation unit 104a determines the position of the extracted image based on the additional information display information input from the display device information acquisition unit 101 so that the main subject does not overlap the display position of the additional information.

  For example, as shown in FIG. 3, when the clock display as additional information is displayed at the bottom of the background image, the center of the face of the person who is the main subject is an area excluding the clock display area of the background image. The position of the extracted image is determined so as to overlap with the center of.

  On the other hand, as shown in FIG. 4, when the additional information is in the center of the screen and the main subject is placed on either the upper side or the lower side (or the right side or the left side) of the additional information, it always overlaps with the additional information. Alternatively, either the upper side or the lower side (or the right side or the left side) may be selected and arranged. For example, the position of the extracted image may be determined so that the main subject is always above the additional information, or the position of the extracted image is determined according to the degree of overlap between the main subject and the additional information. Also good.

  The criterion for determining that the main subject overlaps with the additional information is defined, for example, when the additional information display area overlaps more than half of the area of the main subject. You may make it judge that it is not.

  Further, after calculating the size and position of the extracted image by the extraction position calculation unit 104a, a frame indicating the region of the extraction image in the target image may be superimposed on the target image and displayed on the display unit 142. Further, the user may arbitrarily change the position of the frame by operating a key provided in the operation input unit 141 while the display unit 142 displays the above.

(Supplementary image generation unit 104b)
As shown in FIG. 5, the supplemental image generation unit 104 b displays the deficient area when the extracted image area is less than the necessary background image area at the extracted image position calculated by the extraction position calculating section 104 a. A supplementary image for supplementing is generated. The supplementary image may be, for example, an image configured by pixels of colors obtained by averaging the color information of the target image at the boundary portion with the insufficient portion area. Further, a predetermined amount of region located at the boundary with the target image may be generated by gradation between the color of the boundary portion and the above color.

（ΔＹ、ΔＣｂ及びΔＣｒは、それぞれＹ、Ｃｂ及びＣｒの値の差） (Image color changing unit 104c)
The image color changing unit 104c compares the display color (character color or image color) of the additional information such as a clock or calendar with the color of the extracted image that overlaps the additional information. Correct the color and brightness of the image. For example, based on Equation 2 below, the difference between the values of the YCbCr color system of each color to be compared is obtained, and if the value is smaller than a predetermined threshold, it is determined that the two colors are close, and the image color is corrected. May be performed. The following example shows a case where the threshold value is set to 15.

(ΔY, ΔCb and ΔCr are the differences in the values of Y, Cb and Cr, respectively)

  As described above, the display color of the additional information is compared with the color of the extracted image, and when it is determined that the two colors are close as a result of the comparison, the image color changing unit 104c changes the image color of the extracted image. For example, the image color changing unit 104c may change the image color by changing the luminance value (lightness) Y of the YCbCr value of the image color of the extracted image. For example, if the value of the luminance value Y of the display color of the additional information is Y ≧ 128 (when the YCbCr value is 8 bits), the luminance value Y of the image color of the extracted image is set to Y = Y−64. If the value of the luminance value Y of the display color of information is Y <128, the luminance value Y of the image color of the extracted image may be set to Y = Y + 64.

  Further, as shown in FIG. 6, the image color changing unit 104c changes the color of the boundary portion between the portion where the image color of the extracted image is changed and the portion where the image color is not changed so as to obtain a gradation of both colors. Also good.

  In addition, when the frame indicating the region of the extracted image calculated by the extraction position calculation unit 104a is superimposed on the target image and displayed on the display unit 142, the image of the target image of the portion that is changed by the image color changing unit 104c. The color may be changed and displayed. Furthermore, clocks and calendar characters and images displayed as additional information may be simultaneously superimposed on the target image and displayed so that the user can confirm the image when displayed as the background image.

(Trimming processing unit 104d)
The trimming processing unit 104d extracts a portion of an image used as a background image by performing trimming processing on the target captured image according to the size and position of the extracted image calculated by the extraction position calculating unit 104a. The extracted image is combined with the supplement image generated by the supplement image generation unit 104b and input to the size changing unit 104e.

(Size changer 104e)
The size changing unit 104e generates a final background image by performing size changing processing on the image that is trimmed by the trimming processing unit 104d and combined with the supplementary image. The size changing unit 104e reduces or enlarges the image based on the ratio between the displayable image size of the display device given by the display device information acquiring unit 101 and the size of the image output by the trimming processing unit 104d. Output.

  The configuration for executing the background image generation function in the digital still camera 100 has been described above.

(Example of background image generation processing)
Next, an example of background image generation processing executed by the digital still camera 100 according to the present embodiment will be described with reference to FIGS. 7 and 8 are flowcharts showing the flow of background image generation processing according to this embodiment.

  First, referring to FIG. 7, in step S201, generation of a background image is started in response to an input from the user. The generation of the background image may be immediately after the target image is taken, or even when the user instructs the processing by calling an image stored in a storage medium such as the memory card 143. Good.

  Next, in step S203, the display device information acquisition unit 101 receives information related to the display capability of the display device of the external device that displays the background image (such as the size of the displayable image and the image format) in response to an input from the operation input unit 141. And the display position and display color of the clock, icon, calendar, etc. displayed on the background image of the display device. The acquired information is input to the background image generation unit 104. In step S <b> 205, the subject size acquisition unit 103 acquires the subject size in the same manner and inputs the subject size to the background image generation unit 104.

  Next, in step S207, the subject recognition unit 102 recognizes a main subject included in the target image. The main subject may be recognized by detecting a face image or detecting an object by edge extraction or the like. The subject recognizing unit 102 obtains the size of the image area including the recognized main subject and the position in the target image, and inputs them to the background image generating unit 104.

  Next, in step S209, the extraction position calculation unit 104a of the background image generation unit 104 displays the displayable image size of the display device acquired by the display device information acquisition unit 101 and the main subject area recognized by the subject recognition unit 102. And the size of the extracted image extracted as the background image from the photographed image based on the size of the image and the subject size acquired by the subject size acquisition unit 103. The specific calculation method is as described above.

  Further, in step S211, the extraction position calculation unit 104a compares the size of the extracted image calculated in step S209 with the size of the target image, and determines whether the extracted image having the calculated size can be extracted. . If the extracted image size is larger than the target image size, the process proceeds to step S213, and the extracted image size is corrected according to the size of the target image.

  After step S211 or S213, the extraction position calculation unit 104a determines the position of the extraction image in the target image in step S215. At this time, the extraction position calculation unit 104a determines the position of the extracted image based on the additional information display information input from the display device information acquisition unit 101 so that the center of the main subject does not overlap the display position of the additional information. . For example, the position of the extracted image is determined so that the center of the image area of the main subject overlaps the center of the background image area excluding the display area of the additional information.

  Next, in step S217, it is determined whether or not the extracted image having the size calculated in step S211 or S213 can be extracted at the position determined in step S215. Here, when the region of the image to be extracted as the extracted image protrudes from the target image (see FIG. 5), the process proceeds to step S219, and the supplemental image generation unit 104b performs compensation to compensate for the lacking region. Generate an image. As described above, the supplement image may be, for example, an image configured by pixels of colors obtained by averaging the color information of the target image corresponding to the boundary with the insufficient portion. Further, a predetermined amount of region located at the boundary with the target image may be generated by gradation between the color of the boundary portion and the above color.

  Next, referring to FIG. 8, in step S221, the image color changing unit 104c selects the color of the extracted image where the additional information is displayed and the display color (character color or image color) of the additional information such as a clock or calendar. ) To determine whether the color is the same color or the same brightness. As a result of the comparison, if it is determined that the colors are close to each other, the process proceeds to step S223, and the image color of the extracted image of the portion where the additional information is displayed is changed. For example, the image color changing unit 104c may change the luminance value of the image color of the extracted image of the corresponding part. Further, the color of the boundary portion between the portion where the image color is changed and the portion where the image color is not changed may be changed so as to be a gradation of both colors.

  Next, in step S225, a frame indicating the region of the extracted image calculated by the extraction position calculation unit 104a is superimposed on the target image and displayed on the display unit 142. At this time, the target image may be displayed with the image color changed by the image color changing unit 104c in step S223. Further, a character or an image such as a clock or calendar displayed as additional information may be superimposed on the target image and displayed so that the user can confirm the image when displayed as the background image. Moreover, you may make it display the compensation image produced | generated by step S219 simultaneously.

  Next, in step S227, the background image generation unit 104 can accept an input for changing the position of the extraction image from the operation input unit 141 while a frame indicating the region of the extraction image is displayed. If there is an input to confirm the setting of the extracted image, the process proceeds to step S231.

  On the other hand, in step S227, when there is an input for changing the position of the extracted image from the operation input unit 141, the process proceeds to step S229, and the position of the extracted image calculated by the extracted position calculating unit 104a is changed. Furthermore, as necessary, the compensation image generation unit 104b and the image color change unit 104c generate a compensation image and change the image color based on the extracted image position after the change. The processes in steps S227 and S229 are repeated until the user inputs to confirm the position of the extracted image displayed in step S225.

  When an input for confirming the setting of the extracted image is received in step S227, the process proceeds to step S231, and a background image is generated based on the confirmed position of the extracted image. Specifically, the trimming process is performed based on the position of the extracted image determined for the original target image. Further, in step S233, when the image color is changed by the image color changing unit 104c and the supplementary image is generated by the supplementary image generating unit 104b, the trimmed image and the supplemental image are combined.

  Thereafter, in step S235, the size changing unit 104e generates a final background image by performing size changing processing on the image that is trimmed by the trimming processing unit 104d and combined with the supplementary image. In step S237, the generated background image is stored in the memory card 143 or the like, or transmitted to an external device via the communication interface 138 or the like.

  In an external device, a desired image can be used as a background image simply by setting a background image (wallpaper, standby image) for the image generated in this way. At this time, since the position of the clock or calendar displayed on the background image is also taken into consideration, it is possible to avoid the main subject included in the background image from being hidden by the display. Also, since the background image at the position where the additional information such as a clock or calendar is displayed is adjusted to be easy to see according to the display color of the additional information, the display color setting of the additional information is changed according to the background image. This can save you the trouble.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

It is a block diagram which shows the hardware constitutions of the digital still camera concerning one Embodiment of this invention. It is a functional block diagram which shows schematic structure for implement | achieving the background image generation function of the digital still camera concerning the embodiment. It is a figure which shows an example of the position correction process of an extracted image. It is a figure which shows an example of the position correction process of an extracted image. It is a figure which shows an example of the production | generation process of a compensation image. It is a figure which shows an example of the image color change process of the image used as the background of additional information. It is a flowchart which shows the flow of the background image generation process performed by the digital still camera concerning one Embodiment of this invention. It is a flowchart which shows the flow of the background image generation process performed by the digital still camera concerning one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Digital still camera 101 Display apparatus information acquisition part 102 Subject recognition part 103 Subject size acquisition part 104 Background image generation part 104a Extraction position calculation part 104b Complementary image generation part 104c Image color change part 104d Trimming process part 104e Size change part 110 Imaging process Unit 111 Optical block 112 Drive unit 113 Image sensor 114 Analog processing circuit 120 Image processing unit 130 Control unit 131 MPU
132 System bus 133 ROM
134 RAM
135 Operation Input Interface 136 Display Control Unit 137 Memory Card Interface 138 Communication Interface 139 External Input / Output Interface 141 Operation Input Unit 142 Display Unit 143 Memory Card

Claims (9)

An imaging device that generates a background image for displaying a background on a display device provided in another external device from a captured image,
A display device information acquisition unit that acquires display device information related to the display capability of the display device and additional information display information related to additional information including characters or images displayed on a background image of the display device;
A subject recognition unit for recognizing a subject included in the captured image;
A subject size acquisition unit that acquires information indicating the size of the subject with respect to the background image;
A background image generation unit that extracts all or part of the captured image and generates the background image from the extracted image;
With
The background image generation unit
The image pickup apparatus includes an extraction position calculation unit that calculates a size and a position of the extracted image based on information output from the display device information acquisition unit, the subject recognition unit, and the subject size acquisition unit. apparatus. A display unit that superimposes and displays the position of the extracted image calculated by the extraction position calculation unit on the captured image;
An input unit for performing input so that the position of the extracted image is changed while the position of the extracted image is displayed on the display unit;
Further comprising
The background image generating unit extracts the extracted image from the photographed image based on the position of the extracted image changed by the input from the input unit, and generates the background image using the extracted image. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. The additional information display information includes information about a display position and a display color of the additional information on the display device,
The background image generation unit
Based on the display position of the additional information on the display device, the display position of the additional information when the extracted image is used as the background image is calculated, and the color of the extracted image at the display position and the additional information The imaging apparatus according to claim 1, further comprising: an image color changing unit that changes an image color of the extracted image at the display position when the display color is similar.   The imaging apparatus according to claim 3, wherein the image color changing unit changes a luminance value of the extracted image at the display position.   5. The display according to claim 3, wherein when the position of the extracted image is displayed on the display unit, the color of the extracted image changed by the image color changing unit is changed and displayed. The imaging device described. The background image generation unit
If there is a part outside the area of the photographed image in the area of the extracted image determined by the position of the extracted image calculated by the extraction position calculation unit, the extracted image of the part outside the area of the photographed image is compensated The imaging apparatus according to claim 1, further comprising a supplementary image generation unit that generates a supplementary image to be performed. The supplementary image generation unit
The imaging apparatus according to claim 6, wherein the supplementary image is generated based on a color of a pixel of the extracted image located at a boundary with a portion outside the region of the background image.   The imaging apparatus according to claim 6, wherein when the position of the extracted image is displayed on the display unit, the supplementary image generated by the supplemental image generation unit is displayed simultaneously. . A background image generation method for generating a background image for displaying a background on a display device provided in another external device from a captured image captured by an imaging device,
A display device information acquisition step for acquiring display device information relating to the display capability of the display device and additional information display information relating to additional information including characters or images displayed on a background image of the display device;
A subject recognition step for recognizing a subject included in the captured image;
A subject size acquisition step of acquiring information indicating a size of the subject with respect to the background image;
A background image generation step of extracting all or part of the captured image and generating the background image from the extracted image;
Including
The background image generation step includes
An extraction position calculation step of calculating a size and a position of the extracted image based on information output from the display device information acquisition unit, the subject recognition unit, and the subject size acquisition unit; Image generation method.

Images