KR101427662B1 - Photographing apparatus and method of making background image - Google Patents

Abstract

An image pickup apparatus according to the present invention is an image pickup apparatus for generating a background image to be displayed on a background of a display device of another external apparatus from a captured image, the image pickup apparatus comprising display device information relating to display capability of the display device, A display device information acquiring section for acquiring additional information display information relating to additional information including additional images, a subject recognizing section for recognizing a subject of the shot image, a subject size acquiring section for acquiring information on the size of the subject, , And a background image generation unit for calculating the size and position of the extracted image extracted from the photographed image based on the information output from the display device information acquisition unit, the subject recognition unit, and the subject size acquisition unit, and generating a background image from the extracted image do.

Description

TECHNICAL FIELD [0001] The present invention relates to an imaging apparatus,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus and a background image generation method, and more particularly to an image pickup apparatus and a background image generation method using an image pickup device such as a digital camera, Screen, and idle screen), and a method of generating a background image.

2. Description of the Related Art Generally, an imaging apparatus such as a digital camera can shoot a high-quality image as compared with an imaging function provided in a mobile phone. Therefore, users often want to set a high-quality image taken with a digital camera as a background screen (a standby screen or a desktop screen) of a display mounted on a device such as a cellular phone.

However, there is a problem that it takes a long time to transfer an image photographed by a digital camera to a mobile phone. Furthermore, since the format and the size (file size) of the image data that can be handled by the mobile phone are limited, the input image data of the image pickup apparatus can not be displayed as it is. BACKGROUND ART [0002] In recent years, techniques for appropriately processing an image taken by an imaging device and inputting it to a mobile phone have been developed.

For example, Japanese Patent Laid-Open Publication No. 2004-40601 discloses an image processing method and an image processing method in which image conditions (file format, image size, and the like) required by a service (a print service, a desktop conversion service, And transmits the image data subjected to the image processing to the transmission destination.

Japanese Patent Publication No. 3925051 discloses a technique of receiving information on the ability of an image display apparatus to display an image and converting the image data to fit the information of the received image display capability, To a digital camera.

As described above, a technique for transmitting an image captured by an image capture device such as a digital camera after editing it according to the image display capability of the transmission destination apparatus has been conventionally proposed. However, when an image photographed by a digital camera or the like is to be set on a desktop screen of a cellular phone or the like, various problems that can not be solved by the above-described conventional techniques arise.

For example, in general, a display provided in a digital camera is a horizontally long screen, whereas a majority of a display of a cellular phone is a vertically long screen. Therefore, when an image photographed long in the horizontal direction is displayed on the mobile phone display formed long in the vertical direction, margins are generated in the vertical direction. On the other hand, when the long-side image taken by the digital camera is reduced in accordance with the length in the vertical direction of the display of the mobile phone, both ends are cut off and an area not displayed in the horizontal direction of the image occurs. It takes a lot of time and effort to adjust this to make sure that the main subject included in the image is accurately displayed at the center of the display.

Furthermore, in such a complicated operation, an image suitable for display on a cellular phone display is changed. When a changed image is actually displayed, a clock, calendar, or the like already displayed on the display may cover the main object. Further, when the color of the clock or calendar and the image of the background image are the same series of colors, the display of the clock or calendar is difficult to identify.

An object of the present invention is to provide an image capturing apparatus and an image capturing apparatus capable of displaying an image captured by an image capturing apparatus such as a digital camera on a background image (a desktop screen, an idle screen) of a display device provided in an external device such as a cellular phone by a simple method And a method of generating an image.

The present invention provides an imaging apparatus and a background image generation method for recognizing a subject included in a photographed image, extracting an image based on the size of the subject and display device information, and generating a background image from the extracted image.

An image pickup apparatus according to the present invention is an image pickup apparatus for generating a background image to be displayed on the background of a display device provided in another external apparatus from a photographed picked up image, comprising: display apparatus information relating to display capability of the display apparatus; A display unit information acquiring unit that acquires additional information display information related to additional information including a character to be displayed on the display screen and additional information including an additional image, a subject recognizing unit that recognizes a subject included in the shot image, The size and position of the extracted image extracted from the captured image based on the information output from the display device information acquiring unit, the object recognizing unit, and the object size acquiring unit, And a background image generation unit for generating a background image from the image.

With this configuration, the image photographed by the image capturing apparatus can be easily used as the background image of the display of the external apparatus. In addition, a background image is generated by recognizing a main subject included in the photographed image and taking into account the position and size thereof, so that a user can easily obtain a background image.

In the present invention, the image capturing apparatus may further comprise a display section for displaying the position of the extracted image superimposed on the captured image and an input section for inputting an instruction for changing the position of the extracted image displayed on the display section, It is possible to extract the extracted image from the photographed image based on the position of the extracted image changed by the input of the input unit and generate the background image using the extracted image.

Thereby, the user can confirm in advance which portion of the shot image is extracted as the background image before the background image is generated. If the extracted position is different from the user's intention, the desired background image can be generated by changing the position.

In the present invention, the additional information display information may include information relating to the display position and the display color at which the additional information is displayed on the display device, and the background image generation unit may generate the background image, And an image color changing unit that changes the image color of the extracted image at the additional information display position when the color of the extracted image is similar to the display color of the additional information.

In the present invention, the image color changing unit can change the luminance value of the extracted image at the display position.

In the present invention, the image color changing unit may change the color of the extracted image from the changed image color to the unchanged image color at the boundary between the portion where the image color is changed in the extracted image and the portion where the image color is not changed in the extracted image. The gradation region which gradually changes can be formed.

This makes it possible to prevent a phenomenon in which a clock or a calendar displayed on the background image becomes hard to see when the generated background image is displayed on the display device, and it is possible to prevent the display color of additional information such as a clock, Can be reduced.

In the present invention, when the position of the extracted image is displayed on the display unit, the color of the extracted image changed by the image color changing unit can be changed and displayed. Furthermore, a character or an image representing additional information may be superimposed and displayed at a position where the additional information is displayed. Thereby, the user can confirm the image when the background image is displayed on the display device in advance, and thereby can determine whether to change the position of the image or the like.

In the present invention, when the area of the extracted image includes a portion existing outside the region of the photographed image, the background image generation section generates the supplemental image for supplementing the extracted image of the portion existing outside the region of the photographed image And a generation unit.

In the present invention, the replenishment image generation unit can generate the replenishment image based on the color of the pixel of the extracted image located at the boundary portion of the captured image that is touched by the region of the extracted image existing outside the captured image region.

The supplemental image may have a color obtained by averaging the color information of the pixels of the extracted image located at the boundary of the picked-up image touched by the area of the extracted image existing outside the area of the picked-up image.

Or the supplementary image generation section can gradually generate a supplementary image by gradually changing the color of the pixel of the extracted image located at the boundary portion of the picked-up image touched by the region of the extracted image existing outside the region of the picked-

Thereby, even when the main subject is positioned at the corner of the photographed image, for example, the area around the main subject can be supplemented to generate an image.

In the present invention, the image pickup apparatus may further include a display section for superimposing the position of the extracted image on the photographed image for display, and when the position of the extracted image is displayed on the display section, the supplementary image Can be displayed simultaneously. Thereby, the image of the final background image including the supplemental image can be confirmed in advance.

A background image generation method according to the present invention is a background image generation method for generating a background image to be displayed as a background on a display device provided in another external apparatus from a shot image photographed by an image pickup apparatus, A display device information acquiring step of acquiring additional information display information related to additional information including a character to be displayed on the display device and additional information, a subject recognition step of recognizing the subject included in the shot image An object size acquiring step of acquiring information on the size of the object occupied by the object in the background image, a display device information acquiring step, an object recognizing step, and a step of acquiring the object size extracted in the captured image based on the information output from the object size acquiring step Calculates the size and position of the extracted image, generates a background image from the extracted image It comprises a background image generating step.

The image pickup apparatus and the background image generation method of the present invention as described above can display an image photographed by an image pickup apparatus such as a digital camera as a background image of a display device provided in an external apparatus such as a cellular phone by a simple method .

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the configuration and operation of an image pickup apparatus and a background image generating method according to the present invention will be described in detail with reference to the accompanying drawings. In the present specification and drawings, elements having substantially the same functional configuration are denoted by the same reference numerals, and redundant description will be omitted.

First, an image pickup apparatus according to an embodiment of the present invention will be described. The image pickup apparatus according to the present embodiment has a background image generation function for easily displaying a picked-up image on a background image (a desktop screen, an idle screen) of a display device provided in an external apparatus such as a cellular phone. Hereinafter, a digital still camera will be described as an example of an image pickup apparatus, but the present invention is not limited to such an example. The image pickup apparatus according to the present invention may be an apparatus having a moving image photographing function such as a digital video camera.

■ Hardware configuration of digital still camera (100)

1 is a block diagram schematically showing the hardware configuration of a digital still camera 100 as an image pickup apparatus according to an embodiment of the present invention.

The digital still camera 100 according to the embodiment shown in FIG. 1 includes an image pickup processing section 110, an image processing section 120, and a control section 130.

The image pickup processing unit (110)

The image pickup processing section 110 includes an optical block 111, a driving section 112, an image pickup element 113 and an analog processing circuit 114. The optical block 111 includes a lens group for capturing an object, a diaphragm adjustment mechanism, a focus adjustment mechanism, a zoom mechanism, a shutter mechanism, a flash mechanism, and the like. The driving unit 112 receives the control signal from the control unit 130 and is controlled to drive the diaphragm adjustment mechanism of the optical block 111, the zoom mechanism, the shutter mechanism, and the like.

The image pickup device 113 may be a solid-state image pickup device such as a charge coupled device (CCD), for example. An image passing through the optical block 111 is formed on the image-forming surface of the image pickup device 113. The image pickup element 113 receives the timing signal generated by the analog processing circuit 114 and converts the image of the subject image formed on the image plane into an image pickup signal and supplies the image signal to the analog processing circuit 114.

The analog processing circuit 114 performs processing for converting an analog imaging signal from the imaging element 113 into a digital signal. The analog processing circuit 114 includes an AGC (auto gain control) for automatic gain adjustment of an image pickup signal from the image pickup element 113, an AD (analog digital) converter for converting an analog signal into a digital signal, And a timing generator for generating an image acquisition timing signal for the image pickup element 113 on the basis of a control signal supplied from the timing controller 130. The digital imaging signal output by the analog processing circuit 114 is RAW image data, which is stored in a storage medium such as the RAM 134. [

In the image processing unit 120,

The image processing unit 120 reads raw RAW image data output by the analog processing circuit 114 and stored in the RAM 134 or the like to perform digital image processing and outputs JPEG (Joint Photographic Experts Group), TIFF File Format), and records the converted data on a storage medium such as the memory card 143. [

The image processing section 120 performs various image processing such as pixel interpolation processing, gamma correction processing, and white balance processing on RAW image data. The image processing unit 120 also resizes (resizes, changes the number of pixels), trimming, and crops the photographed image stored in the memory card 143 or the like based on the control signal from the control unit 130. [ crop and so on.

The control unit (130)

1, the control unit 130 includes a micro processing unit (MPU) 131, a read only memory (ROM) 133, a random access memory (RAM) A display control unit 136 for connecting the display unit 142, a memory card interface 137 for charging the memory card 143, a communication interface 138, And an external input / output interface 139 are connected via a system bus 132.

The control unit 130 controls the image pickup processing unit 110 and the image processing unit 120 to control the entire digital still camera 100 and executes image pickup processing. In particular, the control unit 130 has a function (background image generation function) for generating a background image for displaying an image picked up by the image pickup processing unit 110 as a background image of a display device of an external apparatus.

The control unit 130 can execute a background image generation function in various ways. The MPU 131 reads out the program stored in the ROM 133 or the like from the image data read from the memory card 143 or the like or the JPEG image data output from the image pickup processing section 110 and stored in the RAM 134 or the like The background image generation function can be executed. Hereinafter, the configuration of the control unit 130 will be described.

The MPU 131 supplies various control signals generated by executing the control program recorded in the ROM 133 to each part, and controls the whole of the digital still camera 100. The ROM 133 stores a program for controlling the image pickup processing unit 110, the image processing unit 120, the display control unit 136, and the like or the memory card interface 137. The RAM 134 stores the RAW image data output from the analog processing circuit 114 of the image pickup processing unit 110 and the JPEG data read from the memory card 143 and decoded by the image processing unit 120 to be. The RAM 134 is also used as a work area of the MPU 131.

The operation input unit 141 connected to the operation input interface 135 is connected to a plurality of operation keys such as a mode conversion key for changing the photographing mode, a zoom adjusting key, a key for exposure adjustment, a shutter key, Keys. The operation input interface 135 transfers an operation signal from the operation input unit 141 to the MPU 131. [ The MPU 131 discriminates the contents of the operation based on the operation signal input from the operation input interface 135, and performs the control processing according to the discrimination result.

The display unit 142 connected to the display control unit 136 is composed of a liquid crystal display or the like formed of a TFT (Thin Film Transistor) or the like. The display control section 136 displays the image processed by the image processing section 120 and the image read from the memory card 143 on the display section 142 under the control of the MPU 131. [

The memory card interface 137 writes the image data output from the image processing unit 120 to the memory card 143. [ The memory card interface 137 also reads image data from the memory card 143 and supplies the image data to the image processing unit 120 and the like. The memory card 143 is a storage medium read / written by the memory card interface 137. Attached to the memory card 143 are photographed image data and attribute information accompanying the image data.

The communication interface 138 is an interface for communicating with an external device, and receives data transmitted from an external device via a network or transmits data via a network. The external input / output interface 139 is an interface for connecting an external device to the digital still camera 100 to perform data input / output and outputs data stored in the memory card 143 of the digital still camera 100 to an external device And an interface for inputting data input from an external device to the digital still camera 100. [

○ Configuration of background image generation function

2 is a functional block diagram showing a schematic configuration for implementing a background image generating function of the imaging apparatus according to the embodiment shown in Fig. The background image generation function executed by the control unit 130 of the digital still camera 100 will be described with reference to FIG.

2, the digital still camera includes a display device information acquisition unit 101, a subject recognition unit 102, a subject size acquisition unit 103, a background image And a generating unit 104.

2 may be realized by a software program stored in the ROM 133 or the like and executing the functions of the respective parts that can be executed by the MPU 131, . The software program may be executed by reading the program stored in a computer-readable storage medium such as the ROM 133 as described above, or may be provided to the digital still camera 100 through a network or the like.

The display device information acquisition unit (101)

The display device information acquisition unit 101 acquires information about a display device of an external device that displays a background image generated by the function of editing the image and inputs the information to the background image generation unit 104. [ The display device information includes, for example, information about the display capability of the display device provided in the target external device (the size of the displayable image, the format of the image, and the like), a clock displayed together with the background image of the display device Setting information (additional information display information) relating to display of icons, calendars, and the like (hereinafter referred to as additional information). The additional information display information includes, for example, the display position of the additional information, the size of the displayed area, and the display color information.

The display device information acquisition unit 101 acquires the display device information from the operation input unit 141, the communication interface 138, the external input / output interface 139, and the like. For example, the display device information acquisition unit 101 acquires the value of the display device information (the numerical value of the displayable image size of the display device, etc.) input from the operation input unit 141 via the operation input interface 135 It is possible. Or the setting information stored in the storage medium such as the ROM 133 may be input from the operation input unit 141. [

The display device information acquiring unit 101 acquires display device information transmitted from an external device via the network from the communication interface 138 or transmits information input from an external device connected via the external input / May be acquired.

The display device information acquisition unit 101 may also acquire the display device information from the image photographed by the imaging processing unit 110. [ For example, the display device of an external device targeted by the digital still camera 100 is photographed, and the information such as the display position of the additional information, the size of the display area, and the display color are acquired from the image of the photographed display device It is possible. In this case, information on the display capability of the display device is acquired by another method, for example, by input from the operation input section 141. [ The above-described method of acquiring the display device information may be used in combination of two or more methods.

The subject recognition unit 102,

The subject recognition unit 102 automatically recognizes a main subject included in a shot image (hereinafter, referred to as a target image) to be a background image. The number of recognized objects may be one or plural. In the following description, one of the subjects included in the target image is recognized as a main subject. When the main object is recognized in the target image, the image size of the area including the main subject and the position in the target image (for example, the pixel coordinates at the upper left of the area including the main subject) (104).

Recognition of a main subject is performed by object detection by detecting a face image or edge extraction. For example, the subject recognition unit 102 may detect a face image by a method such as pattern matching, and recognize the detected area of the face image as a main subject area. When a plurality of face images are detected, it is also possible to make a face image at a position closest to the center of the image as a main subject or a face image having the largest area as a main subject.

When the face image is not detected, the object recognition unit 102 performs object detection by performing edge extraction or the like on the object image. Here again, when a plurality of objects are detected, the main object may be determined based on the position and the area of the object. In the above-described face image detection processing and object detection processing, image processing techniques such as conventionally known pattern matching and edge extraction can be used.

The subject size acquisition unit 103

The subject size acquiring unit 103 acquires information about the size of a main subject occupied in the generated background image (hereinafter, referred to as subject size) and inputs the information to the background image generating unit 104. [ The subject size can be specified, for example, by the proportion of the main subject occupying the background image (for example, 50% of the background image). The information on the subject size may be a value recorded in advance on a storage medium such as the ROM 133 or a value input from the operation input unit 141. [ Or a selected value among a plurality of values prestored in the ROM 133 or the like.

The background image generation unit 104,

The background image generating unit 104 edits the target image based on the information input from the display device information acquiring unit 101, the object recognizing unit 102 and the object size acquiring unit 103, Thereby generating an image for display as a background image. 2, the background image generating unit 104 includes an extraction position calculating unit 104a, a supplementary image generating unit 104b, an image color changing unit 104c, a trimming processing unit 104d, And a size changing unit 104e.

The extraction position calculation unit 104a

The extraction position calculation section 104a calculates the extraction position calculation section 104a based on the displayable image size of the display device acquired by the display device information acquisition section 101, the size of the main object area recognized by the subject recognition section 102, (Extracted image) extracted as a background image from the photographed image based on the subject size acquired by the imaging unit 103. [ Specifically, the size of the extracted image is calculated so that the size of the area of the main subject recognized by the subject recognizing unit 102 becomes the size of the subject in the background image determined from the subject size.

For example, assuming that the displayable image size of the display device given by the display device information acquisition section 101 is 460 x 800 pixels, the area of the main subject recognized by the subject recognition section 102 is 600 x 800 pixels, It is assumed that the subject size acquired by the subject size acquiring unit 103 is 50% of the background image. In this case, the total number of pixels of the extracted image is 600 x 800 x 1 / 0.5 = 960,000 pixels. If this is distributed to the aspect ratio of the displayable image size of the display device, the size of the extracted image (x is the size in the vertical direction and y is the size in the horizontal direction) is required by the following expression (1).

Therefore, the size of the extracted image is 742 x 1292 pixels.

If the original target image is smaller than the extracted image size obtained in Equation (1), the size of the extracted image is corrected in accordance with the size of the target image. For example, in the above example, when the vertical size (y) of the target image is 1000 pixels, since the size in the y direction is 1000 < 1292 and the vertical length of the target image is not suitable for the extracted image size, Calibration is required. In this case, for example, the vertical length (y) of the extracted image is 1000 pixels, and the horizontal length (x) is corrected as follows.

x = 1000 x 742/1292? 574

Therefore, the size of the extracted image is 574 x 1000 pixels.

The extraction position calculation unit 104a determines the position of the extracted image after calculating the extracted image size. At this time, the extraction position calculation unit 104a determines the position of the extracted image so that the main subject does not overlap the display position of the additional information, based on the additional information display information input from the display device information acquisition unit 101. [

3 is a conceptual diagram showing an example of a position correction process of an extracted image in a background image generating method according to an embodiment of the present invention.

For example, as shown in Fig. 3, when the clock display, which is the additional information, is displayed below the background image, the center of the face of the person as the main subject overlaps the center of the area excluding the clock display area of the background image The position of the extracted image is determined.

4 is a conceptual diagram showing an example of a position correction process of an extracted image in a background image generating method according to an embodiment of the present invention.

On the other hand, as shown in Fig. 4, even if the additional information is located at the center of the screen and the main subject is placed on either the upper side or the lower side (or the right side or the left side) of the additional information, The lower side (or the right side or the left side) may be selected and arranged. For example, the position of the extracted image may be determined so that the main subject is necessarily the upper side of the additional information, or the position of the extracted image may be determined according to the overlapping state of the main subject and the additional information.

For example, a case where a half or more of the area of the main object overlaps with the display area of the additional information is determined to be overlapped with the criterion that the main object and the additional information overlap, It may be determined that they do not overlap.

Furthermore, after the extraction position calculating section 104a calculates the size and position of the extracted image, a rectangle representing the area of the extracted image in the target image may be superimposed on the target image and displayed on the display section 142. [ The present invention is not limited to this, and a symbol disposed along the edge of the extracted image may be displayed to indicate the position of the extracted image. Further, the user can arbitrarily change the position of the rectangle by operating the key provided on the operation input unit 141 while the display unit 142 is displayed.

The supplementary image generation unit 104b generates,

5 is a conceptual diagram showing an example of a supplementary image generation process in a background image generation method according to an embodiment of the present invention.

As shown in Fig. 5, in the case where the extracted image area is not suitable for the area of the background image required at the position of the extracted image calculated by the extraction position calculating unit 104a , A supplementary image for supplementing the region of the deficient portion is generated. The supplemental image can be, for example, an image composed of pixels of a color obtained by averaging the color information of the target image at the boundary portion with the region of the deficient portion. It is also possible to generate the supplementary image by gradation in which a predetermined amount of area located at the boundary with the target image is gradually changed from the color of the boundary portion to the above color.

The image color changing unit 104c

The image color changing unit 104c compares the display color (character color or image color) of the additional information such as a clock or calendar with the color of the extracted image of the portion overlapping with the additional information. If the same color or the same luminance, Correct the color and brightness of the image. For example, the difference between the values of the YCbCr color system of each color to be compared based on the following equation (2) is determined. If the value is smaller than the predetermined threshold value, it is determined that the colors of both colors are close to each other. You may. In the following example, the case where the threshold value is set to 15 is shown.

(DELTA Y, DELTA Cb and DELTA Cr are the difference between the values of Y, Cb and Cr, respectively)

The display color of the additional information is compared with the color of the extracted image as described above, and when it is determined that the colors of the extracted images are close to each other, the image color changing unit 104c changes the image color of the extracted image. For example, the image color changing section 104c can change the image color by changing the luminance value (brightness) (Y) of the YCbCr value of the image color of the extracted image. For example, when the value of the luminance value Y of the display color of the additional information is Y? 128 (when the YCbCr value is 8 bits), the value of the luminance value Y 'of the image color of the extracted image is Y' Y + 64, and if the value of the luminance value (Y) of the display color of the additional information is Y < 128, the value of the luminance value (Y ') of the image color of the extracted image can be set to Y' = Y + 64 .

6 is a conceptual diagram showing an example of an image color change process of an image serving as a background of additional information in a background image generation method according to an embodiment of the present invention.

Further, as shown in Fig. 6, the image color changing unit 104c adds the color of the changed image (the color of the changed image and the color of the changed image to the unchanged portion) The color of the image which has not been changed gradually) can be formed gradually.

When a rectangle representing the area of the extracted image calculated by the extraction position calculation unit 104a is superimposed on the target image and displayed on the display unit 142, the position of the target image of the portion changed by the image color changing unit 104c The image color can be changed and displayed. Thereby, characters or images of a clock or calendar displayed as additional information can be simultaneously displayed on the target image, and all the images when displayed as the background image can be identified by the user.

The trimming processing unit (104d)

The trimming processing unit 104d extracts an image of a portion used as a background image by performing a trimming process on the photographed image as a target according to the size and position of the extracted image calculated by the extraction position calculation unit 104a . The extracted image is combined with the supplementary image generated by the supplementary image generation section 104b and input to the size change section 104e.

In the size changing unit 104e,

The size changing unit 104e generates a final background image by performing a process of trimming the trimming process performed by the trimming process unit 104d and changing the size of the image to which the supplemental image is combined. The size changing section 104e reduces or enlarges the image based on the ratio of the displayable image size of the display device received by the display device information obtaining section 101 and the size of the image output by the trimming processing section 104d Output.

An example of a background image generation processing method

Figs. 7 and 8 are flowcharts showing the flow of background image generation processing executed by the image pickup apparatus according to the embodiment shown in Fig. 1. Fig. An example of a background image generation processing method executed by the digital still camera 100 according to the present embodiment will be described with reference to Figs. 7 and 8. Fig.

Referring to Fig. 7, in step S201, generation of a background image is started in response to a user's input. The start of the generation of the background image may be immediately after the target image is photographed or when an image stored in the storage medium such as the memory card 143 is called up and the user instructs the processing.

Then, in step S203, the display device information acquiring unit 101 acquires information on the display capability of the display device of the external device that displays the background image by input from the operation input unit 141 Etc.), a display position and display color of a clock, an icon, a calendar, etc. displayed together with a background image of the display device. The acquired information is input to the background image generation unit 104. [ Subsequently, in step S205, the subject size acquiring unit 103 acquires the subject size and inputs it to the background image generating unit 104. [

Subsequently, in step S207, the subject recognition unit 102 recognizes a main subject included in the target image. Recognition of a main subject can be performed by detecting a face image or by detecting an object by edge extraction or the like. The subject recognition unit 102 obtains the size of the image area including the recognized main subject and the position in the target image, and inputs the position to the background image generation unit 104. [

Subsequently, in step S209, the extraction position calculation unit 104a of the background image generation unit 104 compares the displayable image size of the display device acquired by the display device information acquisition unit 101 with the displayable image size of the display device acquired by the subject recognition unit 102 The size of the extracted image extracted as the background image from the captured image is calculated based on the size of the recognized main subject area and the size of the subject acquired by the subject size acquiring unit 103. [ The concrete calculation method is as described above.

Further, in step S211, the extraction position calculating unit 104a compares the size of the extracted image calculated in step S209 with the size of the target image, and determines whether or not the extracted image of the calculated size can be extracted. If the extracted image size is larger than the target image size, the flow advances to step S213 to correct the extracted image size in accordance with the size of the target image.

After step S211 or S213, the extraction position calculating unit 104a determines the position of the extracted image in the target image in step S215. At this time, the extraction position calculation unit 104a determines the position of the extracted image so that the center of the main subject does not overlap the display position of the additional information, based on the additional information display information input from the display device information acquisition unit 101. [ For example, the position of the extracted image is determined so that the center of the image area of the main subject overlaps the center of the area of the background image excluding the display area of the additional information.

Then, in step S217, it is determined whether or not the extracted image of the size calculated in step S211 or S213 can be extracted from the position determined in step S215. Here, when the area of the image to be extracted as the extracted image deviates from the target image (see Fig. 5), the process shifts to step S219, and the supplemental image generating unit 104b generates a supplemental image for supplementing the deficient area . As described above, the supplemental image may be, for example, an image composed of pixels of a color obtained by averaging color information of a target image in a portion suitable for a boundary with an area of a deficient portion. Also, a predetermined amount of area located at the boundary with the target image may be generated by gradation that gradually changes from the color of the boundary portion to the above color.

8, in step S221, the image color changing unit 104c compares the color of the extracted image of the portion in which the additional information is displayed and the display color (character color or image color) of the additional information such as a clock or calendar , It is determined whether or not they are the same series color or the same luminance. When it is determined that the colors of both are close to each other, the process proceeds to step S223, and the image color of the extracted image of the portion where the additional information is displayed is changed. For example, the image color changing unit 104c may change the brightness value of the image color of the extracted image of the corresponding portion. In addition, a gradation portion in which the color of both the image (the color of the changed image and the color of the unchanged image) is gradually changed is formed at the boundary between the portion where the image color of the extracted image is changed and the portion where the image color is not changed .

Subsequently, in step S225, a rectangle representing the area of the extracted image calculated by the extraction position calculating unit 104a is superimposed on the target image and displayed on the display unit 142. [ At this time, the target image may be displayed with the image color changed by the image color changing unit 104c in step S223. This enables the user to identify an image when characters or images such as clocks or calendars displayed as additional information are superimposed on the target image and displayed as a background image. It is also possible to simultaneously display the supplementary images generated in step S219.

Subsequently, in step S227, the background image generating unit 104 may receive an input for changing the position of the extracted image from the operation input unit 141 while a rectangle indicating the area of the extracted image is displayed. Here, when there is an input such that the setting of the extracted image is confirmed, the process transitions to step S231.

On the other hand, if there is a position change input of the extracted image from the operation input unit 141 in step S227, the process shifts to step S229 and the position of the extracted image calculated by the extraction position calculation unit 104a is changed. Further, the supplementary image generation section 104b and the image color change section 104c perform the generation of the supplementary image and the change of the image color based on the extracted image position after the change, if necessary. The processes of steps S227 and S229 are repeated until the user inputs an input for confirming the position of the extracted image displayed in step S225.

Upon reception of an input for establishing the setting of the extracted image in step S227, the process transitions to step S231 and a background image is generated based on the determined position of the extracted image. More specifically, trimming processing is performed on the original target image based on the determined position of the extracted image. Furthermore, when the image color is changed by the image color changing unit 104c in step S233, and the supplemental image is generated by the supplemental image generation unit 104b, the image after the trimming process and the supplemental image are combined.

Thereafter, in step S235, the size changing unit 104e performs trimming processing by the trimming processing unit 104d and performs a size changing process on the image to which the supplementary image is combined, thereby generating a final background image. The generated background image is stored in the memory card 143 or the like in step S237 or transmitted to an external device via the communication interface 138 or the like.

In the external apparatus, a desired image can be used as a background image by simply setting an image generated through the above-described steps to a background image (a desktop screen, an idle screen). At this time, since positions of clocks, calenders, and the like displayed on the background image are also taken into consideration, it is possible to avoid that the main subject included in the background image is blocked by the display of the clock or calendar. In addition, since the background image at the position where the additional information such as the clock or the calendar is displayed is easily adjusted in accordance with the display color of the additional information, the time and effort for changing the display color setting of the additional information Can be reduced.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Accordingly, the true scope of the present invention should be determined by the appended claims.

1 is a block diagram schematically showing a hardware configuration of an image pickup apparatus according to an embodiment of the present invention.

2 is a functional block diagram showing a schematic configuration for implementing a background image generating function of the imaging apparatus according to the embodiment shown in Fig.

3 is a conceptual diagram showing an example of a position correction process of an extracted image in a background image generating method according to an embodiment of the present invention.

4 is a conceptual diagram showing an example of a position correction process of an extracted image in a background image generating method according to an embodiment of the present invention.

5 is a conceptual diagram showing an example of a supplementary image generation process in a background image generation method according to an embodiment of the present invention.

6 is a conceptual diagram showing an example of an image color change process of an image serving as a background of additional information in a background image generation method according to an embodiment of the present invention.

7 is a flowchart showing the flow of background image generation processing executed by the imaging apparatus according to the embodiment shown in Fig.

Fig. 8 is a flowchart showing the flow of the background image generation processing following Fig.

DESCRIPTION OF THE REFERENCE NUMERALS

100: digital still camera 120: image processing unit

101: Display device information acquisition unit 130:

102: subject recognition unit 131: MPU

103: Subject size acquisition unit 132: System bus

104: background image generation unit 133: ROM

104a: Extraction position calculating unit 134: RAM

104b: Replacement image generating section 135: Operation input interface

104c: image color changing section 136: display control section

104d: trimming processing unit 137: memory card interface

104e: size changing section 138: communication interface

110: image pickup processing section 139: external input / output interface

111: optical block 141: operation input section

112: driving part 142:

113: Imaging element 143: Memory card

114: analog processing circuit

Claims (12)

An image pickup apparatus for generating a background image for display on a background of a display device provided in another external apparatus from a picked-up image, comprising: A display device information acquisition unit that acquires display device information related to the display capability of the display device and additional information display information related to additional information including a character to be displayed on the display device and an additional image; A subject recognizing unit for recognizing a subject included in the photographed image; A subject size acquiring unit that acquires information on the size of the subject occupied by the subject in the background image; And The size and position of the extracted image extracted from the captured image is calculated based on the information output from the display device information acquisition unit, the object recognition unit, and the object size acquisition unit, and the background image is generated from the extracted image And a background image generation unit. The method according to claim 1, The image pickup apparatus includes: A display unit for superimposing the position of the extracted image on the photographed image and displaying it; And And an input unit for inputting an instruction to change the position of the extracted image displayed on the display unit, Wherein the background image generation unit extracts the extracted image from the photographed image based on the position of the extracted image changed by the input of the input unit and generates the background image using the extracted image. 3. The method of claim 2, Wherein the additional information display information includes information about a display position and a display color at which the additional information is displayed on the display device, Wherein the background image generation unit calculates the difference between the value of the YCbCr color system of each color of the color of the extracted image and the color of the additional information at the position of the background image corresponding to the display position of the additional information, Further comprising an image color changing section for changing an image color of the extracted image at the additional information display position when the additional information is displayed. [Mathematical Expression]

(Where each of? Y,? Cb, and? Cr is the difference between the luminance value Y and the color difference Cb and Cr) The method of claim 3, And the image color changing section changes the luminance value of the extracted image at the display position. The method of claim 3, The image color changing section may change the color between the changed image color of the extracted image and the unchanged image color at the boundary between the portion where the image color is changed in the extracted image and the portion where the image color is not changed in the extracted image Thereby forming gradation regions that gradually change. 6. The method according to any one of claims 3 to 5, And changes the color of the extracted image changed by the image color changing unit when the position of the extracted image is displayed on the display unit. The method according to claim 1, Wherein the background image generation unit comprises: And a supplemental image generating unit that generates a supplemental image for supplementing an extracted image of a portion existing outside the region of the photographed image when the region of the extracted image includes a portion existing outside the region of the photographed image, . 8. The method of claim 7, Wherein the supplemental image generation unit comprises: And generates the supplemental image based on a color of a pixel of the extracted image located at a boundary portion of the captured image that is in contact with an area of the extracted image existing outside the area of the captured image. 9. The method of claim 8, The supplemental image is, And a color obtained by averaging color information of pixels of the extracted image located at a boundary portion of the captured image which is in contact with an area of the extracted image existing outside the area of the captured image. 9. The method of claim 8, Wherein the supplemental image generation unit comprises: And gradually changes the color of a pixel of the extracted image located at a boundary portion of the captured image that is in contact with an area of the extracted image existing outside the area of the captured image to generate the supplemental image. 11. The method according to any one of claims 7 to 10, Wherein the image pickup apparatus further comprises a display section for superimposing the position of the extracted image on the photographed image for display, Wherein when the position of the extracted image is displayed on the display unit, the supplemental image generated by the supplemental image generation unit is simultaneously displayed. A background image generation method for generating a background image to be displayed in a background on a display device provided in another external apparatus from a captured image photographed by an imaging apparatus, A display device information acquiring step of acquiring additional information display information on display device information relating to the display capability of the display device and additional information including characters to be displayed on the display device and additional images; A subject recognition step of recognizing a subject included in the shot image; A subject size acquiring step of acquiring information on the size of the subject occupied by the subject in the background image; And the size and position of the extracted image extracted from the captured image based on the information output from the subject size acquiring step, and extracting the background image from the extracted image A background image generation step of generating a background image;

Images