JP2007243241A - Mobile communication terminal and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile communication terminal capable of easily acquiring more satisfactory photographed images. <P>SOLUTION: When a self-photographing mode is selected as an operation mode of an imaging function by a camera unit 160, a control unit 110 calculates the inclination of the mobile communication terminal 100, on the basis of a posture detection of a posture detecting unit 180. The control unit 110 calculates the degree of distortions, corresponding to the calculated inclination for each distance of a vertical direction from the center position of the image for an image photographed by self-photographing, and corrects the image on the basis of the calculated degree of distortion. In this case, distortions in the horizontal direction of the image is corrected per the coordinate in the vertical direction, and the distortions in the vertical direction of the image is corrected per block having a height in the vertical direction, thereby efficiently readily correcting trapezoidal distortions generated in self-photographing, and outputting the image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a mobile communication terminal and a program, and more particularly to a mobile communication terminal and a program suitable for improving convenience of a mobile communication terminal having an imaging function.

  In mobile communication terminals such as mobile phones, various functions other than a call function which is a basic function are added. For example, an imaging function using an imaging element such as a charge coupled device (CCD) (charge coupled device) Camera function) is installed in many mobile communication terminals.

A mobile communication terminal having such an imaging function is provided with a display unit such as a liquid crystal display device, and is used as a viewfinder at the time of shooting by displaying a captured image. Many mobile communication terminals having a so-called self-shooting function, in which a photographer can shoot while viewing his / her own subject image when such a display unit and a lens used for imaging are in the same direction, have been proposed. (For example, Patent Document 1).
JP 2005-167322 A

  In such a self-portrait, the face portion of the photographer is a subject to be photographed. However, when photographing with the mobile communication terminal held naturally, the position of the mobile communication terminal is lower than the face portion that is the subject of photographing. There are many cases. Thus, when the imaging direction with respect to the imaging target is directed from the bottom to the top, the so-called trapezoidal distortion in which the captured image is distorted in a trapezoidal shape is likely to occur.

  In order to prevent such trapezoidal distortion, it is necessary to shoot in such a posture that the mobile communication terminal is positioned in front of the shooting target, and there is a disadvantage that the degree of freedom of shooting is hindered.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a mobile communication terminal capable of easily obtaining a better captured image.

In order to achieve the above object, a mobile communication terminal according to the first aspect of the present invention includes:
In a mobile communication terminal having an imaging function,
Attitude detecting means for detecting the attitude of the mobile communication terminal;
An inclination calculating means for calculating an inclination of the mobile communication terminal based on the attitude of the mobile communication terminal detected by the attitude detection means when the self-portrait mode is selected by the imaging function;
Image correcting means for correcting distortion of an image captured in the self-portrait mode based on the inclination of the mobile communication terminal calculated by the inclination calculating means;
It is characterized by providing.

  According to such a configuration, for example, by detecting the attitude of the mobile communication terminal using an acceleration sensor or the like, the inclination of the mobile communication terminal is calculated and the captured image is corrected. Trapezoidal distortion caused by tilt can be automatically corrected and displayed and saved.

The mobile communication terminal is
The image correction means corrects lateral distortion in the vertical coordinate unit of the image based on the distance from the center position of the image captured in the self-portrait mode and the inclination of the mobile communication terminal. Is desirable.

  According to such a configuration, the trapezoidal distortion in which the horizontal distortion degree changes according to the vertical distance from the center position of the image is corrected based on the inclination of the mobile communication terminal that affects the distortion degree. Therefore, effective trapezoidal distortion correction can be performed efficiently.

In the mobile communication terminal,
The image correction means sets a plurality of blocks in the vertical direction of the image captured in the self-portrait mode, and based on the distance from the center position of the image and the inclination of the mobile communication terminal for each block. It is desirable to correct the vertical distortion.

  According to such a configuration, the trapezoidal distortion in which the vertical distortion degree changes according to the vertical distance from the center position of the image is corrected based on the inclination of the mobile communication terminal that affects the distortion degree. Therefore, effective trapezoidal distortion correction can be performed efficiently. As for distortion in the vertical direction, effective distortion correction can be efficiently performed by setting a block having a height in the vertical direction on the image and correcting the block in units of blocks.

In order to achieve the above object, a program according to the second aspect of the present invention is:
In a computer that controls a mobile communication terminal having an imaging function,
A function of selecting an operation mode according to the imaging function;
A function of calculating the inclination of the mobile communication terminal when the selfie mode is selected as the operation mode;
A function of correcting an image captured in the self-portrait mode based on the calculated inclination of the mobile communication terminal;
It is characterized by realizing.

  By applying such a program, an existing mobile communication terminal can function as the mobile communication terminal.

  According to the present invention, when taking a selfie with the mobile communication terminal, the captured image is corrected based on the inclination of the mobile communication terminal, so that a better captured image can be obtained.

  Embodiments according to the present invention will be described below with reference to the drawings. In the present embodiment, a case will be described in which the present invention is applied to a mobile communication terminal (a mobile phone or a personal handyphone system (PHS) terminal device) having an imaging function by a camera. A mobile communication terminal 100 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram showing an external configuration of the mobile communication terminal 100.

  As illustrated, the mobile communication terminal 100 according to the present embodiment has a foldable configuration in which two movable housings (a housing 100A and a housing 100B) are connected by a hinge portion 100C. FIG. 1 shows a state where such a mobile communication terminal 100 is deployed. Here, FIG. 1A shows the appearance of the developed mobile communication terminal 100 on the main surface side. As shown in the drawing, a display unit 141, a speaker 151, an imaging lens 161, and the like are arranged on the same surface on the main surface side of the housing 100A. An operation unit 131 such as keys and buttons, a microphone 152, and the like are arranged on the main surface of the housing 100B.

  FIG.1 (b) has shown the structure of the outer surface side used as the other side of the main surface of the mobile communication terminal 100 shown to Fig.1 (a). As shown in the drawing, a display unit 142, an imaging lens 162, and the like are arranged on the same surface on the outer surface side of the housing 100A.

  An imaging lens 161 shown in FIG. 1A is a lens used for so-called “self-portrait” in which a photographer photographs himself / herself. That is, as shown in FIG. 2A, when the photographer holds the mobile communication terminal 100 so that the main surface side of the mobile communication terminal 100 faces itself, the imaging lens 161 faces the photographer. At this time, since the display unit 141 is also suitable for the photographer, by displaying the subject image captured by the imaging lens 161 on the display unit 141, the photographer displays the display unit 141 in which the photographer himself is displayed as the subject image. Can be taken.

  That is, when the mobile communication terminal 100 is deployed and used, self-portrait can be performed by performing shooting using the imaging lens 161. In the present embodiment, it is assumed that a “selfie mode” is prepared as an operation mode for the imaging function of the mobile communication terminal 100, and the self-portrait mode is selected when the mobile communication terminal 100 is deployed. In this case, imaging is performed using the display unit 141 and the imaging lens 161 as described above.

  Further, in the present embodiment, it is assumed that self-portrait can be performed using the display unit 142 and the imaging lens 162 configured on the outer surface side of the mobile communication terminal 100. In this case, the mobile communication terminal 100 is folded and used as shown in FIG. That is, when the self-portrait mode is selected when the mobile communication terminal 100 is folded, an imaging operation using the imaging lens 162 is performed, and a subject image is displayed on the display unit 142 to form a viewfinder. . Also in this case, it is assumed that the photographer holds the folded mobile communication terminal 100 in a posture as shown in FIG.

  In the present embodiment, the vertical direction of the paper surface shown in FIGS. 1 and 2B is the vertical direction of the mobile communication terminal 100 shown therein. In this case, the lens (imaging lens 161 and imaging lens 162) is positioned below the display screen (display unit 141 and display unit 142) with respect to the positional relationship between the display screen and the lens when taking a selfie. . Further, taking a photograph of the photographer's face by the photographing method as shown in FIG. 2A is defined as “selfie”.

  Next, the internal configuration of the mobile communication terminal 100 will be described with reference to FIG. FIG. 3 is a block diagram showing a configuration of mobile communication terminal 100.

  As illustrated, the mobile communication terminal 100 includes a control unit 110, a communication unit 120, an input control unit 130, a display control unit 140, an audio processing unit 150, a camera unit 160, an open / close detection unit 170, an attitude detection unit 180, and a storage. Part 190, and the like.

  The control unit 110 includes, for example, a CPU (Central Processing Unit) and a predetermined storage device (RAM (Random Access Memory)) serving as a work area, and controls each unit of the mobile communication terminal 100. At the same time, each process described later is executed based on a predetermined operation program stored in the storage unit 190. In addition, each structure of the mobile communication terminal 100 mentioned later is each connected with the control part 110, and data transmission / reception between each structure shall be performed via the control part 110. FIG.

  The communication unit 120 includes, for example, a predetermined wireless communication circuit and performs wireless communication with a base station via a predetermined antenna unit 123. The communication unit 120 further includes a voice communication unit 121, a data communication unit 122, and the like.

  The voice communication unit 121 controls communication related to the telephone function (voice call function) of the mobile communication terminal 100. Here, for example, the antenna unit 123 is controlled to perform wireless communication for a voice call through a telephone line network such as a mobile communication network, and perform voice encoding processing, decoding processing, and the like.

  The data communication unit 122 controls communication related to the data communication function of the mobile communication terminal 100 and performs, for example, access to a website and transmission / reception of e-mails.

  The input control unit 130 is a switch circuit connected to an operation unit 131 including various buttons and key groups as shown in FIG. 1A (for example, character keys (ten keys), function buttons, cross keys, etc.). Etc., and an input signal corresponding to the operation of the operation unit 131 is generated and input to the control unit 110. In this case, the input control unit 130 assigns a function to each key of the operation unit 131 according to various modes of the mobile communication terminal 100. Further, the buttons and keys constituting the operation unit 131 are arranged not only on the main surface of the housing 100B as shown in FIG. 1A but also on the side surface of the housing, for example. . In the present embodiment, it is assumed that a shutter button used for the imaging function is disposed on the side surface of the housing 100B.

  The display control unit 140 includes, for example, a display unit 141 configured by a liquid crystal display device and the like, a display circuit connected to the display unit 142, a video memory, and the like, and the display unit 141 based on a control signal from the control unit 110. In addition, various images are displayed on the display unit 141 and the display unit 142 by controlling the display unit 142.

  Here, the display unit 141 configured on the main surface of the mobile communication terminal 100 functions as a main display of the mobile communication terminal 100, and is used as a viewfinder at the time of the above-described selfie. In normal shooting other than the self-portrait mode, it functions as a viewfinder that displays a captured image using the imaging lens 162. In addition, the display unit 141 performs screen display related to most of the functions of the mobile communication terminal 100 such as various menu screens and operation screens.

  On the other hand, the display unit 142 configured on the outer surface of the mobile communication terminal 100 functions as a sub display of the mobile communication terminal 100. That is, the mobile communication terminal 100 is displayed when it is folded when it is not being used. For example, it is normally used for clock display, e-mail or incoming call notification. In the present embodiment, when the imaging function is operated at the time of folding, it is made to function as a self-viewing viewfinder as described above.

  Thus, since the display unit 141 is used as a main display and the display unit 142 is used as a sub-display, a relatively large display device is used for the display unit 141, and a relatively small display is used for the display unit 142. The device will be used. As described above, since any display device is used as a viewfinder at the time of self-portrait, a high-definition color display device is desirable.

  The audio processing unit 150 includes, for example, an analog signal circuit such as an audio codec circuit or an amplifier, converts the digital audio data received by the audio communication unit 121 into an analog audio signal, and outputs the analog audio signal from the speaker 151. The voice input to the microphone 152 is converted into digital data and sent to the voice communication unit 121. Here, when the moving image imaging function is included in the imaging function of the mobile communication terminal 100, the audio processing unit 150 records and reproduces the audio.

  The camera unit 160 is configured to realize the imaging function of the mobile communication terminal 100. For example, the imaging unit such as a CCD (Charge Coupled Device) or an ADC (Analog-Digital Converter) is used. The incident image to the imaging lens 161 and the imaging lens 162 is photoelectrically converted to generate a captured image. In the present embodiment, since the imaging lens 161 and the imaging lens 162 are configured in the mobile communication terminal 100, the camera unit 160 has an imaging device corresponding to each lens.

  Here, the imaging lens 161 configured on the main surface of the housing 100A is a lens dedicated to self-taking because of its arrangement. Here, since the large display unit 141, the speaker 151, and the like are disposed on the main surface of the housing 100A, a relatively small lens is used as the imaging lens 161. However, since the shooting distance is always close and constant if it is dedicated to self-photographing, a necessary and sufficient captured image can be obtained even with a small lens (for example, a fixed focus lens) that does not have high performance. On the other hand, the imaging lens 162 configured on the outer surface of the housing 100A is also used for imaging other than self-portrait. Therefore, the imaging lens 162 may be provided with, for example, an autofocus function or a zoom function. In this case, if the opening / closing of the mobile communication terminal 100 is selected when taking a selfie, a lens having a different performance can be selected and used, so that a selfie can be taken according to the desired image quality.

  The open / close detection unit 170 includes, for example, an angle sensor, and detects the open / close state of the housing by detecting the rotation angle of the hinge unit 100C, for example.

  The posture detection unit 180 includes, for example, an acceleration sensor and detects the posture of the mobile communication terminal 100. In the present embodiment, the posture detection unit 180 detects the inclination of the mobile communication terminal 100. Note that the inclination detected by the posture detection unit 180 is used for correction of a captured image by the camera unit 160 (details will be described later). Therefore, the posture detection unit 180 detects the inclination of the housing 100A in which the imaging lens 161 (162) is configured in the mobile communication terminal 100. Therefore, in this embodiment, it is assumed that the posture detection unit 180 is configured in the housing 100A.

  In the present embodiment, an acceleration sensor is used for the posture detection unit 180. The acceleration sensor in this case is, for example, a piezoelectric acceleration sensor having a configuration in which a piezoelectric element is attached to a diaphragm to which a weight body is attached, and detects an acceleration generated when the mobile communication terminal 100 moves. . More specifically, when the acceleration generated in the mobile communication terminal 100 acts on the weight body, the diaphragm and the piezoelectric element are distorted, and the piezoelectric effect of the piezoelectric element is generated. The generated acceleration is detected by detecting a voltage change due to the piezoelectric effect.

  In addition, the acceleration sensor used in the present embodiment is a three-axis acceleration sensor that detects acceleration generated in three directions of the X direction, the Y direction, and the Z direction. In the present embodiment, since the inclination of the housing 100A is detected, the posture detection unit 180 detects acceleration in three directions in the housing 100A. The X direction in this case is the left-right direction of the housing 100A in the direction shown in FIG. 1 (a) or 2 (b). Similarly, the Y direction is the vertical direction of the housing 100A in the direction shown in FIG. 1 (a) or 2 (b). The Z direction is the depth direction of the casing 100A orthogonal to the X direction and the Y direction.

  The storage unit 190 includes, for example, a predetermined storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory (including a removable memory card), and the like, and an operation executed by the control unit 110 In addition to the program, data necessary for executing each process, data generated by executing each process, and the like are stored. In the present embodiment, as shown in FIG. 3, storage areas such as an image development area 191, a captured image storage area 192, and a program storage area 193 are prepared in the storage unit 190.

  For example, the image development area 191 is a storage area prepared in a RAM or the like, and is an area for developing image data of a subject image acquired during an imaging operation by the camera unit 160. In the present embodiment, as described above, since the display unit 141 (142) is used as a viewfinder, image data is always generated when the camera unit 160 operates. Therefore, the image is displayed on the display unit 141 (142) while developing the image in the image development area 191 as needed. Also, when performing image correction processing to be described later, processing is performed by developing target image data in the image development area 191.

  The captured image storage area 192 is a storage area prepared in, for example, a flash memory, and is an area for storing image data obtained by imaging by the camera unit 160.

  The program storage area 193 is a storage area prepared in, for example, a ROM or a flash memory, and is an area for storing an operation program executed by the control unit 110. The operation program stored in the program storage area 193 stores an operation program for realizing each process to be described later, in addition to an arbitrary basic program that controls the basic operation of the mobile communication terminal 100. Processing performed by the mobile communication terminal 100 to be described later is realized by the control unit 110 executing these operation programs.

  That is, by executing the operation program stored in the storage unit 190, the control unit 110 functions as a configuration as shown in FIG. As illustrated, the control unit 110 functions as an imaging processing unit 111, a mode selection unit 112, an inclination calculation unit 113, an image processing unit 114, and the like.

  The imaging processing unit 111 cooperates with the input control unit 130, the display control unit 140, the camera unit 160, the open / close detection unit 170, the storage unit 190, and the like to execute processing related to the imaging function of the mobile communication terminal 100. Here, in addition to instructing the display control unit 140 to display a screen necessary for the imaging operation, the imaging function is realized by instructing the camera unit 160 to perform an imaging operation corresponding to a shutter operation or the like. In the present embodiment, the display control unit 140 is instructed whether to use the display unit 141 or the display unit 142, and the camera unit 160 determines which of the imaging lens 161 or the imaging lens 162 is used. Instruct. At this time, the imaging processing unit 111 refers to the imaging parameters stored in the storage unit 190 and instructs the camera unit 160 to perform an operation according to the imaging conditions at that time.

  The mode selection unit 112 cooperates with the input control unit 130, the display control unit 140, and the like to perform an operation mode (imaging mode) selection process related to the imaging function of the mobile communication terminal 100. Here, the user (photographer) of the mobile communication terminal 100 selects a shooting mode by a predetermined screen display, and instructs the imaging processing unit 111 to perform an operation related to the selected shooting mode. In the present embodiment, either the display unit 141 or the display unit 142 is used by detecting the open / closed state of the mobile communication terminal 100 in the self-portrait mode in cooperation with the open / close detection unit 170, and It is determined whether the imaging element corresponding to either the imaging lens 161 or the imaging lens 162 is used.

  The inclination calculation unit 113 calculates the inclination of the mobile communication terminal 100 in the self-portrait mode in cooperation with the posture detection unit 180 and the like. In the present embodiment, the vertical inclination of the mobile communication terminal 100 with respect to the photographer is calculated based on the detection value of the posture detection unit 180. More specifically, the inclination calculation unit 113 calculates the inclination of the casing 100A in the vertical direction with respect to the photographer.

  The image processing unit 114 performs image processing on the captured image obtained by the camera unit 160 in cooperation with the input control unit 130, the display control unit 140, the storage unit 190, and the like. Here, image data of an image captured by the camera unit 160 is acquired from the imaging processing unit 111 as needed, and is expanded in the image expansion area 191 of the storage unit 190 for image processing. Particularly in the present embodiment, the image processing is performed in the self-portrait mode. Here, processing for changing the display direction of the subject image displayed at the time of imaging is performed, and processing for correcting distortion of an image captured in the self-portrait mode using the tilt calculated by the tilt calculation unit 113 (details will be described later). ). The image processing unit 114 also causes the display unit 141 (142) to function as a viewfinder by outputting the processed image data to the display control unit 140. The image processing unit 114 performs image processing peculiar to self-shooting as described above, and also performs normal image processing for other shooting modes according to the selected shooting mode.

  In the present embodiment, each functional configuration described above is logically realized by the control unit 110. For example, these functions are configured using a dedicated circuit such as an ASIC (Application Specific Integrated Circuit). These functions may be realized by a physical configuration. In particular, the image processing unit 114 is configured by a dedicated circuit such as a so-called image processing engine, so that higher-speed processing can be realized.

  Each configuration of the mobile communication terminal 100 described above is a configuration necessary for realizing the present invention, and other configurations necessary for basic functions as the mobile communication terminal and configurations necessary for various additional functions. Etc. shall be provided as necessary.

  The operation of the mobile communication terminal 100 having the above configuration will be described below. First, a self-portrait process executed when taking a self-portrait with the image capturing function (here, still image capturing) of the mobile communication terminal 100 will be described with reference to a flowchart shown in FIG. This self-portrait process is started when the user of the mobile communication terminal 100 (photographer) operates a predetermined key or the like of the operation unit 131 to activate the imaging function of the mobile communication terminal 100. Shall. That is, when the input signal instructing activation of the imaging function is input from the input control unit 130 to the control unit 110, the imaging processing unit 111 is activated and starts processing.

  When the process is started, the imaging processing unit 111 acquires a detection result indicating the open / closed state of the mobile communication terminal 100 from the open / close detection unit 170 and notifies the mode selection unit 112 of the detection result. The mode selection unit 112 controls the display control unit 140 in response to the notification from the imaging processing unit 111, and displays a mode selection screen for allowing the user to select a shooting mode (step S101). At this time, based on the information indicating the open / closed state notified from the imaging processing unit 111, the mobile communication terminal 100 is displayed on the display unit 141 when it is expanded, and the mobile communication terminal 100 is displayed when it is folded. Displayed on the unit 142.

  The user (photographer) operates the operation unit 131 and selects a desired shooting mode from the displayed mode selection screen. In this case, an input signal indicating which shooting mode is selected is input from the input control unit 130 to the control unit 110. The mode selection unit 112 determines whether the self-portrait mode is selected based on the input signal from the input control unit 130 (step S102). Here, when a shooting mode other than the self-portrait mode is selected (step S102: No), this process ends. In this case, processing according to the selected shooting mode is executed.

  On the other hand, when the self-portrait mode is selected (step S102: Yes), the mode selection unit 112 notifies the imaging processing unit 111 to that effect. In response to the notification from the mode selection unit 112, the imaging processing unit 111 controls the display control unit 140 and the camera unit 160 to start an imaging operation in the self-portrait mode (step S103). Here, based on the information indicating the open / closed state acquired from the open / close detection unit 170, if the mobile communication terminal 100 is deployed, control is performed so that the image sensor corresponding to the display unit 141 and the imaging lens 161 is used. If it is folded, control is performed so that the image sensor corresponding to the display unit 142 and the imaging lens 162 is used. The imaging processing unit 111 also notifies the inclination calculation unit 113 and the image processing unit 114 that the self-portrait mode has been selected.

  In response to the notification from the imaging processing unit 111, the inclination calculating unit 113 executes an inclination calculating process for calculating the vertical inclination of the housing 100A (step S200). Details of the inclination calculation processing will be described later. The vertical inclination angle of the casing 100A calculated here is assumed to be θ.

  When the inclination calculation unit 113 calculates the inclination of the housing 100A by the inclination calculation process, the inclination calculation unit 113 notifies the imaging processing unit 111 of the fact and notifies the image processing unit 114 of the calculated inclination angle θ. The imaging processing unit 111 acquires captured image data from the camera unit 160 in response to the notification from the inclination calculating unit 113 (step S104), and sequentially supplies the captured image data to the image processing unit 114. At this time, the imaging processing unit 111 supplies the image data while notifying the image processing unit 114 that the image data is for through-displaying the captured image as a viewfinder.

  The image processing unit 114 sequentially develops the supplied image data in the image development area 191 (memory) (step S105). Here, since the obtained images are continuously displayed on the display unit 141 (142) to perform through display for the viewfinder, it is necessary to continuously process the image data. Therefore, the image processing unit 114 develops and processes the image data for through display at a low resolution in order to reduce the processing load.

  The image processing unit 114 determines whether image correction is necessary based on the tilt angle θ notified from the tilt calculation unit 113 (step S106). Here, the relationship between the inclination of the housing 100A, which is a criterion for determination, and the captured image will be described below.

  In self-portrait photography performed in a posture as shown in FIG. 2A, the subject to be imaged is the face of the photographer. FIG. 10A shows the positional relationship between the imaging target and the housing 100A in this case. In FIG. 10A, a horizontal line on the imaging target is H, and a vertical line orthogonal to the horizontal line H is V. In addition, the vertical axis of the housing 100A is Va, and the imaging direction of the imaging lens 161 (162) configured in the housing 100A is Da. Here, the vertical axis Va of the casing 100A is a central axis parallel to the long side direction of the casing 100A.

  FIG. 10B is a diagram showing the extracted lines defined as described above. As shown in the figure, the horizontal line H and the vertical line V are orthogonal to each other, and the vertical axis Va of the housing 100A and the imaging direction Da are orthogonal to each other. Here, the angle between the horizontal line H and the imaging direction Da, that is, the angle of the imaging direction with respect to the imaging target is θ1, and the angle between the vertical line V and the vertical axis Va of the casing 100A, that is, the vertical of the casing 100A. When the inclination of the direction is θ2, since the horizontal line H and the vertical line V are orthogonal to each other, and the vertical axis Va and the imaging direction Da are orthogonal to each other, θ1 and θ2 are equiangular (θ1 = θ2). That is, the angle (θ1) of the imaging direction with respect to the imaging target can be obtained if the vertical inclination (θ2) of the housing 100A is known. In the present embodiment, the imaging direction with respect to the imaging target is obtained by calculating the vertical inclination of the casing 100 </ b> A by the acceleration detection of the posture detection unit 180. Hereinafter, the vertical inclination of the housing 100A is denoted by θ.

  Here, if the imaging direction with respect to the imaging target is horizontal, the captured image is not distorted, but if the imaging direction is angled, the captured image is distorted (details will be described later). Therefore, it is possible to determine whether or not distortion occurs in the captured image based on the vertical inclination θ of the casing 100A. Further, the generated distortion differs depending on the direction of the vertical inclination θ of the housing 100A (details will be described later). In the present embodiment, in order to specify the distortion method based on such an inclination θ, a reference for the inclination θ is defined.

First, as shown in FIG. 10C, it is assumed that there is a casing 100A in front of the imaging target and no distortion occurs in the captured image. In this case, the angle (= θ) formed by the vertical axis Va and the vertical line V of the housing 100A with the imaging lens 161 (162) is 0 °. The inclination of the casing 100A that does not cause distortion in the captured image is set as a reference inclination θ ₀ . The angle of the reference inclination θ ₀ is defined as 0 °.

Here, in the case of the photographing posture as shown in FIG. 2A, the position of the housing 100A is below the object to be imaged. In this case, to image the imaging target, the casing 100A is tilted in the vertical direction as shown in FIG. Thus, the inclination θ when the imaging direction is from the bottom to the top is the right direction with respect to the reference inclination θ ₀ in FIG. 10D. That is, the upper end of the housing 100A is moved away from the imaging target, and the lower end is inclined in a direction approaching the imaging target. In the present embodiment, the inclination θ that is in such a direction with respect to the reference inclination θ ₀ is a positive inclination (+ θ).

On the other hand, for example, when shooting is performed with the mobile communication terminal 100 above the shooting posture shown in FIG. 2A, the position of the housing 100A is closer to the imaging target as shown in FIG. It becomes upper. Also in this case, the casing 100A is tilted in the vertical direction to capture an image, and the imaging direction at this time is from top to bottom. In this case, the inclination θ is leftward from the reference inclination θ ₀ in FIG. That is, the upper end of the housing 100A approaches the imaging target, and the lower end is inclined in a direction away from the imaging target. In the present embodiment, the inclination in such a direction with respect to the reference inclination θ ₀ is a negative inclination (−θ).

The relationship between the inclination θ of the housing 100A defined as described above and the distortion of the captured image will be described. FIG. 11A is an example of a captured image obtained when an image is captured in front of the imaging target as illustrated in FIG. In this way, when the image is taken in front, the subject image is not distorted. That is, when the inclination θ of the housing 100A is 0 ° (that is, the reference inclination θ ₀ ), a captured image without distortion is obtained. Hereinafter, an image without such distortion is referred to as a reference image IM ₀ . It is assumed that the reference image IM ₀ is not only when θ = 0 ° but also has a certain allowable range (reference inclination θ ₀ ± predetermined angle). This allowable range is hereinafter referred to as an allowable range α.

FIG. 11B shows an example of a captured image when the inclination θ is positive as shown in FIG. 10D with respect to the reference image IM ₀ as shown in FIG. When the housing 100A is tilted by + θ, the imaging direction is from the bottom to the top, so that the captured image is distorted in a trapezoidal shape as shown by a broken line (trapezoidal distortion) as illustrated. In other words, each oblique side is distorted as if the image was deformed into a trapezoidal shape with an upper base <lower base contacting the left and right edges of the image when y = 0. Hereinafter, an image in which the trapezoidal distortion is generated by tilting the housing 100A by + θ is referred to as an image IM ₍₊₎ .

Further, a captured image when the inclination θ is negative as shown in FIG. 10E is as shown in FIG. In this case, since the imaging direction is from top to bottom, the image is distorted in a trapezoidal shape (upper base> lower base) that is opposite to that of the image IM ₍₊₎ . Such an image distorted by tilting the housing 100A by −θ is hereinafter referred to as an image IM ₍₋₎ .

As described above, based on the inclination θ of the housing 100A calculated by the inclination calculation process, the image processing unit 114 determines whether or not image correction is necessary in step S106 (FIG. 5). Here, the necessity of image correction is determined by determining whether or not the calculated inclination θ is within an allowable range α in which distortion does not occur (that is, the reference inclination θ ₀ ± predetermined angle). That is, if the inclination θ is within the range, it is determined that image correction is unnecessary, and if it is outside the range, it is determined that image correction is necessary.

  If it is determined that the inclination θ is within the range and no image correction is necessary (step S106: No), the process proceeds to step S107. On the other hand, when it is determined that image correction is necessary (step S106: Yes), an image correction process for correcting distortion of the acquired image data is executed (step S300). Here, image distortion (trapezoidal distortion) that is likely to occur during self-shooting as shown in FIG. 11B or FIG. 11C is corrected. Details of this image correction processing will be described later.

  The image processing unit 114 supplies the captured image data expanded in the image expansion area 191 in step S105 or the image data corrected by the image correction processing to the display control unit 140, thereby displaying the viewfinder on the display unit 141 (142). (Step S107). At this time, the image processing unit 114 displays an image so that the captured subject image is displayed upright on the display unit 141 illustrated in FIG. 1A or the display unit 142 illustrated in FIG. In addition to defining the vertical direction of the image, the image is reversed horizontally so that the photographer can easily determine the composition. By reversing the image from side to side, the subject image that reflects your appearance on the mirror is displayed in the viewfinder, so you can adjust the composition and angle without any discomfort. Here, when the lens being used has an autofocus function and a zoom function, a focus operation and a zooming operation are performed according to the operation of the operation unit 131 by the user. In this case, the imaging processing unit 111 instructs the camera unit 160 to perform an operation according to the input signal from the input control unit 130.

  A view in which captured images are continuously displayed on the display unit 141 or the display unit 142 by repeatedly executing the operations in steps S104 to S107 and step S300 until the shutter operation is performed (step S108: No). Viewfinder display (through display) is performed.

  The user (photographer) operates the operation unit 131 that functions as a shutter button when a desired photo opportunity is reached. In this case, an input signal indicating a shutter operation is input from the input control unit 130 to the control unit 110. If it is determined that the shutter operation has been performed based on the input signal from the input control unit 130 (step S108: Yes), the imaging processing unit 111 instructs the camera unit 160 to perform a shutter operation.

  The camera unit 160 performs a predetermined shutter operation in accordance with an instruction from the imaging processing unit 111, and operates the imaging lens 161 (162) or a corresponding imaging element with parameters according to the imaging conditions at that time to perform imaging. . When the image processing unit 111 acquires image data (captured image data) generated by the image capturing (step S109), the image data is supplied to the image processing unit 114. At this time, the imaging processing unit 111 notifies the image processing unit 114 that the supplied image data is a captured image obtained by a shutter operation.

  The image processing unit 114 develops the image data supplied from the imaging processing unit 111 in the image development area 191 (memory) (step S110). Here, unlike the through display described above, since it is a still image captured by a shutter operation, continuous processing is not required. Since the captured image is based on the photographer's intention, the image processing unit 114 expands the image data in the image expansion area 191 with high resolution so that the image quality can be confirmed.

  Also in this case, the image processing unit 114 determines whether or not image correction is necessary by the same processing as in step S106 (step S111). If image correction is necessary (step S111: Yes), the above-described image correction process (details will be described later) is executed again (step S300), and the captured image developed in the image development area 191 is corrected. On the other hand, if unnecessary (step S111: No), the process proceeds to step S112.

  The image processing unit 114 outputs the image data corrected by the image correction processing to the display control unit 140, thereby displaying a captured image obtained by shooting by the user (photographer) on the display unit 141 or the display unit 142. (Step S112). The image processing unit 114 cooperates with the input control unit 130 and the display control unit 140 to send a message for inquiring whether to save the displayed captured image, a button image for instructing whether to save the image, or the like. Is displayed on the display screen in step S112. The user (photographer) checks the displayed captured image, determines whether or not the image should be saved, and operates the operation unit 131 to input any instruction.

  When an input signal corresponding to the operation of the operation unit 131 is input from the input control unit 130 to the control unit 110, the image processing unit 114 determines whether or not an instruction to save the captured image is input based on the input signal. It discriminate | determines (step S113). When the save instruction is input (step S113: Yes), the image processing unit 114 expands the captured image data that has been expanded in the image expansion area 191 in step S110 and subjected to the image correction processing in step S300 in the captured image storage area 192. Store and save (step S114). On the other hand, if the input instruction is not a save instruction (step S111: No), the image processing unit 114 does not store the captured image in the captured image storage area 192, but clears the captured image expanded in the image expansion area 191. .

  Then, for example, the above-described steps S103 to S114, step S200, and step S300 are repeated until an instruction to end the self-shooting, such as canceling the self-shooting mode or ending the imaging function, is input (step S115: No). Execute. Here, what is performed from the tilt calculation process in step S200 is that the posture of the user (photographer) changes for confirmation of the captured image displayed in step S112, and the tilt of the housing 100A is different from the previous time at the next shooting. This is because they may be different.

  When the user (photographer) operates the operation unit 131 to input an end instruction such as cancellation of the self-portrait mode or the end of the imaging function (step S115: Yes), the control unit 110 ends the process.

  Next, the inclination detection process executed in the above-described self-portrait process will be described with reference to the flowchart shown in FIG.

  When the process is started, the inclination calculating unit 113 acquires the output of the posture detecting unit 180, that is, the acceleration sensor (step S201). As described above, since the posture detection unit 180 is a three-axis acceleration sensor, the posture detection unit 180 detects accelerations in the X direction, the Y direction, and the Z direction, and outputs the acceleration vectors (Ax, Ay, Az). To do. Here, Ax represents an acceleration vector in the X direction, Ay represents an acceleration vector in the Y direction, and Az represents an acceleration vector in the Z direction.

  When each acceleration vector is acquired, the inclination calculation unit 113 counts up using a timer or the like that is realized by the time measuring function of the control unit 110 or the like. At this time, the inclination calculating unit 113 starts counting up after resetting the count value of the timer (step S202).

When the count-up is started, the inclination calculating unit 113 calculates the magnitude of acceleration A ₀ (= √ (Ax ² + Ay ² + Az ² )) from the acceleration vector (Ax, Ay, Az) acquired in step S201 ( Step S203). Then, the calculated acceleration magnitude A ₀ is compared with the gravitational acceleration magnitude g, and whether the difference between the acceleration magnitude A ₀ and the gravitational acceleration magnitude g is within a predetermined error β. It is determined whether or not (step S204).

If the difference between the magnitude A ₀ and gravitational acceleration of magnitude g of the acceleration exceeds the predetermined range of error beta (step S204: No), the slope calculation unit 113, the mobile communication terminal 100 is stationary After determining that the timer has not counted and waiting until the count value of the timer started in step S202 reaches a predetermined period m (step S205), the operations in steps S201 to S204 are executed again.

On the other hand, the difference between the acceleration magnitude A ₀ and gravitational acceleration of magnitude g is, if within a predetermined error range beta (step S204: Yes), the slope calculation unit 113, the mobile communication terminal 100 is still Judge that you are doing. In this case, the inclination calculation unit 113 calculates the vertical inclination angle θ of the mobile communication terminal 100 with respect to the user (photographer) using the acceleration vector in the Z direction at that time (step S206), and FIG. Return to the self-shooting process flow shown in FIG.

Here, calculation of the inclination θ will be described. The inclination of the housing 100A as shown in FIG. 10D or FIG. 10E is caused by the movement of the housing 100A in the Z direction. Therefore, in step S206, the inclination angle θ is calculated using the acceleration vector in the Z direction at that time. In this case, the inclination calculation unit 113 calculates the inclination angle θ by calculating θ = sin ⁻¹ Az ² .

  Thus, when imaging is performed with the housing 100A tilted, trapezoidal distortion as shown in FIG. 11B or FIG. 11C occurs. Therefore, in the present embodiment, the trapezoidal distortion generated in the self-portrait image captured in the self-portrait process is corrected by the image processing unit 114 executing the image correction process (step S300) (trapezoid correction). This image correction processing will be described with reference to the flowchart shown in FIG.

  As shown in the figure, the image correction process includes a horizontal direction correction process (step S400) for correcting the horizontal direction (X direction) of the image and a vertical direction correction process (step for correcting the vertical direction (Y direction) of the image. This is realized by performing S500). Each process will be described below.

In the process described below, image correction is performed based on the coordinates of the captured image. Therefore, in this embodiment, coordinate axes (X axis and Y axis) as shown in FIG. 12 are set in the captured image. The X axis is an axis parallel to the short side of the captured image, and the Y axis is an axis parallel to the long side of the captured image. The X axis and the Y axis intersect at the center position of the captured image. Let this intersection be the origin, and its coordinates be (0, 0). Here, the maximum value of the X coordinate on the captured image is x _max , the minimum value is x _min , the maximum value of the Y coordinate is y _max , and the minimum value is y _min . In this case, since the origin coordinates are (0, 0), x _max and y _max are positive coordinate values, and x _min and y _min are negative coordinate values. Since the origin is the center of the image, the absolute values of x _min and x _{max and} the absolute values of y _min and y _max are the same values (| x _min | = | x _max |, | y _min | = | Y _max |).

For example, if the resolution of the captured image is QVGA, the number of pixels in the long side direction is 320, and thus the number of pixels in the Y direction of the captured image according to the present embodiment is 320. In this case, the range of the coordinate value in the Y direction of the captured image is −160 (y _min ) to +160 (y _max ).

The lateral direction correction processing performed with such coordinate settings will be described with reference to the flowchart shown in FIG. Here, as shown in FIG. 11B, the image IM ₍₊₎ distorted so that the image is gradually reduced or enlarged in the horizontal direction as the position in the vertical direction (Y direction) of the image moves. The case where the image correction is performed will be described below.

In such an image IM ₍₊₎ , the image is distorted into a trapezoidal shape with the upper base <the lower base. This means that the image is distorted in the horizontal direction (X direction) with a different degree of distortion for each Y coordinate of the captured image. Therefore, in the horizontal direction correction processing, correction is performed by obtaining the degree of distortion for each Y coordinate of the captured image.

Here, it is assumed that distortion is corrected for each Y coordinate from the upper end to the lower end of the captured image. Therefore, the image processing unit 114 first sets the maximum value (y _max ) that becomes the Y coordinate of the upper end portion of the image as the target Y coordinate (y) (step S401). In the case of QVGA, +160 is set.

  The image processing unit 114 calculates the distortion degree Vx in the X direction at the Y coordinate (y-th line) set in step S401 (step S402). Here, the distortion degree Vx in the X direction is calculated by the following equation (1).

(Equation 1)
Vx = a · Ry · cos θ + b (a and b are constants)

  In the above equation 1, Ry indicates the distance in the Y direction from the center of the imaging screen to the target Y coordinate, as a coordinate value. As described above, since the center coordinates of the screen are (0, 0), Ry = y-0 is obtained. In the trapezoidal distortion as described above, the degree of distortion increases as the distance from the center of the image increases in the Y direction, and the degree of distortion increases as the tilt angle θ of the housing 100A increases. Therefore, in Equation 1, Ry that is the distance in the Y direction from the center position of the image and the tilt angle θ of the housing 100A are used as elements.

  Here, the constants a and b are numerical values determined under photographing conditions such as self-portrait, for example. For example, it is determined based on a shooting distance at the time of self-taking defined based on a standard arm length or the like, a range of a shooting direction assumed in the case of self-taking. Such constants may be used by setting default values in advance, or when taking a selfie (for example, at the first execution), the constants can be changed to display a corrected image and selected by the user. A value used in the corrected image may be set. The constants a and b are positive values.

In the case of the image IM ₍₊₎ , the inclination θ of the housing 100A is positive (+ θ). Further, since each constant is positive, the positive / negative sign of Vx obtained by Equation 1 corresponds to the positive / negative sign of Ry. Ry is positive when the target Y coordinate is y ≧ 0, and Ry is negative when the Y coordinate is y <0. In the image IM ₍₊₎ , with y = 0 as the boundary, the image is reduced in the X direction as it goes in the + direction of the Y axis, and the image is enlarged in the X direction as it goes in the − direction of the Y axis. Therefore, if the obtained Vx is negative, it indicates an enlargement ratio (degree of enlargement distortion), and if it is positive, it indicates a reduction ratio (degree of reduction distortion).

  The image processing unit 114 corrects the X direction at each Y coordinate based on the distortion degree Vx thus calculated. That is, if Vx indicates an enlargement rate, the image is reduced in the X direction by the degree (that is, the absolute value of Vx), and if Vx indicates the reduction rate, the degree (that is, Vx By magnifying in the X direction by (absolute value), distortion in the X direction at the Y coordinate is corrected. That is, the distortion in the X direction is corrected by scaling in the X direction by the absolute value (| Vx |) of the distortion degree Vx obtained by the Y coordinate (step S403).

When the X direction correction at the Y coordinate set in step S401 is completed, the image processing unit 114 decrements the coordinate value y of the Y coordinate by 1 (step S404). If this value is equal to or greater than the minimum value y _min (step S405: No), the correction in the X direction is performed for all the Y coordinates of the captured image by repeatedly executing steps S402 to S403. Therefore, when the coordinate value y decremented by 1 in step S404 becomes less than the minimum value y _min (step S405: Yes), the target Y coordinate does not exist, and the process returns to the image correction processing flow shown in FIG. In the present embodiment, the Y coordinate is specified and corrected one by one. However, for example, the Y coordinate may be specified and corrected intermittently, or may be corrected in a plurality of Y coordinate units. The processing efficiency may be improved by reducing the number of designated coordinates.

  In the image correction process, a vertical direction correction process for correcting image distortion in the Y direction is subsequently executed. This vertical direction correction processing will be described with reference to the flowchart shown in FIG. In the horizontal direction correction process described above, correction is performed for each Y coordinate. However, in the vertical direction correction process described below, distortion in the Y direction is corrected in units of blocks in the Y direction as shown in FIG. This is because distortion in the Y direction cannot be detected from image data for one pixel in the Y direction, and distortion is detected and corrected in units of blocks having a height of a plurality of pixels in the Y direction.

Here, since the processing is carried out in block units with a height in the Y direction, as a coordinate value that indicates the Y position of each block, the Y coordinate of the block upper and coordinate values y _t, coordinate the Y coordinate of the block lower side the value _{y b.} In addition, the height of each block is the n coordinate in the Y direction. Note that the horizontal width of each block is the same as the horizontal width of the captured image. In addition, when the horizontal width of each block is represented by a coordinate range in the X direction, x _min (−x) to x _max (+ x). The block height n is a divisor of the number of pixels in the Y direction. That is, the height of the block in which the image is equally divided in the Y direction.

Also in the case of the vertical direction correction process, correction is sequentially performed from the upper end to the lower end direction of the captured image, similarly to the above-described horizontal direction correction process. Therefore, the image processing unit 114 first designates a block whose upper side is in contact with the upper end of the image. That is, the _{y t} being the upper side of the Y coordinate is the maximum value _{y max} of the Y coordinate (step S501).

Next, the image processing unit 114 designates the lower side position of the block. In other words, the height of the block is n coordinates fraction, y _b a Y-coordinate of the block lower side is a position lowered by n coordinate component from the specified y _t at step S501. That is, the image processing unit 114 sets the coordinate value obtained by y _t −n to y _b indicating the Y coordinate of the block lower side (step S502).

The image processing unit 114, by thus setting the _{y t} and _{y b,} specifies the block on the image (step S503). That is, a rectangular block in which the upper side Y coordinate is yt and the lower side Y coordinate is yb is designated. Here, the left and right ends of the upper side and the lower side are both the left and right ends of the image, that is, x _min and x _max .

In the case of trapezoidal distortion, the distortion in the Y direction increases as the distance from the center of the image in the Y direction increases, similar to the distortion in the X direction. Therefore, even when correcting the Y-direction distortion, Ry indicating the distance in the Y direction from the origin (0, 0) on the image is used. Here, since it is specified in units of blocks, the distance Ry for each block has a width. In the present embodiment, a distance between the Y coordinate corresponding to the center of the height of each block and the center of the image is defined as a distance Ry. In this case, since the Y coordinates of the upper and lower sides of the block are obtained, the Y coordinate (y _m ) that is the middle point between the coordinates y _t and the coordinates y _b is obtained, and the distance from the center of the image to y _m Is the distance Ry.

That is, the coordinate y _{m which} is the midpoint of the Y direction of the block is y _m = (y _t + y _b ) / 2, and in this case, Ry indicating the Y direction distance from the image center by the coordinate value is Ry = y _m −0. When the distance Ry for the designated block is obtained, the image processing unit 114 calculates the distortion degree Vy in the Y direction in the block (step S504). The distortion degree Vy in the Y direction can be obtained by calculating the following formula 2.

(Equation 2)
Vy = c · Ry · cos θ + d (where c and d are constants)

  Similar to the distortion in the X direction, the degree of distortion in the Y direction increases as the distance from the center of the image increases and the inclination of the housing 100A increases. Therefore, even in Equation 2 for obtaining the degree of distortion Vy in the Y direction, the distance Ry from the image center and the inclination θ of the housing 100A are used as elements. The constants c and d used in Equation 2 are also set to values defined under, for example, a photographing condition of self-taking, similar to the constants a and b used in Equation 1 described above. Also in this case, the constants c and d are assumed to be positive values.

  Like Vx in the lateral correction described above, if Vy is negative, it indicates an enlargement rate (degree of enlargement distortion), and if it is positive, it indicates a reduction rate (degree of reduction distortion). Therefore, the image processing unit 114 corrects the Y direction in the block based on the distortion degree Vy calculated in step S504. That is, if Vy indicates an enlargement rate, the image is reduced in the Y direction by that degree (that is, the absolute value of Vy), and if Vy indicates a reduction rate, that degree (that is, Vy By magnifying in the Y direction by (absolute value), distortion in the Y direction in the block is corrected. That is, the distortion in the Y direction is corrected by scaling the block in the Y direction with the absolute value (| Vy |) of the distortion degree Vy (step S505).

Here, the correction in the Y direction is performed on the image of the entire block. That is, an image of a rectangular area having (x _min , y _t ), (x _max , y _t ), (x _min , y _b ), and (x _max , y _b ) as vertices is corrected by scaling in the Y direction. To do.

In this way, when the block having the Y position designated in step S501 as the upper side is corrected, the image processing unit 114 designates the next block. In this case, the image processing unit 114, a coordinate value _{y b} indicating the lower position of the immediately preceding block -1, and _{y t} is the Y coordinate of the upper side of the next block (step S506). If this value is equal to or greater than the minimum value y _min (step S507: No), the correction in the Y direction is performed for all the blocks set on the captured image by repeatedly executing steps S502 to S505. On the other hand, when the coordinate value y _t of the new upper side Y coordinate is less than the minimum value y _min (y _t <y _min ) (step S507: Yes), the target Y coordinate does not exist, and therefore, as shown in FIG. Returning to the image correction process flow, the process returns to the self-portrait process flow shown in FIG.

  The height n of the block may not be a unique value. For example, in order to reduce the amount of calculation, the value of n may be variable, and the value of n may be increased as the distance from the center of the image increases in the Y direction, so that the number of block divisions as a whole may be further reduced.

In this way, by performing the correction in the horizontal direction and the correction in the vertical direction, the trapezoidal distortion image IM ₍₊₎ as shown in FIG. 11B becomes the reference image IM ₀ as shown in FIG. It is corrected so that

In the above description of the image correction processing, the correction of the image IM ₍₊₎ distorted into the upper base <the trapezoid of the lower base is exemplified, but the upper base as shown in FIG. The image IM ₍₋₎ can also be corrected by performing the image correction process described above. In this case, since the inclination of the housing 100A is −θ, in Equations 1 and 2, Vx and Vy are negative when Ry is positive, and Vx and Vy are positive when Ry is negative. In the case of the image IM ₍₋₎ , with y = 0 as a boundary, the image is distorted in the enlargement direction as it becomes the + direction on the Y axis, and the image is distorted in the reduction direction as it becomes the − direction. That is, Vx and Vy that are negative when the Y coordinate is positive indicate an enlargement ratio, and Vx and Vy that are positive when the Y coordinate is negative indicate a reduction ratio. Therefore, similarly to the correction of the image IM ₍₊₎ described above, when Vx and Vy are negative, they are reduced and scaled by their respective absolute values, and when Vx and Vy are positive, their respective absolute values are used. By enlarging and scaling, the image IM ₍₋₎ can be corrected to become the image IM ₀ .

  As described above, by applying the present invention as in the above embodiment, when taking a self-portrait with a mobile communication terminal having an imaging function, it occurs at the time of self-taking based on the inclination of the mobile communication terminal. A captured image in which easy image distortion is automatically corrected can be obtained. In this case, for the vertical trapezoidal distortion that occurs during self-shooting, the horizontal (x-direction) distortion is corrected in y-coordinate units, and the vertical (y-direction) distortion is corrected in block units. Can be done efficiently.

  Each said embodiment is an example and the application range of this invention is not restricted to this. That is, various applications are possible, and all the embodiments are included in the scope of the present invention.

  For example, in the above embodiment, the attitude detection by the acceleration sensor is performed to calculate the inclination of the mobile communication terminal 100 (housing 100A), but the attitude detection of the mobile communication terminal 100 (housing 100A) can be performed. If present, the sensor used is arbitrary. For example, the posture (tilt) of the mobile communication terminal 100 (housing 100A) may be detected by a geomagnetic sensor or the like.

  Further, in the above embodiment, since the lens used for the self-portrait is configured in the casing 100A, the image correction is performed based on the inclination of the casing 100A. That is, since it is only necessary to detect the inclination (posture) of the portion that affects the imaging direction, the arrangement, direction, number, and the like of the attitude detection unit 180 may be arbitrarily set according to the arrangement of the lenses. It is not limited to the arrangement shown in FIG.

  In the above embodiment, in a foldable mobile communication terminal using two housings, a case where a lens and a display device that displays a viewfinder image captured by the lens are configured on the same housing is illustrated. However, if the imaging direction of the lens and the display direction of the display device are the same or substantially the same, the arrangement of the lens (imaging lens 161 and imaging lens 162) and the display device (display unit 141 and display unit 142) is arbitrary. However, the present invention is not limited to the arrangement shown in the above embodiment.

  For example, when the casing 100A is configured to be biaxially rotatable by configuring the hinge portion 100C of the mobile communication terminal 100 shown in the above embodiment with a biaxial hinge, an imaging lens is formed on the outer surface of the casing 100B. The image captured by the lens may be displayed on the display unit 141 configured on the main surface of the housing 100A in a viewfinder display. In this case, the imaging direction of the imaging lens configured on the outer surface of the housing 100B and the display direction of the display unit 141 are changed by rotating the housing 100A by 180 ° around the vertical central axis by biaxial rotation by the biaxial hinge. Identical or nearly identical. In such a form, self-portrait as shown in the above-described embodiment is possible, so that the present invention can be applied to perform image correction during self-portrait.

  That is, as long as the imaging direction of the imaging lens and the display direction of the captured image by the imaging lens are the same or substantially the same, the form of the housing is arbitrary and is limited to the one shown in the above embodiment. is not. For example, it may be a straight type mobile communication terminal constituted by a single housing.

  In the above-described embodiment, an example in which the two imaging lenses 161 and 162 are configured is shown. However, the number and arrangement of the lenses are shown in the above-described embodiment if self-portrait is possible. It is not limited to the above, but is arbitrary. The same applies to the display unit 141 and the display unit 142, and the number and arrangement thereof are arbitrary.

  It should be noted that the present invention can be realized by a mobile communication terminal having the configuration according to the present invention in advance, such as the mobile communication terminal 100 of the above-described embodiment, as well as an existing mobile communication terminal having a plurality of application execution functions. The present invention can also be realized by applying a program.

  In this case, the computer (for example, CPU) which controls the existing mobile communication terminal applies the program which functions as each function structure implement | achieved by the control part 110 of the said embodiment, By existing mobile communication terminal Can function as the mobile communication terminal 100 of the above embodiment. That is, an operation program similar to the operation program executed by the control unit 110 of the above embodiment is installed in a storage device of an existing mobile communication terminal, and this is executed by the control device (CPU) of the mobile communication terminal. Thus, it can function as a mobile communication terminal according to the present invention.

  Such a program can be provided in any form. For example, the program can be provided by being distributed via a communication medium such as the Internet, and stored in a predetermined recording medium (for example, a memory card). Can be provided.

It is a figure which shows the external appearance structure of the mobile communication terminal concerning embodiment of this invention, (a) shows the main surface side at the time of expansion | deployment, (b) shows the outer surface side at the time of expansion | deployment. It is a figure for demonstrating the usage example of the mobile communication terminal shown in FIG. 1, (a) shows the imaging | photography attitude | position at the time of self-portrait, (b) is the mobile communication terminal at the time of folding used at the time of self-portrait. Appearance is shown. It is a block diagram which shows the structure of the mobile communication terminal shown in FIG. It is a functional block diagram which shows the function implement | achieved by the control part shown in FIG. It is a flowchart for demonstrating the "self-portrait process" concerning embodiment of this invention. It is a flowchart for demonstrating "the inclination calculation process" performed in the self-portrait process shown in FIG. 6 is a flowchart for explaining an “image correction process” executed in the self-portrait process shown in FIG. 5. It is a flowchart for demonstrating the "horizontal direction correction process" performed in the image correction process shown in FIG. It is a flowchart for demonstrating the "vertical direction correction process" performed in the image correction process shown in FIG. (A) And (b) is a figure for demonstrating the inclination of the mobile communication terminal at the time of a self-portrait, (c)-(e) is a figure for demonstrating the reference | standard regarding the inclination of a mobile communication terminal. It is. (A)-(c) is a figure which shows the example of the captured image according to the inclination of a mobile communication terminal. It is a figure for demonstrating the setting on the captured image in the image correction process shown in FIG. It is a figure for demonstrating the setting on the captured image in the vertical direction correction process shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Mobile communication terminal, 100A ... Housing | casing, 100B ... Housing | casing, 100C ... Hinge part, 110 ... Control part, 111 ... Imaging process part, 112 ... Mode selection part, 113 ... Inclination calculation part, 114 ... Image processing part , 120 ... communication section, 121 ... voice communication section, 122 ... data communication section, 123 ... antenna section, 130 ... input control section, 131 ... operation section, 140 ... display control section, 141 ... display section, 142 ... display section, DESCRIPTION OF SYMBOLS 150 ... Sound processing part, 151 ... Speaker, 152 ... Microphone, 160 ... Camera part, 161 ... Imaging lens, 162 ... Imaging lens, 170 ... Opening / closing detection part, 180 ... Attitude detection part, 190 ... Memory | storage part, 191 ... Image expansion | deployment Area, 192 ... captured image storage area, 193 ... program storage area

Claims (4)

In a mobile communication terminal having an imaging function,
Attitude detecting means for detecting the attitude of the mobile communication terminal;
An inclination calculating means for calculating an inclination of the mobile communication terminal based on the attitude of the mobile communication terminal detected by the attitude detection means when the self-portrait mode is selected by the imaging function;
Image correcting means for correcting distortion of an image captured in the self-portrait mode based on the inclination of the mobile communication terminal calculated by the inclination calculating means;
A mobile communication terminal comprising: The image correcting means corrects lateral distortion in the vertical coordinate unit of the image based on the distance from the center position of the image captured in the self-portrait mode and the inclination of the mobile communication terminal.
The mobile communication terminal according to claim 1. The image correction means sets a plurality of blocks in the vertical direction of the image captured in the self-portrait mode, and based on the distance from the center position of the image and the inclination of the mobile communication terminal for each block. Correct vertical distortion,
The mobile communication terminal according to claim 1 or 2. In a computer that controls a mobile communication terminal having an imaging function,
A function of selecting an operation mode according to the imaging function;
A function of calculating the inclination of the mobile communication terminal when the selfie mode is selected as the operation mode;
A function of correcting an image captured in the self-portrait mode based on the calculated inclination of the mobile communication terminal;
A program characterized by realizing.

Images