JP2014003575A - Portable terminal device, image capturing method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a portable terminal device, image capturing method, and computer program capable of preventing an image position from being not correctly determined due to the visual recognition of a deformed image when capturing the image of a subject existing beyond an obstacle positioned higher than a horizontal eye line of an image capturing person.SOLUTION: An acceleration sensor is used to recognize a case where a subject direction 304 is not coincident with an image confirmation angle 303 as shown in Fig. 4. When it is recognized that a subject direction 304 is not coincident with an image confirmation angle 303, a CPU deforms a capturing image according to the image confirmation angle 303 as shown in Fig. 8. Consequently, an image viewed from an image capturing person 302 becomes the one that correctly captures a subject as shown in Fig. 7, so that the image capturing person 302 can confirm the image as a vision viewed from a front. When a shutter is depressed, an image true to the shape of a subject is acquired.

Description

  The present invention relates to a portable terminal device, a photographing method, and a computer program. In particular, when shooting an object beyond the obstacle that is more than the height of the photographer's horizontal line of sight with the attached camera, it is possible to check whether the object is accurately captured in the center of the image frame. The present invention relates to a portable terminal device, a photographing method, and a computer program.

  In recent years, mobile terminal devices with cameras in which a camera is attached to a main body have become widespread for mobile terminal devices. However, from the standpoint of designing the device, such a portable terminal device with a camera has been integrated and miniaturized, and thus the device has a complicated structure that changes the shooting angle of the camera. It has become difficult to have In addition, when photographing a landscape or the like beyond an obstacle present at a position higher than the photographer's horizontal line of sight, the photographer moves the mobile terminal device upward and sets the camera direction to the subject. An operation to turn is required. However, when the photographer looks at the photographing screen after performing such an operation, the photographed image appears to be deformed into a trapezoid. Therefore, in such a case, there is a problem that it is difficult for the photographer to determine whether or not the target subject is captured at the center of the screen.

  As a known example technique in this field, for example, Patent Document 1 discloses a digital camera that compensates for perspective distortion by a processor. Specifically, when perspective distortion appears in the image because the subject was taken up from below, the perspective distortion was removed by enlarging the image with this distortion at a magnification that varies depending on the position by process processing. Try to get an image.

  For example, Patent Document 2 discloses an image display device that corrects screen shake due to vibration at the display position in the vertical and horizontal directions and the display magnification in the front and rear directions. Specifically, the image display device is provided with a camera and has an AF function for measuring the distance to the user's face. The distance data to the user's face measured by the AF mechanism and the amount of movement of the mobile terminal device detected by the built-in acceleration sensor up and down, left and right, and front and rear are transmitted to the CPU. The CPU calculates the enlargement / reduction amount of the display image for correcting the visual size variation of the display image due to the movement in the front-rear direction (far / near) detected by the acceleration sensor, using the distance to the face as an initial value. In addition, the shift direction and the shift amount of the display area of the screen for correcting the vertical and horizontal shake of the screen are calculated from the vertical and horizontal movement amounts of the mobile terminal device detected by the acceleration sensor. Yes.

Japanese Patent No. 4233797 JP 2004-128712 A

  A digital camera that compensates for perspective distortion with a processor, such as the apparatus described in Patent Document 1 described above, is an effective means for removing perspective distortion that appears in an image because a subject is looked up from below. However, such means is not necessarily limited to enlarging an image having the distortion by a process process with a magnification that varies depending on the position.

  In addition, the image display device that corrects the screen shake caused by vibration at the display position in the vertical and horizontal directions, and the display magnification at the front and rear, as in the device described in Patent Document 2 described above, corrects the screen shake caused by the vibration. It is an effective means for doing this. However, the distortion appearing on the display screen is not necessarily caused by vibration.

  By the way, as described above, as a result of progress in integration and miniaturization of recent mobile terminal devices, it is difficult to have a complicated structure that changes the shooting angle of the camera. Further, when photographing a subject (person, landscape, etc.) that exists beyond an obstacle higher than the photographer's horizontal line of sight, the mobile terminal device is moved upward and the camera is directed toward the subject. . Therefore, when the photographer looks at the photographing screen, there is a problem that the photographer appears to be deformed into a trapezoid. Of course, such a problem is not solved by the techniques disclosed in Patent Documents 1 and 2 described above.

  The objective of this invention is providing the portable terminal device, imaging | photography method, and computer program which solve the subject mentioned above.

  In order to solve the above-described problem, the portable terminal device according to the present invention includes a photographing unit that photographs a subject and converts it into photographed image data after the photographer issues a first command, and the first command. , An initialization unit that sets the position of the apparatus as a reference position at a timing at which is issued, a movement state recognition unit that recognizes a movement state of the apparatus from the reference position, and the captured image according to the movement state An image deforming unit that deforms data to obtain deformed image data, a display unit that displays the deformed image data on a display device, and an image that acquires the captured image data at a timing when a photographer issues a second command. Acquisition means.

  Further, the photographing method according to the present invention includes a photographing step of photographing a subject and converting it into photographed image data after the photographer issues a first command, and a timing when the first command is issued, An initialization step for setting the position of the apparatus as a reference position, a movement state recognition step for recognizing the movement state of the apparatus from the reference position, the captured image data is deformed according to the movement state, and a deformed image An image transformation step for obtaining data, a display step for displaying the transformation image data on a display device, and an image acquisition step for obtaining the photographed image data at a timing when a photographer issues a second command.

  Further, the computer program according to the present invention includes a photographing step of photographing a subject and converting it into photographed image data after the photographer issues a first command, and a timing at which the first command is issued, An initialization step for setting the position of the apparatus as a reference position, a movement state recognition step for recognizing the movement state of the apparatus from the reference position, the captured image data is deformed according to the movement state, and a deformed image An image transformation step for obtaining data, a display step for displaying the transformed image data on a display device, and an image acquisition step for obtaining the photographed image data at a timing when the photographer issues a second command are executed on a computer. Let

  As is clear from the above description, according to the present invention, when the other side of the obstacle is photographed, the display image is deformed as if the display image is crushed by looking at the display unit from an oblique direction, and the subject is accurately centered. It is possible to solve the problem that it is difficult to confirm whether or not it is captured.

It is a block diagram which shows the block configuration of the portable terminal device which concerns on embodiment of this invention. It is an external view which shows the external appearance of the portable terminal device which concerns on embodiment of this invention. It is explanatory drawing which shows the method of imaging | photography when an obstruction does not exist in a to-be-photographed object direction. It is explanatory drawing which shows the method of imaging | photography when an obstruction exists in a to-be-photographed object direction. It is explanatory drawing which shows an example of the picked-up image when an obstruction does not exist in a to-be-photographed object direction. It is explanatory drawing which shows an example of the picked-up image image | photographed using the conventional portable terminal device with a camera function, when an obstruction exists in a to-be-photographed object direction. It is explanatory drawing which shows an example of the picked-up image image | photographed using the portable terminal device which concerns on embodiment of this invention when an obstruction exists in a to-be-photographed object direction. It is explanatory drawing which shows the image seen from the front of the picked-up image image | photographed using the portable terminal device which concerns on embodiment of this invention when an obstruction exists in a to-be-photographed object direction. It is a flowchart figure which shows the process sequence at the time of imaging | photography of the portable terminal device which concerns on embodiment of this invention.

  INDUSTRIAL APPLICABILITY The present invention can be applied to a portable terminal device with a camera function when an attempt is made to photograph the other side of an obstacle that is higher than the height of the photographer's horizontal line of sight. That is, in such a case, in a conventional mobile terminal device with a camera function, the photographed image appears to be deformed, and it is difficult for the photographer to confirm the image. As a result, there has been a problem that it is impossible to determine the exact position of the image. Therefore, in the mobile terminal device of the present invention, the captured image is deformed in advance so that the image viewed from the photographer looks normal (that is, a shape faithful to the subject). Thus, an image faithful to the shape of the subject can be acquired, and an image capturing the subject can be displayed in an easy-to-understand manner for the photographer, thereby facilitating the photographing operation of the photographer.

  Hereinafter, embodiments of a portable terminal device, a photographing method, and a computer program of the present invention will be described in detail with reference to the drawings. FIG. 1 is a configuration diagram showing a block configuration of a mobile terminal device according to an embodiment of the present invention. The portable terminal device of this embodiment shown in the figure includes a CPU 1, a camera unit 104, an acceleration sensor 3, a display unit 103, a memory 5, and a key input unit 6. In addition, a communication unit that enables communication with another information processing device via a mobile phone network or the Internet network can be provided.

  In the mobile terminal apparatus shown in FIG. 1, the CPU 1 controls the entire apparatus by a control method based on the stored programming method. The camera unit 104 captures a subject and converts it into captured image data, and outputs the converted captured image data to the CPU 1. The acceleration sensor 3 detects the acceleration applied to the apparatus and recognizes how the apparatus has moved. The display unit 103 displays captured image data and instructions and data for the photographer. The memory 5 stores a program to be executed by the CPU 1 and data necessary for executing various applications including image processing. The key input unit 6 inputs start-up commands and operation commands for the apparatus, and inputs data and commands necessary for instructing image acquisition. Specifically, the mobile terminal device shown in FIG. 1 may be a digital camera, a mobile phone device, a mobile information processing device, and a game machine.

  FIG. 2 is an external view showing the external appearance of the mobile terminal device according to the embodiment of the present invention. As shown in the figure, the display unit 103 is a part attached to the display surface 101. The camera unit 104 is a part attached to the camera mounting surface 102.

  FIG. 3 is an explanatory diagram illustrating a method of performing imaging when no obstacle exists in the direction of the subject. FIG. 4 is an explanatory diagram illustrating a method of performing imaging when an obstacle exists in the subject direction. FIG. 5 is an explanatory diagram illustrating an example of a captured image when no obstacle exists in the subject direction. FIG. 6 is an explanatory diagram illustrating an example of a captured image captured using a conventional mobile terminal device with a camera function when an obstacle exists in the direction of the subject. FIG. 7 is an explanatory diagram illustrating an example of a photographed image photographed using the mobile terminal device according to the embodiment of the present invention when an obstacle exists in the subject direction. Hereinafter, the function of the mobile terminal device with a camera function of the present embodiment will be described with reference to FIGS.

  First, as shown in FIG. 3, when shooting is performed when there is no obstacle in the subject direction 204 using the portable terminal device with a camera function 201 according to the embodiment of the present invention, the subject direction 204 and the image confirmation angle 203 are arranged in a straight line. In this case, since an image as shown in FIG. 5 is seen on the display unit 103 of the mobile terminal device 201 with a camera function, it is easy to recognize an image capturing a subject. Such a situation is the same in a general conventional apparatus.

  Next, as shown in FIG. 4, when the subject behind the obstacle 305 is photographed when the obstacle 305 exists in front of the photographer 302, the image output to the display unit 103 is as it is. As shown in FIG. 6, the image is crushed (deformed image). Therefore, it becomes difficult for the photographer 302 to recognize the subject with an accurate image. Therefore, in the mobile terminal device according to the present embodiment, as shown in FIG. 8, the captured image data is deformed in advance by executing an image processing program (not shown) in the CPU 1. As a result, the screen viewed from the photographer 302 looks as shown in FIG. 7, so that the image capturing the subject can be easily understood and the photographing operation of the photographer 302 can be facilitated.

  FIG. 8 shows an example of an image seen from the front of a photographed image taken using the mobile terminal device with a camera function 301 (the mobile terminal device according to the embodiment of the present invention) when an obstacle exists in the direction of the subject. It is explanatory drawing shown as. In the portable terminal device according to the embodiment of the present invention, when the obstacle 305 is present in front of the photographer 302, when shooting a subject on the other side of the obstacle 305, as described above, It is assumed to be deformed in advance as shown in FIG.

  In order to realize such processing, the mobile terminal device according to the embodiment of the present invention uses the acceleration sensor 3 shown in FIG. 1 to obtain a subject direction 304 and an image confirmation angle 303 as shown in FIG. Recognize when they do not match. If it is recognized that the subject direction 304 and the image confirmation angle 303 do not match, as described above, the image processing program in the CPU 1 corresponds to the image confirmation angle 303 as shown in FIG. Deform the captured image. As a result, as shown in FIG. 7, the image viewed from the photographer 302 is a photograph of the subject faithfully captured at an accurate image position. Therefore, the photographer 302 looks at the image from the front. It becomes possible to confirm with a simple image. Further, when the shutter is pressed, an image faithful to the shape of the subject can be acquired.

  In the case of a general conventional apparatus, if there is an obstacle 305 in front of the photographer 302, a problem occurs when attempting to photograph a subject on the other side of the obstacle 305. That is, in this case, the subject direction 304 and the image confirmation angle 303 are not preferably matched. Therefore, in the case of a general conventional apparatus, the image output to the display unit 103 is a collapsed image (deformed image) as shown in FIG. 6, and the subject can be recognized with an accurate image. It becomes difficult.

  Hereinafter, the operation of the mobile terminal device according to the embodiment of the present invention will be described based on one usage example. However, when the mobile terminal device 301 with a camera function used for shooting (the mobile terminal device according to the embodiment of the present invention) activates the camera function at the position shown in FIG. 3, the image confirmation angle 203 shown in FIG. It is assumed that an image as shown in FIG. Here, it is assumed that the other side of the obstacle 305 is photographed as shown in FIG. In this case, as shown in FIG. 4, when the photographer 302 moves the mobile terminal device with camera function 301 upward, the acceleration sensor 3 (FIG. 1) reacts. In addition, the movement amount (movement distance) and movement direction of the apparatus corresponding to the reaction are output.

  Next, the CPU 1 (FIG. 1) calculates the image confirmation angle 303, deforms the captured image as shown in FIG. 8 according to the calculated image confirmation angle 303, and displays the deformed image on the display unit 103. To do. As a result, as shown in FIG. 7, the photographer 302 shown in FIG. 4 can visually recognize an image with an accurate ratio similar to the case where the photographer 302 is captured in front. Here, the case where the mobile terminal device 301 with a camera function is moved upward is shown. On the other hand, depending on the condition of the obstacle, the mobile terminal device 301 with a camera function can be moved in any other direction including the left-right direction, and in this case also according to the embodiment of the present invention. The mobile terminal device performs the same operation as described above.

  FIG. 9 is a flowchart showing a processing procedure at the time of photographing by the mobile terminal device according to the embodiment of the present invention. Hereinafter, the operation at the time of shooting of the mobile terminal device according to the present embodiment will be described using the flowchart shown in FIG. 9 with reference to FIGS. Here, in the mobile terminal device with camera function shown in FIG. 2, first, in step S1, as shown in FIG. 3, the photographer 202 turns on the camera function in the state shown in FIG. In response, the camera unit 104 is activated. In step S2, the CPU 1 performs initial setting with the position of the present apparatus at this timing as a basic position. Specifically, a front image without deformation is set at an image recognition angle of 0 degree.

  Next, in step S <b> 3, the photographer 202 turns on the display function, whereby the CPU 1 activates the display unit 103. At this time, an unmodified image as shown in FIG. 5 is displayed. Thereafter, in step S4, the acceleration sensor 3 detects the vertical movement of the apparatus. When the acceleration sensor 3 detects the vertical movement of the apparatus, the process continues to step S5. If the acceleration sensor 3 does not detect the vertical movement of the apparatus, the process proceeds to step S7. At this time, the current display image remains an unmodified image as shown in FIG.

  In step S5, the CPU 1 calculates an image confirmation angle 303 as shown in FIG. 4 according to the movement amount (movement distance) and movement direction of the apparatus. Moreover, in continuing step S6, CPU1 deform | transforms a display image via the image process part which is not shown based on said calculation result, makes this a present display image, and passes the flow of a process to step S7. .

  In step S <b> 7, the CPU 1 outputs the current display image to the display screen of the display unit 103. At this time, if the process flow is taken over from step S5, an image as shown in FIG. 8 is displayed. In this case, since this apparatus is moving up and down, the photographer 202 shown in FIG. 3 is like a photographer 302 shown in FIG. 4 (the same person as the photographer 202 shown in FIG. 3). The display image can be visually recognized at an image confirmation angle 303 as shown in FIG. Therefore, for the photographer 302 (the same person as the photographer 202 shown in FIG. 3), the display image looks as shown in FIG. 7, so the photographer 302 (that is, the photographer 202) captures an image capturing the subject. It can be seen with an accurate image seen from the front.

  Therefore, at this time, if the photographer 302 presses the shutter or the like, an image obtained by accurately capturing the subject existing in the direction of the subject direction 304 shown in FIG. 4 with the angle and direction indicated by the subject direction 304 is captured. Can do. Thereafter, in step S8, the CPU 1 verifies whether or not the photographing is finished. If the photographing is finished, the process proceeds to step S9. If shooting has not been completed, the process returns to step S4. In step S <b> 9, the photographer 302 turns off the display function, whereby the CPU 1 stops the display unit 103.

  In the subsequent step S10, the photographer 302 stops the camera function, thereby stopping the camera unit 104. Thereby, the process at the time of imaging | photography is complete | finished.

  According to the mobile terminal device according to the present embodiment, when the other side of the obstacle is photographed, the display image on the display unit 103 is deformed so that the display unit 103 is crushed by looking at the display unit from an oblique direction. It is possible to solve the problem that it is difficult to confirm whether the situation is captured. That is, in the mobile terminal device according to this embodiment, the display screen is deformed in advance and displayed on the display unit 103. In this way, since it is easy for the photographer 302 to confirm that the subject is captured at the center, the photographer 302 obtains a subject having an accurate shape as a photographed image (more specifically, stored in the memory 5). It becomes possible.

  In the above-described embodiment, the movement state (movement direction and movement distance) of the mobile terminal device used for photographing is detected using the acceleration sensor 3 shown in FIG. On the other hand, an image recognition program (not shown) executed by the CPU 1 detects that the position of the camera has moved upward from a change in the image data captured as a photographing result, and the movement status (movement direction). It is also possible to deform the display image according to the movement distance).

  Further, a camera is also provided on the display surface 101 side shown in FIG. 2, and by detecting the direction of the eye of the photographer 302 with a fisheye lens or the like, it is detected that the position of the camera has moved upward in accordance with the eye direction. It is also possible to deform the display image according to the amount of movement. Furthermore, the display image can be forcibly deformed by a key operation by the photographer 302 (FIG. 4).

  Note that at least a part of processing of each component of the mobile terminal device according to the present invention is executed by computer control. A program that causes a computer to execute the above processing according to the procedure shown in the flowchart of FIG. 9 may be distributed by storing it in a computer-readable recording medium such as a semiconductor memory, a CD-ROM, or a magnetic tape. . A computer including at least a microcomputer, a personal computer, and a general-purpose computer may read and execute the program from the recording medium.

1 CPU
3 Acceleration sensor 5 Memory 6 Key input unit 101 Display surface 102 Camera mounting surface 103 Display unit 104 Camera unit 202 Photographer 302 Photographer 203 Image confirmation angle 303 Image confirmation angle 204 Subject direction 304 Subject direction

Claims (8)

Photographing means for photographing a subject and converting it into photographed image data after the photographer issues a first command;
Initialization means for setting the position of the apparatus as a reference position at the timing when the first command is issued;
Movement status recognition means for recognizing the movement status of the apparatus from the reference position;
Image deformation means for deforming the captured image data in accordance with the movement status and obtaining deformed image data;
Display means for displaying the modified image data on a display device;
A portable terminal device comprising: an image acquisition unit configured to acquire the captured image data at a timing when a photographer issues a second command. The mobile terminal device according to claim 1, wherein the movement status recognized by the movement status recognition unit is a movement direction and a movement distance of the apparatus. The mobile terminal device according to claim 2, wherein the moving direction and the moving distance are calculated by an acceleration sensor. The mobile terminal device according to claim 2, wherein the moving direction and the moving distance are recognized by an image recognition program executed by a computer. The portable device according to claim 2, further comprising an additional unit capable of capturing a photographer's line of sight, wherein the movement direction and the movement distance are grasped based on the photographer's line of sight captured by the additional unit. Terminal device. The portable terminal device according to claim 1, wherein the image acquisition unit stores the captured image data in a memory at the timing when the photographer instructs the second instruction. A shooting step of shooting a subject and converting it into shot image data after the photographer issues a first command;
An initialization step of setting the position of the apparatus as a reference position at the timing when the first command is issued;
A movement status recognition step for recognizing the movement status of the apparatus from the reference position;
An image deformation step of deforming the captured image data according to the movement state to obtain deformed image data;
A display step of displaying the modified image data on a display device;
An image acquisition step of acquiring the captured image data at a timing when the photographer issues a second command. A shooting step of shooting a subject and converting it into shot image data after the photographer issues a first command;
An initialization step of setting the position of the apparatus as a reference position at the timing when the first command is issued;
A movement status recognition step for recognizing the movement status of the apparatus from the reference position;
An image deformation step of deforming the captured image data according to the movement state to obtain deformed image data;
A display step of displaying the modified image data on a display device;
A computer program that causes a computer to execute an image acquisition step of acquiring the captured image data at a timing when a photographer issues a second command.

Images