JP2010141839A - Mobile terminal unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile terminal unit capable of displaying characters and images displayed on a display means at a certain location for the user even when a chassis held by the hand is shaking. <P>SOLUTION: Display data outputted from a data storing section 100 is written into a buffer memory 1. A blurring amount calculation section 3 calculates a blurring amount of a chassis when display data is displayed at a display section 2 based on images photographed by a camera 200. A display position control section 4 outputs a display position control signal for changing the display position of the display data on the display section 2 based on the blurring amount calculated by the blurring amount calculation section 3. A display processing section 5 reads display data from the buffer memory 1 and outputs it to the display section 2 based on the display position control signal outputted from the display position control section 4. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a mobile terminal device.

  The mounting of cameras on mobile phone terminals has become widespread, and high performance such as higher pixels and higher resolution is also progressing. As a problem when photographing with such a camera mounted on a mobile phone terminal, there is a blurring of a photographed image due to camera shake of a photographer.

  Therefore, conventionally, a camera shake correction function has been added to a camera so that a clear image can be obtained even if there is a camera shake. As an apparatus for performing such camera shake correction, the image before and after a predetermined time is compared, the movement of the image is detected, the output range from the camera is moved according to the amount of the detected movement, There has been proposed a signal processing device that removes image blur due to camera shake (see, for example, Patent Document 1).

  On the other hand, in recent years, mobile phone terminals have become more sophisticated, and processes that have been performed on personal computers, such as sending and receiving e-mails and web browsing, can now be executed on mobile phone terminals. As a result, opportunities to browse characters, images, and the like in a moving body such as a train are increasing.

However, the moving body that is in progress has shaking of the vehicle body, and the shaking is transmitted to the user. As a result, the mobile phone terminal held by the user is in an unstable state, that is, a state similar to the above-described camera shake during camera shooting, and it is difficult to see characters and images displayed on the screen. The problem occurs.
JP-A-7-177419 (page 3-4, FIG. 1)

  SUMMARY OF THE INVENTION An object of the present invention is to provide a portable terminal device that can display characters and images displayed on the display means at a fixed position to the user even in a state where the hand-held housing is shaking. is there.

  According to one aspect of the present invention, there is provided a mobile terminal device incorporating a camera, wherein a buffer memory in which data for a display area is written and a screen size larger than the display area of the data are stored in the buffer memory. A camera shake amount calculating means for calculating a camera shake amount based on a photographed image of the camera when the data is displayed on the display means for displaying the written data; and the data display means. Display position control means for outputting a display position control signal for changing the display position in accordance with the camera shake amount calculated by the camera shake amount calculation means; and the display means for reading the data from the buffer memory. There is provided a portable terminal device comprising: a display processing means for performing a process of outputting to the display based on the display control signal.

  According to another aspect of the present invention, there is provided a mobile terminal device incorporating a camera, the buffer memory in which data for a display area is written, and the buffer having a screen size larger than the display area of the data A camera shake amount calculating means for calculating a camera shake amount based on a photographed image of the camera when the data is displayed on a display means for displaying the data written in the memory; Write position control means for outputting a write position control signal for changing a write position to the buffer memory in accordance with the shake amount calculated by the shake amount calculation means; and reading the data from the buffer memory and displaying the display There is provided a portable terminal device comprising display processing means for performing processing to output to the means.

  According to the present invention, it is possible to display characters and images displayed on the display means at a fixed position with respect to the user even in a state in which the housing on hand is shaking.

  Embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and the description thereof will not be repeated.

  FIG. 1 is a block diagram illustrating an example of a configuration of a mobile terminal device according to the first embodiment of the present invention.

  The mobile terminal device of the present embodiment is, for example, a camera-equipped mobile phone terminal device, and data for a display area (display data) output from the data storage unit 100 in which data such as received mail and transmitted mail is stored. Is written in the buffer memory 1, the display unit 2 on which the display data written in the buffer memory 1 is displayed, and the camera shake amount of the housing when the display data is displayed on the display unit 2 of the camera 200. A camera shake amount calculation unit 3 that calculates based on the photographed image and a display position control signal for changing the display position of the display data on the display unit 2 according to the camera shake amount calculated by the camera shake amount calculation unit 3 are output. The display position control unit 4 that outputs the display data from the buffer memory 1 and outputs the display data to the display unit 2. It comprises, a display processing unit 5 to perform based on.

  An amount of data that can be displayed in the display area of the display unit 2 is read out from the data stored in the data storage unit 100 and written to the buffer memory 1.

  FIG. 2A shows the relationship between the screen size of the display unit 2 and the display area 21 and the arrangement relationship between the display unit 2 and the camera 200 in the housing 1000 of the mobile terminal device of this embodiment.

  As shown in FIG. 2A, in this embodiment, data such as e-mail is not displayed on the entire surface of the display unit 2, but is displayed in the display area 21 smaller than the screen size. In other words, the screen size of the display unit 2 is larger than the display area 21. Therefore, there is a margin around the display area 21 in the normal display state.

  In the present embodiment, the display unit 2 and the camera 200 are arranged on the same plane of the housing 1000. Therefore, when the camera 200 is operated while a mail or the like is displayed on the display unit 2, the camera 200 is in a state of photographing a user who is browsing the mail or the like. An example of a photographed image 201 photographed by the camera 200 at this time is shown in FIG.

  As shown in FIG. 2B, as the captured image 201, a person image of a user who is browsing the data displayed on the display unit 2 is obtained.

  If the moving body shakes while viewing a mail or the like displayed on the display unit 2 in a moving body such as a train, the shaking is transmitted to the housing 1000 that is being held, and the display unit 2 And the camera 200 shakes. As a result, characters displayed on the display unit 2 appear to shake, and camera shake occurs in the user's image obtained by the camera 200.

  At this time, since the display unit 2 and the camera 200 are arranged on the same plane of the housing 1000, the amount of shaking of the characters being displayed on the display unit 2 and the amount of camera shake of the captured image of the camera 200 are between. Have a certain proportional relationship.

  Therefore, in this embodiment, the amount of camera shake of an image taken by the camera 200 is calculated, and the display position of the display area 21 on the display unit 2 is changed based on the amount of camera shake, and the image being displayed on the display unit 2 is displayed. Try to cancel the shaking of letters.

  FIG. 3 is a block diagram illustrating an example of an internal configuration of the camera shake amount calculation unit 3.

  The camera shake amount calculation unit 3 includes a pattern extraction unit 31 that extracts a human image from an image captured by the camera 200 at a predetermined time interval, a pattern storage unit 32 that stores the human image extracted by the pattern extraction unit 31, and A difference calculation unit that compares the person image stored in the pattern storage unit 32 with the next person image extracted by the pattern extraction unit 31 to calculate a difference in the image position, and outputs the difference amount as a camera shake amount 33.

  The display position control unit 4 outputs a display position control signal in order to change the display position of the display data on the display unit 2 according to the camera shake amount calculated by the camera shake amount calculation unit 3. The operation of the display position control by the display position control unit 4 will be described with reference to FIGS.

  FIG. 4 shows how the display position is controlled when the casing 1000 is shaken in the horizontal direction.

  Now, assume that the housing 1000 located at the position of FIG. 4A swings leftward and moves to the position of FIG. 4B. Let the amount of shaking at this time be d1.

Due to this shaking, camera shake also occurs in the captured image 201 of the camera 200. As shown in FIG. 4C, the camera shake appears in the right direction opposite to the shaking of the housing 1000. Therefore, if this camera shake amount is d2,
d2 = −k · d1
It is expressed. Here, k is a coefficient determined by the magnification of the camera 200 and the distance to the subject (that is, the user). Since the distance between the user viewing the display unit 2 and the camera 200 is considered to be substantially constant, this k can be regarded as a constant.

Therefore, when the camera shake amount d2 is calculated by the camera shake amount calculation unit 3, the shake d1 of the housing 1000 is
d1 = −d2 / k
As required.

  Therefore, the display position control unit 4 outputs a display position control signal so as to change the display position of the display area 21 by d1, that is, −d1 in the direction opposite to the shaking of the housing 1000 at the position of FIG. Output.

  As described above, when the display position of the display area 21 is changed by the same amount as the amount of shaking in the opposite direction to the shaking of the housing 1000, the shaking of the housing 1000 is canceled, and the user receives characters, etc. Appears to be displayed at the same position.

  FIG. 5 shows how the display position is controlled when the casing 1000 is shaken in the vertical direction.

Now, it is assumed that the housing 1000 at the position shown in FIG. 5A swings upward and moves to the position shown in FIG. Let the amount of shaking at this time be h1. At this time, as shown in FIG. 5 (c), when the camera shake amount of the captured image 201 of the camera 200 is generated in the downward direction opposite to the shaking of the housing 1000, the amount of camera shake is h2.
h2 = −k · h1
It is expressed.

Therefore, in this case, the shaking h1 of the housing 1000 is
h1 = −h2 / k
As required.

  Therefore, the display position control unit 4 outputs a display position control signal so as to change the display position of the display area 21 by h1, that is, −h1 in the direction opposite to the shaking of the housing 1000 at the position of FIG. Output. Also in this case, the shaking of the housing 1000 is canceled, and it appears to the user that characters and the like are displayed at the same position.

  The display processing unit 5 reads out display data from the buffer memory 1 and outputs the display data to the display unit 2 based on the display control signal output from the display control unit 4.

  There are two methods for the processing. One is a method of changing the read start timing of the display data from the buffer memory 1, and the other is a method of changing the output start timing when the display data read from the buffer memory 1 is output to the display unit 2. Is the method.

  FIG. 6A is a waveform diagram showing an example in which the timing for starting reading display data from the buffer memory 1 is changed.

  Based on the display control signal output from the display control unit 4, the display position of the display area 21 on the display unit 2 can be changed by changing the timing to start reading display data from the buffer memory 1. .

  FIG. 6B is a waveform diagram showing an example of changing the output start timing when the display data read from the buffer memory 1 is output to the display unit 2.

  By changing the output start timing when the display data read from the buffer memory 1 is output to the display unit 2 based on the display control signal output from the display control unit 4, the display area 21 on the display unit 2 is changed. The display position can be changed.

  Thus, in this embodiment, the display position of the display data stored in the buffer memory 1 on the display unit 2 is changed based on the display control signal output from the display control unit 4.

  On the other hand, when a scroll signal is output to the display unit 2, the data stored in the buffer memory 1 is sequentially rewritten, and the display on the display unit 2 is also rewritten sequentially. Therefore, in such a case, since the display position of characters and the like on the display unit 2 is originally not constant, it is considered that there is little need to correct the shaking of the housing 1000.

  Therefore, as shown in FIG. 7, when a scroll signal for the display unit 2 is output, the output of the display position control signal from the display position control unit 4 may be stopped.

  According to the present embodiment, the amount of shaking of the housing 1000 is calculated based on the amount of shaking of the image taken by the camera 200 arranged on the same surface as the display unit 2, and the shaking is canceled. By changing the display position of the display area 21, it is possible to display characters and images displayed on the display unit 2 at a fixed position to the user even when the hand-held housing 1000 is shaking.

  FIG. 8 is a block diagram illustrating an example of the configuration of the mobile terminal device according to the second embodiment of the present invention.

  The portable terminal device of the present embodiment is provided with a writing position control unit 6 instead of the display position control unit 4 of the first embodiment, and display data to the buffer memory 1 according to the camera shake amount output from the camera shake amount calculation unit 3. The writing position is changed.

  The writing position control unit 6 outputs a writing position control signal to the buffer memory 1 in order to change the writing position of display data to the buffer memory 1 according to the amount of camera shake output from the camera shake amount calculation unit 3.

  The buffer memory 1 of this embodiment has a memory capacity that can store the data amount of the entire display area of the display unit 2. When display data is read from the data storage unit 100, the writing position of the display data in the buffer memory 1 is controlled by the set position of the display area 21. Thereafter, when a write position control signal is output from the write position control unit 6, the display data write position on the buffer memory 1 changes according to the amount of camera shake.

  The display control unit 5A sequentially reads the data in the buffer memory 1 from the head address and outputs the data to the display unit 2. Therefore, the position of the display area displayed on the display unit 2 changes according to the change in the display data writing position.

  FIG. 9 shows the relationship between the image of the display data writing position on the memory area of the buffer memory 1 and the position of the display area on the display unit 2.

  When the display data writing position to the buffer memory 1 changes and changes from the display data writing position image 11A to the display data writing position image 11B, the position of the display area on the display unit 2 is also changed according to the change. To the display area 21B.

  That is, since the display data writing position to the buffer memory 1 changes according to the amount of camera shake, the position of the display area 21 displayed on the display unit 2 also changes according to the amount of camera shake.

  According to the present embodiment, the display data writing position to the buffer memory 1 is changed according to the amount of camera shake, thereby changing the position of the display area 21 displayed on the display unit 2 according to the amount of camera shake. be able to.

The block diagram which shows the example of a structure of the portable terminal device which concerns on Example 1 of this invention. The figure which shows the external appearance of the portable terminal device which concerns on Example 1 of this invention, and the outline of a picked-up image. FIG. 3 is a block diagram illustrating an example of an internal configuration of a camera shake amount calculation unit according to the first embodiment of the present invention. The figure for demonstrating operation | movement of the display position control part of Example 1 of this invention. The figure for demonstrating operation | movement of the display position control part of Example 1 of this invention. The wave form diagram which shows the example of operation | movement of the display process part of Example 1 of this invention. The wave form diagram which shows the example of operation | movement of the display position control part when a scroll signal is output. The block diagram which shows the example of a structure of the portable terminal device which concerns on Example 2 of this invention. The figure which shows the relationship between the writing position image of the buffer memory of Example 2 of this invention, and the position of a display area.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Buffer memory 2 Display part 3 Camera shake amount calculation part 4 Display position control part 5, 5A Display processing part 6 Write position control part 11A, 11B Display data writing position image 21, 21A, 21B Display area 31 Pattern extraction part 32 Pattern storage part 33 Difference calculation unit

Claims (5)

A mobile terminal device with a built-in camera,
A buffer memory in which data for the display area is written;
When the data is displayed on the display means for displaying the data written in the buffer memory having a screen size larger than the display area of the data, the camera shake amount of the housing is measured. Camera shake amount calculation means for calculating based on the image;
Display position control means for outputting a display position control signal for changing the display position of the data on the display means in accordance with the camera shake amount calculated by the camera shake amount calculation means;
A portable terminal device comprising: a display processing unit that reads out the data from the buffer memory and outputs the data to the display unit based on the display control signal. The camera shake amount calculating means
The amount of camera shake is calculated based on an image of a user who is browsing the display unit and is photographed by a camera arranged on the same plane as the display unit. The portable terminal device described. The display processing means
The portable terminal device according to claim 1 or 2, wherein a timing to start reading the data from the buffer memory is changed based on the display control signal. The display processing means
3. The portable terminal device according to claim 1, wherein an output start timing of the data read from the buffer memory to the display unit is changed based on the display control signal. A mobile terminal device with a built-in camera,
A buffer memory in which data for the display area is written;
When the data is displayed on the display means for displaying the data written in the buffer memory having a screen size larger than the display area of the data, the camera shake amount of the housing is measured. Camera shake amount calculation means for calculating based on the image;
Write position control means for outputting a write position control signal for changing the write position of the data to the buffer memory in accordance with the shake amount calculated by the shake amount calculation means;
And a display processing unit that performs a process of reading the data from the buffer memory and outputting the data to the display unit.

Images