JP5654886B2 - Portable terminal, display method, and computer program - Google Patents

Description

  The present invention relates to user interface technology, and more particularly, to a portable terminal having a user interface function and a display method usable in the portable terminal.

  In personal computers, smartphones, etc., icons corresponding to data and applications are displayed on the screen of the display device, and when an operation input such as double-clicking on the icon is accepted, the data corresponding to the icon is displayed, User interfaces for launching applications are widely adopted.

  In recent years, portable game machines, mobile phone terminals, and the like have become widespread, and opportunities to touch such user interfaces in daily life have increased dramatically. Needless to say, the user interface is required to have good operability, but there is also a strong demand for a display that is visually enjoyable and timeless.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a highly entertaining user interface.

  One embodiment of the present invention relates to a mobile terminal. The portable terminal includes a display device, a sensor for detecting the attitude of the portable terminal, a holding unit that holds data, and a display that displays an icon or object corresponding to the data held in the holding unit on the display device A control unit, wherein the display control unit obtains information on the attitude of the mobile terminal, calculates virtual gravity acting on the icon or object according to the tilt of the mobile terminal, and the gravity The display mode of the icon or object is determined based on the above.

  It should be noted that any combination of the above-described constituent elements and a representation of the present invention converted between a method, an apparatus, a system, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to provide a highly entertaining user interface.

It is a figure which shows the external appearance of the portable terminal which concerns on embodiment. It is a figure which shows the external appearance of the portable terminal which concerns on embodiment. It is a figure which shows the structure of a portable terminal. It is a figure which shows the example of the internal data of an icon table. It is a figure which shows the example of the desktop screen which a display control part displays. It is a figure which shows the example of the desktop screen which a display control part displays. It is a figure for demonstrating the function to change the light which hits an icon. It is a figure for demonstrating the function to change the light which hits an icon. It is a figure which shows the example of the desktop screen which a display control part displays. It is a figure which shows the example of the desktop screen which a display control part displays. It is a figure for demonstrating the function to scroll a character. It is a figure for demonstrating the function to scroll a character. It is a figure for demonstrating the function to scroll a character. It is a figure for demonstrating the function to scroll a character. It is a figure for demonstrating the function which switches a background image. It is a figure for demonstrating the function which switches a background image. It is a figure for demonstrating the function to make an icon jump. It is a figure for demonstrating the function to make an icon jump.

  FIG.1 and FIG.2 shows the external appearance of the portable terminal 10 which concerns on embodiment. As shown in FIG. 1, on the front side of the mobile terminal 10, that is, the side facing the user when the user holds and operates the mobile terminal 10, an instruction input button 21, a direction key 22, an R button 23, L A button 20 such as the button 24, a top camera 25, and a display device 68 are provided. The display device 68 is provided with a touch panel 69 for detecting contact with a user's finger or stylus pen.

  As shown in FIG. 2, a rear camera 26 and a rear touch panel 70 are provided on the back side of the mobile terminal 10. Although the display device may be provided on the back side of the mobile terminal 10 as well as the front side, in the present embodiment, only the back touch panel 70 is provided on the back side of the mobile terminal 10 without providing the display device.

  While holding the mobile terminal 10 with both hands, for example, the user operates the instruction input button 21 with the thumb of the right hand, operates the direction key 22 with the thumb of the left hand, operates the R button 23 with the index finger or the middle finger of the left hand, The L button 24 can be operated with the index finger or the middle finger, the touch panel 69 can be operated with the thumbs of both hands, and the back touch panel 70 can be operated with the ring finger or the little finger of both hands. In the case of using a stylus pen or the like, for example, while holding the mobile terminal 10 with the left hand, the touch panel 69 and the instruction input button 21 are operated with the stylus pen or the index finger with the right hand, the direction key 22 is operated with the left hand thumb, and the left hand The L button 24 can be operated with the index finger or the middle finger, and the rear touch panel 70 can be operated with the left ring finger or the little finger.

  As shown in FIG. 1, when an application such as a game is not executed on the mobile terminal 10, the display device 68 receives an instruction input for opening a data file or starting an application from the user. The user interface screen (hereinafter also referred to as “desktop screen”) is displayed. On the desktop screen, icons associated with data files and applications are displayed side by side. In the present embodiment, the icon is expressed as an object having a three-dimensional button shape, and is displayed so as to float with respect to the background image. In the background image, an icon shadow is displayed. The mobile terminal 10 provides a function of changing the display mode of icons and background images according to the position and orientation of the mobile terminal 10.

  FIG. 3 shows the configuration of the mobile terminal 10. The portable terminal 10 includes a top camera 25, a rear camera 26, a button 20, a touch panel 69, a rear touch panel 70, a three-axis gyro sensor 72, a three-axis acceleration sensor 73, a control unit 40, a data holding unit 60, an icon table 62, and a display. A device 68 is provided. In terms of hardware components, these configurations are realized by a CPU of a computer, a memory, a program loaded in the memory, and the like, but here, functional blocks realized by their cooperation are illustrated. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The top camera 25 and the back camera 26 capture an image around the mobile terminal 10. As described above, the button 20, the touch panel 69, and the back touch panel 70 are provided on the surface of the casing of the mobile terminal 10 and receive an operation input from the user. The triaxial gyro sensor 72 is provided inside the housing of the mobile terminal 10 and detects an angle or an angular velocity of the mobile terminal 10 in the XZ plane, the ZY plane, and the YX plane. The triaxial acceleration sensor 73 is provided inside the casing of the mobile terminal 10 and detects acceleration in the XYZ triaxial directions of the mobile terminal 10.

  The data holding unit 60 is configured by a storage medium such as a hard disk, and holds various data files such as documents, music, and moving images used in the mobile terminal 10 and application data for processing the data files. The data holding unit 60 holds various physical quantities and equations of motion that are used when calculating the behavior of the icon by applying a physical law. The icon table 62 holds information on icons associated with data files or applications held in the data holding unit 60.

  FIG. 4 shows an example of internal data of the icon table. The icon table 62 includes an icon ID field 80, an icon name field 81, a shortened name field 85, a path field 82, a coordinate field 83, and a shape data field 84. The icon ID column 80 stores an ID for uniquely identifying an icon. The icon name column 81 stores the name of the icon. The short name column 85 stores the short name of the icon. The abbreviated name may be acquired in advance from a data creator, a user, or the like, or may be automatically generated from an icon name according to a predetermined rule. The path column 82 stores a path indicating a storage location in the data holding unit 60 of the data file associated with the icon or the execution format file of the application. The coordinate column 83 stores coordinates indicating the position of the desktop screen on which the icon is to be displayed. The shape data column 84 stores information such as icon shape data or a data file storing icon shape data. In the present embodiment, all the icons have the same shape, the shape data is held in the data holding unit 60, and the texture image of the surface of each icon is individually held in the data holding unit 60.

  The control unit 40 is configured by a hardware circuit such as a CPU and a memory, and provides a user interface function to the user and executes an application activated via the user interface function. The control unit 40 includes a user interface unit 41 and an application execution unit 48. The application execution unit 48 reads data necessary for the operation of the activated application from the data holding unit 60 to the memory, and executes the application program. The user interface unit 41 includes a display control unit 42, a posture position calculation unit 43, a movement calculation unit 44, a lighting calculation unit 45, and an input reception unit 46.

  The posture position calculation unit 43 acquires information detected by the three-axis gyro sensor 72 and the three-axis acceleration sensor 73, analyzes the information, and calculates the posture and position of the mobile terminal 10. The posture position calculation unit 43 is shifted to a predetermined reference position, for example, when the portable terminal 10 is turned on and the posture position calculation unit 43 is activated, receives a predetermined operation input, or shifts to a predetermined operation mode. The current relative position of the mobile terminal 10 is calculated on the basis of the position of the mobile terminal 10 when an operation input is not received for a predetermined time or longer. In addition, the posture position calculation unit 43 is activated by a predetermined reference posture, for example, the posture of the mobile terminal 10 when the mobile terminal 10 is placed on a horizontal plane or the mobile terminal 10 is turned on. On the basis of the attitude of the mobile terminal 10 such as when an operation input is received, when an operation mode is entered, or when an operation input is not received for a predetermined time or more. The relative angle of is calculated. The function of the present embodiment may be provided only while a predetermined operation input is performed, or until a predetermined end operation is performed after a predetermined start operation is performed. In this case, the posture position calculation unit 43 uses the position and posture of the mobile terminal 10 when a predetermined operation input is started or when a predetermined start operation is performed as a reference for subsequent positions and postures. Calculate the change.

  The motion calculation unit 44 calculates a behavior when a predetermined external force is applied to a three-dimensional object displayed as an icon, a plane or a curved surface displayed as a background image, by applying a physical law. . The motion calculation unit 44 reads shape data of a three-dimensional object modeled using a polygon, a wire frame, or the like from the data holding unit 60, and calculates a position and orientation by a physical simulation using a physics calculation engine. The motion calculation unit 44 may calculate the behavior of the object by applying a physical law such as an equation of motion as it is, or a part of the physical law in order to deform and display part or all of the behavior of the object. Or you may apply after correcting all. For example, a constant value such as gravitational acceleration may be set to a value different from an actual value, or an equation such as an equation of motion may be set to an expression different from an actual equation. The motion calculation unit 44 may perform physical calculation using any known technique. The motion calculation unit 44 calculates the position and orientation of the icon and notifies the display control unit 42 of the calculation.

  When rendering the three-dimensional object displayed as an icon, the lighting calculation unit 45 arranges a light source that emits light of a predetermined intensity at a predetermined position, applies a predetermined lighting model, and increases the brightness of the icon surface. calculate.

  When the input reception unit 46 receives a double-click operation or a double-tap operation on the icon displayed on the desktop screen, the input reception unit 46 opens the data file associated with the icon, or the application associated with the icon. Is requested to the application execution unit 48 to execute.

  The display control unit 42 receives from the user an instruction to select data to be displayed on the desktop screen and an application, and receives image data or shape data necessary for displaying the received data or an icon associated with the application. Obtain and register in the icon table 62. When displaying the desktop screen, the display control unit 42 refers to the icon table 62 and displays an icon on the desktop screen. The display control unit 42 acquires the posture or position of the mobile terminal 10 calculated by the posture position calculation unit 43, and indicates the position and posture of the icon to the motion calculation unit 44 according to a change in the posture or position of the mobile terminal 10. Calculate and determine the icon display mode. The display control unit 42 displays the icon on the desktop screen by rendering the three-dimensional object corresponding to the icon using the calculated position and orientation of the icon. Hereinafter, the details of the function of the mobile terminal 10 according to the present embodiment will be described with reference to the drawings.

1-1. UI that shakes the icon according to the tilt of the mobile device
FIG. 5 shows an example of a desktop screen displayed by the display control unit. On the desktop screen 90, an icon 91 associated with the data or application held in the data holding unit 60 is displayed. The display control unit 42 determines the display mode of the icon 91 according to the inclination of the mobile terminal 10 calculated by the posture position calculation unit 43. The display control unit 42 is set at a predetermined timing, for example, when the power is turned on, when a predetermined operation is performed, when a transition is made to a predetermined operation mode, when no operation is performed for a predetermined time or longer. The vertical direction of the screen at that time is set as a virtual gravity direction with reference to the position and orientation of the mobile terminal 10 when no change in the position or orientation of the mobile terminal 10 is detected for a time or more. The display control unit 42 may set the gravity direction in the real world to a virtual gravity direction. The display control unit 42 uses the user's image captured by the top camera 25 to detect the position of the user's eyes by a technique such as face recognition, and sets the user's line-of-sight direction to a virtual gravity direction. Good. For example, a direction from the position of the user's eyes toward the center of gravity of the mobile terminal 10 may be set as a virtual gravity direction.

  When the user holds the mobile terminal 10 in a stationary state, the display control unit 42 displays the icon 91 so as to sway swaying around the reference position. The display control unit 42 may determine the moving direction and moving amount of the icon 91 using a random number or the like and move the icon 91, or animation using moving image data stored in advance in the data holding unit 60. It may be displayed. In the former case, the display control unit 42 may restrict the icon 91 from moving outside the predetermined range from the reference position. Thereafter, when the inclination of the mobile terminal 10 changes, the behavior of the icon 91 at that time is calculated by the motion calculation unit 44 and reflected in the display. The motion calculation unit 44 obtains the direction of gravity to be applied to the icon from the display control unit 42 and reads physical quantities such as the value of the gravitational acceleration and the rotation moment of the icon 91 from the data holding unit 60 and applies them. The behavior of the icon 91 is calculated. The display control unit 42 reflects the calculated behavior of the icon 91 on the display.

  In the example of this figure, the display control unit 42 fixes a predetermined position of the icon 91, for example, the position of the center of gravity, and causes the motion calculation unit 44 to calculate the posture of the icon 91. That is, as described above, the icon 91 is tilted according to a change in the tilt of the mobile terminal 10 and is displayed so as to shake while the position of the icon 91 is randomly moved within a predetermined range. For example, when the mobile terminal 10 is tilted so that the left side of the mobile terminal 10 is lowered, the icon 91 is tilted so that the right side is relatively lowered. When the mobile terminal 10 is swung left and right, the icon 91 is also swung left and right.

  The display control unit 42 may move the icon 91 by a predetermined amount in the tilted direction when the mobile terminal 10 is tilted. The amount of movement in this case may be larger than the amount of movement of the icon 91 when the mobile terminal 10 is not tilted. That is, when the mobile terminal 10 is not tilted, the display control unit 42 moves the icon 91 in small increments within the first range, and when the mobile terminal 10 is tilted, the display control unit 42 is larger than the first range. Within the range, the icon 91 is moved according to the inclination.

  In this manner, it is as if the three-dimensional object corresponding to the icon 91 is actually arranged on the screen of the display device 68 of the mobile terminal 10 and the object is also shaken in response to the shake of the mobile terminal 10. Since the icon 91 is displayed, a hand shake or the like of the user holding the mobile terminal 10 is reflected in the display of the icon 91, and a highly entertaining visual effect unique to the mobile terminal 10 can be provided.

  The display control unit 42 changes the behavior of the icon according to the attribute of the icon, for example, the type of the data file or application corresponding to the icon, the data amount, the creation date / time, the modification date / time, the activation date / time, the presence / absence of new arrival information, and the like. May be. For example, the value of the virtual gravitational acceleration may be adjusted so that the icon swings larger or smaller as the data amount is smaller, or as the creation date / time, modification date / time, or activation date / time is newer or older. Further, an icon set as a favorite by the user may be shaken larger or smaller than other icons. Thereby, the difference in the attribute of the icon can be displayed in a visually easy-to-understand manner, so that a user interface with good operability can be provided.

  When the display control unit 42 receives a predetermined operation such as an operation of tapping the desktop screen, the display control unit 42 may stop the icon. The display control unit 42 may freeze all icons or a specific icon. For example, when the user taps an area on the desktop screen where no icon is displayed, all icons may be frozen, and when a specific icon is tapped, only that icon may be frozen. The display control unit 42 may determine the icon to be stopped according to the attribute of the icon. For example, only icons set as favorites by the user may be stopped.

1-2. UI that moves icons according to the tilt of the mobile device
FIG. 6 shows an example of a desktop screen displayed by the display control unit. Also in the example of this figure, similarly to the example 1-1, when the inclination of the mobile terminal 10 is smaller than the predetermined value, the display control unit 42 shakes the icon in small increments, and the inclination of the mobile terminal 10 becomes lower than the predetermined value. In the example of this figure, the display control unit 42 does not fix the position of the icon 91 and moves within the range of the desktop screen. Let Therefore, when the mobile terminal 10 is tilted so that the left side of the mobile terminal 10 is lowered, the icon 91 slides down on the desktop screen in the left direction. The motion calculation unit 44 is the direction of gravity to be applied to the icon 91, the value of the gravitational acceleration, the static friction coefficient and the dynamic friction coefficient between the icon and the floor on which the icon is placed, the collision coefficient between icons, and the edge of the desktop screen. A physical quantity such as a collision coefficient between the virtually set wall and the icon is acquired, and applied to calculate the behavior of the icon 91. Thereby, the three-dimensional object corresponding to the icon 91 is actually arranged on the screen of the display device 68 of the mobile terminal 10, and the icon 91 moves so that the object moves in response to the tilt of the mobile terminal 10. Is displayed, it is possible to provide a highly entertaining visual effect unique to the mobile terminal 10.

  If the icon 91 is moved according to the inclination of the mobile terminal 10 as in the example 1-2, it may be difficult for the user to operate the icon 91. Therefore, normally, as in the example 1-1, the icon 91 is displayed so as to be shaken within a predetermined range, and the user interface is not received from the user for a predetermined time or by a hold key (not shown). In an operation mode in which the application is not started by double-clicking the icon, such as when the function is temporarily stopped, the icon is displayed so as to slide down in the tilted direction as in the example 1-2. Also good.

  The display control unit 42 determines the icon movement speed and movement according to the icon attribute, for example, the type of data file or application corresponding to the icon, data amount, creation date / time, modification date / time, activation date / time, presence of new arrival information, The distance, the angle of the mobile terminal 10 when starting to move, and the like may be varied. For example, the value of the virtual gravitational acceleration or dynamic friction coefficient may be adjusted so that the icon moves more greatly as the data amount is smaller, or as the creation date / time, modification date / time, or activation date / time is newer or older. . Thereby, the difference in the attribute of the icon can be displayed in a visually easy-to-understand manner, so that a user interface with good operability can be provided. An icon set as a favorite by the user or an icon with newly arrived information may be left on the spot without moving. Thereby, the icon set as a favorite and other icons can be easily selected. The display control unit 42 may select a moving icon and a non-moving icon based on other icon attributes.

  When the display control unit 42 receives a predetermined operation such as an operation of tapping the desktop screen, the display control unit 42 may stop the icon. The display control unit 42 may freeze all icons or a specific icon. For example, when the user taps an area on the desktop screen where no icon is displayed, all icons may be frozen, and when a specific icon is tapped, only that icon may be frozen. The display control unit 42 may determine the icon to be stopped according to the attribute of the icon. For example, only icons set as favorites by the user may be stopped.

1-3. UI that changes the light hitting the icon according to the attitude or position of the mobile terminal
7 and 8 are diagrams for explaining a function of changing light hitting an icon. As shown in FIG. 7, the lighting calculation unit 45 arranges the mobile terminal 10 with the icon on the surface thereof at a predetermined position in the virtual three-dimensional world 92 and also arranges the light source 93 at a predetermined position, flat shading, and phone shading. The luminance of the icon surface is calculated by applying a predetermined model such as. When the posture or position of the mobile terminal 10 changes, as shown in FIG. 8, the lighting calculation unit 45 changes the position and posture of the icon in the virtual three-dimensional world 92 according to the change. As a result, the relative position between the light source 93 and the icon changes, so that the light hitting the icon changes. As a result, if the three-dimensional object corresponding to the icon 91 is actually arranged on the screen of the display device 68 of the mobile terminal 10 of the mobile terminal 10 and the position or orientation of the mobile terminal 10 is changed, In response, the illumination that hits the icon can also be displayed to change. The illumination calculation unit 45 may change the position of the light source 93 according to the posture or position of the mobile terminal 10.

  9 and 10 show examples of desktop screens displayed by the display control unit. In FIG. 9, since the light hits from the upper left of the icon 91, the left side of the icon 91 is lit, and the shadow of the icon is displayed on the right side of the icon 91. In FIG. 10, the light or the icon 91 is illuminated from the upper right by changing the posture or position of the mobile terminal 10, so the right side of the icon 91 is illuminated and the shadow of the icon is displayed on the left side of the icon 91. Has been. The lighting calculation unit 45 may calculate the shadow of the icon by further arranging the shadow projection plane in the virtual three-dimensional world, or projects one type of texture image for displaying the shadow of the icon or the icon and projection. A plurality of types are stored in the data storage unit 60 according to the distance to the surface, and the shadow of the icon is displayed by adjusting the display position of the texture image according to the relative position between the light source 93 and the icon 91. It may be displayed.

  The display control unit 42 determines how the icon shines according to the attribute of the icon, for example, the type of the data file or application corresponding to the icon, data amount, creation date / time, modification date / time, activation date / time, presence / absence of new arrival information, and the like. It may be different. For example, when calculating the lighting of an icon so that the smaller the amount of data, the more the creation date, modification date, or new activation date, the more intense the icon shines, the distance between the icon and the light source may be shortened. The light intensity of the light source may be increased. Further, an icon set as a favorite by the user, an icon with newly arrived information, or the like may be emitted more intensely than other icons. In addition to the light intensity, the color of the light may be varied.

1-4. UI that scrolls characters when tilting mobile terminal
FIGS. 11-14 is a figure for demonstrating the function to scroll a character. In FIG. 11, a name display area 94 for displaying the name of the icon 91 is provided in the vicinity of the icon 91. The display control unit 42 reads the short name of the icon 91 from the icon table 62 and displays it in the name display area 94 of each icon 91. The shortened name of the icon 91 is set so that the number of characters is smaller than the number of characters that can be displayed in the name display area 94.

  When the desktop screen shown in FIG. 11 is displayed, if the left side of the mobile terminal 10 is tilted by a predetermined angle or more so as to be lowered, the display control unit 42 first displays an icon as shown in FIG. The name 91 is read from the icon table 62 and displayed in the name display area 94 instead of the shortened name. At this time, if the number of characters of the name exceeds the number of characters that can be displayed in the name display area 94, the number of characters that can be displayed from the beginning of the name is displayed. After that, as shown in FIG. 13, the name of the icon displayed in the name display area 94 is scrolled to the left to display the portion that has not been displayed. As a result, even if the name of the icon is too long to be displayed in the name display area 94, it is possible to scroll and view all of the icons, thereby improving user convenience. Moreover, since the name of the icon is displayed so as to slide down by tilting the mobile terminal 10, it is possible to provide a user interface that is visually interesting and intuitive. The display control unit 42 may scroll the name of the icon only for the icon whose name cannot be displayed. The display control unit 42 may display the name of the icon over a plurality of lines. In this case, the icon name may be scrolled up when the mobile terminal 10 is tilted by a predetermined angle or more so that the upper side of the mobile terminal 10 is lowered.

  When the desktop image shown in FIG. 11 is displayed, when the mobile terminal 10 is tilted by a predetermined angle or more so that the right side of the mobile terminal 10 is lowered, the display control unit 42 displays the icon 91 as shown in FIG. The name is read from the icon table 62, and the number of characters that can be displayed from the end of the name is displayed. Thereafter, the name of the icon is scrolled to the right to display the portion that has not been displayed. This allows the user to incline the mobile terminal 10 to the left when checking the first half of the icon name, and to tilt the mobile terminal 10 to the right when checking the second half of the icon name. It is possible to provide a user interface that is easy to understand and easy to operate.

  The display control unit 42 may determine the scrolling speed according to the angle of inclination of the mobile terminal 10. For example, the larger the tilt, the faster the scrolling may be. The display control unit 42 may stop scrolling when the inclination of the mobile terminal 10 becomes smaller than a predetermined angle. The display control unit 42 may display the shortened name of the icon in the name display area 94 again when the state where the inclination of the mobile terminal 10 is smaller than a predetermined angle continues for a predetermined time or longer. The display control unit 42 may stop scrolling when a predetermined operation such as a tap operation in the name display area 94 is accepted while scrolling the name of the icon.

1-5. UI that switches the background image when the mobile terminal is tilted
15 and 16 are diagrams for explaining the function of switching the background image. On the desktop screen shown in FIG. 15, an image when the aquarium is viewed from above is displayed as the background image 95. On the desktop screen shown in FIG. 16, a landscape image is displayed as the background image 96. When the mobile terminal 10 has an inclination less than a predetermined angle from the horizontal, the display control unit 42 displays an image when a certain object is viewed from above as a background image 95 as shown in FIG. When the image is tilted by a predetermined angle or more from the horizontal so that the upper side of the image is on the upper side, as shown in FIG. The background image 95 and the background image 96 are stored in the data holding unit 60 in advance, and the display control unit 42 reads the background image 95 or the background image 96 from the data holding unit 60 according to the inclination of the mobile terminal 10. indicate. As a result, when the user is holding the portable terminal 10 in a nearly horizontal state or placing it on a desk or the like and looking down at the desktop screen of the display device 68 from above, a certain object is looked into from above. A background image 95 is displayed as a background image 95, and when the user lifts the mobile terminal 10 and holds it while tilting it at a predetermined angle or more from the horizontal, an image that looks at a certain landscape is displayed as the background image 96. Therefore, an image that matches the user's line of sight can be displayed in the background, as if there is an object displayed as the background image 95 under the user's feet, and the landscape displayed as the background image 96 is spreading around the user. It is possible to produce like this. Thereby, a highly entertaining user interface can be provided.

  Three or more background images may be stored in the data holding unit 60 and may be switched and displayed according to the inclination of the mobile terminal 10. The position and shape data of the three-dimensional objects constituting the background are stored in the data holding unit 60, the display control unit 42 reads them, and sets the camera viewpoint position and line-of-sight direction according to the inclination of the mobile terminal 10. The background image may be rendered. In this case, the display control unit 42 may continuously change the background image according to the inclination of the mobile terminal 10.

1-6. UI to jump icon when mobile device is flipped up
17 and 18 are diagrams for explaining a function of jumping an icon. When the user moves the mobile terminal 10 vertically upward with respect to the screen of the display device 68 at an acceleration equal to or higher than a predetermined value, the user stops or moves vertically downward as shown in FIG. In addition, the display control unit 42 displays a state in which the icon 91 jumps. The display control unit 42 acquires a value of acceleration that is vertically upward with respect to the screen of the display device 68 of the mobile terminal 10 from the posture position calculation unit 43, and if it is greater than or equal to a predetermined value, the force calculated from the acceleration. Is applied to the icon 91 to cause the motion calculation unit 44 to calculate the behavior of the icon 91 jumping. At this time, the motion calculation unit 44 may read physical quantities such as the mass, rotation moment, and air resistance of the icon 91 from the data holding unit 60 and use them in the calculation. The behavior of the icon may be varied depending on the attribute of the icon, for example, the type of the data file or application corresponding to the icon, the amount of data, the creation date / time, the modification date / time, the activation date / time, the presence / absence of new arrival information, and the like. For example, the smaller the data amount, or the newer the creation date, modification date, or activation date, the smaller the mass, the lower the value of virtual gravity acceleration, or the air The resistance value may be reduced. Further, an icon set as a favorite by the user or an icon having new arrival information may be jumped higher than other icons.

  The display control unit 42 displays an icon using the posture and position of the icon calculated by the motion calculation unit 44, and displays the icon in a visually easy-to-understand manner by blurring the background image. In addition to the background image, the blurred background image is stored in the data holding unit 60, and the display control unit 42 reads the blurred background image from the data holding unit 60, and blurs while the icon jumps. You may switch to a background image, and you may blur a background image using a known image processing technique. In the former case, a plurality of types of background images are stored in the data holding unit 60 in accordance with the height at which the icon is jumping, and the display control unit 42 may switch the background image in accordance with the height of the icon. Good. In the latter case, the degree of blurring may be increased as the height of the icon is higher.

  The display control unit 42 may express how the icon 91 jumps by enlarging the image of the icon 91. The display control unit 42 may express a state in which the icon 91 jumps by bringing the viewpoint position when the icon 91 is rendered closer to the icon 91.

  When the user performs a predetermined operation input such as tapping the desktop screen 90 with a finger while the icon 91 is being flipped up, as shown in FIG. When the operation input is accepted, the icon 91 is stopped. Thereby, since the icon 91 can be enlarged and it can be made easy to see, a user's convenience can be improved. In addition, it is possible to provide a novel and interesting user interface in which the display area is not simply zoomed up, but is enlarged by jumping up and stopping the icon.

  The display control unit 42 may freeze all icons or a specific icon. For example, when the user taps an area on the desktop screen 90 where the icon 91 is not displayed, all icons may be stopped, and when a specific icon is tapped, only that icon may be stopped. The display control unit 42 may determine the icon to be stopped according to the attribute of the icon. For example, only icons set as favorites by the user may be stopped.

  The display control unit 42 is not limited to the vertical upward acceleration with respect to the display screen of the display device 68, but when the acceleration of a predetermined amount or more is applied to the mobile terminal 10 or the angular acceleration of the predetermined amount or more is applied. A force calculated using the acceleration or angular acceleration calculated by the position calculation unit 43 may be applied to the icon 91, and the behavior of the icon 91 may be calculated by the motion calculation unit 44 and reflected on the display. For example, the icon may be shaken as in the example 1-1, the icon may be moved, or the icon may be rotated as in the example 1-2.

  Each of the above functions may be used alone for the mobile terminal 10, or any combination of two or more functions may be used for the mobile terminal 10.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  DESCRIPTION OF SYMBOLS 10 Mobile terminal, 40 Control part, 41 User interface part, 42 Display control part, 43 Posture position calculation part, 44 Motion calculation part, 45 Illumination calculation part, 46 Input reception part, 48 Application execution part, 60 Data holding part, 62 Icon table, 68 display device, 69 touch panel, 70 rear touch panel, 72 3-axis gyro sensor, 73 3-axis acceleration sensor.

Claims (6)

A display device;
A sensor for detecting the attitude of the mobile terminal;
A holding unit for holding data;
A display control unit that displays an icon or an object corresponding to the data held in the holding unit on the display device,
The display control unit acquires information related to the orientation of the mobile terminal , changes the orientation of the icon or object according to the tilt of the mobile terminal while fixing the display position of the icon or object , and Or, when an instruction input to freeze the icon or object is received in an area where no object is displayed, all the displayed icons or objects are stopped and a specific icon or object is displayed. A mobile terminal characterized in that when the instruction input is received in an area, the specific icon or object is stopped . A display device;
A sensor for detecting the attitude of the mobile terminal;
A holding unit for holding data;
A display control unit that displays an icon or an object corresponding to the data held in the holding unit on the display device,
The display control unit acquires information about the attitude of the mobile terminal, moves the icon or object within the screen of the display device according to the tilt of the mobile terminal, and the icon or object is not displayed When an instruction input for stopping the icon or object in the area is received, all the displayed icons or objects are stopped, and the instruction input is received in the area where the specific icon or object is displayed. If so, the mobile terminal is characterized in that the specific icon or object is stationary .   When the display control unit has not received an operation input from the user for a predetermined time or when the function of receiving the operation input from the user is temporarily stopped, the display control unit 3. The portable terminal according to claim 2, wherein an object is moved, and in other cases, the icon or object is displayed so as to shake within a predetermined range. A display control unit acquiring information related to the attitude of the mobile terminal from a sensor for detecting the attitude or position of the mobile terminal;
When the display control unit displays an icon or object corresponding to the data held in the holding unit holding the data on the display device, the display position of the icon or object is fixed according to the inclination of the mobile terminal. While changing the orientation of the icon or object, or moving the icon or object within the screen of the display device ;
When the display control unit receives an instruction input for stopping the icon or object in an area where the icon or object is not displayed, all the displayed icons or objects are stopped When the instruction input is received in the area where the icon or object is displayed, the step of stopping the specific icon or object;
A display method comprising: A function that the display control unit acquires information on the attitude or position of the mobile terminal from a sensor for detecting the attitude or position of the mobile terminal;
When the display control unit displays an icon or object corresponding to the data held in the holding unit holding the data on the display device, the display position of the icon or object is fixed according to the inclination of the mobile terminal. While changing the orientation of the icon or object, or moving the icon or object within the screen of the display device ;
When the display control unit receives an instruction input for stopping the icon or object in an area where the icon or object is not displayed, all the displayed icons or objects are stopped When the instruction input is received in the area where the icon or object is displayed, the function of stopping the specific icon or object;
A computer program for realizing a computer.   The computer-readable recording medium which recorded the computer program of Claim 5.