JP2012252384A - Screen control system, screen control method, and screen control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly display a menu icon 301 on a screen with excellent visibility and operability, thereby enabling intuitive operation.SOLUTION: A screen control system generates an image or the like having the same or a similar shape as the shape of a display screen as a menu icon 301. The system forms a virtual sphere 310 in a three-dimensional virtual space, and rotates or moves the sphere 310 in a virtual manner in response to an operation signal. The system calculates the position of the menu icon 301 on the sphere 310 and the apparent deformation of the menu icon 301 due to the rotation or movement of the sphere 310, the menu icon 301 disposed at a predetermined position on a surface of the sphere 310. The system modifies the menu icon 301 in accordance with the deformation and displays it on the display screen. The system obtains selection operation to the menu icon 301 in accordance with change in the coordinate position of the menu icon 301 due to the rotation or movement of the sphere 310 in response to the operation signal.

Description

  The present invention relates to a screen control system, a screen control method, and a screen control program for appropriately displaying a display screen of an information processing apparatus such as a mobile phone.

  In recent years, with regard to information processing apparatuses such as mobile phones, a graphical user interface (GUI) is known as a user interface that provides users with an operating environment that has excellent visibility and operability and allows intuitive operation. ing. The GUI regards a computer bitmap screen as a desktop, arranges objects such as windows, icons, and menus on the desktop, and freely operates the arranged objects with a pointing device such as a mouse and a touch pad. Today, a three-dimensional GUI in which an operating environment like a real world is constructed by expressing a desktop in a three-dimensional virtual space has been put into practical use (for example, Patent Document 1).

Japanese Patent Laid-Open No. 2000-107772

  Meanwhile, in recent years, information terminals such as mobile phone terminals and personal digital assistants (PDAs) are required to be miniaturized, and on the other hand, multi-functionality such as transmission / reception of e-mails, connection to the Internet, and camera shooting has progressed. In many cases, the selection items such as a large number of icons and files are increased and cannot be displayed on the display screen.

  In particular, in recent years, with the development of liquid crystal displays, 3D liquid crystal displays that can be viewed stereoscopically even with the naked eye have been developed. By using a three-dimensional virtual space, the operability and visibility are not impaired on the screen. Development of a GUI that displays a large amount of information is desired.

  Therefore, the present invention can appropriately display a menu icon displayed on a screen by using a three-dimensional virtual space in an information terminal such as a mobile phone or a mobile terminal. It is an object of the present invention to provide a screen control system, a screen control method, and a screen control program that are excellent and can perform intuitive operations.

In order to solve the above problems, the present invention provides a screen control system for a display screen of an information processing apparatus,
An icon generator that generates screen information displayed on the display screen as a menu icon that is an image or moving image that is the same or similar to the screen shape of the display screen;
A virtual polyhedron forming unit that virtually forms a polyhedron on a three-dimensional virtual space and arranges a frame on which the menu icon is pasted on the polyhedron surface;
An operation signal acquisition unit that acquires an operation signal by an operator and moves the frame along the surface of the polyhedron according to the operation signal;
A deformation calculation unit that calculates an apparent deformation associated with a position on the polyhedron and movement or rotation of the polyhedron of the frame arranged at a predetermined position on the polyhedron surface;
An icon display unit that processes the menu icon in accordance with the deformation of the frame calculated by the deformation calculation unit, pastes the menu icon on the frame, and displays the icon on the display screen.
A selection operation acquisition unit configured to acquire a selection operation for the menu icon according to a change in a coordinate position of the menu icon accompanying rotation or movement of the polyhedron according to the operation signal.

Another invention is a screen control method for a display screen of an information processing apparatus,
(1) A polyhedron is virtually formed on a three-dimensional virtual space, a frame on which the menu icon is pasted is disposed on the surface of the polyhedron, an operation signal from the operator is acquired, and the operation signal is And a virtual polyhedron operation step of moving the frame along the polyhedron surface according to
(2) An icon generation step for generating screen information displayed on the display screen as a menu icon that is an image or moving image that is the same or similar to the screen shape of the display screen;
(3) While calculating the position on the polyhedron of the frame arranged at a predetermined position on the surface of the polyhedron and the apparent deformation accompanying the movement or rotation of the polyhedron, the frame is adjusted according to the calculated deformation of the frame. An icon display step of processing a menu icon, pasting it on the frame, and displaying it on the display screen;
(4) A selection operation acquisition step of acquiring a selection operation for the menu icon according to a change in the coordinate position of the menu icon accompanying rotation or movement of the polyhedron according to the operation signal.

  According to the present invention as described above, by moving or rotating a virtually formed polyhedron in a three-dimensional virtual space, a menu icon on the polyhedron can be easily moved and an arbitrary menu can be moved. An icon can be selected. At this time, for the menu icon on the surface of the polyhedron, the position on the polyhedron and the apparent deformation accompanying the movement or rotation of the polyhedron are calculated, so that the image can be appropriately displayed according to the stop position and visually Intuitive operation is possible.

  In particular, even if the polyhedron is moved and rotated by expressing the frame on the polyhedron according to the latitude and longitude and pasting the menu icon there, it can be operated to rotate the polyhedron vertically. The poles of the polyhedron are not visible, and the menu icons can continue to rotate indefinitely in the matrix arrangement order. In addition, when the image information displayed at the end portion is moved to the center portion, the reduced end portion image is not enlarged and displayed. Can be appropriately performed, and the end portion can be prevented from being blurred or extended.

  In the above invention, further comprising a touch panel in which an operation signal is input by pressure by a touch operation, the operation signal acquisition unit acquires the operation signal through the touch panel, and according to the displacement amount of the input position of the operation signal on the touch panel, It is preferable to change the virtual rotation angle or rotation speed of the polyhedron.

  In this case, since the rotation angle or rotation speed is changed according to the displacement amount of the input position of the operation signal, for example, when the displacement amount is small, the menu icon is selected by reducing the rotation speed. If the amount of displacement is large, the menu icon that was not initially displayed on the screen can be instantly discovered by increasing the rotation speed of the menu icon. Can be made.

  In the above invention, it is preferable that the operation signal acquisition unit displays the polyhedron in an enlarged manner according to the operation signal, and causes the icon display unit to display the menu icons so as to be arranged on a plane in the virtual space. When the polyhedron is enlarged and displayed in this way, the menu icons are displayed as if they are arranged on a plane in the virtual space, so that the user can perform an operation without feeling uncomfortable.

In the above invention, the menu icon is associated with various functions and data of the information processing apparatus, the menu icons are arranged in a matrix according to the attributes of the function and data, and the operation signal acquisition unit is a polyhedron according to the operation signal. It is preferable to increase or decrease the number of menu icons displayed in the menu icon display section by enlarging or reducing the size. In this case, even when the polyhedron is rotated or moved, the related menu icons are displayed on the screen without shifting the arrangement of the menu icons. Further, the information terminal and the screen control method thereof according to the present invention described above can be realized by executing a screen control program described in a predetermined language on a computer. That is, the function module of the information terminal of the present invention can be virtually built on the CPU by installing this screen control program on a computer such as a user terminal or a Web server or an IC chip and executing it on the CPU. The screen control method of the present invention can be implemented by operating this information terminal.

Specifically, it is a screen control program for the display screen of the information processing device, and the arithmetic processing device of the information processing device
(1) A polyhedron is virtually formed on a three-dimensional virtual space, a frame on which the menu icon is pasted is disposed on the surface of the polyhedron, an operation signal from the operator is acquired, and the operation signal is And a virtual polyhedron operation step of moving the frame along the polyhedron surface according to
(2) An icon generation step for generating screen information displayed on the display screen as a menu icon that is an image or moving image that is the same or similar to the screen shape of the display screen;
(3) While calculating the position on the polyhedron of the frame arranged at a predetermined position on the surface of the polyhedron and the apparent deformation accompanying the movement or rotation of the polyhedron, the frame is adjusted according to the calculated deformation of the frame. An icon display step of processing a menu icon, pasting it on the frame, and displaying it on the display screen;
(4) A selection operation acquisition step of acquiring a selection operation for the menu icon according to a change in the coordinate position of the menu icon according to the rotation or movement of the polyhedron according to the operation signal is executed.

  The screen control program can be recorded on a recording medium that can be read by a general-purpose computer, for example, and can be distributed through a communication line. It can be transferred as a packaged application that runs on a computer.

  According to the computer-readable recording medium on which the screen control program is recorded, the information terminal and the screen control method described above can be implemented using a general-purpose computer or a dedicated computer, and the program can be saved. , Transportation and installation can be done easily.

  According to the present invention, in an information terminal such as a mobile phone, a menu icon displayed on a screen can be appropriately displayed using a three-dimensional virtual space, and has excellent visibility and operability. Intuitive operation can be performed.

1 is an external view of a mobile communication terminal 1 according to an embodiment. It is explanatory drawing which shows the example of operation of GUI which concerns on embodiment. It is explanatory drawing which shows the example of operation of GUI which concerns on embodiment. It is a block diagram which shows the internal structure of the mobile communication terminal 1 which concerns on embodiment. It is a block diagram which shows the internal structure concerning the screen control of the portable terminal which concerns on embodiment. It is explanatory drawing which shows the state of the menu icon arrange | positioned on the surface of the sphere which concerns on embodiment. It is explanatory drawing which shows the relationship between the spherical body at the time of operation of GUI which concerns on embodiment, and a menu icon. It is a flowchart figure which shows the operation | movement of the screen control which concerns on embodiment. It is a flowchart explaining the rotation and movement control method in step 111.

  Exemplary embodiments of an information terminal according to the present invention will be described below in detail with reference to the accompanying drawings. FIG. 1 is an external view of a mobile communication terminal 1 according to the present embodiment. As shown in FIG. 1, the mobile communication terminal 1 includes a rectangular main body 100, and a front surface of the main body 100 is provided with a touch panel 300.

(Overview of touch panel display / operation)
The touch panel 300 is an input device that inputs an operation signal by pressure by a touch operation using a user's fingertip or a pen. The touch panel 300 displays an LCD 165 that displays a graphic and an operation signal corresponding to the coordinate position of the graphic displayed on the LCD 165. The touch sensor 168 to be received is superimposed. Note that the mobile communication terminal 1 according to the present embodiment includes an operation button 166 such as a button switch or a direction key in addition to the touch panel 300, and an operation signal can be input using the operation button 166. It has become.

  Here, on the touch panel 300, a general two-dimensional normal display and a sphere home display using a three-dimensional sphere 310 are possible. That is, in the sphere home display, as shown in FIG. 1, a sphere 310 is virtually formed in a three-dimensional virtual space on the graphic screen displayed by the LCD 165, and on the surface of the sphere 310. A plurality of selection items that can be selected by a touch operation are displayed as menu icons 301. The menu icons 301 are such that the sphere 310 is virtually rotated or moved in the vertical and horizontal directions or in an oblique direction by a user operation on the touch panel 300, and the menu icon 301 is moved as the sphere 310 is moved or rotated. It has become. Here, as shown in FIG. 1, the menu icon 302 arranged in the center portion of the screen is displayed as the same shape or similar shape to the screen shape during normal display, and when moved from the center to the outside, the menu icon 302 is moved. Or, along with the rotation, an apparent deformation is made based on the perspective method.

  Further, in the present embodiment, the sphere 310 in the sphere home display is configured such that its radius is expanded or contracted according to a user operation and is expanded or contracted, and the menu icon 301 is displayed according to the expansion or contraction of the sphere 310. The number is increased or decreased. For example, in FIG. 2A, only the standby screen 304 is displayed as the normal screen. When the icon 303 for displaying the menu screen on the standby screen 304 is selected in this state, the sphere home display by the sphere 310 is displayed from the normal display. As shown in FIG. 2B, a screen on which menu icons for 3 vertical columns × 3 horizontal columns are arranged is displayed. In the sphere home display shown in FIG. 2 (b), the sphere 310 has a radius that is maximized and appears to be approaching, and a screen of 3 squares by 3 squares appears. It is displayed as if it were placed on a plane in the virtual space. At this time, by the cancel operation, the radius of the sphere 310 is extended, and the screen can be returned to the standby screen 304 as the original normal screen that is apparently enlarged (approached).

  Further, when the user selects the central square on the screen, as shown in FIG. 2C, the radius of the sphere 310 is reduced, and an apparently retreated screen is displayed. A screen of 7 squares is displayed. The selection operation by the user includes a drag operation, a pinch-in operation, and other editing menu selection operations. In this apparently retreated state, as shown in FIG. 2 (c), the menu icons 301 corresponding to the vertical 7 squares and the horizontal 7 squares are arranged on the surface of the three-dimensional sphere 310. It is displayed. In the state of FIG. 2C, the radius of the sphere 310 is expanded and apparently enlarged (approached) by releasing the finger during the drag operation from the surface, or performing a pinch-out operation or a cancel operation. ) You can return to the original screen.

  Here, the menu icon 301 is associated with various functions and data of the information processing apparatus, and the menu icon 301 is arranged in a matrix according to the function and data attributes, and the screen is enlarged. Even if it is reduced, the displayed arrangement is controlled to be the same as the arrangement order on the matrix. Note that the arrangement order on the matrix can be changed based on a user operation. For example, two menu icons can be selected by long-pressing the menu icon, and a drag and drop operation can be performed. These menu icons can be exchanged or added to a blank position (an icon indicated by “+” in the figure).

  Further, in the present embodiment, when the user performs a slide operation on the screen on which the sphere 310 is displayed, the virtual rotation angle of the spherical surface 310 or the amount of movement (movement distance) from the slide start position of the operation is determined. The rotation speed changes. More specifically, as shown in FIGS. 6 and 7, if the user touches the finger on the LCD 165 and slides it from the start position by ΔL, the sphere 310 is rotated by the rotation angle ΔD. At this time, in the present embodiment, the rotation angle or rotation speed is not increased in proportion to the displacement amount, but the rotation angle or rotation speed is set to increase as the displacement amount increases. For example, as ΔL increases, ΔD increases as a quadratic function, and a setting can be made such that the rotation angle is twice the maximum moving distance on the circumference.

(Internal structure of mobile communication terminal)
A specific internal configuration of the mobile communication terminal 1 including the touch panel capable of display / operation described above will be described in detail below. FIG. 4 is a block diagram showing an internal configuration of the mobile communication terminal 1 according to the present embodiment. As shown in the figure, the mobile communication terminal 1 includes a duplexer 102 connected to an antenna 101, and a reception system module and a transmission system module connected to the duplexer 102.

  The reception system module includes a low noise amplifier 110, a mixer 111, an IF amplifier 112, an orthogonal mixer 113, an A / D converter 114, a demodulator 115, a channel decoder 116, an audio decoder 117, a D / A converter 118, and an amplifier 119 with a switch. And a speaker 120. On the other hand, the transmission system module includes a microphone 140, an amplifier 139, an A / D converter 138, an audio encoder 137, a channel encoder 136, a modulator 135, a D / A converter 134, an orthogonal mixer 133, an IF amplifier 132, a mixer 131, and A power amplifier 130 is provided.

  The mobile communication terminal 1 includes a synthesizer 103, a time base 150, a CPU 200, a RAM 152, a ROM 153, and an EEPROM 151 as control system modules, and includes an acceleration sensor 164, an LCD 165, operation buttons 166, an LED 167, and a touch sensor as user interface modules. 168 and a vibrator 174, and further includes a power supply system battery 171, a power supply 172, and an A / D converter 173 as a power supply system module.

  The antenna 101 transmits and receives signals to and from a base station (not shown) via a radio wave line. The duplexer 102 is a circuit that switches between input and output of a signal to be transmitted and received. The duplexer 102 inputs a signal received by the antenna 101 to the low noise amplifier 110 and outputs a signal output from the power amplifier 130 to the antenna 101.

  In the reception system module, the low noise amplifier 110 amplifies the signal input from the duplexer 102 and outputs the amplified signal to the mixer 111. The mixer 111 receives the output of the low noise amplifier 110, separates it by a specific frequency, and outputs it as an intermediate frequency signal. The IF amplifier 112 amplifies the intermediate frequency signal output from the mixer 111. The orthogonal mixer 113 receives the output of the IF amplifier 112 and performs orthogonal demodulation. The A / D converter 114 digitizes the output of the orthogonal mixer 113. The demodulator 115 demodulates the output of the A / D converter 114. Channel decoder 116 performs error correction on the output of demodulator 115. The error-corrected signal includes a control message and voice data. The control message is sent to the CPU 200 and the voice data is sent to the voice decoder 117.

  The signal input from the channel decoder 116 to the audio decoder 117 is decoded into audio data and delivered to the D / A converter 118. The D / A converter 118 converts the output of the audio decoder 117 into an analog signal. The switch-equipped amplifier 119 is switched at an appropriate timing based on a control signal from the CPU 200 and amplifies the output of the D / A converter 118 when the switch is ON. The speaker 120 amplifies the output of the switch-equipped amplifier 119.

  On the other hand, in the transmission module, the microphone 140 receives an audio signal from the user and outputs the audio signal as an analog signal. The amplifier 139 amplifies the analog signal output from the microphone 140. The A / D converter 138 converts the output of the amplifier 139 into a digital signal. The audio encoder 137 encodes and compresses the output of the A / D converter 138 and outputs it as audio data. The channel encoder 136 combines the control message from the CPU 200 and the audio data from the audio encoder 137 and adds an error correction code.

  The modulator 135 modulates the output of the channel encoder 136. The D / A converter 134 converts the output of the modulator 135 into an analog signal. The orthogonal mixer 133 converts the output of the D / A converter 134 into an IF frequency signal (intermediate frequency signal). The IF amplifier 132 amplifies the output of the quadrature mixer 133. The mixer 131 increases the frequency of the signal output from the IF amplifier 132. The power amplifier 130 amplifies the output of the mixer 131.

  The synthesizer 103 synchronizes the mixer 111, the quadrature mixer 113, the mixer 131, and the quadrature mixer 133 during communication. The time base 150 supplies a clock signal to each unit.

  In the user interface system, the acceleration sensor 164 is a sensor that detects the magnitude and direction of acceleration. The LCD 165 is a liquid crystal display that displays messages, input characters, and the like to the user. The GUI on the touch panel 300 can display graphics such as characters, graphics, and moving images through the LCD 165, and obtains an operation signal through the touch sensor 168 on the touch panel 300.

  The LED 167 is for transmitting a message to the user by turning on and off. The touch sensor 168 detects that the user's finger has touched the surface of the touch panel, and acquires an operation signal by pressure applied to the surface of the touch panel 300. The vibrator 174 is a device that notifies an incoming call, and vibrates when the incoming call is received.

  The power system battery 171 supplies power to the power source 172 and the A / D converter 173. A power source 172 is a power source of the mobile communication terminal 1. The A / D converter 173 supplies a signal to the CPU 200.

  The CPU 200 is an arithmetic processing unit that controls each of the above-described units, and sequentially executes the commands of the program stored in the ROM 153 to perform various functions. The RAM 152 is used as a working memory or the like for the CPU 200, and temporarily stores a calculation result by the CPU 200. Various function programs and data for the CPU 200 are recorded in the ROM 153, and program execution instructions are sequentially output in response to a request from the CPU 200.

  The EEPROM 151 stores user data such as abbreviated dials, machine IDs and telephone numbers, menu icon information, and display information thereof. The menu icons are associated with various functions and data of the information processing apparatus, and are arranged in a matrix on the screen according to the functions and data attributes.

  In the present embodiment, various modules are virtually constructed on the CPU 200 by the CPU 200 executing software such as the screen control program of the present invention.

  Specifically, by executing an application including a screen control program on the display screen of the information processing apparatus of the present invention in the CPU 200, a module by the home application 201 that controls the menu screen as shown in FIG. Virtually constructed. Here, the “module” used in the description refers to a functional unit that is configured by hardware such as an apparatus or device, software having the function, or a combination thereof, and achieves a predetermined operation. Show.

  In the home application 201, modules of a display information generation unit 202, an operation signal acquisition unit 205, and an application control unit 203 are virtually constructed.

  The operation signal acquisition unit 205 is a module that acquires an operation signal from the operator from the input interface (I / F) 175 and controls activation of a menu screen or an application in accordance with the operation signal. The operation signal on 300 is acquired, the coordinate position of the operation signal is collated, the operation command associated with the collation result and the operation signal is selected, and the application control unit 203 or the display information generation unit 202 is selected. input.

  The operation signal acquisition unit 205 includes an operation position detection unit 205a, a deformation calculation unit 205b, and a selection operation acquisition unit 205c. The operation position detection unit 205a is a module that detects the coordinate position of an operation signal input to the touch panel 300. The coordinate point is detected based on the input coordinate position of the operation signal detected by the touch sensor 168. In the present embodiment, when a predetermined position is selected according to the operation signal, the sphere 310 is enlarged / reduced to be displayed, and in the maximum enlarged display state, the radius R of the sphere 310 is set to be infinite, and display information generation is performed. The unit 202 has a function of displaying the menu icon 301 so as to be arranged on a plane in the virtual space.

  The deformation calculation unit 205b is a module that measures the input duration time at a predetermined coordinate position or calculates the displacement amount of the coordinate position of the operation signal by the touch operation. In particular, in the present embodiment, according to the operation signal, An apparent deformation of the menu icon 301 arranged at a predetermined position on the surface of the sphere 310 is calculated as the sphere moves or rotates.

  More specifically, the sphere 310 is not rotated and moved by pasting a menu icon as a texture on the surface of the sphere like a conventional polygon, but as shown in FIGS. 6 (a) and 6 (b), Conceptually, the frame 311 on which the menu icon 301 is pasted moves along the surface of the sphere 310 while being deformed according to the coordinate positions P1 to P3 indicated by the latitude and longitude, and is adapted to the deformation in a rectangular shape. Menu icon 301 is pasted. In accordance with the latitude and longitude, the deformation of the frame 311 approaches the original shape (similar to the shape of the display screen) as it approaches the equator, and the pole side becomes narrower and closer to a triangle as it approaches the pole. Therefore, the apparent deformation of the frame 311 arranged at a predetermined position on the surface of the spherical surface 310 due to the movement or rotation of the polyhedron 310 is based on the deformation on the spherical surface according to the latitude and longitude and the perspective in 3D representation. The deformation is a combined deformation. By adopting a method of deforming the frame on the spherical surface in accordance with the latitude and longitude and pasting the menu icon there, the pole of the sphere 310 can be operated even if the sphere 310 is rotated up and down. 310a is not visible, and the menu icons can continue to rotate indefinitely in the order of the matrix arrangement.

  In addition, the deformation calculation unit 205b has a function of increasing or decreasing the number of menu icons displayed in the display information generation unit 202 by enlarging or reducing the sphere 310 according to the operation signal. For example, as shown in FIG. 2A, when the icon 303 for displaying the menu screen on the standby screen 304 in the normal screen is selected, the screen is switched to the spherical home display as shown in FIG. A screen of 3 vertical cells × 3 horizontal cells arranged on an infinite plane is displayed. At this time, it is possible to return to the original normal screen by the cancel operation. Further, when the user selects the central square on the screen, as shown in FIG. 2C, the radius of the sphere 310 is reduced, and a screen of vertical 7 squares × horizontal 7 squares is displayed.

  As described above, in the round display, when the sphere 310 is enlarged to the maximum and apparently approached, as shown in FIG. 2B, menu icons 301 arranged in a vertical 3 × horizontal 3 square are displayed, At that time, the sphere 310 is displayed so as to be arranged on a plane at infinity in the virtual space so that the radius R of the sphere 310 becomes infinite. Then, when the radius of the sphere 310 is reduced and apparently retreated, the menu icon 301 is displayed as being arranged on the surface of the three-dimensional sphere 310 as shown in FIG. It has come to be.

The deformation calculation unit 205b also has a function of changing the virtual rotation angle or rotation speed of the sphere 310 according to the amount of displacement of the input position of the operation signal when the sphere home is displayed. More specifically, when the user performs a slide operation on the screen on which the sphere 310 is displayed, as shown in FIGS. 6 and 7, it is assumed that the user slides the LCD 165 by ΔL from the start position where the user touched the finger. Then, the sphere 310 is rotated by the rotation angle ΔD. At this time, in the present embodiment, the rotation angle or rotation speed is not increased in proportion to the displacement amount, but the rotation angle or rotation speed is set to increase as the displacement amount increases. For example, when ΔL increases, ΔD increases as a quadratic function, and the maximum rotation angle can be set to twice the movement distance on the circumference. The selection operation acquisition unit 205c This module acquires a selection operation for a menu icon according to a change in the coordinate position of the menu icon accompanying rotation or movement of a polyhedron according to a signal. Specifically, as illustrated in FIGS. 3A and 3B, when the user moves the menu icon 301, the selection operation acquisition unit 205 c displays predetermined coordinates (in the present embodiment, the coordinate center). ) Automatically determines that the user has selected the menu icon located at. In the present embodiment, when the menu icon 301 is moved while being touched, the operation is regarded as a drag operation, and the automatic selection processing at the coordinate center is not executed.

  The input interface (I / F) 175 is a module that receives a user operation signal input from the touch sensor 168, the operation button 166, and other operation devices and inputs the operation signal to the operation signal acquisition unit 205.

  The display information generation unit 202 is a module that generates display information (image, character information, etc.) to be displayed on the LCD 165, and constitutes a GUI in cooperation with the operation signal acquisition unit 205. The display information generation unit 202 includes a virtual polyhedron formation unit 202a, an icon generation unit 202b, and an icon display unit 202c.

  The virtual polyhedron forming unit 202a is a module that virtually forms the sphere 310 in a three-dimensional virtual space. In this embodiment, as shown in FIGS. 6 and 7, a sphere 310 having a radius R centered on C1 is set, and menu icons 301 are arranged on the surface thereof. In the present embodiment, the sphere 310 is virtually formed. However, the present invention is not limited to this, and any three-dimensional shape may be used. For example, various shapes such as an ellipsoid and a polyhedron may be used. A shape may be used.

  As illustrated in FIG. 6, the icon generation unit 202 b is a module that generates screen information displayed on the display screen as a menu icon that is an image or a moving image that is the same or similar to the screen shape of the display screen. In particular, in this embodiment, the icon generation unit 202b menu icon 301 is associated with various functions and data of the information processing apparatus. The icon display unit 202c is a module that processes menu icons in accordance with the deformation calculated by the deformation calculation unit 205b and displays them on the display screen. The menu icons 301 are arranged in a matrix according to functions and data attributes. Is controlled to be.

  In addition, the display information generation unit 202 displays the display information according to the coordinate point (relative position on the touch panel 300) to which the operation signal is input, detected by the operation position detection unit 205a, and the acceleration detected by the acceleration sensor 164. It also has a function to change

  The application control unit 203 is a module that executes another application 210 in accordance with the control signal input from the operation signal acquisition unit 205. In particular, in the present embodiment, when a selection operation for the menu icon 301 is acquired by an input from the selection operation acquisition unit 205c, another application 210 corresponding to the menu icon 301 is activated.

(Screen control method)
By executing the above-described program, the screen control method can be implemented. FIG. 8 is a flowchart showing the screen control method of the mobile communication terminal 1 according to the present embodiment. Here, the mobile communication terminal 1 has many functions such as transmission / reception of e-mail, Internet connection, camera shooting, video / image browsing, game, and the like. A case of selecting these will be described.

  First, as shown in FIG. 2A, when the user performs an input to the touch panel 300 with a finger or a pen tip while the standby screen 304 is displayed in the normal screen display, the input is made. The operation signal is input to the operation signal acquisition unit 205 through the input interface 175 and detected as an operation signal (S101). Then, the operation position detection unit 205a detects the coordinate position of the operation signal input to the touch panel 300 from this operation signal (S102).

  Here, as shown in FIG. 2A, the operation position detection unit 205a determines whether or not the icon 303 for displaying the menu screen is selected based on the coordinate position (S103). If the icon 303 is not selected (“N” in S103), the next operation input is received, and a standby state is entered until an operation signal is detected. On the other hand, when the icon 303 is selected (“Y” in S103), a menu screen with a spherical home display is displayed (S104).

  Specifically, as shown in FIG. 2B, the virtual polyhedron forming unit 202a virtually forms a sphere 310 in a three-dimensional virtual space, and the icon generating unit 202b displays a display screen on the sphere 310. A menu icon 301 having the same shape as or similar to the screen shape is displayed. In the present embodiment, a screen of vertical 3 squares × horizontal 3 squares is displayed, and the menu icon 301 is displayed so as to be arranged on a plane in the virtual space.

  Next, when the next operation signal is input, the input operation signal is input to the operation signal acquisition unit 205 through the input interface 175 and detected as an operation signal (S105). Then, the operation position detection unit 205a detects the coordinate position of the operation signal input to the touch panel 300 from the operation signal.

  Here, as illustrated in FIG. 2B, the operation position detection unit 205a determines whether or not the menu icon 302 at the center of the coordinate is selected based on the coordinate position (S106). Here, when the menu icon 302 is not selected at the coordinate center (“N” in S106), for example, in the case of a drag operation for moving the menu icon 301 while touching, the automatic selection processing at the coordinate center is not executed. . On the other hand, in the case of automatic selection at the coordinate center of the menu icon 302 (“Y” in S106), the operation signal acquisition unit 205 transmits the acquired operation signal to the deformation calculation unit 205b.

  The deformation calculation unit 205b measures the input duration at a predetermined coordinate position (S107), and determines whether the input duration is constant (S108). Here, if the input duration is within a certain time (“N” in S108), the display of the sphere 310 is not changed (S110), and it is determined whether or not there is an operation input while keeping the enlarged display (S110). S112).

  Here, when there is no operation input (“N” in S112), there is a next operation input, and a standby state is entered until an operation signal is detected. When there is an operation input (“Y” in S112), the activation processing of another application 210 is executed in accordance with the operation input (S113).

  On the other hand, when the input duration is equal to or longer than the predetermined time (“Y” in S108), the virtual polyhedron forming unit 202a reduces the sphere 310, and the icon display unit 202c increases the number of display of the menu icons 301. (S109). Specifically, as shown in FIG. 2 (c), a screen with a reduced size of the sphere 310 is displayed, and a screen of vertical 7 squares × horizontal 7 squares is displayed.

  Then, there is a next operation input, and a standby state is entered until an operation signal is detected. When the operation signal acquisition unit 205 detects an operation signal, the user virtually rotates or moves the sphere 310 according to the operation signal (S111).

  Thereafter, the selection operation acquisition unit 205c determines whether there is a selection operation of another application 210 according to a change in the coordinate position of the menu icon 301 accompanying the rotation or movement of the sphere 310 (S112). Specifically, as shown in FIGS. 3A and 3B, it is determined whether the menu icon 302 positioned in the central icon portion is another application 210 as the sphere 310 rotates or moves. .

  If the menu icon 301 is not located in the central icon portion (“N” in S112), there is a next operation input, and a standby state is entered until an operation signal is detected. On the other hand, when the menu icon 301 is present in the central icon portion (“Y” in S112), the selection operation acquisition unit 205c acquires the selection operation for the menu icon 301, and sends another to the application control unit. The application activation process is executed (S113).

(Regarding rotation and movement control of the sphere 310)
Next, a method for controlling the rotation and movement of the sphere 310 executed in step 111 will be described. FIG. 9 is a flowchart for explaining a method for controlling the rotation and movement of the sphere 310 in step 111.

  First, as illustrated in FIG. 2C, when the user inputs an input to the touch panel 300 with a finger or a pen tip while the sphere 310 is displayed in a reduced size, the input operation signal is Then, it is input to the operation signal acquisition unit 205 through the input interface 175 and detected as an operation signal (S201). Then, the operation position detection unit 205a detects the coordinate position of the operation signal input to the touch panel 300 from this operation signal (S202).

  Next, when the user moves the finger or the pen tip to an arbitrary position while touching the touch panel 300, the moved operation signal is input to the deformation calculation unit 205b through the input interface 175. Then, the deformation calculation unit 205b calculates the displacement amount of the coordinate position from the coordinate position of the input start position and the coordinate position of the input stop position (S203).

  Then, the deformation calculation unit 205b determines whether or not the displacement amount is equal to or less than a predetermined value (S204), and changes the virtual rotation angle or rotation speed of the sphere 310 according to the displacement amount of the input position of the operation signal. . Specifically, when the displacement amount is equal to or less than the predetermined value (“Y” in S203), the operation signal acquisition unit 205 virtually displays the frame 311 along the surface of the sphere 310 at a low speed based on the displacement amount. (S205). On the other hand, if the amount of displacement is not less than or equal to the predetermined value (“N” in S203), the operation signal acquisition unit 205 virtually rotates or rotates the frame 311 along the surface of the sphere 310 based on the amount of displacement. Move (S206). Then, the operation signal acquisition unit 205 transmits the displacement amount data and deformation amount data of the frame 311 accompanying these rotations or movements to the icon display unit 202c.

  After that, the icon display unit 202c acquires data on the amount of displacement of rotation or movement, calculates the shape of the frame 311 as an apparent deformation accompanying the movement or rotation of the sphere 310, and selects a menu according to the calculated shape. The icon 301 is processed and pasted and displayed on the display screen (S207).

(Screen control program)
The information terminal and its screen control method described above can be realized by executing a screen control program written in a predetermined language on a computer. That is, the screen control program is installed as a menu application in an information terminal or mobile computer used by a user and executed on the CPU, thereby virtually building the above-described information terminal functional module on the CPU. In addition, the above-described screen control method can be implemented by operating this information terminal.

Specifically, in an information terminal having the touch panel 300,
(1) A polyhedron is virtually formed on a three-dimensional virtual space, a frame on which the menu icon is pasted is disposed on the surface of the polyhedron, an operation signal from the operator is acquired, and the operation signal is And a virtual polyhedron operation step of moving the frame along the polyhedron surface according to
(2) An icon generation step for generating screen information displayed on the display screen as a menu icon that is an image or moving image that is the same or similar to the screen shape of the display screen;
(3) calculating an apparent deformation of the frame arranged at a predetermined position on the surface of the polyhedron with movement or rotation of the polyhedron, and processing the menu icon according to the calculated deformation of the frame; An icon display step to be pasted on the frame and displayed on the display screen;
(4) A process including a selection operation acquisition step of acquiring a selection operation for the menu icon according to a change in the coordinate position of the menu icon according to the rotation or movement of the polyhedron according to the operation signal.

  The screen control program can be distributed through a communication line, for example, and can be transferred as a package application that operates on a stand-alone computer. Such a screen control program can be recorded on a recording medium readable by a general-purpose computer. Specifically, it can be recorded on various recording media such as a RAM card in addition to a magnetic recording medium such as a flexible disk or a cassette tape, or an optical disk such as a CD-ROM or DVD-ROM.

(Action / Effect)
According to the present embodiment described above, when the sphere 310 is moved or rotated in the sphere home display, the menu icon 301 arranged at a predetermined position on the surface of the sphere 310 is also displayed on the frame on the surface of the sphere 310. Since the position 311 and the apparent deformation associated with the movement or rotation of the sphere 310 are calculated, an image can be appropriately displayed according to the stop position, and the visibility and operability are excellent and intuitive operation is possible. Is possible.

  In the sphere home display according to the present embodiment, the rotation angle or the rotation speed of the sphere 310 is changed according to the displacement amount of the input position of the operation signal on the touch panel 300. For example, when the displacement amount is small Can be made easier to select by slowing down the rotation speed of the menu icon. If the amount of displacement is large, the menu icon 301 that is not initially displayed on the screen can be displayed immediately by increasing the rotation speed of the menu icon. User operability can be improved, such as being able to be discovered.

  Further, in the sphere home display according to the present embodiment, the operation signal acquisition unit 205 enlarges and displays the sphere 310 according to the operation signal, and the menu icon 301 is arranged on a plane in the virtual space in the icon display unit 202c. Thus, even when the display screen is displayed three-dimensionally, the user can perform an operation without causing a sense of incongruity.

  In the present embodiment, the menu icon 301 is associated with various functions and data of the information processing apparatus. In the spherical home display, the menu icons 301 are arranged in a matrix according to the function and data attributes. Therefore, even when the sphere 310 is rotated or moved, the menu icons 301 can be appropriately displayed on the screen without shifting the arrangement of the menu icons 301, and the menu icons can be easily searched for. The operability can be improved.

DESCRIPTION OF SYMBOLS 100 ... Main body 101 ... Antenna 102 ... Duplexer 103 ... Synthesizer 110 ... Low noise amplifier 111 ... Mixer 112 ... IF amplifier 113 ... Orthogonal mixer 114 ... Converter 115 ... Demodulator 116 ... Channel decoder 117 ... Audio decoder 118 ... Converter 119 ... Amplifier 120 ... Speaker 130 ... Power amplifier 131 ... Mixer 132 ... IF amplifier 133 ... Orthogonal mixer 134 ... Converter 135 ... Modulator 136 ... Channel encoder 137 ... Audio encoder 138 ... Converter 139 ... Amplifier 140 ... Microphone 150 ... Time base 151 ... EEPROM
152 ... RAM
153 ... ROM
164 ... Acceleration sensor 165 ... LCD
166 ... Operation buttons 167 ... LED
168 ... Touch sensor 171 ... Power supply system battery 172 ... Power supply 173 ... Converter 174 ... Vibrator 175 ... Input interface 200 ... CPU
DESCRIPTION OF SYMBOLS 201 ... Home application 202 ... Display information generation part 202a ... Virtual polyhedron formation part 202b ... Icon generation part 202c ... Icon display part 203 ... Application control part 205 ... Operation signal acquisition part 205a ... Operation position detection part 205b ... Deformation calculation part 205c ... Selection operation acquisition unit 210 ... other application 300 ... touch panel 301 ... menu icon 302 ... menu icon (active)
303 ... Icon 304 ... Standby screen 310 ... Sphere 311 ... Frame

Claims (12)

A screen control system for a display screen of an information processing apparatus,
An icon generator that generates screen information displayed on the display screen as a menu icon that is an image or moving image that is the same or similar to the screen shape of the display screen;
A virtual polyhedron forming unit that virtually forms a polyhedron on a three-dimensional virtual space and arranges a frame on which the menu icon is pasted on the polyhedron surface;
An operation signal acquisition unit that acquires an operation signal by an operator and moves the frame along the surface of the polyhedron according to the operation signal;
A deformation calculation unit that calculates an apparent deformation associated with a position on the polyhedron and movement or rotation of the polyhedron of the frame arranged at a predetermined position on the polyhedron surface;
An icon display unit that processes the menu icon in accordance with the deformation of the frame calculated by the deformation calculation unit, pastes the menu icon on the frame, and displays the icon on the display screen.
A screen control system comprising: a selection operation acquisition unit configured to acquire a selection operation for the menu icon according to a change in a coordinate position of the menu icon accompanying rotation or movement of the polyhedron according to the operation signal. It further includes a touch panel in which an operation signal is input by pressure by a touch operation,
The operation signal acquisition unit acquires the operation signal through an operation on the touch panel, and a virtual rotation angle or rotation of the polyhedron according to a displacement amount of an input position of the operation signal on the touch panel. The screen control system according to claim 1, wherein the speed is changed. Various functions and data of the information processing apparatus are associated with the menu icon,
The menu icons are arranged in a matrix according to the function and data attributes,
3. The screen according to claim 1, wherein the operation signal acquisition unit increases or decreases the number of display of the menu icon in the icon display unit by enlarging or reducing the polyhedron according to the operation signal. Control system.   The operation signal acquisition unit enlarges and displays the polyhedron according to the operation signal, and causes the icon display unit to display the menu icon so as to be arranged on a plane in a virtual space. The screen control system according to any one of 1 to 3. A screen control program for a display screen of an information processing device, the arithmetic processing device of the information processing device,
A polyhedron is virtually formed on a three-dimensional virtual space, a frame on which the menu icon is pasted is disposed on the polyhedron surface, an operation signal is obtained by an operator, and the operation signal is obtained according to the operation signal. A virtual polyhedron operation step for moving the frame along the polyhedron surface;
An icon generation step of generating screen information displayed on the display screen as a menu icon that is an image or moving image that is the same or similar to the screen shape of the display screen;
Calculates the position on the polyhedron of the frame arranged at a predetermined position on the surface of the polyhedron and the apparent deformation associated with the movement or rotation of the polyhedron, and the menu icon according to the calculated deformation of the frame. An icon display step that is processed, pasted on the frame, and displayed on the display screen;
A selection operation acquisition step of acquiring a selection operation for the menu icon in accordance with a change in the coordinate position of the menu icon accompanying rotation or movement of the polyhedron according to the operation signal. Control program. In the virtual polyhedron operation step, the operation signal is acquired through a touch panel to which an operation signal is input by pressure by a touch operation, and the virtual of the polyhedron is determined according to a displacement amount of an input position of the operation signal on the touch panel. The screen control program according to claim 5, wherein a typical rotation angle or rotation speed is changed. In the icon display step, the menu icon is associated with various functions and data of the information processing apparatus, and the menu icons are arranged in a matrix according to the attributes of the function and data,
The screen according to claim 5 or 6, wherein, in the virtual polyhedron operation step, the display number of the menu icons is increased or decreased in the icon display step by enlarging or reducing the polyhedron according to an operation signal. Control program.   6. In the virtual polyhedron operation step, the polyhedron is enlarged and displayed according to an operation signal, and in the icon display step, the menu icons are displayed so as to be arranged on a plane in a virtual space. 8. The screen control program according to any one of 7 to 7. A screen control method for a display screen of an information processing apparatus,
A polyhedron is virtually formed on a three-dimensional virtual space, a frame on which the menu icon is pasted is disposed on the polyhedron surface, an operation signal is obtained by an operator, and the operation signal is obtained according to the operation signal. A virtual polyhedron operation step for moving the frame along the polyhedron surface;
An icon generation step of generating screen information displayed on the display screen as a menu icon that is an image or moving image that is the same or similar to the screen shape of the display screen;
Calculates the position on the polyhedron of the frame arranged at a predetermined position on the surface of the polyhedron and the apparent deformation associated with the movement or rotation of the polyhedron, and the menu icon according to the calculated deformation of the frame. An icon display step that is processed, pasted on the frame, and displayed on the display screen;
And a selection operation acquisition step of acquiring a selection operation for the menu icon in accordance with a change in the coordinate position of the menu icon accompanying rotation or movement of the polyhedron according to the operation signal. In the virtual polyhedron operation step, the operation signal is acquired through a touch panel to which an operation signal is input by pressure by a touch operation, and the virtual of the polyhedron is determined according to a displacement amount of an input position of the operation signal on the touch panel. The screen control method according to claim 9, wherein a general rotation angle or rotation speed is changed. In the icon display step, the menu icon is associated with various functions and data of the information processing apparatus, and the menu icons are arranged in a matrix according to the attributes of the function and data,
The screen according to claim 9 or 10, wherein, in the virtual polyhedron operation step, the display number of the menu icons is increased or decreased in the icon display step by enlarging or reducing the polyhedron according to an operation signal. Control method.   10. The virtual polyhedron operation step displays the polyhedron in an enlarged manner according to an operation signal, and the icon display step displays the menu icon so as to be arranged on a plane in a virtual space. The screen control method according to any one of 11 to 11.