JP5756682B2 - Information processing device - Google Patents

Description

  The present invention relates to an information processing apparatus, and more particularly to an information processing apparatus that executes predetermined processing based on an output of a sensor that detects the attitude of the apparatus.

  Information processing apparatuses such as smartphones and tablet terminals often include a touch panel display. The touch panel display has a function of displaying an image representing data stored in the information processing apparatus on the display and a function of detecting an operation from the user. The user can operate the data represented by the image while referring to the image displayed on the display. Therefore, it is desirable that the operation on the touch panel display is an operation that is visually reflected on the display image.

  Examples of such operations include a tap in which a predetermined process is executed in response to the user pressing the display screen lightly with a finger, and reduction and deletion of the image in response to the user sandwiching an image with two fingers. Pinch-in in which the image is executed, and pinch-out in which enlargement of the image is executed in response to an increase in the interval between two fingers overlapped by the user. An operation performed by a user using a finger on an image displayed on the display like these operation methods is hereinafter referred to as a “gesture operation”. Further, in the following description, an operation method in which the user slides or pays the display image with a finger is referred to as “flick”.

JP 2006-33724 A JP 2003-269956 A JP-A-10-170272

  Recently, apart from these, a technology that has a sensor for detecting the attitude of the apparatus and changes the state of the apparatus in accordance with the sensor output is becoming widespread. For example, Patent Document 1 discloses a technique for changing the assignment of functions to a plurality of operation buttons in accordance with the attitude of a digital camera. As described above, when the function of the operation button is changed according to the posture of the camera, the operability of the button is improved. Furthermore, as a technique for detecting the attitude of the apparatus, Patent Document 2 discloses a technique related to an attitude sensor that can obtain a detection signal with a small noise component, and Patent Document 3 detects the attitude of the apparatus by detecting the rotational movement of the apparatus. Techniques relating to piezoelectric gyro sensors are disclosed.

  These are all relatively new technologies, and have brought about a great change in the interface of the information processing apparatus. However, the range of processing that can be realized by these technologies is not so wide, and there is a problem that only a limited type of instruction can be given to the information processing apparatus.

  Therefore, an object of the present invention is to provide an information processing apparatus that can execute more types of processing determined by a combination of gesture operation performed by a user and the posture of the apparatus.

  An information processing apparatus according to the present invention includes a display unit having a display surface, a posture detection unit that detects a posture of the display surface with respect to a reference coordinate system and outputs a posture detection signal, and a touch position of the user on the display surface. An image displayed on the display means, the coordinate detection means for detecting the coordinates and outputting the coordinate signal, and the process determined by the combination of the value of the coordinate signal and the value of the attitude detection signal in response to the coordinate signal And executing means for executing in connection with.

  According to this information processing apparatus, processing determined according to the combination of the gesture operation performed by the user and the attitude of the information processing apparatus is executed in association with the displayed image. Therefore, the user can perform a gesture operation corresponding to the posture of the information processing apparatus.

  Preferably, the reference coordinate system is an earth reference coordinate system defined by first and second reference coordinate axes that define a horizontal plane and a third reference coordinate axis that is defined perpendicularly to the horizontal plane and facing upward. .

  The attitude of the information processing apparatus is determined by detecting the attitude of the display surface with respect to a prescribed earth reference coordinate system. Therefore, the attitude of the information processing apparatus is uniquely determined.

  More preferably, a first direction and a second direction that define the attitude of the display surface are defined on the display surface, and the execution means is configured so that the first direction detected by the attitude detection signal is the first direction. 3 is displayed on the display surface in response to the change of the touch position of the user represented by the coordinate signal along the first direction when the distance is within a predetermined range centered on the reference coordinate axis 3. A predetermined process is performed on the data represented by the image.

  When the first direction that defines the orientation of the display surface is within a predetermined range centered on the third reference coordinate axis and the touch position of the user changes along the first direction, it is represented by the display image. A predetermined process is performed on the data. Therefore, when the information processing apparatus is in a predetermined posture, flicking the display image along the first direction allows the user to execute predetermined processing on the data represented by the image.

  More preferably, the predetermined process is a process of transferring data represented by an image displayed on the display surface to a predetermined storage device.

  When the attitude of the information processing apparatus is in a predetermined attitude, the user can upload the data represented by the image to the storage apparatus by flicking the display image along the first direction.

  More preferably, it further includes an internal storage device for storing data, and on the display surface, a first direction and a second direction for defining the orientation of the display surface are defined, and the execution means includes an orientation detection signal The first direction detected by the user is within a predetermined range centered on the third reference coordinate axis, and when the image representing the data is displayed on the display surface, the user represented by the coordinate signal In response to the change of the touch position in the direction opposite to the first direction, a predetermined process is performed on the data represented by the image.

  When the posture of the information processing apparatus is in a predetermined posture, the user can execute predetermined processing on the data displayed by the image by flicking the display image along the direction opposite to the first direction.

  More preferably, the predetermined process is a process of transferring data represented by an image from a network to an internal storage device.

  When the orientation of the information processing device is in a predetermined orientation, if the display image is flicked along the direction opposite to the first direction, the user downloads data displayed by the image from the network and stores it in the internal storage device. Can be saved.

  Preferably, the execution unit displays the image displayed on the display surface in a certain direction in response to the change in the touch position of the user represented by the coordinate signal on the display surface along the certain direction. And moving at a speed determined depending on the posture detection signal.

  When the user changes the touch position along a certain direction, the displayed image moves in the same direction as the touched direction at a speed determined by the posture detection signal. Therefore, the user can change the moving speed of the displayed image by changing the inclination of the information processing apparatus.

  More preferably, a first direction and a second direction that define the orientation of the display surface are defined on the display surface, and the information processing apparatus uses the output of the orientation detection means to set the third reference coordinate axis. The display surface changes from a posture in which one of the first direction and the second direction is included in a predetermined range as a center to a posture in which the other of the first direction and the second direction is included in a predetermined range. In response to the change in the posture of the display unit, the display means is configured so that the image displayed on the display surface changes from a state of being upright along one side to a state of being upright along the other side. Further included is an image rotation means for rotating the display direction according to.

  When the posture of the information processing apparatus changes from a posture including one of the first direction and the second direction within a predetermined range centered on the third reference coordinate axis to a posture including the other, The display direction of the image displayed on is rotated. Therefore, the user can change the display direction of the displayed image according to the direction of rotation of the apparatus by rotating the information processing apparatus.

  According to the information processing apparatus according to the present invention, predetermined processing is executed in accordance with a combination of the attitude of the apparatus and the gesture operation from the user. Therefore, the user can perform an operation more intuitive than the conventional operation, and the operability of the information processing apparatus is improved.

It is a figure which shows the external appearance of the information processing apparatus which concerns on one embodiment of this invention. It is a figure which shows the control block of information processing apparatus and a server. It is a figure for demonstrating the method to detect the attitude | position of information processing apparatus. It is a figure for demonstrating the method to detect rotation of information processing apparatus. It is a figure which shows the control structure of the program performed when an upward flick is detected. It is a figure which shows the control structure of the program performed when a downward flick is detected. It is a figure which shows the control structure of the program performed when a rightward or leftward flick is detected. It is a figure which shows the control structure of the program performed when rotation of information processing apparatus is detected. It is a figure which shows the state which the user flicked the icon upwards. It is a figure which shows the state which the user flicked the display screen upwards. It is a figure which shows the state which the user flicked the icon downward. It is a figure which shows the state which the user flicked the display screen downward.

  In the following description and drawings, the same parts are denoted by the same reference numerals. Therefore, detailed description thereof will not be repeated.

[Overall configuration (hardware)]
Referring to FIG. 1, information processing apparatus 100 according to the present embodiment includes a rectangular display screen 110 having a display function. The display screen 110 is a touch panel display, and has not only a display function but also a function of detecting gesture operations performed by the user such as touch and slide. The user can edit the image and data represented by the image while confirming the image displayed on the display screen 110.

  Referring to FIG. 2, in addition to the display screen 110, the information processing apparatus 100 includes a control unit 150 that controls each functional unit, a sensor 160 that detects the attitude and rotation of the information processing apparatus 100, and a user on the display screen 110. The operation detection unit 170 that detects the gesture operation performed in step S102, the storage unit 180 that stores data received by the information processing device 100, and the information processing device 100 and the external device via the network 240 are in a communicable state. A communication unit 190 is further included.

  In the present embodiment, the information processing apparatus 100 is in a state where it can be connected to the server 200 that is an external apparatus via the network 240. The server 200 includes a control unit 210 that controls each functional unit, a storage unit 220 that stores data, and a communication unit 230 that enables communication with an external device via a network 240.

[Detection of attitude and rotation of information processing device]
Referring to FIG. 3, display screen 110 is rectangular as described above. The center of the display screen 110 is a point O, and among the sides constituting the display screen, one of the short sides is a side A and one of the long sides is a side B. The display screen 110 defines a first direction P and a second direction Q that define the orientation of the display screen. In the state shown in FIG. 3, the first direction P is parallel to the long side B, and the second direction Q is parallel to the short side A.

  The X axis and the Y axis shown in FIG. 3 are straight lines that define a horizontal plane, respectively, and the Z axis is a straight line that is defined vertically upward with respect to the horizontal plane. Since the reference coordinate system defined by these three axes is the earth reference coordinate system, it may be considered that this reference coordinate system does not change regardless of the posture of the apparatus.

  The attitude of the information processing apparatus 100 is determined by the positional relationship between the first direction P and the second direction Q with respect to the reference coordinate system. In the state shown in FIG. 3, the first direction P is parallel to the Z axis, and the display screen 110 has the side A upward. That is, the display screen 110 is orthogonal to the XY plane which is a horizontal plane. At this time, it is determined that “the display screen 110 is inclined with the side A facing upward”.

  Furthermore, even if it is not orthogonal to the horizontal plane, the information processing apparatus is determined to be “inclined” if it is inclined more than a certain level with respect to the horizontal plane. A region M shown in FIG. 3 is a predetermined region that spreads in a conical shape with the point O as the center and the Z axis as an axis. The extent of the area M can be set in advance by the user. If the first direction P is included in the region M, the information processing apparatus 100 is determined to be “inclined”. When the direction P exceeds the region M, the information processing apparatus 100 approaches the horizontal state from the orthogonal state, and is determined as “no tilt”.

  When the information processing apparatus 100 is orthogonal to the horizontal plane with the opposite side of the side A as the upward direction (a state rotated 180 degrees from the state shown in FIG. 3), the information processing apparatus 100 is “tilted with the opposite side of the side A facing upward”. Determined. Also in this case, the inclination of the apparatus is determined depending on whether or not the first direction P is included in the region M.

  The rotation of the information processing apparatus 100 will be described by comparing FIG. 3 and FIG. In contrast to the state shown in FIG. 3, in the state shown in FIG. 4, the second direction Q is parallel to the Z axis, and the display screen 110 has the side B as the upward direction. That is, the region M includes not the first direction P but the second direction Q. As described above, when one of the first and second directions changes from the state included in the region M to the state included in the region M, the information processing apparatus 100 is determined to be “with tilt”. . In the state illustrated in FIG. 4, it is determined that “the information processing apparatus 100 is inclined with the side B facing upward”. Even when the side B is directed upward, the inclination of the apparatus is determined depending on whether or not the second direction Q is included in the region M.

Software configuration
With reference to FIGS. 2 and 5-7, a control structure of a program executed in response to detection of a flick will be described. Referring to FIG. 5, when operation detection unit 170 detects that the image displayed on display screen 110 has been flicked from bottom to top in step 250, control unit 150 performs information processing in step 252. It is determined whether the device 100 is tilted. If this determination is affirmative, control proceeds to step 254.

  In step 254, the control unit 150 determines whether the data represented by the image flicked by the user is stored in the storage unit 180 in the information processing apparatus 100. If this determination is affirmative, control proceeds to step 256, and if negative, control ends immediately.

  In step 256, the communication unit 190 transmits the data stored in the storage unit 180 to the server 200. When the process of step 256 ends, the control executed by the information processing apparatus 100 ends. In step 258, the communication unit 230 in the server 200 receives the data. In step 260, the data is stored in the storage unit 220. When the process of step 260 ends, the control executed by the server 200 ends.

  If the determination in step 252 is negative, control proceeds to step 262. In step 262, the control unit 150 moves the image from the bottom to the top in accordance with the inclination of the information processing apparatus 100 detected by the sensor 160 and the speed at which the user flicks the image. (The movement of the display image in this way is hereinafter referred to as “the image scrolls”.)

  In step 264, the control unit 300 determines whether the scrolling of the image has stopped. If this determination is affirmative, the control ends, and if negative, the process returns to step 262 again.

  Referring to FIG. 6, when operation detection unit 170 detects that the image displayed on display screen 110 is flicked from top to bottom in step 280, in step 282, control unit 150 performs information processing. It is determined whether the device 100 is tilted. If this determination is affirmative, control proceeds to step 284.

  In step 284, the control unit 150 determines whether the data represented by the image flicked by the user is stored in the storage unit 220 in the server 200. If this determination is affirmative, control proceeds to step 286, and if negative, control ends immediately.

  In step 286, the control unit 150 transmits a request for transmitting data represented by the image to the server 200. In step 288, the communication unit 230 of the server 200 receives the transmission request via the network 240. In step 290, the control unit 210 of the server 200 reads the data from the storage unit 220 and instructs the communication unit 230 to transmit the data to the information processing apparatus 100. In step 292, the communication unit 190 receives the data transmitted in step 290, and this data is stored in the storage unit 180 (step 294).

  If the determination in step 282 is negative, control proceeds to step 296. In step 296, the control unit 150 scrolls the image from the top to the bottom in accordance with the inclination of the information processing apparatus 100 detected by the sensor 160 and the speed at which the user flicks the image.

  In step 298, the control unit 300 determines whether or not the scrolling of the image has stopped. If this determination is affirmative, control is terminated, and if it is negative, control returns to step 296 again.

  Referring to FIG. 7, when control unit 150 detects a rightward or leftward flick in step 310, an image displayed on display screen 110 is set as the flick direction at a speed corresponding to the inclination of information processing apparatus 100. Scroll in the same direction (step 312). In step 314, the control unit 150 determines whether scrolling of the image has stopped. If this determination is affirmative, the control ends, and if negative, the process returns to step 312 again.

  With reference to FIGS. 2-4 and 8, the control structure of the program executed when the rotation of the information processing apparatus 100 is detected will be described. When the sensor 160 detects that the information processing apparatus 100 has changed from the state illustrated in FIG. 3 to the state illustrated in FIG. 4, in step 320, the control unit 150 determines whether or not the information processing apparatus 100 has rotated. If this determination is affirmative, control proceeds to step 322, and if negative, control returns to step 320 again. In step 322, the image displayed on the display screen 110 is rotated and changed from the state where the side A shown in FIG. 3 is upward to the state where the side B shown in FIG. 4 is upward.

  In step 324, when the sensor 160 detects that the information processing apparatus 100 has changed from the state shown in FIG. 4 to the state shown in FIG. 3, in step 324, the control unit 150 determines whether or not the information processing apparatus 100 has rotated. judge. If this determination is affirmative, control proceeds to step 326, and the image displayed on display screen 110 rotates so that side A faces upward. When the process of step 326 ends, control returns to step 320 again. If the determination in step 324 is negative, control returns to step 324 again.

  In this way, the displayed image is rotated every time the sensor 160 detects the rotation of the information processing apparatus 100. In step 322 or step 326, the image is rotated so that side A or side B faces upward, but the orientation of the image after rotation is determined by which direction the display screen 110 faces upward.

[Operation]
When a flick operation or rotation is detected, the information processing apparatus 100 and the server operate as follows. In the following, reference is mainly made to FIGS. 2 and 5-12.

  With reference to FIGS. 2, 5, and 9, the information processing apparatus 100 is inclined with the side A as the top, and icons 120 and 130 are displayed on the display screen 110. When the user flicks the icon 130 from the bottom to the top, the control unit 150 determines whether the information processing apparatus 100 is tilted (step 252). If it is determined that there is an inclination, the data represented by the icon 130 is transmitted to the server 200 (step 256). The transmitted data is stored in the storage unit 220 in the server (step 260).

  Referring to FIG. 10, when it is determined in step 252 that the information processing apparatus 100 has no inclination, the image flicked by the user is directed upward at a speed determined by the inclination of the information processing apparatus 100 and the flicked speed. (Step 262). Therefore, both the icons 120 and 130 displayed on the display screen 110 are scrolled upward.

  2, 6, and 11, when the user flicks icon 130 from top to bottom, it is determined by control unit 150 whether or not information processing apparatus 100 is tilted (step 282). . If it is determined that there is an inclination, it is determined whether or not the data represented by the icon 130 is stored in the server 200 (step 284). If this determination is affirmative, the information processing apparatus 100 receives the data from the server (step 292) and stores it in the storage unit 180 (step 294).

  Referring to FIG. 12, when it is determined in step 282 that the information processing apparatus 100 has no tilt, the image flicked by the user is downward at a speed determined by the tilt of the information processing apparatus 100 and the flicked speed. (Step 298). Therefore, both the icons 120 and 130 displayed on the display screen 110 are scrolled downward.

  Referring to FIG. 7, when the user flicks the display image to the right or left, the image is scrolled in the same direction as the flick direction at a speed corresponding to the inclination of information processing apparatus 100 (step 312).

  Referring to FIG. 8, when the user rotates information processing apparatus 100 so that side B faces upward, the display image rotates with side B facing upward. Further, when the user rotates the information processing apparatus 100 so that the side A is facing upward, the display image is rotated with the side A facing upward. As described above, when the user rotates the information processing apparatus 100, the image displayed on the display screen 110 also rotates according to the rotated direction.

  According to the information processing apparatus according to the present embodiment, the user can execute various processes based on a combination of the attitude of the information processing apparatus and the gesture operation performed by the user. In this process, the processing contents and the gesture operation performed by the user, such as a process of uploading data when the user flicks the display image upward, and a process of downloading data when the user flicks the display image downward are intuitive. Many are tied together. Therefore, the user can perform a gesture operation while changing the posture of the apparatus, and the operability of the information processing apparatus is improved.

  The information processing apparatus according to the present embodiment transmits / receives data to / from a server that is an external apparatus when an upward or downward flick is detected. However, the process executed in such a case is not limited to the process performed with the external device. For example, when the user flicks an icon upward, the process of displaying the data represented by the icon on the display screen, and when the user flicks the icon downward, the process of deleting the data represented by the icon For example, processing executed only by the information processing apparatus may be used.

  The display screen of the information processing apparatus according to the present embodiment may not be rectangular. For example, it may be oval or square. In such a case, it is desirable that the first direction and the second direction are directions in which the user frequently performs flick operations.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 110 Display screen 120, 130 Icon 150, 210 Control part 160 Sensor 170 Operation detection part 180, 220 Storage part 190, 230 Communication part 240 Network

Claims (8)

Display means having a display surface;
Attitude detection means for detecting the attitude of the display surface relative to a reference coordinate system and outputting an attitude detection signal;
Coordinate detection means for detecting coordinates of a touch position on the display surface of the user and outputting a coordinate signal;
Execution means for executing processing determined by a combination of the value of the coordinate signal and the value of the posture detection signal in response to the coordinate signal in relation to the image displayed on the display means. See
The information processing apparatus , wherein the execution unit switches between a first process and a second process of a type different from the first process according to a value of the posture detection signal .   The reference coordinate system is an earth reference coordinate system defined by first and second reference coordinate axes that define a horizontal plane and a third reference coordinate axis that is defined by facing upwards perpendicular to the horizontal plane. The information processing apparatus according to claim 1. On the display surface, a first direction and a second direction that define the posture of the display surface are defined,
When the first direction detected by the posture detection signal is within a predetermined range centered on the third reference coordinate axis, the execution means determines that the touch position of the user represented by the coordinate signal is In response to the change along the first direction, instead of the process executed as the first process when not within the predetermined range, the image is displayed by the image displayed on the display surface. The information processing apparatus according to claim 2, wherein a predetermined process is executed as the second process on the data to be processed . The process executed as the first process is a process of moving an image displayed on the display surface in the first direction,
The information processing apparatus according to claim 3, wherein the predetermined process is a process of transferring data represented by an image displayed on the display surface to a predetermined storage apparatus. An internal storage device for storing data;
On the display surface, a first direction and a second direction that define the posture of the display surface are defined,
The execution means displays the image representing the data on the display surface, and the first direction detected by the posture detection signal is within a predetermined range centered on the third reference coordinate axis. In response to a change in the touch position of the user represented by the coordinate signal along a direction opposite to the first direction, the first position when the position is not within the predetermined range. The information processing apparatus according to claim 2, wherein a predetermined process is executed as the second process on the data represented by the image instead of the process executed as a process . The process executed as the first process is a process of moving an image displayed on the display surface in a direction opposite to the first direction,
The information processing apparatus according to claim 5, wherein the predetermined process is a process of transferring the data represented by the image from a network to the internal storage device. In response to a change in the touch position of the user represented by the coordinate signal on the display surface along a certain direction, the execution means displays the image displayed on the display surface as the constant The information processing apparatus according to claim 1, wherein a process of moving in a direction at a speed determined depending on the posture detection signal is executed as the first process. . On the display surface, a first direction and a second direction that define the posture of the display surface are defined,
Based on the output of the posture detection means, from the posture in which one of the first direction and the second direction is included in a predetermined range centered on the third reference coordinate axis, the first range is included in the predetermined range. In response to the change in the posture of the display surface to a posture that includes the other of the first direction and the second direction, the image displayed on the display surface is upright along the one side. The information processing apparatus according to claim 2, further comprising an image rotation unit that rotates a display direction of the display unit so as to change from a state in which the display unit is erect along the other side.

Images