WO2013011863A1

WO2013011863A1 - Information processing device, operation screen display method, control program, and recording medium

Info

Publication number: WO2013011863A1
Application number: PCT/JP2012/067526
Authority: WO
Inventors: 正義神原
Original assignee: シャープ株式会社
Priority date: 2011-07-15
Filing date: 2012-07-10
Publication date: 2013-01-24
Also published as: JP2013025410A

Abstract

This information processing device, which is provided with an input unit and a display unit, achieves superior operability. A tablet terminal (100) is characterized by being provided with: a touch information generation unit (21) that acquires the trajectory along which an indicator that points out a position of the screen of the display unit (12) has moved; an object specification unit (22) that, as a selected object, specifies an object of which at least a portion overlaps a selected region specified on the basis of the trajectory; a related item extraction unit (23) that extracts, as a related item, an item associated with the specified object by referring to a related information recording unit (42) that associates and records objects and items related to the objects; and an operation screen processing unit (24) that disposes the icon of the extracted related item at a specific position, and displays the icon at the display unit (12). The tablet terminal is further characterized by the operation screen processing unit (24) disposing the icons of related items having a higher priority rank among the extracted related items closer to the endpoint of the trajectory.

Description

Information processing apparatus, operation screen display method, control program, and recording medium

The present invention relates to a user interface technology of an information processing apparatus including an input unit and a display unit.

In recent years, so-called tablet terminals such as smartphones and tablet PCs are rapidly spreading. The tablet terminal has a flat outer shape and includes a touch panel as a display unit and an input unit. By touching the object displayed on the touch panel with a finger, a pen, or the like, the user can perform various operations on the tablet terminal body.

The tablet terminal can discriminate various contact operations on the user's screen by the touch panel, and can perform object display according to the contact operation. For example, for the touch action, various actions such as tapping (lightly tapping), flicking (flicking, picking), pinching (pinch with a finger), dragging an object displayed on the screen with a finger (pen) There is. The tablet terminal discriminates such various contact actions, and selects / moves objects, scrolls a list, enlarges / reduces an image, etc. according to the discrimination result. The tablet terminal realizes a more intuitive operation by the touch panel as described above, and is supported by many people.

For example, Patent Document 1 discloses a mobile communication terminal including a touch panel display unit. In the portable communication terminal of Patent Document 1, an object (URL, e-mail address, character string, image, etc.) can be selected by touching it with a finger (pen) or touching it so as to surround it. ing. When an object is selected by such an operation, the mobile communication terminal extracts a keyword from the selected object and accesses a related site.

Further, Patent Document 2 discloses a portable device having a touch panel display. The portable device of Patent Document 2 displays a through image (an image reflected on a camera, etc.) on a touch panel display, detects a specific target in the through image selected by touching the surrounding area, and reduces the specific target. The image can be displayed on the edge of the touch panel display as a release button.

Patent Document 3 discloses a website search system using a touch panel. When the keyword displayed in the keyword display area is touched by hand, the website search system of Patent Literature 3 accepts it as a search keyword and displays a first mother icon corresponding to the accepted keyword. Then, the website search system searches the website with a search engine according to the keyword, and displays a thumbnail image of the searched website around the first mother icon.

Patent Document 4 discloses an information processing apparatus including a display panel having a contact sensor. The information processing apparatus disclosed in Patent Document 4 detects rotation of the operating body (finger) by a predetermined angle or more while an object is selected, and displays operation items related to the object around the object.

Further, Patent Document 5 discloses an information processing apparatus including a touch panel unit. The information processing apparatus disclosed in Patent Document 5 acquires a trajectory of a user's touch position, specifies an object image selected by the trajectory, and moves the selected object image to a position corresponding to an end point of the trajectory.

On the other hand, regardless of the tablet terminal and regardless of the size of the device, in general information processing devices, user interface technology that displays a menu and accepts selection is very often used. According to this technology, the information processing apparatus displays a menu so that the user can select a desired item, and accepts the user's selection. Thus, the user can operate the apparatus by selecting a desired item. For example, Patent Documents 6 to 9 disclose information processing apparatuses that realize menu display for the purpose of improving user convenience and operability.

Specifically, Patent Document 6 discloses an information display device that reduces the amount of movement of the cursor on the display screen and reduces the movement of the user's line of sight. Specifically, in the information display device, when the cursor is moved by a pointing device that controls the movement of the cursor position, the information display device detects the locus shape of the cursor. Then, the information display device searches for menu information associated with the trajectory shape, and displays the searched menu information near the end of the cursor trajectory.

Patent Document 7 discloses a window display control device that detects a movement trajectory of a mouse or the like, selects a pop-up menu associated with the trajectory information, and displays it on a display screen. Specifically, the window display control device displays a pop-up menu starting from the cursor position of the mouse, or arranges a plurality of predetermined pop-up menus on the locus of the mouse.

Patent Document 8 discloses a method and apparatus for providing a menu display that speeds up selection of a plurality of items. Specifically, the apparatus displays a combination of an angle marking menu and a straight line menu. In the angle marking menu, an item selected by a stroke pattern created by a pen or the like is determined. In the straight line menu, an item is selected by selecting a position with a pen or the like.

Patent Document 9 discloses a menu display control device and an electronic blackboard system to which the menu display control device is applied. Specifically, the menu display control device recognizes an input history related to a plurality of coordinates input to the device by a user drawing a line or the like on the coordinate input surface with a fingertip. The menu display control device determines whether or not to display the processing menu based on the input history, and displays the processing menu. More specifically, after the user draws a circle on the coordinate input surface with the index finger, the menu display control device touches the coordinate input surface with the middle finger while touching the index finger on the screen, and the middle finger is touched. Display the processing menu.

JP 2010-218322 A (published on September 30, 2010) JP 2010-182023 A (released on August 19, 2010) JP 2009-134738 A (released on June 18, 2009) JP 2011-13980 A (published January 20, 2011) JP 2006-244353 A (published on September 14, 2006) JP-A-8-305535 (released on November 22, 1996) Japanese Patent Laid-Open No. 10-307664 (published November 17, 1998) No. 11-507455 (announced on June 29, 1999) JP 2001-265475 A (published September 28, 2001)

The good operability of the tablet terminal is how to display the final result, which is the user's purpose, with a simple contact operation and a small number of operations, and a natural flow that does not contradict the user's intuition, It depends on whether to display the result based on the contact operation.

Such an improvement in operability is realized by appropriately grasping the user's purpose, the user's state, and the user's tendency. For example, the tablet terminal displays what the user wants to do now, what the user wants to do next, how the user is now operating, where the user is, and how the user moves. It is required to “see” the user's intentions from all points of view, such as whether or not it is natural.

The configurations of the devices described in Patent Documents 1 to 9 are not necessarily sufficient to detect the user's intention.

More specifically, Patent Document 1 discloses that an object is selected by an operation of surrounding the object, but it is disclosed that an item related to the object is extracted and displayed by the above operation. Not. Further, Patent Document 2 discloses that an icon corresponding to an object is displayed by an operation of enclosing the object. However, the object is selected by the above operation, and the object is displayed along with the selection. Extracting and displaying related items is not disclosed. Further, Patent Document 3 discloses that when an object is selected, the object is displayed, and a thumbnail related to the object is displayed around the object. Are not connected. Patent Document 4 discloses that an object is touched to be selected and an icon related to the object is displayed around the object. In order to display an icon around the object, the object is displayed. Apart from the selection, it is necessary to perform complicated operations (operations such as pressing the finger against the touch surface and twisting the finger angle) until the number of operations increases and the desired result (icon) is displayed. The operation becomes very complicated.

Patent Documents

6 and 7 do not mention that an event of selecting an object occurs before menu information is displayed, so the arrangement position of each item of menu information is controlled according to the selection of the object. Can not do it. Moreover, the technique of patent document 8 has the problem that operation becomes very complicated. Specifically, according to Patent Document 8, there are two types of menu lists, and the method of selecting an item is different for each type of menu. As a result, there are two patterns of pen movement and item selection flow, and the pen operation must be used for each type. In the technique of Patent Document 9, the touch panel must be operated using a plurality of fingers, which is complicated and cannot be applied to a device that operates with a pen.

As a result, a series of processes such as object selection, determination, display of the result (related items), selection of the result, display of the final result, a simple contact operation, a small number of operations, and There is a problem that the device cannot be operated by an intuitive contact operation.

The above-mentioned operability problem is not limited to a small tablet terminal excellent in portability, but also an information processing apparatus of any size provided with a touch panel type display unit and input unit, as well as all types of forms not limited to a touch panel. This is a problem that commonly occurs in information processing apparatuses including a display unit and an input unit.

The present invention has been made in view of the above-described problems, and an object thereof is to realize excellent operability in an information processing apparatus including an input unit and a display unit.

In order to solve the above problems, an information processing apparatus according to the present invention is based on a trajectory acquisition unit that acquires a trajectory of movement of an indicator that indicates a position of a screen of a display unit, and a trajectory acquired by the trajectory acquisition unit. With reference to an object specifying means for specifying an object that at least partially overlaps the specified selection area as the selected object, and a related information storage unit that stores the object and an item related to the object in association with each other, A related item extracting unit that extracts an item associated with the object specified by the object specifying unit as a related item, and an icon of the related item extracted by the related item extracting unit is arranged at a specific position, and Operation screen processing means for displaying on the display unit, wherein the operation screen processing means comprises the extracted function. Among the items, as icons higher priority related items, and characterized in that disposed near the end point of the trace obtained by the trace obtaining means.

According to the above configuration, the trajectory acquisition unit acquires a trajectory of an action (movement of the indicator) performed by the user for selecting an object, and based on the trajectory, the object specifying unit performs the operation according to the operation. Identifies the selected object. Subsequently, the related item extracting unit extracts items related to the identified object. Since the related information storage unit stores the object and the item related to the object in association with each other, the related item extracted by the related item extracting unit is related to the object selected by the user. It is. Finally, the operation screen processing means arranges the extracted icons of related items at specific arrangement positions on the display unit.

Here, in more detail, the operation screen processing unit determines which related item icon is to be arranged in which arrangement position as follows.

The operation screen processing means is extracted with reference to the distance between some arrangement positions specified in advance for arranging the icons and the position of the end point of the trajectory acquired by the trajectory acquisition means. Among the related items, the icon of the related item having the higher priority (for example, the possibility of being selected by the user is high) is determined to be arranged at the arrangement position close to the end point.

Thereby, for one operation of the user for selecting an object, the tablet terminal 100 can output a result of selectively displaying icons of related items related to the selected object. In addition, the arrangement of the icons displayed so as to be selectable here is an arrangement in consideration of the user's operation. That is, the higher priority icons are displayed near the position where the user finished the previous operation (end point of the trajectory).

The user is expected to perform an operation of selecting an icon of a related item related to the object immediately after performing the operation of selecting the previous object, and this is a natural flow of operation. In this case, the user moves the indicator (finger, pen, or cursor operated by the mouse) from the position where the previous operation is completed to the position where the target icon is displayed next. .

Here, the longer the distance from the position where the previous operation is completed to the position where the target icon is displayed, the longer the indicator must be moved, and the selection operation becomes troublesome for the user. Such annoyance becomes more prominent as the screen size of the display unit is larger, and becomes a particularly serious problem when the user is operating with one hand and the accessible area is limited. Even when the objects are managed in a hierarchy and the selection operation is performed repeatedly, the annoyance is further increased.

Therefore, as in the present invention, the user can complete the movement of the previous action with a high probability by arranging the icons that are more likely to be selected to be displayed near the completion position of the previous action. A desired icon displayed near the position is selected. As a result, it is possible to avoid the troublesome selection operation described above with a high probability.

Therefore, it is not necessary for the user to unnaturally move the indicator from one place to another on the screen, and the target final result can be reached with a simple operation. In addition, it is possible to suppress the induction of erroneous operations as the moving distance of the indicator is shorter.

In addition, the information processing apparatus of the present invention can preliminarily detect related items that the user will select next after selecting an object, and can display the related items in a selectable manner for the user. Specifically, according to the above configuration of the present invention, the icons arranged on the operation screen by the operation screen processing means are all related item icons extracted as items related to the object selected by the user. is there. In other words, after selecting an object, the related items that will be selected next are displayed as icons in the order from the highest possibility of being selected, so that the user can select the next target item. The icon to be performed can be immediately specified without greatly moving the indicator.

From the above, the information processing apparatus according to the present invention can display the final result desired by the user in a natural flow that does not contradict the user's intuition, while having a simple operation and a small number of operations. As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the input unit and the display unit.

In the information processing apparatus of the present invention, the related information storage unit stores the number of times the icon of the related item is selected by the user for each related item, and the operation screen processing means stores the number of times the selected item is selected. You may arrange | position the icon of many related items near the end point of the said locus | trajectory.

According to the above configuration, when the extracted icon is arranged at a specific arrangement position by the operation screen processing means, the operation screen processing means indicates that the icon of the related item that is selected more frequently becomes the end point of the trajectory. Place it close.

-A related item that is selected many times indicates that the icon is often selected by the user. The icon (related item) most frequently selected by the user is considered to be the icon (related item) having the highest priority, that is, the most likely to be selected next.

As described above, the operation screen processing means arranges the icons based on the past number of selections, so that the icons that are more likely to be selected by the user are arranged closer to the end point of the locus. (Result) can be provided to the user.

Therefore, the final result desired by the user can be displayed in a natural flow that does not contradict the user's intuition while being a simple operation and a small number of operations. As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the input unit and the display unit.

Alternatively, in the information processing apparatus of the present invention, the attribute of the related item is stored for each related item in the related information storage unit, and the operation screen processing unit is similar to the attribute of the selected object. The icon of the related item having a higher attribute may be arranged closer to the end point of the trajectory.

According to the above configuration, when the extracted icon is arranged at a specific arrangement position by the operation screen processing means, the operation screen processing means indicates that the icon of the related item whose attribute is similar to the selected object, Place near the end of the trajectory.

-It is considered that related items having a high degree of similarity between the selected object and the attribute, that is, similar in nature and classification, are more closely related to the selected object or similar in nature.

Rather than selecting objects (icons) whose attributes (classification or property, etc.) are not uniform at all, the user selects objects (icons) with similar (or identical) attributes one after another and browses them. And tend to edit.

In other words, after an object is selected in the previous operation, a related item (its icon) whose attribute is similar to that of the object has a higher priority, that is, is more likely to be selected. I can say that.

For this reason, as described above, the operation screen processing unit arranges icons based on the similarity to the attribute of the object, so that icons that are more likely to be selected by the user are arranged near the end point of the trajectory. As described above, an operation screen (result) can be provided to the user.

Furthermore, in the information processing apparatus of the present invention, the selected object and the related item are photographs, and the related information storage unit stores the shooting date and time of the photograph as an attribute. The screen processing means can arrange the icon of the photograph with the photographing date and time closer to the photographing date and time of the photograph as the selected object closer to the end point of the trajectory.

According to the above configuration, when the extracted photo icon is arranged at a specific arrangement position by the operation screen processing unit, the operation screen processing unit displays the icon of the photo whose shooting date and time is close to the selected object. It is arranged near the end point of the locus.

In other words, after a photo is selected in the previous operation, a photo that has similar attributes to the photo (here, the shooting date and time is close) has a higher priority, that is, may be selected. Can be said to be higher.

For this reason, as described above, the operation screen processing means places a photo icon based on the similarity of the shooting date and time of the photo, so that a photo that is more likely to be selected by the user is closer to the end of the trajectory. It is possible to provide an operation screen (result) to the user so as to be arranged.

Alternatively, in the information processing apparatus of the present invention, the related item is video content, and the related information storage unit stores a recommendation degree indicating a degree of recommending the video content to the user as an attribute. The operation screen processing means may be arranged near the end point of the trajectory as the icon of the moving image content having the higher recommendation level.

According to the above configuration, when the icon of the extracted video content is arranged at a specific arrangement position by the operation screen processing unit, the operation screen processing unit displays the video content having a high recommendation level associated with each video content. The icon is placed closer to the end point of the trajectory.

It is natural that the user has a strong tendency to select the recommended moving image content, and it is considered that the moving image content having a higher degree of recommendation has a higher priority, that is, the possibility of being selected by the user is higher.

For this reason, as described above, the operation screen processing unit arranges icons based on the recommendation level of the moving image content, so that icons that are more likely to be selected by the user are arranged closer to the end point of the trajectory. In addition, an operation screen (result) can be provided to the user.

Alternatively, in the information processing apparatus of the present invention, the trajectory acquisition unit acquires a trajectory of movement of the indicator that has moved the screen of the display unit so as to surround the object displayed on the display unit, and the object identification The means identifies an object at least partially included in the region surrounded by the locus as the selected object, and the operation screen processing means displays the extracted icon of the related item on the outline of the ring. It is preferable to arrange them side by side.

According to the above configuration, the trajectory of the motion enclosed by the trajectory acquisition unit is acquired, and based on the trajectory, the object specifying unit specifies the object selected by the user by the surrounding motion. Subsequently, the related item extracting unit extracts items related to the identified object. Since the related information storage unit stores the object and the item related to the object in association with each other, the related item extracted by the related item extracting unit is related to the object selected by the user. It is. Finally, the operation screen processing means arranges the icons of the extracted related items in a ring shape that is easily associated with the surrounding operation. Note that the operation screen processing means refers to the distance between the specific arrangement position on the contour line and the end point of the trajectory even when the icons are arranged on the contour line of the ring, Decide to place it near the end point.

Thereby, the icons are arranged as described above, and the generated operation screen is presented to the user. As described above, the information processing apparatus according to the present invention is extremely natural and simple in designating an object to “enclose” the object with an indicator (a pen, a finger, or a cursor controlled by a mouse). With the user's action as an opportunity, an operation screen in which icons of related items related to the selected object are arranged in a ring shape can be provided to the user.

Since the trajectory of “enclose” has a shape that encloses something, it can be said that the shape of the ring is similar to the trajectory of movement of the indicator obtained by the enclosing operation. Therefore, the result (an icon arranged in a ring shape) is likely to be associated with the above-described operation surrounded by the user.

In other words, it can be said that the transition from the event of "enclosing the object" to the event of "an operation screen with icons arranged in a circle" is a natural flow that does not contradict the user's intuition. .

Also, the menu list with related item icons arranged in a circle has the following advantages compared to the linear one-dimensional menu list. For example, in a one-dimensional menu list, for example, icons are arranged from top to bottom or from left to right. Therefore, priorities are unintentionally assigned to the icons according to the arrangement positions. On the other hand, according to the circular menu list, it is possible to treat all icons arranged in a circular manner on an equal basis.

As described above, the information processing apparatus of the present invention can display the final result desired by the user in a more natural flow that does not contradict the user's intuition, while having a simple operation and a small number of operations. . As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the input unit and the display unit.

In the information processing apparatus of the present invention, it is preferable that the operation screen processing means determines the position and size of the ring so that an icon is arranged around the selected object.

According to the configuration described above, the information processing apparatus places an icon around the object in response to an extremely natural and simple user action of “enclosing” the object. Can be output.

The user can obtain, as a result, an operation screen in which icons of related items are arranged so as to surround the object that he / she has previously enclosed and selected. The positional relationship between these icons and the object matches the positional relationship between the object and the locus of movement of the indicator by the action previously performed by the user. Further, the movement trajectory of the indicator obtained by surrounding the object is similar to the shape of the ring in which the icon is arranged.

In other words, the transition from the event of “enclosing the object” to the event of “an operation screen with icons arranged around the object” is more natural than the user's intuition. It can be said that.

Furthermore, a menu list in which icons of related items are arranged in a circle around an object has the following advantages compared to a linear one-dimensional menu list. For example, in a one-dimensional menu list, for example, icons are arranged from top to bottom or from left to right. Therefore, priorities are unintentionally assigned to the icons according to the arrangement positions. On the other hand, according to the circular menu list, it is possible to treat all icons arranged in a circular manner on an equal basis. Furthermore, even if a one-dimensional menu list is displayed near the previously selected object, it is difficult to express the relationship between the object and each icon. In contrast, if a circular menu list is displayed around the previously selected object, there is a relationship between the previously selected (enclosed) object and the surrounding icons. This makes it possible for the user to recognize this naturally.

In the information processing apparatus of the present invention, it is preferable that the operation screen processing means determines the trajectory acquired by the trajectory acquisition means, or a similar shape or an approximate shape thereof, as the shape of the ring.

According to the above configuration, the user performs an operation of freely surrounding the object with an arbitrary shape, and the trajectory at this time is held by the trajectory acquisition means. Then, when creating the operation screen, the operation screen processing means sets each icon so as to surround a predetermined area (or the object itself) on the contour line of the ring that is the same or similar to the locus obtained as described above. Deploy.

This makes it possible to output a result that icons are arranged in a ring shape in response to a very natural and simple user action in specifying the object “enclose” around the object. That is, the user can obtain, as a result, an operation screen in which icons of related items are arranged so as to surround a predetermined area (or the object itself).

Furthermore, the shape of the ring in which the icons are arranged is displayed on the operation screen in a state that matches or is similar to the movement trajectory of the operation body obtained by surrounding the object.

That is, when the user surrounds the object, an operation screen on which icons are arranged “as the user surrounded the object” is obtained. It can be said that the transition of this event is a more natural flow that does not contradict the user's intuition.

Further, since the icons are arranged in the shape as enclosed by the user, the user can obtain the operation screen in which the icons are arranged in the desired shape by surrounding the object in the desired shape. Thereby, the playability at the time of displaying an operation screen and operating information processing apparatus increases.

In addition, since the user can predict the arrangement of the icons, surround the object as he / she desires, and display the icons of related items, the operability is further improved.

Furthermore, in the information processing apparatus of the present invention, the operation screen processing means arranges the icon of the related item having the highest priority at the point closest to the end point of the trajectory on the outline of the ring, and the remaining related items. You may arrange | position each icon of an item on the outline of the said ring on the basis of the arrangement position of the said icon with the highest priority.

According to the above configuration, the operation screen processing means can always arrange the icon with the highest priority at the shortest distance from the end point of the locus on the outline of the ring. In other words, the icon that is most likely to be selected by the user can be displayed closest to the position of the user's indicator (such as a finger), and the user does not move the indicator greatly with a high probability. A target icon can be selected.

Alternatively, in the information processing apparatus of the present invention, the trajectory acquisition unit acquires the trajectory of the indicator that has occurred in a predetermined period until the indicator moves to select the object displayed on the display unit, When the operation screen processing means determines that the trajectory acquired during the predetermined period is biased toward a specific area on the screen of the display unit, the icon screen is arranged so that the icon is arranged in the specific area. The position may be determined.

According to the above configuration, the trajectory acquisition means acquires a trajectory not only for an operation for selecting an object but also for an operation that has occurred in a past predetermined period. Subsequently, the operation screen processing means can grasp at which position of the display unit the movement (movement of the indicator) has occurred in the past predetermined period from the acquired trajectory. If the locus is concentrated on a specific area of the display unit, it is possible to detect the deviation of the moving position of the indicator.

The fact that the movement position is biased in this way means that the information processing apparatus can be operated under a special use situation in which only that area can be touched (or other areas are difficult to touch and the indicator cannot be moved). You can guess that you are using it.

Therefore, the operation screen processing means determines the position of the ring so that the icon is arranged in the area where the deviation is detected (that is, the limited area where the indicator can be moved).

Thus, an icon is displayed in an area that can be selected by the user. When the user next performs an operation of selecting an icon, a desired icon can be selected immediately from the selectable area.

The following is a more detailed description using specific examples. For example, when operating with one hand, the touch position tends to be biased toward the lower left area of the touch panel (when operating with the left hand) or the lower right area of the screen (when operating with the right hand). When the user is using the information processing apparatus in such a situation, there is a problem that the operation becomes complicated if an object or icon that requires a contact operation is displayed on the upper part of the screen or the lower part of the screen opposite to the hand. . This is because the user cannot touch the target object immediately and must perform an extra operation of rushing to a contactable area or switching to a two-handed operation.

Therefore, the information processing apparatus according to the present invention can solve the above-mentioned problem by observing the usage status of the user with the above-described configuration. That is, the information processing apparatus according to the present invention can be configured to detect the bias of the contact position of the operating tool and arrange the icon in a region where the user's indicator is estimated to arrive immediately.

As a result, when the user operates the information processing apparatus with one hand, the icon is displayed so as to fit within the lower left (or right) area of the screen of the touch panel. There is no need to drag icons, and a desired icon can be selected immediately.

Alternatively, in the information processing apparatus according to the present invention, the operation screen processing means may be configured such that the icon is arranged in a specific area overlapping the locus of movement of the indicator that selects the object on the screen of the display unit. May be determined.

According to the above configuration, the icon is arranged at or near the position selected by the user with an indicator such as a finger. It can be said that the position where the user has selected the object is a selectable area for the user. Therefore, it is possible to reliably display an icon in the selectable area.

In the information processing apparatus of the present invention, the input unit and the display unit included in the information processing apparatus constitute a touch panel, and the trajectory acquisition unit acquires a trajectory of movement of the indicator that has moved on the touch panel. May be.

According to the above configuration, it is possible to increase the relevance between the contact operation performed by the user for selecting an object and the resultant product obtained according to the operation, and the operation is performed in a natural flow that does not contradict the user's intuition. A screen can be provided. As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the touch panel.

In the information processing apparatus of the present invention, the input unit included in the information processing apparatus inputs an instruction to move the cursor displayed on the display unit to the information processing apparatus, and the trajectory acquisition unit includes the instruction You may acquire the locus | trajectory of the movement of the cursor as a body.

According to the above configuration, it is possible to increase the relevance between the input operation performed for the user to select an object by operating the input unit and the resultant product obtained according to the operation, and the user's intuition The operation screen can be provided in a natural flow that does not violate. As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the display unit and the input unit.

An operation screen display method of the present invention is an operation screen display method in an information processing apparatus to solve the above-described problem, and a trajectory of a movement of an indicator that indicates a position of a screen of a display unit included in the information processing apparatus. A trajectory acquisition step to acquire, an object specifying step to specify, as the selected object, an object that at least partially overlaps the selection area specified based on the trajectory acquired in the trajectory acquisition step, and the object and the object A related item extracting step of extracting an item associated with the object identified in the object identifying step as a related item with reference to a related information storage unit that associates and stores an item associated with Place the icon of the related item extracted in the item extraction step at a specific position, An operation screen processing step to be displayed on the display unit. In the operation screen processing step, among the extracted related items, the icon of the related item having a higher priority is acquired in the trajectory acquisition step. It is characterized by being placed near the end point of the trajectory.

According to the above method, the final result desired by the user can be displayed in a natural flow that does not contradict the user's intuition while being a simple operation and a small number of operations. As a result, there is an effect that excellent operability can be realized in the information processing apparatus including the input unit and the display unit.

The information processing apparatus may be realized by a computer. In this case, an information processing apparatus control program for causing the information processing apparatus to be realized by the computer by causing the computer to operate as the above-described means, and A computer-readable recording medium on which is recorded also falls within the scope of the present invention.

In order to solve the above-described problem, an information processing apparatus according to the present invention provides a trajectory acquisition unit that acquires a trajectory in which an indicator pointing a screen position of a display unit has moved in an information processing apparatus including a touch panel, and the trajectory acquisition. An object specifying means for specifying an object at least partially overlapping the selected area specified based on the trajectory acquired by the means as the selected object, and the object and an item related to the object are stored in association with each other. With reference to the related information storage unit, related item extraction means for extracting an item associated with the object specified by the object specifying means as a related item, and an icon for the related item extracted by the related item extraction means Operation screen processing means arranged at a specific position and displayed on the display unit The operation screen processing unit, in the related item of the extracted more icons higher priority related items, and characterized in that disposed near the end point of the trace obtained by the trace obtaining means.

It is a functional block diagram which shows the principal part structure of the tablet terminal in one Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the tablet terminal in one Embodiment of this invention. It is a top view which shows the external appearance of the tablet terminal in one Embodiment of this invention. It is a figure explaining a mode when a user hold | grips and operates a tablet terminal, (a) is a figure explaining a mode that a tablet terminal is hold | gripped with one hand and is operated with the hand, (b) It is a figure explaining a mode that a tablet terminal is hold | gripped with one hand and is operated with the other hand. It is a figure explaining operation | movement of the object specific part of a tablet terminal, (a) is a figure which shows a mode that the user performed contact operation of "surrounding" an object in order to select the target object, (b) ) Is a diagram showing an example of contact information generated by the contact information generation unit in accordance with the contact operation shown in (a), and (c) is a display unit during the period from t0 to tn when contact is detected. It is a figure which shows an example of the map information of the displayed video frame. It is a figure which shows an example of the relevant information memorize | stored in the relevant information storage part of a tablet terminal. It is a figure which shows the specific example of the icon image memorize | stored in the icon memory | storage part of a tablet terminal. It is a figure which shows an example of the priority given to each icon by the icon order | rank determination part of a tablet terminal. It is a figure explaining the specific example of operation | movement of the icon arrangement | positioning determination part of a tablet terminal, (a) is a figure which shows the specific example of the icon arrangement pattern which an icon arrangement | positioning determination part acquires, (b) is icon arrangement | positioning. It is a figure which shows the specific example of arrangement | positioning of the icon determined by the determination part. It is a figure which shows the specific example of the operation screen obtained as a result of the operation screen production | generation process performed by the operation screen process part of the tablet terminal. It is a flowchart which shows the flow of the operation screen display process by a tablet terminal. It is a figure explaining operation | movement of the contact information generation part of a tablet terminal, and an object specific | specification part, (a) performed the contact operation | movement which "checks" an object in a check point in order for a user to select the target object (B) is a figure which shows an example of the contact information which the contact information generation part produced | generated with the contact operation | movement shown to (a) of the figure. It is a figure which shows the other specific example of the icon arrangement pattern which the icon arrangement | positioning determination part of a tablet terminal acquires. It is a figure which shows the other specific example of the operation screen obtained as a result of the operation screen production | generation process performed by the operation screen process part of the tablet terminal. It is a functional block diagram which shows the principal part structure of the tablet terminal in other embodiment of this invention. It is a figure explaining the processing content of each part of the operation screen process part of a tablet terminal, (a) is a figure explaining an example of the display process of the object by an operation screen process part, (b) is an operation screen process. It is a figure which shows an example of the icon arrangement pattern specialized in the ring shape which the ring shape determination part of the part determined. It is a figure which shows the specific example of the contact information memorize | stored in the contact information memory | storage part of a tablet terminal, (a) is an operation which "encloses" an object in arbitrary shapes in order for a user to select the target object. It is a figure which shows the mode performed, (b) is a figure which shows an example of the contact information which the contact information generation part produced | generated with the contact operation | movement shown to (a). It is a figure which shows the other example of the icon arrangement pattern determined by the ring shape determination part of the tablet terminal. The other example of arrangement | positioning of the object and cyclic | annular icon group in the operation screen which the operation screen process part of a tablet terminal produces | generates is shown. The other example of arrangement | positioning of the object and cyclic | annular icon group in the operation screen which the operation screen process part of a tablet terminal produces | generates is shown. It is a figure which shows the modification of the icon display method of a related item, (a) shows the mode of the ring of the icon initially arrange | positioned small around the object, (b) expands the ring of an icon. A state in the middle is shown, and (c) shows a state in which a ring of icons is finally arranged largely in the center of the screen through (a) to (b). It is a figure which shows one modification of the icon display method of a related item, (a) is a figure which shows a specific example of contact information, (b), (c), (d), (e) It is a figure which shows a mode that a some icon is sequentially displayed at a fixed space | interval, (f) is a figure which shows a specific example of the operation screen finally obtained. It is a figure which shows one modification of the icon display method of a related item, (a) is a figure which shows a specific example of contact information, (b) is a figure which shows an example of the operation screen at the time of t0. (C) is a diagram showing an example of the operation screen at the time point ta, (d) is a diagram showing an example of the operation screen at the time point tb, and (e) is an operation at the time point tc. It is a figure which shows an example of a screen, (f) is a figure which shows an example of the operation screen at the time of tn. It is a flowchart which shows the flow of the operation screen display process by the tablet terminal in other embodiment of this invention. It is a flowchart which shows the flow of the operation screen display process by the tablet terminal in other embodiment of this invention. It is a figure explaining operation | movement of the tablet terminal of this invention which can show an operation screen according to a user's usage condition, (a) demonstrates an example of the condition where the user is operating with the left hand. (B) is a figure which shows the specific example of the contact information produced | generated with the contact operation | movement of (a). It is a figure which shows an example of an operation screen when an icon is arrange | positioned according to the ring shape determined by the ring shape determination part of the operation screen process part. (A)-(c) is a figure which shows the specific example of correlation with the icon arrangement position and priority which the icon arrangement | positioning determination part of a tablet terminal performs. (A) And (b) is a figure explaining a mode that the ring of an icon is rotated by a user dragging on a contactable area | region. It is a figure which shows a mode when the tablet terminal as an information processing apparatus of this invention is implement | achieved by tablet PC. (A) is a figure which shows the specific example of the icon arrangement | positioning pattern by which the locus | trajectory of the indicator and the end point tn of a locus | trajectory were plotted by the icon arrangement | positioning determination part of a tablet terminal, (b) It is a figure which shows the result of which the priority order was linked | related with each icon arrangement position. It is a figure which shows a mode when the tablet terminal as an information processing apparatus of this invention is implement | achieved by tablet PC. (A) is a figure which shows the specific example of the icon arrangement | positioning pattern by which the locus | trajectory of the indicator and the end point tn of a locus | trajectory were plotted by the icon arrangement | positioning determination part of a tablet terminal, (b) It is a figure which shows the result of which the priority order was linked | related with each icon arrangement position. It is a figure which shows an example of the icon arrangement pattern which the icon arrangement | positioning determination part of the tablet terminal determined.

Embodiment 1
Embodiments of the present invention will be described with reference to FIGS. 1 to 14 as follows.

In the embodiment described below, a case where the information processing apparatus of the present invention is applied to a tablet terminal will be described as an example. In the present embodiment, as an example, the tablet terminal is realized by a small smartphone that can be operated with one hand and is excellent in portability.

However, the information processing apparatus of the present invention is not limited to the above-described example, and the information processing apparatus of any size (for example, a notebook-size tablet PC or an electronic blackboard equipped with a large touch panel) can be used. An information processing apparatus may be applied.

[Hardware configuration of tablet terminal]
FIG. 2 is a block diagram illustrating a hardware configuration of the tablet terminal 100 according to the present embodiment. As shown in FIG. 2, the tablet terminal 100 includes at least a control unit 10, an input unit 11, a display unit 12, and a storage unit 19. Furthermore, the tablet terminal 100 may include an operation unit 13, an external interface 14, a communication unit 15, a wireless communication unit 16, an audio output unit 17, and an audio input unit 18 in order to realize inherent functions. .

When the tablet terminal 100 is a multi-function mobile communication terminal such as a smartphone, the tablet terminal 100 is omitted here. However, the tablet terminal 100 includes a call processing unit, an imaging unit (such as a lens / image sensor) that performs imaging, and a broadcast image. Other parts (such as a tuner / demodulation unit), GPS, and sensors (such as an acceleration sensor and an inclination sensor) may be included as well as various components that are typically included in a smartphone.

The input unit 11 is for inputting an instruction signal for the user to operate the tablet terminal 100 via the touch panel. The input unit 11 is a touch surface that accepts contact with an indicator (indicating the screen position of the display unit 12, here, for example, a finger or a pen), and contact / non-contact between the indicator and the touch surface. (Approach / non-approach) and a touch sensor for detecting the contact (approach) position. The touch sensor may be realized by any sensor as long as it can detect contact / non-contact between the indicator and the touch surface. For example, it is realized by a pressure sensor, a capacitance sensor, an optical sensor, or the like.

The display unit 12 displays an object to be processed by the tablet terminal 100 (any display object such as an icon) and a processing result, and displays an operation screen for the user to operate the tablet terminal 100 using a GUI (Graphical (User 表示 Interface) screen. The display unit 12 is realized by a display device such as an LCD (Liquid Crystal Display).

In this embodiment, the input unit 11 and the display unit 12 are integrally formed, and these constitute a touch panel. Therefore, in such an embodiment, an object to be moved (operated) to indicate a screen position, that is, an operation body (here, a finger or a pen) is simultaneously positioned on the screen of the display unit 12. It is also an indicator that indicates

For example, when the touch panel of the tablet terminal 100 of the present invention is realized by a projected capacitive touch panel, specifically, the touch sensor has a transparent electrode pattern in a matrix shape made of ITO (Indium Tin Oxide) or the like. It is formed on a transparent substrate such as glass or plastic. When an indicator (such as a user's finger or pen) touches or approaches the touch sensor, the electrostatic capacity of a plurality of transparent electrode patterns in the vicinity changes. Therefore, the control unit 10 can detect the position where the indicator is in contact or approached by detecting a change in the current or voltage of the transparent electrode pattern.

In the following, the term “contact” when “contact detection”, “contact operation”, “contact position”, etc. is not only the state in which the indicator and the touch surface are in complete contact (contact), It also includes a state in which the indicator and the touch surface are close (approaching) to the extent that the touch sensor can detect.

The operation unit 13 is for the user to directly input an instruction signal to the tablet terminal 100. For example, the operation unit 13 is realized by an appropriate input mechanism such as a button, switch, key, or jog dial. For example, the operation unit 13 is a switch for turning on / off the power of the tablet terminal 100.

The external interface 14 is an interface for connecting an external device to the tablet terminal 100. The external interface 14 is realized by, for example, but not limited to, a socket for inserting an external recording medium (memory card or the like), an HDMI (High Definition Multimedia Interface) terminal, a USB (Universal Serial Bus) terminal, or the like. The The control unit 10 of the tablet terminal 100 can exchange data with an external device via the external interface 14.

The communication unit 15 communicates with an external device via a communication network. The communication unit 15 is connected to various communication terminals via a communication network, and realizes data transmission / reception between the tablet terminal 100 and the communication terminal. Further, when the tablet terminal 100 is a mobile communication terminal such as a smartphone, the communication unit 15 transmits / receives voice call data, e-mail data, and the like to / from other devices via the mobile phone network. To do.

The wireless communication unit 16 communicates with an external device wirelessly. The wireless communication unit 16 is not particularly limited, and may implement any wireless communication means such as infrared communication such as IrDA and IrSS, Bluetooth communication, WiFi communication, and a non-contact type IC card. A plurality of means may be realized. The control unit 10 of the tablet terminal 100 can communicate with devices in the vicinity of the tablet terminal 100 via the wireless communication unit 16, and can exchange data with these devices.

The sound output unit 17 outputs sound data processed by the tablet terminal 100 as sound, and is realized by a speaker, a headphone terminal, headphones, and the like.

The voice input unit 18 receives voice input generated outside the tablet terminal 100, and is realized by a microphone or the like.

The storage unit 19 includes (1) a control program executed by the control unit 10 of the tablet terminal 100, (2) an OS program, and (3) an application program for the control unit 10 to execute various functions of the tablet terminal 100, And (4) storing various data read when the application program is executed. Alternatively, (5) the control unit 10 stores data used for calculation and calculation results in the course of executing various functions. For example, the above data (1) to (4) are stored in a non-volatile storage device such as a ROM (read only memory), a flash memory, an EPROM (Erasable ROM), an EEPROM (Electrically EPROM), an HDD (Hard Disc Drive). Remembered. For example, the data (5) is stored in a volatile storage device such as a RAM (Random Access Memory). Which data is to be stored in which storage device is appropriately determined based on the purpose of use, convenience, cost, physical restrictions, and the like of the tablet terminal 100.

The control unit 10 performs overall control of each unit included in the tablet terminal 100. The control unit 10 is realized by, for example, a CPU (central processing unit). The functions of the tablet terminal 100 are such that the CPU as the control unit 10 reads a program stored in a ROM or the like into a RAM or the like and executes the program. It is realized by doing. Various functions (particularly, the operation screen display function of the present invention) realized by the control unit 10 will be described later with reference to other drawings.

[Appearance of tablet terminal]
FIG. 3 is a plan view showing the appearance of the tablet terminal 100. As illustrated in FIG. 3, the tablet terminal 100 includes an input unit 11 and a display unit 12 as a touch panel. The tablet terminal 100 includes an operation unit 13, an external interface 14, a wireless communication unit 16, an audio output unit 17, an audio input unit 18, and the like, although these are not essential components. For example, when the wireless communication unit 16 is realized by infrared communication means, an infrared transmission / reception unit is provided as the wireless communication unit 16 on the side surface of the tablet terminal 100.

FIG. 4 is a diagram illustrating a state when the user holds and operates the tablet terminal 100. More specifically, FIG. 4A is a diagram illustrating a state in which the tablet terminal 100 is gripped with one hand and is operated with that hand, and FIG. It is a figure explaining a mode that it is hold | gripped with a hand and operated with the other hand.

In the present embodiment, the tablet terminal 100 is a palm-sized information processing apparatus that can be held with one hand. As shown in FIG. 4A, the tablet terminal 100 is held with the thumb of the hand while holding the tablet terminal 100 with one hand. The touch surface of the input unit 11 can be operated. For example, when there is an icon to be operated at a position where the thumb does not reach, the tablet terminal 100 draws the icon near the thumb by flicking, and surrounds or taps the icon with the thumb. The icon can be selected.

Further, as shown in FIG. 4B, the user may hold the tablet terminal 100 with one hand and operate the touch surface of the input unit 11 with the finger of the other hand. Alternatively, although not shown, the tablet terminal 100 may be horizontally long, hold both sides with both hands, and operate the touch surface of the input unit 11 with the thumbs of both hands.

[Tablet function]
Next, the functional configuration of the tablet terminal 100 will be described. FIG. 1 is a functional block diagram illustrating a main configuration of the tablet terminal 100 according to the present embodiment.

As shown in FIG. 1, the control unit 10 of the tablet terminal 100 according to the present embodiment includes at least a contact information generation unit 21, an object specification unit 22, and related functions as functional blocks for realizing the operation screen display function of the present invention. An item extraction unit 23 and an operation screen processing unit 24 are provided. In more detail, the operation screen processing unit 24 includes an icon rank determining unit 31 and an icon arrangement determining unit 33.

Each functional block of the control unit 10 described above includes a CPU (central processing unit) that stores a program stored in a non-volatile storage device realized by a ROM (read only memory) or the like (RAM (random access memory)). It can be realized by reading and executing the above.

In addition, the storage unit 19 is specifically a frame map storage unit 41 or a related unit as a storage unit for reading or writing data when the above-described units of the control unit 10 execute the operation screen display function. An information storage unit 42, an icon storage unit 43, and a contact information storage unit 44 are included.

The contact information generation part 21 processes the signal output from the touch sensor of the input part 11, and produces | generates contact information. The contact information includes at least contact coordinate information indicating the coordinate position of the contact position of the indicator (for example, a finger). Thereby, each part of the control part 10 can acquire the locus | trajectory of the movement of the said indicator from the said contact information. In the present embodiment, the contact time information indicating the time when the contact has occurred (the movement time information indicating the movement time of the indicator) is further associated with each point constituting the trajectory as necessary. Also good.

The contact information storage unit 44 stores the contact information generated by the contact information generation unit 21. The contact information may be temporarily stored in a storage unit (such as a cache) (not shown) so that the object specifying unit 22 can be used immediately. Furthermore, in the present embodiment, the contact information can be used when each unit of the operation screen processing unit 24 and the operation screen processing unit 24 executes an operation screen generation process (including a process for displaying an icon). It is stored in the contact information storage unit 44. Whether or not the contact information storage unit 44 is realized by a non-volatile storage device, that is, whether or not the contact information is stored in a nonvolatile manner depends on the purpose of the operation screen display function executed by the operation screen processing unit 24 and the assumed use. It is determined as appropriate from the environment, the purpose of use of the tablet terminal 100 itself, convenience, cost, physical restrictions, and the like.

More specifically, the contact information generation unit 21 generates contact information after the touch sensor of the input unit 11 detects the contact between the touch surface and the indicator (in this embodiment, a finger). Until the non-contact is detected, the contact information generation unit 21 acquires a signal output from the touch sensor. This signal includes information indicating that “contact” has been detected and information indicating the contact position. Based on this signal, the contact information generation unit 21 generates contact coordinate information indicating the contact position in coordinates. Generate. Further, the contact information generation unit 21 measures the time from when contact is detected until it becomes non-contact, and associates the contact time information with the contact coordinate information. The contact information generation unit 21 may acquire and use absolute time information held by the clock unit mounted on the tablet terminal 100. However, in the present embodiment, the contact information generation unit 21 detects contact. Then, timing is started and relative contact time information is obtained. For example, the contact information generation unit 21 measures the elapsed time with the time point when the contact is first detected (t0) as 0.00 seconds, and continues the measurement until the time point when the contact is finally detected (tn). Thus, the relative contact time information corresponding to the contact position may be acquired. The contact information generation unit 21 generates contact information by associating the obtained contact time information with the contact coordinate information. In the present embodiment, the generated contact information is supplied to the object specifying unit 22 and used by the object specifying unit 22. Moreover, in this embodiment, the contact information generation part 21 acquires the coordinate information of the position where contact operation was complete | finished at least, and produces | generates contact information including this information. Furthermore, the contact information generation unit 21 may also acquire coordinate information of a position where the contact operation is started. Furthermore, the contact information generation unit 21 may associate contact time information with the start and end positions of the contact operation. The contact information generated in this way is supplied to the operation screen processing unit 24 and used by each unit included in the operation screen processing unit 24 and the operation screen processing unit 24. For example, the operation screen processing unit 24 can recognize at least the position of the end point of the trajectory based on the contact information having the above data structure.

The object specifying unit 22 specifies an object selected by the user's contact operation. The object specifying unit 22 compares the contact information generated by the contact information generating unit 21 with the map information of the video frame displayed on the display unit 12 while the contact is occurring. Thereby, the object specifying unit 22 can specify the object pointed to by the contact operation from among the objects being displayed on the display unit 12.

The frame map storage unit 41 stores map information of the video frame output to the display unit 12 at the time of contact. The map information is information indicating the layout of the video frame displayed on the touch panel. Specifically, the map information includes information for individually identifying each object displayed, and information on the shape, size, and display position of each object. That is, the map information is obtained by plotting each object corresponding to the coordinate system of the touch panel.

FIG. 5 is a diagram for explaining the operation of the object specifying unit 22. More specifically, FIG. 5A is a diagram showing that the user has performed a contact operation of “enclosing” an object in order to select the target object. FIG. 5B is a diagram illustrating an example of contact information generated by the contact information generation unit 21 in accordance with the contact operation illustrated in FIG. FIG. 5C is a diagram illustrating an example of map information of a video frame displayed on the display unit 12 during a period from t0 to tn in which contact is detected.

As shown in (a) of FIG. 5, it is assumed that the user performs a contact operation of “enclosing” one of the objects 80 (here, photographs) displayed on the touch panel of the tablet terminal 100. For example, it is assumed that the contact operation is performed in a period from t0 to tn so that the contact point passes the position of the broken line in FIG.

The object specifying unit 22 acquires contact information as shown in FIG. 5B from the contact information generating unit 21. In the present embodiment, the coordinate system of the contact information corresponds to the coordinate system of the touch panel of the tablet terminal 100, and has the leftmost upper end of the panel as the origin. In FIG. 5B, among the trajectories of the “enclose” contact movement of the user, the start point is indicated as t0 and the end point is indicated as tn. However, contact time information may also be associated with each point in between.

The object specifying unit 22 acquires the map information shown in FIG. 5C (that is, the layout of the video frame displayed on the display unit 12 during the period from t0 to tn) from the frame map storage unit 41. Then, the object specifying unit 22 compares the contact information with the map information, and selects an object 80 that completely or substantially overlaps an area (selected area) specified by the trajectory of the user's finger obtained from the contact information. Identified as an object. In the example shown in FIG. 5, the object specifying unit 22 specifies “Picture 1” in FIG. 5C as the selected object. The object specifying unit 22 supplies information on the specified object to the related item extracting unit 23.

For the selection area specified by the trajectory, when the trajectory is a closed curve, the object specifying unit 22 specifies the area surrounded by the trajectory (the hatched area in the thick frame in FIG. 5C) as the selection area. Alternatively, even if the locus is a closed curve, the circumscribed rectangle of the locus may be specified as the selection region. Based on the trajectory, the object specifying unit 22 obtains a specific selection region based on a predetermined rule that is not limited to the above example.

FIGS. 5A to 5C show, as an example, a method for identifying an object when the contact action for the user to select an object is “enclose”. The configuration is not limited to the above. For example, when the user points to a desired object, the user may select the object by simply “tapping” a touch action (single tap or double tap), or check mark (such as a check mark or a cross mark) on the object. ) May be selected by a contact action of “check”, or may be selected by a “flick” contact action of paying the object diagonally from top to bottom, or the object may be selected for a predetermined period of time. You may select by the contact operation | movement of "long press" which continues pushing. The tablet terminal 100 may assign any of the contact actions described above to “an action for selecting an object”, and the object specifying unit 22 appropriately specifies the object pointed to by the user according to the assigned touch action. It is a possible configuration.

The related item extracting unit 23 extracts related items related to the object specified by the object specifying unit 22, that is, the object selected by the user. When an object is selected, an item deeply related to the selected object is extracted by the related item extraction unit 23.

For example, when the object is data such as “photo”, the photo is transferred to “display”, “edit”, “send as an email”, “peripheral device (TV etc.)” It is assumed that operations such as “print” and “print” are executed. Therefore, an item corresponding to an “action” executed on an object that is an “action target” may be extracted as a related item of the object.

Or, if the object is data such as “photograph”, it is assumed that the photo is sent to a specific person. Therefore, an item corresponding to the relationship of “action partner” when an action is executed on an object that is an action target may be extracted as a related item.

Alternatively, if the object is an “album” or “folder” that includes multiple photos or other data, the user may want the photos or data contained in the object. In this way, items belonging to the lower layer of the object may be extracted as related items.

The related information storage unit 42 stores related information indicating the relationship between objects and items. FIG. 6 is a diagram illustrating an example of related information stored in the related information storage unit 42.

As shown in FIG. 6, the related information is information in which at least “related items” are associated with each “object”. The association information indicates the association between the object and the item by this association.

When an object is specified by the object specifying unit 22, the related item extracting unit 23 refers to related information stored in the related information storage unit 42 and extracts items related to the specified object as related items. To do.

For example, as shown in FIGS. 5A to 5C, it is assumed that the object specifying unit 22 specifies that the selected object is “Photo 1”. In this case, since “Photo 1” is classified as “Photo” as an object, the related item extraction unit 23 extracts a related item group 60 associated with the object “Photo” from the related information.

Information on the related items extracted by the related item extracting unit 23 is supplied to the operation screen processing unit 24. The extracted related items are displayed so as to be selectable (for example, as icons) as items related to the previously selected object.

Although not limited to this configuration, in this embodiment, as shown in FIG. 6, an icon may be assigned to each “related item”. For example, the icon “1: TV” is associated with the related item “display on television (transfer to television)” associated with the object “photo”. The icon “1: TV” is, for example, an icon on which an illustration of a TV or the like is drawn, and is preferably a picture reminiscent of “sending a photograph to the TV for display”. Other related items are also assigned icons with appropriate patterns that recall the contents of the related items.

Based on such related information, the related item extracting unit 23 may supply the operation screen processing unit 24 with icons (or icon identification information) corresponding to the extracted related items. Thereby, the operation screen processing unit 24 can proceed to display the icon specified by the related item extraction unit 23.

In the present embodiment, the related information holds information indicating the nature or classification of the related item, that is, “attribute” for each related item. In the present embodiment, it is preferable that the attribute is an index for obtaining the possibility (height) that the related item is selected by the user. In the present embodiment, as an example of the attribute, information of “number of selections” is stored in the related information storage unit 42 for each related item. The attribute “number of selections” is information indicating how many times the related item has been selected so far by a user operation. The number of times of selection may be one in which the cumulative number of times since the tablet terminal 100 started to be used has been counted. Alternatively, the selected number of times may be reset every time an event such as power-off, elapse of a predetermined period, or history deletion, and be counted from the beginning each time. The attribute of the related item is not limited to the above, and the related information may hold any type of attribute as long as it is information indicating the nature or classification of the related item. The attribute of the related item is read by the operation screen processing unit 24 and each unit as necessary.

The operation screen processing unit 24 performs processing (operation screen generation processing) for generating an operation screen for displaying an object and a related item (its icon) related to the selected object in a selectable manner for the user. It is.

FIG. 7 is a diagram showing a specific example of the icon image stored in the icon storage unit 43. As shown in FIG. As shown in FIG. 7, in this embodiment, each icon image can be identified by icon identification information. For example, the icon identification information “1: TV” is associated with an icon image depicting a TV. In addition, although not shown, a portrait or an avatar image of the person may be used as an icon representing personal information such as an acquaintance who often makes a call.

The operation screen processing unit 24 reads the icon images assigned to the related items extracted by the related item extraction unit 23 from the icon storage unit 43, and performs operations so that these are displayed at an appropriate position and an appropriate timing. A screen is generated and output to the display unit 12 via a display control unit (not shown).

Specifically, in the present embodiment, the operation screen processing unit 24 has a function of displaying icons of related items related to the object selected by the contact operation along a predetermined icon arrangement pattern.

In the present embodiment, the operation screen processing unit 24 includes at least an icon rank determining unit 31 and an icon arrangement determining unit 33. The icon rank determination unit 31 and the icon arrangement determination unit 33 are responsible for some functions of the operation screen generation process executed by the operation screen processing unit 24.

The icon ranking determining unit 31 gives priority to the related items extracted by the related item extracting unit 23 based on the attributes of the related items. Alternatively, the icon rank determining unit 31 may give a priority to the icons associated with the extracted related items.

In this embodiment, as described above, the related information includes a field of “number of selections” as one of the attributes of the related item. Therefore, the icon order determination unit 31 assigns a priority to each related item or each icon in descending order of the “selection count” of the extracted related items. Here, it is considered that the related item having a larger number of selections is more likely to be selected by the user, and accordingly, a higher priority is given.

FIG. 8 is a diagram illustrating an example of the priority order assigned to each icon by the icon order determination unit 31.

For example, as shown in (a) or (c) of FIG. 5, when the object 80 (that is, the photograph 1) is selected and specified by the contact operation, the related item extracting unit 23 performs the relation shown in FIG. It is assumed that the related item group 60 is extracted from the items.

The icon rank determination unit 31 includes an attribute “number of selections” and icon identification information associated with each of the related items extracted from the related information (FIG. 6) stored in the contact information storage unit 44. And read. Then, the icon order determination unit 31 assigns priorities to the respective icons in the order of the number of selections indicated by the corresponding “number of selections”. In the example shown in FIGS. 6 and 8, when the icons are sorted in descending order of the “number of selections”, “2: printer”, “4: photo display”, “1: television”, “8: memory card”, “ “3: Mail”, “6: Palette”, “7: Trash”, and “5: Information display”, the priority order from the first place to the eighth place is given to these. The result of assigning the priority is returned to the operation screen processing unit 24.

The icon arrangement determination unit 33 determines the arrangement of icons of related items. The function of the icon arrangement determination unit 33 will be described with reference to FIG. FIG. 9 is a diagram illustrating a specific example of the operation of the icon arrangement determining unit 33. More specifically, FIG. 9A illustrates a specific example of the icon arrangement pattern acquired by the icon arrangement determining unit 33. FIG. 9B is a diagram illustrating a specific example of the icon arrangement determined by the icon arrangement determining unit 33.

In the present embodiment, first, the icon arrangement determining unit 33 acquires an icon arrangement pattern, and associates the above-described priority order with each icon arrangement position defined in the icon arrangement pattern. The icon arrangement pattern is defined by patternizing how many icons and where the icons are arranged on the touch panel. In the present embodiment, as an example, the tablet terminal 100 holds one fixed icon arrangement pattern in any area (not shown) of the storage unit 19, and the icon arrangement determination unit 33 displays the icon arrangement pattern. To get. However, the tablet terminal 100 is not limited to the above configuration, and the storage unit 19 may hold a plurality of types of icon arrangement patterns, and the icon arrangement determination unit 33 selects one icon arrangement pattern as necessary. It may be configured to. Alternatively, a configuration in which an arrangement pattern determination unit (not shown) dynamically determines an icon arrangement pattern according to the input movement trajectory of the contact operation and supplies the icon arrangement pattern to the icon arrangement determination unit 33 may be adopted.

In the present embodiment, as shown in FIG. 9A, an icon arrangement that defines that eight icons are arranged at equal intervals on the outline of a ring (vertically long ellipse) arranged over the entire screen of the touch panel. The pattern is held in the storage unit 19 as an example. However, FIG. 9A is not intended to limit the icon arrangement pattern of the present invention to this specific example.

In the present embodiment, the icon arrangement determining unit 33 associates priorities with the arrangement positions of the eight icons defined in the icon arrangement pattern shown in FIG.

Here, the icon arrangement determining unit 33 associates the icon arrangement position with the priority order based on the contact information stored in the contact information storage unit 44.

Specifically, the icon arrangement determination unit 33 acquires the contact information generated previously from the contact information storage unit 44. Here, the icon arrangement determination unit 33 may acquire only the contact coordinate information of the end point tn of the locus among the contact information. Next, the icon arrangement determining unit 33 plots the end point tn of the locus on the acquired icon arrangement pattern. And the icon arrangement | positioning determination part 33 associates a high priority in order from the arrangement position with the short distance with the end point tn.

Specifically, as shown in FIG. 9A, when the eight arrangement positions A to H are arranged in order of the distance from the end point tn, H, A, G, B, C, Assuming F, D, and E, the icon arrangement determining unit 33 associates the priority orders from the first place to the eighth place in this order.

As a result, the icon arrangement determining unit 33 can generate the rank association result shown in FIG. According to the ranking association result shown in FIG. 9B, the second position at the A position, the fourth position at the B position, the fifth position at the C position, the seventh position at the D position, the eighth position at the E position, The sixth position is associated with the F position, the third position with the G position, and the first position with the H position.

Next, the icon arrangement determining unit 33 arranges icons according to the priority of each icon determined by the icon order determining unit 31 and the arrangement position for each priority determined by itself. For example, the icon arrangement determining unit 33 determines to arrange the icon “1: TV” to which the priority “third place” is assigned at the third position (originally G) in FIG. To do.

The icon arrangement result determined by the icon arrangement determining unit 33 in this way is returned to the operation screen processing unit 24.

The operation screen processing unit 24 generates an operation screen in which each icon extracted according to the determination of the icon rank determination unit 31 and the icon arrangement determination unit 33 is arranged at a predetermined position, and is displayed on the display unit 12 via the display control unit. Output.

9A and 9B, the broken line indicating the outline of the ring is the shape of the ring held as information inside the tablet terminal 100, and actually the display unit 12 May not be displayed. Similarly, the broken line indicating the outline of the ring in each of the drawings shown below is not actually displayed on the display unit 12.

FIG. 10 is a diagram illustrating a specific example of the operation screen obtained as a result of the operation screen generation process executed by the operation screen processing unit 24. The example shown in FIG. 10 is a specific example of the operation screen obtained when the object 80 (object “Photo 1”) is selected, as in FIGS.

The operation screen processing unit 24 extracts “1: TV”, “2: Printer”, “3: Mail”, “4: Photo display”, “5: Information display”, “6” extracted according to the above-described procedure. : Palette ”,“ 7: Trash ”, and“ 8: Memory card ”are read from the icon storage unit 43 (FIG. 7). The operation screen processing unit 24 generates an operation screen according to the priority order determined as shown in FIG. 8 and the arrangement position determined as shown in FIG. 9B. The generated operation screen is output to the display unit 12 and displayed on the display unit 12 as shown in FIG.

The operation screen processing unit 24 may arrange the selected object 80 in the center of the screen as shown in FIG. In the above description, the operation screen processing unit 24 performs the process of arranging the object 80 in the center, but this is not an essential structure. However, in the present embodiment, the operation screen processing unit 24 arranges the object 80 in the center from the viewpoint of making it easy to see the icon so as not to overlap the object because the icon is arranged in a ring shape along the icon arrangement pattern. It is preferable.

[Operation screen display flow]
Next, a processing flow when the tablet terminal 100 executes the operation screen display function will be described. FIG. 11 is a flowchart showing a flow of operation screen display processing by the tablet terminal 100.

When the input unit 11 detects that an indicator (such as a user's finger) has touched the touch surface of the touch panel (YES in S101), the contact information generation unit 21 starts the finger from that time (t = t0). Acquisition of the contact coordinate information indicating the contact position is started, and this is acquired over time (S102). This tracking of the contact position is continued until no contact is detected between the touch surface and the finger (NO in S103). When contact is not detected in the input unit 11 (YES in S103), the contact information generation unit 21 obtains the contact coordinate information and the contact time information acquired from t = t0 to this time (t = tn). Corresponding contact information is generated (S104). When the contact operation for selecting an object is a double tap or the like, that is, when a non-contact occurs temporarily in a single contact operation for an extremely short time, the contact is temporarily made in a short time. What is necessary is just to comprise the contact information production | generation part 21 so that tracking may be continued as the contact operation is not over even if it is not detected.

The object specifying unit 22 compares the contact information generated in S104 (for example, (b) in FIG. 5) with the map information (for example, (c) in FIG. 5) stored in the frame map storage unit 41. Then, the object overlapping the area where the locus touched by the user exists is specified as the selected object (S105). In the example shown in FIG. 5C, the object 80 “Photo 1” is specified.

The related item extraction unit 23 refers to the related information (for example, FIG. 6) in the related information storage unit 42 based on the object specified in S105, and extracts the related item of the specified object (S106). Alternatively, identification information of icons assigned to related items may be extracted.

Subsequently, the operation screen processing unit 24 executes an operation screen generation process. First, the icon order determination unit 31 determines the priority order for each of the related items extracted in S106 (S107). The icon order determination unit 31 associates the determined priority order with the related item or the icon of the related item (for example, FIG. 8).

On the other hand, the icon arrangement determining unit 33 reads a predetermined icon arrangement pattern (for example, (a) in FIG. 9) from the storage unit 19, and assigns priority to some icon arrangement positions defined in the icon arrangement pattern. Associate (S108). The icon arrangement determining unit 33 associates the icon arrangement position with the priority order based on the contact information obtained in S104. More specifically, the icon arrangement determination unit 33 performs association so that a higher rank is assigned to an icon arrangement position with a shorter distance from the end point of the finger trajectory (for example, (a) and ( b)). Note that S107 and S108 may be executed in parallel, or may be executed sequentially in an arbitrary order in series.

Next, the icon arrangement determining unit 33 determines the arrangement of each extracted icon according to the priority determined by the icon order determining unit 31 and the arrangement position determined by itself (S109). The icon arrangement determining unit 33 returns the icon arrangement result to the operation screen processing unit 24.

Finally, the operation screen processing unit 24 acquires the icon image of the related item extracted in S106 from the icon storage unit 43 (for example, FIG. 7). Then, according to the determination in S109, an operation screen on which the acquired icon image is arranged is generated (S110). In the above-described example, the operation screen processing unit 24 generates an operation screen in which the selected object is arranged in the center and each icon is arranged in a ring shape around the selected object.

The video signal of the operation screen generated as described above is output to the display unit 12. As shown in FIG. 10, the operation screen is displayed on the display unit 12 of the tablet terminal 100. According to the operation screen generated by the tablet terminal 100 according to the present embodiment, each icon has a higher priority, that is, a “selection count”, that is, an icon that is more likely to be selected by the user. It is arranged near the end point tn.

According to the above-described configuration and method of the present invention, the tablet terminal 100 displays the icons of related items related to the selected object in a selectable manner with respect to one contact operation of the user for selecting the object. The result can be output. In addition, the arrangement of the icons displayed so as to be selectable here is an arrangement in consideration of the user's contact operation. In other words, the icons that are more likely to be selected by the user are displayed closer to the position (end point tn of the trajectory) where the user has finished the previous contact operation.

The user is expected to perform the contact operation of selecting the icon of the related item related to the object immediately after performing the contact operation of selecting the previous object, and this is a natural flow of operation. In this case, the user moves the indicator (such as a finger) from the position where the previous contact operation is completed to the position where the target icon is displayed next.

Here, the longer the distance from the position where the previous contact operation has been completed to the position where the target icon is displayed, the longer the indicator must be moved, and the selection operation becomes troublesome for the user. . Such annoyingness becomes more noticeable as the screen size of the touch panel is larger, and when the user is operating with one hand as shown in FIG. It becomes a particularly serious problem. Even when the objects are managed in a hierarchy and the selection operation is performed repeatedly, the annoyance is further increased.

Therefore, as in the present invention, by arranging the icons that are more likely to be selected to be displayed near the completion position of the previous contact operation, the user can predict the previous contact operation with a high probability. It is only necessary to select a desired icon displayed near the completion position. As a result, it is possible to avoid the troublesome selection operation described above with a high probability.

As described above, according to the tablet terminal 100 of the present invention, it is not necessary for the user to unnaturally move the indicator from one place to another on the screen, and the target final result can be reached with a simple contact operation. it can. In addition, it is possible to suppress the induction of erroneous operations as the moving distance of the indicator is shorter.

From the above, the tablet terminal 100 can display the final result desired by the user in a natural flow that does not contradict the user's intuition while having a simple contact operation and a small number of operations. As a result, the tablet terminal 100 including the touch panel has an effect that it is possible to realize excellent operability.

(Other examples of contact movement and icon arrangement pattern)
In the present embodiment, the case where the user's contact operation for selecting an object is an operation of “enclosing” has been described. Further, as a specific example of the fixed icon arrangement pattern, a case has been described in which an icon arrangement pattern that defines that eight icons are arranged at equal intervals on a vertically long elliptical outline has been described.

However, the configuration of the tablet terminal 100 of the present invention is not limited to the above. The tablet terminal 100 can determine a gesture other than the contact operation “enclose” as a gesture for selecting an object. Moreover, the tablet terminal 100 may hold | maintain the icon arrangement pattern different from the icon arrangement pattern shown to (a) of FIG.

FIG. 12 is a diagram for explaining the operation of the contact information generating unit 21 and the object specifying unit 22. More specifically, FIG. 12A is a diagram illustrating that the user has performed a contact operation of “checking” an object at a check point in order to select a target object. FIG. 12B is a diagram illustrating an example of contact information generated by the contact information generation unit 21 in accordance with the contact operation illustrated in FIG.

As shown in FIG. 12A, it is assumed that the user performs a contact operation of “checking” the object 80 (here, the photograph 2) displayed on the touch panel of the tablet terminal 100 at a check mark. For example, it is assumed that the contact operation is performed in a period from t0 to tn so that the contact point passes the position of the broken line in FIG.

The contact information generation unit 21 generates the contact information shown in FIG. Of the user's “check” contact movement trajectory, the start point is indicated as t0 and the end point is indicated as tn. However, the contact information generation unit 21 may associate the contact time information with each point in between.

The object specifying unit 22 acquires the contact information shown in FIG. 12B from the contact information generating unit 21. Then, the object specifying unit 22 acquires map information shown in FIG. 5C from the frame map storage unit 41. Then, the object specifying unit 22 compares the contact information with the map information, and an object 80 (here, Photo 2) that overlaps with or substantially overlaps the region where the locus of the user's finger obtained from the contact information exists. Is identified as the selected object. Or in this modification, you may specify the object which overlaps the point with the largest Y coordinate (namely, the turning point of a check point) among each point which comprises the locus | trajectory of a check point as an object selected by the user.

Next, the related item extracting unit 23 extracts related items of the specified object 80 (photo 2). Here, it is assumed that the related item group 60 (FIG. 6) is extracted as in the first embodiment, and the icon order determination unit 31 assigns a priority to each related item as in the first embodiment (FIG. 8). Shall.

On the other hand, the icon arrangement determining unit 33 may acquire the icon arrangement pattern shown in FIG. 13 from the storage unit 19 instead of the icon arrangement pattern shown in FIG. FIG. 13 is a diagram illustrating another specific example of the icon arrangement pattern acquired by the icon arrangement determining unit 33.

In the example shown in FIG. 13, the icon arrangement pattern divides the touch panel screen evenly into vertical × horizontal = 4 × 3 blocks, and icons are added to the surrounding 10 blocks excluding the blocks near the center among the divided blocks. It is defined that they are arranged one by one.

Subsequently, the icon arrangement determining unit 33 obtains the distance between the end point tn of the trajectory and the icon arrangement position determined at the center of each block. Then, the placement position and the priority order are associated with each other so that a higher priority order is assigned to the placement position whose distance from the end point tn is shorter. The numbers shown at the respective icon arrangement positions in FIG. 13 indicate the priorities associated with the icon arrangement determining unit 33 in this modification. As shown in FIG. 13, the priority “first place” is naturally associated with the arrangement position of the upper right block closest to the end point tn.

FIG. 14 is a diagram illustrating another specific example of the operation screen obtained as a result of the operation screen generation process executed by the operation screen processing unit 24. The example shown in FIG. 14 shows an operation obtained when the object 80 (here, photo 2) is selected based on the contact information shown in FIG. 12B and the icon arrangement pattern shown in FIG. It is a specific example of a screen.

The operation screen processing unit 24 extracts “1: TV”, “2: Printer”, “3: Mail”, “4: Photo display”, “5: Information display”, “6” extracted according to the above-described procedure. : Palette ”,“ 7: Trash ”, and“ 8: Memory card ”, the priority determined by the icon order determination unit 31, and the icon arrangement determination unit 33 as shown in FIG. The operation screen is generated according to the arrangement position. The generated operation screen is output to the display unit 12 and displayed on the display unit 12 as shown in FIG.

Although the following is not an essential configuration, the operation screen processing unit 24 may arrange the selected object 80 at the center of the screen as shown in FIG. In the present embodiment, the operation screen processing unit 24 arranges icons around the screen of the touch panel along the icon arrangement pattern shown in FIG. 13, so that the icons do not overlap with the objects and are easy to see. 80 is preferably arranged in the center.

As shown in FIG. 13, even when the icon arrangement pattern defines the arrangement positions of 10 icons, there may be cases where the related items actually extracted by the related item extraction unit 23 are less than 10. Also good. In the example illustrated in FIG. 14, the icon arrangement determination unit 33 determines that no icon is arranged in the blocks associated with the priority orders “9th” and “10th”.

According to the tablet terminal 100 of the present invention in the above-described example, the user does not need to unnaturally move the indicator around the screen, and can reach the target final result with a simple contact operation. . In addition, it is possible to suppress the induction of erroneous operations as the moving distance of the indicator is shorter.

<< Embodiment 2 >>
Another embodiment of the information processing apparatus according to the present invention will be described below with reference to FIGS. 15 to 24B. For convenience of explanation, members having the same functions as those in the drawings described in the first embodiment are denoted by the same reference numerals, and description of the same contents as those in the first embodiment is omitted.

In the present embodiment, the tablet terminal 100 is configured to accept an “enclose” contact operation as an operation for selecting an object, and the operation screen processing unit 24 has a ring shape associated with the “enclose” contact operation. It is the structure which arrange | positions an icon along. Thereby, according to a user's intuition, the result recalled more intuitively from contact operation | movement can be output.

[Tablet function]
FIG. 15 is a functional block diagram illustrating a main configuration of the tablet terminal 100 according to the present embodiment.

The tablet terminal 100 according to the present embodiment has a configuration in which the control unit 10 further includes a ring shape determining unit 30 as a functional block, as compared with the tablet terminal 100 (FIG. 1) of the first embodiment.

Here, in the present embodiment, the control unit 10 of the tablet terminal 100 is not essential, but may further include a gesture determination unit 25 and an animation determination unit 32 as functional blocks as necessary. .

The ring shape determination unit 30 determines an icon arrangement position when the operation screen processing unit 24 arranges an icon. In particular, when arranging icons in a ring shape, the ring shape determination unit 30 determines the shape of the ring. Is.

In the first embodiment, the operation screen processing unit 24 is configured to arrange icons in accordance with an arbitrary fixed icon arrangement pattern. In the present embodiment, when an object is selected by the touch action “enclose”, the ring shape determination unit 30 determines an icon arrangement pattern that defines that icons are arranged in a ring shape, and the operation screen processing unit Each icon is arranged and displayed in a ring shape according to the icon arrangement pattern 24 determined.

That is, the ring shape determining unit 30 is a functional block configured to determine the icon arrangement pattern by specializing the arrangement pattern determining unit (not shown) described in the first embodiment to the shape of the ring.

In accordance with the icon arrangement pattern based on the ring shape determined by the ring shape determining unit 30, the operation screen processing unit 24 arranges the extracted icons. The ring shape information determined by the ring shape determination unit 30 may further include ring size information and / or ring position information.

As described above, it is preferable that the control unit 10 of the tablet terminal 100 further includes the gesture determination unit 25. When various types of contact actions (gestures) performed on the input unit 11 other than “enclose” are assumed, it is determined whether the gesture is “enclose” or another gesture. Because it is necessary to do.

The gesture determination unit 25 determines what gesture it is for the contact operation performed on the input unit 11. For example, the gesture determination unit 25 can determine gestures such as “tap”, “flick”, “pinch”, “drag”, and “enclose”. A known technique can be appropriately employed as an algorithm for determining a gesture.

The gesture determination unit 25 instructs each unit of the control unit 10 to execute processing corresponding to the determined gesture according to the determination result.

In this embodiment, when the gesture determination unit 25 determines that the detected contact action is a gesture of “enclose”, the contact information generation unit 21 sends the generated contact information to the contact information storage unit 44. It is preferable to instruct to store. As a result, the operation screen processing unit 24 can refer to all information (such as the position, size, locus, contact time, contact point movement timing, etc.) of the gesture “enclose”, and the contact operation can be performed. If the gesture is other than the “enclose” gesture, it is possible to avoid unnecessary writing to the contact information storage unit 44. However, the contact information generation unit 21 may be configured to write all contact information in the contact information storage unit 44 regardless of the determination result of the gesture determination unit 25.

As described above, the control unit 10 of the tablet terminal 100 may further include an animation determination unit 32. The animation determination unit 32 determines an animation to be given to all objects to be arranged on the operation screen, that is, objects, icons, rings, and the like. Thereby, when displaying an object and an icon, the visual effect (namely, animation) can be given to how to display.

Note that the animation determination unit 32 may give a visual effect such as fade-in (change of transparency) in addition to the movement of the object, icon, or ring.

Further, the animation determination unit 32 may move the icons so that the icons are finally terminated on the contour line from different places instead of causing the icon to be displayed in a circular shape to appear on the contour line of the ring from the beginning. . For example, after the icons are aggregated around the center of the ring, the animation determination unit 32 adds a movement that diffuses each icon, and moves each icon so that it is finally arranged on the outline of the ring. May be attached.

(Example of arrangement of object and ring shape 1)
FIG. 16 is a diagram illustrating the processing contents of each unit of the operation screen processing unit 24 in the present embodiment. More specifically, FIG. 16A is a diagram for explaining an example of object display processing executed by the operation screen processing unit 24, and FIG. 16B is a diagram illustrating the operation screen processing unit 24. It is a figure which shows an example of the icon arrangement pattern specialized in the ring shape which the ring shape determination part 30 determined.

In this embodiment, as shown in FIG. 16A, the operation screen processing unit 24 determines to reposition the object 80 selected by the previous “enclose” contact operation at the center. At this time, the animation determination unit 32 may give the object 80 an animation in which the object 80 gradually moves from the original position to the center.

Next, in the present embodiment, the ring shape determination unit 30 of the operation screen processing unit 24 determines an icon arrangement pattern based on the ring shape. Specifically, the ring shape determination unit 30 acquires the icon arrangement pattern stored in the storage unit 19, thereby to obtain the ring shape, position, size, number of icons, icon arrangement for arranging icons. The position etc. can be determined.

In FIG. 16B, the ring shape determination unit 30 determines an icon arrangement pattern that defines that eight icons are arranged uniformly along the outline of an elliptical ring around the object 80. An example is shown.

Here, the example of FIG. 16B in which the shape of the reference “ring” that defines the icon arrangement position defined in the icon arrangement pattern is an ellipse is merely an example, and the present invention There is no intention to limit the shape of the ring. Further, the “ring” does not necessarily mean a shape formed by a curve. For example, the ring shape determining unit 30 may define the shape of the ring as a circle, square, rectangle, or other polygon, or may be a complex shape, irregular shape, or non-geometric shape. A figure having an outline that separates the outside from the outside may be defined as a ring. Further, “ring” does not necessarily mean a closed curve. Even if the starting point and the ending point of the contour line of the ring are not completely coincident with each other, a contour line that largely separates the inside and the outside may be defined as the ring. The operation screen processing unit 24 can place an icon on the outline of the ring of any shape defined as described above according to the determination of the ring shape determination unit 30.

(Example of arrangement of object and ring shape 2)
In the present embodiment, in the example shown in FIGS. 16A and 16B, the ring shape determination unit 30 defines an icon arrangement in which a predetermined shape, a predetermined position, and a predetermined size are defined for a ring for arranging icons. The icon arrangement pattern is determined by acquiring the pattern from the storage unit 19.

However, the present invention is not limited to such a configuration, and the ring shape determination unit 30 according to the present embodiment uses the input contact movement “enclose” movement trajectory (contact information stored in the contact information storage unit 44). On the basis of this, the icon arrangement pattern may be determined by dynamically determining the shape of the ring. More specifically, the ring shape determination unit 30 can change the shape of the trajectory of the “surround” operation into a ring shape for arranging icons as it is. Also, the size of the ring can be determined based on the size of the area enclosed by the “enclose” operation. Further, the position of the ring can be determined based on the position of the enclosed region. The ring shape determination unit 30 may further determine the ring shape based on the map information stored in the frame map storage unit 41. That is, the size and position of the ring may be determined according to the display position and size of the enclosed object. The operation of the operation screen processing unit 24 in such a case will be described with reference to FIGS. 17 and 18.

FIG. 17 is a diagram illustrating a specific example of the contact information stored in the contact information storage unit 44. More specifically, FIG. 17A is a diagram showing that the user has performed a contact operation of “enclosing” an object in an arbitrary shape in order to select a target object. FIG. 17B is a diagram illustrating an example of contact information generated by the contact information generation unit 21 in accordance with the contact operation illustrated in FIG.

As shown in FIG. 17A, the user performs a contact operation of “enclosing” an object (here, photo 1) displayed on the touch panel of the tablet terminal 100 in an arbitrary shape (for example, a heart shape). For example, it is assumed that the contact operation is performed in the period from t0 to tn so that the contact point passes through the position of the broken line in FIG.

The gesture determination unit 25 acquires contact information as illustrated in FIG. 17B from the contact information generation unit 21. In FIG. 17B, among the traces traced by the user, the start point is indicated as t0 and the end point is indicated as tn. However, the contact time information may be associated with each point in between.

The gesture determination unit 25 determines that this contact operation is a gesture of “enclose” based on the contact information shown in FIG. The gesture determination unit 25 instructs the contact information generation unit 21 to store the contact information shown in FIG. 17B in the contact information storage unit 44. Thereby, each part of the operation screen process part 24 can refer to the contact information shown by (b) of FIG. 17 memorize | stored in the contact information storage part 44, when performing the process which displays an icon. .

FIG. 18 is a diagram illustrating another example of the icon arrangement pattern determined by the ring shape determining unit 30.

First, the operation screen processing unit 24 can place the selected object 80 (here, Photo 1) in the center. Next, in the present embodiment, the ring shape determination unit 30 acquires the contact information stored in the contact information storage unit 44. Based on the movement trajectory of the finger tip (contact point) obtained from the contact information, the ring shape determination unit 30 determines a shape that is the same as or similar to the trajectory as the ring shape for arranging the icons. To do. In the present embodiment, as an example, the ring shape determination unit 30 determines to place the ring at the center and to place it as large as possible on the touch screen. As shown in FIGS. 17A and 17B, the object 80 is surrounded by a heart shape. Therefore, the ring shape determination unit 30 determines the ring shape 81 so that the similar shape of the heart-shaped locus is arranged in the center of the screen as shown by the broken line in FIG. At this time, for each icon arrangement position on the ring shape 81, the ring shape determination unit 30 may determine that the icons are arranged at equal intervals, or the icon can be placed at an arbitrary position on the contour line according to another rule. You may decide to arrange.

And the icon arrangement | positioning determination part 33 of the operation screen process part 24 makes the end point tn of the locus | trajectory shown in FIG.17 (b), and each icon arrangement position on the ring shape 81 determined by the ring shape determination part 30. FIG. A priority order is associated with each icon arrangement position according to the distance.

The operation screen processing unit 24 arranges an icon on the outline of the ring shape 81 determined by the ring shape determination unit 30 as shown in FIG. At this time, the icon arrangement determination unit 33 determines the icon arrangement position so that the priority order given to each icon by the icon order determination unit 31 matches the priority order associated with the arrangement position.

The ring shape determining unit 30 of the operation screen processing unit 24 may determine the approximate shape of the trajectory as the ring shape when the trajectory has an extremely complicated shape. By rounding a fine and distorted line of a locus with a straight line or a curve, the amount of information defining the shape of the ring can be reduced, and the processing load for arranging icons can be reduced.

(Example of arrangement of object and ring shape 3)
In each of the above-described arrangement examples, the operation screen processing unit 24 displays the selected object (for example, the object 80 in FIG. 16A, FIG. 16B, or the object 80 in FIG. 18) when the icon is arranged. The ring shape determination unit 30 is configured to determine the position of the ring so as to arrange icons around the center object 80. However, the configuration of the present invention is not limited to the above. The operation screen processing unit 24 may be configured to maintain the display position of the selected object 80 as it is. Even in this case, the ring shape determination unit 30 may display the ring shape on the screen as shown in FIG. The position and size of the ring may be determined so as to be displayed large in the center. FIG. 19 shows another arrangement example of the object and the circular icon group.

(Example 4 of arrangement of object and ring shape)
Alternatively, in the configuration in which the display position of the selected object 80 is maintained as it is, the ring shape determination unit 30 may cause the object 80 at the original position to be in the center of the ring as shown in FIG. The position and size of the ring shape may be determined. FIG. 20 shows another arrangement example of the object and the circular icon group.

(Object and ring shape arrangement example 5-with animation)
In the configuration in which the display position of the selected object 80 is maintained as it is, when the ring shape is largely arranged at the center of the screen regardless of the object 80, it is said that “the icon is displayed surrounding the object 80”. There is a problem that the relationship between the event and the finally obtained result (for example, FIG. 19) is weakened. On the other hand, in the configuration in which the display position of the object 80 is maintained as it is, when icons are displayed around the object 80 in association with the display position of the object 80, depending on the position where the object 80 is originally displayed, There is a problem that the shape is not arranged so as to fit on the screen with a sufficient size, and as a result, the visibility of the icon decreases (for example, FIG. 20).

Therefore, when the tablet terminal 100 includes the animation determination unit 32, the animation determination unit 32 solves the above problem by adding animation to a ring for arranging icons as shown in FIG. May be. Specifically, the animation determination unit 32 sets the ring shape determined by the ring shape determination unit 30 based on the original display position and size of the object 80 so that the ring shape is largely arranged in the center of the screen over a certain period of time. Add animation to the ring. As a result, the ring of icons once arranged small around the object 80 gradually changes its shape with the passage of time, and is finally arranged large in the center of the screen. 21A shows a state of an icon ring initially arranged around the object 80, and FIG. 21B shows a state in which the icon ring is being expanded. (C) shows a state in which the ring of icons is finally arranged largely at the center of the screen from (a) of FIG.

According to the above configuration, the icon can be displayed in a sufficient size without impairing the relevance between the user's contact operation and the displayed result.

Note that the animation determination unit 32 may give an animation that gradually increases the size of each icon according to the size of the ring, or the icon size is independent of the size of the ring and the size of the icon. The animation may be given such that the size of the icon is fixed and the interval between the icons is gradually increased.

(Icon display timing)
In each of the above arrangement examples, the operation screen processing unit 24 is configured to simultaneously arrange a plurality of icons around the selected object. However, the present invention is not limited to this, and when the operation screen processing unit 24 includes the animation determination unit 32, the animation determination unit 32 may determine the display timing of each icon.

FIG. 22 is a diagram showing a modification of the related item icon display method.

For example, it is assumed that contact information as shown in FIG. 22A is obtained from the user's “enclose” contact operation. The animation determination unit 32 refers to the contact information stored in the contact information storage unit 44, and recognizes that the “enclose” gesture has occurred clockwise from t0 to tn. The animation determination unit 32 determines that the first to eighth icons appear one by one at regular intervals in a clockwise direction so as to match this movement. For example, the animation determining unit 32 causes icons to appear in order at regular intervals (b), (c), (d), (e),... In FIG. The display timing of the icons is controlled so that the operation screen shown in FIG.

According to the above configuration, a result with substantially the same movement as the movement (clockwise) when the user is surrounded can be obtained, so that the relevance between the user's contact operation and the displayed result can be further increased. As a result, the operation screen can be provided in a natural flow that does not contradict the user's intuition.

Here, the icon arrangement determining unit 33 roughly matches the display position of the first icon with the start point of the finger trajectory (contact position at time t0), and the display position of the last icon and the end point (tn It is preferable that the contact position at the time is approximately the same.

This makes it possible to provide an operation screen with a more natural flow by further linking the contact operation and the result.

Note that, here, the order in which icons appear and the priority assigned by the icon order determination unit 31 do not have to match. For example, in the example shown in FIG. 22, the icon of the priority “1st place” is arranged at the H arrangement position ((f) in FIG. 22) closest to the end point tn of the trajectory. Instead of appearing in the second place, it will appear in the fourth place ((e) in FIG. 22).

(Icon display timing 2)
Furthermore, it is preferable that the animation determination unit 32 sequentially causes each icon to appear in accordance with not only the finger movement (clockwise or counterclockwise) but also the speed at which the finger moves when the object is surrounded.

Specifically, it is assumed that contact information as shown in FIG. This contact information indicates a trajectory enclosed in a clockwise direction from time t0 to time tn. More specifically, the contact position (tip of the finger) is at the left of the object at the time ta, the contact position is at the upper left of the object at the time tb, and the contact position is at the upper right of the object at the time tc. It can be seen from this contact information.

Therefore, when the eight icons are arranged at equal intervals on the ellipse, the animation determination unit 32 causes the first icon to appear immediately below the object at time t0, and then matches the finger speed. The icon appears up to the left (third) of the object at the time ta, the icon appears up to the upper left (fourth) of the object at the time tb, and the object appears at the time tc. The icons appear up to the upper right (sixth), and finally, it is determined that all objects appear at the time point tn. FIG. 23B shows the operation screen at time t0. FIG. 23C shows an operation screen at the time point ta. (D) of FIG. 23 has shown the operation screen at the time of tb. FIG. 23E shows the operation screen at the time tc. FIG. 23 (f) shows the operation screen at time tn.

According to the above configuration, a result is obtained that has substantially the same movement as the movement of the user (clockwise) and the speed of the movement. Therefore, the relationship between the user's contact operation and the displayed result is obtained. The operation screen can be provided in a natural flow that does not contradict the user's intuition.

Also here, the order in which the icons appear and the priority order given by the icon order determination unit 31 do not have to match. In the example shown in FIG. 23, the icon of the priority “1st place” is arranged at the H arrangement position ((f) in FIG. 23) closest to the end point tn of the trajectory. Instead of appearing, it appears fourth ((d) in FIG. 23).

[Operation screen display flow]
24A and 24B are flowcharts showing a flow of operation screen display processing by the tablet terminal 100 in the present embodiment.

When the input unit 11 detects that an indicator (such as a user's finger) has touched the touch surface of the touch panel (YES in S201), the contact information generation unit 21 starts the finger from that time (t = t0). The acquisition of the contact coordinate information indicating the contact position is started and acquired over time (S202). This tracking of the contact position is continued until no contact is detected between the touch surface and the finger (NO in S203). When contact is not detected in the input unit 11 (YES in S203), the contact information generation unit 21 obtains the contact coordinate information and the contact time information acquired from t = t0 to this time (t = tn). Corresponding contact information is generated (S204).

Here, the gesture determination unit 25 may determine the gesture of the contact operation based on the contact information (S205). In the present embodiment, if the determined gesture is not “enclose” (NO in S206), the gesture determination unit 25 instructs each unit of the control unit 10 to execute a process according to the determined other gesture. Each unit performs processing according to the determined gesture (S207).

On the other hand, when the determined gesture is “enclose” (YES in S206), the gesture determination unit 25 instructs the contact information generation unit 21 to store the contact information in the contact information storage unit 44. . The contact information generation unit 21 stores the contact information generated in S204 in the contact information storage unit 44 (S208).

The object specifying unit 22 includes contact information stored in the contact information storage unit 44 (for example, FIG. 5B or FIG. 17B) and map information stored in the frame map storage unit 41 ( For example, by comparing with (c) of FIG. 5, an object that overlaps the area surrounded by the user is specified as the selected object (S209). In the example shown in FIG. 5C, the object 80 “Photo 1” is specified.

The related item extraction unit 23 refers to the related information (for example, FIG. 6) in the related information storage unit 42 based on the object specified in S209, and extracts the related item of the specified object (S210). Alternatively, identification information of icons assigned to related items may be extracted.

Subsequently, the operation screen processing unit 24 may acquire the contact information generated in S204 from the contact information storage unit 44 as necessary (S211). Then, the ring shape determination unit 30 of the operation screen processing unit 24 dynamically determines an icon arrangement pattern based on the ring shape based on the contact information (S212). Alternatively, the operation screen processing unit 24 may determine the icon arrangement pattern by acquiring the icon arrangement pattern held in the storage unit 19. Here, the ring shape determination unit 30 determines an icon arrangement pattern based on the ring shape, which position and size the ring is arranged with respect to the object, how many icons are arranged, It is determined what kind of shape each icon is placed on the outline of the ring, where the icon is placed on the outline (for example, FIG. 9A, FIG. 16B, FIG. 18 to 23).

Next, the operation screen processing unit 24 executes an operation screen generation process. Specifically, the icon ranking determining unit 31 of the operation screen processing unit 24 gives priority to each of the related items extracted in S210 (S213). Specifically, the icon rank determination unit 31 reads the extracted attribute of the related item (for example, “the number of selections” in FIG. 6) from the related information storage unit 42. Then, the order of each related item is determined in order of the possibility of being selected by the user determined based on the attribute (for example, FIG. 8). The icon order determination unit 31 may associate a priority order with the icon image or icon identification information associated with the related item.

Meanwhile, the icon arrangement determining unit 33 of the operation screen processing unit 24 associates each icon arrangement position defined in the icon arrangement pattern determined in S212 with the priority (S214). Specifically, the icon arrangement determining unit 33 acquires the coordinates of the end point of the trajectory from the contact information obtained in S211 and associates the priorities in order from the arrangement position with the shortest distance from the end point. Note that S213 and S214 may be executed in parallel, or may be executed sequentially in an arbitrary order in series. Furthermore, S213 may be executed before S211 or S212 as long as it is a step after S210.

Subsequently, the icon placement determination unit 33 sets each priority so that the priority assigned to each icon by the icon order determination unit 31 in S213 matches the priority order associated with each icon placement position in S214. It is determined to arrange an icon (S215).

Here, when the tablet terminal 100 further includes an animation determination unit 32, the animation determination unit 32 includes an arrangement object (object, ring, and each) whose arrangement position is determined in each upstream process. An animation may be given to an icon or the like as necessary (S216). For example, the animation determination unit 32 determines the appearance timing of each placement object, decides to gradually change the position, shape, and size of the placement object, fade-in (changes in transparency), and the like. Or other visual effects may be determined.

Finally, the operation screen processing unit 24 acquires the icon image of the related item extracted in S210 from the icon storage unit 43 (for example, FIG. 7). Then, according to the content determined in each upstream process, the object specified in S209 is arranged, and the acquired icon image is placed on the outline of the ring arranged around the object or at a predetermined position with respect to the object. The operation screen is generated by arranging them (S217). For example, the operation screen processing unit 24 places the object in the center and places each icon on the ring shape arranged around the object, that is, on the ring-shaped outline determined in S212. Moreover, in the final product, the icons arranged are arranged closer to the trajectory end point of the “enclose” contact operation as the possibility of being selected is higher. The video signal of the operation screen generated as described above is output to the display unit 12.

According to the above-described configuration and method of the present invention, the tablet terminal 100 displays an icon around the object in response to an extremely natural and simple user contact operation of “enclosing” the object. Can output an operation screen (final result) on which is placed.

The user can obtain an operation screen in which icons of related items are arranged so as to surround the object as a result. The positional relationship between these icons and the object matches the positional relationship between the object and the trajectory of the finger by the previous contact operation previously performed by the user. In addition, the finger trajectory obtained by surrounding is similar to a ring shape in which icons are arranged.

In other words, the transition from the phenomenon of “contacting the object to“ enclose ”the object” to the phenomenon of “obtaining an operation screen with icons arranged around the object” is a natural transition that does not contradict the user's intuition. It can be said that there is.

In addition, the tablet terminal 100 can preliminarily detect related items that the user will select after selecting an object, and can display the related items in a selectable manner for the user. Specifically, according to the above configuration of the present invention, the icons displayed around the object are all related item icons extracted as items related to the object. That is, after the user surrounds and selects an object, the user can immediately designate “motion”, “motion target”, “action partner”, and the like related to the object from surrounding icons.

In addition, on the operation screen as the final result, the icons that are more likely to be selected are arranged closer to the movement completion position of the previous contact operation. Accordingly, the user selects (touches) an icon displayed near the movement completion position of the previous contact operation with a high probability as a desired icon. As a result, it is possible to avoid the troublesome selection operation of moving the indicator unnaturally from place to place on the screen with high probability.

Furthermore, according to the configuration in which the ring shape determining unit 30 dynamically determines the icon arrangement pattern (specifically, the ring shape) based on the movement trajectory of the indicator, the following effects can be obtained.

According to the above-described configuration and method of the present invention, the tablet terminal 100 displays an icon around the object in response to an extremely natural and simple user contact operation of “enclosing” the object. The operation screen for placing the image can be output as the final result. That is, the user can obtain, as a result, an operation screen in which icons of related items are arranged so as to surround the object.

Specifically, the user performs an operation of surrounding the object in an arbitrary shape with a free hand, and the locus at this time is held in the tablet terminal 100. Then, on the operation screen created by the tablet terminal 100, each icon is arranged so as to surround the object on the same or similar ring-shaped contour line as the obtained trajectory.

The positional relationship between these icons and the object matches the positional relationship between the object and the locus of the finger by the contact operation previously performed by the user. Further, the trajectory of the finger obtained by enclosing it matches the ring shape where the icon is arranged.

That is, when an object is enclosed, an operation screen in which icons are arranged "as enclosed" around the object can be obtained. It can be said that the transition of this event is a more natural flow that does not contradict the user's intuition. In addition, since the icons are arranged in the shape as enclosed by the user, the playability when the operation screen is displayed and the tablet terminal 100 is operated is improved. In addition, since the user can predict the arrangement of the icons, surround the object as he / she desires, and display the icons of the related items, the operability is further improved.

The icons displayed in the surroundings after selecting an object indicate related items that are deeply related to the object and are likely to be selected next.

From the above, the tablet terminal 100 can display the final result desired by the user with a more natural flow that does not contradict the user's intuition, while having a simple contact operation and a small number of operations. As a result, the tablet terminal 100 including the touch panel has an effect that it is possible to realize excellent operability.

≪Modification≫
A tablet terminal 100 in which the first and second embodiments are appropriately combined also falls within the scope of the present invention. That is, the control unit 10 of the tablet terminal 100 according to each of the first and second embodiments partially includes the gesture determination unit 25, the ring shape determination unit 30, and the animation determination unit 32 when not essential in each embodiment. Or all of them may be provided.

(Priority determination method)
In each of the above-described embodiments, the attribute associated with the related item in the related information is “selection count”, and the icon rank determination unit 31 may refer to the attribute “selection count” and be selected by the user. The configuration in which the priority order is given according to the height of the above has been described. However, the configuration of the tablet terminal 100 of the present invention is not limited to the above. The icon order determination unit 31 may assign priorities according to various attributes other than the attribute “number of times of selection”.

For example, as described below, a related item having a higher correlation with the object compared to the attribute of the selected object may be determined as a related item that is more likely to be selected by the user. Good.

More specifically, it is assumed that there is a single “photo” with the selected object, and other photos related to the photo are related as a plurality of related items in the related information. In this case, in the related information, information such as “photographing date / time”, “photographer”, “camera model”, “photograph title” and the like are associated with each of the related items and stored in the related information storage unit 42. Has been.

Therefore, for example, the icon order determination unit 31 associates the related items (other photographs) taken at the photographing date and time closer to the photographing date and time of the selected object “photo” so that the priority order is higher. The order of items may be determined. Therefore, the icon arrangement determination unit 33 sets the icon closer to the end point of the trajectory as “photograph” as the selected object is closer to the photographed date and time with respect to “the plurality of other photos” as the extracted related items. Decide to place.

It is assumed that the user wants to select related photos one after another. Photos with similar shooting dates and times are considered to be more closely related. Next to “Photo” as the previously selected object, “other photos” that are related to it, that is, with a similar shooting date and time. Is likely to be selected.

Therefore, as in the present invention, by arranging the icons that are closer to the shooting date and time, that is, icons that are more likely to be selected, to be displayed near the movement completion position of the previous contact operation, the user can A desired icon displayed near the movement completion position of the previous contact operation is selected with high probability. As a result, it is possible to avoid the troublesome selection operation of moving the indicator unnaturally from place to place on the screen with high probability.

Note that the icon rank determination unit 31 does not determine the high possibility of selection based on one attribute, but rather includes a plurality of attributes (for example, “shooting date”, “photographer”, “camera model”). ”,“ Photo title ”, etc.) may be used comprehensively to determine the likelihood of being selected. For example, the icon rank determination unit 31 compares a plurality of attributes of the selected object with a plurality of attributes of each related item to obtain a total similarity, and a higher similarity is selected. It may be determined that there is a high possibility that Here, it is considered that a related item having an attribute similar to that of the selected object is more likely to be selected by the user, and accordingly, a higher priority is given.

Alternatively, for example, it is assumed that the selected object is a tool “display video content list” and a plurality of “video content” are arranged as icons as related items. In this case, in the related information, information such as “recommendation degree”, “genre”, and “performer name” is associated with each of the related items and stored in the related information storage unit 42. “Recommendation level (recommendation level)” is information indicating the degree of recommendation for viewing to the user here, based on user preference information, viewing history, etc., or by the video content provider, It is predetermined.

Therefore, the icon ranking determining unit 31 may determine the ranking of the related items such that the higher the “recommended degree” the related items (video content), the higher the priority. Therefore, the icon placement determination unit 33 determines that the icon is placed closer to the end point of the trajectory as the “recommendation degree” is higher for the “video content” as the extracted related item. Here, it is considered that a related item having a higher recommendation level is more likely to be selected by the user, and therefore, a higher priority is given.

In this way, by arranging the icons that are highly recommended, that is, the icons that are more likely to be selected, to be displayed closer to the movement completion position of the previous contact operation, the user can have a higher probability. The desired icon displayed near the movement completion position of the contact operation is selected. As a result, it is possible to avoid the troublesome selection operation of moving the indicator unnaturally from place to place on the screen with high probability.

Further, the icon rank determination unit 31 compares the user preference information set in advance with the attributes of each related item (such as “genre” and “performer name”), and the similarity to the user preference. It may be determined that the higher the similarity is, the higher the possibility of selection is. Here, it is considered that a related item having an attribute close to the user's preference is more likely to be selected by the user, and thus a higher priority is given.

Alternatively, in the related information, “selected date / time” indicating the recently selected date / time may be stored as one of the attributes of the related item. The icon rank determination unit 31 may refer to the attribute “selected date and time” and give a higher priority to the latest date and time when the icon of the related item was last selected. Here, it is considered that the related item that the user has recently selected with interest is more likely to be selected by the user, and therefore, a higher priority is given. On the contrary, a higher priority may be given to the older date and time when the icon of the related item was last selected. According to this configuration, the related items that have not been selected recently can be arranged closer to the user's fingertips, so that the probability of being selected by the user can be increased.

("See" user usage)
As shown in FIGS. 4A and 4B, when the tablet terminal 100 is a small portable terminal and can be operated with one hand or both hands, when operating with one hand, operate with both hands. It is assumed that the area where the user can touch the screen with a finger is different from the time when the user is present. As shown in FIG. 4B, when operating with both hands, any area of the touch panel can be touched. On the other hand, as shown in FIG. 4A, when the operation is performed with one hand, the contact position is the area on the lower left side of the screen (when operated with the left hand) or the area on the lower right side of the screen (operated with the right hand). Tend to be biased. When the user is using the tablet terminal 100 in such a situation, there is a problem that the operation becomes complicated if an object or icon that requires a contact operation is displayed on the upper part of the screen or the lower part of the screen opposite to the hand. . This is because the user cannot touch the target object immediately and must perform an extra operation of rushing to a contactable area or switching to a two-handed operation.

Therefore, the tablet terminal 100 of the present invention solves the above problem by observing the usage status of the user. Specifically, the tablet terminal 100 of the present invention can be configured to detect the bias of the contact position of the finger and place the icon in an area where the user's finger is expected to reach immediately.

In this modification, the contact information generation unit 21 of the tablet terminal 100 does not depend on contact / non-contact switching, and further, regardless of whether the contact operation is a gesture of “enclose” (for example, In this configuration, contact information indicating a user's contact operation that occurred within a few seconds to several minutes) is generated and stored in the contact information storage unit 44.

FIG. 25 is a diagram for explaining the operation of the tablet terminal 100 of the present invention capable of presenting an operation screen in accordance with the usage status of the user. More specifically, FIG. 25A is a diagram illustrating an example of a situation where the user is operating with the left hand. (B) of FIG. 25 is a figure which shows the specific example of the contact information produced | generated with the contact operation | movement of (a) of the figure.

As shown in FIG. 25 (a), for example, it is assumed that the user performs a contact operation by dragging a target object to a contactable region by a flick operation with a thumb and surrounding it.

The contact information generation unit 21 generates contact information as shown in FIG. 25B for the above-described series of contact operations for a predetermined period (for example, the past several seconds to several minutes), and a contact information storage unit 44. When the memory capacity of the contact information storage unit 44 that stores the locus is limited, the contact information generation unit 21 may be configured to delete the oldest locus every time a new locus is stored. .

Here, it is determined by the gesture determination unit 25 that a gesture for selecting an object (for example, “enclose”) has occurred. The operation screen processing unit 24 refers back to the past several seconds to several minutes of contact information stored in the contact information storage unit 44, and detects whether or not the finger touch position is biased. In the example shown in FIGS. 25A and 25B, the finger trajectory is biased toward the area 82 on the left side of the lower part of the screen. The operation screen processing unit 24 detects this bias and identifies the user accessible area as the area 82 on the lower left side of the screen. Note that the area 82 on the lower left side of the screen and the area 83 on the lower right side of the screen are defined in advance.

The ring shape determining unit 30 of the operation screen processing unit 24 determines the shape, size, and arrangement position of the ring so that the ring for arranging the icons fits in the area 82 on the lower left side of the screen.

FIG. 26 is a diagram illustrating an example of an operation screen when icons are arranged according to the icon arrangement pattern (or ring shape) determined by the arrangement pattern determining unit (or ring shape determining unit 30). As shown in FIG. 26, since the related items of the selected object 80 are displayed so as to fit within the area 82 on the lower left side of the screen, the user does not need to drag the target icon with the thumb, The icon can be selected.

Further, the operation screen processing unit 24 arranges each extracted icon near the locus end point based on the contact operation for selecting an object, as the next icon is more likely to be selected by the user. Yes. For this reason, the user can avoid the troublesome operation of moving the finger unnaturally around the screen with a high probability. Since one-handed operation is less operable than two-handed operation, avoiding the operational inconvenience as described above is particularly effective in improving operability when operating with one hand. .

Note that the tablet terminal 100 determines whether or not the user is operating with the thumb based on the thickness of the line of the finger trajectory in order to determine the usage situation that the user is operating with one hand. If it is determined that the operation is performed, it may be determined that the operation is performed with one hand, and an icon may be displayed at the bottom of the screen. Alternatively, a sensor is provided in the casing of the tablet terminal 100, and the tablet terminal 100 determines whether the casing is gripped by four fingers or five fingers, and accordingly One-handed operation or two-handed operation may be determined.

In addition, the operation screen processing unit 24 of the tablet terminal 100 refers to the contact coordinate information of the area surrounded by the locus of the finger, identifies the locus area, and the locus area and the vicinity thereof are user accessible areas. It may be determined that an icon ring is arranged there.

(Adjusting the number of icons displayed)
If the icon arrangement determining unit 33 determines that all icons extracted by the related item extracting unit 23 cannot be displayed in consideration of the determined ring shape size and icon size, the icon arrangement determining unit 33 should display the icons. It may be decided to reduce the number of icons.

Further, the icon arrangement determining unit 33 may determine the number of icons to be displayed based on the absolute size of the finger trajectory (or the enclosed area) with reference to the contact information. Thus, the user can intentionally adjust the number of icons to be displayed next by changing whether the object is surrounded by a smaller ring or a larger ring.

Here, when the number of icons to be displayed is reduced, the icon arrangement determining unit 33 can decrease the icons in descending order of priority determined by the icon order determining unit 31.

(Association method of icon position and priority)
In each of the above-described embodiments, the icon placement determination unit 33, when associating priorities with the placement positions of some icons defined in a predetermined icon placement pattern, from the placement position that is close to the end point tn of the trajectory. In order, the higher priority order was associated.

However, the configuration of the information processing apparatus (tablet terminal 100) of the present invention is not limited to the above. The icon arrangement determining unit 33 may associate the icon arrangement position with the priority order by the method described below.

27 (a) to 27 (c) are diagrams showing specific examples of association between icon arrangement positions and priorities performed by the icon arrangement determining unit 33. FIG.

The predetermined icon arrangement pattern referred to by the icon arrangement determining unit 33 is the case where the end point of the finger trajectory when the object is selected is at the position of the end point tn shown in FIG. As shown, it is assumed that eight icons are arranged at equal intervals on the outline of the elliptical ring.

In the example shown in FIG. 27A, the icon arrangement determining unit 33 specifies a point P closest to the end point tn of the trajectory on the outline of the ring. Then, the icon arrangement determining unit 33 associates a higher priority with an arrangement position having a shorter distance from the point P to the icon arrangement positions A to H (FIG. 9A) following the outline of the ring. .

That is, as shown in FIG. 27A, the icon arrangement determining unit 33 sets the icon arrangement position A to the eighth priority, B to the sixth, C to the fourth, D to the second, E Associate 1st position with F, 3rd position with F, 5th position with G and 7th position with H.

According to the above configuration, as shown in FIGS. 28A and 28B, the tablet terminal 100 can rotate the icon ring clockwise and counterclockwise by a user's drag operation. There is a merit when displaying the operation screen.

Specifically, after the user selects an object at the end point tn, the icon ring displayed with the user is rotated by a drag operation, and the target icon is located at the position where the fingertip is located (that is, near the end point tn). Suppose that you want to crawl on. In this case, the icon at a position where the distance to follow the outline from the end point tn becomes shorter, and the target icon can be dragged with fewer drag operations (rotation operations).

In other words, if an icon is more likely to be selected by the user, if the icon is arranged at a position where the distance to follow the outline becomes shorter, the user will move the target icon with a lower number of drag operations with a higher probability. be able to.

Therefore, the user can reach the target final result with a small number of operations. That is, it is possible to avoid the troublesome selection operation.

Alternatively, when the icons are arranged in an elliptical shape, it is assumed that the user naturally moves his / her finger so as to trace the elliptical outline when searching for and selecting the target icon. Therefore, the icon arrangement determining unit 33 determines to arrange an icon having a high priority at an icon arrangement position where the distance to follow the outline from the end point tn is short.

Thereby, an icon having a high possibility of being selected by the user can be arranged at an icon arrangement position having a short distance on the contour line from the end point tn. For this reason, the user can reach the target icon at a shorter distance with high probability by running the finger from the position where the object has been selected.

Therefore, it is not necessary for the user to unnaturally move the operating body from one place to another on the screen, and the user can reach the desired final result with a simple contact operation. That is, it is possible to avoid the troublesome selection operation. In addition, the shorter the movement distance of the operating body, the more the erroneous operation can be prevented.

In the example shown in (b) of FIG. 27, the icon arrangement determining unit 33 associates the highest priority with the arrangement position closest to the end point tn of the locus among the predetermined icon arrangement positions A to H. As shown in the figure, the first place is associated with the icon arrangement position E. Then, the second and subsequent priorities are sequentially associated with each arrangement position in the counterclockwise direction from the first E. That is, as shown in FIG. 27 (b), D is 2nd, C is 3rd, B is 4th, A is 5th, H is 6th, G is 7th, and F is Associate 8th place.

According to the above configuration, as shown in FIG. 28A, the tablet terminal 100 has an advantage when displaying an operation screen that can rotate the ring of icons clockwise by a user's drag operation. There is.

Specifically, after the user selects an object at the end point tn, the icon ring displayed along with it is rotated clockwise by a drag operation, and the target icon is moved to the position where the fingertip is located (that is, the end point). Assume a case of rushing to (near tn). In this case, each icon shown in FIG. 27B is dragged from the first-ranked icon to the user's fingertip (near the end point tn) in the counterclockwise order as the ring rotates clockwise.

That is, the order in which they are gathered matches the priority shown in FIG. 27B (the order in which the user is likely to be selected). For this reason, the user can search for the target icon with a high probability with a smaller number of drag operations.

In the example shown in (c) of FIG. 27, the icon arrangement determining unit 33 associates the first place with the icon arrangement position E as in the example shown in (b) of FIG. Then, in order clockwise from the first E, the second and subsequent priorities are sequentially associated with each placement position. That is, as shown in FIG. 27 (c), F is 2nd, G is 3rd, H is 4th, A is 5th, B is 6th, C is 7th, and D is Associate 8th place.

According to said structure, when the tablet terminal 100 displays the operation screen which can rotate the ring of an icon counterclockwise by a user's drag operation as shown in FIG.28 (b). There are benefits.

More specifically, after the user selects an object at the end point tn, the user rotates the icon ring displayed in a counterclockwise direction by dragging, and moves the target icon to the position where his / her fingertip is located (that is, Assume a case of rushing to the vicinity of the end point tn. In this case, each icon shown in FIG. 27C is dragged from the first-ranked icon to the user's fingertip (near the end point tn) in the clockwise order as the ring rotates counterclockwise.

That is, the order in which they are gathered matches the priority shown in FIG. 27C (the order in which the user is likely to be selected). For this reason, the user can search for the target icon with a high probability with a smaller number of drag operations.

(Application to tablet PC)
In each of the above-described embodiments, the tablet terminal 100 as the information processing apparatus of the present invention has been described on the assumption that it is a small and portable smartphone that can be operated with one hand. The information processing apparatus of the present invention is not limited to a small smartphone, but can be applied to a tablet PC having a notebook size screen, an electronic blackboard having a larger screen than that, and the like.

Tablet PCs, electronic blackboards, etc. have a larger display screen than smartphones. Therefore, in order for the user to select a desired object or icon, an indicator (finger or pen) or the like must be moved more than when operating the smartphone.

According to the information processing apparatus of the present invention, the movement of the indicator can be further reduced when selecting a desired object or icon. That is, if the information processing apparatus of the present invention is used in a tablet PC having a large display screen or an electronic blackboard, the advantage of the present invention that realizes excellent operability can be enjoyed more.

FIG. 29 and FIG. 31 are views showing a state where the tablet terminal 100a as the information processing apparatus of the present invention is realized by a tablet PC.

30 and 32, (a) is a diagram showing a specific example of the icon arrangement pattern in which the locus of the indicator and the end point tn of the locus are plotted by the icon arrangement determining unit 33, and (b) is an icon. It is a figure which shows the result by which the priority order was linked | related with each icon arrangement position by the arrangement | positioning determination part.

As shown in FIGS. 29 and 31, for example, it is assumed that two photo objects are displayed on the entire screen of the display unit 12 of the tablet terminal 100a, and the user selects one of them by moving the indicator. Specifically, it is assumed that the user performs a contact operation by moving a finger (indicator) so as to surround the target photographic object.

First, in the example shown in FIG. 29, the user draws a circle so as to surround the photo object from around the lower right of the target photo object, and similarly finishes drawing the circle around the lower right.

And the icon arrangement | positioning determination part 33 acquires the locus | trajectory of the said finger | toe acquired by the contact information generation part 21, and its end point tn, and as shown to (a) of FIG. 30, it is made into a predetermined icon arrangement pattern. Plot.

Subsequently, the icon arrangement determination unit 33 follows each predetermined rule defined in the pattern based on the end point tn according to a predetermined rule (for example, a rule that associates a higher priority in the order of the shortest distance from the end point tn). The priority positions 1 to 8 are associated with the icon arrangement positions A to H. For example, when the icon arrangement determining unit 33 associates the priority order with the icon arrangement position according to the above rules, the result is as shown in FIG. The number assigned to each icon arrangement position represents the associated priority order.

The operation screen processing unit 24 arranges each icon at the arrangement position according to the determination by the icon arrangement determining unit 33 so that the icon order determining unit 31 matches the priority order previously assigned to the icons.

Thus, an icon with a higher priority (highly likely to be selected by the user) is displayed near the end point tn where the fingertip is considered to be located. Therefore, even if the user has a large screen of the tablet terminal 100a, the user can move to an operation of selecting a target icon without moving the finger greatly with a high probability.

On the other hand, the method of the contact operation varies depending on the user. For example, as shown in FIG. 31, the user may draw a circle so as to surround the photo object from the upper left of the target photo object, and finish drawing the circle around the upper left as well.

In this case, the icon arrangement determining unit 33 plots the finger trajectory and its end point tn in the icon arrangement pattern as shown in FIG.

Subsequently, according to the above rules, the icon arrangement determining unit 33 associates the higher priority with the icon arrangement position having a shorter straight line distance with the end point tn. Here, for example, when the icon arrangement determining unit 33 associates the priority order with the icon arrangement position according to the above rule, the result is as shown in FIG.

Thus, an icon with a higher priority (highly likely to be selected by the user) is displayed near the end point tn where the fingertip is considered to be located. Therefore, even if the user has a large screen included in the tablet terminal 100a, the user can move to an operation of selecting a target icon without moving the finger greatly with a high probability.

As described above, in the tablet terminal (information processing apparatus) 100a having a large screen, the moving distance of the indicator when selecting an object becomes long, so that the position of the end point (fingertip) reaches everywhere on the screen. To do. Then, it is possible that the icon to be selected next is displayed at a position away from a certain position. In this case, the user unnaturally moves the indicator from one place to another on the screen, and there is a problem that operability is degraded. Note that the larger the screen of the information processing apparatus, the more troublesome is the operation of moving the indicator further away, and the above problem is more serious.

However, according to the tablet terminal 100a of the present invention, as described above, even if the position of the end point (fingertip) reaches any position throughout the screen, the position closer to the end position is selected by the user. Since the icon that is highly likely to be displayed is displayed, the user can select the target icon with a high probability without moving the finger too much. For example, as shown in the above-described example, even when the same photographic object is enclosed, the way of enclosure varies depending on the user, and different results are obtained accordingly. In the contact operation shown in FIG. 29, the highest priority is associated with the icon arrangement position E as shown in FIG. 30, and in the contact operation shown in FIG. 31, the icon arrangement position G is the highest as shown in FIG. A higher priority is associated. In this way, as the information processing apparatus has a large screen, the final product differs greatly depending on how the user is enclosed, and the above serious problems can be solved.

As described above, according to the tablet terminal 100a of the present invention, it is not necessary for the user to unnaturally move the indicator from one place to another on the screen, and the target final result can be reached with a simple contact operation. it can. In addition, it is possible to suppress the induction of erroneous operations as the moving distance of the indicator is shorter. As a result, there is an effect of realizing excellent operability. As described above, if the information processing apparatus of the present invention is used in a tablet PC having a large display screen, an electronic blackboard, or the like, the effect obtained is particularly great.

(Modification of icon arrangement pattern)
In each of the above-described embodiments, the icon arrangement determining unit 33 is configured to recognize the icon arrangement position based on a predetermined icon arrangement pattern (for example, (a) in FIGS. 9, 30, and 32). . However, it is not limited to such a configuration. The icon arrangement determination unit 33 may determine an icon arrangement pattern by itself based on a finger trajectory acquired by the contact information generation unit 21 when the user performs a contact operation.

FIG. 33 is a diagram illustrating an example of an icon arrangement pattern determined by the icon arrangement determining unit 33.

In this modification, there is no predetermined icon arrangement pattern, and only the ring shape and the arrangement position are determined by the ring shape determination unit 30.

33, first, the icon arrangement determining unit 33 acquires information on the ring shape and the arrangement position determined by the ring shape determining unit 30, and plots the end point tn of the trajectory with respect thereto. Next, the icon arrangement determination unit 33 specifies a point P on the contour line of the ring that is closest to the end point of the locus, and determines that point as the icon arrangement position with the highest priority (first place). A more detailed icon arrangement position of the first icon is not particularly limited. For example, a position where the center of the icon coincides with the point P may be determined as the first icon arrangement position.

Subsequently, the icon arrangement determining unit 33 determines the remaining icon arrangement positions based on the previously determined first icon arrangement position. For example, when there are seven icons to be arranged, as shown in FIG. 33, the remaining icon arrangement positions are all equally spaced on the contour line with reference to the first icon arrangement position. A to G are determined. The association between the remaining icon arrangement positions A to G and the priority order may be performed according to any of the rules described above.

According to the above configuration, the operation screen processing unit 24 can always arrange the icon with the highest priority in the shortest distance from the end point tn on the outline of the ring.

Alternatively, the icon arrangement determination unit 33 may determine the arrangement position of the icon with the highest priority (first place) as follows.

Specifically, as shown in FIG. 30A, the icon arrangement determining unit 33 first acquires a point tn−1 in addition to the end point tn from the trajectory. Then, the icon arrangement determining unit 33 specifies a position where the extended line of the straight line extending from the point tn−1 to the end point tn first intersects with the contour line of the ring (here, the dashed ellipse), that is, the intersection point Q. And the icon arrangement | positioning determination part 33 may determine the intersection Q as the icon arrangement position of the highest priority (1st place).

This configuration is effective when the end point tn is inside the ring. In this case, when the user moves the operating tool (finger or pen) in a circle so as to surround the object, the intersection Q is the point that the operating tool reaches the earliest naturally on the outline of the ring. Conceivable.

By arranging the icon with the first priority at the position of the intersection Q in this way, it is possible to display the icon with the first priority at the position where the operating body reaches the earliest naturally. . As a result, the user can easily select the first icon by the operation tool in the next operation.

(Applied to information processing equipment with separate display and input)
Regarding the configuration of the tablet terminal 100 as the information processing apparatus of the present invention, that is, the configuration in which an icon having a high priority is arranged and displayed near the end point of the movement locus of the indicator, this is represented by the input unit 11 and the display unit 12. And may be applied to an information processing apparatus provided separately. For example, an information processing apparatus in which the input unit 11 is configured by an input device such as a mouse and the display unit 12 is configured by a display device such as an LCD can be considered. Here, the cursor displayed on the display unit 12 indicates the position of the display unit 12 on the screen. Then, when the user operates the mouse to perform an input operation, the cursor moves. Therefore, in such an embodiment, the mouse is the operating body, the cursor is the pointing body, and the pointing body displayed on the display unit 12 moves as the operating body moves. When the user selects an object by moving the cursor on the screen of the display unit 12 by operating the mouse, the information processing apparatus interlocks the position of the cursor held in advance with the movement of the mouse, The trajectory of the movement is held. When the icon is arranged at a predetermined position, an icon having a high priority (high possibility of being selected by the user) is arranged at an arrangement position close to the end point of the locus. According to the above-described configuration, an operation screen in which icons that are often selected are arranged near the position where the user has finished moving the mouse is obtained. It becomes possible to move to operation.

As the input unit 11 configured separately from the display unit 12, various input devices such as a keyboard, a joystick, a digitizer, a tablet, and a stylus pen can be employed in addition to the mouse.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

Note that the information processing apparatus and operation screen display method of the present invention can also be expressed as follows.

In an information processing apparatus including a touch panel, at least a selection region specified based on a trajectory acquired by the contact motion acquisition unit that acquires a trajectory of movement of the indicator that has moved on the touch panel Referring to the object specifying means for specifying the partially overlapping object as the selected object and the related information storage unit for storing the object and the item related to the object in association with each other, the object specifying means specifies the object. Related item extraction means for extracting an item associated with the selected object as a related item, and operation screen processing for displaying the icon of the related item extracted by the related item extraction means at a specific position and displaying it on the touch panel And the operation screen processing means is extracted as described above. Among related items, an icon of a related item having a higher priority (such as a descending order of possibility of being selected by the user) is arranged near the end point of the trajectory acquired by the contact operation acquisition unit. Information processing apparatus.

In an operation screen display method in an information processing apparatus including a touch panel, a touch motion acquisition step for acquiring a trajectory of movement of an indicator that has moved on the touch panel, and a specification based on the trajectory acquired in the touch motion acquisition step An object specifying step for specifying an object at least partially overlapping the selected area as the selected object, and a related information storage unit for storing the object and an item related to the object in association with each other. A related item extracting step for extracting an item associated with the object specified in the object specifying step as a related item, and an icon specifying position of the related item extracted by the related item extracting step. Operation screen processing steps to be displayed on In the operation screen processing step, among the extracted related items, the icons of related items having higher priority (such as the order in which the user is more likely to be selected) are acquired in the contact operation acquisition step. An operation screen display method characterized by being arranged near the end point of the trajectory.

[Example of software implementation]
Finally, each block of the tablet terminal 100, in particular, the contact information generating unit 21, the object specifying unit 22, the related item extracting unit 23, the operation screen processing unit 24, the gesture determining unit 25, the ring shape determining unit 30, the icon rank determining unit 31, the animation determination unit 32, and the icon arrangement determination unit 33 may be configured by hardware logic, or may be realized by software using a CPU as follows.

That is, the tablet terminal 100 includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, a RAM (random access memory) that develops the program, A storage device (recording medium) such as a memory for storing the program and various data is provided. An object of the present invention is to provide a recording medium on which a program code (execution format program, intermediate code program, source program) of a control program of the tablet terminal 100, which is software that realizes the functions described above, is recorded so as to be readable by a computer. This can also be achieved by supplying the tablet terminal 100 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

Further, the tablet terminal 100 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

The present invention can be widely applied to information processing apparatuses including an input unit and a display unit. For example, but not limited to this, a digital TV, personal computer, smartphone, tablet PC, notebook computer, mobile phone, PDA (Personal Digital Assistant), electronic book reader, electronic dictionary, portable, provided with an input unit and a display unit -It can be used suitably for a home game machine, an electronic blackboard, etc. Furthermore, if the present invention is applied to an information processing apparatus including a touch panel, it is possible to realize even better operability.

10 Control unit 11 Input unit (touch panel)
12 Display unit (touch panel)
13 operation unit 14 external interface 15 communication unit 16 wireless communication unit 17 audio output unit 18 audio input unit 19 storage unit 21 contact information generation unit (trajectory acquisition unit / contact operation acquisition unit)
22 Object identification part (object identification means)
23 Related Item Extraction Unit (Related Item Extraction Unit)
24 Operation screen processing unit (operation screen processing means)
25 Gesture determination unit (gesture determination means)
30 Ring shape determination unit (ring shape determination means / arrangement pattern determination means)
31 Icon ranking determining unit (icon ranking determining means)
32 Animation determination unit (animation determination means)
33 Icon arrangement determining unit (icon arrangement determining means)
41 Frame map storage unit 42 Related information storage unit 43 Icon storage unit 44 Contact information storage unit 100 Tablet terminal (information processing apparatus)

Claims

A trajectory acquisition means for acquiring a trajectory by which the indicator pointing to the position of the screen of the display unit has moved;
Object specifying means for specifying an object that at least partially overlaps the selection area specified based on the trajectory acquired by the trajectory acquiring means, as a selected object;
A related item extracting unit that extracts an item associated with the object identified by the object identifying unit as a related item with reference to a related information storage unit that associates and stores the object and an item associated with the object; ,
An operation screen processing means for arranging the icon of the related item extracted by the related item extracting means at a specific position and displaying it on the display unit;
The operation screen processing means includes
An information processing apparatus characterized in that, among the extracted related items, an icon of a related item having a higher priority is arranged closer to the end point of the trajectory acquired by the trajectory acquisition unit.
In the related information storage unit, the number of times the icon of the related item is selected by the user is stored for each related item,
The operation screen processing means includes
The information processing apparatus according to claim 1, wherein icons of related items that are selected more frequently are arranged closer to an end point of the trajectory.
In the related information storage unit, attributes of related items are stored for each related item,
The operation screen processing means includes
The information processing apparatus according to claim 1, wherein icons of related items having attributes having a high similarity to the attribute of the selected object are arranged closer to the end point of the trajectory.
The selected object and the related item are photographs, and the related information storage unit stores the shooting date and time of the photograph as an attribute,
The operation screen processing means includes
4. The information processing apparatus according to claim 3, wherein an icon of a photograph with a shooting date and time closer to the shooting date and time of the selected object is arranged closer to the end point of the trajectory.
The related item is video content, and the related information storage unit stores a recommendation level indicating a level of recommending the video content to the user as an attribute,
The operation screen processing means includes
The information processing apparatus according to claim 1, wherein icons of moving image content having a higher recommendation level are arranged closer to an end point of the trajectory.
The trajectory acquisition means acquires a trajectory of movement of the indicator that has moved the screen of the display unit so as to surround the object displayed on the display unit,
The object specifying means specifies an object including at least a part of the area surrounded by the locus as a selected object,
6. The information processing apparatus according to claim 1, wherein the operation screen processing unit arranges the icons of the extracted related items side by side on an outline of a ring.
The operation screen processing means includes
The information processing apparatus according to claim 6, wherein the position and size of the ring are determined so that an icon is arranged around the selected object.
The operation screen processing means includes
The information processing apparatus according to claim 6 or 7, wherein a trajectory acquired by the trajectory acquisition unit, or a similar shape or an approximate shape thereof, is determined as the shape of the ring.
The operation screen processing means includes
Place the icon of the related item with the highest priority at the point closest to the end point of the trajectory on the outline of the ring,
9. The icons of the remaining related items are arranged on the outline of the ring with reference to the arrangement position of the icon having the highest priority. Information processing device.
The trajectory acquisition means includes
Obtaining a locus of the indicator that has occurred in a predetermined period until the indicator moves to select an object displayed on the display unit;
The operation screen processing means includes
When it is determined that the trajectory acquired during the predetermined period is biased to a specific area on the screen of the display unit, the position of the icon is determined so that the icon is arranged in the specific area. The information processing apparatus according to any one of claims 1 to 9.
The operation screen processing means includes
10. The position of the icon is determined so that the icon is arranged in a specific area overlapping with a movement locus of the indicator that selects the object on the screen of the display unit. The information processing apparatus according to any one of claims.
The input unit and the display unit included in the information processing apparatus constitute a touch panel.
The information processing apparatus according to any one of claims 1 to 11, wherein the trajectory acquisition unit acquires a trajectory of movement of the indicator that has moved on the touch panel.
The input unit included in the information processing apparatus inputs an instruction to move the cursor displayed on the display unit to the information processing apparatus.
The information processing apparatus according to claim 1, wherein the trajectory acquisition unit acquires a trajectory of movement of the cursor as the indicator.
An operation screen display method in an information processing apparatus,
A trajectory acquisition step for acquiring a trajectory of movement of an indicator that indicates the position of the screen of the display unit included in the information processing apparatus;
An object specifying step for specifying an object that at least partially overlaps the selected area specified based on the track acquired in the track acquiring step as a selected object;
A related item extracting step of extracting an item associated with the object identified in the object identifying step as a related item with reference to a related information storage unit that associates and stores the object and an item associated with the object When,
An operation screen processing step of placing the icon of the related item extracted in the related item extraction step at a specific position and displaying the icon on the display unit,
In the above operation screen processing step,
An operation screen display method characterized in that, among the extracted related items, an icon of a related item having a higher priority is arranged near the end point of the track acquired in the track acquisition step.
A control program for causing a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 13.
A computer-readable recording medium on which the control program according to claim 15 is recorded.