JP5173001B2 - Information processing apparatus, screen display method, control program, and recording medium - Google Patents

Abstract

In order to achieve excellent operability in an information processing device provided with an input unit and a display unit, this tablet terminal (100) is characterized by being provided with: a contact information generation unit (21) that acquires a path along which an indicator indicating the position of a screen on a display unit (12) has moved; a margin selection assessment unit (28) that assesses whether a selection area, which has been specified on the basis of the path acquired by the contact information generation unit (21), has been selected as a margin area in which information is not displayed on the display unit (12); a related item extraction unit (23) which, when the selection area has been assessed as having been selected as the margin area by the margin selection assessment unit (28), references a related information storage unit, which stores items related to an event in which a margin area has been selected, and extracts items that are associated with the event as related items; and an operating screen processing unit (24) that displays icons corresponding to the related items extracted by the related item extraction unit (23) on the display unit (12).

Description

  The present invention relates to a user interface technology of an information processing apparatus including an input unit and a display unit.

  In recent years, so-called tablet terminals such as smartphones and tablet PCs are rapidly spreading. The tablet terminal has a flat outer shape and includes a touch panel as a display unit and an input unit. By touching the object displayed on the touch panel with a finger, a pen, or the like, the user can perform various operations on the tablet terminal body.

  The tablet terminal can discriminate various contact operations on the user's screen by the touch panel, and can perform object display in accordance with the contact operations. For example, for the touch action, various actions such as tapping (lightly tapping), flicking (flicking, picking), pinching (pinch with a finger), dragging an object displayed on the screen with a finger (pen) There is. The tablet terminal discriminates such various contact actions, and selects / moves objects, scrolls a list, enlarges / reduces an image, etc. according to the discrimination result. The tablet terminal realizes a more intuitive operation by the touch panel as described above, and is supported by many people.

  In addition, regardless of the tablet terminal, regardless of the size of the device, the user interface technology that displays a display object such as a menu, an image, and an icon and accepts selection in a multi-functional information processing device in general is very good. It is used. According to this technology, the information processing apparatus displays a display object so that the user can select a desired item, and accepts the user's selection. Thus, the user can operate the apparatus by selecting a desired item.

  As a method of displaying a display object (for example, a menu), for example, a call button to which a function for calling a menu is assigned is provided physically or on a touch screen, and the menu is pressed by pressing the button. It is common to call. Specifically, when a menu button provided on the remote control is pressed, a menu is displayed on the TV, or when a function key provided on the mobile phone is pressed, a function list is displayed on the display screen of the mobile phone. Things are being done.

  Further, Patent Documents 1 and 2 disclose information processing apparatuses that realize display of display objects such as menus, images, and icons for the purpose of improving user convenience and operability.

  Patent Document 1 discloses a display device including a touch input unit having an input area that overlaps a display screen. This display device accepts user instruction operation input by touch. Then, a circular menu including a plurality of menu items arranged in a substantially ring shape around the accepted touch position is displayed on the display screen.

  Patent Document 2 discloses a menu display control device and an electronic blackboard system to which the menu display control device is applied. Specifically, the menu display control device recognizes an input history related to a plurality of coordinates input to the device by a user drawing a line or the like on the coordinate input surface with a fingertip. The menu display control device determines whether or not to display the processing menu based on the input history, and displays the processing menu. More specifically, after the user draws a circle on the coordinate input surface with the index finger, the menu display control device touches the coordinate input surface with the middle finger while touching the index finger on the screen, and the middle finger is touched. Display the processing menu.

JP 2006-139615 A (published June 1, 2006) JP 2001-265475 A (published September 28, 2001)

  The good operability of the tablet terminal is how to display the final result, which is the user's purpose, with a simple contact operation and a small number of operations, and a natural flow that does not contradict the user's intuition, It depends on whether to display the result based on the contact operation.

  Such an improvement in operability is realized by appropriately grasping the user's purpose, the user's state, and the user's tendency. For example, the tablet terminal displays what the user wants to do now, what the user wants to do next, how the user is now operating, where the user is, and how the user moves. It is required to “see” the user's intentions from all points of view, such as whether or not it is natural.

  The configurations of the devices described in Patent Documents 1 and 2 are not necessarily sufficient to sense the user's intention.

  More specifically, Patent Document 1 discloses that the user can arbitrarily determine the position for displaying the menu according to the touch position. However, the user may have touched the screen to select another object, and accurately determine whether or not the user touched the screen with the intention of calling a menu unrelated to the specific object. I can't.

  Also, in Patent Document 2, a special gesture is assigned as an operation for the user to intentionally call the menu. However, the special gesture is complicated and the user has a hard time to call the menu. You must learn how to do that.

  As a result, a series of processes such as user operation and display of the result (menu, image, icon, etc.), user operation for selecting the result, ... display of the final result are simplified. There is a problem that the device cannot be operated with a contact operation, a small number of operations, and an intuitive and natural contact operation.

  The above-mentioned operability problem is caused by not only a small tablet terminal having excellent portability but also an information processing apparatus of any size (for example, an electronic blackboard having a large touch panel) provided with a touch panel type display / input unit. Etc.), and not only the touch panel, but also a problem that occurs in common in information processing apparatuses including display units and input units of all forms.

  The present invention has been made in view of the above problems, and an object thereof is to realize excellent operability in an information processing apparatus including an input unit and a display unit.

  In order to solve the above problems, an information processing apparatus according to the present invention is based on a trajectory acquisition unit that acquires a trajectory of movement of an indicator that indicates a position of a screen of a display unit, and a trajectory acquired by the trajectory acquisition unit. A margin selection determination means for determining whether or not the selected selection area has selected a blank area for which information on the display unit is not displayed, and the selection area is determined to be a blank area by the margin selection determination means. When it is determined that the item is selected, the related item extraction unit extracts the item associated with the event as a related item with reference to the related information storage unit that stores the item associated with the event for which the blank area is selected. And a screen processing means for displaying an icon corresponding to the related item extracted by the related item extracting means on the display unit.

  According to the above configuration, the information processing apparatus of the present invention receives a selection operation of selecting a blank area having no information from the user. In the information processing apparatus, a function of calling (displaying) a target display object (here, one or a plurality of icons) is assigned to the user's selection operation of “selecting a blank area”. When a blank area is selected by the user, the information processing apparatus extracts related items associated with the event in advance and displays an image corresponding to each related item.

  More specifically, when the trajectory acquisition unit receives the selection operation, a trajectory of movement of the indicator (for example, a user's finger, pen, or cursor) can be obtained. The margin selection determination means determines whether or not the area specified by the locus (that is, the selection area selected by the user) is a margin area. If the selected area is included in a blank area where no information is displayed on the display unit, the margin selection determining means determines that the blank area has been selected by the selection area. In other words, an event that “a blank area is selected by the user” occurs in the information processing apparatus, and it is recognized in the information processing apparatus that this event has occurred.

  As described above, when it is recognized by the margin selection determination that the event that “the blank area is selected by the user” has occurred, the related item extracting unit then determines that the event “the blank area has been selected”. The related items associated are extracted from the related information storage unit. The related information storage unit stores one or more items as related items in association with the event “margin area has been selected”.

  Subsequently, the screen processing means displays an icon corresponding to the extracted related item on the display unit. When a plurality of related items are extracted, an icon corresponding to each related item may be displayed on the display unit.

  As described above, the information processing apparatus according to the present invention allows the user to perform a selection operation of “selecting” a “margin area” in which no information is displayed, and a target display target for the selection operation. A function for calling an object (here, an icon) is assigned. In other words, the normal user's selection operation of selecting some information or area and the “margin area” having no information are combined, and the image when the margin area is selected is selected in this selection operation. The function of calling is assigned. Here, an image to be stored in association with the event “margin area has been selected” is a multi-faceted, general-purpose, general, or attribute-free image that does not depend on specific display contents (objects). can do.

  As described above, from the active operation of “selecting a blank area where no information is displayed”, it can be easily expressed as “multi-faceted, general-purpose, general, independent of specific contents, or It is associated with the desire for an “attributeless display object (icon)”.

  In other words, it is possible to cause the user to intentionally and actively input information such as “no information is selected” or “desired display target not depending on specific contents” to the information processing apparatus. . Thereby, the user's intention of desiring a multi-faceted, general-purpose, general, or attribute-free icon can be more accurately sensed.

  In addition, the operation for selecting the margin area may be the same as a general selection operation usually performed when selecting some information (an object such as a file), and a special operation for selecting the margin area is performed. Gestures are not included. Therefore, the user is not forced to learn the operation. The user can call an image that does not depend on specific contents simply by selecting an area in which no information is displayed (vacant) in the same manner as when selecting some information.

  As a result, a series of processes including user operation and display of the final result (icon) associated therewith can be realized in the information processing apparatus with simple operations, a small number of operations, and intuitive and natural operations. It becomes possible. As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the display unit and the input unit.

  In the information processing apparatus of the present invention, the margin selection determination unit specifies the size of the selection region for selecting a margin region based on the trajectory, and the related item extraction unit determines that the margin region is selected. The number of related items to be extracted may be determined according to the size of the selection area.

  According to the above configuration, the related item extracting unit adjusts the number of related items to be extracted according to the size of the selection operation performed by the user, that is, how large the selection area selected by the user is. As a result, the final number of icons displayed on the display unit by the screen processing means is also variable according to the size of the selection area.

  For example, if the selection area selected by the user is small, the number of icons displayed is reduced accordingly, and if the selection area is large, the number of icons displayed is increased accordingly. Can be considered.

  In this way, since the selection operation actually performed by the user and the final result displayed on the display unit are linked, the user can operate the information processing apparatus of the present invention by a more intuitive and natural operation. Can be operated.

  Alternatively, in the information processing apparatus of the present invention, the margin selection determination unit specifies the size of the selection region for selecting a margin region based on the trajectory, and the screen processing unit includes icons corresponding to the related items. Or the size of the image display area for displaying the icon may be determined according to the size of the selection area.

  According to the above configuration, the screen processing means is configured to display the size of the icon to be displayed or one or more depending on the size of the selection operation performed by the user, that is, how large the selection area selected by the user is. The size of the image display area for displaying the icon is adjusted. Thereby, the range in which one or more icons displayed on the display unit are displayed is variable according to the size of the selection area.

  For example, if the selection area selected by the user is small, the range in which the icon is displayed is reduced accordingly, and if the selection area is large, the range in which the icon is displayed is increased accordingly. I think that.

  As described above, since the selection operation actually performed by the user and the final result displayed on the display unit are linked, the user can call the target image as intended with a smaller number of operations. The information processing apparatus of the present invention can be operated by a more intuitive and natural operation.

  Furthermore, in the information processing apparatus of the present invention, it is preferable that the screen processing means displays the icon in the selection area determined to have selected a blank area.

  According to the above configuration, the screen processing means can display an icon at a position where the selection operation performed by the user has occurred.

  As described above, since the selection operation actually performed by the user and the final result displayed on the display unit are linked, the user can call the target icon as intended with a smaller number of operations. The information processing apparatus of the present invention can be operated by a more intuitive and natural operation.

  Furthermore, in the information processing apparatus of the present invention, the margin selection determination unit specifies the size of the selection region for selecting a margin region based on the locus, and the screen processing unit displays one or more icons. The screen processing means may determine the size of the ring in accordance with the size of the selection area.

  According to the above configuration, the screen processing means defines a ring for arranging icons, arranges the icons on the outline of the ring, and displays them on the display unit.

  Here, the screen processing means adjusts the size of the ring according to the size of the selection operation performed by the user, that is, how large the selection area selected by the user is. Thereby, the display range of the ring of icons becomes variable according to the size of the selection area.

  For example, if the selection area selected by the user is small, the ring is reduced to reduce the area where the icon is displayed, and if the selection area is large, the ring is enlarged accordingly. It is possible to increase the range in which icons are displayed.

  As described above, since the selection operation actually performed by the user and the final result displayed on the display unit are linked, the user can call the target image as intended with a smaller number of operations. The information processing apparatus of the present invention can be operated by a more intuitive and natural operation.

  Alternatively, the information processing apparatus according to the present invention further includes an image processing unit that processes an image and displays the image on the display unit, and the margin selection determination unit determines the size of the selection region for selecting a margin region as the locus. The image processing means is activated when an image processing icon displayed by the screen processing means is selected, and selects the size of the image display area for displaying the image. The size may be determined according to the size of the area, and the image may be scaled or trimmed.

  According to the above configuration, the screen processing unit displays the image on the display unit after executing the image processing for changing the size of the image.

  Here, the screen processing means adjusts the size of the image according to the size of the selection operation performed by the user, that is, how large the selection area selected by the user is. More specifically, the image is enlarged or reduced, or the image is trimmed when it is the same magnification.

  As a result, the size of the displayed image becomes variable according to the size of the selection area.

  For example, if the selection area selected by the user is small, the displayed image is reduced by reducing or trimming the image accordingly, and if the selection area is large, the image is adjusted accordingly. It is conceivable to enlarge an image to be displayed by enlarging the image.

  As described above, since the selection operation actually performed by the user and the final result displayed on the display unit are linked, the user can display a target image as intended with a smaller number of operations. The information processing apparatus of the present invention can be operated by a more intuitive and natural operation.

  Alternatively, the information processing apparatus of the present invention includes a state determination unit that determines a screen display state of the information processing apparatus according to a screen displayed on the display unit by the information processing apparatus, and the related information storage unit The related item related to the event for which the margin area is selected is further stored in association with each screen display state, and the related item extracting means is the margin determined by the state determining means. You may extract the related item linked | related with the screen display state of the said information processing apparatus when it determines with the selection area | region selecting the blank area | region by the selection determination means.

  According to the above configuration, the information processing apparatus of the present invention is optimal for the scene in consideration of what screen the information processing apparatus is displaying (that is, the screen display state of the information processing apparatus). Icons for related items can be presented to the user.

  Therefore, by letting the user select a margin area, the user intentionally activates that he wants a “multi-faceted, general-purpose, general, or attribute-free image that does not depend on specific contents”. Accordingly, the information processing apparatus can input the information, and in response to this, the information processing apparatus can present an icon (or icon group) independent of specific contents to the user. In addition, the information processing apparatus can present the optimal icon corresponding to each scene to the user in consideration of the screen display state of the information processing apparatus when the blank area is selected. It is possible to provide the user with a final result that more accurately reflects the intention.

  Alternatively, in the information processing apparatus according to the present invention, the margin selection determination unit selects a margin region based on the trajectory with reference to division information defining a plurality of division regions obtained by dividing the screen of the display unit. The divided area including the selected area is specified, and the related information storage unit further stores a related item related to the event for which the blank area is selected, and is associated with each divided area. The item extracting unit may extract a related item associated with a divided region including the selected region specified by the margin selection determining unit.

  According to the above configuration, the information processing apparatus according to the present invention presents the icon of the related item that is optimal in the operation situation in consideration of which position (divided area) of the display unit the blank area has been selected. it can.

  It is assumed that there are an infinite number of display objects (icons) “multi-faceted, general-purpose, general, or attribute-free that do not depend on specific contents”.

  Innumerable icons include icons that users call frequently, favorite icons, icons often called at work, icons often called when playing, icons often called when using both hands at home, when going out and moving There will be various groups for each user, such as icons that are often called when used with one hand.

  On the other hand, the screen position (divided region) where the selection operation is frequently performed should be biased for each user and for each use scene. For example, some users use one hand, some users use both hands, some right-handed users, and some left-handed users. For example, when the information processing apparatus includes a touch panel as an example, a person who frequently operates with the right hand (one hand) should concentrate on the lower right of the touch panel screen.

  Here, for example, the structure of the data stored in the related information storage unit is divided into the upper most of the related items that the user uses most frequently in the divided area “bottom right of the screen” where the contact operation occurs most frequently. It can be set as the structure which links | relates.

  Therefore, according to the information processing apparatus of the present invention, when the user selects a blank area on the “bottom right of the screen” that is most often touched, the icon of the related item that is most frequently used by the user is displayed. Can be presented.

  From the above, by allowing the user to select a blank area, the user wants “a multi-faceted, general-purpose, general, or attribute-free display object (icon) that does not depend on specific contents”. Can be intentionally actively input to the information processing apparatus. Then, the information processing apparatus can present an icon that does not depend on specific contents to the user in response to the input. In addition, the usage situation is estimated in consideration of the position of the locus when the blank area is selected (which divided area is included in the selected area), and an optimal icon (or icon) is linked to the estimated situation. Group) can be presented to the user, so that it is possible to provide the user with a final result that more accurately reflects the user's intention.

  Alternatively, in the information processing apparatus of the present invention, the margin selection determining unit specifies the size of the selection region for selecting a margin region based on the trajectory, and the selection region is divided into a plurality of sections based on the size. In the related information storage unit, the related information related to the event for which the margin area is selected is further stored in association with each category of the size of the selected area. The extracting means may extract related items associated with the size classification of the selected area classified by the margin selection determining means.

  It is assumed that there are an infinite number of display objects (icons) “multi-faceted, general-purpose, general, or attribute-free that do not depend on specific contents”. Then, as described above, various groupings may be made for each user.

  On the other hand, for example, when the information processing apparatus includes a touch panel, it is considered that the ease of performing a large contact operation is different from the ease of performing a small contact operation. For example, in the case of a contact operation that “encloses” an area like a circle, a smaller circle is easier to draw. It may be difficult to draw a large circle that fills the screen without using both hands.

  Here, for example, the user uses the data structure stored in the related information storage unit most frequently in the category “small” of the size of the selected area, which is presumed that the contact operation that is most easily performed has occurred. It is possible to have a structure in which related items to be associated are associated with each other.

  Therefore, according to the information processing apparatus of the present invention, when the margin area is selected by the user at the size of the contact operation that is most easily performed (that is, the size of the category “small”), It is possible to present the user with images of related items that are most frequently used by the user.

  Or, as another example, when a large contact operation is performed, the user has intentionally performed a relatively difficult contact operation, so that a “general independent” display object is displayed. It can be estimated that the user's willingness to do so is stronger.

  In such a case, for example, the user stores the structure of data stored in the related information storage unit in the category “large” of the size of the selection region where the contact operation that is most difficult to perform is estimated, A structure in which related items frequently used in specific cases (at home, when moving, etc.) can be associated.

  Therefore, according to the information processing apparatus of the present invention, when the user selects a blank area at the size of the contact operation that is most difficult to perform (that is, the size of the category “large”), It is also possible to present to the user images of related items used in special situations.

  As described above, by allowing the user to select a margin, it is desired that “a multi-faceted, general-purpose, general, or attribute-free display object (icon) that does not depend on specific contents” is desired. A user can intentionally and actively input information to the information processing apparatus. And the said information processing apparatus can show the user the icon which does not depend on specific content according to the said input. In addition, the operation situation can be estimated in consideration of the area of the margin area when the margin is selected, and an optimal icon can be presented to the user in conjunction with the estimated operation situation. It is possible to provide the user with the final result reflected more accurately.

  In the information processing apparatus of the present invention, the margin selection determination unit may recognize an object region in which a selectable object is displayed as a non-margin region and a region in which no selectable object is displayed as a margin region. .

  Furthermore, in the information processing apparatus of the present invention, the margin selection determination unit determines that the selection region has selected a margin region when the selection region does not include the object region, and When the selection area includes at least a part of the object area, the ratio of the object area overlapping the selection area to the selection area is obtained. If the ratio is less than the threshold, the selection is performed. It is preferable to determine that the area has selected a blank area, and to determine that the selected area has selected an object area if the writing ratio is equal to or greater than the threshold value.

  According to the above configuration, the margin selection determination unit can immediately determine that the margin area has been selected unless the selection area includes an object area. On the other hand, if the selection area includes an object area, the margin selection determination means does not immediately determine that the object has been selected, but obtains the proportion of the object area in the selection area and includes the object area. If the ratio is less than the predetermined threshold, it can be determined that the blank area has been selected.

  As a result, it is possible to avoid that the margin selection determination means erroneously determines that the object has been selected when the user has unintentionally selected an object in an attempt to select a margin. .

  Therefore, an appropriate related item is extracted based on the correct determination of the margin selection determination means, and an icon is displayed.

  Furthermore, in the information processing apparatus according to the present invention, when the margin selection determination unit determines that the selection area has selected an object area, an object at least partially overlapping the selection area is selected by the user. An object specifying means for specifying as an object is provided, and the related information storage section further stores an object and an item related to the object in association with each other, and the related item extraction means includes the related information storage section The item associated with the object specified by the object specifying means may be extracted as a related item.

  According to the above configuration, the screen processing means can present different icons (groups) to the user when the margin area is selected and when the object is selected. In other words, when a blank area is selected, an icon corresponding to a multi-faceted, general-purpose, general, or no-attribute related item that does not depend on specific contents is displayed, while an object is selected. In this case, an icon corresponding to a related item specialized for the selected object can be displayed.

  Even in such an information processing apparatus that produces different results, the selection operation performed by the user may be the same type of selection operation when selecting a margin and when selecting an object. There is no need to learn special gestures. By performing the same selection operation, the user can easily specify whether to call an icon that does not depend on a specific content or an icon specialized for a specific object.

  In the information processing apparatus of the present invention, the input unit and the display unit included in the information processing apparatus constitute a touch panel, and the trajectory acquisition unit acquires a trajectory of movement of the indicator that has moved on the touch panel. May be.

  According to the above configuration, it is possible to increase the relevance between the contact operation performed by the user for selecting an object and the resultant product obtained according to the operation, and the operation is performed in a natural flow that does not contradict the user's intuition. A screen can be provided. As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the touch panel.

  Alternatively, in the information processing apparatus of the present invention, the input unit included in the information processing apparatus inputs an instruction to move the cursor as the indicator displayed on the display unit to the information processing apparatus. The locus acquisition means may acquire a locus of movement of the cursor.

  According to the above configuration, it is possible to increase the relevance between the input operation performed for the user to select an object by operating the input unit and the resultant product obtained according to the operation, and the user's intuition The operation screen can be provided in a natural flow that does not violate. As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the display unit and the input unit.

  In order to solve the above problems, the screen display method of the present invention is a screen display method in an information processing apparatus, and acquires a trajectory obtained by moving a pointer that indicates the position of the screen of the display unit, A margin selection determination step for determining whether or not the selection region identified based on the locus acquired in the locus acquisition step selects a margin region in which information on the display unit is not displayed; and When it is determined in the margin selection determination step that the selected area has selected a margin area, the margin information is associated with the event by referring to a related information storage unit that stores an item related to the selected event. A related item extracting step for extracting the selected item as a related item, and a screen for displaying an icon corresponding to the related item extracted in the related item extracting step on the display unit It is characterized by comprising a management step.

  The information processing apparatus may be realized by a computer. In this case, an information processing apparatus control program for causing the information processing apparatus to be realized by the computer by causing the computer to operate as the above-described means, and A computer-readable recording medium on which is recorded also falls within the scope of the present invention.

  In order to solve the above problems, an information processing apparatus according to the present invention is based on a trajectory acquisition unit that acquires a trajectory of movement of an indicator that indicates a position of a screen of a display unit, and a trajectory acquired by the trajectory acquisition unit. A margin selection determination means for determining whether or not the selected selection area has selected a blank area for which information on the display unit is not displayed, and the selection area is determined to be a blank area by the margin selection determination means. When it is determined that the item is selected, the related item extraction unit extracts the item associated with the event as a related item with reference to the related information storage unit that stores the item associated with the event for which the blank area is selected. And a screen processing means for displaying an icon corresponding to the related item extracted by the related item extracting means on the display unit.

  In order to solve the above problems, the screen display method of the present invention is a screen display method in an information processing apparatus, and acquires a trajectory obtained by moving a pointer that indicates the position of the screen of the display unit, A margin selection determination step for determining whether or not the selection region identified based on the locus acquired in the locus acquisition step selects a margin region in which information on the display unit is not displayed; and When it is determined in the margin selection determination step that the selected area has selected a margin area, the margin information is associated with the event by referring to a related information storage unit that stores an item related to the selected event. A related item extracting step for extracting the selected item as a related item, and a screen for displaying an icon corresponding to the related item extracted in the related item extracting step on the display unit It is characterized by comprising a management step.

  As a result, there is an effect that it is possible to realize excellent operability in the information processing apparatus including the input unit and the display unit.

It is a functional block diagram which shows the principal part structure of the tablet terminal in one Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the tablet terminal in one Embodiment of this invention. It is a top view which shows the external appearance of the tablet terminal in one Embodiment of this invention. It is a figure explaining a mode when a user hold | grips and operates a tablet terminal, (a) is a figure explaining a mode that a tablet terminal is hold | gripped with one hand and is operated with the hand, (b) It is a figure explaining a mode that a tablet terminal is hold | gripped with one hand and is operated with the other hand. (A) is a figure which shows that the user performed the contact operation | movement which "encloses" a margin area | region with the intention of selecting a margin, (b) is shown to (a) of the figure. It is a figure which shows an example of the contact information which the contact information generation part produced | generated with contact operation | movement, (c) shows an example of the map information of the video frame displayed on the display part in the period when the contact was detected. FIG. It is a figure which shows an example of the relevant information memorize | stored in the relevant information storage part of a tablet terminal. It is a figure which shows the specific example of the icon image memorize | stored in the icon memory | storage part of a tablet terminal. It is a figure which shows the specific example of the icon image memorize | stored in the icon memory | storage part of a tablet terminal. It is a figure which shows the example of arrangement | positioning when the operation screen process part of a tablet terminal arrange | positions an icon. It is a figure which shows the specific example of the operation screen obtained as a result of the operation screen production | generation process which the operation screen process part of the tablet terminal performed. It is a flowchart which shows the flow of the operation screen display process by a tablet terminal. It is a functional block diagram which shows the principal part structure of the tablet terminal in other embodiment of this invention. It is a figure which shows an example of the relevant information memorize | stored in the relevant information storage part of the tablet terminal in other embodiment. It is a figure which shows the specific example of the operation screen obtained as a result of the operation screen production | generation process performed by the operation screen process part in other embodiment. (A) is a figure which shows that the user performed the "surround" contact operation | movement with the intention of selecting a blank area | region, when the tablet terminal is in the state which is displaying the "photo application screen". (B) is a figure which shows the map information of the image | video frame displayed on the display part in the period when the contact was detected, and the selection area | region specified by the said "surrounding" contact operation | movement. It is a figure which shows the icon displayable area | region in the said "photo application screen", and the example of an icon arrangement | positioning. It is a figure which shows the specific example of the operation screen obtained as a result of the operation screen production | generation process performed by the operation screen process part in the said other embodiment. It is a flowchart which shows the flow of the operation screen display process by the tablet terminal in said other embodiment. It is a figure which shows the specific example of the map information containing the division information memorize | stored in the frame map memory | storage part. (A) is a diagram showing that the user has performed an “enclose” contact operation with the intention of selecting a blank area, and (b) is map information including division information and the above “enclose”. It is a figure which shows the selection area | region specified by contact operation. It is a figure which shows an example of the relevant information memorize | stored in the relevant information storage part in further another embodiment of this invention. It is a figure which shows the specific example of the operation screen obtained as a result of the operation screen production | generation process performed by the operation screen process part in the said further another embodiment. It is a figure which shows some examples of a locus | trajectory when a user performs the "enclose" contact operation | movement of various magnitude | sizes with the intention of selecting a margin area | region. It is a figure which shows an example of the relevant information memorize | stored in the relevant information storage part in other embodiment. (A) is a figure which shows the specific example of the operation screen produced by the said operation screen process part when the contact operation of a predetermined | prescribed magnitude | size arises, (b) is a figure of (a) of the figure. It is a figure which shows the specific example of the operation screen produced by the said operation screen process part, when the contact operation | movement of the predetermined magnitude | size different from the case arises. It is a functional block diagram which shows the principal part structure of the tablet terminal in further another embodiment of this invention. It is a figure which shows an example of the relevant information memorize | stored in the relevant information storage part in the said further another embodiment. It is a figure which shows the specific example of the operation screen produced by the operation screen process part in the said further another embodiment, when the contact operation | movement of a predetermined magnitude | size arises. It is a figure which shows the specific example of the operation screen produced by the said operation screen process part, when the contact operation | movement of the predetermined magnitude | size different from the case of FIG. 27 arises. It is a figure which shows the specific example of the operation screen produced by the said operation screen process part, when the contact operation | movement of the predetermined magnitude | size different from the case of FIG. 27 and FIG. 28 arises. It is a figure which shows the specific example of the correspondence table of "margin area" and "the number of arrangement | positioning icons" memorize | stored in a related information storage part. It is a functional block diagram which shows the principal part structure of the tablet terminal in further another embodiment of this invention. It is a figure which shows an example of the relevant information memorize | stored in the relevant information storage part in the said further another embodiment. It is a figure which shows an example of the database of the registration image memorize | stored in the registration image memory | storage part. It is a figure which shows the specific example of the wallpaper display screen produced by the registration image process part, when the contact operation | movement of a predetermined magnitude | size arises in a predetermined position at the time of a tablet terminal displaying a home screen. (A) is a figure which shows the specific example of the wallpaper display screen produced by the registration image process part, when the contact operation of the predetermined position and magnitude | size different from the case of FIG. 34 arises at the time of a home screen display. (B) is created by the registered image processing unit when a contact operation having a predetermined position and size different from those in FIGS. 34 and 35 (a) occurs when the home screen is displayed. It is a figure which shows the specific example of the set wallpaper display screen. (A) is a diagram showing that when the tablet terminal is displaying the “schedule application screen”, the user has performed a “surrounding” contact operation with the intention of selecting a blank area; b) is a diagram showing map information of a video frame displayed on the display unit during a period in which contact is detected, and a selection area specified by the “enclose” contact operation. It is a figure which shows the specific example of the operation screen obtained as a result of the operation screen production | generation process performed by the operation screen process part in other embodiment of this invention.

Embodiment 1
The embodiment of the present invention will be described below with reference to FIGS.

  In the embodiment described below, a case where the information processing apparatus of the present invention is applied to a tablet terminal will be described as an example. In the present embodiment, as an example, the tablet terminal is realized by a small smartphone that can be operated with one hand and is excellent in portability.

  However, the information processing apparatus of the present invention is not limited to the above example, and the information processing apparatus of the present invention can be applied to an information processing apparatus of any size (for example, an electronic blackboard equipped with a large touch panel). Good.

[Hardware configuration of tablet terminal]
FIG. 2 is a block diagram illustrating a hardware configuration of the tablet terminal 100 according to the present embodiment. As shown in FIG. 2, the tablet terminal 100 includes at least a control unit 10, an input unit 11, a display unit 12, and a storage unit 19. Furthermore, the tablet terminal 100 may include an operation unit 13, an external interface 14, a communication unit 15, a wireless communication unit 16, an audio output unit 17, and an audio input unit 18 in order to realize inherent functions. .

  When the tablet terminal 100 is a multi-function mobile communication terminal such as a smartphone, the tablet terminal 100 is omitted here. However, the tablet terminal 100 includes a call processing unit, an imaging unit (such as a lens / image sensor) that performs imaging, and a broadcast image. Other parts (such as a tuner / demodulation unit), GPS, and sensors (such as an acceleration sensor and an inclination sensor) may be included as well as various components that are typically included in a smartphone.

  The input unit 11 is for inputting an instruction signal for the user to operate the tablet terminal 100 via the touch panel. The input unit 11 is a touch surface that accepts contact with an indicator (indicating the screen position of the display unit 12, here, for example, a finger or a pen), and contact / non-contact between the indicator and the touch surface. (Approach / non-approach) and a touch sensor for detecting the contact (approach) position. The touch sensor may be realized by any sensor as long as it can detect contact / non-contact between the indicator and the touch surface. For example, it is realized by a pressure sensor, a capacitance sensor, an optical sensor, or the like.

  The display unit 12 displays an object to be processed by the tablet terminal 100 (any display object such as an icon) and a processing result, and displays an operation screen for the user to operate the tablet terminal 100 using a GUI (Graphical (User Interface) screen. The display unit 12 is realized by a display device such as an LCD (Liquid Crystal Display).

  In this embodiment, the input part 11 and the display part 12 are integrally molded, and these constitute a touch panel. Therefore, in such an embodiment, an object to be moved (operated) to indicate a screen position, that is, an operation body (here, a finger or a pen) is simultaneously positioned on the screen of the display unit 12. It is also an indicator that indicates

  For example, when the touch panel of the tablet terminal 100 of the present invention is realized by a projected capacitive touch panel, specifically, the touch sensor has a matrix-like transparent electrode pattern made of ITO (Indium Tin Oxide) or the like, It is formed on a transparent substrate such as glass or plastic. When an indicator (such as a user's finger or pen) touches or approaches the touch sensor, the electrostatic capacity of a plurality of transparent electrode patterns in the vicinity changes. Therefore, the control unit 10 can detect the position where the indicator is in contact or approached by detecting a change in the current or voltage of the transparent electrode pattern.

  In the following, the term “contact” when “contact detection”, “contact operation”, “contact position”, etc. is not only the state in which the indicator and the touch surface are in complete contact (contact), It also includes a state in which the indicator and the touch surface are close (approaching) to the extent that the touch sensor can detect.

  The operation unit 13 is for the user to directly input an instruction signal to the tablet terminal 100. For example, the operation unit 13 is realized by an appropriate input mechanism such as a button, switch, key, or jog dial. For example, the operation unit 13 is a switch for turning on / off the power of the tablet terminal 100.

  The external interface 14 is an interface for connecting an external device to the tablet terminal 100. The external interface 14 is realized by, for example, but not limited to, a socket for inserting an external recording medium (memory card or the like), an HDMI (High Definition Multimedia Interface) terminal, a USB (Universal Serial Bus) terminal, or the like. The The control unit 10 of the tablet terminal 100 can exchange data with an external device via the external interface 14.

  The communication unit 15 communicates with an external device via a communication network. The communication unit 15 is connected to various communication terminals via a communication network, and realizes data transmission / reception between the tablet terminal 100 and the communication terminal. Further, when the tablet terminal 100 is a mobile communication terminal such as a smartphone, the communication unit 15 transmits / receives voice call data, e-mail data, and the like to / from other devices via the mobile phone network. To do.

  The wireless communication unit 16 communicates with an external device wirelessly. The wireless communication unit 16 is not particularly limited, and may implement any wireless communication means such as infrared communication such as IrDA and IrSS, Bluetooth communication, WiFi communication, and a non-contact type IC card. A plurality of means may be realized. The control unit 10 of the tablet terminal 100 can communicate with devices in the vicinity of the tablet terminal 100 via the wireless communication unit 16, and can exchange data with these devices.

  The audio output unit 17 outputs audio data processed by the tablet terminal 100 as audio, and is realized by a speaker, a headphone terminal, headphones, and the like.

  The voice input unit 18 receives voice input generated outside the tablet terminal 100 and is realized by a microphone or the like.

  The storage unit 19 includes (1) a control program executed by the control unit 10 of the tablet terminal 100, (2) an OS program, and (3) an application program for the control unit 10 to execute various functions of the tablet terminal 100, And (4) storing various data read when the application program is executed. Alternatively, (5) the control unit 10 stores data used for calculation and calculation results in the course of executing various functions. For example, the above data (1) to (4) are stored in a nonvolatile storage device such as a ROM (read only memory), a flash memory, an EPROM (Erasable Programmable ROM), an EEPROM (Electrically EPROM), an HDD (Hard Disc Drive), etc. Is remembered. For example, the data (5) is stored in a volatile storage device such as a RAM (Random Access Memory). Which data is to be stored in which storage device is appropriately determined based on the purpose of use, convenience, cost, physical restrictions, and the like of the tablet terminal 100.

  The control unit 10 performs overall control of each unit included in the tablet terminal 100. The control unit 10 is realized by, for example, a CPU (central processing unit) or the like. The functions of the tablet terminal 100 are such that the CPU as the control unit 10 reads a program stored in a ROM or the like into a RAM or the like and executes the program. It is realized by doing. Various functions (particularly, the operation screen display function of the present invention) realized by the control unit 10 will be described later with reference to other drawings.

[Appearance of tablet terminal]
FIG. 3 is a plan view showing the appearance of the tablet terminal 100. As illustrated in FIG. 3, the tablet terminal 100 includes an input unit 11 and a display unit 12 as a touch panel. The tablet terminal 100 includes an operation unit 13, an external interface 14, a wireless communication unit 16, an audio output unit 17, an audio input unit 18, and the like, although these are not essential components. For example, when the wireless communication unit 16 is realized by infrared communication means, an infrared transmission / reception unit is provided as the wireless communication unit 16 on the side surface of the tablet terminal 100.

  FIG. 4 is a diagram illustrating a state when the user holds and operates the tablet terminal 100. More specifically, FIG. 4A is a diagram illustrating a state in which the tablet terminal 100 is gripped with one hand and is operated with that hand, and FIG. It is a figure explaining a mode that it is hold | gripped with a hand and operated with the other hand.

  In the present embodiment, the tablet terminal 100 is a palm-sized information processing apparatus that can be held with one hand. As shown in FIG. 4A, the tablet terminal 100 is held with the thumb of the hand while holding the tablet terminal 100 with one hand. The touch surface of the input unit 11 can be operated. For example, when there is an icon to be operated at a position where the thumb does not reach, the tablet terminal 100 draws the icon near the thumb by flicking, and surrounds or taps the icon with the thumb. The icon can be selected.

  Further, as illustrated in FIG. 4B, the user may hold the tablet terminal 100 with one hand and operate the touch surface of the input unit 11 with the finger of the other hand. Alternatively, although not shown, the tablet terminal 100 may be horizontally long, hold both sides with both hands, and operate the touch surface of the input unit 11 with the thumbs of both hands.

[Tablet function]
Next, the functional configuration of the tablet terminal 100 will be described. FIG. 1 is a functional block diagram illustrating a main configuration of the tablet terminal 100 according to the present embodiment.

  As shown in FIG. 1, the control unit 10 of the tablet terminal 100 according to the present embodiment includes at least a contact information generation unit 21, a margin selection determination unit 28, as functional blocks for realizing the operation screen display function of the present invention. A related item extraction unit 23 and an operation screen processing unit 24 are provided. Preferably, the control unit 10 may further include an object specifying unit 22.

  Each functional block of the control unit 10 described above includes a CPU (central processing unit) that stores a program stored in a non-volatile storage device realized by a ROM (read only memory) or the like (RAM (random access memory)). It can be realized by reading and executing the above.

  In addition, the storage unit 19 is specifically a frame map storage unit 41 or a related unit as a storage unit for reading or writing data when the above-described units of the control unit 10 execute the operation screen display function. An information storage unit 42 and an icon storage unit 43 are included.

  The contact information generation part 21 processes the signal output from the touch sensor of the input part 11, and produces | generates contact information. The contact information includes at least contact coordinate information indicating the coordinate position of the contact position of the indicator (for example, a finger). Thereby, each part of the control part 10 can acquire the locus | trajectory of the movement of the said indicator from the said contact information. In the present embodiment, contact time information indicating a time when contact has occurred may be associated with each point constituting the trajectory as necessary.

  More specifically, the contact information generation unit 21 from when the touch sensor of the input unit 11 detects contact between the touch surface and the indicator (in this embodiment, a finger) until it detects non-contact. Acquires a signal output from the touch sensor. This signal includes information indicating that “contact” has been detected and information indicating the contact position. Based on this signal, the contact information generation unit 21 generates contact coordinate information indicating the contact position in coordinates. Generate.

  Furthermore, although not essential, the contact information generation unit 21 may measure the time from when contact is detected until it becomes non-contact, and associate the contact time information with the contact coordinate information. The contact information generation unit 21 may acquire and use absolute time information held by the clock unit mounted on the tablet terminal 100. However, in the present embodiment, the contact information generation unit 21 detects contact. Then, timing is started and relative contact time information is obtained. For example, the contact information generation unit 21 measures the elapsed time with the time point when the contact is first detected (t0) as 0.00 seconds, and continues the measurement until the time point when the contact is finally detected (tn). Thus, the relative contact time information corresponding to the contact position may be acquired. The contact information generation unit 21 generates contact information by associating the obtained contact time information with the contact coordinate information.

  In the present embodiment, the generated contact information is supplied to the margin selection determination unit 28 and is used by the margin selection determination unit 28. In the configuration in which the control unit 10 includes the object specifying unit 22, the contact information may be supplied to the object specifying unit 22, and the object specifying unit 22 may use the contact information to specify the object.

  The margin selection determination unit 28 determines which portion of the content displayed on the display unit 12 is selected by the user's contact operation. The margin selection determination unit 28 determines which of a non-margin area where some information is displayed and a margin area where no information is displayed is selected by the user. In particular, in this embodiment, the margin selection determination unit 28 determines whether the selected object is any object or no object is selected. That is, it is determined whether an object area (non-margin area) is selected or an area where no object is displayed (margin area) is selected.

  In the present embodiment, the tablet terminal 100 performs different processes depending on whether an object is selected or a margin is selected. In the present embodiment, a function (operation screen display function) of displaying a menu (that is, an operation screen) is assigned to a user operation of selecting a margin. Therefore, when the margin selection determination unit 28 determines that a margin is selected, each unit of the control unit 10 executes a process for displaying a menu. In particular, the operation screen processing unit 24 executes an operation screen generation process. On the other hand, when the margin selection determination unit 28 determines that some object is selected instead of the margin, each unit of the control unit 10 performs processing (object-dependent processing) according to the selected object. The object-dependent processing is not limited to this, but includes, for example, processing in which the object specifying unit 22 specifies an object, and displays each related item associated with the object specified by the object specifying unit 22 as a menu. It is done. Also in this case, the operation screen processing unit 24 executes an operation screen generation process.

  More specifically, the margin selection determination unit 28 compares the contact information generated by the contact information generation unit 21 with the map information of the video frame displayed on the display unit 12 while the contact is occurring. .

  The frame map storage unit 41 stores map information of the video frame output to the display unit 12 at the time of contact. The map information is information indicating the layout of the video frame displayed on the touch panel. Specifically, the map information includes information for individually identifying each object displayed, and information on the shape, size, and display position of each object. That is, the map information is obtained by plotting each object corresponding to the coordinate system of the touch panel.

  The margin selection determination unit 28 reads the map information stored in the frame map storage unit 41 and executes the above-described determination process.

  FIG. 5 is a diagram for explaining the operation of the margin selection determination unit 28. More specifically, FIG. 5A is a diagram showing that the user has performed a contact operation of “enclosing” a margin (a region other than an object) with the intention of selecting the margin. FIG. 5B is a diagram illustrating an example of contact information generated by the contact information generation unit 21 in accordance with the contact operation illustrated in FIG. (C) of FIG. 5 is a figure which shows an example of the map information of the video frame displayed on the display part 12 in the period of t0-tn in which the contact was detected.

  As illustrated in FIG. 5A, it is assumed that the user performs a contact operation of “enclosing” a blank area (an area where no object is shown) displayed on the touch panel of the tablet terminal 100. For example, it is assumed that the contact operation is performed in a period from t0 to tn so that the contact point passes the position of the broken line in FIG.

  The margin selection determination unit 28 acquires contact information as illustrated in FIG. 5B from the contact information generation unit 21. In the present embodiment, the coordinate system of the contact information corresponds to the coordinate system of the touch panel of the tablet terminal 100, and has the leftmost upper end of the panel as the origin. In FIG. 5B, among the trajectories of the “enclose” contact movement of the user, the start point is indicated as t0 and the end point is indicated as tn. However, contact time information may also be associated with each point in between. Or in this embodiment, there is no need of contact time information.

  The margin selection determination unit 28 acquires the map information (that is, the layout of the video frame displayed on the display unit 12 during the period from t0 to tn) shown in FIG. 5C from the frame map storage unit 41. In the example shown in FIG. 5, three buttons (objects) are displayed at the bottom of the touch panel screen. Therefore, the map information shown in FIG. 5C shows object areas A to C corresponding to these three buttons.

  The margin selection determination unit 28 compares the contact information with the map information to determine whether or not the object region overlaps the selection region specified by the user's contact operation. Determine whether they overlap. Based on the selection area, it is determined whether a blank area or an object is selected. As a method for the margin selection determination unit 28 to specify the selection area, for example, as shown in FIG. 5C, within the area surrounded by the locus of the user's finger obtained from the contact information (the hatched area within the thick frame) ) May be specified as the selection area, or the circumscribed rectangular area of the trajectory may be specified as the selection area.

  If the object area does not overlap the selection area specified by the trajectory, the margin selection determination unit 28 determines that the object has not been selected, that is, the blank area has been selected. Alternatively, as shown in FIG. 5C, even if the object region partially overlaps the selection region (bold frame hatched region), the overlapping object region is a predetermined threshold% (for example, 20%) of the selection region. %), The margin selection determination unit 28 can determine that the margin area has been selected without determining that the object has been selected. For example, the selection area shown in FIG. 5C overlaps part of the object area A and the object area B. However, this overlapping object area is less than 10% of the entire selected area. That is, it is less than 20% of the predetermined threshold. As a result, the margin selection determination unit 28 can determine that it is not the object region A or the region B but the blank region that is selected by the “enclose” contact operation. In other words, the margin selection determination unit 28 performs a highly accurate determination that more accurately reflects the user's intention by considering the ratio of the portion of the object region that overlaps the selection region to the selection region. Can do.

  The margin selection determination unit 28 may further determine whether an object has been selected or a margin region has been selected in consideration of the positional relationship between the selection region and the object region. This configuration is particularly useful in the following situations.

  That is, if the object area that the user wants to select is very small, even if the user surrounds the object of interest, the user's “enclose” contact motion is relative to the size of the object. There may be a situation in which it must be extremely large. The object area of a small object is extremely small with respect to the selection area specified by the trajectory of the “enclose” contact movement, and the object area may occupy less than 20% of the selection area. Can happen. In this case, there is a disadvantage that the margin selection determination unit 28 erroneously determines that the margin area has been selected even though the user has drawn a trajectory with the intention of selecting the small object.

  Therefore, in order to eliminate such inconvenience, the margin selection determination unit 28 is a case where the object region overlaps the selected region, and the overlapping object region is less than a predetermined threshold% of the selected region. If the area is not occupied, it is not determined immediately that the blank area has been selected, and the positional relationship between the selected area and the object area is further considered.

  Specifically, the margin selection determination unit 28, even when the object region is small relative to the area of the selection region when the very small object region is included near the center of the selection region. It may be determined that the object in the object area has been selected. On the other hand, if the very small object area is not included near the center of the selected area, the margin selection determining unit 28 may still determine that the margin area has been selected.

  When the user draws a circle to surround the margin, it is conceivable that a small object is included unintentionally toward the end of the circle, but it is unlikely that an object is included in the middle of the circle. On the other hand, when the user selects a target object (for example, “encloses”), it is natural to draw a circle so that the target object is substantially at the center regardless of the size of the circle. Conceivable.

  Therefore, according to the above configuration, even when the selection area selected by the user is extremely large with respect to the object, the margin selection determination unit 28 immediately selects the “margin area” for such a situation. It is possible to verify in more detail whether the blank area has been selected or whether the object area has been selected in consideration of the positional relationship between the selected area and the object area. Thus, the margin selection determination unit 28 can output a determination result that reflects the user's intention more accurately.

  In the present embodiment, when the margin selection determination unit 28 determines that a margin region has been selected, the margin selection is supplied to the related item extraction unit 23, and the margin is selected by the related item extraction unit 23. Prompt you to execute the process. If the margin selection determination unit 28 determines that any object has been selected, the margin selection determination unit 28 supplies the determination result to, for example, the object specifying unit 22 and specifies the selected object to the object specifying unit 22. Prompt. After the object is specified, appropriate processing associated with the specified object is executed by each unit (including components not shown) of the control unit 10.

  The object specifying unit 22 specifies an object selected by a user's contact operation. The object specifying unit 22 specifies what the selected object is when the margin selection determining unit 28 determines that an object has been selected by the user.

  Specifically, the object specifying unit 22 compares the contact information generated by the contact information generating unit 21 with the map information of the video frame displayed on the display unit 12 while the contact is occurring. Thereby, the object specification part 22 can specify the object area | region which overlaps with the selection area specified by the locus | trajectory obtained from contact operation, and can specify what the object corresponding to the object area | region is. For example, when most of the selection area overlaps the object area A in FIG. 5C, the object specifying unit 22 specifies a button object corresponding to the object area A, and this is selected by the selection area. Can be determined. When this button is assigned a function for starting a mailer, a mailer function (not shown) activates a mail application in accordance with the identification result of the object identification unit 22.

  Alternatively, the object specifying unit 22 may supply information on the specified object to the related item extracting unit 23 and prompt the related item extracting unit 23 to extract related items related to the object.

  If an object is selected and the process to be executed by the tablet terminal 100 is uniquely determined, it is sufficient for the margin selection determination unit 28 to determine that some object has been selected. In this case, the object specifying unit 22 may not be provided in the control unit 10.

  The related item extracting unit 23 extracts related items according to the determination result of the margin selection determining unit 28. For example, if the margin selection determination unit 28 determines that a margin region has been selected, the related item extraction unit 23 stores the related item associated with the event “margin region has been selected” as related information storage. Extracted from a plurality of related items stored in the unit 42.

  If the object specifying unit 22 is provided, the related item extracting unit 23 may extract the related item related to the object specified by the object specifying unit 22, that is, the object selected by the user. Good. When an object is selected, items that are closely related to the selected object are extracted by the related item extraction unit 23.

  The related information storage unit 42 stores related information indicating the relationship between “an event in which a blank area is selected” and an item to be displayed as a menu. FIG. 6 is a diagram illustrating an example of related information stored in the related information storage unit 42. Further, the related information storage unit 42 may store related information indicating the relationship between the “object” and the item.

  As shown in FIG. 6, the related information is information in which “related item” is associated with at least an event in which a blank area is selected (hereinafter, “object“ none ”). The association information indicates the association between the “event in which the margin is selected” and the item by this association. Further, the related information may be information indicating the relationship between the object and the item.

  If the margin selection determination unit 28 determines that the “margin area has been selected (no object selection)”, the related item extraction unit 23 refers to the related information stored in the related information storage unit 42 and reads “object Items related to “None” are extracted as related items.

  In the event of “no object”, it is considered that the user did not select a specific object of interest but rather performed a contact operation for the purpose of calling the menu anyway. Therefore, it is preferable that a general item often performed by the user is associated as a related item with respect to the event of the object “none”. For example, in the related information shown in FIG. 6, three related items, R1 to R3, are associated with the event “object none”. Therefore, the related item extraction unit 23 extracts the three related items R1 to R3 when the margin selection determination unit 28 determines that a blank area has been selected.

  The item associated with the event “none” of the object is not always determined in advance as in the example illustrated in FIG. 6. For example, the configuration may be such that some items such as functions recently used by the user and frequently used functions are always dynamically associated.

  In addition, when the object specifying unit 22 specifies an object, the related item extracting unit 23 refers to the related information stored in the related information storage unit 42 and selects an item associated with the specified object. You may extract as a related item.

  For example, when the specified object is data such as “photograph”, for a photo, “display”, “edit”, “send to mail”, “peripheral devices (TV etc.) It is assumed that operations such as “transfer to” and “print” are executed. Therefore, an item corresponding to an “action” executed on an object that is an “action target” may be associated as an associated item of the object (related items R4 to R11 in FIG. 6).

  Alternatively, when the object is data such as “photo”, it is assumed that the photo is sent to a specific person. Therefore, an item corresponding to the relationship of “action partner” when an action is executed on an object that is an action target may be extracted as a related item.

  Alternatively, if the object is an “album” or “folder” that contains multiple photos or other data, the user may want the photos or data contained in the object. In this way, items belonging to the lower layer of the object may be extracted as related items.

  Information on the related items extracted by the related item extraction unit 23 is supplied to the operation screen processing unit 24. Then, the extracted related item is displayed so as to be selectable (for example, as an icon) as an item related to the event in which the blank area is selected (or the previously selected object).

  Although not limited to this configuration, in this embodiment, as shown in FIG. 6, an icon may be assigned to each “related item”. For example, the icon “1: TV” is associated with the related item R1 “Activate TV” associated with “no object”. The icon “1: TV” is, for example, an icon on which a television illustration or the like is drawn, and is preferably a pattern that reminds of “watching TV”. Other related items are also assigned icons with appropriate patterns that recall the contents of the related items.

  Based on such related information, the related item extraction unit 23 may supply the operation screen processing unit 24 with icons (or icon identification information) corresponding to the extracted related items. Thereby, the operation screen processing unit 24 can proceed to display the icon specified by the related item extraction unit 23.

  The operation screen processing unit 24 performs processing (operation screen generation processing) for generating an operation screen for displaying the related item (its icon) extracted by the related item extraction unit 23 in a selectable manner for the user. .

  7A and 7B are diagrams illustrating specific examples of icon images stored in the icon storage unit 43. FIG. As shown in FIGS. 7A and 7B, in this embodiment, each icon image can be identified by icon identification information. For example, the icon identification information “1: TV” is associated with an icon image depicting a TV. In addition, although not shown, a portrait or an avatar image of the person may be used as an icon representing personal information such as an acquaintance who often makes a call.

  The operation screen processing unit 24 reads the icon images assigned to the related items extracted by the related item extraction unit 23 from the icon storage unit 43, and performs operations so that these are displayed at an appropriate position and an appropriate timing. A screen is generated and output to the display unit 12 via a display control unit (not shown).

  Specifically, in the present embodiment, the operation screen processing unit 24 has a function of circularly displaying icons of related items related to the blank area (or object) selected by the touch operation “enclose”. Yes.

  FIG. 8 is a diagram for explaining the processing contents of the operation screen processing unit 24. More specifically, FIG. 8 is a diagram showing an example of icon arrangement of related items executed by the operation screen processing unit 24.

  In the present embodiment, the operation screen processing unit 24 first identifies the shape and position of the ring for placing the extracted icons of related items, as indicated by the broken line in FIG. And each icon is arrange | positioned along the outline of a ring. Although how to determine the shape of the ring, the position of the ring, and the arrangement of the icons is not particularly limited, an example is shown below.

  For example, in the present embodiment, the operation screen processing unit 24 is assumed to have a ring shape that is predetermined as a circle or an ellipse. Then, the operation screen processing unit 24 defines a ring as large as possible in an area where the icons can be displayed so that the icons are displayed in a larger or larger size. For example, in FIG. 5A, the area other than the area where the battery information, date information, and other objects are already displayed on the touch panel screen is set as an area where icons can be displayed. In this case, as illustrated in FIG. 8, the operation screen processing unit 24 defines a ring (circle or ellipse) that is as large as possible in an area where an icon can be displayed (image display area; white area in the figure).

  Subsequently, in the present embodiment, the operation screen processing unit 24 arranges each icon of the extracted related item on the defined outline of the ring. FIG. 8 shows an example in which, when three related items are extracted, the operation screen processing unit 24 arranges three icons at equal intervals along the outline of the defined elliptical ring. .

  Here, regarding the shape of the reference “ring” indicating the arrangement position of the icon, the oval shape is merely an example, and there is no intention to limit the shape of the ring of the present invention. Further, the “ring” does not necessarily mean a shape formed by a curve. For example, the operation screen processing unit 24 may define the shape of the ring as a circle, square, rectangle, or other polygon, or may include a complicated shape, irregular shape, or non-geometric shape. A figure having an outline that separates the outside from the outside is defined as a ring. Further, “ring” does not necessarily mean a closed curve. Even if the starting point and the ending point of the contour line of the ring are not completely coincident with each other, a contour line that largely separates the inside and the outside may be defined as the ring. The operation screen processing unit 24 arranges icons on the contour lines of the rings having any shapes defined as described above.

  The broken line indicating the outline of the ring shown in FIG. 8 is the shape of the ring that the tablet terminal 100 holds as information inside, and may not actually be displayed on the display unit 12. Similarly, the broken line indicating the outline of the ring in each of the drawings shown below is not actually displayed on the display unit 12.

  The order in which the icons are arranged is not particularly limited in the present embodiment. For example, the icons may be arranged in the order extracted by the related item extraction unit 23 in the clockwise direction from the top of the ring.

  FIG. 9 is a diagram illustrating a specific example of the operation screen obtained as a result of the operation screen generation process executed by the operation screen processing unit 24. The example shown in FIG. 9 is a specific example of the operation screen obtained when the blank area is surrounded by the home screen (standby screen) of the tablet terminal 100 as shown in FIG.

  The related item extraction unit 23 refers to the related information shown in FIG. 6 and extracts the related items R1 to R3 associated with the blank area (object “none”). In the present embodiment, the related item extracting unit 23 further includes icon identification information associated with the related items R1 to R3, that is, “1: TV”, “13: Web browser application”, and “3 : Mail "may be extracted.

  Based on the icon identification information extracted by the related item extraction unit 23, the operation screen processing unit 24 reads a corresponding icon image from the icon storage unit 43 as shown in FIGS. 7A and 7B. Then, a ring is defined based on a region where icons can be displayed, and the read icon image is arranged on the outline. In the example shown in FIG. 9, three icons “1: TV”, “13: Web browser application”, and “3: Mail” are arranged on the contour line in the order of extraction in the clockwise direction and at equal intervals. And displayed on the display unit 12.

[Operation screen display flow]
Next, a processing flow when the tablet terminal 100 executes the operation screen display function will be described. FIG. 10 is a flowchart showing a flow of operation screen display processing by the tablet terminal 100.

  When the input unit 11 detects that an indicator (such as a user's finger) has touched the touch surface of the touch panel (YES in S101), the contact information generation unit 21 starts the finger from that time (t = t0). Acquisition of the contact coordinate information indicating the contact position is started, and this is acquired over time (S102). This tracking of the contact position is continued until no contact is detected between the touch surface and the finger (NO in S103). When contact is not detected in input unit 11 (YES in S103), contact information generation unit 21 generates contact information including contact coordinate information acquired from t = t0 to this time (t = tn). (S104). In the present embodiment, the contact time information may or may not be included in the contact information.

  The margin selection determination unit 28 uses the contact information generated in S104 (for example, the locus shown in FIG. 5B) and the map information stored in the frame map storage unit 41 (for example, FIG. 5C). ) To determine whether the user has enclosed a margin (a margin region has been selected) or an object region has been enclosed (some object has been selected) (S105).

  In the example shown in FIG. 5C, since the object area does not substantially overlap the selection area specified by the locus (the hatched area in FIG. 5C), the margin selection determination unit 28 Then, it is determined that a blank area has been selected (A in S105).

  In this case, the related item extraction unit 23 refers to the related information (for example, FIG. 6) in the related information storage unit 42 based on the event “(no selected object)” identified in S105, Related items of “none” are extracted (S106). Alternatively, identification information of icons assigned to related items may be extracted.

  The operation screen processing unit 24 acquires the icon image of the related item extracted in S106 from the icon storage unit 43 (for example, FIGS. 7A and 7B). Then, the acquired icon image is arranged on the outline of the ring that is defined based on a predetermined rule, and an operation screen is generated (S107).

  The video signal of the operation screen generated as described above is output to the display unit 12. Then, as shown in FIG. 9, the operation screen is displayed on the display unit 12 of the tablet terminal 100.

  On the other hand, in S105, when most of the selection area specified by the locus (the hatched area in FIG. 5C) overlaps the object area, the margin selection determination unit 28 selects an object. (B in S105).

  In this case, each unit responsible for the downstream process of the margin selection determination unit 28 executes processing when an object is selected (S108). Although there is no intention to limit the present invention, an example of the processing is as follows.

  For example, the object specifying unit 22 specifies a selection region based on the trajectory acquired from the contact information generated in S104, and compares this selection region with the map information stored in the frame map storage unit 41. To do. Then, an object area almost overlapping with the selected area is specified, and an object corresponding to the object area is specified as an object selected by the user (that is, the selected area is selected) (SS1). The related item extracting unit 23 refers to the related information (FIG. 6) stored in the related information storage unit 42, and extracts related items associated with the identified object (SS2). For example, when the selected object is specified as “photograph”, the related item extracting unit 23 extracts related items R4 to R11. And the operation screen process part 24 produces | generates the operation screen for displaying the icon image corresponding to each of eight extracted related items (SS3).

  According to the above-described configuration and method of the present invention, the tablet terminal 100 causes the user to perform an operation of “enclosing” (selecting) a “blank area” having no object or the like. A function for calling a display object (here, a menu list such as an icon) is assigned. More specifically, in the tablet terminal 100, in order to specify or select something, such as “enclose” an area, there is no natural object, such as a simple user contact operation, and no object. The “margin area” is linked. Then, a function of calling a multi-faceted, general-purpose, general, or attribute-free display object (menu) that does not depend on a specific object is assigned to the contact operation associated with the blank area.

  As described above, from the active action of “selecting a blank area where no object is displayed”, it can be easily described as “multiple, general, general, or independent of a specific object, or It is associated with the desire for an “attributeless display object”.

  That is, it is possible to cause the user to intentionally and actively input information such as “no object is selected” or “desired to be a display object independent of a specific object” to the tablet terminal 100. Thereby, a user's intention can be perceived more accurately.

  In addition, the operation of selecting a specific region represented by “enclose” is a general operation performed when selecting something, does not include special gestures, and learns the operation from the user. There will be no hardship for you. The user only needs to select an area where no object is displayed (open).

  This allows the tablet terminal 100 to execute a series of processes including user operation and display of the final result accompanying the user operation with a simple contact operation, a small number of operations, and an intuitive and natural contact operation. become. As a result, the tablet terminal 100 including the touch panel has an effect that it is possible to realize excellent operability.

<< Embodiment 2 >>
Another embodiment of the information processing apparatus according to the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as those in the drawings described in the first embodiment are denoted by the same reference numerals, and description of the same contents as those in the first embodiment is omitted.

  In the above-described first embodiment, the tablet terminal 100 is uniform when a blank area is selected without considering what screen the tablet terminal 100 is displaying (screen display state). It was a configuration that extracted specific related items. The tablet terminal 100 according to the present embodiment is configured to vary the related items to be extracted depending on what screen display state the tablet terminal 100 is in and the blank area selected.

[Tablet function]
FIG. 11 is a functional block diagram illustrating a main configuration of the tablet terminal 100 according to the present embodiment.

  Compared with the tablet terminal 100 (FIG. 1) of the first embodiment, the tablet terminal 100 according to the present embodiment further includes a state determination unit 29 as a functional block. The storage unit 19 further includes a state storage unit 47.

  The state determination unit 29 monitors the state of the tablet terminal 100 and determines the current state of the tablet terminal 100. In particular, in the present embodiment, the state determination unit 29 monitors the screen display state of the tablet terminal 100. The current screen display state of the tablet terminal 100 determined by the state determination unit 29 is stored in the state storage unit 47.

  The screen display state is information indicating what screen the tablet terminal 100 is displaying on the display unit 12.

  For example, as shown in FIG. 5A, when a home screen (a so-called standby screen) that is displayed first when the tablet terminal 100 is turned on is displayed, the state determination unit 29 The screen display state of the tablet terminal 100 is determined to be the “home screen”, and the determination result, that is, “screen display state; home screen” is stored in the state storage unit 47. The screen display state stored in the state storage unit 47 is preferably maintained by the state determination unit 29 so as to always indicate the latest information following the screen transition of the display unit 12.

  FIG. 12 is a diagram illustrating an example of related information stored in the related information storage unit 42 in the present embodiment. FIG. 12 shows only examples of related items associated with “no object”. Similar to the related information shown in FIG. 6, in this embodiment, the relationship between “object” and related items is also shown. May be further stored.

  The difference between the related information of the present embodiment shown in FIG. 12 and the related information of the first embodiment shown in FIG. 6 is that related items are associated with each “screen display state” in addition to “object”. Is a point.

  In the first embodiment, “no object” is associated with each of the related items R1 to R3 regardless of the screen display state. In this embodiment, “no object” is further associated with each screen display state. Different related item groups are associated with each other.

  The related item extraction unit 23 identifies and extracts related items based on the current screen display state of the tablet terminal 100 determined by the state determination unit 29. For example, it is assumed that the tablet terminal 100 is displaying a home screen as shown in FIG. 5A, and a blank area is selected at this time. At this time, “screen display state; home screen” is stored in the state storage unit 47. The related item extracting unit 23 extracts related item groups R1 to R5 associated with “screen display state; home screen” among related items associated with “no object”.

  FIG. 13 is a diagram illustrating a specific example of the operation screen obtained as a result of the operation screen generation process executed by the operation screen processing unit 24 of the present embodiment. As in the above example, when a blank area is selected on the home screen shown in FIG. 5A, the related item extraction unit 23 extracts related item groups R1 to R5 from the related information shown in FIG. . In the present embodiment, the related item extraction unit 23 may further specify icon identification information associated with each extracted related item, and supply this to the operation screen processing unit 24. In the example shown in FIG. 12, the icon identification information of “23: calculator”, “24: calendar”, “25: communication connection setting”, “17: telephone”, and “26: camera” is displayed on the operation screen processing unit. 24.

  Based on the icon identification information extracted by the related item extraction unit 23, the operation screen processing unit 24 is necessary from each icon image (for example, FIG. 7A and FIG. 7B) stored in the icon storage unit 43. Read icon image. Then, the operation screen processing unit 24 arranges the read icons according to a predetermined rule. For example, as described above, the predetermined rule is a rule that icons are arranged at regular intervals in the extracted order on an elliptical outline defined in the image display area. This rule is an example and is not intended to limit the present invention. The operation screen generated by the operation screen processing unit 24 is output to the display unit 12 and displayed so as to be visible and selectable by the user as shown in FIG. In the example illustrated in FIG. 13, icons “23: calculator”, “24: calendar”, “25: communication connection setting”, “17: telephone”, and “26: camera” are displayed on the display unit 12. ing.

  On the other hand, an example when the tablet terminal 100 is in another screen display state will be described as follows.

  (A) of FIG. 14 shows that when the tablet terminal 100 is in the screen display state displaying the “photo application screen”, the user performs the “enclose” contact operation with the intention of selecting the blank area. It is a figure which shows that. FIG. 14B is a diagram showing the map information of the video frame displayed on the display unit 12 during the period in which contact is detected, and the selection area specified by the “enclose” contact operation.

  As shown in FIG. 14A, while the tablet terminal 100 is activating the photo application, the screen display state of the tablet terminal 100 is in a “photo application screen” (displaying) state. In such a case, the state determination unit 29 stores the determination result “screen display state; photo application screen” in the state storage unit 47.

  Here, it is assumed that a blank area is selected by the user as shown by a broken line in FIG. First, the margin selection determination unit 28 specifies a selection area based on the locus indicated by the broken line. For example, as shown in FIG. 14B, the margin selection determination unit 28 can specify the inner area (thick frame hatched area) surrounded by the locus as the selection area. Note that the selection area may be specified by a circumscribed rectangle of the trajectory.

  Based on the presence / absence of an object region that overlaps the selected region, the margin selection determination unit 28 determines whether the selected region is a blank region or an object (region). In the example shown in FIG. 14B, since there is no object area overlapping the selection area, the margin selection determination unit 28 determines that the margin area has been selected here. Even when the selection area overlaps the object area, if the ratio of the overlapping area in the selection area is less than the predetermined threshold%, the margin selection determination unit 28 determines whether the selection area and the object area are It may be determined that the objects do not overlap (that is, the object is not selected by the selection area).

  Subsequently, the related item extraction unit 23 is associated with “no object” and “screen display state; photo application screen” based on “screen display state; photo application screen” stored in the state storage unit 47. Extract related items. In the example illustrated in FIG. 12, the related item extraction unit 23 extracts a related item group of R6 to R9. In the present embodiment, the related item extracting unit 23 further supplies icon identification information associated with each of the related items R6 to R9 to the operation screen processing unit 24.

  Based on the supplied icon identification information, the operation screen processing unit 24 stores the icon images of “10: album edit”, “27: sort”, “28: home”, and “26: camera” as the icon storage unit 43. The operation screen is generated by arranging each icon in the image display area.

  FIG. 15 is a diagram showing an icon displayable area (image display area) and an icon arrangement example on the photo application screen shown in FIG. For example, the operation screen processing unit 24 defines an area in which no object is displayed on the photo application screen as an image display area as shown in the white background area in FIG. Define. Then, the operation screen processing unit 24 arranges the read icon images at equal intervals on the elliptical outline.

  FIG. 16 is a diagram illustrating a specific example of the operation screen obtained as a result of the operation screen generation process executed by the operation screen processing unit 24 of the present embodiment. The operation screen illustrated in FIG. 16 is an example of an operation screen that is displayed when a blank area is selected when the tablet terminal 100 displays the photo application screen. When the photo application screen is displayed, items more closely related to “photo” are extracted as related items than when the home screen is displayed. Therefore, when a blank area is selected when the photo application screen is displayed, icons of related items related to the photo are displayed. For example, as shown in FIG. 16, an icon “10: Edit album” for creating a new album, an icon “27: Sort” for sorting photos and albums in order of date, and an icon “ 26: Camera ”is displayed. Furthermore, when the screen is in a state other than the home screen, it is preferable that the icon “28: house” for returning to the home screen immediately is displayed because it is convenient for the user.

[Operation screen display flow]
Next, a processing flow when the tablet terminal 100 executes the operation screen display function will be described. FIG. 17 is a flowchart showing a flow of operation screen display processing by the tablet terminal 100.

  When the input unit 11 detects that an indicator (such as a user's finger) has touched the touch surface of the touch panel (YES in S201), the contact information generation unit 21 starts the finger from that time (t = t0). The acquisition of the contact coordinate information indicating the contact position is started and acquired over time (S202). This tracking of the contact position is continued until no contact is detected between the touch surface and the finger (NO in S203). When contact is not detected in the input unit 11 (YES in S203), the contact information generation unit 21 generates contact information including contact coordinate information acquired from t = t0 to this time (t = tn). (S204). In the present embodiment, the contact time information may or may not be included in the contact information.

  The margin selection determination unit 28 uses the contact information generated in S204 (for example, the locus shown in FIG. 5B) and the map information stored in the frame map storage unit 41 (for example, FIG. 5C). ) To determine whether the user has enclosed a margin (a margin region has been selected) or an object region has been enclosed (some object has been selected) (S205).

  In the example shown in (c) of FIG. 5 or (b) of FIG. 14, the selection area (the hatched area in (c) of FIG. 5 or (b) of FIG. 14) identified by the locus has an object area. Are substantially non-overlapping, the margin selection determination unit 28 determines that a margin region has been selected (A in S205).

  In this case, the related item extraction unit 23 specifies the current screen display state of the tablet terminal 100 (S206). The screen display state obtained here may be determined by the state determination unit 29 in S206, or the determination result stored in the state storage unit 47 by the state determination unit 29 in advance is used as the related item extraction unit. 23 may be referred to.

  When the screen display state of the tablet terminal 100 is identified (YES in S207), the related item extraction unit 23 is associated with the identified screen display state among the related items associated with “no object”. Related items are extracted (S208). For example, when the screen display state is specified as “home screen”, the related items R1 to R5 associated with the “home screen” are extracted from the related information illustrated in FIG. The related item extraction unit 23 supplies the operation screen processing unit 24 with information on the extracted related items or icon identification information associated with the extracted related items. In S207, when other screens not defined in advance are displayed or the determination result is not stored in the state storage unit 47 and the screen display state cannot be specified (NO in S207). ), The related item extraction unit 23 extracts the related items of R17 to R19 associated with “unknown or other screen” regardless of the screen display state (S209). Alternatively, the related item extraction unit 23 may extract all the related items associated with “no object”.

  The operation screen processing unit 24 generates an operation screen for displaying the extracted icons of related items (S210). The operation screen is displayed on the display unit 12 (for example, FIG. 13 or FIG. 16).

  On the other hand, in S205, when most of the selection area specified by the locus (the hatched area in FIG. 5C or FIG. 14B) overlaps the object area, the margin selection determination unit 28 Determines that some object has been selected (B in S205).

  In this case, each unit responsible for the downstream process of the margin selection determination unit 28 executes processing when an object is selected (S211). Note that an example of the processing is as described based on FIG. 10, and therefore description thereof will not be repeated here.

  According to the above-described configuration and method, the tablet terminal 100 according to the present embodiment considers what screen the tablet terminal 100 is displaying (screen display state), and related item icons that are optimal for the scene. Can be presented to the user. Therefore, by letting the user select the margin, the user intentionally wants “a multi-faceted, general-purpose, general, or attribute-free display object that does not depend on a specific object”. The tablet terminal 100 can be actively input, and a menu list that does not depend on a specific object can be presented to the user. In addition, considering the screen display state of the tablet terminal 100 when the margin is selected, an optimal menu list corresponding to the scene can be presented to the user, so that the user's intention is reflected more accurately. Things can be provided to the user.

<< Embodiment 3 >>
Another embodiment of the information processing apparatus according to the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as those in the drawings described in the above-described embodiments are denoted by the same reference numerals, and description of contents overlapping with the above-described embodiments is omitted.

  In the present embodiment, when a blank area is selected, the tablet terminal 100 is configured to vary the related items to be extracted depending on the position of the selected blank area on the screen.

[Tablet function]
The principal part structure of the tablet terminal 100 in this embodiment can be represented by the functional block diagram shown in FIG. 1 or FIG.

  The tablet terminal 100 according to the present embodiment is different from the tablet terminal 100 according to the above-described embodiments in the following points.

  1stly, the map information memorize | stored in the frame map memory | storage part 41 of this embodiment differs in the point further including the division information which divided | segmented the screen of the touchscreen into several blocks (division area | regions). .

  Second, in the related information stored in the related information storage unit 42 of the present embodiment, the related item associated with “no object” is further divided into “division” on the touch panel screen of the selected blank area. Each area is different in that it is related.

(About division information of map information)
FIG. 18 is a diagram illustrating a specific example of map information including division information. The division information shown in FIG. 18 is an example, and the present invention is not limited to this. The touch panel may be divided into less than four divided areas, or may be divided into more divided areas. The shape of each divided region is not limited to a quadrangle, and may have any shape, and each divided region may have a different shape.

  FIG. 18 shows an example of division information when the touch panel screen is equally divided into two parts vertically and horizontally and divided into a total of four divided areas for the sake of simplicity.

  In the present embodiment, identification information of “upper left”, “upper right”, “lower left”, and “lower right” is assigned to each of the four divided regions in correspondence with the actual positions, and the tablet terminal 100 can identify them. It can be done.

  The margin selection determination unit 28 according to the present embodiment compares the selection area specified by the locus with the map information and determines whether or not a margin is selected. In addition to this, the margin selection determination unit 28 determines which divided region defined by the division information includes the selected margin region (selected region).

  FIG. 19A is a diagram showing that the user performs a “surrounding” contact operation with the intention of selecting a blank area. FIG. 19B is a diagram illustrating map information for a period in which contact is detected, including division information, and a selection area specified by the “enclose” contact operation.

  Here, it is assumed that a blank area is selected by the user as shown by a broken line in FIG. First, the margin selection determination unit 28 specifies a selection area based on the locus indicated by the broken line. For example, as shown in FIG. 19B, the margin selection determination unit 28 can specify the inner area (thick frame hatched area) surrounded by the locus as the selection area.

  Then, as described above, the margin selection determination unit 28 determines whether a margin area is selected or an object is selected. That is, it is determined whether the selected area belongs to a blank area or a non-blank area. In the present embodiment, when the margin selection determination unit 28 determines that a margin region has been selected (the selection region is a margin region), based on the position of the selection region, which division of the selection region It is determined whether it is included in the area. Specifically, the margin selection determination unit 28 determines to which divided area the four selected areas defined in FIG. 18 belong.

  In addition, when the selection area is overlapped over a plurality of divided areas, all the overlapping divided areas may be specified as belonging divided areas, or the divided area belonging to the divided area having the widest overlap with the selected area may be specified. It may be specified as a region. In the present embodiment, as shown in FIG. 19B, since the selected region overlaps the “upper left” divided region most widely, the margin selection determination unit 28 determines that the selected region is the “upper left” divided region. Is determined to be included.

  The margin selection determination unit 28 transmits the content that “the margin region is selected” and the content that “the selection region belongs to the“ upper left ”divided region” to the related item extraction unit 23 as a determination result. To do.

(About the division area of related information)
FIG. 20 is a diagram illustrating an example of related information stored in the related information storage unit 42 in the present embodiment. FIG. 20 shows only examples of related items associated with “no object”. Similar to the related information shown in FIG. 6, in this embodiment, the relationship between “object” and related items is also shown. May be further stored.

  The difference between the related information of this embodiment shown in FIG. 20 and the related information of each of the above-described embodiments shown in FIG. 6 or 12 is that the related item is classified by “divided area” in addition to “object”. It is a point that is associated.

  In the present embodiment, as shown in FIG. 20, “no object” is further associated with a different related item group for each divided region. In the example shown in FIG. 18, the touch panel screen is defined by being divided into four divided areas of “upper left”, “upper right”, “lower left”, and “lower right”. Related items are related.

  When it is determined that the margin area is selected by the margin selection determination unit 28, the related item extraction unit 23 determines the related item based on the divided area including the selection area determined by the margin selection determination unit 28. To extract. For example, as illustrated in FIG. 19B, when the margin selection determination unit 28 determines that the divided region to which the selected region belongs is “upper left”, the related item extraction unit 23 selects “no object” “division” Related items R7 to R9 associated with “region; upper left” are extracted. The related item extraction unit 23 may further extract icon identification information associated with these related items, as in the other embodiments. Here, the related item extraction unit 23 extracts three pieces of icon identification information “application 7 icon”, “application 8 icon”, and “application 9 icon”, and supplies them to the operation screen processing unit 24.

  FIG. 21 is a diagram illustrating a specific example of the operation screen obtained as a result of the operation screen generation process executed by the operation screen processing unit 24 of the present embodiment. The operation screen shown in FIG. 21 is an example of a display screen that is displayed when a blank area is selected in the “upper left” divided area of the touch panel screen. In another example, for example, when a blank area is selected in “lower right” of the divided area, the icons of the applications 4 to 6 are displayed on the display unit 12 according to the related information example illustrated in FIG. 20. Will be.

  Although the application is “multi-faceted, general-purpose, general, or attribute-free that does not depend on a specific object”, various multifunctional tablet terminals 100 are equipped with various applications. In other words, there are as many “general object-independent” icons to be displayed as there are as many applications as there are.

  Countless application icons include icons that users select frequently, favorite icons, icons used at work, icons used when playing, icons used when using both hands at home, when going out and moving There will be various groupings for each user, such as icons used when using with one hand.

  On the other hand, the screen position where the contact operation is frequently performed should be biased for each user and for each use scene. This is because, as shown in FIGS. 4A and 4B, some users use one hand, some use both hands, some right-handed users, and some left-handed users. For example, a person who frequently operates the tablet terminal 100 with the right hand (one hand) should concentrate the contact operation on the screen position “lower right” shown in FIG.

  Therefore, according to the present embodiment, for example, the related information data structure shown in FIG. 20 is used in the “screen position; lower right” where the contact operation occurs most frequently. It is possible to have a structure in which some of the top (for example, icons of applications 4 to 6 in FIG. 20) are associated with each other.

  Therefore, according to the tablet terminal 100 according to the present embodiment, when the user selects a blank area at the screen position “lower right” that is most touched, the icon of the application most frequently used by the user is displayed. (The icons of the applications 4 to 6) can be presented to the user in a selectable manner.

  From the above, by letting the user select the margin, the user wants a “multi-faceted, general-purpose, general, or attribute-free display object that does not depend on a specific object”. It can be intentionally actively input to the tablet terminal 100, and a menu list that does not depend on a specific object can be presented to the user. In addition, it is possible to estimate the use scene in consideration of the screen position of the margin area when the margin is selected, and to present the optimal menu list to the user in conjunction with the estimated scene. As a result, it is possible to provide the user with a result reflecting the above more accurately.

<< Embodiment 4 >>
Another embodiment of the information processing apparatus according to the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as those in the drawings described in the above-described embodiments are denoted by the same reference numerals, and description of contents overlapping with the above-described embodiments is omitted.

  In the present embodiment, when a blank area is selected, the tablet terminal 100 is configured to vary the related items to be extracted according to the area of the selected blank area.

[Tablet function]
The principal part structure of the tablet terminal 100 in this embodiment can be represented by the functional block diagram shown in FIG. 1 or FIG.

  The tablet terminal 100 according to the present embodiment is different from the tablet terminal 100 according to the above-described embodiments in the following points.

  In the related information stored in the related information storage unit 42 of the present embodiment, the related item associated with “no object” is further the size of the area of the selected blank area, that is, “margin area”. It differs in that it is associated separately.

  The margin selection determination unit 28 according to the present embodiment compares the selection area specified by the locus with the map information and determines whether or not a margin is selected. In addition to this, in the present embodiment, the margin selection determination unit 28 determines whether the area of the selected margin area is classified as “large”, “medium”, or “small”. . The area is not limited to the above-described three categories, and may be classified into any number of categories.

  FIG. 22 is a diagram illustrating some examples of trajectories when the user performs “enclose” contact operations of various sizes in order to select a blank area.

  For example, it is assumed that the user moves the finger along the broken line 97a to perform the contact operation. The margin selection determination unit 28 specifies the selection region based on the locus obtained from the broken line 97a, and determines that the margin region is selected by comparing with the map information ((c) of FIG. 5). Then, based on the size of the area of the selected region, the blank area of the broken line 97a is classified into one of “large”, “medium”, and “small”.

  In the present embodiment, the condition and threshold value for determining that the margin area is “large” is set in advance as “the ratio of the selected area in the entire touch panel screen is not less than m%”. For example, when the ratio of the occupied area of the selected area specified from the locus of the broken line 97a on the entire touch panel screen is equal to or greater than m%, the margin selection determining unit 28 determines that the margin area of the selected area is “large”. .

  In the present embodiment, the condition and threshold value for determining the margin area as “medium” is set in advance as “the ratio is n% or more and less than m%”. For example, when the ratio of the area occupied on the entire touch panel screen of the selection area specified from the locus of the broken line 97b is n% or more and less than m%, the margin selection determination unit 28 sets the margin area of the selection area to “medium”. Is determined.

  In the present embodiment, the condition and threshold value for determining that the margin area is “small” is set in advance as “the ratio is less than n%”. For example, when the ratio of the area occupied on the entire touch panel screen of the selection area specified from the locus of the broken line 97c is less than n%, the margin selection determination unit 28 determines that the margin area of the selection area is “small”. .

  The margin selection determination unit 28 associates the content that “the margin region is selected” and the content that “the area of the selected region is“ (large, medium, or small) ”” as a determination result. This is transmitted to the item extraction unit 23.

  FIG. 23 is a diagram illustrating an example of related information stored in the related information storage unit 42 in the present embodiment. FIG. 23 shows only examples of related items associated with “no object”. Similar to the related information shown in FIG. 6, in this embodiment, the relationship between “object” and related items is also shown. May be further stored.

  The difference between the related information of this embodiment shown in FIG. 23 and the related information of each of the above-described embodiments shown in FIG. 6, FIG. 12 or FIG. This is a point associated with each “margin area”.

  In the present embodiment, as shown in FIG. 23, “no object” is further associated with a different related item group for each blank area. In the example shown in FIG. 23, the size of the margin area is classified into three types of “large”, “medium”, and “small”, and different related item groups are associated with each size. .

  When it is determined that the margin area is selected by the margin selection determination unit 28, the related item extraction unit 23 selects the related item based on the area (margin area) of the margin area determined by the margin selection determination unit 28. Extract.

  For example, when a contact operation indicated by a broken line 97a shown in FIG. 22 occurs, the margin selection determination unit 28 determines that the margin area is “large”. Therefore, the related item extraction unit 23 extracts the related items R1 to R3 associated with “margin area; large” of “no object”. The related item extraction unit 23 may further extract icon identification information associated with these related items, as in the other embodiments. Here, the related item extraction unit 23 extracts three pieces of icon identification information “application 1 icon”, “application 2 icon”, and “application 3 icon”, and supplies them to the operation screen processing unit 24.

  FIG. 24A is a diagram illustrating a specific example of the operation screen created by the operation screen processing unit 24 of the present embodiment when a contact operation with a predetermined size occurs.

  FIG. 24B is a specific example of the operation screen created by the operation screen processing unit 24 of the present embodiment when a contact operation having a predetermined size different from the case of FIG. FIG.

  In the present embodiment, FIG. 21 is created by the operation screen processing unit 24 of the present embodiment when a contact operation having a predetermined size different from that in FIGS. 24A and 24B occurs. It is a figure which shows the specific example of an operation screen.

  The operation screen shown in FIG. 24A is an example of a display screen that is displayed when a blank area surrounded by a broken line 97a in FIG. 22 is selected. The area of the margin area selected by the broken line 97a is determined as “large” by the margin selection determination unit 28 as described above. Therefore, based on the related information shown in FIG. 23, the “related item icon”, “the icon of application 2”, and “the icon of application 3” are extracted by the related item extraction unit 23. The operation screen processing unit 24 arranges these extracted icons and displays them on the display unit 12 as shown in FIG.

  The operation screen shown in FIG. 24B is an example of a display screen that is displayed when a blank area indicated by a broken line 97b in FIG. 22 is selected. The area of the margin area selected by the broken line 97b is determined as “medium” by the margin selection determination unit 28 as described above. Therefore, based on the related information shown in FIG. 23, the related item extraction unit 23 extracts “the icon of application 4”, “the icon of application 5”, and “the icon of application 6”. The operation screen processing unit 24 arranges these extracted icons and displays them on the display unit 12 as shown in FIG.

  The operation screen shown in FIG. 21 is an example of a display screen that is displayed when a blank area indicated by a broken line 97c in FIG. 22 is selected. The area of the margin area selected by the broken line 97c is determined to be “small” by the margin selection determination unit 28 as described above. Therefore, based on the related information shown in FIG. 23, the related item extracting unit 23 extracts “icon of application 7”, “icon of application 8”, and “icon of application 9”. As shown in FIG. 21, the operation screen processing unit 24 arranges these extracted icons and displays them on the display unit 12.

  Although the application is “multi-faceted, general-purpose, general, or attribute-free that does not depend on a specific object”, various multifunctional tablet terminals 100 are equipped with various applications. In other words, there are as many “general object-independent” icons to be displayed as there are as many applications as there are.

  Countless application icons include icons that users select frequently, favorite icons, icons used at work, icons used when playing, icons used when using both hands at home, when going out and moving There will be various groupings for each user, such as icons used when using with one hand.

  On the other hand, it is considered that the ease of performing a large contact operation is different from the ease of performing a small contact operation. For example, in the case of the “enclose” contact operation, a smaller circle is easier to draw. It may be difficult to draw a large circle that fills the screen without using both hands.

  According to the present embodiment, for example, in the related information data structure shown in FIG. 23, the “margin area; Icon).

  Therefore, according to the tablet terminal 100 according to the present embodiment, when the margin area is selected by the user at the size of the contact operation that is most easily performed (that is, the margin area “small”), It is possible to present to the user selectable icons of applications that are most frequently used by the user.

  Conversely, when a large circle is drawn, the user intentionally wants to display a “general object-independent” display object because the user intentionally performed a relatively difficult contact motion. It can be estimated to be strong.

  According to the present embodiment, for example, in the data structure of the related information shown in FIG. 23, when the user is in a specific case (at the time of being at home, It is possible to have a structure in which related items (icons) that are frequently used when moving) are associated.

  Therefore, according to the tablet terminal 100 according to the present embodiment, when the user selects a blank area at the size of the contact operation that is most difficult to perform (that is, the blank area “large”), The icon of the application used in a special case can be presented to the user in a selectable manner.

  As described above, by letting the user select a margin, the user intends to desire “a multi-faceted, general-purpose, general, or attribute-free display object that does not depend on a specific object”. The tablet terminal 100 can be actively input, and a menu list that does not depend on a specific object can be presented to the user. In addition, the use scene can be estimated in consideration of the area of the margin area when the margin is selected, and an optimal menu list can be presented to the user in conjunction with the estimated scene. It becomes possible to provide the user with a result reflected more accurately.

<< Embodiment 5 >>
Another embodiment of the information processing apparatus according to the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as those in the drawings described in the above-described embodiments are denoted by the same reference numerals, and description of contents overlapping with the above-described embodiments is omitted.

  In each of the above-described embodiments, the tablet terminal 100 uniformly determines a predetermined area without considering the contact operation (the position or the size of the trajectory) when the blank area is selected. It was the composition which arranges and displays the icon by the method. The tablet terminal 100 according to the present embodiment is configured to vary the icon arrangement method according to the position and / or size of the trajectory.

[Tablet function]
FIG. 25 is a functional block diagram illustrating a main configuration of the tablet terminal 100 according to the present embodiment.

  Compared with the tablet terminal 100 (FIGS. 1 and 11) of each embodiment, the tablet terminal 100 according to the present embodiment further includes an operation screen processing unit 24 of the control unit 10 as a functional block as a ring shape determining unit. 30. The storage unit 19 includes a contact information storage unit 44.

  The contact information storage unit 44 stores the contact information generated by the contact information generation unit 21. In each of the above-described embodiments, the contact information is temporarily stored in a storage unit (such as a cache) (not shown) so that the margin selection determination unit 28 or the object specifying unit 22 can be used. In the present embodiment, the contact information is used so that each part of the operation screen processing unit 24 and the operation screen processing unit 24 can perform operation screen generation processing (including processing for displaying icons). It is stored in the storage unit 44. Whether or not the contact information storage unit 44 is realized by a non-volatile storage device, that is, whether or not the contact information is stored in a nonvolatile manner depends on the purpose of the operation screen display function executed by the operation screen processing unit 24 and the assumed use. It is determined as appropriate from the environment, the purpose of use of the tablet terminal 100 itself, convenience, cost, physical restrictions, and the like.

  The ring shape determination unit 30 determines the shape of the ring when the operation screen processing unit 24 places an icon on the outline of the ring. In each of the above-described embodiments, the operation screen processing unit 24 is configured to arrange icons along a predetermined ring shape. For example, the icon is arranged on an elliptical outline of a predetermined position and a predetermined size. In the present embodiment, the ring shape determination unit 30 determines the shape of the ring for arranging the icons based on the contact information stored in the contact information storage unit 44. That is, the ring shape determination unit 30 determines the ring shape based on the finger trajectory obtained from the user's contact operation (for example, “enclose”).

  More specifically, the ring shape determining unit 30 determines the size of the ring for arranging icons in proportion to the size of the trajectory (in this embodiment, a substantially circular trajectory obtained from the contact operation “enclose”). To decide. Furthermore, it is preferable that the ring shape determination unit 30 determines the position of the ring according to the screen position of the touch panel on which the locus is drawn.

  Alternatively, as a procedure for determining the size of the ring, the ring shape determining unit 30 first determines the size of the image display area based on the size of the trajectory, and the size of the ring so as to fit in the determined image display area. You may decide. Further, the ring shape determination unit 30 determines the position of the image display area based on the object area of the object already displayed and the position of the trajectory, and determines the position of the ring so as to fit in the image display area. May be.

  In accordance with the ring shape information (ring size and position information) determined by the ring shape determining unit 30, the operation screen processing unit 24 arranges the extracted icons. The ring shape information determined by the ring shape determination unit 30 may further include ring position information in addition to the ring size information. That is, the operation screen processing unit 24 can place a ring at a determined position with a determined size, and can place an icon on the outline of the ring.

  Further, in the present embodiment, the margin selection determination unit 28 is selected by the locus in addition to determining whether or not the margin is selected by the locus, similarly to the margin selection determination unit 28 of the fourth embodiment. Classify the size of the area of the margin area. Also in the present embodiment, the margin selection determination unit 28 determines whether the size of the selected margin area is classified as “large”, “medium”, or “small”.

  Further, in the related information stored in the related information storage unit 42 of the present embodiment, each of the related items is associated with a “condition” for displaying the icon of the related item. This is different from the data structure of the embodiment.

  FIG. 26 is a diagram illustrating an example of related information stored in the related information storage unit 42 in the present embodiment. FIG. 26 shows only examples of related items associated with “no object”. Similar to the related information shown in FIG. 6, in this embodiment, the relationship between “object” and related items is also shown. May be further stored.

  The difference between the related information of this embodiment shown in FIG. 26 and the related information of each embodiment described above shown in FIG. 6, FIG. 12, FIG. 20, or FIG. Are associated with each other.

  “Condition” indicates a condition for the icon of the related item to be actually displayed on the display unit 12. For example, for the icon of the related item R1, which is associated with “condition:“ large, medium, small ””, the size of the locus (area of the selected region) determined by the margin selection determination unit 28 is “large”. ", Medium, Small" is displayed. On the other hand, the icon of the related item R4, which is associated with “condition:“ when large ””, is displayed only when the size of the trajectory determined by the margin selection determination unit 28 is “large”. In other words, “condition; when“ large ”” means that the icon of the related item R4 is not displayed on the display unit 12 when the size of the locus is “medium” or “small”. .

  The related item extraction unit 23 relates to the related item (or icon identification information) to be supplied to the operation screen processing unit 24 according to the size of the trajectory determined by the margin selection determination unit 28 and the condition associated with each related item. ).

  That is, according to the above-described data structure of the related information, even in the same event that “margin area is selected”, the icon actually displayed on the display unit 12 is proportional to the size of the trajectory (margin area). The number can be adjusted.

  In the example illustrated in FIG. 26, when a blank area is selected and the selected blank area is determined to be “large”, the related item extraction unit 23 is associated with “no object”. Extract icons of all related items (five icons R1 to R5). Alternatively, when the margin area is determined to be “medium”, the related item extraction unit 23 extracts three icons of the related items R1 to R3. Alternatively, when the margin area is determined to be “small”, the related item extraction unit 23 extracts two icons of the related items R1 and R2.

  As a result, when the size of the trajectory drawn by the user is “large”, the operation screen processing unit 24 arranges and displays five icons on a ring having a size proportional to the trajectory. When the size is “medium”, three icons are arranged and displayed on a ring whose size is proportional to the trajectory, and when the size of the trajectory is “small”, it is proportional to the trajectory. It becomes possible to arrange and display two icons on the ring of size.

  FIG. 22 is a diagram illustrating an example of a trajectory when the user performs “enclose” contact operations of various sizes with the intention of selecting a blank area.

  FIG. 27 is a diagram illustrating a specific example of the operation screen created by the operation screen processing unit 24 of the present embodiment when a contact operation with a predetermined size occurs.

  FIG. 28 is a diagram illustrating a specific example of the operation screen created by the operation screen processing unit 24 of the present embodiment when a contact operation having a predetermined size different from that in FIG. 27 occurs.

  FIG. 29 is a diagram illustrating a specific example of the operation screen created by the operation screen processing unit 24 of the present embodiment when a contact operation having a predetermined size different from that in FIGS. 27 and 28 occurs. .

  The operation screen shown in FIG. 27 is an example of a display screen that is displayed when a blank area surrounded by a broken line 97a in FIG. 22 is selected. The area of the margin area selected by the broken line 97a is determined as “large” by the margin selection determination unit 28 as described above. Therefore, the related item extracting unit 23 supplies the five icon identification information of the applications 1 to 5 to the operation screen processing unit 24 based on the related information shown in FIG.

  As shown in FIG. 27, the ring shape determining unit 30 of the operation screen processing unit 24 determines the size of the ring and the position of the ring based on the contact information obtained from the broken line 97a (ring 98a in FIG. 27). The operation screen processing unit 24 arranges the five icons of the applications 1 to 5 on the outline of the ring 98 a determined by the ring shape determination unit 30 and displays them on the display unit 12.

  The operation screen shown in FIG. 28 is an example of a display screen displayed when a blank area surrounded by a broken line 97b in FIG. 22 is selected. The area of the margin area selected by the broken line 97b is determined as “medium” by the margin selection determination unit 28 as described above. Therefore, the related item extraction unit 23 supplies the three icon identification information of the applications 1 to 3 to the operation screen processing unit 24 based on the related information shown in FIG.

  As shown in FIG. 28, the ring shape determining unit 30 of the operation screen processing unit 24 determines the size of the ring and the position of the ring based on the contact information obtained from the broken line 97b (ring 98b in FIG. 28). The operation screen processing unit 24 arranges the three icons 1 to 3 on the outline of the ring 98 b determined by the ring shape determination unit 30 and displays the icons on the display unit 12.

  The operation screen shown in FIG. 29 is an example of a display screen displayed when a blank area surrounded by a broken line 97c in FIG. 22 is selected. The area of the margin area selected by the broken line 97c is determined to be “small” by the margin selection determination unit 28 as described above. Therefore, the related item extracting unit 23 supplies the two icon identification information of the applications 1 and 2 to the operation screen processing unit 24 based on the related information shown in FIG.

  As shown in FIG. 29, the ring shape determination unit 30 of the operation screen processing unit 24 determines the size of the ring and the position of the ring based on the contact information obtained from the broken line 97c (ring 98c in FIG. 29). The operation screen processing unit 24 arranges two icons of the applications 1 and 2 on the outline of the ring 98 c determined by the ring shape determination unit 30 and displays them on the display unit 12.

  According to the said structure, according to the contact operation | movement of selecting a user's margin, the display target object (here icon list) which is associated with a margin and does not depend on a specific object can be shown to a user. Furthermore, the tablet terminal 100 according to the present embodiment determines the size and position of the ring on which the icon is arranged according to the size and position of the trajectory of the contact operation by the user.

  That is, the user can influence the range and position of the display object (ie, the size of the ring of the icon list) by one contact operation (for example, “enclose”) for calling the target display object (icon list). (Position) can also be specified.

  Conventionally, there is an operation in which a desired display object (for example, a menu list or the like) is called and an operation for designating a position for displaying the display object (for example, Patent Document 1 and Patent Document 2). . However, in the conventional technique, one operation for calling a display object (in the present invention, for example, a contact operation of “enclose”) also serves as an operation for designating a display range in addition to the display position of the display object. This configuration could not be realized. Further, in the conventional technique, a configuration in which one operation also serves as an operation for designating the number of sub-items included in the desired display object cannot be realized.

  Not only an operation for calling a display object such as a menu list, but also one operation that is a trigger for activating a specific function specifies both a position and a range in which the execution result of the function affects. A configuration that also serves as an operation is desirable. The present invention realizes this configuration that could not be realized in the past.

  As described above, according to the tablet terminal 100 of the present invention, information indicating that “no object is selected” or “desired display target object (here, a menu list) independent of a specific object” is intentionally given to the user. The tablet terminal 100 can be actively input, and by this input, the range and position where the display object is displayed can be designated without requiring a separate operation.

  The coincidence between the size of the contact operation and the size of the final product (here, the ring of icons) is an event that can be naturally understood by the user. In addition, the fact that the position where the contact operation is performed matches the position where the final result is displayed is an event that can be naturally understood by the user.

  From the above, the tablet terminal 100 is caused to execute a series of processes including user operation and display of the final result accompanying the user operation with a simple contact operation, a small number of operations, and an intuitive and natural contact operation. Is possible. As a result, the tablet terminal 100 including the touch panel has an effect that it is possible to realize excellent operability.

  In the above description, the ring shape determination unit 30 of the present embodiment has been described as having a configuration in which both the position and size of the ring are substantially matched with the position and size of the locus of the contact operation. The configuration of the ring shape determining unit 30 is the most preferred embodiment, but the ring shape determining unit 30 of the present invention is not limited to this. The ring shape determination unit 30 may be configured to determine only the position of the ring in consideration of the position of the trajectory, or may be configured to determine only the size of the ring in consideration of the size of the trajectory. May be. Such a configuration also falls within the scope of the present invention.

  In the above description, the configuration in which the related item extraction unit 23 adjusts the number of icons to be extracted according to the size of the trajectory has been described. However, when the number of icons to be extracted is constant, the operation screen processing unit 24 adjusts the size of the ring according to the size of the trajectory and arranges it according to the determined size of the ring. A good-looking icon list may be displayed by enlarging or reducing the size of each icon.

(Modification 5-1)
In the above description, in the present embodiment, the size of the ring determined by the ring shape determination unit 30 is determined according to the size of the trajectory of the contact operation in order to select the margin. However, the configuration of the tablet terminal 100 of the present invention is not limited to the above. For example, the ring shape determination unit 30 may determine the size of the ring according to the number of related items (or icon identification information) extracted by the related item extraction unit 23.

  For example, when the related item extracting unit 23 extracts six or more icons, the ring shape determining unit 30 determines the size of the ring as shown by a broken line in FIG. Alternatively, when five are extracted, the size of the ring is determined as shown by the broken line in FIG. 27, and when the related item extracting unit 23 extracts three icons, the size of the ring is determined by the broken line shown in FIG. When the related item extraction unit 23 extracts two or one icon, the size of the ring may be determined as shown by a broken line in FIG.

(Modification 5-2)
In the above description, in the present embodiment, the related item extraction unit 23 extracts a predetermined number of related items (icons) based on the “condition” set for each related item in the related information illustrated in FIG. 26. It was the composition to do. However, the configuration of the tablet terminal 100 of the present invention is not limited to the above. For example, when the related information stored in the related information storage unit 42 does not include the “condition” field, the related information storage unit 42 instead of the “margin area” and the “number of arranged icons” A correspondence table may be stored.

  FIG. 30 is a diagram showing a specific example of a correspondence table of “margin area” and “arranged icon count” stored in the related information storage unit 42.

  As shown in FIG. 30, the correspondence table corresponds to the “number of icons to be arranged” indicating the number of icons to be arranged in the ring by the operation screen processing unit 24 for each “margin area” classification determined by the margin selection determination unit 28. It has a data structure attached.

  According to the correspondence table shown in FIG. 30, the related item extraction unit 23 extracts five related items (or icons) when the size of the locus (that is, the blank area) is determined to be “large”. To do. Alternatively, if it is determined as “small”, the number of related items is limited to two. The related item extracting unit 23 extracts only a part of the related items in an ascending order (or descending order) of IDs (for example, “R1”) assigned to the related items. Alternatively, they may be extracted in the order of priority given separately.

  Even in such a configuration, as a result, the related item extraction unit 23 can transmit an optimal number of icons to the operation screen processing unit 24 in proportion to the size of the trajectory.

Embodiment 6
Another embodiment of the information processing apparatus according to the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as those in the drawings described in the above-described embodiments are denoted by the same reference numerals, and description of contents overlapping with the above-described embodiments is omitted.

  In each of the embodiments described above, the tablet terminal 100 has been described as having a configuration in which a function (operation screen display function) for calling a menu list (icon ring) is assigned to a contact operation for selecting a margin. In this embodiment, the contact operation for selecting a margin is further assigned a function for designating a position or size for displaying the image in a function for displaying a predetermined image on the background (hereinafter referred to as a wallpaper display function). It is the composition which is done.

[Tablet function]
FIG. 31 is a functional block diagram illustrating a main configuration of the tablet terminal 100 according to the present embodiment.

  Compared with the tablet terminal 100 of each embodiment (FIGS. 1, 11, and 25), the control unit 10 as a functional block is added to the operation screen processing unit 24 in the tablet terminal 100 according to the present embodiment. Further, the registration image processing unit 36 is included. The storage unit 19 further includes a registered image storage unit 48.

  Further, in the tablet terminal 100 of the present embodiment, the control unit 10 may include the state determination unit 29 and the storage unit 19 may include the state storage unit 47 as in the second embodiment (FIG. 11). .

  Furthermore, in the tablet terminal 100 of the present embodiment, the storage unit 19 includes the contact information storage unit 44 as in the fifth embodiment (FIG. 25).

  Further, in the present embodiment, the related information storage unit 42 has a data structure including at least “wallpaper setting application (starts)” as a related item corresponding to “no object” in the related information. .

  FIG. 32 is a diagram illustrating an example of related information stored in the related information storage unit 42 in the present embodiment. FIG. 32 shows only examples of related items associated with “no object”. Similar to the related information shown in FIG. 6, in this embodiment, the relationship between “object” and related items is also shown. May be further stored.

  As shown in FIG. 32, in the present embodiment, “no object” is associated with at least a related item R4 of “wallpaper setting application (runs)”. In the configuration including the state determination unit 29, the related item R4 may be associated with “no object” and the screen display state “home screen”.

  The “wallpaper setting application” is an application that the registered image processing unit 36 executes in order to realize the above-described wallpaper display function. Specifically, an application for realizing a function in which the registered image processing unit 36 performs appropriate image processing on a registered image designated by a user or the like and displays the image on a background such as a home screen (standby screen). It is.

  When the margin selection determination unit 28 determines that a margin area has been selected, the related item extraction unit 23 first extracts a related item associated with “no object”. According to the related information shown in FIG. 32, the related item extracting unit 23 extracts related item groups R1 to R4 including the related item R4 of “wallpaper setting application”.

  For example, when a blank area is selected (for example, when a blank area is selected when the tablet terminal 100 is displaying the home screen), the related item extraction unit 23 is shown in FIG. Related items R1 to R4 are extracted.

  The operation screen processing unit 24 reads the icon corresponding to the extracted related item from the icon storage unit 43, arranges the icon based on a predetermined rule, and displays the icon on the touch panel display unit 12, as in the above-described embodiment. indicate. Here, four icons, that is, “1: TV”, “13: Web browser application”, “3: Mail”, and “*: Wallpaper setting application icon” are read from the icon storage unit 43, Is displayed. For example, the operation screen displayed in the present embodiment is an operation screen in which “*: wallpaper setting application icon” is arranged on the outline of the ring in addition to the three icons in the operation screen shown in FIG. .

  When “*: wallpaper setting application icon” displayed on the operation screen is selected by the user, the registered image processing unit 36 activates the wallpaper setting application.

  The registered image processing unit 36 executes a wallpaper setting application, processes a registered image designated by the user, and causes the display unit 12 to display the wallpaper. In the present embodiment, first, when the user selects “*: wallpaper setting application icon” by the user, the registered image processing unit 36 reads a registered image for wallpaper display from the registered image storage unit 48.

  The registered image storage unit 48 stores a registered image registered for display as a wallpaper by the wallpaper display function of the registered image processing unit 36. FIG. 33 is a diagram illustrating an example of a registered image database stored in the registered image storage unit 48.

  As shown in FIG. 33, one or more registered images used for wallpaper are stored in the database in a state where registered image identification information is given.

  In the present embodiment, when a plurality of registered images are stored in the registered image storage unit 48, the registered image processing unit 36 displays a list of registered images so that the user can select a registered image to be displayed on the wallpaper. , Display to the user as selectable.

  For example, when the registered image “xxxx.jpg” is selected by the user, the registered image processing unit 36 registers the registered image 9 selected by the user based on the registered image identification information “xxxx.jpg”. Read from the storage unit 48.

  In the present embodiment, when the registered image processing unit 36 displays the selected registered image 9, first, the contact information generated by the contact information generation unit 21 stored in the contact information storage unit 44 is displayed. Browse information. Then, the registered image processing unit 36 performs image processing corresponding to the locus drawn by the user on the read registered image, and arranges the registered image at a position corresponding to the locus. As a result, the registered image 9 is displayed on the display unit 12 (the wallpaper of the registered image 9 is set as the background of the home screen). A more detailed operation of the registered image processing unit 36 will be described below with reference to the drawings.

  FIG. 22 is a diagram illustrating an example of a trajectory when the user performs “enclose” contact operations of various sizes at various positions with the intention of selecting a blank area.

  FIG. 34 is a diagram illustrating a specific example of the wallpaper display screen created by the registered image processing unit 36 when a contact operation having a predetermined size occurs at a predetermined position during home screen display.

  FIG. 35A shows a specific example of the wallpaper display screen created by the registered image processing unit 36 when a contact operation with a predetermined position and size different from the case of FIG. 34 occurs when the home screen is displayed. It is a figure which shows an example.

  FIG. 35B is generated by the registered image processing unit 36 when a contact operation with a predetermined position and size different from the case of FIG. 34 and FIG. It is a figure which shows the specific example of the made wallpaper display screen.

  The wallpaper display screen shown in FIG. 34 is an example of a screen that is displayed when a blank area surrounded by a broken line 97a in FIG. 22 is selected on the home screen. The registered image processing unit 36 can specify the position and size of the selection region based on the broken line 97 a based on the contact information stored in the contact information storage unit 44. The registered image processing unit 36 compares the size of the registered image read from the registered image storage unit 48 with the size of the selected region, and determines whether the selected region is sufficiently larger than the size of the registered image. To do. Even if the selected area is smaller than the registered image, the registered image processing unit 36 provides a certain allowable range, and if the difference in the area is within the allowable range, “the selected area is sufficient for the registered image. It may be judged as “large”. Here, as an example, the registered image is registered image 9 (xxxx.jpg) in FIG. The registered image processing unit 36 compares the selected area based on the broken line 97a with the registered image 9 and determines that “the selected area is sufficiently larger than the registered image 9”.

  If the registered image processing unit 36 determines that “the selected area is sufficiently large”, the registered image is not subjected to image processing such as reduction or trimming on the registered image, and the registered image is kept at the same position as the selected area with the same magnification. You may arrange. The registered image processing unit 36 may arrange the registered image at a substantially overlapping position even when the registered image protrudes from the selection region. Alternatively, when the selected area is considerably larger than the registered image 9, the registered image processing unit 36 may perform image processing for enlarging the registered image 9.

  The registered image processing unit 36 determines the display position and display range of the registered image 9 based on the broken line 97a (FIG. 22), and displays a wallpaper display screen using the registered image 9 as wallpaper as shown in FIG. To the unit 12.

  The wallpaper display screen shown in FIG. 35A is an example of a screen that is displayed when a blank area surrounded by a broken line 97b in FIG. 22 is selected on the home screen. The registered image processing unit 36 specifies the position and size of the selected region based on the locus of the broken line 97b. The registered image processing unit 36 compares the size of the registered image 9 with the size of the selected area. Here, it is assumed that the registered image processing unit 36 determines that the selection area based on the broken line 97 b is smaller than the registered image 9.

  In this case, the registered image processing unit 36 performs appropriate image processing on the registered image 9 so that the registered image 9 substantially overlaps the selected area within the allowable range. In the example shown in FIG. 35A, the registered image processing unit 36 performs image processing for reducing the registered image 9 so as to fit the selected area. Then, the registered image processing unit 36 arranges the reduced registered image 9 at a position that substantially overlaps the selected area. As a result, as shown in (a) of FIG. 35, the registered image 9 is displayed as a wallpaper at a position and range that substantially match the locus (broken line 97b) drawn by the user.

  The wallpaper display screen shown in (b) of FIG. 35 is an example of a screen that is displayed when a blank area surrounded by a broken line 97c in FIG. 22 is selected on the home screen. The registered image processing unit 36 specifies the position and size of the selected region based on the locus of the broken line 97c. The registered image processing unit 36 compares the size of the registered image 9 with the size of the selected area. Here, it is assumed that the registered image processing unit 36 determines that the selection area based on the broken line 97 c is smaller than the registered image 9.

  In this case, the registered image processing unit 36 performs appropriate image processing on the registered image 9 so that the registered image 9 substantially overlaps the selected area within the allowable range. In the example shown in FIG. 35B, the registered image processing unit 36 performs image processing for trimming the registered image 9 so as to match the selected region. Then, the registered image processing unit 36 arranges the trimmed registered image 9 at a position that substantially overlaps the selected area. As a result, as shown in FIG. 35 (b), the trimmed registered image 9 is displayed as a wallpaper at a position and range that substantially match the locus drawn by the user (broken line 97c).

  According to the above configuration, the related item (here, the icon of the wallpaper setting application for calling the wallpaper display function) associated with the margin, which does not depend on the specific object, is associated with the contact operation of selecting the margin of the user. It can be provided to the user in a selectable manner. Furthermore, when the related item is a function for displaying some display object (such as wallpaper), the display processing unit (the registered image processing unit 36 in the above example) that executes the function of the related item is: The trajectory (that is, contact information) when the blank area is selected by the above-described contact operation can be taken over. And the said display process part determines the position and range which arrange | position a display target object according to the magnitude | size and position of the locus | trajectory of the contact operation by a user. Further, appropriate image processing can be performed on the display object according to the determined range.

  That is, the user can display a display object to be processed by the display processing unit by one contact operation (for example, “enclose”) performed to call a function of displaying some display object (for example, a wallpaper display function). It is also possible to specify the range and position that are affected (that is, the display range and position of the wallpaper image).

  In the conventional technique, one operation for calling a display object (in the present invention, for example, a touching operation of “enclose”) serves as an operation for designating a display range in addition to the display position of the display object. Could not be realized.

  Not only an operation for calling a display object such as a menu list or a wallpaper image, but also an operation that is a trigger for activating a specific function, both in a position and a range where the execution result of the function affects. A configuration that also serves as an operation for designating is desirable. The present invention realizes this configuration that could not be realized in the past.

  As described above, according to the tablet terminal 100 of the present invention, it is possible to cause the user to intentionally and actively input information “no object is selected” to the tablet terminal 100. In addition, when an instruction to “display some display object (in this case, wallpaper image)” is given, a separate operation is required for the range and position where the display object is displayed by the above input. It is possible to specify without.

  The coincidence between the size of the contact operation and the size of the final result (wallpaper image) is an event that can be naturally understood by the user. In addition, the fact that the position where the contact operation is performed matches the position where the final result is displayed is an event that can be naturally understood by the user.

  From the above, the tablet terminal 100 is caused to execute a series of processes including user operation and display of the final result accompanying the user operation with a simple contact operation, a small number of operations, and an intuitive and natural contact operation. Is possible. As a result, the tablet terminal 100 including the touch panel has an effect that it is possible to realize excellent operability.

<< Embodiment 7 >>
Another embodiment of the information processing apparatus according to the present invention will be described below with reference to FIGS. 11, 12, 36, and 37. FIG. For convenience of explanation, members having the same functions as those in the drawings described in the above-described embodiments are denoted by the same reference numerals, and description of contents overlapping with the above-described embodiments is omitted.

  In the second embodiment described above, the configuration of the tablet terminal 100 that performs icon display in consideration of the screen display state has been described. Specifically, the case where the margin is selected when the screen display state is the “home screen” and the case where the margin is selected when the screen display state is the “photo application screen” have been described.

  In the present embodiment, the tablet terminal 100 is equipped with a “schedule application” function for managing a schedule. Then, during execution of the schedule application, the tablet terminal 100 transitions to a state in which the “schedule application screen” is displayed. In the present embodiment, the operation of the tablet terminal 100 will be described for a case where a margin is selected when the screen display state is “schedule application screen”.

  In the present embodiment, the tablet terminal 100 is configured to recognize, as an object, a bar indicating that the schedule is filled in the schedule book displayed on the “schedule application screen” during execution of the schedule application. And it is the structure which recognizes the area | region where no bar is displayed, ie, the area | region where no plan is input, as a blank area.

  FIG. 36A shows that when the tablet terminal 100 is in the screen display state displaying the “schedule application screen”, the user performs an “enclose” contact operation with the intention of selecting a blank area. It is a figure which shows that. FIG. 36B is a diagram showing the map information of the video frame displayed on the display unit 12 during the period in which the contact is detected, and the selection area specified by the “enclose” contact operation.

  As shown in FIG. 36A, while the tablet terminal 100 is activating the schedule application, the screen display state of the tablet terminal 100 is a “schedule application screen” (displaying) state. In such a case, the state determination unit 29 stores the determination result “screen display state; schedule application screen” in the state storage unit 47.

  Here, it is assumed that a blank area (an area in which no schedule is filled in the schedule book) is selected by the user as indicated by a broken line in FIG. First, the margin selection determination unit 28 specifies a selection area based on the locus indicated by the broken line. For example, as shown in FIG. 36B, the margin selection determination unit 28 can specify the inner area (thick frame hatched area) surrounded by the locus as the selection area. As shown in FIG. 36 (a), if the trajectories do not intersect by themselves or the start point and the end point do not match, and the closed area cannot be specified, the blank space For example, as illustrated in FIG. 36B, the selection determination unit 28 may specify the selection region by supplementing a line (for example, a straight line) connecting the start point and the end point of the trajectory. On the other hand, when the locus is a closed curve, the margin selection determination unit 28 may specify the region surrounded by the locus as the locus region.

  Alternatively, the margin selection determining unit 28 may specify the circumscribed rectangle of the linear trajectory as a trajectory region even if the trajectory is a closed curve. The margin selection determination unit 28 can specify the selection region based on a predetermined rule that is not limited to the above example based on the trajectory.

  In the present embodiment, the map information on the schedule application screen stored in the frame map storage unit 41 includes an area in which a bar indicating that the schedule is filled is displayed as shown in FIG. Define as an object area. Then, an area where no bar is displayed in the schedule book is defined as a blank area. The bar (object) of the schedule book shown in (a) of FIG. 36 is defined as an object area 99 on the map information as shown in (b) of FIG.

  The margin selection determination unit 28 determines whether a margin region has been selected or an object region has been selected based on the presence / absence of an object region overlapping the selection region. Here, the object area is a bar representing a schedule in FIG. 36A, and refers to the object area 99 shown in FIG. In the example shown in FIG. 36B, the object area 99 overlaps the selection area, but the ratio of the overlapping object area in the selection area is less than a predetermined threshold (for example, less than 20%, etc.). Therefore, the margin selection determination unit 28 determines that a margin area has been selected.

  Subsequently, the related item extracting unit 23 refers to the related information shown in FIG. 12 and, based on the “screen display state; schedule application screen” stored in the state storage unit 47, “no object” and “screen display”. The related items associated with “status: schedule application screen” are extracted. In the example illustrated in FIG. 12, the related item extraction unit 23 extracts a related item group of R10 to R13. In the present embodiment, the related item extracting unit 23 further supplies icon identification information associated with each of the related items R10 to R13 to the operation screen processing unit 24.

  Based on the supplied icon identification information, the operation screen processing unit 24 stores the icon images of “23: Calculator”, “1: Television”, “7: Trash”, and “6: Palette” from the icon storage unit 43. Each icon is arranged in the read-out displayable area (image display area), and an operation screen is generated.

  FIG. 37 is a diagram illustrating a specific example of the operation screen obtained as a result of the operation screen generation process executed by the operation screen processing unit 24 of the present embodiment. The operation screen illustrated in FIG. 37 is an example of an operation screen that is displayed when a blank cell is selected when the tablet terminal 100 displays the schedule application screen.

  When the blank area is selected when the tablet terminal 100 is displaying the schedule application screen, the user naturally wants to fill the time zone (margin area) where the schedule is not filled with some schedule. Can be guessed.

  Therefore, as shown in FIG. 12, in the related information, the relation indicating the contents of the schedule with respect to “no object (in this case, the time zone where the schedule is not filled)” and “screen display state; schedule application screen” is shown. The item is associated with an icon that recalls the contents of the schedule. According to such a data structure, when the blank area of the schedule book is selected by the user, the related item extracting unit 23 can extract a candidate candidate to be filled as a related item.

  Therefore, the operation screen processing unit 24 can annularly arrange icon images that recall the contents of the extracted schedule. When the screen is sufficiently wide, the icon image may be arranged in a ring shape so that the selected margin area is at the center and the margin area is surrounded. In the example shown in FIG. 12 and FIG. 37, the operation screen processing unit 24 displays an icon “23: Calculator” indicating the schedule “R10; attach household account book” and “1: An icon of “TV”, an icon of “7: Trash” indicating the schedule “R12; garbage disposal”, and an icon of “6: Palette” indicating the schedule “R13; hobby / learning” are displayed in a ring shape. Further, as shown in FIG. 37, the operation screen processing unit 24 may display the contents of the schedule (contents described in “Related Items” in FIG. 12) with the icons.

  After the operation screen shown in FIG. 37 is displayed, for example, the user performs an operation of dragging a desired schedule icon and releasing it on the blank area of the desired time slot, thereby making the desired schedule the desired time slot. It is possible to realize a user interface that can be easily put in This further improves user convenience in managing the schedule.

≪Application to information processing device with separate display and input unit≫
Based on the configuration of the tablet terminal 100 as the information processing apparatus of the present invention, that is, based on the movement trajectory of the indicator, different icon groups are arranged depending on whether the object is selected or the margin is selected. About the structure to display, you may apply this to the information processing apparatus with which the input part 11 and the display part 12 are provided separately. For example, an information processing apparatus in which the input unit 11 is configured by an input device such as a mouse and the display unit 12 is configured by a display device such as an LCD can be considered. Here, the cursor displayed on the display unit 12 indicates the position of the display unit 12 on the screen. Then, when the user operates the mouse to perform an input operation, the cursor moves. Therefore, in such an embodiment, the mouse is the operating body, the cursor is the pointing body, and the pointing body displayed on the display unit 12 moves as the operating body moves. When the user operates the mouse to move the cursor on the screen of the display unit 12 to select an intended area, the information processing apparatus links the position of the cursor held in advance with the movement of the mouse. Thus, the locus of movement of the cursor (indicator) is held. Then, it is determined whether or not the selection area specified by the trajectory includes the object area, or it is determined whether or not the object area is selected by obtaining the proportion of the object area in the selection area. As a result, the information processing apparatus can determine whether the object area is selected or whether the blank area is selected, extract an appropriate related item according to the determination result, and display the icon.

  As the input unit 11 configured separately from the display unit 12, various input devices such as a keyboard, a joystick, a digitizer, a tablet, and a stylus pen can be employed in addition to the mouse.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  The information processing apparatus and the screen display method of the present invention can be expressed as follows.

  In an information processing apparatus including a touch panel, a trajectory acquisition unit that acquires a trajectory of movement of the indicator that has moved on the touch panel, and a selection area that is specified based on the trajectory acquired by the trajectory acquisition unit is the touch panel. A margin selection determination unit that determines whether or not the above information is a blank region that is not displayed, and a margin region is selected when the selection region is determined to be a blank region by the margin selection determination unit. Corresponding to the related item extracted by the related item extracting means, the related item extracting means for extracting the item related to the event as a related item with reference to the related information storage unit that stores the items related to the event An information processing apparatus comprising screen processing means for displaying an icon to be displayed on the touch panel.

  In a screen display method in an information processing apparatus including a touch panel, a trajectory acquisition step for acquiring a trajectory of movement of an indicator that has moved on the touch panel, and a selection specified based on the trajectory acquired in the trajectory acquisition step When it is determined in the margin selection determination step that determines whether or not the region is a margin region in which information on the touch panel is not displayed, and in the margin selection determination step, the selection region is a margin region, With reference to a related information storage unit that stores an item related to the event for which the blank area is selected, a related item extracting step for extracting the item related to the event as a related item, and the related item extracting step And a screen processing step for displaying an icon corresponding to the extracted related item on the touch panel. Surface display method.

[Example of software implementation]
Finally, each block of the tablet terminal 100, in particular, the contact information generation unit 21, the object specification unit 22, the related item extraction unit 23, the operation screen processing unit 24, the margin selection determination unit 28, the state determination unit 29, and the ring shape determination unit 30, the registered image processing unit 36 may be configured by hardware logic, or may be realized by software using a CPU as follows.

  That is, the tablet terminal 100 includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, a RAM (random access memory) that expands the program, A storage device (recording medium) such as a memory for storing the program and various data is provided. An object of the present invention is to provide a recording medium on which a program code (execution format program, intermediate code program, source program) of a control program of the tablet terminal 100, which is software that realizes the functions described above, is recorded so as to be readable by a computer. This can also be achieved by supplying the tablet terminal 100 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the tablet terminal 100 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered Trademark), 802.11 wireless, HDR (High Data Rate), mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention can be widely applied to information processing apparatuses including an input unit and a display unit. For example, but not limited to this, a digital TV, personal computer, smartphone, tablet PC, notebook computer, mobile phone, PDA (Personal Digital Assistant), e-book reader, electronic dictionary, portable, provided with an input unit and a display unit -It can be used suitably for a home game machine, an electronic blackboard, etc. Furthermore, if the present invention is applied to an information processing apparatus including a touch panel, it is possible to realize even better operability.

10 Control unit 11 Input unit (touch panel)
12 Display unit (touch panel)
13 Operation unit 14 External interface 15 Communication unit 16 Wireless communication unit 17 Audio output unit 18 Audio input unit 19 Storage unit 21 Contact information generation unit (trajectory acquisition means)
22 Object identification part (object identification means)
23 Related Item Extraction Unit (Related Item Extraction Unit)
24 Operation screen processing unit (screen processing means)
28 Margin selection determination unit (margin selection determination means)
29 State determination unit (state determination means)
30 Ring shape determination unit (screen processing means)
36 Registered image processing unit (image processing means)
41 Frame map storage unit (divided information storage unit)
42 related information storage unit 43 icon storage unit 44 contact information storage unit 47 state storage unit 48 registered image storage unit 100 tablet terminal (information processing apparatus)

Claims (15)

A trajectory acquisition means for acquiring a trajectory by which the indicator pointing to the position of the screen of the display unit has moved;
Margin selection determination means for determining whether or not the selection area identified based on the locus acquired by the locus acquisition means has selected a margin area in which information on the display unit is not displayed;
When it is determined by the margin selection determination means that the selected area has selected a margin area, the margin information is associated with the event by referring to a related information storage unit that stores an item associated with the selected event. A related item extracting means for extracting the selected item as a related item;
Screen processing means for displaying on the display unit an icon corresponding to the related item extracted by the related item extracting means ,
The margin selection determination means specifies the size of the selection region for selecting a margin region based on the locus,
The information processing apparatus according to claim 1, wherein the related item extracting unit determines the number of related items to be extracted related to an event in which the blank area is selected according to a size of the selected area .   The screen processing means determines the size of the icon corresponding to the related item or the size of the image display area for displaying the icon according to the size of the selection area. The information processing apparatus according to claim 1.   The information processing apparatus according to claim 1, wherein the screen processing unit displays the icon in the selection area that is determined to have selected a blank area.   The screen processing means arranges and arranges one or a plurality of icons on the outline of the ring and displays them on the display unit.
  The information processing apparatus according to any one of claims 1 to 3, wherein the screen processing unit determines the size of the ring according to the size of the selection region.   In accordance with the screen displayed by the information processing apparatus on the display unit, the information processing apparatus includes state determination means for determining a screen display state of the information processing apparatus,
  In the related information storage unit, related items related to the event for which the blank area is selected are further stored in association with each screen display state,
  The related item extracting unit is associated with the screen display state of the information processing apparatus when the margin selection determining unit determines that the selected region has selected a margin region, which is determined by the state determining unit. The information processing apparatus according to any one of claims 1 to 4, wherein the related item is extracted.   The margin selection determining means refers to division information defining a plurality of divided areas obtained by dividing the screen of the display unit, and identifies a divided area including the selected area for selecting a margin area based on the trajectory. And
  In the related information storage unit, related items related to the event in which the blank area is selected are further stored in association with each divided area,
  The said related item extraction means extracts the related item linked | related with the division area where the said selection area specified by the said margin selection determination means is included, The any one of Claim 1-5 characterized by the above-mentioned. The information processing apparatus described in 1.   The margin selection determination means specifies the size of the selection region for selecting a margin region based on the trajectory, classifies the selection region into one of a plurality of categories based on the size,
  In the related information storage unit, a related item related to the event in which the blank area is selected is further stored in association with each size of the selected area,
  7. The related item extracting unit extracts a related item associated with a size category of the selected area classified by the margin selection determining unit. The information processing apparatus described in 1.   The input unit and the display unit included in the information processing apparatus constitute a touch panel.
  The information processing apparatus according to any one of claims 1 to 7, wherein the trajectory acquisition unit acquires a trajectory of movement of the indicator that has moved on the touch panel.   The input unit included in the information processing apparatus inputs an instruction to move the cursor as the indicator displayed on the display unit to the information processing apparatus.
  The information processing apparatus according to claim 1, wherein the trajectory acquisition unit acquires a trajectory of movement of the cursor.   A trajectory acquisition means for acquiring a trajectory by which the indicator pointing to the position of the screen of the display unit has moved;
  Margin selection determination means for determining whether or not the selection area identified based on the locus acquired by the locus acquisition means has selected a margin area in which information on the display unit is not displayed;
  When it is determined by the margin selection determination means that the selected area has selected a margin area, the margin information is associated with the event by referring to a related information storage unit that stores an item associated with the selected event. A related item extracting means for extracting the selected item as a related item;
  Screen processing means for displaying an icon corresponding to the related item extracted by the related item extracting means on the display unit;
  Image processing means for processing the image and displaying it on the display unit,
  The margin selection determination means specifies the size of the selection region for selecting a margin region based on the locus,
  The image processing means is activated when the image processing icon displayed by the screen processing means is selected, and the size of the image display area for displaying the image is set according to the size of the selection area. The information processing apparatus is characterized in that the image is enlarged and reduced or trimmed. A trajectory acquisition means for acquiring a trajectory by which the indicator pointing to the position of the screen of the display unit has moved;
Margin selection determination means for determining whether or not the selection area identified based on the locus acquired by the locus acquisition means has selected a margin area in which information on the display unit is not displayed;
When it is determined by the margin selection determination means that the selected area has selected a margin area, the margin information is associated with the event by referring to a related information storage unit that stores an item associated with the selected event. A related item extracting means for extracting the selected item as a related item;
Screen processing means for displaying on the display unit an icon corresponding to the related item extracted by the related item extracting means,
The margin selection determination means includes:
Recognize areas where selectable objects are displayed as object areas, areas where selectable objects are not displayed as margin areas,
Above Symbol selection area, if it does not contain the above object area, it is determined that the selected region is selected margin area,
When the selection area includes at least a part of the object area, the ratio of the object area overlapping the selection area to the selection area is calculated.
If the ratio is less than the threshold, it is determined that the selection area has selected a blank area,
If the ratio is above the threshold, the selection region information processing device shall be the determining means determines that the currently selected object region. A trajectory acquisition means for acquiring a trajectory by which the indicator pointing to the position of the screen of the display unit has moved;
Margin selection determination means for determining whether or not the selection area identified based on the locus acquired by the locus acquisition means has selected a margin area in which information on the display unit is not displayed;
When it is determined by the margin selection determination means that the selected area has selected a margin area, the margin information is associated with the event by referring to a related information storage unit that stores an item associated with the selected event. A related item extracting means for extracting the selected item as a related item;
Screen processing means for displaying on the display unit an icon corresponding to the related item extracted by the related item extracting means,
The margin selection determination means recognizes an area where a selectable object is displayed as an object area, and an area where no selectable object is displayed as a margin area,
Further, when the selection area by the upper Symbol margin selection determining means is determined to select the object region, at least partially overlaps an object in the selected area, the specific object identifying means as the object selected by the user With
The related information storage unit further stores an object and items related to the object in association with each other,
The related item extracting means, the related information with reference to the storage unit, information you and extracting the item associated with the object identified by the object specifying means as the related item paper processing apparatus.   A screen display method in an information processing apparatus,
  A trajectory acquisition step for acquiring a trajectory that the indicator pointing to the position of the screen of the display unit has moved;
  A margin selection determination step for determining whether or not the selection region identified based on the locus acquired in the locus acquisition step selects a margin region in which information on the display unit is not displayed;
  If it is determined in the margin selection determination step that the selected area has selected a margin area, the event is referred to by referring to a related information storage unit that stores an item related to the event for which the margin area is selected. A related item extraction step for extracting related items as related items;
  A screen processing step for displaying an icon corresponding to the related item extracted in the related item extracting step on the display unit,
  In the margin selection determination step, the size of the selection region for selecting a margin region is specified based on the locus,
  In the related item extracting step, the number of related items to be extracted related to the event in which the blank area is selected is determined according to the size of the selected area.   A control program for causing a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 12.   The computer-readable recording medium which recorded the control program of Claim 14.

Images