JP2015049773A - Object operation system, object operation control program and object operation control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow grouping of objects or taking-out of the objects from groups to be easily and surely executed.SOLUTION: An object operation system comprises: a display part that displays an object; an operation part that enables an operation of the object; and a control part that controls the display part and the operation part. The control part determines whether a preliminarily determined specific gesture is included in an operation of causing the predetermined object to move on the display part, and depending upon whether the specific gesture is included in the operation, the control part determines whether the operation is an operation of defining only an arrangement of the predetermined object on the display part, or an operation of defining a group relationship between the arrangement of the predetermined object on the display part and other objects displayed on the display part.

Description

  The present invention relates to an object operation system, an object operation control program, and an object operation control method, and more particularly to an object operation system capable of displaying and operating objects, and grouping a plurality of objects in the system and extracting objects from the groups. The present invention relates to an object operation control program to be controlled and an object operation control method.

  In recent years, an electronic conference or the like that uses a display screen (referred to as a shared screen) that can be used by a plurality of users and writes and discusses display elements (hereinafter referred to as objects) such as characters, graphics, and images on the shared screen Has been done. In such a shared screen, multiple users can write various objects, move the written objects to any location in the shared screen, or group any objects displayed on the shared screen. Can do.

  As a technique related to the grouping of the objects, for example, in Patent Document 1 below, an operation unit, a first layer displayed in a state where information can be selected, and the first layer are selected by being superimposed on the first layer. A controller configured to display on the display unit a screen composed of a transparent or semi-transparent second layer for grouping information, and the controller is connected to the first layer via the operation unit. An information processing apparatus is disclosed in which when the displayed information is selected, the selected information is displayed on the second layer to be a grouping target.

JP 2012-145980 A

  The shared screen is a system in which the appearance of objects and the logical configuration (group configuration) of the objects are independent. For example, when two objects are overlapped, the relationship between the two objects is two There are two cases where the objects are independent and are simply displayed in an overlapping manner, and when the two objects are in one group. Therefore, when overlapping objects or moving overlapping objects, it is necessary for the user to instruct which of the above-described object relationships should be set.

  For example, in PowerPoint (registered trademark) of Microsoft (registered trademark), the designation of the arrangement of the object and the designation of the logical configuration of the object are different operations. So, for example, when grouping two objects, first move the object or surround the object with a frame to select the two objects, and group the selected objects. The instructing operation must be performed separately, and the operation is complicated.

  As another method for grouping objects, for example, a mode for grouping objects is provided, and when objects are overlapped when the mode is selected, a method for grouping the overlapped objects is performed. Can be considered. However, this method requires an operation for selecting a mode, and thus the operation becomes complicated. In addition, since the shared screen is used by a plurality of users at the same time, if a certain user selects a mode without permission, there is a possibility that the operation of another user will not be intended.

  As another method of grouping objects, there is a method of executing grouping when objects are arranged within a certain distance, as in Patent Document 1. However, this method makes it impossible to perform operations such as simply overlapping objects without grouping, and the degree of freedom of object placement is limited.

  Furthermore, as another method of grouping objects, a method of applying the technique of Patent Document 1 and executing grouping when a certain time elapses with a certain object overlaid on another object is also considered. It is done. However, with this method, grouping may be executed at a timing unintended by the user, such as when a certain period of time has passed and the user's operation may result in an unintended result.

  The same problem occurs not only when objects are grouped but also when a predetermined object among the objects once grouped is taken out of the group. However, in the conventional method, when taking out an object from a group, an operation for instructing to take out the object or an operation for selecting a mode have to be performed separately, which complicates the operation.

  The present invention has been made in view of the above problems, and a main object thereof is an object operation system and object operation control capable of simply and reliably executing object grouping and object extraction from the group. To provide a program and an object operation control method.

  One aspect of the present invention includes an object operation system including a display unit that displays an object, an operation unit that can operate the object, and a control unit that controls the display unit and the operation unit. The unit determines whether a predetermined specific gesture is included in an operation of moving a predetermined object on the display unit, and depending on whether the specific gesture is included in the operation, The operation is an operation that prescribes only an arrangement of the predetermined object on the display unit, or an arrangement of the predetermined object on the display unit and a group relationship with another object displayed on the display unit. It is characterized by determining whether the specified operation.

  One aspect of the present invention is an object operation control program that operates in a control unit that controls a display unit that displays an object and an operation unit that enables the object to be operated. The first process for determining whether or not the predetermined gesture is included in the operation to be moved on the display unit, and the operation is performed according to whether or not the specific gesture is included in the operation. An operation that prescribes only the arrangement of the predetermined object on the display unit, or an operation that prescribes the arrangement of the predetermined object on the display unit and the group relationship with other objects displayed on the display unit, The second process of determining the above is executed.

  One aspect of the present invention is an object operation control method in a system including a display unit that displays an object, an operation unit that can operate the object, and a control unit that controls the display unit and the operation unit. A first process for determining whether a predetermined gesture is included in an operation of moving a predetermined object on the display unit, and whether the operation includes the specific gesture. Accordingly, the operation is an operation that prescribes only the arrangement of the predetermined object on the display unit, or the arrangement of the predetermined object on the display unit and another object displayed on the display unit. And a second process for determining whether the operation defines a group relationship.

  According to the object operation system, the object operation control program, and the object operation control method of the present invention, it is possible to easily and reliably execute the grouping of objects and the extraction of objects from the groups.

  The reason for this is that, in a system including a display unit that displays an object, an operation unit that can operate the object, and a control unit that controls the display unit and the operation unit, the control unit changes the appearance of the object. It is determined whether a predetermined gesture is included in an operation (for example, moving an object), and if the specific gesture is included, the object overlaps with another object at the end of the operation. This is because it is determined whether or not these objects are grouped according to whether or not they are present. Further, at the end of the operation, it is determined whether or not to take out the object from the group depending on whether or not there is another object in the vicinity of the object.

It is a figure which shows typically the external appearance structure of the object operation system which concerns on 1st Example of this invention. It is a figure which shows typically the other external appearance structure of the object operation system which concerns on 1st Example of this invention. It is a figure which shows typically the other external appearance structure of the object operation system which concerns on 1st Example of this invention. It is a figure which shows typically the other external appearance structure of the object operation system which concerns on 1st Example of this invention. It is a block diagram which shows the structure of the object operation system which concerns on 1st Example of this invention. It is a flowchart figure which shows the process (grouping) of the object operation system which concerns on 1st Example of this invention. It is a flowchart figure which shows the process (extraction of the object from a group) of the object operation system which concerns on 1st Example of this invention. It is a figure which shows typically the procedure of the grouping of the object which concerns on 1st Example of this invention. It is a figure which shows the example of a display of the object in movement. It is a figure which shows typically the procedure which takes out an object from the group which concerns on 1st Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example (example which groups the grouped object group and object) of the grouping of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example (example which groups an object and the group of objects grouped) of the object which concerns on 1st Example of this invention. It is a figure which shows typically the other example (example which groups the grouped object groups) of the grouping of the object which concerns on 1st Example of this invention. It is a flowchart figure which shows the process (grouping) of the object operation system which concerns on 2nd Example of this invention. It is a figure which shows typically the procedure of the grouping of the object which concerns on 2nd Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 2nd Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 2nd Example of this invention. It is a figure which shows typically the other example of grouping of the object which concerns on 2nd Example of this invention.

  As shown in the background art, when discussing by operating an object displayed on the shared screen, the object can be moved, moved to a group, or extracted from the group. Various operations are performed. Here, when moving an object, it is possible to move the object easily by moving a finger or the like that touches the object. However, when grouping multiple objects, an object such as moving multiple objects is used. The operation for making the selected state and the operation for instructing the grouping of the selected object have to be performed separately, and the operation is complicated. Also, when an object is taken out from a group, the operation for setting the target object to the selected state and the operation for instructing the removal of the selected object from the group must be performed separately. It was complicated.

  Regarding this grouping operation, there is a method of providing a mode for grouping objects, but this method requires an operation of selecting a mode, and a shared screen is used by a plurality of users at the same time. If a mode is selected without permission, inconvenience arises in the operation of other users. There are also methods such as grouping when objects are placed within a certain distance, and grouping when a certain amount of time has passed with other objects superimposed, but with these methods, the objects are simply stacked. There is a risk of grouping. This problem occurs not only when the objects are grouped but also when the objects are extracted from the group.

  That is, an operation of moving an object or the like to select the object (an operation to change the appearance state) and an operation to instruct the selected object to be grouped or taken out (change the logical configuration) If the operation is performed as a separate and independent operation, the operation becomes complicated. On the other hand, in order to simplify the operation, when the object state satisfies a certain condition, the grouping is automatically performed (that is, only the operation that changes the appearance state), the user's intention The result may be contrary to the above. Against this background, there is a demand for a method that simplifies the operation as much as possible and that allows objects to be grouped and taken out from the group as intended by the user.

  Therefore, in an embodiment of the present invention, when a predetermined gesture is included in an operation for changing the appearance arrangement of an object, such as an object movement operation, the object grouping or the object from the group When the above operation is completed, grouping of objects or extraction of objects from the group is executed according to the positional relationship between the object to be operated and other objects.

  Specifically, in the case of grouping, if a specific gesture such as a long press is performed when moving an object, the object can be grouped, and the moved object overlaps another object. If so, group those objects.

  Also, in the case of taking out an object from a group, when a specific gesture such as a long press is performed when moving a predetermined object in the group, the object can be taken out (the predetermined object is If there is no other object in the group near the moved object, the predetermined object is excluded from the group.

  With the control described above, the user can designate an operation target object without taking time and obtain a result as intended by the user. In addition, since the mode is not specified but only the object to be operated is specified, it is possible to prevent contention even in an environment where a plurality of users operate the object at the same time. Further, since the operations are consistent between grouping the objects and taking out the objects from the group, the operations can be easily understood.

  In order to describe the above-described embodiment of the present invention in more detail, the object operation system, the object operation control program, and the object operation control method according to the first embodiment of the present invention will be described with reference to FIGS. explain. 1 to 4 are diagrams schematically showing the configuration of the object operation system according to the present embodiment, and FIG. 5 is a block diagram showing the configuration of the object operation system. 6 and 7 are flowcharts showing processing of the object operation system of this embodiment, and FIGS. 8 to 10 are schematic views showing specific examples thereof. FIG. 11 to FIG. 20 are schematic diagrams showing variations of object grouping.

  The present invention is applicable to both cases where there is one operator and a plurality of operators. In this embodiment, a system having a shared work area that can be operated by a plurality of operators will be described. There are four possible forms of this system. As shown in FIG. 1, the first form includes a touch panel having a display unit 40 that displays an object and an operation unit 50 that operates the object, and a control unit 20 that controls the display unit 40 and the operation unit 50. It is the form comprised with the apparatus united. In the second embodiment, as shown in FIG. 2, a touch panel having a display unit 40 and an operation unit 50 and a control unit 20 that controls the display unit 40 and the operation unit 50 are configured separately, and these are wired. Or it is a form connected by radio | wireless. In the third embodiment, as shown in FIG. 3, a device in which the display unit 40 and the control unit 20 are integrated and an operation unit 50 that sends an operation signal to the control unit 20 in a wired or wireless manner are separately provided. It is the form comprised by. As shown in FIG. 4, the fourth mode is a mode in which the display unit 40, the operation unit 50, and the control unit 20 are configured separately and are connected by wire or wirelessly. Hereinafter, in order to simplify the description, the description will be made on the premise of the first embodiment of FIG.

  The object operation system 10 of this embodiment is a display panel, an electronic blackboard, a tablet terminal, a notebook computer device, or the like having a calculation function. As shown in FIG. 5A, the control unit 20 and the storage unit 30 are used. And the display unit 40 and the operation unit 50.

  The control unit 20 includes a CPU (Central Processing Unit) 21 and a memory such as a ROM (Read Only Memory) 22 and a RAM (Random Access Memory) 23. The CPU 21 receives a control program from the ROM 22 and the storage unit 30. The entire operation of the object operation system 10 is controlled by calling and developing the program in the RAM 23 and executing it. Further, as shown in FIG. 5B, the control unit 20 also functions as a processing unit 20a, an operation determination unit 20b, and an object management unit 20c.

  The processing unit 20a causes the display unit 40 to display an object according to the content of the operation determined by the operation determining unit 20b. When the operation determination unit 20b detects a specific gesture for instructing grouping, an object to be grouped is added by adding a frame to the object to be grouped or changing the display form. Allow users to identify. When grouping is performed, an object to be grouped is surrounded by a frame or the background is changed so that the user can identify the group configuration.

  In addition, when the operation determination unit 20b detects a specific gesture instructing extraction of an object from the group, the processing unit 20a adds a frame to the object to be extracted from the group or changes the display form to remove the object from the group. Allows the user to identify the object to retrieve. Also, when removing an object from a group, the user can identify that the object is being removed from the group by changing the frame surrounding the object in the group according to the position of the object to be removed. To do.

  The operation determination unit 20b determines an operation performed by the operation unit 50. For example, when the touch is performed on the object and the touch position is moved, it is determined that the operation is an operation for moving the object or a group including the object (drag operation). In addition, a specific gesture such as when the object is continuously touched (so-called long press) for a predetermined time or more at the same position or when the object is touched one or more times (so-called tap or double tap) within a predetermined time or less. If it is detected, it is determined that an instruction to group objects or take out an object from the group is given.

  Specifically, it is determined whether a specific gesture is included in an operation of moving a certain object to a position overlapping with another object (for example, whether a specific gesture has been performed in a state of overlapping with another object), If a specific gesture is included, it is determined that the instruction is to group these objects. Then, after the object is moved, when the touch is released, it is determined whether there is another object that overlaps the object, and if there is another object, the object management unit 20c is instructed to group these objects. .

  Also, whether a specific gesture is included in an operation of moving a predetermined object among a plurality of grouped objects (for example, a specific gesture is performed on a predetermined object in a grouped state). If a specific gesture is included, it is determined that the instruction is to take out a predetermined object from the group. Then, after the predetermined object is moved, when the touch is released, it is determined whether there is another object belonging to the group in the vicinity of the predetermined object. If there is no other object, the predetermined object from the group is detected. The object management unit 20c is instructed to take out (exclude the predetermined object from the group).

  When object grouping is instructed from the operation determination unit 20b, the object management unit 20c identifies objects to be grouped and registers them as a group in a group management table provided in advance. When the operation determination unit 20b instructs to take out an object from the group, the object to be taken out from the group is specified, and the specified object is excluded from the group management table.

  The processing unit 20a, the operation determination unit 20b, and the object management unit 20c may be configured as hardware, or software that causes the control unit 20 to function as the processing unit 20a, the operation determination unit 20b, and the object management unit 20c. (Operation control program) may be configured so that the CPU 21 executes the operation control program.

  The storage unit 30 is configured by a memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, and stores information on the contents of operations performed on the operation unit 50 and objects displayed on the display unit 40. In addition, the group management table and the like are stored.

  The display unit 40 includes an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, and the like, and provides a shared work area where a plurality of operators can operate the object. The operation unit 50 includes a touch sensor composed of grid-like electrodes arranged on the display unit 40, hard keys, and the like. In addition to normal object operations (object movement, etc.), grouping of objects, It accepts a specific gesture that instructs to take out an object from the group. The display unit 40 and the operation unit 50 constitute a touch panel.

  Hereinafter, an object operation control method using the object operation system 10 configured as described above will be described. First, object grouping will be described. The CPU 21 executes the processing shown in the flowchart of FIG. 6 by developing the operation control program stored in the ROM 22 or the storage unit 30 in the RAM 23 and executing it. In the following description, a plurality of objects are displayed in advance on the display unit 40 (touch panel), and the user operates the objects on the touch panel.

  First, when a predetermined object displayed on the touch panel is touched by a user operation and the touch position is moved, the operation determination unit 20b determines that a drag operation of the object is being performed, and the processing unit 20a Then, the display position of the object is moved in accordance with the movement of the touch position (S101).

  Next, the operation determination unit 20b determines whether the drag operation of the object is completed (whether the movement of the touch position is stopped and the touch is released) (S102). Since it is a movement, a process is complete | finished.

  When the drag operation of the object continues, the operation determination unit 20b determines whether a specific gesture (for example, long press) has been performed (S103), and if the specific gesture has not been performed, Since the drag operation continues, the process returns to S102 and the same processing is repeated.

  On the other hand, when a specific gesture is performed, the operation determination unit 20b determines whether the object being operated overlaps another object (S104). If the object being operated does not overlap another object, there is no other object to be grouped, so the operation determination unit 20b determines that the drag operation of the object continues, and the process proceeds to S102. Return and repeat the same process. On the other hand, if the object being operated overlaps another object, the operation determination unit 20b determines that the operation is an instruction to instruct grouping of objects (S105).

  Here, if it is determined that the operation is an instruction for grouping, the grouping process may be executed immediately. However, at this stage, when the user cancels the grouping of the object or when another operation at another position is performed. When grouping with an object, the position of the object being operated may be adjusted. Therefore, the operation determination unit 20b determines whether or not the overlapping of the objects continues (S106), and determines that the grouping is canceled or the grouping with another object is performed when the overlapping is eliminated. Then, the process returns to S102. On the other hand, if the object being operated continues to overlap another object, the position of the object being operated may be adjusted, so the operation determination unit 20b determines whether the drag operation has ended. (S107), if the drag operation has not ended, the process returns to S106 to determine whether the overlapping of the operated objects continues.

  When the drag operation is finished (touch is released) in a state where the object being operated overlaps with another object, the operation determination unit 20b determines that the objects are grouped. Instructing the management unit 20c, the object management unit 20c registers the object being operated and other objects overlapping the object in the group management table (S108).

  FIG. 8 schematically shows this series of operations. Here, in order to facilitate understanding, two objects are provided, each is represented by a simple rectangle, and the hatching type is changed to distinguish the two objects.

  First, the user touches one object (see (a)) and moves the object to a position where it overlaps the other object (see (b)). When the touch is released in this state (see (c)), the process is terminated without performing grouping because the object is simply moved. On the other hand, when a specific gesture (here, long press) is performed with one object overlaid on the other object (see (d)), it is determined that the operation is an instruction to group objects. . At that time, in order to allow the user to easily identify the object to be grouped, a frame is added to the object to be grouped, and the form (color and density) of the object is changed. (In the figure, a thick frame is added).

  Here, as described above, there may be a case where the position of the object is adjusted during grouping, the grouping of the object is canceled, or the grouping with another object is performed. In the stage, the grouping process is not executed, the object is held in a movable state, the object is moved when the movement operation is performed (see (e)), and then the grouping operation is performed when the touch is released. Is determined, and the overlapping objects are grouped (see (f)). At that time, in order to make it easy for the user to identify the grouped object, a frame surrounding the grouped object can be added, or the background can be colored (the object is shown in the figure). A surrounding frame is added).

  Further, when the object is further moved in the state of FIG. 8E (see (h)) and the touch is released without overlapping with the other object (see (i)), the grouping of the objects is canceled. Therefore, the process is terminated without grouping. At this time, since it is considered that the grouping of objects is canceled in a state where there is no overlap with other objects (the state in FIG. 8H), the frame added to the object may be deleted. Further, since grouping with another object is also conceivable, the groupable state may be maintained (the frame is displayed) until the touch is released.

  Further, when the object is moved, the display form of the object may be changed so that the user can identify that the object is moving. For example, when one object is moved when two objects are displayed as shown in FIG. 9A, the object is displayed lightly as shown in FIG. 9), the object is displayed in a reduced size as shown in FIG. 9C, or only the frame of the object is displayed as shown in FIG. 9D. You can also. By performing such processing, it is possible to eliminate the inconvenience that the screen is difficult to see due to the object to be moved.

  Next, processing for extracting an object from a group will be described. The CPU 21 executes the processing shown in the flowchart of FIG. 7 by developing the operation control program stored in the ROM 22 or the storage unit 30 in the RAM 23 and executing it. In the following description, a plurality of objects are grouped in advance, and the user operates the objects on the touch panel.

  First, the operation determination unit 20b determines whether a specific gesture (for example, long press) has been performed on any one of a plurality of objects grouped in advance (S201). If a specific gesture has not been performed, the operation determination unit 20b determines whether a drag operation has started on any object (S202). If the drag operation has not started, the operation determination unit 20b returns to S201. Similar processing is repeated. When a drag operation is started on any object, the operation determination unit 20b determines that the drag operation is for the entire group, and moves the entire object in the group in accordance with the drag (S203). Then, the operation determination unit 20b determines whether or not the drag operation has ended (S204). If the drag operation has not ended, the operation determination unit 20b returns to S203 to continue the drag operation, and when the drag operation ends, the series of processing is performed. finish.

  On the other hand, when a specific gesture is performed on any one of the plurality of objects in S201, the operation determination unit 20b determines that the operation is an instruction to take out an object from the group, When the drag operation is started (Yes in S205), only the touched object is moved in response to the drag operation (S206). Next, the operation determination unit 20b determines whether or not the drag target object is more than a certain distance from other objects in the group (S207).

  If it is not separated from other objects by a certain distance or more, the operation determination unit 20b determines whether the drag operation has ended (S208). If the drag operation ends without being separated from another object, the operation determination unit 20b determines that the object has not been removed from the group, ends the series of processes, and continues the drag operation. Returning to S207, the distance to other objects is monitored.

  In S207, when the distance from the other object exceeds a certain distance, the operation determination unit 20b instructs the object management unit 20c to take out the object from the group, and the object management unit 20c rewrites the group management table, The object is excluded from the group (S209). Thereafter, the operation determination unit 20b determines whether or not the drag operation has ended (S210). If the drag operation has not ended, the operation determination unit 20b moves the object along with the drag, and ends the series of processes when the drag operation ends. To do.

  FIG. 10 schematically shows this series of operations. Again, for ease of understanding, there are two objects, each represented by a simple rectangle, and the hatching type is changed to distinguish the two objects.

  When two objects are grouped (see (a)), when the user touches one object in the group and moves the touch position, it is determined as a drag operation to the entire group, and the entire group Is moved (see (b)). On the other hand, when a specific gesture (here, long press) is performed on one object (see (c)), it is determined that the operation is an instruction to take out the object from the group, and then the touch is performed. When the position is moved, it is determined that the drag operation is performed only on the object, and only the object is moved (see (d)). When the object is further moved away from the other objects in the group by a certain distance or more, the object is taken out from the group (see (e)).

  At that time, in order to make it easy for the user to identify the object to be taken out from the group, a frame is added to the object on which a specific gesture has been performed, or the form (color or density) of the object is changed. Thus, it may be distinguished from other objects. Further, in order to allow the user to easily determine that an operation for taking out an object from the group is performed, the frame surrounding the group may be deformed as the object moves. In addition, in order to allow the user to easily determine that the extraction of the object from the group has been completed, the frame added to the object can be deleted, or the form of the object can be restored. In addition, when there are two objects constituting a group, the group is canceled, so the frame and background indicating the group are deleted.

  In FIG. 10D, the frame surrounding the group is enlarged as the object moves away. In order to allow the user to easily determine visually that the object is to be taken out from the group, The frame surrounding the group can be deformed as it moves. For example, it can be deformed from an elliptical frame in FIG. 10 (f) to a saddle-shaped frame in FIG. 10 (g).

  The basic configuration of object grouping and object removal from the group has been described above. However, the content of a specific gesture, the timing of performing a specific gesture, etc. can be changed if the same control can be realized. is there.

  For example, regarding the grouping of two objects, in FIG. 8, only one object is operated to perform grouping, but as shown in FIG. 11, both objects are moved by touching both objects ( (See (a)), both objects are overlapped (see (b)), and then the grouping may be executed when the touch of both objects is released (see (c)). In this case, an operation of touching both objects and overlapping the objects corresponds to a specific gesture.

  In the above description, the long press is exemplified as the specific gesture. However, for example, as illustrated in FIG. 12, the object is moved by touching one object (see (a)) and overlapped with the other object. When tapping (operation that touches once for a short time) or double tap (operation that touches twice at short time intervals) is performed (see (b)), it is determined as an operation that instructs grouping (see (c)). Thereafter, when the touch of the object is released (see (d)), the grouping may be executed.

  In this method, the touch of the object is temporarily released when tapping or double-tapping, but the time from releasing the touch to tapping, the time of tapping or double-tapping, and again after tapping or double-tapping. By appropriately setting the time until the touch, it can be regarded as a series of operations. Also, this time interval is monitored, and when a predetermined time interval is exceeded (for example, when the time from when touch is released until tapping exceeds a predetermined time), grouping is not performed. Thus, it is possible to prevent the inconvenience of erroneously determining a normal operation as a group operation. This method can also be applied to the case of taking out an object from a group. For example, by performing a tap or a double tap instead of a long press in the state shown in FIG. It becomes possible to take out the object which becomes.

  In the above description, the case of operating with one finger has been described, but the operation can also be determined by the number of fingers. For example, an operation with one finger is assigned as an operation such as moving an object, and an operation with two or more fingers is assigned as a group operation. Then, as shown in FIG. 13, when one object touches and moves with two fingers (see (a)) and overlaps the other object, it is determined that grouping is instructed (see (b)). Then, when the touch of the object is released (see (c)), the grouping may be executed. In the case of this method, grouping can be executed simply by moving an object and superimposing it on another object, so that the grouping operation can be further simplified. This method can also be applied to the case of taking out an object from a group. For example, by touching with two fingers instead of long pressing in the state of FIG. 10 (c) or (f), It becomes possible to take out the target object.

  In FIG. 13, an operation is assigned according to the number of fingers in advance, but it can also be determined that a specific gesture has been performed when the number of fingers is changed during the operation. For example, as shown in FIG. 14, when an object is moved by touching the object with one finger (see (a)), and the number of fingers to be touched is changed while overlapping the other object (here, When the number of fingers is changed from one to two (see (b)), it is determined as an operation for instructing grouping, and then the grouping is executed when the touch is released (see (c)). You may make it do.

  In the above description, it is determined that grouping is instructed by performing a specific gesture with the objects overlapped. For example, as illustrated in FIG. By performing a specific gesture (long press, tap / double tap, etc.), it is determined that the grouping is instructed (see (a)), and the object is grouped when it is moved and overlaid on another object. Grouping may be executed when the touch of the object is released (see (c)). In the case of this method, since a specific gesture such as long press, tap, or double tap is performed in a state where the objects are separated from each other, a problem that a specific gesture is erroneously performed on another object is prevented in advance. And the grouping operation can be performed more reliably.

  In the above description, the grouping is instructed by performing a specific gesture. However, each time a specific gesture is performed, the groupable state and the normal state can be switched. For example, as shown in FIG. 16, one object is touched (see (a)), and a specific gesture (long press, tap / double tap, etc.) is performed with the object moved and superimposed on the other object. Then, the grouping is possible (see (b)). In this state, if a specific gesture is further performed once, the groupable state is canceled (see (c)), and the identification is performed again. It is also possible to return to a state where grouping is possible by performing the gesture of once. Thus, by making it possible to switch between a groupable state and a normal state, operability can be improved.

  This method can also be applied to the case of taking out an object from a group. For example, when a long press is performed in the state of FIG. 10C or FIG. The state in which only the touched object can be moved) is released, and the state in which the object can be taken out is canceled when a long press is performed, and the two states are switched each time a specific gesture is performed. can do. In addition, here, the same operation as a specific gesture is performed to switch the state. However, for example, if a long press is performed, a grouping or an object can be taken out from the group, and tap / double tap is performed. If done, the state can be switched by a different specific gesture, such as returning to the normal state.

  In the above description, it is determined whether the operation type is a normal object operation or an operation for instructing grouping of objects or taking out an object from a group. For example, as shown in FIG. 17, the upper half of the object is assigned as a normal object operation area, and the lower half of the object is assigned as an object grouping area.

  In this case, touch the lower half of the object (see (a)), move the object, and if it overlaps with another object, it is determined that grouping is instructed (see (b)), and then the touch is released. Grouping is performed (see (c)). In other words, in this case, an operation of touching a specific area (here, the lower half) of the object corresponds to a specific gesture. On the other hand, the upper half of the object is touched (see (d)), and the object is moved, and it is not determined that grouping is instructed even if it is superimposed on another object (see (e)). When released, the object is placed at that position (see (f)). In this way, by performing the grouping only when a specific area of the object is touched, the grouping operation can be simplified and an erroneous operation can be prevented in advance.

  In the above description, the case where the objects are grouped is described. However, the objects and the grouped object groups can be grouped, and the grouped object groups can be further grouped. For example, as shown in FIG. 18, any object in the grouped object group or the background in the frame surrounding the grouped object group is touched (see (a)), and the grouped object When a specific gesture (for example, long press) is performed with the group moved and overlaid on another object, grouping is possible (see (b)), and then the object is untouched (See (c)), the object group and other objects are grouped. In this case, the group may be registered as a group that is one layer above the existing group, or the existing group may be temporarily destroyed, and the entire object group and other objects may be registered as a new group.

  Further, as shown in FIG. 19, the object is touched (see (a)), the object is moved, and the object in the grouped object group or the frame surrounding the grouped object group is enclosed. When a specific gesture (for example, long press) is performed in a state of being overlaid on the background, grouping becomes possible (see (b)), and then the object is untouched (see (c)). The object and the object group are grouped. In this case, similar to the above, it may be registered as a group one layer above the existing group, or the existing group may be temporarily destroyed and the object group and the entire object may be registered as a new group.

  Further, as shown in FIG. 20, the object in the grouped object group is touched (see (a)) by touching one of the objects in the grouped object group or the background in the frame surrounding the grouped object group. If you move a group and make a specific gesture (for example, long press) while overlaid on one of the objects in another group of objects or the background in a frame surrounding another group of objects, the group Then, when the touch of the object is released (see (c)), the object group of the moved group and the object group of another group are grouped. The In this case, as described above, these groups may be registered as a group on one layer above, or these groups may be temporarily destroyed and both object groups may be registered as a new group.

  In addition, here, the case where the object or the object group is pressed for a long time is described as an example, but the object and the object group or both object groups may be touched and overlapped as in FIG. It can also be grouped in the same way. Similarly to FIG. 12, when an object and an object group, or both of the object groups are tapped or double-tapped, they can be similarly grouped. Similarly to FIG. 13, when a plurality of fingers are touched to an object or a group of objects, they can be similarly grouped. Similarly to FIG. 14, when one finger is touched to an object or a group of objects and a plurality of fingers are touched in a state where the object and the object group, or both object groups are overlapped, grouping is performed in the same manner. You can also. Similarly to FIG. 15, when an object or an object group is touched and pressed for a long time and then the object and the object group, or both object groups are overlapped, they can be similarly grouped. Similarly to FIG. 17, when a specific area of an object or an object group is touched and the object and the object group, or both object groups are overlapped, they can be similarly grouped.

  Next, an object operation system, an object operation control program, and an object operation control method according to the second embodiment of the present invention will be described with reference to FIGS. FIG. 21 is a flowchart showing the operation of the object operation system of the present embodiment, and FIGS. 22 to 25 are schematic views showing specific examples thereof.

  As a method of grouping objects, there is a method of drawing a line surrounding the objects, selecting an object to be grouped, and then instructing the selected object to be grouped. Two types of operations, that is, an operation for selecting an object and an operation for instructing the selected object to be grouped must be performed, which complicates the operation. Another method is to set the object grouping mode, and then select the object to be grouped. However, this method allows other users not to group. Even when an object is selected, there is a disadvantage that it is grouped.

  That is, when grouping objects, it is necessary to be able to instruct grouping by a series of operations and to prevent mixing of normal object operations and group operations.

  To deal with this problem, in the first embodiment described above, when a specific gesture is included in a series of operations on a predetermined object, it is grouped with an object overlapping the predetermined object. However, in this embodiment, when a specific gesture is included in an operation on a plurality of objects instead of an operation on a predetermined object, the plurality of objects are grouped. Specifically, when drawing a line that surrounds multiple objects in order to select an object, if a line that surrounds multiple objects is drawn after performing a specific gesture, the instruction is to group Judgment is made to group a plurality of objects surrounded by the line. Note that the configuration of the object operation system 10 in this case is the same as that of the first embodiment, and a description thereof will be omitted.

  Hereinafter, object grouping will be described. The CPU 21 executes the processing shown in the flowchart of FIG. 21 by developing the operation control program stored in the ROM 22 or the storage unit 30 in the RAM 23 and executing it. In the following description, it is assumed that a plurality of objects are displayed on the touch panel in advance, and the user operates the objects by touching the touch panel.

  First, the operation determination unit 20b determines whether a specific gesture (here, long press) has been performed (S301).

  If a specific gesture is not performed, the operation determination unit 20b determines whether drawing of a line (referred to as a surrounding line) surrounding a plurality of objects has started (S302), and if not, the process proceeds to S301. Returning and monitoring a specific gesture, when the drawing of the surrounding line is started, it is a normal operation, so the surrounding line is drawn (S303). Thereafter, the operation determination unit 20b determines whether drawing of the surrounding line has been completed (S304). If not, the operation determination unit 20b returns to S303 to continue drawing the surrounding line, and when drawing of the surrounding line is completed. Then, it is determined that the operation is to select an object, and the object surrounded by the enclosing line is selected (S305).

  On the other hand, when a specific gesture is performed in S301, the operation determination unit 20b determines that the instruction is for an object grouping. Then, the operation determination unit 20b determines whether drawing of a line surrounding a plurality of objects has been started (S306). If not, the operation determination unit 20b waits for the start, and when drawing of the surrounding line starts, Since this is an operation for instructing grouping, a surrounding line is drawn in a form (color, thickness, etc.) different from the normal operation (object selection operation) in S303 (S307). Thereafter, the operation determination unit 20b determines whether drawing of the surrounding line has been completed (S308). If the drawing has not been completed, the process returns to S307 to continue drawing the surrounding line. Then, the object management unit 20c is instructed to group the objects, and the object management unit 20c registers the objects surrounded by the enclosing line in the group management table and performs grouping (S309).

  FIG. 22 schematically shows this series of operations. Again, for ease of understanding, there are two objects, each represented by a simple rectangle, and the hatching type is changed to distinguish the two objects.

  First, in a state where a plurality of (here, two) objects are displayed on the screen (see (a)), the screen is touched and the touched finger is moved so as to surround the plurality of objects ((b )), And then release the touch. In this case, since it is a simple object selection operation, a plurality of objects are selected without being grouped. In this object selection operation, in order to make it easy for the user to recognize that the object has been selected, the object is displayed in a different form from that of grouping, for example, by shadowing the object. can do.

  On the other hand, when the user touches the screen and performs a specific gesture (here, long press) at that position (see (d)), it is determined that the operation is an instruction to group objects. Then, after the touched finger is moved so as to surround the plurality of objects, the grouping of the objects is executed when the touch is released. At that time, in order to enable the user to easily determine that the objects have been grouped, a line surrounding a plurality of objects to be grouped can be drawn, or the background design can be changed.

  It should be noted that an object that has already been grouped may be included in the object that is the target of this grouping operation, and in that case, it may be registered as a group that is one layer above the existing group. Alternatively, an existing group may be temporarily destroyed, and the entire object surrounded by a surrounding line may be registered as a new group.

  In the above description, a specific gesture has been described as long press, but a tap or double tap may be used. For example, as shown in FIG. 23, when a plurality of (here, two) objects are displayed on the screen (see (a)), the screen is touched and a tap or a specific gesture is performed at that position. When double tapping is performed (see (b)), it is determined that the operation is an instruction to instruct grouping of objects. Then, after the touched finger is moved so as to surround the plurality of objects, the grouping of the objects is executed when the touch is released.

  Also, as shown in FIG. 24, a line surrounding the object is drawn using fingers of both hands (for example, a semicircle is drawn in the opposite direction with fingers of each hand), or two lines are drawn as shown in FIG. When a line surrounding the object is drawn with the finger, it may be determined that the instruction is for grouping the objects.

  In addition, this invention is not limited to the said Example, The structure and control can be changed suitably, unless it deviates from the meaning of this invention.

  For example, in the above embodiment, when the objects overlap, the overlapping objects are grouped. However, the arrangement of the objects and the form of the individual objects are various, and it is clear that the objects overlap. In some cases, it may be difficult to recognize. In such a case, if an object to be grouped is clear, grouping may be performed on an object near the object. The neighborhood range in that case can be set as appropriate according to the arrangement of other objects (whether or not objects that are not grouped are separated).

  Further, in the above embodiment, it is assumed that one user performs an operation. However, if objects to be operated are different, when a plurality of users perform an operation at the same time, The control of the present invention can be applied.

  In the above embodiment, the case where the object is operated on the touch panel has been described. However, the present invention can be similarly applied to the case where the object is operated using an operation unit such as a laser mouse. In that case, touching an object on the touch panel corresponds to an operation of moving the mouse cursor over the object and left-clicking. The tap / double tap of the object on the touch panel corresponds to an operation of left-clicking once or twice on the object for a short time without moving the position of the mouse cursor.

  In the above embodiment, the object is described as a rectangular figure, but the form of the object is arbitrary, and the method of the present invention is similarly applied to figures, images, photographs, text, etc. of any shape. can do.

  The present invention relates to a system capable of operating objects such as characters and graphics, in particular, a system in a system capable of joint operation by a plurality of operators, an operation control program that operates in the system, and a recording in which the operation control program is recorded The present invention can be used for a medium and an operation control method for controlling an operation of an object in the system.

DESCRIPTION OF SYMBOLS 10 Object operation system 20 Control part 20a Processing part 20b Operation determination part 20c Object management part 21 CPU
22 ROM
23 RAM
30 storage unit 40 display unit 50 operation unit

Claims (33)

A display for displaying objects;
An operation unit that makes the object operable,
A control unit for controlling the display unit and the operation unit,
The controller is
It is determined whether or not a predetermined specific gesture is included in an operation of moving a predetermined object on the display unit, and the operation is performed depending on whether or not the specific gesture is included in the operation. An operation for defining only the arrangement of the predetermined object on the display unit, or an operation for defining the arrangement of the predetermined object on the display unit and a group relationship with other objects displayed on the display unit To judge,
An object operation system characterized by that. The controller is
If the specific gesture is included in the operation and the predetermined object overlaps at least a part of the other object at the end of the operation, the predetermined object and the other object are grouped together. Register as,
The object operation system according to claim 1. The controller is
When the specific gesture is included in the operation and the predetermined object overlaps at least a part of the other object, the display form of the predetermined object and the other object is changed.
The object operation system according to claim 2, wherein: The controller is
When the overlap between the predetermined object and the other object is released, the display form of the predetermined object and the other object is restored to the original state.
The object operation system according to claim 3. The controller is
When registering the predetermined object and the other object as a group, a frame surrounding the predetermined object and the other object is displayed on the display unit.
The object operation system according to any one of claims 2 to 4, wherein The controller is
The predetermined object and the other object are registered in advance as a group, the specific gesture is included in the operation, and the predetermined object and the other object are at a predetermined distance at the end of the operation. If it is far away, the predetermined object is excluded from the group.
The object operation system according to claim 1. The controller is
When the specific gesture is included in the operation, the display form of the predetermined object is changed.
The object operation system according to claim 6. The controller is
Displaying a frame surrounding the predetermined object and the other object registered in advance as a group, and changing the shape of the frame as the predetermined object moves;
The object operation system according to claim 7. The predetermined object and / or the other object is one object or a plurality of grouped objects.
The object operation system according to claim 1, wherein the object operation system is an object operation system. A display for displaying objects;
An operation unit that makes the object operable,
A control unit for controlling the display unit and the operation unit,
The controller is
It is determined whether or not a predetermined specific gesture is included in an operation for selecting a plurality of objects, and the operation is performed based on whether or not the specific gesture is included in the operation. Determining whether the operation is a selection only operation or selection of the plurality of objects and a group relationship of the plurality of objects.
An object operation system characterized by that. The operation unit is a touch panel,
The specific gesture includes any one of an operation of continuously touching the object at the same position for a predetermined time or more, an operation of touching the object one or more times within a predetermined time or less, and an operation of touching the object with a plurality of fingers.
The object operation system according to claim 1, wherein An object operation control program that operates in a control unit that controls a display unit that displays an object and an operation unit that allows the object to be operated,
In the control unit,
A first process for determining whether a predetermined specific gesture is included in an operation of moving a predetermined object on the display unit;
Depending on whether or not the specific gesture is included in the operation, the operation is an operation that defines only the arrangement of the predetermined object on the display unit, or the arrangement of the predetermined object on the display unit And a second process for determining whether the operation defines a group relationship with another object displayed on the display unit.
An object operation control program characterized by the above. In the second process,
If the specific gesture is included in the operation and the predetermined object overlaps at least a part of the other object at the end of the operation, the predetermined object and the other object are grouped together. Register as,
The object operation control program according to claim 12, wherein: In the second process,
When the specific gesture is included in the operation and the predetermined object overlaps at least a part of the other object, the display form of the predetermined object and the other object is changed.
The object operation control program according to claim 13. In the second process,
When the overlap between the predetermined object and the other object is released, the display form of the predetermined object and the other object is restored to the original state.
15. The object operation control program according to claim 14, wherein: In the second process,
When registering the predetermined object and the other object as a group, a frame surrounding the predetermined object and the other object is displayed on the display unit.
16. The object operation control program according to claim 13, wherein the object operation control program is any one of claims 13 to 15. When the predetermined object and the other object are registered in advance as a group,
In the second process,
If the specific gesture is included in the operation, and the predetermined object and the other object are separated by a predetermined distance or more at the end of the operation, the predetermined object is excluded from the group.
The object operation control program according to claim 12, wherein: In the second process,
When the specific gesture is included in the operation, the display form of the predetermined object is changed.
The object operation control program according to claim 17, wherein: When a frame surrounding the predetermined object registered in advance as a group and the other object is displayed,
In the second process,
As the predetermined object moves, the shape of the frame is changed.
The object operation control program according to claim 18. The predetermined object and / or the other object is one object or a plurality of grouped objects.
The object operation control program according to any one of claims 12 to 19, wherein An object operation control program that operates in a control unit that controls a display unit that displays an object and an operation unit that allows the object to be operated,
In the control unit,
A first process for determining whether a predetermined gesture is included in an operation of selecting a plurality of objects;
Depending on whether or not the specific gesture is included in the operation, the operation is only an operation for selecting the plurality of objects, or an operation for defining the selection of the plurality of objects and a group relation of the plurality of objects. A second process for determining whether or not
An object operation control program characterized by the above. The operation unit is a touch panel,
The specific gesture includes any one of an operation of continuously touching the object at the same position for a predetermined time or more, an operation of touching the object one or more times within a predetermined time or less, and an operation of touching the object with a plurality of fingers.
The object operation control program according to any one of claims 12 to 21, wherein An object operation control method in a system comprising: a display unit that displays an object; an operation unit that enables operation of the object; and a control unit that controls the display unit and the operation unit,
A first process for determining whether a predetermined specific gesture is included in an operation of moving a predetermined object on the display unit;
Depending on whether or not the specific gesture is included in the operation, the operation is an operation that defines only the arrangement of the predetermined object on the display unit, or the arrangement of the predetermined object on the display unit And a second process for determining whether the operation defines a group relationship with another object displayed on the display unit.
An object operation control method characterized by the above. In the second process,
If the specific gesture is included in the operation and the predetermined object overlaps at least a part of the other object at the end of the operation, the predetermined object and the other object are grouped together. Register as,
The object operation control method according to claim 23. In the second process,
When the specific gesture is included in the operation and the predetermined object overlaps at least a part of the other object, the display form of the predetermined object and the other object is changed.
25. The object operation control method according to claim 24. In the second process,
When the overlap between the predetermined object and the other object is released, the display form of the predetermined object and the other object is restored to the original state.
26. The object operation control method according to claim 25. In the second process,
When registering the predetermined object and the other object as a group, a frame surrounding the predetermined object and the other object is displayed on the display unit.
The object operation control method according to any one of claims 24 to 26, wherein: When the predetermined object and the other object are registered in advance as a group,
In the second process,
If the specific gesture is included in the operation, and the predetermined object and the other object are separated by a predetermined distance or more at the end of the operation, the predetermined object is excluded from the group.
The object operation control method according to claim 23. In the second process,
When the specific gesture is included in the operation, the display form of the predetermined object is changed.
The object operation control method according to claim 28, wherein: When a frame surrounding the predetermined object registered in advance as a group and the other object is displayed,
In the second process,
As the predetermined object moves, the shape of the frame is changed.
30. The object operation control method according to claim 29. The predetermined object and / or the other object is one object or a plurality of grouped objects.
The object operation control method according to any one of claims 23 to 30, wherein: An object operation control method in a system comprising: a display unit that displays an object; an operation unit that enables operation of the object; and a control unit that controls the display unit and the operation unit,
A first process for determining whether a predetermined specific gesture is included in an operation of selecting a plurality of objects;
Depending on whether or not the specific gesture is included in the operation, the operation is only an operation for selecting the plurality of objects, or an operation for defining the selection of the plurality of objects and a group relation of the plurality of objects. A second process for determining whether or not
An object operation control method characterized by the above. The operation unit is a touch panel,
The specific gesture includes any one of an operation of continuously touching the object at the same position for a predetermined time or more, an operation of touching the object one or more times within a predetermined time or less, and an operation of touching the object with a plurality of fingers.
33. The object operation control method according to any one of claims 23 to 32, wherein:

Images