JP5705768B2 - Operation information input system and content search method executed by operation information input system - Google Patents

Description

  The present invention relates to an operation information input system and a content search method executed by the operation information input system.

  Conventionally, with the widespread use of touch screens and tablet-type information processing terminals, the operation of information processing terminals is changing from mouse and keyboard operations to touch operations. In accordance with this, attempts to realize operations such as scrolling and enlarging / reducing with more intuitive human actions are becoming popular. Opportunities for sharing large touch screens with multiple people are also increasing.

  In recent years, “Kurumuri”, a technology that changes the content displayed by intuitive operation, using a frame-type physical device to cut out only the point of interest on the touch screen and rotate the frame-type device, has been proposed. Has been. Note that the frame-type device may be called a frame-type object.

Ryosuke Aoki, Hiromu Miyashita, Masayuki Ihara, Takehiko Ohno, Hiroaki Chiaki, Satoshi Kobayashi, Kurumuru: Tablet operation method using frame-type physics objects with multiple conductive parts, HCI Study Group (144th), 2011

  However, in the above-described conventional technology, it may be difficult to perform continuous input operations on the information processing terminal using the frame-type device. Specifically, in the above-described prior art, since the information processing terminal is operated by a discrete input operation for each single frame device, a plurality of inputs cannot be performed simultaneously, and the frame device In some cases, it is difficult to execute continuous input operations on the information processing terminal. For example, in the above-described conventional technology, it is difficult to finely adjust the rotation and enlargement of the 3D-CAD model projected on the touch screen using the frame device, and information processing using the frame device is difficult. It may be difficult to perform continuous input operations on the terminal. That is, in the above-described prior art, for example, in order to perform an operation of enlarging the clipped portion, an operation of adjusting with a scroll bar or the like separately projected on the touch screen is required. The input is required to move the viewpoint, and the input becomes complicated.

  Therefore, the technology according to the present application has been made in view of the above-described problems of the prior art, and an operation information input system and an operation information input that enable continuous input operations to an information processing terminal using a frame-type device. An object of the present invention is to provide a content search method executed by a system.

  In order to solve the above-described problems and achieve the object, an operation information input system in which an input operation is performed on an information processing apparatus by operating a physical device on a touch screen, , A first device of a frame type, and a second device disposed within a frame of the first device, wherein the first device and the second device are respectively sensed by the touch screen Each of the plurality of sensed units, and the information processing apparatus stores a device information storing unit information in which a predetermined process is associated with each arrangement pattern of the plurality of sensed units, and the touch screen. A detection unit for identifying and detecting the detected units of the first device and the second device from a plurality of the detected units detected by A processing unit that refers to chair information and executes processing corresponding to the arrangement pattern of the sensed units of the first device and the second device detected by the detection unit, respectively. To do.

  The operation information input system according to the present application makes it possible to improve the operability of an information processing terminal using a frame-type device.

FIG. 1 is a diagram illustrating an example of the configuration of the operation information input system according to the first embodiment. FIG. 2 is a diagram illustrating an example of the configuration of the physical device according to the first embodiment. FIG. 3 is a diagram for explaining the position of the conductive portion in the physical device according to the first embodiment. FIG. 4 is a diagram illustrating an example of the configuration of the information processing apparatus according to the first embodiment. FIG. 5 is a diagram illustrating a first example of information stored by the device information storage unit according to the first embodiment. FIG. 6 is a diagram illustrating a second example of information stored by the device information storage unit according to the first embodiment. FIG. 7 is a schematic diagram illustrating the conductive part identification process performed by the grounding device detection processing unit according to the first embodiment. FIG. 8 is a flowchart of a process procedure performed by the operation information input system according to the first embodiment. FIG. 9 is a flowchart illustrating the procedure of the combination determination process performed by the grounding device detection processing unit according to the first embodiment. FIG. 10 is a flowchart illustrating the procedure of the arrangement pattern specifying process performed by the grounding device detection processing unit according to the first embodiment. FIG. 11 is a flowchart of a process procedure performed by the grounding device direction calculation processing unit according to the first embodiment. FIG. 12 is a diagram for explaining content search using the operation information input system according to the first embodiment. FIG. 13 is a diagram illustrating an example of a content search according to the first embodiment.

  Exemplary embodiments of an operation information input system and a content search method executed by the operation information input system according to the present application will be described below in detail with reference to the accompanying drawings. The operation information input system and the content search method executed by the operation information input system according to the present application are not limited to the following embodiments.

[Configuration of Operation Information Input System According to Embodiment 1]
First, the configuration of the operation information input system according to the embodiment will be described. FIG. 1 is a diagram illustrating an example of a configuration of an operation information input system 300 according to the first embodiment. As illustrated in FIG. 1, the operation information input system 300 includes a physical device 100 and an information processing apparatus 200. As illustrated in FIG. 1, the information processing apparatus 200 includes a touch screen 210 and receives an information input operation by the physical device 100. For example, the touch screen 210 is a capacitive touch screen 210 or a pressure-sensitive touch screen 210.

  Here, the capacitive touch screen accepts an information input operation by detecting a weak voltage change and determining contact or non-contact. On the other hand, a pressure-sensitive touch screen accepts information input operations by detecting pressure. In this embodiment, a case where an information processing apparatus having a capacitive touch screen 210 is used will be described as an example.

  As illustrated in FIG. 1, the physical device 100 includes a lower layer device 110 and an upper layer device 120 stacked on the lower layer device 110. For example, as shown in FIG. 1, the lower layer device 110 has a hollow cylindrical shape and includes a conductive portion (also referred to as a sensed portion) 111 on a side surface. The conductive portion 111 is formed of a conductor, and is grounded when a user's hand is touched.

  The upper layer device 120 has a hollow convex shape and has a conductive portion 121 on a side surface. The conductive part 121 is formed of a conductor, and is grounded when a user's hand is touched. Here, the upper layer device 120 is stacked on the lower layer device 110 by fitting the convex portion 122 into the hollow of the lower layer device 110.

  Details of the physical device 100 shown in FIG. 1 will be described with reference to FIGS. FIG. 2 is a diagram illustrating an example of the configuration of the physical device 100 according to the first embodiment. Here, in FIG. 2, the upper layer device 120 and the lower layer device 110 have a front surface (a surface viewed from the upper side in FIG. 1), a rear surface (a surface viewed from the lower side in FIG. 1), and a side surface (a horizontal surface in FIG. 1). The surface viewed from the direction). As illustrated in FIG. 2, the upper layer device 120 has, for example, a ring-shaped convex shape, and the conductive portion 121 is provided in the circumferential direction of the side surface of the portion that is not the convex portion 122. And in the upper layer device 120, the electroconductive part 121 is extended from the side surface to the convex part 122 direction, and the electroconductive part 121 of three places is formed in a back surface.

  That is, if the user's hand touches the conductive part 121 on the side surface of the upper layer device 120 and the back surface of the upper layer device 120 is brought into contact with the touch screen 210, it is grounded by the three conductive parts 121 on the back surface, Each conductive part 121 is detected by the touch screen 210. Note that portions other than the conductive portion 121 in the upper layer device 120 are formed of a dielectric.

  For example, the lower layer device 110 has a ring shape, and the conductive portion 111 is provided in the circumferential direction of the side surface. In the lower layer device 110, three conductive portions 111 are formed from the side surface to the back surface. That is, also in the lower layer device 110, each conductive portion 111 is detected by the touch screen 210 as in the upper layer device 120. Further, the portion other than the conductive portion 111 in the lower layer device 110 is formed of a dielectric, as in the upper layer device 120.

  FIG. 3 is a diagram for explaining the position of the conductive portion in the physical device 100 according to the first embodiment. FIG. 3 shows a case where the upper layer device 120 is stacked so that the back side of the upper layer device 120 is in contact with the front side of the lower layer device 110. Further, in FIG. 3, the grounding points that are positions where the conductive portions of the upper layer device 120 and the lower layer device 110 are detected by the touch screen 210 are indicated by circles.

  As shown in FIG. 2, since the conductive portion 121 in the upper layer device 120 is provided on the back side of the convex portion 122, the ground point of the upper layer device 120 is the ground point of the lower layer device 110 as shown in FIG. It will be located inside. That is, when the back surface comes into contact with the touch screen 210 while holding the conductive portions on the side surfaces of the upper layer device 120 and the lower layer device 110, the conductive portion 121 of the upper layer device 120 and the conductive portion 111 of the lower layer device 110 are distinguished. It becomes possible.

  Here, as shown in FIGS. 2 and 3, in the lower layer device 110 and the upper layer device 120, the arrangement pattern of the conductive portions in the ring is different. That is, as shown in FIG. 3, the touch screen 210 simultaneously detects the three conductive portions 111 of the lower layer device in the arrangement pattern indicated by the ground point of the lower layer device 110, whereas the arrangement indicated by the ground point of the upper layer device 120. The conductive part 121 of the upper layer device is simultaneously detected by the pattern.

  The physical device 100 according to the first embodiment has the above-described configuration without having an electronic circuit. For example, in the physical device 100, the conductive portions 111 and 121 are formed of a conductor such as aluminum foil, and the ring-shaped dielectric and the ring-shaped and convex-shaped dielectric are formed of wood, foamed polystyrene, or the like. The

  Note that the configuration of the physical device 100 illustrated in FIGS. 1 to 3 is merely an example, and the embodiment is not limited thereto. That is, the shape of the physical device 100 is not limited to a cylindrical shape, and may be a case where, for example, hollow square pillars are stacked. Further, the number of devices to be stacked is not limited to two. For example, three or more devices may be stacked. In addition, the conductive portion may have any shape as long as it can identify that the touch screen 210 is in contact.

  As described above, the physical device 100 includes the upper layer device 120 and the lower layer device 110 having conductive portions having different arrangement patterns. Then, the user operates the physical device 100 on the touch screen 210 to cause the information processing apparatus 200 to execute a desired process. The information processing apparatus 200 according to the first embodiment executes processing according to the position of the conductive part detected by the touch screen 210. Details of the information processing apparatus 200 will be described below. FIG. 4 is a diagram illustrating an example of the configuration of the information processing apparatus 200 according to the first embodiment.

  As shown in FIG. 4, for example, the information processing apparatus 200 includes a database 220, a grounding point detection processing device 230, a real-time processing generation device 240, and a grounding point information processing device in addition to a touch screen 210 (not shown). 250 and a display screen processing device 260, and is connected to the database 400 via the network 500. The network 500 is, for example, a local area network (LAN) or a wide area network (WAN).

  As shown in FIG. 4, the database 400 includes an information storage unit 410. The information storage unit 410 stores various types of information acquired by a grounding point information processing device 250 described later. For example, the information storage unit 410 stores image data acquired by the ground point information processing apparatus 250 and transferred via the network 500.

  As illustrated in FIG. 4, the database 220 includes a grounding point information storage unit 221, a device information storage unit 222, an image information storage unit 223, and a grounding device center coordinate and direction storage unit 224. The database 220 is, for example, a storage device such as a hard disk or an optical disk, or a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, and stores various programs executed by the information processing device 200. To do.

  The contact point information storage unit 221 stores information on the contact point acquired by the contact point detection processing device 230. The ground point means a point where the conductive part of the physical device 100 is grounded to the touch screen 210 and detected. Specifically, the ground point information storage unit 221 stores the coordinates of the ground point in the coordinate system of the touch screen 210 and the time when the coordinates of the ground point are acquired in association with each other.

  The device information storage unit 222 stores information regarding the physical device 100. Specifically, the device information storage unit 222 stores the operation content for each arrangement pattern of the conductive units that can be formed in the upper layer device 120 and the lower layer device 110 constituting the physical device 100. FIG. 5 is a diagram illustrating an example of information stored by the device information storage unit 222 according to the first embodiment.

  For example, as illustrated in FIG. 5, the device information storage unit 222 stores device information in which the distance ratio between the conductive parts and the distance and orientation with respect to Origin Point are associated with each position of the conductive parts. Here, the “position of the conductive part” indicates an arrangement pattern of the conductive part. The “distance ratio between the conductive parts” indicates, for example, the ratio of the distances between the centers of the conductive parts. In addition, “distance and orientation with respect to Origin Point” means that, when a conductive part that forms the shortest distance and the longest distance among conductive parts among the plurality of conductive parts is “Origin Point”, This means information indicating the distance and direction from “Origin Point” to the center of the physical device as a vector.

  Here, the device information storage unit 222 stores different applications and operation details (for example, enlargement / reduction processing) in association with each conductive portion position (arrangement pattern). That is, the device information storage unit 222 identifies what device the conductive unit detected by the touch screen 210 is, and stores information for executing processing corresponding to the device.

  Note that the information stored by the device information storage unit 222 is not limited to the information shown in FIG. FIG. 6 is a diagram illustrating a second example of information stored by the device information storage unit 222 according to the first embodiment. For example, as illustrated in FIG. 6, the device information storage unit 222 stores device information in which the angle of the conductive part, the distance to the origin point, and the orientation are associated with each position of the conductive part. That is, the angle of the conductive portion is stored instead of the distance ratio between the conductive portions. Here, the “angle of the conductive portion” indicates, for example, each angle of a triangle formed by a straight line connecting the conductive portions.

  Returning to FIG. 4, the image information storage unit 223 stores an image to be displayed on the touch screen 210. The ground device center coordinate and direction accumulation unit 224 stores the processing result of the ground point information processing apparatus 250. Specifically, the ground device center coordinate and direction accumulation unit 224 stores the center and orientation of the upper layer device 120 or the lower layer device 110 acquired by the ground point information processing apparatus 250. These pieces of information will be described in detail later.

  As shown in FIG. 4, the contact point detection processing device 230 includes a contact point position coordinate acquisition processing unit 231 and a contact point number processing unit 232. When the physical device 100 is placed on the touch screen 210 with the conductive part on the side being held, the ground point position coordinate acquisition processing unit 231 detects the ground point (coordinates) and touches it together with the detection time. Stored in the spot information storage unit 221. The ground point number processing unit 232 determines the number of ground points at the same time based on the coordinates of the ground point stored by the ground point information storage unit 221 and the acquisition time. For example, the grounding point number processing unit 232 determines whether the number of grounding points is 3, 6, or any other number.

  The real-time processing generator 240 issues an event that causes the information processing apparatus 200 to execute processing at a predetermined time interval. For example, the real-time processing generator 240 issues an event that causes the ground point acquisition process on the touch screen 210 to be executed every 33 ms.

  As shown in FIG. 4, the ground point information processing apparatus 250 includes a ground device detection processing unit 251, a ground device direction calculation processing unit 252, a ground device position coordinate calculation processing unit 253, a frame coordinate calculation processing unit 254, A grounding device information addition processing unit 255 and a grounding device information transmission processing unit 256 are included.

  The ground device detection processing unit 251 detects the arrangement pattern of the conductive parts of the physical device grounded to the touch screen 210. Specifically, the grounding device detection processing unit 251 touches the user from the grounding point coordinates detected by the grounding point position coordinate acquisition processing unit 231 and the number of grounding points detected by the grounding point number processing unit 232. The device operating on the screen 210 is determined, and the arrangement pattern of the grounded conductive parts is detected. Here, the ground device detection processing unit 251 can identify whether the user is operating the lower layer device 110 or the upper layer device 120.

  FIG. 7 is a diagram schematically illustrating the conductive part identification processing by the grounding device detection processing unit 251 according to the first embodiment. FIG. 7 shows a case in which each conductive portion is identified in a physical device in which devices of three layers (upper layer, middle layer, lower layer) each having three conductive portions are stacked. FIG. 7 shows processing after the ground point is detected by the ground point position coordinate acquisition processing unit 231 and shows the ground point from which each point in the figure is detected.

  For example, as shown in FIG. 7A, the grounding device detection processing unit 251 first, among the circles passing through the three grounding points at the grounding point detected by the grounding point position coordinate acquisition processing unit 231, A circle including all points other than the ground contact point forming the circle (minimum inclusion circle) is acquired. For example, the grounding device detection processing unit 251 draws a circle by randomly selecting three points from the nine grounding points shown in FIG. 7A, and all other grounding points are included in the circle. It is determined whether or not. Here, when not all the grounding points are included, the grounding device detection processing unit 251 repeatedly executes the same processing with another combination.

  Then, as shown in FIG. 7A, the grounding device detection processing unit 251 extracts a circle C1 including all other grounding points, and determines the three points forming the circle C1 as grounding points of the lower layer device. Is identified. Similarly, as shown in FIG. 7B, the ground device detection processing unit 251 includes a circle C2 including three points other than the three ground points that form a circle from the six ground points included in the circle C1. And the three points forming the circle C2 are identified as the grounding points of the middle layer device. Furthermore, as shown in FIG. 7C, the ground device detection processing unit 251 identifies the remaining three points as ground points of the upper layer device. That is, the grounding device detection processing unit 251 extracts grounding points that form circles with different distances (radius) from the center of the physical device 100, and identifies the grounding points as the conductive portions of the devices of the respective layers. Note that the number of conductive portions in each layer device is preset by the user.

  Further, the ground device detection processing unit 251 specifies the arrangement pattern of the conductive parts in the identified devices of each layer. Specifically, the grounding device detection processing unit 251 refers to the device information stored by the device information storage unit 222 and identifies the arrangement pattern by the distance ratio between the conductive units or the angle of the conductive units. For example, when the arrangement pattern is specified by the distance ratio between the conductive parts, the grounding device detection processing unit 251 calculates the distance between the coordinates of the grounding points in the specified devices of the same layer, and is normalized with the shortest distance. By calculating the distance, the distance ratio is calculated. Then, the grounding device detection processing unit 251 identifies the arrangement pattern corresponding to the calculated distance ratio with reference to the device information (FIG. 5).

  Further, for example, when specifying the arrangement pattern by the angle of the conductive part, the grounding device detection processing unit 251 draws a triangle connecting the coordinates of the grounding point in the specified device of the same layer, and calculates each angle of the triangle. calculate. Then, the grounding device detection processing unit 251 specifies the arrangement pattern corresponding to the calculated angle with reference to the device information (FIG. 6).

  The ground device direction calculation processing unit 252 calculates the direction of the device from the coordinates of the conductive part of the device operated by the user. Specifically, the ground device direction calculation processing unit 252 first determines the shortest distance and the longest distance between the conductive parts in the conductive part of the device in which the arrangement pattern is detected by the ground device detection processing part 251. One conductive part to be formed is extracted, and the extracted conductive part is determined as Origin Point (hereinafter referred to as OP). Then, the ground device direction calculation processing unit 252 determines the conductive part that forms the shortest distance from the OP as a second point (hereinafter referred to as SP).

  Thereafter, the ground device direction calculation processing unit 252 calculates the inclination of the line segment formed by OP and SP in the coordinate system of the touch screen 210 as the device direction. The grounding device direction calculation processing unit 252 determines a conductive part other than OP and SP as Third Point (hereinafter referred to as TP). As described above, by calculating the direction of the device, it is possible to calculate how much the device is rotated clockwise or counterclockwise when the device of each layer is rotated. .

  The ground device position coordinate calculation processing unit 253 calculates the position of the physical device 100 operated by the user. Specifically, the ground device position coordinate calculation processing unit 253 refers to the “distance and orientation with respect to Origin Point” of the device information, and determines the position of the physical device 100 from the OP determined by the ground device direction calculation processing unit 252. calculate. That is, the grounding device position coordinate calculation processing unit 253 adds the coordinates obtained by adding a vector of “distance and orientation with respect to Origin Point” to the coordinates of the OP determined by the grounding device direction calculation processing unit 252. And decide. As described above, by determining the position of the physical device on the touch screen 210, it is possible to determine where the physical device 100 is placed with respect to the image displayed on the touch screen 210. become able to.

  When the hollow physical device is grounded, the frame coordinate calculation processing unit 254 calculates the coordinates of the frame when executing processing for the image of the hollow portion. Specifically, the frame coordinate calculation processing unit 254 calculates the coordinates of the hollow portion from the position of the physical device 100 with reference to the device information (the size of the hollow portion) stored in the database 220 in advance.

  The ground device information addition processing unit 255 adds (maps) information to the arrangement pattern detected by the ground device detection processing unit 251 and stores the information in the information storage unit 410. For example, the ground device information addition processing unit 255 adds information to the arrangement pattern when a physical device (for example, a finger) different from the physical device 100 is newly grounded. The ground device information transmission processing unit 256 outputs the information added by the ground device information addition processing unit 255 and transmits it to another device.

  As shown in FIG. 4, the display screen processing device 260 includes an in-frame image enlargement / reduction processing unit 261 and an image update processing unit 262. The in-frame image enlargement / reduction processing unit 261 enlarges or reduces the image in the frame based on the position, orientation, and frame coordinates of the physical device 100 calculated by the contact point information processing apparatus 250. The in-frame image enlargement / reduction processing unit 261 executes other content selection processing and the like.

  For example, it is assumed that the arrangement pattern of the conductive portion 111 of the lower layer device 110 corresponds to an image enlargement / reduction process, and the arrangement pattern of the conductive portion 121 of the upper layer device 120 corresponds to an image rotation process. In such a case, when the user rotates the lower layer device 110 and the upper layer device 120 on the touch screen 210, the display screen processing device 260 executes image enlargement / reduction in accordance with the rotation of the lower layer device 110, and The image is rotated according to the rotation of the device 120.

  In the operation information input system 300 according to the first embodiment, as described above, operations by devices in each layer can be continuously executed. As a result, the operation information input system 300 can cope with complicated input operations, and enables continuous input operations to the information processing terminal using the frame-type device.

[Procedure for Processing by Operation Information Input System According to Embodiment 1]
Next, a processing procedure performed by the operation information input system 300 according to the first embodiment will be described with reference to FIG. FIG. 8 is a flowchart of a process procedure performed by the operation information input system 300 according to the first embodiment. As shown in FIG. 8, in the operation information input system 300 according to the first embodiment, first, the real-time processing generator 240 determines whether or not a predetermined time (for example, 33 ms) has elapsed (step S101). .

  Here, when it is determined that a predetermined time has elapsed, the contact point position coordinate acquisition processing unit 231 calculates the coordinates of the contact point (step S102). Then, the grounding device detection processing unit 251 determines whether or not the number of grounding points determined by the grounding point number processing unit 232 is a multiple of 3 (step S103). If the number of grounding points is a multiple of 3 (Yes at step S103), the grounding device detection processing unit 251 determines a combination of grounding points (step S104), and sets a device for each grounding point combination. Is determined (step S105).

  Then, the ground device direction calculation processing unit 252 determines a position and a direction for each device (step S106). Thereafter, the in-frame image enlargement / reduction processing unit 261 executes a process corresponding to the device that has received the input (step S107), and the image update processing unit 262 updates the display image (step S108).

  On the other hand, if the number of contact points is not a multiple of 3 in step S103 (No in step S103), the in-frame image enlargement / reduction processing unit 261 executes a process in which the user performs an input operation with a finger or the like (step S103). S110). When any one of the processes is executed and the display image is updated, the real-time processing generator 240 determines whether or not an end command has been received from the user (step S109). If the end command has not been received from the user (No at Step S109), the real-time processing generator 240 returns to Step S101 and determines whether or not a predetermined time has elapsed. On the other hand, when an end command is received from the user (Yes at Step S109), the operation information input system 300 ends the process.

[Combination Determination Processing Procedure by Grounding Device Detection Processing Unit According to Embodiment 1]
Next, the procedure of the combination determination process by the ground device detection processing unit 251 according to the first embodiment will be described. FIG. 9 is a flowchart illustrating the procedure of the combination determination process performed by the grounding device detection processing unit 251 according to the first embodiment. Here, the flowchart shown in FIG. 9 corresponds to the processing in step S104 in FIG. Note that FIG. 9 shows a process in the case where there are three conductive portions of devices in each layer.

  As shown in FIG. 9, the grounding device detection processing unit 251 obtains the number of grounding points (step S201), and calculates the quotient “q” and the remainder “r” obtained by dividing the number of grounding points by 3. In this case, it is determined whether or not “r = 0”. If “r = 0” (Yes at Step S202), the grounding device detection processing unit 251 selects three grounding points that are not combined (Step S203), and the selected grounding point is selected. Form a circle that passes through If “r = 0” is not satisfied (No at Step S202), the operation information input system 300 executes the process at Step S110 in FIG.

  Then, the grounding device detection processing unit 251 determines whether or not all other uncombined grounding points are included in the circle formed by the selected grounding points (step S204). Here, if all other uncombined grounding points are included (Yes in step S204), the grounding device detection processing unit 251 is a combination of the conductive units in the devices on the same layer with the three selected grounding points. Is determined (step S205).

  On the other hand, if all other uncombined grounding points are not included (No at Step S204), the grounding device detection processing unit 251 returns to Step S203 and selects three grounding points that are not combined. When the three grounding points are determined as combinations, the grounding device detection processing unit 251 determines whether or not “q” combinations have been determined (step S206).

  If “q” combinations have not been determined (No at Step S206), the grounding device detection processing unit 251 returns to Step S203 and selects three grounding points that are not combined. On the other hand, when “q” combinations are determined (Yes at Step S206), the grounding device detection processing unit 251 ends the combination determination process.

[Procedure for Arrangement Pattern Specification Processing by Grounding Device Detection Processing Unit According to Embodiment 1]
Next, the procedure of the arrangement pattern specifying process performed by the ground device detection processing unit 251 according to the first embodiment will be described. FIG. 10 is a flowchart illustrating the procedure of the arrangement pattern specifying process performed by the grounding device detection processing unit 251 according to the first embodiment. Here, the flowchart shown in FIG. 10 corresponds to the processing in step S105 in FIG. FIG. 10 shows a case where the arrangement pattern is specified using the distance ratio.

  When the combination of the conductive parts is determined in step S104, the ground device detection processing unit 251 calculates the distance between the conductive parts based on the coordinates of the ground point as shown in FIG. 10 (step S301). Then, the ground device detection processing unit 251 calculates the distance ratio by normalizing the calculated distance between the conductive parts (step S302).

  Thereafter, the ground device detection processing unit 251 refers to the device information storage unit 222, specifies the arrangement pattern of the conductive parts in the device from the calculated distance ratio (step S303), and ends the process.

[Processing Procedure by Grounding Device Detection Processing Unit According to Embodiment 1]
Next, a processing procedure performed by the grounding device direction calculation processing unit 252 according to the first embodiment will be described. FIG. 11 is a flowchart of a process procedure performed by the grounding device direction calculation processing unit 252 according to the first embodiment. Here, the flowchart shown in FIG. 11 corresponds to the processing in step S106 in FIG.

  When the device arrangement pattern is specified, the grounding device direction calculation processing unit 252 refers to the distance between the conductive units as shown in FIG. 11, and determines the conductive unit that forms the shortest distance and the longest distance. It determines as OP (step S401). Then, the ground device direction calculation processing unit 252 determines that the conductive part that is not OP among the two conductive parts forming the shortest distance is SP (step S402), and determines the remaining conductive part is TP (step S402). S403).

  Thereafter, the ground device position coordinate calculation processing unit 253 calculates the center coordinates of the vector from the coordinates of the OP, and determines the calculated coordinates as the position of the device (step S404). Further, the ground device direction calculation processing unit 252 determines the angle of the line segment formed by OP and SP with respect to the touch screen 210 as the device direction (step S405), and ends the process.

  As described above, the operation information input system 300 according to the first embodiment can continuously input a plurality of processes by operating the physical device 100 on the touch screen 210. Therefore, an example of content search will be described as a usage example of the operation information input system 300 according to the first embodiment. FIG. 12 is a diagram for explaining content search using the operation information input system 300 according to the first embodiment.

  FIG. 12 shows a schematic diagram of a content search example. The content search example shown in FIG. 12 shows a situation where countless content uploaded by a general user is pasted on a map projected on a tablet PC, such as a video content search application having position information. In recent years, with the spread of GPS-equipped cameras and smartphones, the opportunity to record video and images with location information metadata has increased, and as a result, enormous amounts of content with location information have overflowed on the Internet. By arranging them on the map, it is possible to browse the places where the user is interested and the moving images taken there.

  However, in the current application, only thumbnail images are displayed on the map, so if there is a large amount of content on the touch screen, individual contents can be surveyed unless the scale is increased. Can not. On the other hand, if the scale is large, you will not know what content is on the map. Therefore, a physical device as in the present application is utilized.

  For example, on the touch screen 210 shown in FIG. 12, the user brings the physical device 100 to an area of his / her interest on the map, and rotates the lower layer device 110 and the upper layer device 120 at that position to perform different processes. Can be executed. FIG. 13 is a diagram illustrating an example of a content search according to the first embodiment.

  For example, the enlargement / reduction process of the image is associated with the arrangement pattern of the conductive unit 111 of the lower layer device 110, and the content selection / volume adjustment process is associated with the arrangement pattern of the conductive unit 121 of the upper layer device 120. . Thereby, for example, when the user brings the physical device 100 to an area of interest on the map and rotates the lower layer device 110, a map enlarged to the desired area is displayed as shown in FIG. Is done.

  Then, the user rotates the upper layer device 120 in a state where the desired area is enlarged, thereby sequentially switching the contents existing in the zoomed-up area as shown in FIG. Each content can be viewed while making adjustments.

  Until now, the entire map has been expanded to the point where the visibility of the thumbnail images of the contents can be secured, and viewing of the contents in each map has been handled with a single click. By using it, content can be viewed with a simple operation such as moving or rotating the physical device. As described above, the operation information input system 300 according to the first embodiment makes it possible to significantly improve the user operability.

[Effect of Example 1]
As described above, according to the first embodiment, the physical device 100 includes the frame-type lower layer device 110 and the upper layer device 120 arranged in the frame of the lower layer device 110. The lower layer device 110 and the upper layer device 120 have a plurality of conductive parts that are sensed by the touch screen 210, respectively. In the information processing apparatus 200, the device information storage unit 222 stores device information in which a predetermined process is associated with each arrangement pattern of the plurality of conductive units. Then, the ground device detection processing unit 251 identifies and detects the conductive units of the lower layer device 110 and the upper layer device 120 from among the plurality of conductive units sensed by the touch screen 210. The in-frame image enlargement / reduction processing unit 261 executes processing corresponding to the arrangement pattern of the conductive portions of the lower layer device 110 and the upper layer device 120 detected by the ground device detection processing unit 251 with reference to the device information. To do. Therefore, the operation information input system 300 according to the first embodiment enables continuous input operations to the information processing terminal using a frame-type device using a physical device formed from a plurality of devices.

  That is, the user can use the physical device 100 having no power in an overlapping manner so that translation on the plane of the touch screen 210 and continuous rotation with a plurality of degrees of freedom in a plane parallel to the plane of the touch screen 210 are performed. Input is possible. Furthermore, the physical device 100 can be freely moved on a large touch screen, and a rotational input can be freely given with the left and right hands while gazing at an area cut out by the physical device 100. Thus, the user can perform various inputs with a simple operation of turning without distracting the viewpoint from the content of interest.

  In addition, according to the first embodiment, the ground device detection processing unit 251 detects the coordinates of the plurality of conductive parts on the touch screen 210, and forms the shortest conductive part distance and the longest conductive part distance. Then, SP, which is another conductive part that forms the shortest distance between the conductive parts, is extracted, and the inclination of the straight line formed by OP and SP in the coordinate system of the touch screen 210 is calculated as the direction of the physical device. Therefore, the operation information input system 300 according to the first embodiment makes it possible to accurately detect the orientation of the physical device.

  Further, according to the first embodiment, the device information storage unit 222 further stores the device information in association with the distance from the OP to the center of the physical device 100 and the direction. Then, the ground device detection processing unit 251 refers to the physical device information, and adds the distance to the center of the physical device 100 and the direction with respect to the conductive unit detected by the touch screen 210, so that the physical device 100 A position relative to the touch screen 210 is determined. Therefore, the operation information input system 300 according to the first embodiment makes it possible to accurately detect the position of the physical device 100.

  According to the first embodiment, when the physical device 100 is hollow, the device information storage unit 222 further stores the shape of the frame of the physical device 100 in the device information. Then, the ground device detection processing unit 251 calculates the position and direction of the frame from the direction and position of the physical device 100 with reference to the device information. Therefore, the operation information input system 300 according to the first embodiment makes it possible to execute a process using a frame.

  Further, according to the first embodiment, the conductive portion is a conductor, and the touch screen 210 is a capacitance type. Therefore, the operation information input system 300 according to the first embodiment can be applied to an information processing apparatus that detects contact by a change in voltage due to static electricity.

  Although the first embodiment has been described so far, the embodiment according to the present application is not limited to the first embodiment. That is, these embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made.

  In the first embodiment described above, the case where the capacitive touch screen 210 is used has been described as an example. However, the embodiment is not limited to this. For example, the pressure-sensitive touch screen 210 may be used. In such a case, the physical device 100 is used in which a protrusion that allows the touch screen 210 to sense pressure is provided at the position of the conductive portion of each of the lower layer device 110 and the upper layer device 120. And it is realizable by performing a predetermined process for every arrangement pattern of a projection part.

  In Example 1 mentioned above, the case where each layer device was provided with three electroconductive parts was demonstrated. However, the embodiment is not limited to this, and may include, for example, four or more conductive portions. In such a case, the combination of the conductive portions in the devices of each layer is executed as described below, for example.

  That is, as the procedure 1, first, a circle including all the grounding points detected by the touch screen 210 is determined. Then, as the procedure 2, the center of the determined circle is acquired. In step 3, the radius of the physical device 100 is registered in the information processing apparatus 200 in advance, and each ground point is associated with each layer device based on the radius. By performing the above-described processing, it is possible to use a physical device having conductive parts other than three.

  In addition, for example, the specific form (for example, the form of FIG. 4) of the distribution / integration of each device is not limited to the illustrated one, and all or a part thereof can be arbitrarily set according to various loads or usage conditions. It can be distributed or integrated functionally or physically in units. For example, the contact point position coordinate acquisition processing unit 231 and the contact point number processing unit 232 may be integrated as one contact point detection unit, and the in-frame image enlargement / reduction processing unit 261 executes each process. You may distribute to a some process part.

  Further, the ground point information processing device 250 may be connected as an external device of the information processing device 200 via a network, or the ground device detection processing unit 251 and the ground device direction calculation processing unit 252 may be connected by another device. The functions of the information processing apparatus 200 described above may be realized by having each connected and connected to a network.

  These embodiments and modifications thereof are included in the invention disclosed in the claims and equivalents thereof as well as included in the technology disclosed in the present application.

DESCRIPTION OF SYMBOLS 100 Physical device 110 Lower layer device 111,121 Conductive part 120 Upper layer device 122 Convex part 200 Information processing apparatus 230 Grounding point detection processing apparatus 240 Real-time processing generator 250 Grounding point information processing apparatus 260 Display screen processing apparatus

Claims (7)

An operation information input system in which an input operation is performed on an information processing apparatus by operating a physical device on a touch screen,
The physical device is
A frame-shaped first device; and a second device disposed within a frame of the first device;
Each of the first device and the second device has a plurality of sensed parts that are sensed by the touch screen,
The information processing apparatus includes:
A device information storage unit that stores device information in which a predetermined process is associated with each of the arrangement patterns of the plurality of sensed units;
A detecting unit for identifying and detecting the sensed parts of the first device and the second device from a plurality of sensed parts sensed by the touch screen;
A processing unit that refers to the device information and executes processing corresponding to an arrangement pattern of the sensed part of the first device and the second device detected by the detection unit;
An operation information input system characterized by comprising:   The detection unit detects coordinates of the plurality of sensed parts on the touch screen, respectively, and forms a shortest sensed part distance and a longest sensed part distance, and the shortest sensed part. A second sensed part, which is another sensed part that forms a distance between sensed parts, and the first sensed part and the second sensed part in the coordinate system of the touch screen, 2. The operation information input system according to claim 1, wherein an inclination of a straight line formed by the calculation is calculated as a direction of the physical device. The device information storage unit further stores the device information in association with a distance and a direction from the first sensed unit to the center of the physical device,
The detection unit refers to the device information and adds the distance and the direction to the center of the physical device with respect to the sensed unit detected by the touch screen. The operation information input system according to claim 2, wherein the position is determined. When the physical device is hollow, the device information storage unit further stores the shape of the physical device frame in the device information,
The operation information input system according to claim 2, wherein the detection unit calculates the position and direction of the frame from the direction and position of the physical device with reference to the device information.   The operation information input system according to claim 1, wherein the sensed part is a conductor, and the touch screen is a capacitive type. The operation information input system according to claim 1, wherein the sensed part protrudes in a direction of grounding with the touch screen, and the touch screen senses pressure. Content that is executed by an operation information input system that causes an information processing device to search for content by operating a physical device on a touch screen that displays a content search screen in which a plurality of contents are linked on a map A search method,
The physical device is
A frame-shaped first device; and a second device disposed within a frame of the first device;
Each of the first device and the second device has a plurality of sensed parts that are sensed by the touch screen,
The information processing apparatus is
A device information storage unit for storing device information in which processing related to content search is associated with each of the arrangement patterns of the plurality of sensed units;
A detection step of identifying and detecting the sensed parts of the first device and the second device from a plurality of sensed parts sensed by the touch screen;
A process step of referring to the device information and executing a process corresponding to an arrangement pattern of the sensed part of the first device and the second device detected by the detection step;
The content search method characterized by including.

Images