WO2018079383A1

WO2018079383A1 - Information processing apparatus, control method, program, and recording medium

Info

Publication number: WO2018079383A1
Application number: PCT/JP2017/037732
Authority: WO
Inventors: 洋一西牧; 泰史奥村
Original assignee: 株式会社ソニー・インタラクティブエンタテインメント
Priority date: 2016-10-28
Filing date: 2017-10-18
Publication date: 2018-05-03
Also published as: JPWO2018079383A1; JP6687749B2

Abstract

This information processing apparatus: receives a result of detection of a hand angle and a result of detection of a finger bending/stretching state by a finger sensor of a device; determines, on the basis of the received results of detection, whether there is a match with any hand poses identified in information stored in advance; acquires, when it is determined that there is a match with one of the hand poses, process identification information associated with the information that identifies the matching hand pose; and executes a process identified by the acquired process identification information.

Description

Information processing apparatus, control method, program, and recording medium

The present invention relates to an information processing apparatus, a control method, a program, and a recording medium.

2. Description of the Related Art Controllers for controlling the movement of characters in a virtual space are widely used in information processing apparatuses such as home game machines. In recent years, it has been considered that the movement of the user's hand is reflected in the movement of the finger of the character in the virtual space.

However, a finger sign may indicate some meaning depending on the region or scene, and when controlling the movement of a character's fingers, the user does not intend to present an inappropriate sign to the character in an online game or the like. There was a case where I was allowed to.

The present invention has been made in view of the above circumstances, and an object thereof is to provide an information processing apparatus, a control method, a program, and a recording medium that can prevent an inappropriate sign from being presented.

The present invention for solving the problems of the above-described conventional example is an information processing apparatus connected to a device fixed to a user's hand, and the device detects a bending / extending state of the finger for at least some of the fingers. A finger sensor; and an angle sensor for detecting a fixed hand angle, wherein the information processing apparatus is configured to specify information for identifying a hand pose and processing to be performed for at least one hand pose. Based on the received detection result, the receiving means connected to the holding means for holding the information in association with each other, and receiving the detection result of the bending state of the finger by the finger sensor and the detection result of the hand angle from the device, It is determined whether or not it matches one of the hand poses specified by the information held in the holding means. Determining means for acquiring process specifying information associated with the information specifying the over's, and execution means for executing the processing of said determination means is specified by the acquired process specifying information is obtained by a comprise.

According to the present invention, inappropriate signing can be prevented.

It is a schematic perspective view showing the example of the controller device connected to the information processing apparatus of embodiment of this invention. It is a block diagram showing the example of a structure of the controller device connected to the information processing apparatus of embodiment of this invention. It is a block diagram showing the example of a structure of the information processing apparatus of embodiment of this invention. It is a functional block diagram showing the example of the information processing apparatus of embodiment of this invention. It is explanatory drawing showing the example of the content of the pose database which the information processing apparatus of embodiment of this invention uses. It is a flowchart figure showing the process example of the information processing apparatus of embodiment of this invention.

Embodiments of the present invention will be described with reference to the drawings. In the following description, the sizes, ratios, and arrangements of the respective parts are examples, and the examples of the present embodiment are not limited to the illustrated sizes, ratios, and arrangements.

The information processing apparatus 1 according to the embodiment of the present invention is a computer device such as a home game machine, and a controller device 20 is connected to the information processing apparatus 1.

The controller device 20 is attached and fixed to the left or right hand of the user. In the following description, when it is necessary to distinguish between the controller device 20 fixed to the user's left hand and the controller device 20 fixed to the right hand, the controller device 20 fixed to the user's left hand is referred to as the controller device 20L, The controller device 20 fixed to the user's right hand is distinguished from the controller device 20R by the reference numerals L and R respectively.

An example of the controller device 20 includes a device main body 210 and a fixture 220 fixed to the device main body 210 as illustrated in FIG. As shown in FIG. 2, the controller device 20 includes a control unit 21, a storage unit 22, an operation interface 23, a sensor unit 24, and a communication unit 25. In the present embodiment, the circuit unit is housed in the device main body 210.

In the following description, the longitudinal direction of the device body 210 is the Z axis (when the device body 210 is fixed to the hand, the direction crossing the palm, and the side where the user's thumb reaches is the positive direction), and the front side The direction (depth direction) connecting the device body 210 to the hand and the back side when looking at the palm of the hand and the back side (depth direction) is the Y axis, and the left and right direction (device body 210 is fixed to the hand When opened (stretched fingers), the direction from the wrist side to the fingertip side is taken as the X axis (FIG. 1).

The device main body 210 may be the same shape as that attached to the user's left hand and that attached to the right hand. In the example of the present embodiment, the fixing tool 220 has a flexible belt in an annular shape, and the device main body 210 is passed through the fixing tool 220 from the user's index finger (so-called index finger) to the little finger (so-called pink finger). Is fixed to a position where it is brought into contact with the root of the user's thumb (the position corresponding to the MP joint of the little finger from the index finger).

The size of the controller device 20 is such that when the user naturally holds the device body 210, one end thereof is positioned slightly outside the position where the tip of the user's thumb (so-called thumb) reaches, and the other end. However, a certain size is set at a position slightly protruding from the base of the little finger (position corresponding to the MP joint). Even if the user opens his / her hand with the controller device 20 mounted, the controller device 20 is fixed to the user's hand by the fixing tool 220, and therefore does not fall.

The device main body 210 includes a gripping part 211 gripped by a user, an operation part 212, and a position presentation part 213. In one example of the present embodiment, the gripping portion 211 has a substantially polygonal column shape. The operation unit 212 is formed continuously from the grip unit 211 and includes a button operation unit 231 including a plurality of buttons that can be operated with the thumb of the user in the example of FIG. Further, the side surface side of the device main body 210 (the surface facing the finger when the back surface of the device main body 210 is brought into contact with the palm of the user) and the button operation unit 231 are respectively provided with a sensor unit 24 described later. A finger sensor 241 is provided.

The position presentation unit 213 is arranged on the back side of the device main body 210 (on the thumb side when fixed to the user's hand) and includes at least one light emitting element such as an LED. It is assumed that the position presentation unit 213 emits light of a predetermined color unique to each controller device 20 during operation of the controller device 20. Further, the position presentation unit 213 only needs to be able to detect the position of each controller device 20 such as a color marker unique to each controller device 20 from the outside, and is not necessarily a light emitting element.

In an example of the present embodiment, the finger sensor 241 is an optical sensor, and includes a light emitting unit that emits infrared light and a light receiving unit that detects infrared light, and light emitted from the light emitting unit is an object. The distance to the obstacle in the infrared radiation direction is detected by measuring the ratio of the light reflected and incident on the light receiving unit. Specifically, here, the finger sensor 241 provided in the button operation unit 231 detects the distance to the thumb (thumb) for operating the button of the button operation unit 231. Also, finger sensors corresponding to the respective fingers from the index finger to the little finger provided on the side surface of the device body 210 (the surface facing the finger when the back surface of the device body 210 is in contact with the palm of the user) 241 detects the distance to the corresponding finger (the obstacle in the direction in which it should be). The finger sensor 241 then outputs the detected distance information as detection result information.

In this example, when the user moves the hand from the state in which the device main body 210 is fixed to the hand and the index finger is extended from the little finger to the shape in which the device main body 210 is gripped, the finger sensor 241 is in the state in which the finger is extended. The distance to the object in the direction of the tip of the finger (for example, the floor surface if the user extends the finger downward) will be detected.

Further, when the user's finger is bent, the distance to the surface of the second joint or the third joint of the bent finger is detected, and when the user grasps the device body 210, the finger sensor 241 receives the finger surface. Therefore, the distance detected by the finger sensor 241 is “0”. In the present embodiment, the finger sensor 241 functions as a sensor that detects the bending / extending state of the finger. However, the sensor for detecting the bending / extending state of the finger is not limited to the optical finger sensor as described here as long as the state of bending / extending (bending or extending) of each finger can be detected.

In the present embodiment, the control unit 21 is a program control device such as a CPU, and operates according to a program stored in the storage unit 22. In the present embodiment, the control unit 21 receives input of information representing the content of the operation performed by the user on the operation unit 212 from the operation interface 23 and outputs the information to the information processing apparatus 1 via the communication unit 25. . Further, the control unit 21 outputs information output from the sensor included in the sensor unit 24 to the information processing apparatus 1 via the communication unit 25.

The storage unit 22 is a memory device or the like, and holds a program executed by the control unit 21. This program may be provided by being stored in a computer-readable and non-transitory recording medium and may be copied to the storage unit 22 or provided via a communication means such as a network. It may be stored in the unit 22. The storage unit 22 also operates as a work memory for the control unit 21.

The operation interface 23 outputs information representing the content of the operation performed by the user at the operation unit 212 to the control unit 21. The sensor unit 24 includes at least one sensor and outputs information output from the sensor to the control unit 21. In an example of the present embodiment, the sensor unit 24 may include an inclination sensor 242 and an acceleration sensor 243 that detect the inclination of the device main body 210 of the controller device 20 in addition to the finger sensor 241 described above.

Here, the tilt sensor 242 detects information on the angle with respect to the direction of gravity of the Z axis in the longitudinal direction of the device body 210, and calculates information on the elevation angle φ from the horizontal plane of the Z axis of the device body 210. The tilt sensor 242 also detects information about the rotation angle θ around the Z axis, and outputs detection result information including the rotation angle θ around the Z axis and the elevation angle φ of the Z axis from the horizontal plane. The acceleration sensor 243 detects the acceleration of the controller device 20 in the X, Y, and Z axis directions, and outputs detection result information representing the detection result. In addition, the acceleration sensor 243 is used when the hand (including a finger) wearing the controller device 20 hits another object (the other hand, another person's hand, a wall, etc.) and examines the impact (time change of acceleration) When the acceleration increases in a pulse shape), the impact is detected, and detection result information indicating that the impact has been detected is output.

The communication unit 25 is a wired interface such as a USB interface or a wireless interface such as Bluetooth (registered trademark), and outputs various information to the information processing apparatus 1 according to instructions input from the control unit 21. .

As illustrated in FIG. 3, the information processing apparatus 1 includes a control unit 11, a storage unit 12, an interface unit 13, an imaging unit 14, and an output unit 15. The control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12.

In this embodiment, the control unit 11 is connected to a pose database that associates pose specifying information for specifying a pose of a hand with process specifying information indicating processing to be performed for at least one hand pose. The control unit 11 also receives detection result information representing the detection result of the finger bending / extension state, the detection result of the hand angle, and the detection result of the impact from the controller device 20. When the hand pose represented by the received detection result information matches one of the hand poses specified by the pose specifying information registered in advance in the pose database, the control unit 11 The process specifying information registered in association with the specific information is acquired, and the process specified by the process specifying information is executed. Detailed operation of the control unit 11 will be described later.

The storage unit 12 is a memory device, a disk device, or the like, and holds a program executed by the control unit 11. The program may be provided by being stored in a computer-readable non-transitory recording medium and stored in the storage unit 12. Further, it may be provided via a communication means such as a network and stored in the storage unit 12. The storage unit 12 also operates as a work memory for the control unit 11.

In the present embodiment, as illustrated in FIG. 4, the storage unit 12 includes, for at least one hand pose, pose specifying information (P) for specifying a hand pose and processing specifying information (P) indicating processing to be performed ( Q) is associated with the pose database. Here, the pose identification information (P) includes information relating to the bending / extending state of each finger from the thumb detected by the finger sensor 241 (whether it is bent or stretched) and the hand angle (rotation angle around the Z axis). This is a determination condition using at least one of the range and the information relating to the elevation angle of the Z axis with respect to the horizontal plane.

For example, the pose identification information for identifying a so-called thumbs-up pose is “the thumb is extended, the index finger, middle finger, ring finger, and little finger are bent, and the Z-axis is vertically upward”. It can be described as a condition. The pose where the index finger and middle finger form a V-shaped sign is that the thumb, ring finger, little finger are bent, the index finger and middle finger are stretched, and the Z axis is in the range of ± 10 degrees from the horizontal direction. Can be described as

The process specifying information (Q) specifies, for example, a process related to a character hand drawing process. As an example, the process specifying information (Q) is information specifying a process for discarding the received detection result information (in this case, each finger of the character remains in the pose of the immediately preceding hand).

The interface unit 13 is connected to the controller device 20 wirelessly or in a wired manner. Information indicating the contents of the user's operation from the controller device 20 and information on the distance detected by the finger sensor 241 (the finger sensor 241 corresponds to each finger). If there are a plurality of distances, information on a plurality of distances detected by the finger sensor 241 corresponding to each finger is received and output to the control unit 11.

The imaging unit 14 is a camera or the like installed with the range where the user is positioned as the imaging range, and repeatedly captures images including the user at predetermined timings, and outputs data of the images to the control unit 11. The output unit 15 has an interface for outputting video and the like to a home television, such as an HDMI (registered trademark) interface. The output unit 15 outputs information on a video to be displayed in accordance with an instruction input from the control unit 11.

Next, the operation of the control unit 11 of this embodiment will be described. The control unit 11 according to the present embodiment functionally includes a receiving unit 111, a determination unit 112, and a process execution unit 113 as illustrated in FIG.

Here, the receiving unit 111 receives the detection result of the bending / extending state of the finger and the detection result of the hand angle at every predetermined timing (for example, every time a certain time elapses) from the controller device 20. In addition, when the controller device 20 detects an impact, the receiving unit 111 receives information indicating that the impact has been detected.

The discriminating unit 112 discriminates the hand pose based on the information received by the receiving unit 111. Specifically, the determination unit 112 determines whether each finger is bent or stretched based on the detection result information of the bending state of the finger received by the receiving unit 111. As described above, in the detection result information output from the controller device 20, when the finger is bent, the detection result information (distance to the finger) of the finger sensor 241 corresponding to the bent finger is compared. The value is close to the target “0”. On the other hand, when the finger is extended, the detection result information (distance to the finger) of the finger sensor 241 corresponding to the bent finger is a relatively large value. Therefore, the determination unit 112 compares the detection result information of each finger with a predetermined threshold value, and when the detection result information is larger than the threshold value, the corresponding finger is stretched. Is determined. If the detection result information is less than the threshold value, it is determined that the corresponding finger is bent.

Accordingly, the determination unit 112 determines whether each finger of the user's hand to which the controller device 20 is fixed is stretched (hereinafter referred to as “extension”) or bent (hereinafter referred to as “bending”). .

Further, the determination unit 112 refers to the pose database held by the storage unit 12 and determines the bending / extension state of each finger and the angle of the hand determined based on the detection result information of the bending / extension state of the finger received by the receiving unit 111. The pose specifying information that matches the detection result information (the pose specifying information specifying the pose determined by each detection result information) is searched. When the matching pose specifying information is found as a result of the search, the determining unit 112 reads the process specifying information (Q) associated with the pose specifying information (P) and outputs it to the process executing unit 113.

As an example, from the detection result information of the bending state of the finger received by the receiving unit 111, it is determined that “the thumb, the index finger, the middle finger, the ring finger, and the little finger are stretched, bent, bent, bent, bent, respectively”, and the angle of the hand is determined. When the detection result information to be expressed indicates that “Z-axis is vertically upward”, the determination unit 112 satisfies the condition of the pose specifying information corresponding to the so-called thumb-up pose (“the thumb is extended and the index finger is extended). The middle finger, the ring finger, and the little finger are bent and the Z-axis is vertically upward ”. Then, the process specifying information associated with the thumb up pose specifying information is read out and output to the process executing unit 113.

Further, the determination unit 112 determines that there is no pose specifying information related to a condition that the information received by the receiving unit 111 satisfies in the pose database held by the storage unit 12 (no matching pose specifying information can be found by the search). Sometimes, information indicating that there is no corresponding pose is output to the pose database.

The process executing unit 113 executes a process specified by the process specifying information output from the determination unit 112. The process executed here is, for example, a process for controlling the pose instructed by the user not to be reflected on the character, or a process for concealing by a predetermined method such as applying a Gaussian blur to the image portion on which the character's hand is displayed. In addition to the above, various processes such as a process of generating a sound and a process of synthesizing a predetermined image may be used.

Further, when the determining unit 112 outputs information indicating that there is no pose corresponding to the pose database, the process executing unit 113 executes a predetermined process. For example, the processing execution unit 113 controls the pose of the character in the virtual space so that the finger bending / extending state, the hand bending angle, and the hand angle received by the receiving unit 111 are the same.

[Consideration of elapsed time]
In the description so far, the determination unit 112 searches for the pose specification information that matches the detection result of the bending / extension state of the finger and the detection result of the hand angle received by the reception unit 111, and the pose specification information that matches the detection result. In the above description, the process specifying information (Q) associated with the pause specifying information (P) is immediately read out and output to the process execution unit 113. However, the present embodiment is not limited to this. Absent.

That is, when the determination unit 112 has the same pose specification information determined to match the detection result received by the receiving unit 111 for a predetermined time (assumed to be received a plurality of times within this time). That is, the detection result information received by the receiving unit 111 a plurality of times during a predetermined time period is the same hand among the poses of any hand specified by the pose specification information stored in the pose database. When it is determined that the pose matches (the condition relating to the same pose specification information is satisfied), the process specification information associated with the pose specification information for specifying the matched hand pose is acquired, and the process execution unit It is good also as outputting to 113.

According to this example, for example, when a pose specified by any one of the pose specifying information is temporarily formed while the user is changing the pose of the hand, the process specifying associated with the pose specifying information is specified. It is possible to prevent the process specified by the information from being executed, and to prevent the extra process from being performed in the process of changing the pose of the character.

[Operation]
Next, a basic operation example of the information processing apparatus 1 according to the embodiment of the present invention will be described. In the following example of the present embodiment, in the pose database of the storage unit 12, “the thumb is extended, the index finger, the middle finger, the ring finger, and the little finger are bent, and the Z axis is vertically downward. In response to the pose specifying information for specifying the pose indicating the condition “so-called thumbs-down pose”, each character's finger is moved to the hand pose (indicated by the finger sensor 241 of the user's finger detected by the user). It is assumed that the process specifying information for specifying the process of continuing the pose of the immediately preceding hand is stored in association with it as if there was no instruction without setting to the (bending / extending state).

Here, when the user who fixed the controller device 20 to the left and right hands, for example, stretches the thumb of the right hand and bends the other index finger to the little finger to hold the controller device 20 and points the thumb vertically upward, The plurality of finger sensors 241 of the controller device 20 output detection result information indicating the distance from the corresponding thumb to the little finger. In addition, the tilt sensor 242 of the sensor unit 24 outputs detection result information indicating that the Z-axis direction (the longitudinal direction of the controller device 20 and the user's thumb side) is vertically upward as the hand angle. .

The control unit 11 of the information processing apparatus 1 includes a variable p representing pose identification information searched last time, which is initially initialized to “” (blank character string), and a counter variable c which is initially initialized to “0”. 6 is executed, and the detection result information output from the controller device 20 is accepted (S11). The control part 11 produces | generates the information showing the bending / extension state of each finger | toe based on the detection result information of the finger sensor 241 among the detection result information received by process S11 (S12). Here, the control part 11 will produce | generate the information that it exists in the bending / stretching state of "elongation, bending, bending, bending, bending".

Then, the control unit 11 can search the pose database for pose specifying information including a condition that satisfies the information indicating the bending state and the information on the hand angle among the detection results received in the process S11. (S13). In this example, the pose database does not include a condition that “the thumb is extended, the index finger, the middle finger, the ring finger, and the little finger are bent, and the Z axis is vertically upward”. Therefore, the control unit 11 determines that the pose identification information cannot be found by the search (S13: No), and executes a predetermined process. Here, the control unit 11 initializes the variable p to “” (blank character string) and the counter variable c to “0”, and as an example, the bending / extension of the finger is the same as the bending / extension state of the finger or the angle of the hand received in step S11. The pose of the character in the virtual space is controlled so as to be in the state and the hand angle (S14), and the process returns to the process S11 to continue the process.

Next, when the user extends the thumb of the right hand and bends from the other index finger to the little finger and grips the controller device 20, when the thumb is directed vertically downward, the plurality of finger sensors 241 of the controller device 20 correspond to each other. Detection result information indicating the distance from the thumb to the little finger is output. In addition, the tilt sensor 242 of the sensor unit 24 outputs detection result information indicating that the Z-axis direction (the longitudinal direction of the controller device 20 and the user's thumb is present) is vertically downward as the hand angle. .

In this case, based on the detection result information received in step S11, the control unit 11 obtains information indicating that the finger bending / stretching state is “stretching, bending, bending, bending, bending, bending” in step S12. Generate. And in the process S13, the condition that “the thumb is extended, the index finger, the middle finger, the ring finger, and the little finger are bent, and the Z axis is vertically downward” is included in the pose database. It is determined that the pose specifying information for specifying the so-called thumbs-down pose has been retrieved (S13: Yes).

Then, the control unit 11 determines whether or not the previously searched pose specifying information (represented by the variable p) matches the pose specifying information searched this time in the process S13 (S15). If the pose specifying information searched last time matches the pose specifying information searched this time in step S13 (S15: Yes), the counter variable c is incremented by "1" (S16), and the counter variable c It is determined whether or not exceeds a predetermined threshold value (count threshold value) (S17). Here, if it is determined that the counter variable c has exceeded a predetermined count threshold value (S17: Yes), the detection result (accepted multiple times within this time) for a predetermined time is met. Since the determined pose specifying information is the same, the control unit 11 stores the processing specifying information (Q) stored in the pose database in association with the searched pose specifying information (represented by the variable p). Is acquired (S18).

And the control part 11 performs the process specified by the process specific information acquired here (S19). For example, here, the shape of the character's hand is set to a predetermined shape (for example, a shape in which all fingers are stretched), and the process returns to step S11 to continue the process.

In process S15, if the pose specifying information searched last time does not match the pose specifying information searched this time in process S13 (S15: No), variable p is searched this time in process S13. The pause identification information is set, the counter variable c is set to “1” (S20), and the process returns to the process S11 to continue the process.

In process S17, when the counter variable c does not exceed the predetermined count threshold value (S17: No), the process proceeds to process S14 and the process is continued.

In the example here, the controller device 20 outputs the detection result information by detecting the bending state of the finger, the hand angle, etc., multiple times per 0.5 seconds. The number of times detection result information is input in 0.5 seconds. In this way, the character poses a thumbs-down pose for a maximum of 0.5 seconds, but after that, the character is changed to a predetermined pose and is a pose that is inappropriate in the scene (here, the thumbs). (Pause down) is prevented.

[If you can specify how your fingers open]
Further, in the present embodiment, when the controller device 20 connected to the information processing apparatus 1 can determine the tilt of the finger for at least some fingers (a part of the finger sensor 241 is not limited to the distance to the object). In the case where the direction of a relatively close target object can also be determined), the pose may be determined using not only the information on the bending / stretching state but also information on the tilt of the finger.

For example, if you can determine the tilt of the index finger, whether the index finger and middle finger are stretched in parallel with the index finger and middle finger stretched (attached to each other), or the index finger and middle finger are stretched to create a V-shape. Can be distinguished. In this case, the pose specifying information is that the pose in which a V-shaped sign is formed by the index finger and the middle finger is “the thumb, ring finger, little finger are bent, the index finger, the middle finger are extended, and the index finger is away from the middle finger. It can be described as a condition that “the Z-axis is in a range of ± 10 degrees from the horizontal direction”, and the control unit 11 is distinguished from a state in which the index finger and the middle finger extend in parallel (a state in which the index finger is not V-shaped). It is possible to execute the process.

[Hand contact judgment]
Furthermore, in an example of the present embodiment, detection result information of the acceleration sensor 243 of the controller device 20 may be used. For example, using the detection result information of the acceleration sensor 243, the pose identification information indicates the high touch pose. “The first to little finger is stretched, and the Z axis is in the range of ± 10 degrees from the horizontal direction. If the acceleration sensor 243 detects an impact ”, it is possible to execute processing corresponding to a pose such as high touch.

[Hand movement information]
Furthermore, the information processing apparatus 1 according to the present embodiment may obtain information on the movement of the user's hand using detection result information from the acceleration sensor 243 and the tilt sensor 242. Since there is a widely known method for acquiring information on the movement of the user's hand using the detection result information of the acceleration sensor 243 and the tilt sensor 242 of the controller device 20, a detailed description thereof is omitted here.

When the user's hand movement information can be acquired in this way, the pose specifying information indicates the high touch pose, “the first to little finger is extended, and the Z axis is ± 10 degrees from the horizontal direction. If the condition is that the user's hand has risen and the acceleration sensor 243 has detected an impact, the information processing apparatus 1 can execute processing corresponding to a pose such as high touch more reliably. Is possible.

[Hand position determination]
In another example of the present embodiment, the position of the hand may be detected and used to determine the pose. That is, the information processing apparatus 1 according to the present embodiment detects the position presentation unit 213 of the device 20 from the image data output from the imaging unit 14.

Specifically, in this device position detection process, the image data output by the imaging unit 14 is received, and the position presentation unit 213 of the controller device 20 fixed to the user's hand is detected from the image data. Since such a method of detecting the position presentation unit is widely known, a detailed description thereof is omitted here.

When the user's hand position information can be acquired in this way, the pose specifying information indicates the high touch pose, “the first to little finger is stretched and the Z axis is ± 10 degrees from the horizontal direction. If the condition is that the user's hand is higher than the user's head and the acceleration sensor 243 has detected an impact, the information processing apparatus 1 can make a pose such as high touch more reliably. Corresponding processing can be executed.

[Pattern registration by application]
Note that the information processing apparatus 1 according to the present embodiment executes an application program, and performs processing for registering / deleting pause specifying information and processing specifying information associated therewith in the pause database in accordance with an instruction of the application program. May be.

According to this, for example, in a scene where a joy expression such as a high touch is inappropriate in a game application or the like, the pose specifying information indicating the high touch and the character do not perform the high touch action (for example, a predetermined upright posture of the character). By registering in association with processing specific information (which is set to a predetermined posture such as), it is possible to perform processing so that an inappropriate pose is not presented according to the scene.

[Game rating, processing considering the region]
Considering that the validity of hand poses also depends on the user's age, residential area, etc., the pose database is used when the pose specification information and the pose specified by the pose specification information are instructed. In addition, a condition for executing a process (hereinafter referred to as an execution condition) and process specifying information may be held in association with each other.

Here, the execution conditions are, for example, the rating information (age restriction information) of the application program being executed, the conditions related to the area in which the application program is being executed (or the language selected for the message display, etc.), etc. The rating may be a condition such as “A” or “B” ”, or a condition that“ the language selected for displaying the message is “Greek” ”. .

In the pose database, for example, in Greece, considering that the pose of the hand with the index finger and middle finger extended and bending the other finger and pointing the palm side toward the other party is socially inappropriate, Executed for pose specific information that represents a pose that presents a sign, such as “the thumb, ring finger, little finger are bent, index finger, middle finger are extended, and the Z axis is within ± 10 degrees from the horizontal direction” The condition is “The language selected for displaying the message is“ Greek ”and the game rating is other than“ Z ””, and the character is instructed to perform the process specified by the process specifying information. Processing specific information that does not present a pose (for example, sets a character to a predetermined posture such as a predetermined upright posture) is associated and registered.

According to this example, during the execution of an online game whose rating is “A”, the user (including the user of the information processing apparatus 1 as well as the opponent of the online game) can display messages. Information on the selected language is acquired, and when “Greek” is included in the acquired language information, it is determined that one of the users (the user of the information processing apparatus 1 is satisfied) as the execution condition is satisfied. Even if an online game opponent (including opponents of online games) takes a pose of presenting a V-shaped sign, the V-shaped sign will be displayed by the user's character even if the V-shaped sign is formed by a hand with the controller device 20 fixed. It is no longer presented and the user of the opponent who selects the Greek language is not inadvertently discomforted.

[Modification]
In the present embodiment illustrated in FIG. 6, the counter threshold value to be compared in the process S17 may be different for each piece of pause specifying information. In this case, the counter threshold value may be registered in the pose database in association with the pose specifying information. In this example, the control unit 11 sets the variable p in the process S20 to the pose specific information searched this time in the process S13, sets the counter variable c to “1”, and searches the current time in the process S13. The counter threshold value registered in association with the pose specifying information may be read and used in step S17.

[Relative position with other devices]
In the description so far, as a method of detecting the position of the controller device 20 (the position of the hand of the user wearing the controller device 20), the controller device 20 provided in the information processing apparatus 1 is imaged. Although the method has been described, in the present embodiment, the method for detecting the position of the controller device 20 may be another method.

As an example, when the user wears a device (for example, a display device such as a head mounted display worn on the head) in addition to the controller device 20, the controller device 20 is based on the relative position with respect to the other device. May be detected.

Specifically, if the other device is a head-mounted display, the information processing apparatus 1 detects the position of the head-mounted display based on the image captured by the imaging unit 14 (such as the angle of the head-mounted display with respect to the vertical direction). At the same time, the position of the controller device 20 is detected.

The information processing apparatus 1 then detects the position of the detected head-mounted display (the optical axis direction of the imaging unit 14 and the vertical direction (the direction of gravity) and a direction orthogonal thereto in a plane whose normal direction is the optical axis direction). Using the position on each axis as a reference position, the coordinates of the relative position of the controller device 20 with respect to the reference position are obtained. In the following, the vertically upward direction is defined as a positive direction.

Since the head mounted display is mounted on the user's head, it can be assumed that the value of the vertical component of the position coordinates of the head mounted display represents the height of the user's head. Therefore, the information processing apparatus 1 has a value obtained by subtracting the value of the vertical component of the position coordinate of the head mounted display from the value of the vertical component of the position coordinate of the controller device 20 exceeds a predetermined threshold value, and When the value is a positive value (the controller device 20 is higher than the head-mounted display), it can be determined that the user's hand is higher than the user's head.

Further, the information processing apparatus 1 determines that the value obtained by subtracting the value of the vertical component of the position coordinate of the head mounted display from the value of the vertical component of the position coordinate of the controller device 20 exceeds a predetermined threshold value. If the value is negative (the controller device 20 is higher than the head-mounted display), it can be determined that the user has lowered his / her hand.

Further, in an example of the present embodiment, the head mounted display may have a camera, and the relative position from the head mounted display to the tip of the hand (or the controller device 20) may be detected.

In this case, when the controller device 20 is located vertically above the head mounted display and at a distance exceeding a predetermined threshold from the image captured by the camera of the head mounted display, the user It can be determined that the hand is raised (the user's hand is higher than the user's head).

Furthermore, although the head-mounted display is taken as an example here as another device, the present embodiment is not limited to this, and the headphone worn on the user's head, the headband, and the wristwatch type worn on the user's arm. The movement of the user's hand may be determined by detecting the relative position of the controller device 20 from various devices.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus, 11 Control part, 12 Storage part, 13 Interface part, 14 Imaging part, 15 Output part, 20 Controller device, 21 Control part, 22 Storage part, 23 Operation interface, 24 Sensor part, 25 Communication part, 111 Acceptance unit, 112 determination unit, 113 processing execution unit, 210 device main body, 211 gripping unit, 212 operation unit, 213 position presentation unit, 220 fixture, 231 button operation unit, 241 finger sensor, 242 tilt sensor, 243 acceleration sensor.

Claims

An information processing apparatus connected to a device fixed to a user's hand,
The device includes a finger sensor that detects a bending / extending state of a finger for at least some of the fingers;
An angle sensor for detecting the angle of the fixed hand;
Including
The information processing apparatus includes:
For at least one hand pose, connected to holding means for holding information specifying the hand pose and processing specifying information indicating processing to be performed in association with each other,
A receiving means for receiving a detection result of a finger bending / extension state by a finger sensor and a detection result of a hand angle from the device;
Based on the received detection result, it is determined whether or not it matches any of the hand poses specified by the information held in the holding means. Determining means for acquiring process specifying information associated with information specifying the pose of the hand that has been performed;
Execution means for executing a process specified by the process specifying information acquired by the determination means;
An information processing apparatus including:
The information processing apparatus according to claim 1,
The receiving means receives a detection result of a finger bending / extension state by a finger sensor and a detection result of a hand angle repeatedly from the device at predetermined timings,
The determination unit determines that all the detection results repeatedly received over a predetermined time coincide with the same hand pose among any of the hand poses specified by the information held in the holding unit. An information processing apparatus that acquires processing specifying information associated with information for specifying the matching hand pose when the determination is made.
The information processing apparatus according to claim 1, wherein:
The information processing apparatus including the processing specification information indicating that the received detection result information should be discarded.
A finger sensor that is fixed to a user's hand and that detects at least a part of the finger and detects a bending / extension state of the finger, and an angle sensor that detects an angle of the fixed hand, and is connected to at least one device. For the hand pose, using an information processing apparatus connected to a holding unit that holds information for specifying the pose of the hand and process specifying information representing the process to be performed,
The acquisition means includes at least a finger detection result of the finger bending / extension state from the device, a hand angle detection result, a hand position detection result, and an acceleration detection result of the device. Obtaining a detection result including a detection result of a bending state and a detection result of a hand angle;
When the determining means determines whether or not it matches any of the hand poses specified by the information held in the holding means based on the acquired detection result, And obtaining process identification information associated with information identifying the matched hand pose,
A step of executing a process specified by the acquired process specifying information;
A method for controlling an information processing apparatus including:
Connected to a device that is fixed to a user's hand and includes at least some fingers, a finger sensor that detects a flexion / extension state of the finger, and an angle sensor that detects an angle of the fixed hand;
For at least one hand pose, an information processing apparatus connected to holding means for holding information specifying the hand pose and processing specifying information indicating processing to be performed in association with each other,
A receiving means for receiving a detection result of a finger bending / extension state by a finger sensor and a detection result of a hand angle from the device;
Based on the received detection result, it is determined whether or not it matches any of the hand poses specified by the information held in the holding means. Determining means for acquiring process specifying information associated with information specifying the pose of the hand that has been performed;
Execution means for executing a process specified by the process specifying information acquired by the determination means;
Program to function as.
A finger sensor that is fixed to a user's hand and that detects at least a part of the finger and detects a bending / extension state of the finger, and an angle sensor that detects an angle of the fixed hand, and is connected to at least one device. An information processing apparatus connected to a holding unit that holds information for specifying a hand pose and processing specifying information indicating processing to be performed in association with the hand pose.
A receiving means for receiving a detection result of a finger bending / extension state by a finger sensor and a detection result of a hand angle from the device;
Based on the received detection result, it is determined whether or not it matches any of the hand poses specified by the information held in the holding means. Determining means for acquiring process specifying information associated with information specifying the pose of the hand that has been performed;
Execution means for executing a process specified by the process specifying information acquired by the determination means;
A computer-readable recording medium storing a program that functions as a computer.