JP5817322B2 - Information processing apparatus, information processing system, and operation device - Google Patents

Description

  This technology realizes the movement of the operation object according to the movement of the operation device based on the information obtained from the movement sensor when the operation device having the movement sensor is operated in the three-dimensional space by the user. The present invention relates to an information processing apparatus, an information processing system, an information processing method, and an operation device that execute an operation for performing the operation.

  The remote operation device described in Patent Literature 1 detects the posture and motion of the housing by a sensor, and outputs a signal for operating an operation target device such as a TV based on the detected posture and motion amount. Specifically, the casing of the remote control device is formed in a cube, and the remote control device outputs an operation command (command) signal corresponding to the combination of the output of the acceleration sensor and the output of the gyro sensor of the remote control device. (For example, refer to paragraphs [0050] to [0052] and FIGS. 4 to 6 of Patent Document 1).

JP 2009-1003006 A

  In recent years, the performance of operation target devices such as TVs, PCs (Personal Computers), and game devices has been improved. As the performance of the operation target device is improved, the contents of operations performed by the remote operation device have become complicated and diversified, and the remote operation device has a large number of fine buttons. Therefore, a situation occurs in which the button corresponding to the operation intended by the user cannot be easily found. In addition, the user needs to select and determine an operation item while viewing a large number of menus on application software having a complicated hierarchical structure.

  In view of the circumstances as described above, an object of the present technology is to provide an information processing apparatus, an information processing system, an operation device, and an information processing method related to an operation device that can improve user operability.

In order to achieve the above object, an information processing apparatus according to the present technology includes an acquisition unit, a storage unit, a detection unit, a generation unit, and a transmission unit.
The acquisition unit acquires motion information detected by the first sensor when the user operates the operating device to rotate with the operating device placed on the operating surface. The operation device includes a housing including a surface having a plurality of operation regions, a first sensor that detects movement of the housing, and a second sensor that can detect an external force applied to the plurality of operation regions.
The storage unit stores information on a plurality of postures of the operation device for defining the plurality of operation areas, and information on a plurality of operation commands respectively corresponding to the plurality of operation areas. .
The detection unit detects an attitude of the operation device operated by the user based on the acquired movement information.
In the second sensor, the generation unit is a region that is not in contact with the operation surface of the surface of the casing having the detected posture, and that an upper side region of the casing is pressed by the user. If detected by the above, an operation command corresponding to posture information that defines an operation region located at a predetermined position on the surface of the housing among the plurality of operation regions is generated.
The transmission unit transmits at least the generated operation command to an operation target device of the operation device.

  In this technology, the posture corresponding to the operation region located at a predetermined position of the upper side region triggered by the user pressing the upper side region, which is a position that is easily seen by the user when the user rotates the operation device. An operation command corresponding to the information is generated. Thereby, user operability can be improved.

  The “first sensor” and the “second sensor” may be the same sensor. The “same sensor” here includes both the meaning that the first sensor also functions as the second sensor and the meaning that these sensors are structurally the same type of sensor.

  The generation unit corresponds to the posture information that defines the vertical operation area, with the vertical operation area through which the vertical axis in the three-dimensional dimension passes as the operation area located at the predetermined position among the plurality of operation areas. The operation command may be generated. Often, the region through which the vertical axis passes in the upper region is a region that is easily visible to the user. Therefore, according to the present technology, user operability can be improved. In this case, even if an area not corresponding to the vertical operation area is pushed in by the user, an operation command corresponding to the vertical operation area is generated.

  The generation unit may generate the operation command corresponding to the posture information that defines the vertical operation area when it is detected that the vertical operation area is pressed by the user.

  The generation unit can generate the operation command corresponding to the posture information that defines at least one operation region adjacent to the vertical operation region in the upper side region. Thereby, the operation corresponding to the surrounding operation area including the vertical operation area can be executed.

  The operation target device may be capable of outputting a plurality of related images respectively related to the plurality of operation areas for display. In that case, the transmission unit causes the operation target device to output a related image related to the operation region located at the predetermined position among the plurality of related images for display based on the motion information. In addition, the acquired motion information or the detected posture detection information may be transmitted to the operation target device. Accordingly, the user can select one operation area from among the plurality of operation areas while viewing an image related to the operation area generated and displayed by the operation target device. This improves operability.

  The transmission unit acquires the motion information acquired or the detected posture in order to selectively highlight a related image related to the operation region located at the predetermined position according to the motion of the operation device. The detection information may be transmitted to the operation target device. This improves visibility and operability for the user.

  Of the surfaces of the casing of the operation device, at least a surface that contacts the operation surface may be a spherical surface.

  The operation device may include a display that is provided on a surface of the housing and displays a plurality of related images respectively associated with the plurality of operation regions. Thereby, the user can operate the operation device while looking at the display.

  The information processing apparatus may further include a stop unit and a release unit. The said stop part stops the output of the detection information of the attitude | position of the said operation device by the said detection part, or stops the transmission to the said operation target apparatus of the said movement information by the said transmission part. The release unit releases the stop of the transmission by the stop unit when a predetermined external force is detected by the second sensor.

The information processing system according to the present technology includes the above-described operation target device including an acquisition unit, a storage unit, a detection unit, a generation unit, a transmission unit, a reception unit, and an output unit. It comprises.
The receiving unit receives at least the transmitted operation command.
The output unit outputs, for display, related images respectively associated with the plurality of operation areas based on the received operation command information.

Another information processing system according to the present technology includes an information processing device including an acquisition unit, a detection unit, a generation unit, and a transmission unit, a storage unit, a reception unit, and an operation target device including an execution unit. It comprises.
The acquisition unit acquires motion information detected by a first sensor when the operation device is operated to rotate by the user while the operation device is placed on the operation surface. The operation device includes a housing, a first sensor that detects movement of the housing, and a second sensor that can detect an external force applied to the housing.
The detection unit detects an attitude of the operation device operated by the user based on the acquired movement information.
In the second sensor, the generation unit is a region that is not in contact with the operation surface of the surface of the casing having the detected posture, and that an upper side region of the casing is pressed by the user. If detected by, an execution command is generated.
The transmission unit transmits the generated execution command and the acquired motion information or the detected posture detection information to the operation target device of the operation device.
The storage unit includes information on a plurality of coordinate ranges represented in a virtual plane area for defining a plurality of operation areas, and information on a plurality of operation commands respectively corresponding to the plurality of operation areas. Store in association with each other.
The receiving unit receives information transmitted from the information processing apparatus.
The execution unit corresponds to one coordinate range of the plurality of coordinate ranges that can be selected according to the received motion information or posture detection information when the execution command is received by the reception unit. Execute the operation command.

The operation device according to the present technology includes a housing, a first sensor, a second sensor, an acquisition unit, a storage unit, a detection unit, a generation unit, and a transmission unit.
The housing includes a surface having a plurality of operation areas.
The first sensor detects the movement of the housing.
The second sensor can detect an external force with respect to the plurality of operation areas.
The acquisition unit acquires motion information detected by the first sensor when the user operates the operating device to rotate with the operating device placed on the operating surface.
The storage unit stores information on a plurality of postures of the operation device for defining the plurality of operation areas, and information on a plurality of operation commands respectively corresponding to the plurality of operation areas. .
The detection unit detects an attitude of the operation device operated by the user based on the acquired movement information.
In the second sensor, the generation unit is a region that is not in contact with the operation surface of the surface of the casing having the detected posture, and that an upper side region of the casing is pressed by the user. If detected by the above, an operation command corresponding to posture information that defines an operation region located at a predetermined position on the surface of the housing among the plurality of operation regions is generated.
The transmission unit transmits at least the generated operation command to an operation target device of the operation device.

Information processing method according to the present technology. An information processing apparatus for an operation device, comprising: a housing including a surface having a plurality of operation regions; a first sensor that detects movement of the housing; and a second sensor that can detect an external force applied to the plurality of operation regions. It is a method to execute.
The information processing method stores information on a plurality of postures of the operation device for defining each of the plurality of operation areas and information on a plurality of operation commands respectively corresponding to the plurality of operation areas. Including doing.
In a state where the operation device is placed on the operation surface, when the user operates the operation device to rotate, motion information detected by the first sensor is acquired.
Based on the acquired movement information, the attitude of the operation device operated by the user is detected.
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. Of the plurality of operation areas, an operation command corresponding to information on a posture that defines an operation area located at a predetermined position on the surface of the casing in the upper side area is generated.
At least the generated operation command is transmitted to the operation target device of the operation device.

  The present applicant did not intend to describe the premise of “having a surface, which is,” with the intention of the prior art, but the description of the premise makes it easy to understand the contents of the present technology. It is a description for making.

  As described above, according to the present technology, it is possible to improve user operability.

FIG. 1 is a diagram illustrating an information processing system according to an embodiment of the present technology. FIG. 2 is a diagram illustrating a hardware configuration of the operation device. FIG. 3 is a diagram illustrating a hardware configuration of the display device. FIG. 4 is a diagram for explaining the detection axes of the sensors of the operation device and their relative arrangement. FIG. 5 is a diagram showing the relationship between the local coordinate system and the global coordinate system. FIG. 6 is a flowchart showing processing by the operation device. FIG. 7 is a diagram for explaining the principle of the arithmetic processing in FIG. FIG. 8 is a diagram for explaining the principle of the arithmetic processing in FIG. FIG. 9 is a diagram for explaining a basic operation method of the operation device by the user. 10A and 10B are diagrams for explaining an operation example 1 by the operation device. FIG. 11 is a diagram for explaining the principle of the setting method when setting the operation content in the information on the angle range, which is the posture information. FIG. 12 is a diagram for explaining an operation example 2 by the operation device. FIG. 13 is a diagram for explaining an operation example 3 by the operation device. FIG. 14 is a diagram for explaining an operation example 4 by the operation device. FIG. 15A is a diagram for explaining an operation example 5 by the operation device. FIG. 15B is a diagram for explaining an operation example 5 by the operation device. FIG. 16 is a diagram for explaining an operation example 7 by the operation device. FIG. 17 is a diagram illustrating an operation example of an operation device according to another embodiment of the present technology. FIG. 18 is a diagram for describing an operation example of the operation device illustrated in FIG. 17. FIG. 19 is a diagram for explaining the principle of operation region detection in the operation step shown in FIG. 20A and 20B show another example for an operating device having an eccentric center of gravity.

  Hereinafter, embodiments of the present technology will be described with reference to the drawings.

  [Configuration of information processing system]

  FIG. 1 is a diagram illustrating an information processing system according to an embodiment of the present technology.

  This information processing system includes an operation device 10 operated by a user, and a display device 50 that receives operation information transmitted from the operation device 10 and executes display processing based on the information. The operation device 10 and the display device 50 are electrically connected, and in this embodiment, communication is performed by radio such as infrared rays or radio waves.

  A part that realizes information processing of the operation device 10 functions as an information processing apparatus. The display device 50 functions as an operation target device.

  The operation device 10 is formed in a size that can be grasped by a hand, and has, for example, a substantially spherical shape. The shape of the operation device 10 is not limited to a spherical shape, and may be a shape that allows the user to recognize the orientation and orientation of the operation device 10 (a shape other than a spherical shape). As will be described later, each item such as a menu screen displayed by the display device 50 can be selected and designated by the user's operation using the operation device 10.

  FIG. 2 is a diagram illustrating a hardware configuration of the operation device 10. The operation device 10 includes an MCU (Micro Control Unit) 1 that functions as a CPU, a power supply 2, a RAM 3, a ROM 4, a communication device 9, an acceleration sensor 5, an angular velocity sensor 6, a magnetic sensor 7, a pressure sensor 8, and the like. The acceleration sensor 5, the angular velocity sensor 6, and the magnetic sensor 7 function as a motion sensor (first sensor) and are detected by at least one of these sensors 5 to 7 out of acceleration, angular velocity, and magnetic intensity. At least one piece of information functions as motion information.

  Further, the operation device 10 includes a rewritable memory or the like (not shown). As an alternative to the MCU 1, programmable hardware such as an FPGA (Field Programmable Gate Array) may be used.

  As the angular velocity sensor 6, for example, a device that detects the angular velocity itself using Coriolis force is used.

  Here, the communication device 9 mainly functions as a transmitter that transmits information generated by the MCU 1.

  The ROM 4 or a memory (not shown) stores software necessary for realizing the present technology.

  These hardware are arranged in a spherical casing 11 so as to be fixed to the casing 11. The surface of the casing 11 has a spherical shape as described above.

  For example, a plurality of pressure sensors 8 are attached near the surface of the housing 11. These pressure sensors 8 function as sensors (second sensors) that detect external forces on the surface of the casing 11.

  For example, at least four pressure sensors 8 are provided at equiangular intervals in the hemispherical portion (eight in the whole sphere). However, the number and arrangement of the pressure sensors 8 can be set as appropriate, and the pressure sensors 8 are arranged so that the force can be detected no matter where the user presses. Further, the pressure sensor 8 is configured to detect the pressed position with a predetermined accuracy regardless of the position pressed by the user.

  For example, a strain gauge, a piezoelectric sensor, or the like is used as the pressure sensor.

  FIG. 3 is a diagram illustrating a hardware configuration of the display device 50. The display device 50 includes a CPU 53, a ROM 54, and a RAM 55, as well as a general computer, and also includes a display unit 52, a communication unit 56, and a storage unit 57. Here, the communication unit 56 mainly functions as a receiver. The storage unit 57 is typically an auxiliary (secondary) storage unit for the ROM 54 and the RAM 55.

  The display device 50 is integrated with the display unit 52 and a device (operation target device) that receives the information transmitted from the operation device 10 and generates and outputs an image for display. It has a configuration. However, they may be connected to each other so that they can be communicated separately by wire or wirelessly.

  The ROM 54 and the storage unit 57 of the display device 50 store software for realizing the present technology.

  Typically, in this information processing system, information detected by the sensors 5 to 7 of the operation device 10 (obtained by processing it with the MCU 1) and information of operation commands generated by the calculation of the MCU 1 are included. And transmitted to the display device 50 via the communication device 9 (see FIG. 2). In this case, at least one of the MCU 1 and the communication device 9 functions as a transmission unit. Then, the display device 50 receives the information via the communication unit 56 (see FIG. 3), and executes a predetermined process based on the information.

  FIG. 4 is a diagram for explaining the detection axes of the sensors of the operation device 10 and their relative arrangement. The acceleration sensor 5, the angular velocity sensor 6, and the magnetic sensor 7 all have three orthogonal detection axes. That is, the acceleration sensor 5 has sensors (5a, 5b, 5c) corresponding to three detection axes. Similarly, the angular velocity sensor 6 and the magnetic sensor 7 have sensors (6a, 6b, 6c) and (7a, 7b, 7c), respectively.

  For example, all of these sensors 5, 6 and 7 are packaged in a common package. Alternatively, these sensors 5, 6 and 7 are packaged by separate packaging and mounted on a common sensor substrate 20 (see FIGS. 5, 7 and the like).

  FIG. 5 is a diagram showing the relationship between the local coordinate system and the global coordinate system.

  A display device 50 is placed on the ground. Here, the coordinate system fixed to the ground or the display device 50 is referred to as a global coordinate system. A coordinate system that can freely move with respect to the global coordinate system and is fixed to the sensor substrate 20 of the operation device 10 is referred to as a local coordinate system. As described above, the sensor substrate 20 is a common substrate on which the acceleration sensor 5, the angular velocity sensor 6, and the magnetic sensor 7 are mounted.

  For convenience of explanation, the global coordinate system is represented by uppercase letters (X, Y, Z) and the local coordinate system is represented by lowercase letters (x, y, z). However, in order to make the sentence easier to understand, explanations will be made using expressions of “local” and “global” as much as possible. In the global coordinate system, the ground is the XY plane, and the plane parallel to the principal plane of the local coordinate system is the xy plane.

  [Process by operation device]

  FIG. 6 is a flowchart showing processing by the operation device 10. This process is realized by cooperation between software resources stored in the storage device (ROM or RAM) of the operation device 10 and hardware resources such as the MCU 1. Hereinafter, for the sake of convenience, the subject of processing will be described as MCU1.

  Since the outer shape of the operation device 10 is a spherical shape, the MCU 1 needs to recognize the posture of the operation device 10 at the present time because the upper, lower, left, and right of the operation device 10 are not determined. MCU1 acquires the information detected by each sensor 5-7, and MCU1 functions as an acquisition part in this case.

At the start of the operation by the operation device 10, in step 201, based on the information on the acceleration detected by the acceleration sensor 5 (particularly, acceleration (a _x , a _y ) in the x and y axis directions), the global X And an initial posture angle around the Y axis are calculated. FIG. 7 is a diagram for explaining the principle of the arithmetic processing, and shows a state where the inclination (posture) of the sensor substrate 20 is inclined in the global coordinate system.

For example, when calculating the initial posture angle around the global Y axis, that is, the angle θ _x from the X axis, it is calculated by the following expression 1 based on the information of the acceleration a _x . In Equation 1, as shown in FIG. 8, for example, the value of gravity 1G detected by the x-axis acceleration sensor 5a is A _xG and the sensor substrate 20 is tilted (the acceleration sensor 5 (5a) is tilted). in, and the value of the acceleration sensor 5a and a _x.

Similarly, based on the information on the acceleration a _y , the initial posture angle around the global X axis, that is, the angle θ _y from the Y axis is calculated by Equation 2. The value of the y-axis acceleration sensor 5b with respect to gravity 1G is assumed to be _AyG .

A _xG , A _yG : Gravitational acceleration 1G detected by the x and y axis acceleration sensors 5a and 5b, respectively.
a _x , a _y , a _z : Current values (local acceleration) detected by the x, y, z axis acceleration sensors 5a, 5b, 5c, respectively.
θ _x , θ _y : Initial posture angle of the sensor board in the global coordinate system from the X and Y axes (θ _x is the angle around the Y axis (pitch angle), θ _y is the angle around the X axis (roll angle) ).)

  Although the sine is used in the equations 1 and 2, the initial posture angle can be calculated also by a cosine or another calculation method.

  Next, in step 202, based on the information calculated in step 201 and the magnetic intensity information in each axial direction in the local coordinate system detected by the magnetic sensor 7, the initial value around the global Z axis of the operating device 10 is displayed. An attitude angle (azimuth) is calculated.

The following formula 3 is used for this calculation. In Expression 3, the magnetic intensities (local magnetic intensities) detected by the x, y, and z axis magnetic sensors 7a, 7b, and 7c are h _x , h _y , and h _z , respectively. In addition, the magnetic strengths (global magnetic strengths) in the global X, Y, and Z axis directions obtained by the calculation are H _x , H _y , and H _z .

h _x , h _y , h _z : magnetic strength in the x, y, z axis directions in the local coordinate system (local magnetic strength)
H _x , _Hy : Global X and Y axis magnetic strength (global magnetic strength)
θ _z : initial posture angle (azimuth) around the global Z axis (θ _z is the yaw angle)

  By using the magnetic sensor 7 that detects geomagnetism in this way, the operation device 10 can recognize the orientation of the sensor substrate 20 around the global Z axis. In the case of calculating Equations 1 to 3, the MCU 1 and software describing the calculation contents function as an initial posture angle calculation unit.

  As described above, the operation device 10 can recognize the initial posture (inclination with respect to the X, Y, and Z axes) of the sensor substrate 20 in the global coordinate system by the arithmetic processing in steps 201 and 202. That is, the operation device 10 can recognize its own posture without the user being aware of how to grip the operation device 10 and its direction. As a result, the user can start the operation using the operation device 10 in an arbitrary posture within the two-dimensional plane.

Next, in step 203, the angular velocity detected by the angular velocity sensor 6 at the start of the operation, that is, immediately after the start of the operation, based on the information of the initial posture angles (θ _x , θ _y , θ _z ) calculated in steps 201 and 202. Is converted to a global angular velocity in the global coordinate system. This is a rotational coordinate transformation. Formula 4 is used for this calculation process. In this case, the MCU 1 and the software describing the calculation contents function as a coordinate conversion unit.

w _x , w _y , w _z : Angular velocity around the local x, y, z axis (local angular velocity)
W _x , W _y , W _z : Angular velocities around the global X, Y, and Z axes (global angular velocities)

In step 204, an angle _Rx around the global X axis is calculated using the global angular velocity thus calculated. That is, the global angle (its X-axis direction component) is calculated. Formula 5 is used for this calculation process. This calculation process uses a trapezoidal integration method. The angles R _y and R _z around the global Y and Z axes are also calculated in the same manner as R _x . The meaning of the processing in step 204 is to change the initial posture angle (θ _x , θ _y , θ _z ) to a new posture angle (R _x , R _y , R _z ), that is, update the posture angle. . In this case, the MCU 1 and the software in which the calculation contents are described function as an update unit.

R _x : Angle around the X axis in the global coordinate system (global angle (global attitude angle))
(t _n ): means the _nth value obtained Δt = t _n -t _n-1

When the global angle is calculated in this way, this information is transmitted to the display device 50 by the communication device 9 (step 205). At this time, this global angle is the initial (ie, at the start of the operation) global angle. The display device 50 receives this information. The display device 50 sets values obtained by multiplying (R _y , R _x ) by constants among the initial global angles (R _x , R _y , R _z ) on the screen of the display unit 52 and other pointers and other values. Assigned to coordinate values (X, Y) for generating an operation target image including an image or the like. In this way, the initial posture of the operation device 10 corresponds to the initial position and initial posture of the operation target image. In this case, the MCU 1 functions as a posture detection unit for the operation device.

  When the display device 50 receives information on the second and subsequent global angles, the display device 50 displays an operation target image having a position or orientation on the screen according to the global angle on the display unit 52. In this case, the position of the operation target image changes or the operation target image itself changes according to the rotation angle of the user's operation device 10.

  In this way, the MCU 1 continues to generate information on the global angle according to the movement of the operation device 10 and continues to recognize the attitude of the operation device 10. Also in this case, the MCU 1 functions as a posture detection unit of the operation device.

  Note that the global angle can be obtained by various integration processes such as the midpoint method and the Simpson method, without being limited to the trapezoidal integration method as shown in Equation 5.

Here, initially, in step 203, the coordinate conversion unit executes the coordinate conversion process for the local angular velocity (initial value) based on the information on the initial posture angles (θ _x , θ _y , θ _z ). However, in the processing of step 203 after the second time (other than the initial time), the coordinate conversion unit determines the local angular velocity (after the second time) based on the information on the global angles (R _x , R _y , R _z ) calculated in step 204 The coordinate conversion process is executed for the value of. The following formula 6 is used for this calculation process.

  The global angle of the operation device 10 changes gradually (every moment). Therefore, specifically, in Equation 6, rotational coordinate conversion is performed based on a value obtained by sequentially adding global angles as shown in the following equation.

R _x '= R _x (t _n ) + R _x (t _{n + 1} )
R _y '= R _y (t _n ) + R _y (t _{n + 1} )
R _z '= R _z (t _n ) + R _z (t _{n + 1} )

However, (R _x , R _y , R _z ) is used instead of (R _x ′, R _y ′, R _z ′) in Equation 6 depending on the software specifications for image display by the display device 50. May be.

  As described above, if the initial posture angle is calculated first, the global angle calculated in step 204 is used in the second and subsequent calculations without using the initial posture angle. Hereinafter, this reason will be described.

  When the user starts operating the operation device 10, the local angular velocity is converted into the global angular velocity using information on the initial posture angle calculated based on information including the acceleration obtained by the acceleration sensor 5. That is, at the moment when the operation is started, the acceleration sensor 5 substantially detects only the gravitational acceleration, the initial posture angle is calculated based on the information including the gravitational acceleration, and the coordinate conversion is performed based on the initial posture angle. Done.

  However, during subsequent user operations, the acceleration sensor 5 detects a value obtained by adding motion acceleration (inertial acceleration) to gravity acceleration. That is, during the user's operation, the posture of the sensor substrate 20 changes every moment. Therefore, if the posture angle is calculated based on the acceleration including the motion acceleration obtained by the acceleration sensor 5 during the user's operation, an error may occur.

  Therefore, in the present embodiment, the local angular velocity acquired after the local angular velocity acquired as the initial value is information that does not include motion acceleration information, that is, the posture angle updated using the local angular velocity at least once. Conversion processing is executed based on information (information obtained through steps 203 and 204 at least once). This is because the local angular velocity value is not affected by the motion acceleration.

  According to such a calculation process, it is possible to suppress a situation in which the influence of the motion acceleration generated during the operation of the operation device 10 by the user affects the calculation of the attitude angle of the operation device 10 and causes an error in the calculation. be able to.

  As described above, the MCU 1 can detect the attitude of the operation device 10 in the global coordinate system. Thereby, as will be described later, the MCU 1 is a region that does not come into contact with the operation surface on the surface of the housing, and can detect the position of the upper side region (described later) of the housing in the three dimensions. Further, since the MCU 1 can detect the attitude of the operation device 10, the display device 50 can change the operation target image displayed on the display unit 52 as described above.

  In this process, until the output from the angular velocity sensor 6 becomes substantially 0, that is, until the movement of the operation device 10 substantially stops, the MCU 1 uses the global angle calculated in step 204 to Calculate global angular velocity. Processing after the movement of the operation device 10 substantially stops starts from Step 201.

The information on the global angles (R _x , R _y , R _z ) calculated in step 205 is also used for the following processing. That is, the coordinate conversion means converts the local acceleration into the global acceleration by using the global angle information as in Expression 6. This calculation process uses the following Expression 7. When the user moves the operation device 10 away from the operation surface and performs a translation operation in the air, the MCU 1 executes the calculation process of Expression 7. Thereby, the user can use the operation device 10 as a space operation device.
A _x , A _y , A _z : Acceleration around the global X, Y, and Z axes (global acceleration)
As described above, also in the process of converting the local acceleration into the global acceleration, the coordinate conversion unit performs the coordinate conversion based on the information on the global angle except in the initial stage.

  [Operation method of operation device]

(Basic operation method)
FIG. 9 is a diagram for explaining a basic operation method of the operation device 10 by the user.

  In the present technology, as illustrated in the upper part of FIG. 9, the operation device 10 is placed on the operation surface 90, and the operation device 10 is operated to rotate in a state where the casing 11 is in contact with the operation surface 90. Hereinafter, this operation may be referred to as a “first operation step”.

  Here, the rotation means that the operation device 10 does not rotate once, that is, includes rotation (tilt) of less than 360 degrees. The operation surface 90 may be anything such as a floor surface, a table surface, a desk surface, etc., and these need not be flat.

  The user rotates the operation device 10 as described above while looking at the display unit 52 of the display device 50. Thereby, as will be described later, for example, a menu item displayed on the display unit 52 can be selected.

  Then, as shown in the lower left, lower center, or lower right of FIG. 9, the user selects by pressing at least a part of the upper region of the casing 11 that is an area that does not contact the operation surface 90 of the casing 11. Specify (or execute) the selected item. Hereinafter, this operation may be referred to as a “second operation step”.

  In the state shown in FIG. 9, the upper side region is a region of the upper hemisphere in the surface of the housing 11 or a region narrower than that and further on the upper side.

  The figure shown in the lower left of FIG. 9 shows an example in which the pressure sensor 8 detects the applied pressure when the user presses the upper region of the housing 11.

  9 illustrates an example in which the acceleration sensor 5 detects the excitation force generated in the operation device 10 when the user hits the upper side region of the housing 11. In this way, the action of the user “hitting” the casing 11 is mechanically equivalent to the action of “pushing (applying force)”. In this case, the vibration may be detected by an acceleration sensor instead of the pressure sensor 8. In that case, it functions as a vibration sensor that detects the presence or absence of vibration.

  9 illustrates an example in which the user touches the operation area 15 to be selected that is located at a predetermined position in the upper area of the housing 11. This “touching” action is mechanically equivalent to a “pushing” action, and is equivalent to an action in which an external force is applied. In this case, an electrostatic sensor may be used as the second sensor instead of the pressure sensor 8.

  Hereinafter, various operation examples using the first and second operation steps will be specifically described.

(Operation example 1)
10A and 10B are diagrams for explaining an operation example 1 by the operation device 10.

  The surface of the housing 11 of the operation device 10 according to this operation example 1 has a plurality of operation areas 15. The housing 11 has a soccer ball-like shape similar to a truncated icosahedron, and the surfaces defined by pentagons and hexagons correspond to the operation areas 15, respectively.

  In addition, although each regular pentagon and each regular hexagonal surface in the truncated icosahedron are configured by planes, the surface of the casing 11 of the operation device 10 is substantially formed by a spherical surface. However, the casing 11 of the operation device 10 may be a truncated icosahedron formed of a regular pentagon and a regular hexagon.

  Each operation area 15 has various operation contents such as “CH 1”, “CH 2”,... For selecting a TV broadcasting station, “VOL +”, “VOL-” for volume adjustment, and the like. Each character is printed. In addition, in the ROM 4 of the operation device 10 or a memory (not shown), information on a plurality of postures of the operation device 10 (information on angle ranges to be described later) for defining each operation region 15 and the operation regions 15 are provided. Information of a plurality of corresponding operation commands is stored in association with each other. The operation command is a command for executing various processes corresponding to the operation area 15 such as channel selection and volume adjustment described above.

  Information on a plurality of postures of the operation device 10 for defining each operation region 15 will be described later.

  In the present embodiment, when the user selects an operation content to be operated (operation content to be selected), the operation region 15 corresponding to the operation content is located at a predetermined position on the surface of the housing 11 in the upper side region. As shown, the first operation step is performed. As described with reference to FIG. 6, the operation device 10 transmits global angle information to the display device 50.

  The display device 50 receives the information, and generates a related image 58 related to at least a part of the operation area 15 in the upper area based on the information, and outputs the related image 58 for display. In this case, the CPU 53 functions as an output unit. In order to realize this, the display device 50 includes a plurality of related images 58 corresponding to each operation region 15 (or information on a plurality of postures of the operation device 10 for defining each operation region 15). You only have to memorize the information.

  A typical example of this technique is shown in FIG. 10A. The user positions the operation area 15 with a predetermined channel number vertically upward, that is, the position through which the vertical axis V passes in three dimensions is set as the predetermined position, and the operation content to be selected by the user is corresponding to that position. The rotation operation is performed so as to position the operation area 15 to be operated. At this time, the display device 50 displays a related image 58 related to the operation region 15 positioned vertically upward (hereinafter referred to as the vertical operation region 15V) in accordance with the movement of the operation device 10 by the rotation operation of the user. Generate this and output it for display.

  As shown in FIG. 10A, the related image 58 may be an image having substantially the same content as the characters printed on the operation area 15 on the casing 11, or the characters, symbols, and the like related thereto. It may be an image. The related image 58 may be anything as long as the user can recognize that the related image 58 is an image showing the operation region 15 that the user intends to select from among the plurality of operation regions 15.

  Here, for example, immediately after the power source 2 of the operation device 10 is turned on, the posture of the casing 11 of the operation device 10 is detected in steps 201 and 202 shown in FIG. 6 as described above. Further, after that, when the first operation step is performed by the user, as described above, the MCU 1 repeats the processes of steps 203 to 205 to detect the posture of the casing 11 of the operation device 10.

  Thus, MCU1 can detect the attitude | position of arbitrary operation devices 10, and can specify the upper side area | region of the housing 11 by this. That is, the MCU 1 is located at the upper part of the position of the surface of the housing 11 in the current posture, or more specifically, the position of the surface of the housing 11 is located at which position in three dimensions. Can be specified. For example, the inclination angle of the sensor substrate 20, that is, the global angle (or initial posture angle) is calculated as described above with reference to the state where the global coordinate axis and the local coordinate axis of the sensor substrate 20 coincide with each other. This is possible.

  Specifically, as shown in FIG. 10A, the MCU 1 has operation contents corresponding to information on an angle range (that is, posture information) that defines an operation area 15 passing through the vertical axis V that is a predetermined position in the upper area. Can be specified (an operation command is generated).

  Next, when the 2nd operation step by a user is performed (refer FIG. 10B), MCU1 determines whether the force by the pressing operation by the pressure sensor 8 exceeds a threshold value.

  At this time, the position where the user presses the upper side region may be an arbitrary area and an arbitrary position in the upper side region (for example, the upper hemisphere).

  When the pressing force by the user exceeds the threshold value, the MCU 1 defines the operation region 15 that is currently selected (displayed by the display device 50), that is, the operation region 15 at a predetermined position in the upper region. An operation command corresponding to the information is generated. In this case, the MCU 1 functions as a generation unit. Then, the MCU 1 transmits information on the generated operation command to the display device 50.

  Based on the received information, the display device 50 executes processing such as channel selection and volume adjustment corresponding to the operation command.

  FIG. 11 is a diagram for explaining the principle of the setting method in the case where the operation content is set in the above-described angle range information that is “posture information”.

In this example, a polar coordinate system is used as the global coordinate system. Among the initial posture angles (θ _x , θ _y , θ _z ) obtained in steps 201 and 202 shown in FIG. 6, the yaw angle θ _z and the tilt angle θ from the Z axis are used to express in polar coordinates. An operation area 15 is represented. The tilt angle θ from the Z-axis is obtained by the following formula using the same method as formulas 1 and 2.

θ = arccos (a _z / A _ZG ) (8)

Alternatively, the yaw angle of these initial posture angles is not limited to θ _z , and R _z that is an angle after this is globally converted may be used. Since the movement of the operation device 10 substantially temporarily stops after the user selects the operation area 15 in the first operation step and moves to the second operation step, the MCU 1 is configured to have the initial posture angle θ of the yaw angle. _z can be used.

In the example shown in FIG. 11, for example, r is a constant value, and an area where 0 ° ≦ θ <30 ° and 30 ° <θ _z ≦ 60 ° is set as the operation area 15 for “volume up”. . As described above, such a setting can be realized from the tilt angle of the sensor substrate 20 based on the state where the global coordinate axis and the local coordinate axis of the sensor substrate 20 coincide with each other. As described above, information on these angle ranges and operation contents (operation commands) are associated with each other and stored in the ROM 4 or a memory (not shown).

  As described above, in the present embodiment, the upper side area, which is a position that is easy for the user to see when the user rotates the operation device 10, is positioned at a predetermined position in the upper side area triggered by being pressed by the user. An operation command corresponding to the posture information corresponding to the operation area 15 to be generated is generated. Thereby, user operability can be improved.

  In particular, the position of the area to be selected is set in advance as the position of the vertical operation area 15V. Since the vertical operation region 15V is a position that is easy for the user to see, the operability for the user can be improved.

(Operation example 2)
FIG. 12 is a diagram for explaining an operation example 2 by the operation device 10. In the following description, the description of the same part as the operation example 1 is omitted or simplified.

  In the operation example 1, the pressing position in the second operation step of the user is an arbitrary position in the upper side region. However, in this operation example 2, as shown in FIG. 12, by pressing only the operation area 15 to be selected by the user in the upper area, the operation command corresponding to the posture information of the operation area 15 is executed. Is done. In this case, in the first operation step, the user rotates the operation device 10 so that the operation region 15 to be selected is located at an arbitrary position (for example, a position through which the vertical axis V passes) in the upper region. Good.

  In order to realize this operation example 2, although not shown, the pressure sensor 8 may be provided for each operation region 15, for example. Alternatively, even if the pressure sensor 8 is not arranged, a program capable of calculating the pressurization position for each operation region 15 by a predetermined algorithm may be stored in the ROM 4 or the like. In this case, a strain sensor may be used instead of the pressure sensor 8.

  In order to realize this operation example 2, similarly to the operation example 1, the MCU 1 detects the attitude of the operation device 10 and detects the current position for each operation region 15 in the three dimensions.

  In the second operation example, the display device 50 may or may not output the related image 58 related to the operation area 15 of the operation device 10.

(Operation example 3)
FIG. 13 is a diagram for explaining an operation example 3 by the operation device 10.

  In this operation example 3, the display device 50 outputs a plurality of related images 58 corresponding to a plurality of operation areas 15 similar to the form of the operation device 10 (that is, representing a spherical surface) for display. In the example shown in FIG. 13, the CPU 53 of the display device 50 enlarges and displays, for example, the vertical operation area 15 </ b> V that is a selection target area and the operation area 15 adjacent thereto.

  In the first operation step, the CPU 53 changes the related image 58 in accordance with the movement of the operation device 10 (updates the display of the related image 58), and further, one related image of the plurality of operation areas 15. 58 is selectively highlighted. The highlighted related image 58 is indicated by reference numeral 58V.

  The operation device 10 may transmit information on the detected posture of itself to the display device 50. Alternatively, if the operation device 10 transmits its own movement information to the display device 50, the display device 50 receives it, and based on the received information, detects the attitude of the operation device 10 as shown in FIG. Good.

  The second operation step is the same as in the above operation examples 1 and 2.

  As described above, in this operation example 3, the related image 58 similar to that of the operation device 10 is displayed, so that the user's intuition is improved. Further, since the related image 58 corresponding to the operation region 15 to be selected located at a predetermined position (vertically upward) is highlighted, the visibility and operability of the user are improved.

  In this operation example 3, the CPU 53 enlargedly displays the vertical operation area 15V and the operation area 15 adjacent thereto. However, the CPU 53 does not perform such enlarged display, and as a result, for example, as shown in the left side of FIG. 14 to be described next, the entire relationship of the operation device 10 viewed from a predetermined direction (typically obliquely upward) An image may be displayed.

(Operation example 4)
FIG. 14 is a diagram for explaining an operation example 4 by the operation device 10.

  In the fourth operation example, no character or the like representing the operation content is printed on the surface of the casing 11 of the operation device 10. In this case, the display device 50 displays the related images 65 related to a plurality of operation areas similar to the form of the operation device 10. Further, the related image 65V to be selected is highlighted.

  The processing content of the MCU 1 of the operation device 10 is the same as that of the operation example 1, and the processing content of the CPU 53 of the display device 50 is the same as that of the operation example 3.

(Operation example 5)
FIG. 15A is a diagram for explaining an operation example 5 by the operation device 10.

  In this operation example 5, as shown in FIG. 15A, for example, the operation area 15 to be selected is not only the operation area 15 passing through a predetermined position (here, the position through which the vertical axis V passes), but at least adjacent to the operation area 15 One operation area 15 is also included. These regions that are selection candidates are hereinafter referred to as a selection candidate region group (region 25 indicated by hatching).

  In this example, “OK” is the vertical operation region 15V, and a plurality of direction keys are provided adjacent to it. The direction key is used, for example, when the pointer is moved within the screen. Of course, the present invention is not limited to such determination and direction keys, and may be anything such as an operation area related to the media playback function.

  In this operation example 5, the processing of the MCU 1 of the operation device 10 is the same as in the operation example 1 or 2. The MCU 1 executes an operation command corresponding to the posture information that defines one operation region 15 in the selection candidate region group 25. The processing content of the CPU 53 of the display device 50 may be any one of the above operation examples.

  In order to detect which operation region 15 of the selection candidate region group 25 is designated by the user, the operation device may have the following configuration. For example, a touch sensor (a capacitance sensor, a strain gauge, or the like) may be disposed on the surface of the housing 11 so as to be able to detect a touch on each operation region 15 in the selection candidate region group 25.

(Operation example 6)
FIG. 15B is a diagram for explaining an operation example 6 by the operation device 10.

  As shown in FIG. 15B, one operation area 15 of the operation device 10 described so far may be divided into a plurality of operation areas 15. In this case, one piece of posture information and one selection candidate region group 35 among the plurality of pieces of posture information of the housing 11 are stored in association with each other. The user selects one selection candidate area group 35 as a selection target in the first operation step, and touches one operation area 15 from the selection candidate area group 35 (see the lower right in FIG. 9). An operation command corresponding to the area 15 is executed.

  The processing content of the operation device 10 according to the operation example 6 is the same as that of the operation example 5 described above.

(Operation example 7)
FIG. 16 is a diagram for explaining an operation example 7 by the operation device.

  The operation device 110 according to the operation example 6 includes a display that is provided on the surface of the casing 111 and displays a plurality of related images respectively related to the plurality of operation regions 15. The display may have a function such as a pressure-sensitive or capacitive touch panel.

  As shown in the left side of FIG. 16, the operation device 110 has a TV / VIDEO mode for controlling a TV, a recording function or a recording device as shown in FIG. 16, and a music function or a music device as shown in the right side of FIG. MUSIC mode for control. The operation device 110 may have three or more control modes.

  Switching between the two modes may be performed by, for example, a gesture using the operation device 110, a switch (not shown) provided in the operation device 110, or application software that can be displayed on the display device 50.

  The processing contents of the MCU of the operation device 110 and the CPU of the display device are the same as any one of the operation examples 1 to 4 described above.

  In the operation example 7, the related image related to the operation area 15 may not be displayed on the display device. The display of the operation device 110 may highlight an area to be selected at a predetermined position.

(Operation example 8)
An operation example 8 by the operation device will be described.

  For example, the operation device 10 according to the operation example 8 stops the calculation (output) of information as a result of detecting its own posture, or transmits the calculation (output) of the posture to the display device 50. You may have the function to stop doing. In this case, the MCU 1 functions as a stop unit. Accordingly, since the detection information of the attitude of the operation device 10 is not transmitted to the display device 50, the CPU 53 of the display device 50 does not change the display.

  In the example shown in FIG. 10A, even if the user operates the operation device 10 to rotate, the display device 50 does not change the related image 58 of the selection target area. For example, the predetermined related image 58 Alternatively, the related image 58 corresponding to the operation area 15 designated immediately before is displayed. When there is a predetermined gesture using the operation device 10 by the user, the MCU 1 generates an enable signal triggered by the gesture. That is, the MCU 1 releases the above stop. In this case, the MCU 1 functions as a release unit.

  The predetermined gesture is, for example, a switch or button (not shown) provided on the operation device 10, for example, when the user grasps the casing 11 of the operation device 10 with a predetermined pressure, shakes with a predetermined excitation force and a predetermined number of times. There is such as pushing.

  [Operation devices according to other embodiments]

  17 and 18 illustrate an operating device according to another embodiment of the present technology. In the following description, the hardware, functions, and the like included in the operation device 10 and the information processing system according to the above-described embodiment will be described with a focus on different points while simplifying or omitting the description.

  The center of gravity of the casing 211 of the operation device 210 is eccentric, and the casing 211 has a shape similar to an ellipsoid or an egg. The upper and lower sides of the casing 211 are set in advance, and are eccentric so that the weight of the lower side region having the region in contact with the operation surface 90 is heavier than that of the upper side region. As a result, the operation device 210 can move as if it were getting up. The reference posture of the casing 211 of the operation device 210 is self-supporting so that the longitudinal direction of the casing 211 coincides with the direction of gravity as if the casing 211 is raised and spilled. From this reference posture state, the first operation step is performed such that the user rotates (tilts) in an arbitrary direction as shown in FIG.

(Operation example 1 of an operation device having an eccentric shape)
Since the reference posture of the operation device 210 is set in advance, the operation device 210 is operated by the user with the vertical direction determined. Therefore, the operation device 210 can detect the posture by at least the three-axis acceleration sensor 5 among the motion sensors. In this case, the angular velocity sensor 6 and the magnetic sensor 7 may be used as auxiliary sensors for increasing the accuracy of detecting the posture.

  As shown in FIG. 17, when the user keeps the operation device 210 tilted, the MCU 1 detects the posture of the casing 211. Specifically, the MCU 1 calculates the above-described initial posture angle or global angle (see FIG. 6) as the tilt angle of the posture, and transmits the calculated information to the display device 50. The display device 50 receives the information and controls the display to scroll the related image 58 according to the posture (that is, the tilt angle and the direction). In this case, the CPU 53 increases the scrolling speed as the tilt angle increases, and scrolls the related image 58 in a direction corresponding to the tilt direction.

  In this case, among the related images 58 corresponding to the plurality of operation regions, the related image 58V of the region to be selected is highlighted. The position of the highlighted related image 58V does not move within the screen at least during scrolling.

  As shown in FIG. 18, when the user performs the second operation step, the operation device 210 is returned to the reference posture, and the upper region of the housing 211 of the operation device 210, here the region through which the vertical axis V passes. Apply external force. The external force by the user can be detected by the pressure sensor 8, the vibration sensor, and the contact sensor as described above. By this second operation step, the MCU of the operation device 210 executes the operation command and transmits it to the display device 50. Thereby, the area selected by highlighting on the screen of the display device 50 is designated.

  FIG. 19 is a diagram for explaining the principle of operation region detection in the first operation step shown in FIG. In this operation example, the CPU 53 performs an operation so as to move in a virtual plane represented by a two-dimensional orthogonal coordinate system in accordance with the attitude (tilt angle and tilt direction) of the operation device 210. A plurality of operation areas 105 are virtually set in advance in each coordinate area in the plane. By the first operation step, among the operation areas 105 on the plane, the operation area 15 where the moving pointer is located becomes an area to be selected (highlighted) at that time.

  In the example shown in FIG. 19, a coordinate range of 20 ≦ x <40 and 10 <y ≦ 30 is set as the operation area 105 for “volume up”. That is, the operation command is stored in, for example, the storage unit 57 of the display device 50 or the ROM 54 (see FIG. 3) so that the operation command is associated with each coordinate range of the operation area 105. In the case of this operation example 1, since the operation device 210 itself has not detected which coordinate range corresponds to the operation area 105 corresponding to the current pointer position, the operation command transmitted from the operation device 210 is The operation content becomes a trigger command (execution command) for causing the display device 50 to start execution.

  The operation device 210 returns to the reference posture until the operation device 210 moves from the first operation step to the second operation step. At the time of this return, if the display device 50 is controlled not to follow the position of the pointer to be controlled, the operability is improved.

  In the present embodiment, when the pointer moves to the end of coordinates (maximum or minimum), the coordinate value of the pointer may jump (loop) to the counter electrode. In the case of the example shown in FIG. 19, when x = + 60 (−60) is exceeded (below), it may fly to x = −60 (+60). Thus, the CPU 53 can control the display so that the operation target area returns to the original operation area 105 before one rotation even when the ball rotates one or more times on the screen.

(Operation example 2 of operation device with eccentric shape)
In the description of the operation example 1, the operation device 210 having an eccentric shape does not detect which coordinate range corresponds to the position of the pointer and the display device 50 performs this operation. It was. However, in the second operation example, when the operation device 210 includes the acceleration sensor 5, the angular velocity sensor 6, and the magnetic sensor 7, if the operation region in the global coordinates is detected as in the spherical operation device 210 described above. Good. That is, the detection principle of the operation region by the operation device 210 according to the second operation example is different from the detection principle of the operation region 105 as illustrated in FIG. 19, and the operation device 210 detects the detection values of these sensors 5 to 7. Posture information, which is combination information, and operation command information are stored in association with each other.

  For example, either the surface of the casing 211 of the operation device 210 and the operation surface 90 with which the operation device 210 comes into contact have a soft material, the operation surface 90 is uneven, or the operation surface 90 is messed up. This operation example 2 is also effective when a cloth or the like is laid. In this case, even when the operation device 210 is eccentric, the operation device 210 may not return to the reference posture depending on the shape of the operation device 210, the degree of eccentricity, and the like. In such a case, the operation device 210 stores the posture information and the operation command information in association with each other in advance, so that the region that is the operation target of the casing 211 is the vertical axis V that is originally the operation target. It does not have to be located at a position passing through. Even when a region other than the vertex is located at a position passing through the vertical axis V, the MCU can detect a vertical operation region other than the vertex.

(Examples of operation devices with other eccentric shapes)
20A and 20B show another example for an operating device having an eccentric center of gravity. As shown in these figures, the present technology can also be applied to the operation devices 310 and 410 having shapes other than the shape similar to the egg shape.

  [Other embodiments]

  The present technology is not limited to the embodiments described above, and other various embodiments can be realized.

  In each of the embodiments described above, the operation device performs all the processes shown in FIG. However, the display device may execute part of the processing illustrated in FIG. For example, the operation device executes up to AD conversion processing of the detection values of the sensors 5 to 7 in FIG. 6, and transmits information obtained by AD conversion to the display device. And a display device may perform the steps 201-205. Alternatively, the operation device may execute any one of the steps 201 to 204 and the display device 50 may execute the remaining processes. Which process the operation device 10 and the display device 50 are in charge of may be appropriately determined from the environment such as their processing capacity, their cost, chip size, and the like.

  In each operation example described above, the operation content corresponding to the vertical operation region is set as a selection target by the user. However, an operation content corresponding to an operation region that is an upper region of the housing and does not pass through the vertical axis may be set as an operation target.

  In the operation examples 3 and 4, the display device displays a related image in a form similar to the shape of the operation device. However, the display device is not limited to this, and the related image is displayed in another display form such as a grid display or a list display. Also good.

  The shape of the outer surface of the casing that can come into contact with the operation surface when operated by the user of the operation device according to the embodiment has been described as a curved surface. However, an operation device in which at least a part of the outer surface of the housing is formed by a set of fine planes and the surface formed by the set of planes is operated in contact with the operation surface is also included in the scope of the present technology.

  The casing of the eccentric shaped operation device according to the embodiment shown in FIGS. 17, 18 and 20 is formed in a shape whose vertical direction is predetermined. However, even when the box is in a shape other than a sphere, a shape whose vertical direction is not determined in advance, such as a rectangular parallelepiped, a cube, an ellipsoid, a rugby ball, or at least one of these three shapes and a sphere The housing may have a shape such as a combination with a part of the housing.

  The shape of the truncated icosahedron is just an example as the shape of the plurality of operation regions provided on the surface of the operation device in the embodiment. The plurality of operation areas may be regular polyhedrons, for example, or may be partitioned by lines such as a plurality of globes and / or latitude lines. Alternatively, the shape of each operation region may not be regular, and may be defined by an irregularly shaped surface (or irregularly shaped bubbles).

  The operation device whose vertical direction is determined in advance may have an acceleration sensor and an angular velocity sensor having two orthogonal detection axes.

  The calculation method for the operation device to detect its own posture is not limited to the process shown in FIG. 6, and any other known calculation method can be used.

  It is also possible to combine at least two feature portions among the feature portions of each embodiment described above.

The present technology can be configured as follows.
(1) An operation device having a housing including a surface having a plurality of operation regions, a first sensor that detects movement of the housing, and a second sensor that can detect an external force with respect to the plurality of operation regions is operated. An acquisition unit that acquires movement information detected by the first sensor when the operation device is operated by the user to rotate while being placed on a surface;
A storage unit that associates and stores information on a plurality of postures of the operation device and information on a plurality of operation commands respectively corresponding to the plurality of operation regions for defining the plurality of operation regions;
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generating unit that generates an operation command corresponding to information on a posture that defines an operation region located at a predetermined position on a surface of the casing among the upper side region among the plurality of operation regions;
An information processing apparatus comprising: a transmission unit that transmits at least the generated operation command to an operation target device of the operation device.
(2) The information processing apparatus according to (1),
The generation unit corresponds to the posture information that defines the vertical operation area, with the vertical operation area through which the vertical axis in the three-dimensional dimension passes as the operation area located at the predetermined position among the plurality of operation areas. An information processing apparatus that generates the operation command.
(3) The information processing apparatus according to (2),
The generation unit generates the operation command corresponding to information on the posture that defines the vertical operation area when it is detected that the vertical operation area is pressed by the user.
(4) The information processing apparatus according to (2),
The information processing apparatus can generate the operation command corresponding to the posture information that defines at least one operation region adjacent to the vertical operation region in the upper side region.
(5) The information processing apparatus according to any one of (1) to (4),
The operation target device can output a plurality of related images respectively related to the plurality of operation areas for display,
The transmission unit is configured to cause the operation target device to output, for display, a related image related to the operation region located at the predetermined position among the plurality of related images, based on the motion information. An information processing apparatus that transmits acquired motion information or detection information of the detected posture to the operation target device.
(6) The information processing apparatus according to (5),
The transmission unit acquires the motion information acquired or the detected posture in order to selectively highlight a related image related to the operation region located at the predetermined position according to the motion of the operation device. An information processing device that transmits the detection information to the operation target device.
(7) The information processing apparatus according to any one of (1) to (6),
Of the surfaces of the casing of the operation device, at least a surface in contact with the operation surface is a spherical surface.
(8) The information processing apparatus according to any one of (1) to (7),
The operation device includes a display that is provided on a surface of the housing and displays a plurality of related images respectively associated with the plurality of operation regions.
(9) The information processing apparatus according to any one of (5), (6), and (8),
Stop the output of the detection information of the attitude of the operation device by the detection unit, or a stop unit to stop transmission of the motion information to the operation target device by the transmission unit,
An information processing apparatus further comprising: a release unit that cancels the stop of the transmission by the stop unit when a predetermined external force is detected by the second sensor.
(10) An operation device having a housing including a surface having a plurality of operation areas, a first sensor for detecting movement of the housing, and a second sensor capable of detecting an external force applied to the plurality of operation areas is operated. An acquisition unit that acquires movement information detected by the first sensor when the operation device is operated by the user to rotate while being placed on a surface;
A storage unit that associates and stores information on a plurality of postures of the operation device and information on a plurality of operation commands respectively corresponding to the plurality of operation regions for defining the plurality of operation regions;
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generating unit that generates an operation command corresponding to information on a posture that defines an operation region located at a predetermined position on a surface of the casing among the upper side region among the plurality of operation regions;
An information processing apparatus including: a transmission unit that transmits at least the generated operation command to an operation target device of the operation device;
A receiving unit for receiving at least the transmitted operation command;
An information processing system comprising: an operation target device including: an output unit configured to output, for display, related images respectively associated with the plurality of operation areas based on the received operation command information.
(11) In a state where an operation device having a housing, a first sensor that detects movement of the housing, and a second sensor that can detect an external force with respect to the housing is placed on the operation surface, the operation is performed by the user. An acquisition unit that acquires movement information detected by the first sensor when the device is operated to rotate;
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generation unit for generating an execution command;
An information processing apparatus comprising: a transmission unit that transmits the generated execution command and the acquired motion information or the detected posture detection information to an operation target device of the operation device;
Information on a plurality of coordinate ranges represented in a virtual plane area for defining a plurality of operation areas and information on a plurality of operation commands respectively corresponding to the plurality of operation areas are stored in association with each other. A storage unit;
A receiving unit for receiving information transmitted from the information processing apparatus;
When the execution command is received by the receiving unit, an operation command corresponding to one coordinate range of the plurality of coordinate ranges that can be selected according to the received motion information or posture detection information is executed. An information processing system comprising an operation target device having an execution unit.
(12) a housing including a surface having a plurality of operation areas;
A first sensor for detecting movement of the housing;
A second sensor capable of detecting an external force on the plurality of operation areas;
An acquisition unit that acquires movement information detected by the first sensor when the operation device is operated so as to rotate by the user while the operation device is placed on the operation surface;
A storage unit that associates and stores information on a plurality of postures of the operation device and information on a plurality of operation commands respectively corresponding to the plurality of operation regions for defining the plurality of operation regions;
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generating unit that generates an operation command corresponding to information on a posture that defines an operation region located at a predetermined position on a surface of the casing among the upper side region among the plurality of operation regions;
An operation device comprising: a transmission unit that transmits at least the generated operation command to an operation target device of the operation device.
(13) Information of an operation device having a housing including a surface having a plurality of operation regions, a first sensor that detects movement of the housing, and a second sensor that can detect an external force applied to the plurality of operation regions. A method performed by a processing device,
A plurality of operation device information for defining the plurality of operation regions, and a plurality of operation command information respectively corresponding to the plurality of operation regions are stored in association with each other;
In a state where the operation device is placed on the operation surface, when the operation device is operated so as to rotate by a user, movement information detected by the first sensor is acquired,
Based on the acquired movement information, the posture of the operation device operated by the user is detected,
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. Generating an operation command corresponding to posture information defining an operation region located at a predetermined position on the surface of the casing among the plurality of operation regions,
An information processing method for transmitting at least the generated operation command to an operation target device of the operation device.

1 ... MCU
DESCRIPTION OF SYMBOLS 5 ... Acceleration sensor 6 ... Angular velocity sensor 7 ... Magnetic sensor 8 ... Pressure sensor 9 ... Communication apparatus 10, 110, 210, 310, 410 ... Operation device 11, 111, 211 ... Case 15, 105 ... Operation area 15V ... Vertical operation Area 50 ... Display device 53 ... CPU
56 ... Communication unit 57 ... Storage unit 58 ... Related image 65 ... Related image 90 ... Operation surface

Claims (12)

An operation device having a housing including a surface having a plurality of operation regions, a first sensor that detects movement of the housing, and a second sensor that can detect an external force applied to the plurality of operation regions is placed on the operation surface. An acquisition unit that acquires movement information detected by the first sensor when the operation device is operated so as to rotate by a user in a state in which the operation device is rotated;
A storage unit that associates and stores information on a plurality of postures of the operation device and information on a plurality of operation commands respectively corresponding to the plurality of operation regions for defining the plurality of operation regions;
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generating unit that generates an operation command corresponding to information on a posture that defines an operation region located at a predetermined position on a surface of the casing among the upper side region among the plurality of operation regions;
A transmission unit that transmits at least the generated operation command to an operation target device of the operation device ;
The operation device includes, as the first sensor, an acceleration sensor having three orthogonal detection axes in a local coordinate system, an angular velocity sensor having the three orthogonal detection axes, and a magnetic sensor having the three orthogonal detection axes. An operating device with
The acquisition unit acquires information on acceleration, angular velocity, and magnetic intensity detected by the acceleration sensor, the angular velocity sensor, and the magnetic sensor, respectively, as the movement information,
The detector is
Coordinate transformation that converts the angular velocity and acceleration acquired by the acquisition unit into global angular velocity and global acceleration in the global coordinate system using information on the attitude angle of the operation device in a global coordinate system representing a three-dimensional space, respectively. And
Based on the information on the acceleration and the magnetic strength acquired by the acquisition unit at the start of operation of the operation device by the user, among the posture angles, the initial posture angle in the global coordinate system of the operation device An initial posture angle calculation unit for calculating an initial posture angle;
An update unit that updates the posture angle in the global coordinate system of the operation device based on the information on the global angular velocity converted by the coordinate conversion unit;
Using the information on the initial posture angle calculated by the initial posture angle calculation unit, the coordinate conversion unit obtains the first angular velocity and the first acceleration acquired by the acquisition unit at the start of the operation as the global The second angular velocity and the second acceleration acquired after the first angular velocity and the first acceleration are acquired using the updated attitude angle information, respectively converted into the angular velocity and the global acceleration. And an information processing apparatus having a control unit that converts the global angular velocity and the global acceleration, respectively . The information processing apparatus according to claim 1,
The generation unit corresponds to information on the posture that defines the vertical operation area, with a vertical operation area through which a vertical axis in the three-dimensional space passes as an operation area located at the predetermined position among the plurality of operation areas. An information processing apparatus that generates the operation command to be performed. An information processing apparatus according to claim 2,
The generation unit generates the operation command corresponding to information on the posture that defines the vertical operation area when it is detected that the vertical operation area is pressed by the user. An information processing apparatus according to claim 2,
The information processing apparatus can generate the operation command corresponding to the posture information that defines at least one operation region adjacent to the vertical operation region in the upper side region. The information processing apparatus according to any one of claims 1 to 4,
The operation target device can output a plurality of related images respectively related to the plurality of operation areas for display,
The transmission unit is configured to cause the operation target device to output, for display, a related image related to the operation region located at the predetermined position among the plurality of related images, based on the motion information. An information processing apparatus that transmits acquired motion information or detection information of the detected posture to the operation target device. The information processing apparatus according to claim 5,
The transmission unit acquires the motion information acquired or the detected posture in order to selectively highlight a related image related to the operation region located at the predetermined position according to the motion of the operation device. An information processing device that transmits the detection information to the operation target device. The information processing apparatus according to any one of claims 1 to 6,
Of the surfaces of the casing of the operation device, at least a surface in contact with the operation surface is a spherical surface. The information processing apparatus according to any one of claims 1 to 7,
The operation device includes a display that is provided on a surface of the housing and displays a plurality of related images respectively associated with the plurality of operation regions. The information processing apparatus according to any one of claims 5, 6, and 8,
Stop the output of the detection information of the attitude of the operation device by the detection unit, or a stop unit to stop transmission of the motion information to the operation target device by the transmission unit,
An information processing apparatus further comprising: a release unit that cancels the stop of the transmission by the stop unit when a predetermined external force is detected by the second sensor. An operation device having a housing including a surface having a plurality of operation regions, a first sensor that detects movement of the housing, and a second sensor that can detect an external force applied to the plurality of operation regions is placed on the operation surface. An acquisition unit that acquires movement information detected by the first sensor when the operation device is operated so as to rotate by a user in a state in which the operation device is rotated;
A storage unit that associates and stores information on a plurality of postures of the operation device and information on a plurality of operation commands respectively corresponding to the plurality of operation regions for defining the plurality of operation regions;
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generating unit that generates an operation command corresponding to information on a posture that defines an operation region located at a predetermined position on a surface of the casing among the upper side region among the plurality of operation regions;
An information processing apparatus including: a transmission unit that transmits at least the generated operation command to an operation target device of the operation device;
A receiving unit for receiving at least the transmitted operation command;
An output unit that outputs, for display, related images respectively associated with the plurality of operation areas based on the information of the received operation commands, and includes an operation target device .
The operation device includes, as the first sensor, an acceleration sensor having three orthogonal detection axes in a local coordinate system, an angular velocity sensor having the three orthogonal detection axes, and a magnetic sensor having the three orthogonal detection axes. An operating device with
The acquisition unit acquires information on acceleration, angular velocity, and magnetic intensity detected by the acceleration sensor, the angular velocity sensor, and the magnetic sensor, respectively, as the movement information,
The detector is
Coordinate transformation that converts the angular velocity and acceleration acquired by the acquisition unit into global angular velocity and global acceleration in the global coordinate system using information on the attitude angle of the operation device in a global coordinate system representing a three-dimensional space, respectively. And
Based on the information on the acceleration and the magnetic strength acquired by the acquisition unit at the start of operation of the operation device by the user, among the posture angles, the initial posture angle in the global coordinate system of the operation device An initial posture angle calculation unit for calculating an initial posture angle;
An update unit that updates the posture angle in the global coordinate system of the operation device based on the information on the global angular velocity converted by the coordinate conversion unit;
Using the information on the initial posture angle calculated by the initial posture angle calculation unit, the coordinate conversion unit obtains the first angular velocity and the first acceleration acquired by the acquisition unit at the start of the operation as the global The second angular velocity and the second acceleration acquired after the first angular velocity and the first acceleration are acquired using the updated attitude angle information, respectively converted into the angular velocity and the global acceleration. And an information processing system including a control unit that converts the global angular velocity and the global acceleration, respectively . The operation device is rotated by a user in a state where an operation device having a housing, a first sensor that detects the movement of the housing, and a second sensor that can detect an external force applied to the housing is placed on the operation surface. An acquisition unit that acquires movement information detected by the first sensor when operated to
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generation unit for generating an execution command;
An information processing apparatus comprising: a transmission unit that transmits the generated execution command and the acquired motion information or the detected posture detection information to an operation target device of the operation device;
Information on a plurality of coordinate ranges represented in a virtual plane area for defining a plurality of operation areas and information on a plurality of operation commands respectively corresponding to the plurality of operation areas are stored in association with each other. A storage unit;
A receiving unit for receiving information transmitted from the information processing apparatus;
When the execution command is received by the receiving unit, an operation command corresponding to one coordinate range of the plurality of coordinate ranges that can be selected according to the received motion information or posture detection information is executed. An operation target device having an execution unit ,
The operation device includes, as the first sensor, an acceleration sensor having three orthogonal detection axes in a local coordinate system, an angular velocity sensor having the three orthogonal detection axes, and a magnetic sensor having the three orthogonal detection axes. An operating device with
The acquisition unit acquires information on acceleration, angular velocity, and magnetic intensity detected by the acceleration sensor, the angular velocity sensor, and the magnetic sensor, respectively, as the movement information,
The detector is
Coordinate transformation that converts the angular velocity and acceleration acquired by the acquisition unit into global angular velocity and global acceleration in the global coordinate system using information on the attitude angle of the operation device in a global coordinate system representing a three-dimensional space, respectively. And
Based on the information on the acceleration and the magnetic strength acquired by the acquisition unit at the start of operation of the operation device by the user, among the posture angles, the initial posture angle in the global coordinate system of the operation device An initial posture angle calculation unit for calculating an initial posture angle;
An update unit that updates the posture angle in the global coordinate system of the operation device based on the information on the global angular velocity converted by the coordinate conversion unit;
Using the information on the initial posture angle calculated by the initial posture angle calculation unit, the coordinate conversion unit obtains the first angular velocity and the first acceleration acquired by the acquisition unit at the start of the operation as the global The second angular velocity and the second acceleration acquired after the first angular velocity and the first acceleration are acquired using the updated attitude angle information, respectively converted into the angular velocity and the global acceleration. And an information processing system including a control unit that converts the global angular velocity and the global acceleration, respectively . A housing including a surface having a plurality of operation areas;
A sensor for detecting the movement of the housing, an acceleration sensor having three orthogonal detection axes in a local coordinate system, an angular velocity sensor having the three orthogonal detection axes, and a magnetic sensor having the three orthogonal detection axes a first sensor configured in,
A second sensor capable of detecting an external force on the plurality of operation areas;
When the operation device is operated so that the operation device rotates with the operation device placed on the operation surface, the acceleration sensor, the angular velocity sensor, and the motion information detected by the first sensor An acquisition unit for acquiring information on acceleration, angular velocity, and magnetic intensity detected by the magnetic sensor ;
A storage unit that associates and stores information on a plurality of postures of the operation device and information on a plurality of operation commands respectively corresponding to the plurality of operation regions for defining the plurality of operation regions;
Based on the acquired movement information, a detection unit that detects an attitude of the operation device operated by the user;
When the second sensor detects that the user has pressed the upper side area of the casing that is not in contact with the operation surface of the surface of the casing having the detected posture. A generating unit that generates an operation command corresponding to information on a posture that defines an operation region located at a predetermined position on a surface of the casing among the upper side region among the plurality of operation regions;
A transmission unit that transmits at least the generated operation command to an operation target device of the operation device ;
The detector is
Coordinate transformation that converts the angular velocity and acceleration acquired by the acquisition unit into global angular velocity and global acceleration in the global coordinate system using information on the attitude angle of the operation device in a global coordinate system representing a three-dimensional space, respectively. And
Based on the information on the acceleration and the magnetic strength acquired by the acquisition unit at the start of operation of the operation device by the user, among the posture angles, the initial posture angle in the global coordinate system of the operation device An initial posture angle calculation unit for calculating an initial posture angle;
An update unit that updates the posture angle in the global coordinate system of the operation device based on the information on the global angular velocity converted by the coordinate conversion unit;
Using the information on the initial posture angle calculated by the initial posture angle calculation unit, the coordinate conversion unit obtains the first angular velocity and the first acceleration acquired by the acquisition unit at the start of the operation as the global The second angular velocity and the second acceleration acquired after the first angular velocity and the first acceleration are acquired using the updated attitude angle information, respectively converted into the angular velocity and the global acceleration. And a control unit that converts the global angular velocity and the global acceleration, respectively .