US20160325161A1 - Motion support method, terminal apparatus, and motion support program - Google Patents

Motion support method, terminal apparatus, and motion support program Download PDF

Info

Publication number
US20160325161A1
US20160325161A1 US15/110,438 US201515110438A US2016325161A1 US 20160325161 A1 US20160325161 A1 US 20160325161A1 US 201515110438 A US201515110438 A US 201515110438A US 2016325161 A1 US2016325161 A1 US 2016325161A1
Authority
US
United States
Prior art keywords
motion
output
swing
detected
motion support
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US15/110,438
Inventor
Shoichi Nakajo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Seiko Epson Corp
Original Assignee
Seiko Epson Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Seiko Epson Corp filed Critical Seiko Epson Corp
Assigned to SEIKO EPSON CORPORATION reassignment SEIKO EPSON CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NAKAJO, SHOICHI
Publication of US20160325161A1 publication Critical patent/US20160325161A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00Training appliances or apparatus for special sports
    • A63B69/36Training appliances or apparatus for special sports for golf
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B60/00Details or accessories of golf clubs, bats, rackets or the like
    • A63B60/46Measurement devices associated with golf clubs, bats, rackets or the like for measuring physical parameters relating to sporting activity, e.g. baseball bats with impact indicators or bracelets for measuring the golf swing
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
    • A63B24/0006Computerised comparison for qualitative assessment of motion sequences or the course of a movement
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0021Tracking a path or terminating locations
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B27/0172Head mounted characterised by optical features
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • A63B2071/0625Emitting sound, noise or music
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B2071/0647Visualisation of executed movements
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B2071/0658Position or arrangement of display
    • A63B2071/0661Position or arrangement of display arranged on the user
    • A63B2071/0666Position or arrangement of display arranged on the user worn on the head or face, e.g. combined with goggles or glasses
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00Measuring of physical parameters relating to sporting activity
    • A63B2220/40Acceleration
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00Measuring of physical parameters relating to sporting activity
    • A63B2220/50Force related parameters
    • A63B2220/51Force
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00Measuring of physical parameters relating to sporting activity
    • A63B2220/80Special sensors, transducers or devices therefor
    • A63B2220/83Special sensors, transducers or devices therefor characterised by the position of the sensor
    • A63B2220/833Sensors arranged on the exercise apparatus or sports implement
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00Measuring of physical parameters relating to sporting activity
    • A63B2220/80Special sensors, transducers or devices therefor
    • A63B2220/83Special sensors, transducers or devices therefor characterised by the position of the sensor
    • A63B2220/836Sensors arranged on the body of the user
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/0132Head-up displays characterised by optical features comprising binocular systems
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/014Head-up displays characterised by optical features comprising information/image processing systems

Definitions

  • the present invention relates to a motion support method, a terminal apparatus, and a motion support program.
  • a golf swing support apparatus can be given as an example of a motion support apparatus.
  • the golf support apparatus informs the player of, for example, an address position, a top position, and an impact position with a sound through a wireless earphone, using a metronome (PTL 1) or turns on an LED row provided to the foot of the player in relay to inform the player of the positions (PTL 2).
  • Another golf support apparatus optically detects a position during a back swing after a swing starts, for example, the position of a clubhead that is 20 cm away from the address position, outputs a signal to the player after a predetermined period of time has elapsed from the detection of the position, and informs the player of the time when the swing changes to a down swing (PTL 3).
  • An object of some aspects of the invention is to provide a motion support method, a terminal apparatus, and a motion support program that can allow an external output device to generate a motion support output at the exact time and to output the motion support output to the outside, without an operator inputting the time when the external output device generates the motion support output and outputs the motion support output to the outside with operation keys.
  • An aspect of the invention relates to a motion support method including: a step of determining whether a motion of an object to be detected satisfies a preset condition, on the basis of an output signal from an inertia sensor which is mounted to the object to be detected; and a step of generating an output start trigger when the motion satisfies the condition, in which the output start trigger is used as a start signal which allows an external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside.
  • the object to be detected is moved and an output signal indicating the detection of the motion of the object to be detected is obtained from the inertia sensor which is mounted to the object to be detected. It is determined whether the motion of the object to be detected satisfies the preset condition on the basis of the output signal. When the motion satisfies the condition, the output start trigger is generated.
  • the output start trigger is used as the start signal which allows the external output device to generate the motion support output (for example, an image, a sound, or vibration) for supporting the motion of the object to be detected and to output the motion support output to the outside.
  • the output start trigger makes it possible to automatically start an operation of allowing the external output device to generate the motion support output and to output the motion support output to the outside on the basis of the motion of the object to be detected. Therefore, a special operation other than the motion of the object to be detected is not needed and the external output device can generate the motion support output and output the motion support output to the outside, on the basis of the motion of the object to be detected.
  • the external output device is preferably a wearable apparatus which is mounted to an operator who is the object to be detected or an operator who operates an equipment, which is the object to be detected, or a portable apparatus which can be carried by the operator. This is because it is easy for the operator to acquire the motion support output.
  • the condition may include a support target motion of the object to be detected, a motion associated with the support target motion, or a stationary state of the object to be detected for a predetermined period of time or more.
  • the conditions for generating the output start trigger include a motion that is indispensable in the prescribed motion of the object to be detected or a stationary state, rather than a motion simply related to a decision.
  • a motion simply related to the decision is an extra motion and is likely to be forgotten.
  • the support target motion corresponds to, for example, a practice golf swing
  • the associated motion corresponds to, for example, waggle
  • the stationary state corresponds to an address state.
  • the motion or the stationary state is a natural motion in the support target motion, for example, the golf swing. It is possible to detect the natural motion and to generate the output start trigger.
  • motion support information which is used by the external output device to generate the motion support output may be transmitted to the external output device, on the basis of the output start trigger.
  • the external output device does not need to store the motion support information (for example, an image signal, a sound signal, or a frequency signal) for outputting the motion support output (for example, an image, a sound, or vibration).
  • the external output device can generate the motion support output and output the motion support output to the outside, on the basis of the motion support information which is transmitted to the external output device after the output start trigger is generated.
  • the output start trigger may be transmitted to the external output device that stores motion support information for generating the motion support output.
  • the motion support output may be an image. In this case, it is possible to support the motion of the object to be detected, using the output of the image.
  • the motion support output may be an image.
  • the motion support output may be a sound.
  • a sound for example, a rhythm sound.
  • the motion support output may include an image for a swing period from the start of the swing to at least an impact or a rhythm sound for the swing period. It is possible to support a swing from a swing start position to an impact position, using the image or the rhythm sound for the swing period.
  • analysis information which is obtained by analyzing the motion of the object to be detected may be collected on the basis of an output from the inertia sensor, and a comparison result between comparison information included in the motion support information and the analysis information may be transmitted to the external output device. Since the comparison result is output to the outside, it is possible to accurately support the motion of the object to be detected.
  • the comparison information may include a travel time from a swing start position to a swing turn position, a waiting time at the swing turn position, a travel time from the swing turn position to an impact position, and information about the swing turn position or a deceleration position of the grip during a down swing. It is possible to accurately evaluate the swing of the object to be detected on the basis of the comparison information.
  • Another aspect of the invention relates to a terminal apparatus that is connected to an inertia sensor mounted to an object to be detected and an external output device by wire or wirelessly and supports a motion of the object to be detected, the terminal apparatus including: a determination unit that determines whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and a trigger generation unit that generates an output start trigger when the motion satisfies the condition, in which the output start trigger is used as a start signal which allows the external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside.
  • the terminal apparatus including: a determination unit that determines whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and a trigger generation unit that generates an output start trigger when the motion satisfies the condition, in which the output start trigger is used as
  • Still another aspect of the invention relates to a motion support program that is used in a terminal apparatus which includes a computer, is connected to an inertia sensor mounted to an object to be detected and an external output device by wire or wirelessly, and supports a motion of the object to be detected, in which the motion support program causes the computer to perform: a procedure for determining whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and a procedure for generating an output start trigger which allows the external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside when the motion satisfies the condition.
  • the motion support program is executed, it is possible to implement the motion support method or the terminal apparatus according to the aspect of the invention, using software.
  • FIG. 1 is a diagram illustrating the entire structure of a swing support system according to an embodiment of the invention.
  • FIG. 2 is a block diagram illustrating a swing support apparatus illustrated in FIG. 1 .
  • FIG. 3 is a block diagram illustrating a terminal apparatus illustrated in FIG. 2 .
  • FIG. 4 is a diagram illustrating a head-mounted display and a wireless earphone which are examples of an external output device.
  • FIG. 5 is a diagram illustrating processes of a swing support method according to an embodiment of the invention.
  • FIG. 6 is a diagram illustrating a golf swing.
  • FIG. 7 is a diagram illustrating an example of an image displayed on a display device which is the external output device.
  • FIG. 8 is a diagram illustrating processes of a swing support method according to another embodiment of the invention.
  • FIG. 9 is a block diagram illustrating an example of a position calculation unit illustrated in FIG. 3 .
  • FIG. 10 is a diagram illustrating the deceleration of a grip from the maximum value of the speed of the grip.
  • FIG. 1 is a diagram illustrating the entire structure of a swing support system (a motion support system in a broad sense) according to the invention which is applied to golf swing support.
  • servers 12 and 14 and a base station 16 are connected to a network such as an Internet 10 .
  • the server 12 is a program distribution server which distributes, for example, a swing support program or a swing analysis program.
  • a terminal apparatus 30 is a portable terminal, such as a mobile phone which can communicate with the program distribution server 12 through the base station 16 and the Internet 10 , or a personal computer which can communicate with the program distribution server 12 through the server 14 and the Internet 10 .
  • a swing support apparatus (a motion support apparatus in a broad sense) 20 includes the terminal apparatus 30 , an inertia sensor 40 which is attached to at least one of an operator (player), who is an object to be detected, and a golf club, and an external output device 50 which generates a swing support output (a motion support output in a broad sense) and outputs the swing support output to the outside.
  • a storage unit of the terminal apparatus 30 stores the swing support program or the swing analysis program downloaded from the program distribution server 12 .
  • an acceleration sensor or a gyro sensor (angular speed sensor) is incorporated into the inertia sensor 40 .
  • the acceleration sensor can detect acceleration in each of the directions of three axes x, y, and z which are perpendicular to each other.
  • the gyro sensor can detect an angular speed about each of the three axes x, y, and z which are perpendicular to each other.
  • the inertia sensor 40 outputs detection signals of the acceleration and the angular speed for each axis.
  • the y-axis is aligned with the axial direction of a shaft 102 and the x-axis is aligned with a hitting direction A.
  • the inertia sensor 40 is attached to, for example, a golf club (exercise equipment) 100 .
  • the golf club 100 includes a grip 101 and the shaft 102 .
  • a clubhead 103 is coupled to the leading end of the shaft 102 .
  • the inertia sensor 40 is attached to the grip 101 or the shaft 102 of the golf club 100 .
  • the terminal apparatus 30 includes an arithmetic processing circuit 300 .
  • the inertia sensor 40 is connected to the arithmetic processing circuit 300 through an interface 301 .
  • the interface 301 is connected to the inertia sensor 40 by wire or wirelessly.
  • the detection signal is supplied from the inertia sensor 40 to the arithmetic processing circuit 300 .
  • a storage device 302 is connected to the arithmetic processing circuit 300 .
  • the storage device 302 can store a program 303 , such as a swing support program (motion support program) or a swing analysis software program (motion analysis program), and data.
  • the program 303 is downloaded from the program distribution server 12 and is then stored.
  • the arithmetic processing circuit 300 executes the swing support program using the detection signal from the inertia sensor 40 to control the external output device 50 .
  • the arithmetic processing circuit 300 executes the golf swing analysis software program to perform golf swing analysis.
  • the storage device 302 can include, for example, a dynamic random access memory (DRAM) or a high-capacity storage device unit and a non-volatile memory.
  • DRAM dynamic random access memory
  • the program 303 is stored in the DRAM.
  • the terminal apparatus 30 includes a high-capacity storage device unit, such as a hard disk drive (HDD)
  • the program 303 and data can be stored in the HDD.
  • the non-volatile memory stores a relatively low-capacity program, such as a basic input/output system (BIOS), or data.
  • BIOS basic input/output system
  • An input device 304 is connected to the arithmetic processing circuit 300 .
  • the input device 304 includes at least alphabet keys and a numeric keypad. Text information or numerical information is input from the input device 304 to the arithmetic processing circuit 300 .
  • An output processing circuit 305 is connected to the arithmetic processing circuit 300 .
  • the arithmetic processing circuit 300 transmits predetermined output data to the output processing circuit 305 .
  • the output data (swing support information) processed by the output processing circuit 305 is transmitted to the external output device 50 through the interface 301 .
  • the output data is image data.
  • the external output device is a sound output device, such as an earphone, a headphone, or a speaker
  • the output data is sound data.
  • the output data is image data and sound data which are synchronized with each other.
  • the arithmetic processing circuit 300 can include a calculation unit 310 , a determination unit 320 , a trigger generation unit 330 , and a data transmission unit 340 .
  • the determination unit 320 determines whether the motion of the golf club 100 , which is the object to be detected, satisfies a preset condition on the basis of the detection signal from the inertia sensor 40 .
  • the determination unit 320 may receive the detection signal from the inertia sensor 40 through the interface 301 or may receive the calculation result of the calculation unit 310 on the basis of the detection signal.
  • the preset condition may be a motion simply related to a decision.
  • the preset condition preferably includes a motion that is indispensable in the prescribed motion of the golf club 100 , which is the object to be detected, or a stationary state.
  • a motion which is clearly distinguished from a swing motion for example, a motion of tapping the golf club on the ground and swinging the shaft in one direction or the forward and reverse directions
  • Examples of the motion which is indispensable in the prescribed motion of the golf club 100 or the stationary state can include a practice swing motion, an address motion, and a waggle motion.
  • the practice swing corresponds to a support target motion or a preparatory action having the same content as the support target motion.
  • the waggle (which will be described in detail below) corresponds to a motion associated with the support target motion and the address state corresponds to a stationary state.
  • the motion or the stationary state is a natural motion in the golf swing. It is possible to detect the natural motion or the stationary state and to generate an output start trigger. For example, for the address motion, it is possible to determine whether the golf club 100 is in a stationary state at an address position (swing start position) for a predetermined period of time (for example, a few seconds) or more.
  • the calculation unit 310 calculates, for example, the acceleration, speed, or position of the clubhead 103 , using the output from the inertia sensor 40 .
  • the determination unit 320 can determine that the golf club 100 is at the address position. Further, the determination unit 320 can determine whether the golf club 100 satisfies the preset condition, such as a stationary state, a swing state, or a motion other than a swing, on the basis of the output signal from the inertia sensor 40 .
  • the calculation unit 310 calculates the acceleration asj (including gravity g) of the clubhead 103 , using, for example, the following expression.
  • the calculation unit 310 specifies the position lsj of the clubhead 103 according to a local coordinate system ⁇ s unique to the inertia sensor 40 .
  • the position lsj of the clubhead 103 can be designated through, for example, the input device 304 and then stored in the storage device 302 .
  • An acceleration output as and an angular speed output ⁇ s are obtained from the inertia sensor 40 .
  • Angular acceleration (which is represented by ⁇ s having a dot at the top in the following expression) is obtained by differentiating the angular speed output ⁇ s.
  • ⁇ sj ⁇ s + ⁇ dot over ( ⁇ ) ⁇ s ⁇ l sj + ⁇ s ⁇ ( ⁇ s ⁇ l sj )+ g [Math. 1]
  • the calculation unit 310 calculates the moving speed of the clubhead 103 on the basis of the calculated acceleration.
  • an integral process is performed on the acceleration at a prescribed sampling interval dt, according to the following expression.
  • the calculation unit 310 calculates the position of the clubhead 103 on the basis of the calculated speed.
  • an integral process is performed on the speed at a prescribed sampling interval dt.
  • the trigger generation unit 330 When the determination unit 320 determines that the above-mentioned conditions are satisfied, the trigger generation unit 330 generates an output start trigger.
  • the data transmission unit 340 reads swing support information, which has been stored in the storage device 302 in advance, to the output processing circuit 305 on the basis of the output start trigger.
  • the output start trigger may be a read signal.
  • the swing support information processed by the output processing circuit 305 is transmitted to the external output device 50 through the interface 301 .
  • the external output device 50 generates a swing support output on the basis of the input swing support information and outputs the swing support output to the outside.
  • the swing support information which has been stored in the storage device 302 in advance may be transmitted to the external output device 50 through the interface 301 , without passing through the output processing circuit 305 .
  • the output processing circuit 305 is not needed.
  • the data transmission unit 340 is not needed, which will be described below.
  • the output start trigger generated by the trigger generation unit 330 is input to the external output device 50 through the interface 301 .
  • FIG. 4 is a head-mounted display device (head-mounted display: HMD) 500 which is an example of a display device as the external output device 50 .
  • the HMD 500 includes a spectacle body 501 which is mounted to the head of an operator 1 .
  • a display unit 502 is provided in the spectacle body 501 .
  • the display unit 502 displays a virtual image, which is the swing support output, on the basis of the swing support information transmitted from the terminal apparatus 30 , so as to overlap a real image that can be seen through the spectacle body 501 .
  • the display unit 502 can include, for example, an image display unit 503 , such as a liquid crystal display (LCD), a first beam splitter 504 , a second beam splitter 505 , a first concave reflecting mirror 506 , a second concave reflecting mirror 507 , a shutter 508 , and a convex lens 509 .
  • the first beam splitter 504 is provided in front of the left eye of the operator and partially transmits and reflects light from the image display unit 503 .
  • the second beam splitter 505 is provided in front of the right eye of the operator and partially transmits and reflects light from the first beam splitter 504 .
  • the first concave reflecting mirror 506 is provided in front of the first beam splitter 504 and partially reflects the partially reflected light from the first beam splitter 504 so as to be guided to the left eye of the operator through the first beam splitter 504 .
  • the second concave reflecting mirror 507 is provided in front of the second beam splitter 505 and partially reflects the partially reflected light from the second beam splitter 505 so as to be guided to the right eye of the operator through the second beam splitter 505 .
  • the convex lens 509 guides the partially transmitted light from the second beam splitter 505 to the outside of the HMD 500 when the shutter 508 is opened.
  • FIG. 4 also illustrates an earphone 510 which is an example of a sound output device as the external output device 50 .
  • the earphone 510 can be, for example, a wireless earphone which is wirelessly connected to the terminal apparatus 30 .
  • the earphone 510 may be connected to the terminal apparatus 30 by wire.
  • This structure holds for the HMD 500 .
  • the earphone 510 generates a rhythm sound (swing support output) on the basis of the swing support information from the terminal apparatus 30 and outputs the rhythm sound to the outside.
  • the operator 1 can listen to the rhythm sound which is the swing support output from the earphone 510 , without being affected by an ambient environmental sound.
  • FIG. 5 illustrates the operation of the operator 1 (golf club 100 ), the terminal apparatus 30 , and the display device 500 .
  • FIG. 6 illustrates a swing trajectory R of the clubhead 103 of the golf club 100 operated by the operator 1 .
  • the swing trajectory R includes a swing start (address) position P 1 , a top (swing turn) position P 2 , an impact position P 3 , and a finish position P 4 .
  • the calculation unit 310 calculates the positions Psj(t) of the clubhead 103 during a swing using the above-mentioned expression to calculate the swing trajectory R.
  • the terminal apparatus 30 can monitor the motion of the operator 1 (golf club 100 ) using the output from the inertia sensor 40 .
  • the calculation unit 310 calculates, for example, the acceleration ⁇ sj and the position Psj of the clubhead 103 using the above-mentioned expressions, if necessary.
  • the determination unit 320 determines whether the motion of the operator 1 (golf club 100 ) satisfies a preset condition, on the basis of the detection signal (acceleration, angular speed) from the inertia sensor 4 or the calculation result of the calculation unit 310 .
  • the determination unit 320 can determine that the motion satisfies the condition.
  • the determination unit 320 can determine whether the clubhead 103 is in a stationary state for a predetermined period of time or more at the position P 1 or in the vicinity of the position P 1 .
  • the determination unit 320 may determine a waggle motion which reciprocates the clubhead 103 a plurality of times between the position P 1 and a position that is away from the position P 1 by a distance corresponding to a short stroke in a take-back direction. The waggle motion is made immediately before a swing starts.
  • the determination unit 320 can determine the waggle motion on the basis of information about the acceleration ⁇ sj and the position Psj of the clubhead 103 .
  • the trigger generation unit 330 When the determination unit 320 determines that the motion satisfies the condition, the trigger generation unit 330 generates the output start trigger. Then, the data transmission unit 340 reads the swing support information from the storage device 302 and transmits the swing support information to the HMD 500 , which is the external output device 50 , through the interface 301 . Then, an image, which is the swing support output, is displayed on the HMD 500 . Therefore, the operator 1 does not need to operate a start switch and can start the display of an image on the HMD 500 while holding the grip 101 .
  • FIG. 7 illustrates an example of the display of the swing support output.
  • the swing support output is a moving picture indicating the movement of a clubhead 103 A of a golf club 100 A operated by an avatar 1 A.
  • the movement trajectory of a back swing is represented by R 1
  • the movement trajectory of a down swing is represented by R 2 .
  • the golf club 100 A is moved in an exemplary rhythm along the movement trajectories R 1 and R 2 .
  • the swing support output preferably includes an image from the start (address) of a swing to at least an impact and more preferably includes an image from start to finish.
  • the operator 1 can repeatedly view the swing support output (moving picture) illustrated in FIG. 7 with the HMD 500 . Then, the operator 1 actually takes a swing with reference to the swing support output (moving picture).
  • the swing motion is detected by the inertia sensor 40 and is then input to the terminal apparatus 30 .
  • the swing motion of the operator 1 is analyzed by the terminal apparatus 30 and is acquired as swing analysis information.
  • the swing analysis information can be displayed on the HMD 500 , similarly to FIG. 7 .
  • the swing support information and the actual swing (swing analysis information) can be displayed as moving pictures in parallel or alternately so as to be contrasted with each other.
  • the terminal apparatus 30 can further perform the following process.
  • the determination unit 320 can output, to the HMD 500 , the comparison result between comparison information included in the swing support information and the swing analysis information.
  • the comparison information can include a travel time from the swing start position P 1 to the swing turn position P 2 , a waiting time at the swing turn position P 2 , a travel time from the swing turn position P 2 to the impact position P 3 , or information about the swing turn position P 2 in FIG. 6 .
  • the determination unit 320 may create a correction point on the basis of the comparison result, in addition to the comparison result.
  • terminal apparatus 30 can output the following text or sound to the HMD 500 or the earphone 510 through the output processing circuit 305 : “Please take a swing more slowly.” when the travel time is short during a back swing and/or a down swing; “Please hold the golf club at the top.” when the waiting time is short at the turn position P 2 ; or “Please raise the position of the top.” when the turn position is low.
  • the terminal apparatus 30 can determine that the swing of the golf club 100 is stopped and the golf club 100 is in a stationary state for a predetermined period of time or more, using the output from the inertia sensor 40 and then stop the transmission of an image to the HMD 500 .
  • FIG. 8 illustrates another swing support method.
  • a coach 2 there are a coach 2 , another inertia sensor 60 which is mounted to the coach 2 , and another terminal apparatus 70 which is connected to the inertia sensor 60 , in addition to the operator 1 , the terminal apparatus 30 , the inertia sensor 40 , and the HMD 500 illustrated in FIG. 7 .
  • FIG. 8 differs from FIG. 7 in that an example swing and comments of the coach 2 are displayed on the HMD 500 , instead of the transmission of the image of the correction point illustrated in FIG. 7 .
  • the coach 2 can check the swing of the operator 1 , which is transmitted from the terminal apparatus 30 , with a display unit of the terminal apparatus 70 through the Internet and the server 12 illustrated in FIG. 1 .
  • the coach 2 When viewing the swing, the coach 2 demonstrates an example swing for teaching the swing of the operator 1 to be corrected and obtains sensor data for the example swing from the inertia sensor 60 .
  • the sensor data and the input comment data are transmitted together with an example swing registration command from the terminal apparatus 70 to the server 12 through the Internet 10 .
  • the terminal apparatus 30 After a notice indicating the completion of the registration of the example data is transmitted from the server 12 to the terminal apparatus 30 through the Internet 10 , the terminal apparatus 30 can acquire the example swing and the comments from the server 12 .
  • the operator can check the example swing and the comments with the display unit of the HMD 500 or the terminal apparatus 30 .
  • the determination unit 320 illustrated in FIG. 3 can use the deceleration position of the grip 101 during a down swing as the comparison information when comparing the comparison information included in the swing support information with the swing analysis information.
  • the calculation unit 310 illustrated in FIG. 3 can include the structure illustrated in FIG. 9 .
  • a grip acceleration calculation unit 350 calculates the acceleration ash of the grip 101 from a length lsh from the inertia sensor 40 to the grip 101 , on the basis of the swing data stored in the storage device 302 , using the above-mentioned Math. 1 , instead of the length lsj of the clubhead 103 .
  • a maximum value extraction unit 352 extracts the maximum value of the moving speed of the grip 101 .
  • An impact time extraction unit 353 extracts the moving speed of the grip 101 at the time of impact.
  • a change rate calculation unit 354 calculates the change rate r of the speed on the basis of the magnitude of the moving speed extracted by the maximum value extraction unit 352 and the impact time extraction unit 353 , using the following expression.
  • FIG. 10 illustrates a change in the speed of the grip 101 when change rate r of the speed is “0.3”. It should be noted that FIG. 10 also illustrates a change in the speed of the clubhead 103 for reference.
  • the movement of the grip 101 is decelerated from the maximum value of the moving speed on the time axis based on an impact. Therefore, when the position (time) of the maximum value K of the moving speed is specified, the grip 101 is decelerated from the position of the maximum value. Therefore, the deceleration position of the grip 101 is specified.
  • the moving speed of the grip 101 is not reduced, the moving speed at the time of the impact corresponds to the maximum value of the moving speed. As a result, the change rate becomes “0 (zero)”, which means that the grip 101 is not decelerated during a down swing.
  • the determination unit 320 which is connected to the maximum value extraction unit 342 and the change rate calculation unit 344 compares the calculated deceleration position of the grip 101 with the comparison information which has been stored in the storage device 302 in advance.
  • the determination unit 320 can output the following comparison result: “Please roll the wrists in the impact zone.” and “Please roll the wrists early (late) when the deceleration position of the grip 101 deviates.”.
  • the external output device 50 is the sound output device (for example, an earphone) 510 illustrated in FIG. 5 , it is possible to output a rhythm sound synchronously with the image in which the golf club 100 A is moved in an exemplary rhythm along the movement trajectories R 1 and R 2 illustrated in FIG. 7 or without being associated with the image.
  • the external output device 50 may be, for example, a metronome used only for golf.
  • the HMD 500 includes a storage unit
  • the swing support information can be stored in the storage unit of the HMD 500 . In this case, the HMD 500 starts to display an image on the basis of an output start trigger from the terminal apparatus 30 .
  • the motion support method according to the invention is not necessarily applied to the golf swing and can be applied to support various motions including ball games, such as tennis and baseball, medical practices whose exemplary motions need to be mastered, and other motions.

Landscapes

  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A terminal apparatus is connected to an inertia sensor which is mounted to an object to be detected and an external output device by wire or wirelessly. The external output device generates a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside. The terminal apparatus includes: a determination unit that determines whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and a trigger generation unit that generates an output start trigger when the motion satisfies the condition. The output start trigger is used as a start signal which allows the external output device to generate a motion support output and to output the motion support output to the outside.

Description

    TECHNICAL FIELD
  • The present invention relates to a motion support method, a terminal apparatus, and a motion support program.
  • BACKGROUND ART
  • For example, a golf swing support apparatus can be given as an example of a motion support apparatus. The golf support apparatus informs the player of, for example, an address position, a top position, and an impact position with a sound through a wireless earphone, using a metronome (PTL 1) or turns on an LED row provided to the foot of the player in relay to inform the player of the positions (PTL 2). Another golf support apparatus optically detects a position during a back swing after a swing starts, for example, the position of a clubhead that is 20 cm away from the address position, outputs a signal to the player after a predetermined period of time has elapsed from the detection of the position, and informs the player of the time when the swing changes to a down swing (PTL 3).
  • CITATION LIST Patent Literature
  • PTL 1: JP-A-2012-168510
  • PTL 2: JP-A-9-271554
  • PTL 3: JP-A-2012-165860 (Abstract, [0033])
  • SUMMARY OF INVENTION Technical Problem
  • In PTLs 1 and 2, in order for the external output device, such as the wireless earphone or the LED row, to generate a swing support output and outputs the swing support output to the outside, it is necessary to operate, for example, a start switch. Therefore, the player needs to release the golf club and operate, for example, the start switch, which is troublesome. In addition, after operating, for example, the start switch, the player needs to hold the golf club again and it is difficult for the player to take a swing according to the swing support output. In addition, when the LED row is provided along the foot of the player as in PTL 2, the player looks away from a ball during a swing and takes a swing different from the actual swing.
  • On the other hand, In PTL 3, when the player does not actually start a swing, it is difficult for the output device to generate the swing support output. However, it is very difficult to synchronize the swing with a signal which is output during a swing.
  • An object of some aspects of the invention is to provide a motion support method, a terminal apparatus, and a motion support program that can allow an external output device to generate a motion support output at the exact time and to output the motion support output to the outside, without an operator inputting the time when the external output device generates the motion support output and outputs the motion support output to the outside with operation keys.
  • Solution to Problem
  • (1) An aspect of the invention relates to a motion support method including: a step of determining whether a motion of an object to be detected satisfies a preset condition, on the basis of an output signal from an inertia sensor which is mounted to the object to be detected; and a step of generating an output start trigger when the motion satisfies the condition, in which the output start trigger is used as a start signal which allows an external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside.
  • According to the aspect of the invention, the object to be detected is moved and an output signal indicating the detection of the motion of the object to be detected is obtained from the inertia sensor which is mounted to the object to be detected. It is determined whether the motion of the object to be detected satisfies the preset condition on the basis of the output signal. When the motion satisfies the condition, the output start trigger is generated. The output start trigger is used as the start signal which allows the external output device to generate the motion support output (for example, an image, a sound, or vibration) for supporting the motion of the object to be detected and to output the motion support output to the outside. The use of the output start trigger makes it possible to automatically start an operation of allowing the external output device to generate the motion support output and to output the motion support output to the outside on the basis of the motion of the object to be detected. Therefore, a special operation other than the motion of the object to be detected is not needed and the external output device can generate the motion support output and output the motion support output to the outside, on the basis of the motion of the object to be detected. It should be noted that the external output device is preferably a wearable apparatus which is mounted to an operator who is the object to be detected or an operator who operates an equipment, which is the object to be detected, or a portable apparatus which can be carried by the operator. This is because it is easy for the operator to acquire the motion support output.
  • (2) In the aspect of the invention, the condition may include a support target motion of the object to be detected, a motion associated with the support target motion, or a stationary state of the object to be detected for a predetermined period of time or more.
  • Preferably, the conditions for generating the output start trigger include a motion that is indispensable in the prescribed motion of the object to be detected or a stationary state, rather than a motion simply related to a decision. This is because the motion simply related to the decision is an extra motion and is likely to be forgotten. For example, when a support target is a golf swing, the support target motion corresponds to, for example, a practice golf swing, the associated motion corresponds to, for example, waggle, and the stationary state corresponds to an address state. The motion or the stationary state is a natural motion in the support target motion, for example, the golf swing. It is possible to detect the natural motion and to generate the output start trigger.
  • (3) In the aspect of the invention, motion support information which is used by the external output device to generate the motion support output may be transmitted to the external output device, on the basis of the output start trigger.
  • In this case, the external output device does not need to store the motion support information (for example, an image signal, a sound signal, or a frequency signal) for outputting the motion support output (for example, an image, a sound, or vibration). The external output device can generate the motion support output and output the motion support output to the outside, on the basis of the motion support information which is transmitted to the external output device after the output start trigger is generated.
  • (4) In the aspect of the invention, the output start trigger may be transmitted to the external output device that stores motion support information for generating the motion support output.
  • (5) In the aspect of the invention, when the external output device includes a display device, the motion support output may be an image. In this case, it is possible to support the motion of the object to be detected, using the output of the image.
  • (6) In the aspect of the invention, when the external output device includes a head-mounted display that is mounted to the object to be detected, the motion support output may be an image. In this case, it is possible to support the motion of the object to be detected, using the output of the image. Therefore, the operator can recognize the image while maintaining the line of sight required for a motion.
  • (7) In the aspect of the invention, when the external output device includes a sound output device, the motion support output may be a sound. In this case, it is possible to support the motion of the object to be detected, using a sound, for example, a rhythm sound.
  • (8) In the aspect of the invention, when the object to be detected is an operator who swings exercise equipment or the exercise equipment, the motion support output may include an image for a swing period from the start of the swing to at least an impact or a rhythm sound for the swing period. It is possible to support a swing from a swing start position to an impact position, using the image or the rhythm sound for the swing period.
  • (9) In the aspect of the invention, analysis information which is obtained by analyzing the motion of the object to be detected may be collected on the basis of an output from the inertia sensor, and a comparison result between comparison information included in the motion support information and the analysis information may be transmitted to the external output device. Since the comparison result is output to the outside, it is possible to accurately support the motion of the object to be detected.
  • (10) In the aspect of the invention, when the object to be detected is an operator who holds a grip of exercise equipment and swings the exercise equipment or the exercise equipment, the comparison information may include a travel time from a swing start position to a swing turn position, a waiting time at the swing turn position, a travel time from the swing turn position to an impact position, and information about the swing turn position or a deceleration position of the grip during a down swing. It is possible to accurately evaluate the swing of the object to be detected on the basis of the comparison information.
  • (11) Another aspect of the invention relates to a terminal apparatus that is connected to an inertia sensor mounted to an object to be detected and an external output device by wire or wirelessly and supports a motion of the object to be detected, the terminal apparatus including: a determination unit that determines whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and a trigger generation unit that generates an output start trigger when the motion satisfies the condition, in which the output start trigger is used as a start signal which allows the external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside. According to the another aspect of the invention, it is possible to preferably perform the motion support method according to the aspect of the invention.
  • (12) Still another aspect of the invention relates to a motion support program that is used in a terminal apparatus which includes a computer, is connected to an inertia sensor mounted to an object to be detected and an external output device by wire or wirelessly, and supports a motion of the object to be detected, in which the motion support program causes the computer to perform: a procedure for determining whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and a procedure for generating an output start trigger which allows the external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside when the motion satisfies the condition. When the motion support program is executed, it is possible to implement the motion support method or the terminal apparatus according to the aspect of the invention, using software.
  • BRIEF DESCRIPTION OF DRAWINGS
  • FIG. 1 is a diagram illustrating the entire structure of a swing support system according to an embodiment of the invention.
  • FIG. 2 is a block diagram illustrating a swing support apparatus illustrated in FIG. 1.
  • FIG. 3 is a block diagram illustrating a terminal apparatus illustrated in FIG. 2.
  • FIG. 4 is a diagram illustrating a head-mounted display and a wireless earphone which are examples of an external output device.
  • FIG. 5 is a diagram illustrating processes of a swing support method according to an embodiment of the invention.
  • FIG. 6 is a diagram illustrating a golf swing.
  • FIG. 7 is a diagram illustrating an example of an image displayed on a display device which is the external output device.
  • FIG. 8 is a diagram illustrating processes of a swing support method according to another embodiment of the invention.
  • FIG. 9 is a block diagram illustrating an example of a position calculation unit illustrated in FIG. 3.
  • FIG. 10 is a diagram illustrating the deceleration of a grip from the maximum value of the speed of the grip.
  • DESCRIPTION OF EMBODIMENTS
  • Hereinafter, an embodiment of the invention will be described with reference to the accompanying drawings. The following embodiment does not unduly limit the content of the invention described in the scope of the appended claims and all of the structures described in this embodiment are not indispensable as the solutions of the invention.
  • (1) Motion Support System and Motion Support Apparatus
  • FIG. 1 is a diagram illustrating the entire structure of a swing support system (a motion support system in a broad sense) according to the invention which is applied to golf swing support. As illustrated in FIG. 1, for example, servers 12 and 14 and a base station 16 are connected to a network such as an Internet 10. The server 12 is a program distribution server which distributes, for example, a swing support program or a swing analysis program.
  • A terminal apparatus 30 is a portable terminal, such as a mobile phone which can communicate with the program distribution server 12 through the base station 16 and the Internet 10, or a personal computer which can communicate with the program distribution server 12 through the server 14 and the Internet 10. A swing support apparatus (a motion support apparatus in a broad sense) 20 includes the terminal apparatus 30, an inertia sensor 40 which is attached to at least one of an operator (player), who is an object to be detected, and a golf club, and an external output device 50 which generates a swing support output (a motion support output in a broad sense) and outputs the swing support output to the outside. A storage unit of the terminal apparatus 30 stores the swing support program or the swing analysis program downloaded from the program distribution server 12.
  • For example, an acceleration sensor or a gyro sensor (angular speed sensor) is incorporated into the inertia sensor 40. The acceleration sensor can detect acceleration in each of the directions of three axes x, y, and z which are perpendicular to each other. The gyro sensor can detect an angular speed about each of the three axes x, y, and z which are perpendicular to each other. The inertia sensor 40 outputs detection signals of the acceleration and the angular speed for each axis. The y-axis is aligned with the axial direction of a shaft 102 and the x-axis is aligned with a hitting direction A.
  • The inertia sensor 40 is attached to, for example, a golf club (exercise equipment) 100. The golf club 100 includes a grip 101 and the shaft 102. A clubhead 103 is coupled to the leading end of the shaft 102. The inertia sensor 40 is attached to the grip 101 or the shaft 102 of the golf club 100.
  • The terminal apparatus 30 includes an arithmetic processing circuit 300. The inertia sensor 40 is connected to the arithmetic processing circuit 300 through an interface 301. The interface 301 is connected to the inertia sensor 40 by wire or wirelessly. The detection signal is supplied from the inertia sensor 40 to the arithmetic processing circuit 300.
  • A storage device 302 is connected to the arithmetic processing circuit 300. The storage device 302 can store a program 303, such as a swing support program (motion support program) or a swing analysis software program (motion analysis program), and data. The program 303 is downloaded from the program distribution server 12 and is then stored. The arithmetic processing circuit 300 executes the swing support program using the detection signal from the inertia sensor 40 to control the external output device 50. The arithmetic processing circuit 300 executes the golf swing analysis software program to perform golf swing analysis. The storage device 302 can include, for example, a dynamic random access memory (DRAM) or a high-capacity storage device unit and a non-volatile memory. For example, the program 303 is stored in the DRAM. When the terminal apparatus 30 includes a high-capacity storage device unit, such as a hard disk drive (HDD), the program 303 and data can be stored in the HDD. The non-volatile memory stores a relatively low-capacity program, such as a basic input/output system (BIOS), or data.
  • An input device 304 is connected to the arithmetic processing circuit 300. The input device 304 includes at least alphabet keys and a numeric keypad. Text information or numerical information is input from the input device 304 to the arithmetic processing circuit 300.
  • An output processing circuit 305 is connected to the arithmetic processing circuit 300. The arithmetic processing circuit 300 transmits predetermined output data to the output processing circuit 305. The output data (swing support information) processed by the output processing circuit 305 is transmitted to the external output device 50 through the interface 301. Here, when the external output device 50 is a display device, the output data is image data. When the external output device is a sound output device, such as an earphone, a headphone, or a speaker, the output data is sound data. When the external output device 50 includes both the display device and the sound output device, the output data is image data and sound data which are synchronized with each other.
  • Here, as illustrated in FIG. 3, the arithmetic processing circuit 300 can include a calculation unit 310, a determination unit 320, a trigger generation unit 330, and a data transmission unit 340. The determination unit 320 determines whether the motion of the golf club 100, which is the object to be detected, satisfies a preset condition on the basis of the detection signal from the inertia sensor 40. The determination unit 320 may receive the detection signal from the inertia sensor 40 through the interface 301 or may receive the calculation result of the calculation unit 310 on the basis of the detection signal.
  • The preset condition may be a motion simply related to a decision. However, rather, the preset condition preferably includes a motion that is indispensable in the prescribed motion of the golf club 100, which is the object to be detected, or a stationary state. The reason is that a motion which is clearly distinguished from a swing motion (for example, a motion of tapping the golf club on the ground and swinging the shaft in one direction or the forward and reverse directions) is the motion simply related to a decision, is not necessary for a swing, and is likely to be forgotten. Examples of the motion which is indispensable in the prescribed motion of the golf club 100 or the stationary state can include a practice swing motion, an address motion, and a waggle motion. The practice swing corresponds to a support target motion or a preparatory action having the same content as the support target motion. The waggle (which will be described in detail below) corresponds to a motion associated with the support target motion and the address state corresponds to a stationary state. The motion or the stationary state is a natural motion in the golf swing. It is possible to detect the natural motion or the stationary state and to generate an output start trigger. For example, for the address motion, it is possible to determine whether the golf club 100 is in a stationary state at an address position (swing start position) for a predetermined period of time (for example, a few seconds) or more. In any case, the calculation unit 310 calculates, for example, the acceleration, speed, or position of the clubhead 103, using the output from the inertia sensor 40. For example, when the clubhead 103 is located at the position of a ball, the determination unit 320 can determine that the golf club 100 is at the address position. Further, the determination unit 320 can determine whether the golf club 100 satisfies the preset condition, such as a stationary state, a swing state, or a motion other than a swing, on the basis of the output signal from the inertia sensor 40.
  • When the position of the clubhead 103 is detected, the calculation unit 310 calculates the acceleration asj (including gravity g) of the clubhead 103, using, for example, the following expression. When the acceleration is calculated, the calculation unit 310 specifies the position lsj of the clubhead 103 according to a local coordinate system Σs unique to the inertia sensor 40. The position lsj of the clubhead 103 can be designated through, for example, the input device 304 and then stored in the storage device 302. An acceleration output as and an angular speed output ωs are obtained from the inertia sensor 40. Angular acceleration (which is represented by ωs having a dot at the top in the following expression) is obtained by differentiating the angular speed output ωs.

  • αsjs+{dot over (ω)}s ×l sjs×(ωs ×l sj)+g  [Math. 1]
  • The calculation unit 310 calculates the moving speed of the clubhead 103 on the basis of the calculated acceleration. Here, an integral process is performed on the acceleration at a prescribed sampling interval dt, according to the following expression.
  • V sj ( 0 ) = 0 V sj ( t ) = n = 1 t α sj ( n ) · dt ( t = 1 , , N ) [ Math . 2 ]
  • In addition, the calculation unit 310 calculates the position of the clubhead 103 on the basis of the calculated speed. Here, an integral process is performed on the speed at a prescribed sampling interval dt.
  • P sj ( t ) = n = 1 t V sj ( n ) · dt ( t = 1 , , N ) [ Math . 3 ]
  • When the determination unit 320 determines that the above-mentioned conditions are satisfied, the trigger generation unit 330 generates an output start trigger. The data transmission unit 340 reads swing support information, which has been stored in the storage device 302 in advance, to the output processing circuit 305 on the basis of the output start trigger. In this case, the output start trigger may be a read signal. The swing support information processed by the output processing circuit 305 is transmitted to the external output device 50 through the interface 301. The external output device 50 generates a swing support output on the basis of the input swing support information and outputs the swing support output to the outside. It should be noted that the swing support information which has been stored in the storage device 302 in advance may be transmitted to the external output device 50 through the interface 301, without passing through the output processing circuit 305. In this case, the output processing circuit 305 is not needed. In addition, when the swing support information has been stored in the external output device 50, the data transmission unit 340 is not needed, which will be described below. In this case, the output start trigger generated by the trigger generation unit 330 is input to the external output device 50 through the interface 301.
  • (2) External Output Device
  • FIG. 4 is a head-mounted display device (head-mounted display: HMD) 500 which is an example of a display device as the external output device 50. As illustrated in FIG. 4, the HMD 500 includes a spectacle body 501 which is mounted to the head of an operator 1. A display unit 502 is provided in the spectacle body 501. The display unit 502 displays a virtual image, which is the swing support output, on the basis of the swing support information transmitted from the terminal apparatus 30, so as to overlap a real image that can be seen through the spectacle body 501.
  • The display unit 502 can include, for example, an image display unit 503, such as a liquid crystal display (LCD), a first beam splitter 504, a second beam splitter 505, a first concave reflecting mirror 506, a second concave reflecting mirror 507, a shutter 508, and a convex lens 509. The first beam splitter 504 is provided in front of the left eye of the operator and partially transmits and reflects light from the image display unit 503. The second beam splitter 505 is provided in front of the right eye of the operator and partially transmits and reflects light from the first beam splitter 504. The first concave reflecting mirror 506 is provided in front of the first beam splitter 504 and partially reflects the partially reflected light from the first beam splitter 504 so as to be guided to the left eye of the operator through the first beam splitter 504. The second concave reflecting mirror 507 is provided in front of the second beam splitter 505 and partially reflects the partially reflected light from the second beam splitter 505 so as to be guided to the right eye of the operator through the second beam splitter 505. The convex lens 509 guides the partially transmitted light from the second beam splitter 505 to the outside of the HMD 500 when the shutter 508 is opened. When the HMD 500 is used as the external output device 50, the operator 1 can see the swing support output displayed on the HMD 500 while facing in the eye direction required for a swing operation.
  • FIG. 4 also illustrates an earphone 510 which is an example of a sound output device as the external output device 50. The earphone 510 can be, for example, a wireless earphone which is wirelessly connected to the terminal apparatus 30. However, the earphone 510 may be connected to the terminal apparatus 30 by wire. This structure holds for the HMD 500. For example, the earphone 510 generates a rhythm sound (swing support output) on the basis of the swing support information from the terminal apparatus 30 and outputs the rhythm sound to the outside. When the earphone 510 is used as the external output device 50, the operator 1 can listen to the rhythm sound which is the swing support output from the earphone 510, without being affected by an ambient environmental sound.
  • (3) Swing Support Method
  • A swing support method will be described with reference to FIGS. 5 to 7. FIG. 5 illustrates the operation of the operator 1 (golf club 100), the terminal apparatus 30, and the display device 500. FIG. 6 illustrates a swing trajectory R of the clubhead 103 of the golf club 100 operated by the operator 1. The swing trajectory R includes a swing start (address) position P1, a top (swing turn) position P2, an impact position P3, and a finish position P4. The calculation unit 310 calculates the positions Psj(t) of the clubhead 103 during a swing using the above-mentioned expression to calculate the swing trajectory R.
  • In FIG. 5, when the operator 1 starts a swing, the terminal apparatus 30 can monitor the motion of the operator 1 (golf club 100) using the output from the inertia sensor 40. When the detection signal from the inertia sensor 40 is input to the terminal apparatus 30, the calculation unit 310 calculates, for example, the acceleration αsj and the position Psj of the clubhead 103 using the above-mentioned expressions, if necessary. The determination unit 320 determines whether the motion of the operator 1 (golf club 100) satisfies a preset condition, on the basis of the detection signal (acceleration, angular speed) from the inertia sensor 4 or the calculation result of the calculation unit 310. The practice full swing illustrated in FIG. 6 can be given as an example of the condition. When a full swing having four positions P1 to P4 illustrated in FIG. 6 is detected on the basis of the calculation result of the calculation unit 310, the determination unit 320 can determine that the motion satisfies the condition. As another example of the condition, the determination unit 320 can determine whether the clubhead 103 is in a stationary state for a predetermined period of time or more at the position P1 or in the vicinity of the position P1. As still another example of the condition, the determination unit 320 may determine a waggle motion which reciprocates the clubhead 103 a plurality of times between the position P1 and a position that is away from the position P1 by a distance corresponding to a short stroke in a take-back direction. The waggle motion is made immediately before a swing starts. The determination unit 320 can determine the waggle motion on the basis of information about the acceleration αsj and the position Psj of the clubhead 103.
  • When the determination unit 320 determines that the motion satisfies the condition, the trigger generation unit 330 generates the output start trigger. Then, the data transmission unit 340 reads the swing support information from the storage device 302 and transmits the swing support information to the HMD 500, which is the external output device 50, through the interface 301. Then, an image, which is the swing support output, is displayed on the HMD 500. Therefore, the operator 1 does not need to operate a start switch and can start the display of an image on the HMD 500 while holding the grip 101.
  • FIG. 7 illustrates an example of the display of the swing support output. In FIG. 7, the swing support output is a moving picture indicating the movement of a clubhead 103A of a golf club 100A operated by an avatar 1A. In FIG. 7, the movement trajectory of a back swing is represented by R1 and the movement trajectory of a down swing is represented by R2. The golf club 100A is moved in an exemplary rhythm along the movement trajectories R1 and R2. As illustrated in FIG. 7, the swing support output preferably includes an image from the start (address) of a swing to at least an impact and more preferably includes an image from start to finish.
  • The operator 1 can repeatedly view the swing support output (moving picture) illustrated in FIG. 7 with the HMD 500. Then, the operator 1 actually takes a swing with reference to the swing support output (moving picture). The swing motion is detected by the inertia sensor 40 and is then input to the terminal apparatus 30. The swing motion of the operator 1 is analyzed by the terminal apparatus 30 and is acquired as swing analysis information. The swing analysis information can be displayed on the HMD 500, similarly to FIG. 7. Preferably, the swing support information and the actual swing (swing analysis information) can be displayed as moving pictures in parallel or alternately so as to be contrasted with each other.
  • In this embodiment, the terminal apparatus 30 can further perform the following process. The determination unit 320 can output, to the HMD 500, the comparison result between comparison information included in the swing support information and the swing analysis information. The comparison information can include a travel time from the swing start position P1 to the swing turn position P2, a waiting time at the swing turn position P2, a travel time from the swing turn position P2 to the impact position P3, or information about the swing turn position P2 in FIG. 6. The determination unit 320 may create a correction point on the basis of the comparison result, in addition to the comparison result. For example, terminal apparatus 30 can output the following text or sound to the HMD 500 or the earphone 510 through the output processing circuit 305: “Please take a swing more slowly.” when the travel time is short during a back swing and/or a down swing; “Please hold the golf club at the top.” when the waiting time is short at the turn position P2; or “Please raise the position of the top.” when the turn position is low.
  • Even when the display of an image on the HMD 500 is stopped, it is possible to use the output from the inertia sensor 40. For example, the terminal apparatus 30 can determine that the swing of the golf club 100 is stopped and the golf club 100 is in a stationary state for a predetermined period of time or more, using the output from the inertia sensor 40 and then stop the transmission of an image to the HMD 500.
  • FIG. 8 illustrates another swing support method. In FIG. 8, there are a coach 2, another inertia sensor 60 which is mounted to the coach 2, and another terminal apparatus 70 which is connected to the inertia sensor 60, in addition to the operator 1, the terminal apparatus 30, the inertia sensor 40, and the HMD 500 illustrated in FIG. 7. FIG. 8 differs from FIG. 7 in that an example swing and comments of the coach 2 are displayed on the HMD 500, instead of the transmission of the image of the correction point illustrated in FIG. 7. The coach 2 can check the swing of the operator 1, which is transmitted from the terminal apparatus 30, with a display unit of the terminal apparatus 70 through the Internet and the server 12 illustrated in FIG. 1. When viewing the swing, the coach 2 demonstrates an example swing for teaching the swing of the operator 1 to be corrected and obtains sensor data for the example swing from the inertia sensor 60. The sensor data and the input comment data are transmitted together with an example swing registration command from the terminal apparatus 70 to the server 12 through the Internet 10. After a notice indicating the completion of the registration of the example data is transmitted from the server 12 to the terminal apparatus 30 through the Internet 10, the terminal apparatus 30 can acquire the example swing and the comments from the server 12. The operator can check the example swing and the comments with the display unit of the HMD 500 or the terminal apparatus 30.
  • This embodiment has been described in detail above. It will be readily understood by those skilled in that art that various modifications and changes to the invention can be made, without substantially departing from the new matters and effects of the invention. Therefore, all of the modification examples are included in the scope of the invention. For example, a term which is described in the specification or the drawings at least one time together with a different term having a wider or similar meaning can be replaced with the different term at any position of the specification or the drawings.
  • For example, the determination unit 320 illustrated in FIG. 3 can use the deceleration position of the grip 101 during a down swing as the comparison information when comparing the comparison information included in the swing support information with the swing analysis information. In this case, the calculation unit 310 illustrated in FIG. 3 can include the structure illustrated in FIG. 9.
  • In FIG. 9, a grip acceleration calculation unit 350 calculates the acceleration ash of the grip 101 from a length lsh from the inertia sensor 40 to the grip 101, on the basis of the swing data stored in the storage device 302, using the above-mentioned Math. 1, instead of the length lsj of the clubhead 103. A grip speed calculation unit 351 calculates the speed Vsh(t) of the grip 101, using Vsh(0)=0 and the acceleration ash of the grip 101 calculated by the grip acceleration calculation unit 350, in the above-mentioned Math. 2.
  • A maximum value extraction unit 352 extracts the maximum value of the moving speed of the grip 101. An impact time extraction unit 353 extracts the moving speed of the grip 101 at the time of impact. A change rate calculation unit 354 calculates the change rate r of the speed on the basis of the magnitude of the moving speed extracted by the maximum value extraction unit 352 and the impact time extraction unit 353, using the following expression.

  • η=l−(the speed at the time of impact/the maximum value of the speed)  [Math. 4]
  • Here, FIG. 10 illustrates a change in the speed of the grip 101 when change rate r of the speed is “0.3”. It should be noted that FIG. 10 also illustrates a change in the speed of the clubhead 103 for reference. The movement of the grip 101 is decelerated from the maximum value of the moving speed on the time axis based on an impact. Therefore, when the position (time) of the maximum value K of the moving speed is specified, the grip 101 is decelerated from the position of the maximum value. Therefore, the deceleration position of the grip 101 is specified. When the moving speed of the grip 101 is not reduced, the moving speed at the time of the impact corresponds to the maximum value of the moving speed. As a result, the change rate becomes “0 (zero)”, which means that the grip 101 is not decelerated during a down swing.
  • The determination unit 320 which is connected to the maximum value extraction unit 342 and the change rate calculation unit 344 compares the calculated deceleration position of the grip 101 with the comparison information which has been stored in the storage device 302 in advance. When the change rate 1 calculated from the swing data is “0”, the determination unit 320 can output the following comparison result: “Please roll the wrists in the impact zone.” and “Please roll the wrists early (late) when the deceleration position of the grip 101 deviates.”.
  • Further, when the external output device 50 is the sound output device (for example, an earphone) 510 illustrated in FIG. 5, it is possible to output a rhythm sound synchronously with the image in which the golf club 100A is moved in an exemplary rhythm along the movement trajectories R1 and R2 illustrated in FIG. 7 or without being associated with the image. In addition, the external output device 50 may be, for example, a metronome used only for golf. When the HMD 500 includes a storage unit, the swing support information can be stored in the storage unit of the HMD 500. In this case, the HMD 500 starts to display an image on the basis of an output start trigger from the terminal apparatus 30.
  • In addition, for example, the motion support method according to the invention is not necessarily applied to the golf swing and can be applied to support various motions including ball games, such as tennis and baseball, medical practices whose exemplary motions need to be mastered, and other motions.
  • REFERENCE SIGNS LIST
    • 1: operator
    • 10: network
    • 12: swing support program distribution server
    • 20: motion support apparatus (swing support apparatus)
    • 30: terminal apparatus
    • 40: inertia sensor
    • 50: external output device
    • 100: golf club
    • 103: clubhead
    • 300: arithmetic processing circuit
    • 301: interface
    • 302: storage device
    • 304: input device
    • 305: output processing circuit
    • 310: calculation unit
    • 320: determination unit
    • 330: trigger generation unit
    • 340: data transmission unit
    • 500: head-mounted display
    • 510: sound output device (earphone)

Claims (12)

1. A motion support method comprising:
a step of determining whether a motion of an object to be detected satisfies a preset condition, on the basis of an output signal from an inertia sensor which is mounted to the object to be detected; and
a step of generating an output start trigger when the motion satisfies the condition,
wherein the output start trigger is used as a start signal which allows an external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside.
2. The motion support method according to claim 1, wherein
the condition includes a support target motion of the object to be detected, a motion associated with the support target motion, or a stationary state of the object to be detected for a predetermined period of time or more.
3. The motion support method according to claim 1, wherein
motion support information which is used by the external output device to generate the motion support output is transmitted to the external output device, on the basis of the output start trigger.
4. The motion support method according to claim 1, wherein
the output start trigger is transmitted to the external output device that stores motion support information for generating the motion support output.
5. The motion support method according to claim 1, wherein,
when the external output device includes a display device, the motion support output is an image.
6. The motion support method according to claim 1, wherein,
when the external output device includes a head-mounted display that is mounted to the object to be detected, the motion support output is an image.
7. The motion support method according to claim 1, wherein,
when the external output device includes a sound output device, the motion support output is a sound.
8. The motion support method according to claim 5, wherein,
when the object to be detected is an operator who swings exercise equipment or the exercise equipment, the motion support output includes an image for a swing period from the start of the swing to at least an impact or a rhythm sound for the swing period.
9. The motion support method according to claim 3, wherein:
analysis information which is obtained by analyzing the motion of the object to be detected is collected on the basis of an output from the inertia sensor, and
comparison result between comparison information included in the motion support information and the analysis information is transmitted to the external output device.
10. The motion support method according to claim 9, wherein,
when the object to be detected is an operator who holds a grip of exercise equipment and swings the exercise equipment or the exercise equipment, the comparison information includes a travel time from a swing start position to a swing turn position, a waiting time at the swing turn position, a travel time from the swing turn position to an impact position, and information about the swing turn position or a deceleration position of the grip during a down swing.
11. A terminal apparatus that is connected to an inertia sensor mounted to an object to be detected and an external output device by wire or wirelessly and supports a motion of the object to be detected, comprising:
a determination unit that determines whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and
a trigger generation unit that generates an output start trigger when the motion satisfies the condition, wherein
the output start trigger is used as a start signal which allows the external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside.
12. A motion support program that is used in a terminal apparatus which includes a computer, is connected to an inertia sensor mounted to an object to be detected and an external output device by wire or wirelessly, and supports a motion of the object to be detected and causes the computer to perform:
a procedure for determining whether the motion of the object to be detected satisfies a preset condition, on the basis of an output signal from the inertia sensor; and
a procedure for generating an output start trigger which allows the external output device to generate a motion support output for supporting the motion of the object to be detected and to output the motion support output to the outside when the motion satisfies the condition.
US15/110,438 2014-03-27 2015-03-24 Motion support method, terminal apparatus, and motion support program Abandoned US20160325161A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2014066073A JP2015188506A (en) 2014-03-27 2014-03-27 Motion support method, terminal device, and motion support program
JP2014-066073 2014-03-27
PCT/JP2015/001673 WO2015146156A1 (en) 2014-03-27 2015-03-24 Motion support method, terminal device, and motion support program

Publications (1)

Publication Number Publication Date
US20160325161A1 true US20160325161A1 (en) 2016-11-10

Family

ID=54194703

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/110,438 Abandoned US20160325161A1 (en) 2014-03-27 2015-03-24 Motion support method, terminal apparatus, and motion support program

Country Status (3)

Country Link
US (1) US20160325161A1 (en)
JP (1) JP2015188506A (en)
WO (1) WO2015146156A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150224380A1 (en) * 2014-02-13 2015-08-13 Mizuno Corporation Measurement System and Measurement Device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5718639A (en) * 1994-06-20 1998-02-17 Thrustmaster, Inc. Opto-electric golf club swing sensing system having vertically offset sensors
US20070206837A1 (en) * 2006-03-03 2007-09-06 Kirby Richard A Portable Swing Analyzer
US20070298896A1 (en) * 2006-06-21 2007-12-27 Nusbaum Mark E Electronically controlled golf swing analyzing/training mat system with ball striking-related feedback
US20110270135A1 (en) * 2009-11-30 2011-11-03 Christopher John Dooley Augmented reality for testing and training of human performance
US20110313552A1 (en) * 2010-05-11 2011-12-22 Golf Impact Llc Golf Free Swing Measurement and Analysis System

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04146777A (en) * 1990-10-11 1992-05-20 Seiko Instr Inc Electronic sport swing trainer
JPH10272216A (en) * 1997-03-31 1998-10-13 Tokico Ltd Swing diagnosing device
JP2008073210A (en) * 2006-09-21 2008-04-03 Seiko Epson Corp Golf club and its swing evaluation support apparatus
JP2009106323A (en) * 2007-10-26 2009-05-21 Yamaha Corp Motion form evaluation device and motion form evaluation program
JP2009125507A (en) * 2007-11-27 2009-06-11 Panasonic Electric Works Co Ltd Golf improvement support device
JP5761506B2 (en) * 2011-06-09 2015-08-12 セイコーエプソン株式会社 Swing analysis apparatus, swing analysis system, swing analysis method, swing analysis program, and recording medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5718639A (en) * 1994-06-20 1998-02-17 Thrustmaster, Inc. Opto-electric golf club swing sensing system having vertically offset sensors
US20070206837A1 (en) * 2006-03-03 2007-09-06 Kirby Richard A Portable Swing Analyzer
US20070298896A1 (en) * 2006-06-21 2007-12-27 Nusbaum Mark E Electronically controlled golf swing analyzing/training mat system with ball striking-related feedback
US20110270135A1 (en) * 2009-11-30 2011-11-03 Christopher John Dooley Augmented reality for testing and training of human performance
US20110313552A1 (en) * 2010-05-11 2011-12-22 Golf Impact Llc Golf Free Swing Measurement and Analysis System

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150224380A1 (en) * 2014-02-13 2015-08-13 Mizuno Corporation Measurement System and Measurement Device

Also Published As

Publication number Publication date
WO2015146156A1 (en) 2015-10-01
JP2015188506A (en) 2015-11-02

Similar Documents

Publication Publication Date Title
US8998717B2 (en) Device and method for reconstructing and analyzing motion of a rigid body
US20150111657A1 (en) Movement analysis method, movement analysis apparatus, and movement analysis program
JP6467766B2 (en) Motion analysis method, motion analysis apparatus, and motion analysis program
JP6467698B2 (en) Baseball batting practice support system
US10512833B2 (en) Presentation method, swing analysis apparatus, swing analysis system, swing analysis program, and recording medium
US20170203172A1 (en) Equipment display method, swing analysis apparatus, swing analysis system, swing analysis program, and recording medium
KR20140148298A (en) Motion analysis method and motion analysis device
US20150283427A1 (en) Motion analysis system and motion analysis method
TW201501751A (en) Motion analysis device
US20170239520A1 (en) Motion analysis apparatus, motion analysis system, motion analysis method, recording medium, and display method
US20170120122A1 (en) Electronic apparatus, system, method, program, and recording medium
US20170203153A1 (en) Electronic apparatus, system, determination method, determination program, and recording medium
US10286285B2 (en) Display method, display apparatus, motion analysis system, motion analysis program, and recording medium
JP2016067408A (en) Sensor, arithmetic unit, movement measurement method, movement measurement system, and program
US20170024610A1 (en) Motion analysis apparatus, motion analysis system, motion analysis method, and display method and program of motion analysis information
TWI530821B (en) Head-mounted display system and operation method thereof
US20170011652A1 (en) Motion analysis method, motion analysis apparatus, motion analysis system, and program
US20170203188A1 (en) Display method, motion analysis apparatus, motion analysis system, motion analysis program, and recording medium
US20160325161A1 (en) Motion support method, terminal apparatus, and motion support program
JP2019187501A (en) Swing analysis system and swing analysis method
KR102000151B1 (en) System for analyzing golf swing pattern by using wearable device
US11074826B2 (en) Frameworks and methodologies configured to enable real-time adaptive delivery of skills training data based on monitoring of user performance via performance monitoring hardware
JP6340646B2 (en) Sway detection device, sway detection system, and sway detection program
US10946242B2 (en) Swing analysis device, swing analysis method, and swing analysis system
CN107229400B (en) Display device support method, display device support system, and electronic device

Legal Events

Date Code Title Description
AS Assignment

Owner name: SEIKO EPSON CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NAKAJO, SHOICHI;REEL/FRAME:039106/0482

Effective date: 20160422

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION