US20150057775A1

US20150057775A1 - System and method for capturing and using move data

Info

Publication number: US20150057775A1
Application number: US13/971,138
Authority: US
Inventors: Naxin Dong
Original assignee: Individual
Current assignee: Individual
Priority date: 2013-08-20
Filing date: 2013-08-20
Publication date: 2015-02-26
Also published as: WO2015025221A3; WO2015025221A2

Abstract

A system and method for capturing and using move data from a user are disclosed. The system may use one or more sensors to capture raw data about a physical activity of the user. The raw data about the physical activity of the user may be processed into one or more pieces of move data relating to the physical activity of the user. The processed move data may be used for various purposes, such as, for example, to play a virtual game, to provide to a social network system or to provide training to the user.

Description

FIELD

The disclosure relates generally to a system for capturing move data while a user plays a game.

BACKGROUND

Currently, the majority of available sports gear and accessories cannot record personal exercise/training data of a user and track the changes over time. In addition, current sport gear and accessories do not allow users to participate in social functions or to extract the real exercise data and use the real exercise data for other purposes.
Soccer balls exist that have sensors within the soccer ball that may be known as smart soccer balls. However, the design of current smart soccer balls lack sufficient sensors to capture the interaction between the ball and the players. Specifically, current smart soccer balls have sensors that can be used only to capture simple data, such as kicking force, direction and height, but do not capture more complicated data such as goals scored or passes between players. No system for group or individual sports currently exists that allows users to capture this kind of data.
Due to the limitations of the smart soccer balls, users cannot collect data and determine tell who kicked the ball, where the ball was kicked, and to whom the ball was kicked. Therefore, users cannot incorporate data from current smart soccer balls into social media platforms or combine real-life game play data into virtual worlds or video games.
Some prior art methods also required players to separately attach sensors to the ball, which significantly increased the cost.
Thus, it is desirable to provide a system for collecting data from game play to allow users to incorporate it into social media platforms, virtual worlds, or video games, and it is to this end that the disclosure is directed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an implementation of a system for capturing and using move data from a user;

FIG. 2 is a diagram illustrating an example of a ball that may be used with the system;

FIG. 3 is a diagram illustrating an example of user worn sensor that may be used with the system;

FIG. 4 illustrates a circuit block diagram of a sensor that may be used by the system;

FIG. 5 is a diagram illustrating an example of the system used for a basketball game;

FIG. 6 is a flowchart illustrating the system used to generate move data from raw data gathered from sensors;

FIGS. 7 and 8 illustrate raw data being generated for two different basketball games;

FIG. 9 is a chart illustrating an example of raw data gathered from one or more sensors;

FIG. 10 is a chart illustrating an example of basketball move data generated from an application programming interface;

FIGS. 11A and 11B illustrate a method for generating basketball move data;

FIG. 12 illustrates an example of a basketball game implement with a set of sensors; and

FIG. 13 illustrates an example of basketball statistics that may be generated by the system using the move data.

DETAILED DESCRIPTION OF ONE OR MORE EMBODIMENTS

The disclosure will be described in detail as it pertains to a specific example of a sport (basketball) in which move data is captured and used and to a specific example of a use of the move data (a virtual game.) However, the system and method is not limited to the examples used for illustration purposes. For example, the system may be used to capture data about various different physical activities of the user, such as soccer, football, badminton, baseball, tennis, golf, ice hockey, volleyball, cricket, squash or any other event/game/sport/exercise in which it would be desirable to be able to capture raw data about a plurality of movements of one or more users. Furthermore, the system may provide the move data of the one or more users to a social network, a training aid and other systems that may use the move data.
The system is described below as being implemented as a system with one or more sensors and a backend system that gathers the raw data, converts the raw data into a set of move data or each player and then sends the generated move data to various external systems and sites. However, the system may be implemented in which each sensor may perform its own processing of the raw data to generate the move data and then interact with the external systems (without a backend.)
FIG. 1 is a diagram of an implementation of a system 100 for capturing and using move data from a user. The system may use one or more sensors 102 to capture raw data about a physical activity of the user. For example, depending on the physical activity of the user, the sensor may be located on a body part of a user 104, such as an arm or leg, attached to or embedded in a ball 106 or attached to or embedded in an implement 108 used for the physical activity, such as a bat for a baseball activity, a racket for tennis or badminton, a basketball hoop for basketball, a soccer goal for soccer and the like. In addition, for certain physical activities, such as a game of basketball, there may also be one or more sensors near a basketball rim at end of the court that may collect raw data about a ball being near the basketball rim and passing through the basketball rim to indicate a successful shot. Similar, for a soccer game, the goal at each end of the soccer field may have one or more sensors to provide raw data about when the soccer ball goes into the goal. Other physical activities may have similar additional sensors. In addition, the system 100 may be used by multiple users who each have the sensors 102 to measure the movement and actions of the user during the physical activity.
Each sensor 102 may capture raw data and may be a sensor without a memory or a sensor that includes a memory. Thus, in one embodiment, each sensor 102 may store the raw data about the player and then periodically upload the data over a link 110 to a backend component 120. In another embodiment, each sensor may store the raw data and upload it to a computing device (described in more detail with reference to FIG. 5) that then uploads the data to the backend component 120. In another embodiment, each sensor captures the raw data and immediately sends the raw data to the computing device. In yet another embodiment, each sensor 102 may have sufficient processing power and memory to capture the raw data and generate the move data from the raw data (described in more detail below) and the system does not have the backend component 120 since the move data from the sensors may be provided directly to the social network 122 and/or the other systems 124, such as games or training devices. In some embodiments, the sensor(s) 102 in the ball 106 or implement 108 used for the physical activity may be embedded resulting in the ball 106 or implement 108 being ready to use in the system out of the package and reducing the overall cost of the system to the user.
Each sensor 102 that is associated with a ball 106 or an implement 108 may measure various data about a physical activity of the user as described below in more detail. Alternatively, each sensor in an implement, such as a basketball rim or soccer goal, may measure the presence of the ball 106 to indicate a field goal of the user in basketball or a goal in a soccer game. Each sensor 102 on the user 104 may identify the particular user and may determine when the user is near the ball 106 or implement 108.
In the illustrated embodiment, the raw data from each sensor may be communicated over the link 110 to the backend component 120. The link 110 may be a wired or wireless link. For example, the link may be Ethernet, the Internet, a wireless data network, a wireless cellular data network, a WiFi network and the like. The backend component 120 may be implemented using one or more computing resources, such as one or more server computers, one or more cloud computing resources and the like that move one or more processors, memory and other typical components in which at least one processor of the one or more computing resources may execute a plurality of lines of computer code (stored in the memory, for example) that implement the elements of the backend component. Alternatively, the backend component may be implemented in hardware with one or more programmed integrated circuits and the like that implement the elements of the backend component. In the illustrated embodiment, the backend component 120 also may have a move data processor 122 that receives the raw data from each of the sensors and generates the various different types of move data depending on the physical activity of the user. In other embodiments, there may be a plurality of move data processors 122 in which each move data processor 122 receives the raw data and then generates the move data for a particular physical activity, such as a soccer move data processor that generates soccer specific move data or a baseball move data processor that generates baseball specific move data. Thus, there may be a move data processor 122 for each different physical activity of the system. In embodiments in which the sensors 102 generate their own move data, the backend component may not be present or may be used as a gateway to communicate the move data.
In the illustrated embodiment, the backend component 120 may also have a store 124 that stores the data of the system including the user data, move data of each user and the like. The store 124 may also store the plurality of lines of computer code that generate the move data. The store 124 may be a database that may be implemented in hardware or software. In the illustrated embodiment, the backend component 120 may communicate, over one or more interfaces 121 a, 121 b, such as an application programming interface (API), the move data to one or more other systems, such as a social network system 126 or other systems 128 that may use the move data for various purposes. For example, the social network system may use the move data to generate statistics for each user based on the move data (or the backend component may generate the statistics.) For example, the other systems 128 may be a virtual game that uses the move data of the user to play a virtual game or a training system that generates training data based on the move data. For illustration purposes only, the system may be used to capture basketball move data and then used to play a virtual basketball game as shown in FIG. 6 in which move data from one or more users may be used to play the virtual game.
Thus, using the system and method, raw physical activity data is gathered from one or more sensors 102, processed into move data (at various different locations) and sent to an interface where the user may share the move data, such as with her social network, to enhance an avatar's performance in a video game or virtual game or use the move data to improve real-life sports performance.
FIG. 2 is a diagram illustrating an example of a ball 106 that may be used with the system. The ball may have a bladder 106 a and a cover 106 b having a thickness as is typical with a ball that is inflated to use in a physical activity, such as soccer or basketball. In one example, the sensor 102 in the ball may have a set of components 106 c may be embedded into the cover portion as shown. The set of components may also be attached to the outside of the ball or within the bladder of the ball. For a ball that is not inflated during use, such as a baseball, softball, golf ball and the like, the components 106 c may be embedded into the material of the ball or attached to the outside of the ball. When the sensor 102 is attached to the user, ball or implement, the sensor may be attached using a fastener, Velcro and the like.
The set of components 106 c may include one or more sensors, such as an accelerometer, thermometer, light sensor, pressure sensor, rotation angle sensor, speed sensor, a magnetic field direction sensor or a magnetic field strength sensor, one or more processors that control the overall operation of the set of components, a memory to store data used by the processor and store the sensor data, a power source, such as a battery, to power the set of components and a communications module to wirelessly communicate the raw data from the sensors. The set of components 106 c may also measure ambient light, humidity or any other property. The set of components also may include additional electronics such as a system for determining position, such as a GPS sensor, and a charger so that the battery may be recharged wherein the charger may be a wireless charger. The set of components also may have one or more LED lights and/or one or more speakers that provide feedback to the user. For example, the lights may flash or the speaker may emit a sound when a ball is placed into a bag that has been tagged with an ID chip, signaling that game play has ended.
FIG. 3 is a diagram illustrating an example of user worn sensor 102 that may be used with the system. In FIG. 3, only a portion of the sensor 102 is shown and the sensor may have an RFID chip 102 a. In the example in FIG. 3, the RFID chip 102 a may be in a band 300 worn by the user 104. In one example, the sensor 102 may identify the user when the user is involved in a physical activity and then be used to determine when the user is interacting with the ball 106, such as dribbling and/or shooting the basketball, dribbling and/or shooting the soccer ball or holding the bat or badminton racket. Using the raw data from the sensor connected to the user and then ball or implement sensor, the system may determine when the user is interacting with the ball or implement and thus generate the move data for the user as described below in more detail.
FIG. 4 shows a circuit block diagram of a sensor system 102 that may be used by the system. The sensor may have one or more sensor elements 400, a processor 402, RAM 404 and memory 406 that are connected to each other. The one or more sensor elements 400 may directly transmit the raw data through a universal serial bus (USB), WiFi, Bluetooth, near field communication (NFC), 3G, or 4G data networks to the processor 402. The sensor elements 400 may also transmit the raw data to the memory 404, such as a random access memory (RAM), which then transmits data to the processor 402.
FIG. 5 is a diagram illustrating an example of the system used for a basketball game in which a basketball 106 has one or more sensors that collect raw data for the basketball game. In the basketball game example in FIG. 5, each user who participates in the basketball game have at least one sensor connected to them. In the example of the system in FIG. 5, the raw data from the sensors 102 (including the sensor attached to the user) are sent across the link 110 over the interfaces 121 a, 121 b to the social network system 126 or a virtual game 128. For example, the link may be one or more computing devices 110 a and a computer network 110 b. Each of the one or more computing devices may be a smartphone device, a tablet computer, a wireless access point, a computer or a personal computer as shown. In this example, the raw data of the sensors may be processed into the move data at each sensor or by one of the computing devices 110 a. In this example, the move data may be used to allow one or more users to play a virtual game of basketball as shown.
FIG. 6 is a flowchart of an example of the system used to generate move data from raw data gathered from sensors 102 and then delivered to other systems over the interfaces 121 a, 121 b. As described above, raw data may be generated from the sensor-containing accessories 102, such as a ball and from the sensors attached to the player. The raw data of the sensors may be transmitted in real time to a computing device 110 a, such as a smartphone, tablet, PC, WiFi hotspot, or cradle. The computing device 110 a may then upload the raw data to the backend system 120, such as a server, housing one or more move data processors and the interfaces. The one or more move processors or the interfaces of the backend 120 may have modules containing algorithms and instructions for processing the raw data into move data according to a particular physical activity for which the system is being used. As shown in FIG. 6, the interface 121 a may be used to send the move data for the physical activity to a social network system 126 and specifically with one or more friends of the social network system. The interface 121 b may be used to send the move data for the physical activity to the game 128 so that the move data of one or more users may be used to affect the game play. For example, as shown in FIGS. 7 and 8, the system may be used to gather raw data from one or more users in different basketball games (Andy, Nel, Tom, Peter, Jack and Jerry on a first basketball court and Kebi, Wade, Jordan, Rose, James and Pippen on a second basketball court) and then use that data (once converted into move data for each user) for a virtual game. The raw data for each basketball court may be gathered contemporaneously at the same location or the raw data may be gathered at different times or at different locations. Furthermore, in the example in FIGS. 7 and 8, the physical activity in FIG. 7 may be a basketball game being played by a set of amateur players while the physical activity in FIG. 7 may be a basketball game being played by professional basketball players so that the team of amateur players may virtually play against the team of professional basketball players.
For the basketball game example shown in FIGS. 5 and 7-8, the implement 108 may be a basketball hoop as shown in FIG. 12 through which a ball 106 may pass when a user of the system makes a field goal. The basketball hoop may thus have one or more sensors 102, such as ID A, ID B and IDC, that determine when the ball 106 pass near and/or through the basketball hoop. For example, the sensors may generate raw data that may be used to determine if a field goal is made by a user or if a user gets a rebound of the basketball.
FIG. 9 is a chart illustrating an example of raw data gathered from one or more sensors for a basketball game and FIG. 10 is a chart illustrating an example of basketball move data generated from an application programming interface. For the basketball game, each user may have a user sensor 102 that has an RFID chip which will enable the system to identify each user. In combination with the other sensors 102, the system can determine the movement of the basketball between specific players, such as when the ball is passed to a teammate or to a member of the opposing team (known as a steal.) For example, the move data processor may compare a time when the user sensor indicates that the ball sensor in near the user to determine that the user has the ball. When the user dribbles the ball, passes the ball or shoots the ball, the raw data of the user sensor indicates that the ball is no longer near the user (for a pass or a shot) or returns to the user when the user is dribbling the ball. The sensors may be attached to the hoop, backboard, and basket, to enable the system to triangulate and determine when a user shoots the ball and whether it goes in or bounces off the backboard or rim, depending on the position of the ball relative to the sensors on the backboard and hoop.
Using raw data from the various sensors, the system may also determine other basketball move data, such as dribbling, shooting, holding, rebound, steal, blocked shot, and three-pointers. The system may also determine other move data useful to a player, such as the geographic location of the game, temperature of the surroundings, pressure within the ball, rotation speed and angle of the ball, and magnetic field direction and strength. For example, the move data processor may receive the raw data from the sensors of the user, the ball sensor raw data and the hoop sensor raw data and then generate a dribbling movement of the user based on the raw data for whatever period of time that the user is dribbling the ball, Similarly, the move data may process the raw data to generate move data for shooting, holding, rebound, steal, blocked shot, and three-pointers of the user. FIG. 10 shows an example of the types of basketball move data determined by the system. As shown in FIG. 10, each piece of move data (labeled #1 to #14) may include a date field, a time field, a start time field, stop time field (such as a start and stop time of the user dribbling the ball), a field indicating that the move data is dribbling move data, a field indication that the move data is pass move data, a field indicating that the particular move data is shooting move data, etc. The backend system may then send the data to a social media platform, such as Twitter, Google+, and Facebook, where the user may upload the move data to the platform to share. The backend system may also send the data to a video game, where the user may use the move data to affect video game play by enhancing the performance of an avatar.
FIGS. 11A and 11B illustrate a method 1100 for generating basketball move data for a user that may be carried out by the move data processor of the backend component, within the sensor or within the computing device 110 a. The described method may be performed for each user of the system that uses the sensors for a particular period of time. In the method, the raw sensor data of the user is received 1102 and the method determines if the ball (based on the sensor raw data of the ball) is near the user (based on the proximity of the user sensor to the ball sensor) 1104. If the ball is not near the user during the particular period of time (indicating that the user is not interacting with the ball), then the method is completed and no move data is generated for that user for that particular period of time. However, the method may be repeated for each user during a number of different periods of time since the user may interact with the ball during a different period of time and move data for the user should be generated.
If the user is near the ball initially during the time period, the method then determines if the user is not near the ball following a subsequent time period (based on the proximity of the ball sensor and the user sensor) 1106. If the ball is still near the user, then the method may generate ball holding move data for the user 1108 and store that move data. If the user is no longer near the ball during the subsequent time period, the method then determines if the ball is near the user during a next subsequent time period 1110. If the user is not near the ball during the next subsequent time period, the method may generate stealing move data for the user (if the ball sensor indicates that the ball is near a user on a different team), shooting move data for the user (if the ball sensor and the hoop sensors indicate that the ball has gone through the hoop) and/or passing move data for the user (if the ball sensor indicates that the ball is near a user on the same team). For the shooting move data, the user sensor raw data may contain raw data about the positioning of the user on the court (and the position of the three point line on the court) so that the system can determine if the user scored a field goal or a three point shot. If the user is again near the ball during the next subsequent time period, the method may generate dribbling move data for the user until the ball sensor indicates that the user is no longer near the ball.
In addition to the basketball move data shown in FIG. 10 and describe above, the system may also generate passing success rate move data (a percentage of successful passes versus a percentage of steals), a shooting success rate, rebound move data (indicating when a user is near the ball and the ball is near the hoop, but not passing through the hoop) and/or blocked shot move data. The system may also generate, for the various different physical activities, time of playing the game move data, geographical location of the game, temperature at the time of the game, pressure within the ball during the game, rotational speed of the ball during the game, rotational angle of the ball during the game, participants in the game (based on the sensors on the players and/or brightness of the playing environment. The system may also be used to generate additional types of move data for basketball or other physical activities not specifically described above since the above list is merely representative of the different type of move data.
The basketball move data may then be imported into a social network or video game. FIG. 13 depicts an example of the statics generated based on the move data from a video game in which the user may alter video game play by enhancing his/her video game avatar with move data from a real-life basketball game previously played. The user may enhance the video game avatar's performance to play with the avatars of professional basketball players. Similarly, volleyball move data that is similar to the basketball move data may be generated.
In a second implementation, the system may be used to generate move data for a racket sport, such as tennis or badminton. In this implementation, the racket (an example of the implement 108) may have tension sensors built into the strings to generate raw data, which may be processed with the tennis or badminton module on the API to determine the point at which the racket made contact with the ball. The racket may also have tri-axial acceleration sensors and a gyroscope to generate raw data, which may be processed to determine motion of the racket. Additionally, the racket may have a GPS to determine location information of the players, as well as a processor, memory devices, battery, and charger. For the racket sport, the system may generate level of strength/intensity/force of each racket swing, racket speed when hitting the ball, angle of hitting, ball point of contact with the racket, grip strength, time of game, and/or the player's body temperature during the game.
In a third implementation, the system may determine baseball move data. The baseball bat may have sensors built into it to generate raw data. The raw data may be processed by the baseball module on the API to determine baseball move data such as level of strength/intensity/force of each swing, speed of the bat at contact time, angle of swing, relative location between the ball and the bat, grip strength, and a player's body temperature.
In a fourth implementation, the system may determine soccer move data. The user may attach additional identifying sensors to the player and the goal to generate additional raw data. The raw data may be processed by the soccer module on the API to determine soccer move data such as a pass, success rate of passes, shooting, force of ball contact, goals, steals, rotation speed and/or angle of the ball, and player information.
In a fifth implementation, the system may determine the system may determine football or ice hockey move data, such as a pass, run, snap, interception, fumble, touchdown, field goal, force of ball contact, rotation speed and/or angle of the ball, and player information. In a sixth implementation, the system may determine golf move data, such as force/power of each swing, speed of hit, angle of the club, relative location of the club, point of contact between ball and club, grip strength, and/or a user's body temperature during the game.
The system and method brings a traditionally offline activity, such as a playing a basketball game or doing a physical activity, into the digital age, allowing users to connect with other social network users, enhance their video game avatars, or improve upon their real-life game play using move data of past performances.
While the foregoing has been with reference to a particular embodiment of the invention, it will be appreciated by those skilled in the art that changes in this embodiment may be made without departing from the principles and spirit of the disclosure, the scope of which is defined by the appended claims.

Claims

1. A method for generating move data from a physical activity, comprising:

providing a sensor for each user in the physical activity, at least one sensor for a ball or implement used during the physical activity;

gathering raw data from each sensor being used in the physical activity;

processing the raw data from each sensor into one or more pieces of move data for the physical activity;

outputting the one or more pieces of move data using an interface; and

using the one or more pieces of move data by one of a social network and a virtual game.

2. The method of claim 1, wherein outputting the one or more pieces of move data using an interface further comprises outputting the one or more pieces of move data using an application programming interface.

3. The method of claim 1, wherein the physical activity is a sport physical activity.

4. The method of claim 1, wherein processing the raw data from each sensor into one or more pieces of move data for the physical activity further comprises processing the raw data into one or more pieces of move data for one of a basketball physical activity, a soccer physical activity, a tennis physical activity, a badminton physical activity, a football physical activity, an ice hockey physical activity, a cricket physical activity, a squash physical activity and a baseball physical activity.

5. The method of claim 1, wherein gathering the raw user sensor data further comprising gathering an identifier of the user and a location of the user.

6. The method of claim 5, wherein gathering the raw data for the ball sensor further comprises generating one or more of acceleration data of the ball, temperature of the ball, pressure data of the data, rotational angle of the ball and speed of the ball, a magnetic field of the ball, ambient light striking the ball and humidity around the ball.

7. The method of claim 1, wherein gathering the raw data further comprises generating raw data from one or more sensors connected to a goal of the physical activity.

8. The method of claim 7, wherein generating raw data from one or more sensors connected to a goal of the physical activity further comprises generates raw data about a proximity of the ball to the goal of the physical activity.

9. An apparatus for generating move data from game play with sensor-containing accessories, comprising:

a computer implemented move data processor that receives raw data from at least one sensor associated with a ball being used for a physical activity and at least one sensor from a user participating in the physical activity and generates one or more pieces of move data for the physical activity based on the raw data from the sensors;

an interface, coupled to the computer implemented move data processor, that outputs the pieces of move data for the physical activity; and

one of a social network and a virtual game that uses the one or more pieces of move data.

10. The apparatus of claim 9, wherein the interface is an application programming interface.

11. The apparatus of claim 9, wherein the physical activity is a sport physical activity.

12. The apparatus of claim 9, wherein a basketball physical activity, a soccer physical activity, a tennis physical activity, a badminton physical activity, a football physical activity, an ice hockey physical activity, a cricket physical activity, a squash physical activity and a baseball physical activity.

13. The apparatus of claim 9 further comprising one or more user sensors wherein each user sensor gathers an identifier of the user and a location of the user.

14. The apparatus of claim 9 further comprising one or more ball sensors that gather raw data about acceleration data of the ball, temperature of the ball, pressure data of the data, rotational angle of the ball and speed of the ball, a magnetic field of the ball, ambient light striking the ball and humidity around the ball.

15. The apparatus of claim 9 further comprising a goal sensor.

16. The apparatus of claim 15, wherein the goal sensor is one of a basketball hoop sensor and a soccer goal sensor.

17. The apparatus of claim 9 further comprising a computer implemented backend component that can be coupled to the sensors wherein the computer implemented backend component includes the move data processor.

18. The apparatus of claim 12, wherein the move data processor further comprises a move data processor for each type of physical activity.

19. The apparatus of claim 17, wherein the move data processor further comprises a move data processor for each type of physical activity.