WO2017004998A1

WO2017004998A1 - System for directing action of self-propelled physical object and method thereof

Info

Publication number: WO2017004998A1
Application number: PCT/CN2016/075537
Authority: WO
Inventors: Zheng Shi; Yeliao TAO
Original assignee: Zheng Shi
Priority date: 2015-07-03
Filing date: 2016-03-03
Publication date: 2017-01-12
Also published as: CN105094423A; CN105094423B

Abstract

An interactive system and method for directing the action of a self-propelled physical object (105) are disclosed. The interactive system includes an interactive surface (101), embedded with a sensor array with at least one sensor; an input actuator (104) that changes the output value of a sensor once the input actuator (104) is placed on the interactive surface (101) and within the detection range of the sensor; a self-propelled physical object (105), configured to move on the interactive surface (101); a control circuit (102), configured to detect changes in the output value of a sensor within the sensor array; and a processor (103), operatively linked to the control circuit (102). Once the input actuator (104) is placed on the interactive surface (101), the processor (103) is configured to receive changes in output values from the sensor array, and derive action parameters of the input actuator (104) from the changes in output values, and once the self-propelled physical object (105) is placed on the interactive surface (101), the processor (103) is further configured to direct the action of the self-propelled physical object (105) based on an action instruction derived from the action parameters.

Description

[Title established by the ISA under Rule 37.2] SYSTEM FOR DIRECTING ACTION OF SELF-PROPELLED PHYSICAL OBJECT AND METHOD THEREOF

TECHNICAL FIELD

The present invention relates to human-computer interaction upon an interactive surface. More specifically, the present invention relates to collecting movement data from an input actuator and using this data to control a self-propelled physical object.

BACKGROUND

With the development of technology, touchpads are increasingly used for gaming and other similar electronic devices.

Currently, these touchpads (e.g. touch screen smart phone) can recognize finger gestures and track physical objects on their surfaces but are currently unable to control physical object movement themselves.

The American patent US 8737035 B2 describes a method that controls movable objects upon an electronic equipment surface through magnetic fields. But the drawbacks of such a method lies in that only a single object can move upon the surface and the direction of the object cannot be determined.

Patent CN102039045A discloses an interactive surface with a radio frequency (RF) antenna array which is configured to read and recognize the RFIG tags of multiple objects placed upon the interactive surface. Patent US 20110272884 A1 discloses a system whereby an object can move on the surface using a mechanical transmission device. The combination of both patents describes a system whereby multiple objects upon an interactive surface can be made to physical move according to instructions from a computer. However, such system cannot accurately recognize the direction of object relative to the interactive surface, and cannot guide objects to move along a specific path.

SUMMARY OF INVENTION

The present invention provides an electronic interactive surface that can detect and record the movement of an input actuator and use this information to generate action instructions for a self-propelled physical object located on the interactive surface to perform.

One aspect of the embodiment of the present invention provides an interactive system for directing the action of a self-propelled physical object, comprising:

-an interactive surface embedded with a sensor array with at least one sensor；

-an input actuator that changes the output value of a sensor upon the input actuator being placed on the interactive surface and within the detection range of the sensor；

-a self-propelled physical object that is configured to move on the interactive surface；

-a control circuit that is configured to detect changes in the output value of a sensor within the sensor array；

-a processor that is operatively linked to the control circuit.

Once the input actuator is placed on the interactive surface, the processor is configured to receive changes in output values from the sensor array, and derive action parameters of the input actuator from the changes in output values； and once the self-propelled physical object is placed on the interactive surface, the processor is further configured to direct the action of the self-propelled physical object based on an action instruction derived from the action parameters.

Optionally, the sensor array comprises capacitive sensors.

Optionally, the input actuator is one or more fingers.

Optionally, the self-propelled physical object comprises an RFID chip, and the processor is configured to recognize the ID of RFID chip.

Optionally, the processor is configured to direct the action of the self-propelled object on the interactive surface through wireless communication.

Optionally, the action parameters comprise location, direction, speed, and acceleration of the input actuator.

Optionally, the interactive system further includes a mode switch. The processor records the action parameters of an input actuator when the mode switch is set to recording mode, and the processor directs the action of the self-propelled physical object when the mode switch is set to directing mode.

Optionally, the processor is configured to direct the actions of a plurality of self-propelled physical objects based on one action instruction.

Optionally, the processor is configured to direct the action of the self-propelled physical object based on a sequential set of action instructions.

Another aspect of the embodiment of the present invention provides an interactive method for directing the action of a self-propelled physical object, comprising:

-placing an input actuator on an interactive surface embedded with a sensor array,

-detecting changes in output values from the sensor array by a control circuit embedded in the interactive surface,

-receiving changes in output values from the sensor array by a processor,

-deriving action parameters of the input actuator from the changes in output values by the processor,

-placing a self-propelled physical object on the interactive surface,

-generating an action instruction for the self-propelled physical object from the action parameters by the processor,

-directing the action of a self-propelled physical object based on the action instruction.

Optionally, the interactive method further comprises recognizing, by the processor, the ID of the RFID chip embedded in the self-propelled physical object.

Optionally, the interactive method further comprises directing, by the processor, the action of the self-propelled object on the interactive surface through wireless communication.

Optionally, the interactive method further comprises recording, by the processor, the action parameter of an input actuator when a mode switch is set to recording mode, and directing, by the processor, the action of the self-propelled physical object when the mode switch is set to directing mode.

Optionally, the interactive method further comprises directing, by the processor, the actions of a plurality of self-propelled physical objects based on one action instruction.

Optionally, the interactive method further comprises directing, by the processor, the actions of a self-propelled physical object based on a sequential set of action instructions.

BRIEF DESCRIPTION OF THE DRAWINGS

To better illustrate the technical features of the embodiments of the present invention, various embodiments of the present invention will be briefly described in conjunction with the accompanying drawings. It should be obvious that the drawings are only for exemplary embodiments of the present invention, and that a person of ordinary skill in the art may derive additional drawings without deviating from the principles of the present invention.

FIG. 1 is an exemplary schematic diagram illustrating the interactive system in accordance with one embodiment of the present invention.

FIG. 2 is an exemplary structure diagram of the self-propelled physical object in accordance with one embodiment of the present invention.

FIG. 3 is an exemplary schematic diagram illustrating the interactive system process flow in accordance with one embodiment of the present invention.

FIG. 4 is an exemplary schematic diagram illustrating the interactive system in accordance with one embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to various embodiments of the invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with the embodiments, it will be understood that this is not intended to limit the scope of the invention to these specific embodiments. The invention is intended to cover all alternatives, modifications and equivalents within the spirit and scope of invention, which is defined by the apprehended claims.

Furthermore, in the detailed description of the present invention, specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be obvious to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuits are not described in details to avoid unnecessarily obscuring a clear understanding of the present invention.

In addition, while a sensor system making use of capacitive sensing between a capacitance sensor embedded on the interactive board and the objects is disclosed in connection with embodiments of the present invention, other sensor technologies such as Hall effect sensors, pressure sensors, resistance touch sensors or piezo touch sensors can also be adapted and are within the scope of the present invention.

In addition, while a touch by a finger is repeatedly used to represent an event that alters the capacitive property of the sensors, the use of other means, such as a stylus, an electric conductive material, or a material with high dielectric constant, to alter the capacitive property of the sensors can also be used and are within the scope of the present invention.

The present invention may be better understood and its numerous objects and advantages will become apparent to those skilled in the art by reference to the accompanying drawings.

One aspect of the embodiment of the present invention provides an interactive system for directing the action of a self-propelled physical object, comprising an interactive surface embedded with a sensor array with at least one sensor； an input actuator that changes the output value of a sensor once the input actuator is placed on the interactive surface and within the detection range of the sensor； a self-propelled physical object that is configured to move on the interactive surface； a control circuit that is configured to detect changes in the output value of a sensor within the sensor array； and a processor that is operatively linked to the control circuit.

Within the system described above, once the input actuator is placed on the interactive surface, the processor is configured to receive changes in output values from the sensor array, and derive action parameters of the input actuator from the changes in output values； and once the self-propelled physical object is placed on the interactive surface, the processor is further configured to direct the action of the self-propelled physical object based on an action instruction derived from the action parameters.

FIG. 1 is an exemplary schematic diagram illustrating the interactive system in accordance with one embodiment of the present invention. As shown in FIG. 1, the interactive system includes an interactive surface 101, a control circuit 102, a processor 103, a user’s finger touch (i.e., an input actuator) 104, a self-propelled physical object 105, a mode switch 106, a wireless transmission module 107, and an on/off switch 108. The interactive system is configured to recognize in real-time the position of the input actuator 104 and the self-propelled physical object 105 placed on the interactive surface 101.

In this embodiment, the interactive surface 101 includes a sensor array that comprises at least one sensor. The interactive surface 101 is set to comprise multiple capacitance sensors that are designed to capacitively couple with the input actuator 104 and the self-propelled physical object 105. The processor 103 being able to detect the change in capacitance values (i.e., output value) of the capacitance sensors within the detection range of the input actuator 104 or self-propelled physical object 105 whenever the input actuator 104 and the self-propelled physical object 105 are placed upon the sensor array. Within the confines of the embodiment of the present invention, all the capacitance sensors are set with the same initial baseline capacitance values but with each sensor assigned different coordinates. The control circuit 102 in the control circuit array 102 continuously collects the output values of each individual capacitance sensor and transmits these to the processor 103 in real time.

According to one embodiment of the present invention, the system can be switched to a recording mode or a directing mode via the mode switch 106.

Whenever the mode switch 106 is switched to the recording mode, a user may place an input actuator 104 upon the interactive surface 101. The input actuator 104 can be either fingers or any physical object that can capacitively couple with the capacitance sensors (e.g., a stylus) . Whenever the input actuator 104 is placed upon the interactive surface 101, capacitive coupling between the input actuator 104 and the one or more sensors of the sensor array located under the input actuator 104 occurs. The processor 103 is configured to detect any changes in output values experienced by any of the sensors of the sensor array. Thus, the position of the input actuator 104 relative to the interactive surface 101 is determined in real time. As the input actuator 104 is moved along the interactive surface 101, the processor 103 can further detect and record the changes in output values experienced by the various sensors that are located under the input actuator’s 104 path on the interactive surface 101. Using the sensors coordinates and time as parameters, the processor 103 can deduce the input actuator’s 104 direction, speed and acceleration (i.e., the input actuator’s action parameters) .

Using the input actuator’s 104 position and movement data (i.e., action parameters) , the processor 103 is further configured to generate a predefined action instruction for one or more self-propelled physical objects 105 to perform.

Specifically, the change in output value experienced by a sensor of the sensor array is detected by the processor 103 via the control circuit 102 at time t, the processor 103 is then configured to assign the coordinates of this capacitance sensor as the current position of the input actuator 104

As the input actuator 104 is moved along the interactive surface 101, the processor 103 collects more position vectors in time so that:

wherein

is the position of the input actuator 104 at the start of the movement (time t0) at the starting coordinates

and

is the position of the input actuator 104 at the end of the movement (time t_m) at the ending coordinates

Thus, using the abovementioned space/time data variable, the processor 103 can determine the input actuator’s 104 direction, speed, and acceleration (i.e., action parameters) . For example, for each time parameter ti, speed and acceleration can be determined using the following algorithms:

speed

acceleration

As the input actuator 104 is moved along the interactive surface 101, the processor 103 collects and stores the input actuator’s 104 movement data. It will be noted that the array of movement input data comprises all the time t_i, position

speed

and acceleration

wherein position

speed

and acceleration

are all vectors that comprise movement information.

As the mode switch is switched to the directing mode, a user may place a self-propelled physical object 105 on the interactive surface 101. Through a predefined set of algorithms that generate action instructions using the input actuator’s 104 movement data as inputs, the processor 103 is configured to generate action instructions to be performed by the self-propelled physical object 105.

FIG. 2 is an exemplary structure diagram of the self-propelled physical object 105. As shown in FIG. 2, the self-propelled physical object 105 comprises an identification device 201, a second wireless transmission module 202, a self-propelled module 203, and a control module 204.

Once the processor 103 has generated an action instruction, the processor 103 then proceeds to transmit the action instruction to the self-propelled physical object 105 via wireless communication. In the embodiment illustrated in FIG. 2, the self-propelled physical object 105 is embedded with an RFID (i.e., the identification device 201) which the processor 103 uses to identify the self-propelled physical object 105 via wireless communication. Once the identification is achieved the processor 103 proceeds to transmit wirelessly the action instruction to the self-propelled physical object’s 105 second wireless communication module 202. The control module 204 then processes and instructs the self-propelled module 203 what actions (e.g., direction, speed and time) need to be performed.

Based on type of programming, the processor 103 can assign multiple sets of action instructions to one self-propelled physical object 105 to be performed in a sequence. The processor 103 can also assign the same set of action instructions to multiple self-propelled physical objects 105 located on the interactive surface 101 (the use of a time delay may be used in such an instance) . The processor 103 can further set up action time delay for different self-propelled physical objects.

FIG. 3 is an exemplary schematic diagram illustrating the interactive system process flow in accordance with one embodiment of the present invention. As shown in FIG. 3, the specific steps of this method are as follows:

S301: a user selects either the recording mode or the directing mode.

S302: the user selects the recording mode.

S303: the processor initializes the recording mode in which the processor 103 detects and records in real-time any changes in the output values experienced by the sensors whenever a user places an input actuator 104 (e.g., fingers or any physical object that can capacitively couple with the capacitance sensors) upon the interactive surface 101. The user causes the input actuator 104 to move along the interactive surface 101 and the processor 103 determines the position and movement trajectory (including speed and acceleration) of the input actuator 104 upon the interactive surface 101 and stores the information as an array of action data (i.e., action parameters) .

S304: the user may decide to continue or cease the recording mode once the recording of the input actuator’s movement on the interactive surface is finished. In case the user decides to continue recording, the processor will re-execute the step described in S303. In such a case, the user may use the same input actuator 104 to record a different movement or they may change input actuator 104 (e.g., stylus) to record. In case the user decides to end the recording mode S302, they may move onto step S309.

S305: the user selects the directing mode.

S306: the user places a self-propelled physical object 105 on the interactive surface 101 in order to assign the previously recorded movement actions to the self-propelled physical object 105.

S307: the processor 103 generates an action instruction from the recording action data (S304) based on an electronic program.

S308: The processor 103 recognizes the UID of the self-propelled physical object 105 via wireless communication with its second wireless transmission module 202 (e.g., RF module) and subsequently transmits an action instruction that the control module 204 processes. The control module 204 will thereafter direct the self-propelled module 203 to perform this movement instruction. Upon the end of the self-propelled physical object’s 105 movement action, the user may continue the action by placing the same or another self-propelled physical object 105 back to its original starting point. In case the user decides to end the directing mode, they may move onto step S309.

S309: the user may continue or end the program. If the use decides to continue the program, they may go back to step S301 selecting either the recording mode or the directing mode. In case the user decides to end the program, they may proceed to step S310.

S310: the program ends.

FIG. 4 is an exemplary schematic diagram illustrating the interactive system in accordance with one embodiment of the present invention. As shown in FIG. 4, the interactive surface 401 comprises a top surface 402 and a substrate 403. It will be pointed out that, although the top surface 402 and the substrate 403 are separated in FIG. 4, this is only done for illustration sake and in reality these two surfaces would be affixed to one another.

The input actuator 405, which is a stylus in this particular embodiment, and the self-propelled physical object 406 are configured to interact with the top surface 402 of the interactive surface 401.

The substrate 403 comprises a plurality of sensors 407 which constitute the sensor array and a peripheral unit 404. The peripheral unit 404 comprises the processor 408 that is connected to the sensors 407 via a control circuit as well as an RF antenna 409 that is connected to the processor 408. The processor 408 wirelessly communicates with the self-propelled physical object’s 406 identification device 201 (in order to recognize the UID) and control module 204 (in order to direct it movement instructions) via the second wireless transmission module 202.

The method of the interactive system for the embodiment illustrated in FIG. 4 may be described as follows. Auser places a self-propelled physical object 406 upon the top surface 402 of the interactive surface 401. The output value of the sensor S_T0*changes and this change in output is detected by the processor 408. Upon detecting a change in a sensor’s 407 value, the processor 408 is configured to scan for the self-propelled physical object 406 through a wireless communication between the RF antenna 409 and the identification device 201 (via the second wireless transmission module 202) . In such a manner, the processor 408 detects the presence of the self-propelling physical object 406 and recognizes and records its location (i.e., the coordinates of the sensor S_T0*) as well as its UID.

The user then proceeds to use the input actuator 405 (i.e., the stylus embedded with an RFID tag 410 in the embodiment illustrated in FIG. 4) to touch the top surface 402 of the interactive surface 401. The output value of the sensor S_T0 changes and this change in output is detected by the processor 408. Upon detecting a change in a sensor’s 407 value, the processor 408 scans, recognizes and records the input actuator’s 405 UID through a wireless communication between the RF antenna 409 and the RFID tag 410. In such a manner, the processor 408 detects the presence of the input actuator 405 and recognizes and records its location (i.e., the coordinates of the sensor S_T0) as well as its UID.

The user then proceeds to move the input actuator 405 from S_T0 at time T₀ to S_T1 at time T₁. The processor 408 detects and records the sensors 407 that have experienced a change in output value and determines the movement path of the input actuator 405. It will be noted that parameters such as orientation, speed and acceleration are all separate variables related to the movement path that the processor 408 can be made to record.

The recorded input movement path (i.e., action parameters) of the input actuator 405 is subsequently used by the processor 408 in order to generate a specific movement instruction to be directed at the self-propelled physical object 406. It will be noted that how this data is used to generate a movement instruction is based on software programming and there exists a large array of potential options as to what kind of movement instructions are generated from the specific movement path (i.e., action parameters) of the input actuator 405.

For illustration sake, the present embodiment only describes a configuration by which the self-propelled physical object 406 mimics the same movement path as the input actuator 405 from a different departure point (i.e., S_T0*) . But it is understood that the scope of the present invention is not limited to these simple movement instructions.

As explained, the processor 408 is configured to use the input actuator’s 405 movement path (i.e., action parameters) to generate a specific movement instruction to be directed at the self-propelled physical object 406. In the embodiment illustrated in FIG. 4, this is a simple mimicking of the input actuator’s 405 movement path by which the processor 408 directs the self-propelled physical object 406 to move from S_T0*at time T_0*to S_T1 at time T_1*. Specifically, the manner in which the processor 408 directs the movement instructions is done wirelessly via its RF antenna 409 and the second wireless transmission module 202. The self-propelled physical object’s 406 control module 204 is then configured to process this data and instruct the movement module 203 to perform the required movement.

The process flow in accordance with the embodiment illustrated in FIG. 4 is as follows:

-placing an input actuator on an interactive surface embedded with a sensor array；

-detecting changes in output values from the sensor array by a control circuit embedded in the interactive surface；

-receiving changes in output values from the sensor array by a processor；

-generating from the output value an action instruction for the self-propelled object by the processor；

-placing a self-propelled physical object on the interactive surface, prior or after the placing of the input actuator on the interactive surface；

-directing the action instruction to the self-propelled physical object by the processor；

-performing the action instruction by the self-propelled physical object.

In the several embodiments of the present invention, it should be understood that the apparatus and method disclosed may be realized by other means. For example, the embodiment of the apparatus described is only schematic. For example, the division of the unit is only a division of logical function； in practice, it can be divided in other ways. For example multiple units or components can be combined or integrated to another system, or some features can be ignored, or not executed. At another point, the mutual coupling or direct coupling or communication connection displayed or discussed can be through indirect coupling or communication connection of a number of interfaces, devices or units, which can be electrical, mechanical or otherwise.

The unit described as a separate component can be either physically separated or not, the component as display unit can be either physical unit or not, namely it can be placed in one place, or be distributed to multiple network units. By choosing part or the entire unit according to the actual need, the purpose of this embodiment can be realized.

In addition, each function unit if each embodiment of the present invention can be integrated to a processing unit or each unit physically exists alone, or two or more units are integrated to one unit. The integrated unit can be realized in the form of hardware or realized in the form of hardware plus software.

The integrated unit realized in the form of software unit, can be stored in a computer readable storage medium. The software unit is stored in a storage medium, comprising several instructions to make a computer device (e.g., a personal computer, server, or network equipment) or processor execute part of the steps in each embodiment of the present invention. The storage medium may comprise: a U disk, mobile hard disk, read only memory read only memory (ROM) , random access memory (RAM) , disk or CD-ROM and other various program code storage medium.

People skilled in the art will notice that the above embodiments are only used to convey the inventive concept and should not be construed as a restriction upon the scope of the invention in itself. In practical application, the functions of the present invention can be realized by different modules and processes according to the need (e.g., internal structure of the device can be divided into different functional modules) to realize the entire or part of the functions described in the detailed description.

Claims

An interactive system for directing the action of a self-propelled physical object, comprising:

- an interactive surface, embedded with a sensor array with at least one sensor；

- an input actuator that changes the output value of a sensor upon the input actuator being placed on the interactive surface and within the detection range of the sensor；

- a self-propelled physical object, configured to move on the interactive surface；

- a control circuit, configured to detect changes in the output value of a sensor within the sensor array；

- a processor, operatively linked to the control circuit；

wherein, upon the input actuator being placed on the interactive surface, the processor is configured to receive changes in output values from the sensor array, and derive action parameters of the input actuator from the changes in output values； and wherein, upon the self-propelled physical object being placed on the interactive surface, the processor is further configured to direct the action of the self-propelled physical object based on an action instruction derived from the action parameters.
The interactive system in claim 1, wherein the sensor array comprises capacitive sensors.
The interactive system in claim 1, wherein the input actuator is one or more fingers.
The interactive system in claim 1, wherein the self-propelled physical object comprises an RFID chip, and the processor is configured to recognize the ID of RFID chip.
The interactive system in claim 1, wherein the processor is configured to direct the action of the self-propelled object on the interactive surface through wireless communication.
The interactive system in claim 1, wherein the action parameters comprise location, direction, speed, and acceleration of the input actuator.
The interactive system in claim 1, further comprising a mode switch, wherein the processor records the action parameters of an input actuator when the mode switch is set to recording mode, and wherein the processor directs the action of the self-propelled physical object when the mode switch is set to directing mode.
The interactive system in claim 1, wherein the processor is configured to direct the actions of a plurality of self-propelled physical objects based on one action instruction.
The interactive system in claim 1, the processor is configured to direct the action of the self-propelled physical object based on a sequential set of action instructions.
An interactive method for directing the action of a self-propelled physical object, comprising:

- placing an input actuator on an interactive surface embedded with a sensor array,

- detecting changes in output values from the sensor array by a control circuit embedded in the interactive surface,

- receiving changes in output values from the sensor array by a processor,

- deriving action parameters of the input actuator from the changes in output values by the processor,

- placing a self-propelled physical object on the interactive surface,

- generating an action instruction for the self-propelled physical object from the action parameters by the processor,

- directing the action of the self-propelled physical object based on the action instruction.
The interactive method in claim 10, wherein the sensor array comprises capacitive sensors.
The interactive method in claim 10, wherein the input actuator is one or more fingers.
The interactive method in claim 10, further comprising recognizing, by the processor, the ID of the RFID chip embedded in the self-propelled physical object.
The interactive method in claim 10, further comprising, directing, by the processor, the action of the self-propelled object on the interactive surface through wireless communication.
The interactive method in claim 10, wherein the action parameters comprise location, direction, speed, and acceleration of the input actuator.
The interactive method in claim 10, further comprising, recording, by the processor, the action parameter of an input actuator when a mode switch is set to recording mode, and directing, by the processor, the action of the self-propelled physical object when the mode switch is set to directing mode.
The interactive method in claim 10, further comprising, directing, by the processor, the actions of a plurality of self-propelled physical objects based on one action instruction.
The interactive method in claim 10, further comprising, directing, by the processor, the actions of a self-propelled physical object based on a sequential set of action instructions.