CN107053214B - A robot battle device and control method based on somatosensory control - Google Patents

Abstract

The invention discloses a robot fight device based on somatosensory control and a control method, wherein the device comprises a fighter robot, a Kinect device for identifying actions of players and collecting action information, and a control device for receiving the action information collected by the Kinect device and controlling the fighter robot according to the action information; the Kinect device comprises a Kinect sensor, a camera for motion recognition, a depth sensor and a PC end, wherein the Kinect sensor, the camera and the depth sensor are all connected with the PC end, the control device comprises a WIFI module, and the Kinect device is communicated with the control device through the WIFI module; the control device is arranged in the war chariot robot. The device acquires the player action signals by using the computer-man-machine interaction module, and collects and extracts the actions of the original player, thereby realizing the control of the war chariot robot.

Description

A robot battle device and control method based on somatosensory control

technical field

The invention relates to the research field of automatic control of human-computer interaction, in particular to a robot battle device and control method based on somatosensory control.

Background technique

Most of the battle games currently on the market are mainly software games, or battle games with remote control of the handle. The operation mode of the software game is mainly through peripherals such as keyboard and mouse; while the remote control of the handle is through the remote control. In these types of battle games, players just tap the buttons to fight, without any real combat experience, and the long-term game is easy to tire the body and produce a sense of boredom.

Kinec t is a 3D somatosensory camera with functions such as real-time motion capture, image recognition, microphone input, and voice recognition. The Kinect device launched by Microsoft has excellent performance in human body tracking and attitude assessment. Human body movements and gestures to understand the operator's intentions, so as to use the computer to effectively operate the robot.

Using the somatosensory control system of the Kinect device to control the robot battle increases the diversity of game operations and the entertainment of the game. At the same time, this operation method is more in line with one of the future trends in robot development. It embodies the concept of human-computer interaction, allowing players to truly experience the thrill of "combat". The somatosensory-controlled chariot robot battle device allows two players to choose to control the robot part or the car part respectively, and two players cooperate to play a battle game.

Contents of the invention

The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, to provide a robot fighting device and control method based on somatosensory control, through which the control of the chariot robot can be realized through the Kinect device.

In order to achieve the above object, the present invention adopts the following technical solutions:

The present invention provides a robot combat device based on somatosensory control and a control method, including a chariot robot, and a Kinect device for recognizing player actions and collecting action information, and receiving the action information collected by the Kinect device, and according to the action information. The control device that the chariot robot controls; the Kinect device includes a Kinect sensor, a Kinect somatosensory camera for motion recognition, a depth sensor and a PC terminal, and the Kinect sensor, Kinect somatosensory camera and a depth sensor are all connected to the PC terminal , the control device includes a WIFI module, and the Kinect device communicates with the control device through the WIFI module; the control device is arranged in the chariot robot.

As a preferred technical solution, the control device includes a single-chip microcomputer, a power supply module, a communication module, a motor drive module and a function key module, and the power supply module, the communication module, the motor drive module and the function key module are all connected to the single-chip microcomputer.

As a preferred technical solution, the chariot robot includes a motor and a steering gear, and the single-chip microcomputer sets two I/O ports for controlling the motor. By changing the high and low levels of the two I/O ports and the square wave The duty cycle is used to control the forward rotation, reverse rotation and speed of a motor; the single chip microcomputer is also provided with an I/O port for controlling the steering gear, and in the working cycle of the steering gear, by changing the I/O port The high and low level duty cycle is used to control the rotation of the servo from 0° to 180°.

As a preferred technical solution, the power supply module uses a 7.4V rechargeable battery and a voltage stabilizing chip to supply power to the control device.

As a preferred technical solution, the chariot robot is provided with a plurality of micro switches and a plurality of LED lights, and the plurality of micro switches and a plurality of LED lights are all connected to the control device. Once, one LED light goes out, and when all the LED lights go out, the WIFI module disconnects the connection between the Kinect device and the control device.

The present invention also provides a control method of a robot combat device based on somatosensory control, comprising the following steps:

1) the Kinect device is connected with the host server computer, so that it can collect the player's body motion before the Kinect somatosensory, and use the Kinect device to collect the player's motion information in real time;

2) Turn on the power supply of the chariot robot, so that the client single-chip microcomputer on the chariot robot is connected with the host server computer PC;

3) The PC analyzes and processes the position information of each player's joints read by the Kinect somatosensory camera, obtains effective joint angles or posture intentions, analyzes motion information, and reads the control information required to control the chariot robot;

4) The PC establishes a connection with the client MCU through the wireless WIFI module, and sends control information to the MCU;

5) The single-chip microcomputer receives the control information collected by the PC in real time, and analyzes the control information through a specific algorithm, and then the single-chip microcomputer drives the motor drive circuit and the steering gear drive circuit according to the obtained information, and changes the duty cycle of the PWM wave input by the motor drive. The rotation speed of the motor can be changed by changing the duty cycle of the PWM wave input by the steering gear to change the rotation angle of the steering gear to realize the control of the chariot robot.

As a preferred technical solution, step 3) is specifically:

The Kinect device begins to recognize and capture the player's skeleton within 1-3 meters in front of the Kinect somatosensory camera, and collects the player's body movements. The Kinect device can map out the 3D scene map within the range of the Kinect somatosensory camera and give three-dimensional space coordinates; the Kinect device Obtain the effective joint angle of the player or recognize the gesture intention through spatial positioning, analyze the action information, and read the control information required to control the chariot robot; the information read includes: the angles of the six joints of the hands of the player controlling the robot, The rotation direction of the waist and the tilt angle of the upper body; the relative position of the hands of the player controlling the car, specifically the angle between the line between the palms of both hands and the horizontal line; the information collected by the former controls the movement of the robot, and the information collected by the latter Then control the forward, backward, turning and speed of the car.

As a preferred technical solution, step 4) is specifically:

When the single-chip microcomputer is initialized and the connection between the single-chip microcomputer and the PC is established through the WIFI module, the single-chip microcomputer is in the state of waiting to receive the control data of the PC;

When the data is sent to the single-chip microcomputer through the WIFI module, the single-chip microcomputer will analyze the received information and extract the control information of the required steering gear and motor, including the angle control information of 9 steering gears and the control information of 2 motors; Change the PWM duty cycle of the output port to change the rotation angle of each steering gear. Through different combinations of different angles of the steering gear, different actions of the chariot robot are presented, so as long as you input a specific angle for each steering gear, it will The robot can make expected actions; the single-chip microcomputer keeps updating the player's action information collected by the Kinect device. In the case of a small update delay, the effect of synchronizing the action of the chariot robot with the player can be achieved; at the same time, through the motor drive module, Control changes the speed and steering of the motor. When there is a speed difference between the two driving wheels, the car can turn.

9. The control method of the robot battle device based on somatosensory control according to claim 6, characterized in that 5 LED lights are installed on the chariot robot as the life bar of the chariot robot, when the device enters the normal working state, The micro switch is in the sensing state, and all 5 LED lights are on at the same time. When any one of the micro switches senses a touch, it will turn off one LED light. When all 5 LED lights are off, the client MCU will The connection to the server computer will be disconnected, data cannot be transmitted, and the player cannot operate the tank robot.

10. The control method of the robot combat device based on somatosensory control according to claim 6, wherein the chariot robot device uses a 7.4V rechargeable battery and a voltage stabilizing chip to supply power to each module and components.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The Kinect device of the present invention realizes the connection with the control device through the WIFI module. When the Kinect device analyzes the actions collected by the player and obtains the required action joint information, the signal is sent to the control device through the wireless communication module. Single-chip microcomputer: The single-chip microcomputer controls the rotation of the steering gear and the motor to control the action of the chariot robot according to the signal received from the communication module and after data processing.

2. The present invention uses the computer-human-computer interaction module to obtain player action signals, collects the original player actions and extracts information, thereby realizing the control of the chariot robot.

3. The somatosensory control of the Kinect of the present invention changes the single operation of the previous games, and makes the concept of human-computer interaction more thoroughly displayed, so that players can really put themselves into the "battle". Two players can choose to control the robot part or the car part respectively, and the two players cooperate to play the battle game. The player who controls the robot can make upper body movements to make the robot imitate; the player who controls the car part can control the forward, backward and turning of the car by simulating the driving posture, so as to realize the mobile function of the robot. The two players cooperate with each other and fight together.

Description of drawings

Fig. 1 is the circuit structure block diagram of the robot battle device based on somatosensory control of the present invention;

Fig. 2 (a) is the minimum system circuit schematic diagram of Arduino mega 2560 of the present invention;

Fig. 2 (b) is the schematic diagram of the motor drive module circuit of the present invention;

Fig. 2 (c) is the circuit principle diagram of 5V, 3V voltage stabilizing module of the present invention;

Fig. 2 (d) is the circuit schematic diagram of the chip pin corresponding to the WIFI module of the present invention and each circuit interface;

Fig. 3 is the working flow chart of the single-chip microcomputer of the robot fighting device based on somatosensory control in the present invention;

Fig. 4 is a flow chart of the control method of the robot fighting device of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

Example

As shown in Fig. 1, Fig. 2 (a) - Fig. 2 (d), a kind of robot battle device based on somatosensory control in this embodiment includes a chariot robot, a Kinect device for recognizing player actions and collecting action information, and a receiver The action information that Kinect device gathers and controls the chariot robot according to this action information; Described Kinect device comprises Kinect sensor, camera for motion recognition, depth sensor and PC end, described Kinect sensor, camera And the depth sensor is connected with the PC end, the control device includes a WIFI module, and the Kinect device communicates with the control device through the WIFI module; the control device is arranged in the chariot robot.

The Kinect sensor adopts Kinect Xbox one, and the depth sensor is embedded inside the Kinect sensor.

The WIFI module adopts the ESP8266 WIFI module to realize the wireless transmission of data between the upper computer (computer, PC) and the lower computer (single-chip microcomputer).

The control device includes a single-chip microcomputer, a power supply module, a communication module, a motor drive module and a function key module, and the power supply module, the communication module, the motor drive module and the function key module are all connected to the single-chip microcomputer.

The chariot robot includes a motor and a steering gear. The single-chip microcomputer sets two I/O ports for controlling the motor, and controls one by changing the high and low levels of the two I/O ports and the square wave duty cycle Forward rotation, reverse rotation and speed of the motor; the single-chip microcomputer is also provided with an I/O port for controlling the steering gear. In the working cycle of the steering gear, by changing the high and low level duty cycle of the I/O To control the rotation of the steering gear from 0° to 180°.

The single-chip microcomputer adopts the Arduino mega 2560 single-chip microcomputer, and uses the Arduino mega 2560 single-chip microcomputer as the main control chip to realize analyzing the action data collected and transmitted by the computer in real time.

The motor drive module adopts L298P, and drives the motor to perform corresponding actions through the L298P.

The power supply module uses a 7.4V rechargeable battery and a voltage stabilizing chip to supply power to the control device, which can provide a stable voltage to the control device and ensure that the robot battle device can continue to work.

The chariot robot is provided with a plurality of micro switches and a plurality of LED lights, and the plurality of micro switches and a plurality of LED lights are connected to the control device. When the micro switch is hit once, the LED lights go out One, when all the LED lights are off, the WIFI module disconnects the Kinect device from the control device.

The working voltage of the fighting robot is 7.4VDC～8.6VDC; the working voltage of the Kinect device is 220V.

Described AT series single-chip microcomputer control unit is made of MEGA2560 single-chip microcomputer and its peripheral circuit produced by ATMEL company.

The working process of the robot fighting device of the present embodiment is as follows:

Referring to Figure 3 and combined with Figure 2, when the ATmega2560 MCU is initialized, the ESP8255WIFI module on the MCU will establish the connection between the MCU and the PC connected to the Kinect device through the initialization settings. At this time, the WIFI module is used as the client, and the PC is used as the server. ; After the wireless communication is established, the Kinect device will start to recognize the player's actions and collect valid information. After certain processing, the data will be sent to the MCU through the WIFI module. The single-chip microcomputer analyzes the received information, and extracts the required control information of the steering gear and the motor. The process of analysis and information extraction adopts conventional means. At the same time, set two output I/O ports, and control the forward rotation, reverse rotation and speed of a motor by changing the high and low levels of the two I/O ports and the square wave duty cycle. Set an output I/O port. During the working cycle of the steering gear, change the high and low level duty cycle to control the rotation of the steering gear from 0° to 180°. The motor information collected by the Kinect device can include forward and reverse rotation and rotational speed; the collected steering gear information can include the steering gear angle of each shutdown of the robot. After the single-chip microcomputer outputs the processed information to the drive circuit, it can control the movement of the chariot and the action of the robot. In addition, several micro switches are installed on the body of the robot. When the micro switches are hit, one of the five LED lights on the chariot will be turned off. When all the lights are off, the WIFI module will disconnect from the PC, the robot will not be able to receive information, and the player will not be able to control the chariot robot through the Kinect device.

As shown in Figure 4, the control method of the robot combat device based on somatosensory control in this embodiment includes the following steps:

Turn on the power supply of the Kinect device, connect the Kinect device to the host server computer, complete the initialization of the Kinect device, and turn on the power switch of the chariot robot at the same time, initialize the single-chip microcomputer, and establish a connection between the client single-chip microcomputer and the PC.

After the connection is completed, the Kinect device begins to recognize and capture the player's skeleton within about 1 to 3 meters in front of the somatosensory camera, and collect the player's body movements. The PC analyzes and processes the position information of each bone joint of the player read by the Kinect somatosensory camera. The Kinect device can map a 3D scene map within the range of the camera and give three-dimensional space coordinates. The Kinect device can obtain the effective joint angle of the player or recognize the gesture intention through spatial positioning, analyze the action information, and read the control information required to control the chariot robot. The information read includes: the angles of the six joints of the hands of the player controlling the robot, the direction of waist rotation, and the inclination angle of the upper body; the relative position of the hands of the player controlling the car, specifically the line between the palms of both hands and the horizontal line angle between. The information collected by the former controls the movement of the robot, while the information collected by the latter controls the forward, backward, turning and speed of the car.

Analyzing action information Analyzing action is to judge the player's action intention by using the obtained position information through the following methods, such as comparison of relative positions, measurement and comparison of angles, setting of time parameters, etc.

Set a specific action model according to the action characteristics that you need to collect. If the detected action characteristics conform to the set action model, the corresponding execution program will be triggered. like:

1. Detection of raised hands: Take the height from the shoulders to the head as a unit. If the palm is higher than the head by one or more units, it is considered that the player's hands are raised.

2. Detect arm raising: Take the body plane as a coordinate reference system. If the angle formed by the arm bone and the vertical axis is 90°±10°, it is considered as the player’s arm raising.

3. Detect punch action: set a time parameter. Take the length of the shoulder as a length unit. If the distance between the palm and the shoulder increases from one unit to greater than or equal to two length units within this period, it is considered that the player has punched.

In this embodiment, other actions are also established according to action characteristics.

After collecting the required control information, the PC sends the packaged and packaged control information to the microcontroller through the connection established between the WIFI module and the client microcontroller.

When the single-chip microcomputer is initialized and the connection between the single-chip microcomputer and the server computer is established through the WIFI module, the single-chip microcomputer is in a state of waiting to receive control data from the server computer. When the data is sent to the MCU through the WIFI module. The single-chip microcomputer analyzes the received information and extracts the required control information of the steering gear and motor. Specifically, it includes the angle control information of 9 steering gears and the control information of 2 motors. The single-chip microcomputer changes the rotation angle of each servo by changing the PWM duty cycle of the output port. Different combinations of different angles of each servo can present different movements of the robot. So as long as you input a specific angle to each servo, you can make the robot make the desired action. The single-chip microcomputer continuously updates the player's action information collected by the Kinect device. In the case of a small update delay, the effect of synchronizing the robot's action with the player can be achieved. In addition, through the motor drive module, the speed and direction of the motor can be changed. When there is a speed difference between the speeds of the two driving wheels, the car can be turned. The larger the speed difference, the larger the turning range, and the forward and backward functions can be realized by changing the direction of the motor.

The above-mentioned single-chip microcomputer analyzes the received information, and extracts the required steering gear and motor control information as follows:

(1) The PC side packs the data and sends it to the microcontroller. The specific packing method is: data discriminator + skill judgment data + steering gear and motor control information + check digit.

(2) The data discriminator is used as the header of the data group to remind the MCU to receive information; the skill judgment data is also two-digit hexadecimal data, including the types of skills that cannot be used. If the skill is triggered, the remaining data will be skipped and executed directly Skill program; if it is a non-skill command, analyze the steering gear and motor control information, convert the required 9 steering gear angles and 2 motor control information into two-digit hexadecimal, and arrange them in a certain order The check digit is to take the remainder of 256 after accumulating and summing the hexadecimal data except the data discriminator, and use the remainder as the check digit.

(3) After the single-chip microcomputer receives a piece of valid information, it accumulates and sums the data except the check digit and takes the remainder of 256, and compares it with the check digit in the data. If the check digit is incorrect, it does not analyze the information and receives The next piece of information; if the check digit is correct, extract the skill data or the control data of the steering gear and the motor, perform reverse decoding and convert it into decimal control information, and then control the circuit.

In addition, a number of micro switches are installed on the body parts of the robot, and these micro switches are used as touch points when the robot is hit. There are 5 LED lights at the rear of the car, which are used as the life value of the robot. When the device enters the normal working state, the micro switch is in the sensing state, and all 5 LED lights are on at the same time. When any one of the micro switches senses a touch, it turns off an led light. When the 5 LED lights are all off, the client MCU will disconnect from the PC, the data cannot be transmitted, and the player cannot control the chariot robot through Kinect somatosensory.

When the action command is triggered, the player only needs to make the set action of the corresponding skill. When the computer successfully recognizes the action as a skill action, it will send a specific skill command to the single-chip microcomputer, and the single-chip microcomputer will execute the corresponding program command to make the robot perform the skill action set in advance.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (7)

1. The robot fight device based on somatosensory control comprises a war chariot robot, wherein the war chariot robot comprises a robot part and a trolley part, and is characterized by further comprising a Kinect device for identifying actions of players and collecting action information and a control device for receiving the action information collected by the Kinect device and controlling the war chariot robot according to the action information; the Kinect device comprises a Kinect sensor, a Kinect somatosensory camera for motion recognition, a depth sensor and a PC end, wherein the Kinect sensor, the Kinect somatosensory camera and the depth sensor are all connected with the PC end, the control device comprises a WIFI module, and the Kinect device is communicated with the control device through the WIFI module; the control device is arranged in the war chariot robot;

the control device comprises a singlechip, a power module, a communication module, a motor driving module and a function key module, wherein the power module, the communication module, the motor driving module and the function key module are all connected with the singlechip;

the war chariot robot comprises a motor and a steering engine, wherein the singlechip is provided with two I/O ports for controlling the motor, and the forward rotation, the reverse rotation and the rotating speed of one motor are controlled by changing the high and low levels of the two I/O ports and the square wave duty ratio; the single chip microcomputer is also provided with an I/O port for controlling the steering engine, and the rotation of the steering engine from 0-180 degrees is controlled by changing the high-low level duty ratio of the I/O port in the working period of the steering engine;

the Kinect device recognizes and captures bones of a player within a range of 1-3 meters in front of the somatosensory camera, collects body actions of the player, the PC analyzes and processes the read position information of each bone joint of the player through the Kinect somatosensory camera, the Kinect device maps out a 3D scene graph within the camera range, gives out three-dimensional space coordinates, the Kinect device obtains effective joint angles or gesture recognition intentions of the player through space positioning, analyzes the action information, and reads control information required by controlling the warcraft robot; wherein the read information includes: controlling angles of six joints, waist rotation direction and upper body inclination angles of hands of a player of the robot; controlling the relative positions of hands of a player of the trolley, specifically the angles between the connecting lines between the palms of the hands and the horizontal line; the information collected by the former is used for controlling the action of the robot, and the information collected by the latter is used for controlling the advancing, retreating, turning and vehicle speed of the trolley.

2. The robot combat device based on motion sensing control of claim 1, wherein the power module employs a 7.4V rechargeable battery and a voltage stabilizing chip to power the control device.

3. The robot fight device based on motion sensing control according to claim 1, wherein the chariot robot is provided with a plurality of micro switches and a plurality of LED lamps, the micro switches and the LED lamps are connected with the control device, when the micro switches are hit once, one LED lamp is extinguished, and when all the LED lamps are extinguished, the WIFI module disconnects the Kinect device from the control device.

4. A control method of a robot fighter device based on motion sensing control according to any one of claims 1-3, characterized by comprising the steps of:

1) Connecting the Kinect device with a host server computer to enable the Kinect device to collect body actions of a player before the Kinect somatosensory, and collecting action information of the player in real time by using the Kinect device;

2) Starting a power supply of the war chariot robot to enable a client singlechip on the war chariot robot to be connected with a host server computer PC;

3) The PC machine analyzes and processes the position information of each joint of the player read by the Kinect somatosensory camera to obtain effective joint angle or gesture intention, analyzes action information and reads control information required by controlling the warfare robot;

4) The PC establishes connection with the client singlechip through the wireless WIFI module and sends control information to the singlechip; the method comprises the following steps:

after the singlechip completes initialization and establishes connection between the singlechip and the PC through the WIFI module, the singlechip is in a state of waiting for receiving control data of the PC;

when data is sent to the singlechip through the WIFI module, the singlechip analyzes the received information and extracts the needed control information of the steering engine and the motor, wherein the control information comprises the angle control information of 9 steering engines and the control information of 2 motors; the singlechip changes the rotating angle of each steering engine by changing the PWM duty ratio of the output port, and shows different actions of the warfare robot through different combinations of different angles of the steering engines, so that the robot can make expected actions only by inputting a specific angle to each steering engine; the single chip microcomputer continuously updates the action information of the player acquired by the Kinect device, and the action and the synchronization effect of the warfare robot can be achieved under the condition of low updating delay; meanwhile, the motor driving module is used for controlling and changing the rotating speed and steering of the motor, when the rotating speeds of the two driving wheels have the rotating speed difference, the trolley can realize the turning function, and the larger the rotating speed difference is, the larger the turning amplitude is, the motor steering is changed, so that the forward and backward functions can be realized;

5) The singlechip receives control information acquired by the PC in real time, analyzes the control information through a specific algorithm, drives the motor driving circuit and the steering engine driving circuit according to the obtained information, changes the rotating speed of the motor by changing the duty ratio of PWM waves input by motor driving, and changes the rotating angle of the steering engine by changing the duty ratio of PWM waves input by the steering engine, thereby realizing the control of the warfare robot.

5. The method for controlling a robot fighter device based on motion sensing control of claim 4, wherein the step 3) specifically comprises:

the Kinect device recognizes and captures bones of a player within a range of 1-3 meters in front of the somatosensory camera, collects body actions of the player, the PC analyzes and processes the read position information of each bone joint of the player through the Kinect somatosensory camera, the Kinect device maps out a 3D scene graph within the camera range, gives out three-dimensional space coordinates, the Kinect device obtains effective joint angles or gesture recognition intentions of the player through space positioning, analyzes the action information, and reads control information required by controlling the warcraft robot; wherein the read information includes: controlling angles of six joints, waist rotation direction and upper body inclination angles of hands of a player of the robot; controlling the relative positions of hands of a player of the trolley, specifically the angles between the connecting lines between the palms of the hands and the horizontal line; the information collected by the former is used for controlling the action of the robot, and the information collected by the latter is used for controlling the advancing, retreating, turning and vehicle speed of the trolley.

6. The control method of the robot fight device based on somatosensory control according to claim 4, wherein 5 LED lamps are installed on the warfare robot as life bars of the warfare robot, when the device enters a normal working state, the micro switch is in a inductable state, meanwhile, the 5 LED lamps are all on, when any one of the micro switches senses touch, one LED lamp is extinguished, when the 5 LED lamps are all off, the client singlechip is disconnected from the server computer, data cannot be transmitted, and a player cannot operate the warfare robot.

7. The method for controlling a robot fighter device based on motion sensing control of claim 4, wherein the fighter vehicle robot fighter device uses a 7.4V rechargeable battery and a voltage stabilizing chip to supply power to each module and component.