WO2015143968A1

WO2015143968A1 - Method for remotely controlling robot, and robot avatar network

Info

Publication number: WO2015143968A1
Application number: PCT/CN2015/073488
Authority: WO
Inventors: 余浪
Original assignee: 余浪
Priority date: 2014-03-23
Filing date: 2015-03-02
Publication date: 2015-10-01
Also published as: CN103901856A

Abstract

A method for remotely controlling a robot, and a robot avatar network. Users remotely control a robot (3) functioning as a user avatar to move in a large range so as to explore a remote environment. Multiple users make reservations on a server (1) to remotely control the robot avatar (3) in different time segments; when a user sends connection invitation signaling, the server (1) determines, according to stored reservation information, whether the user has made a reservation for the specified robot avatar (3); in a case where the reservation has been made, the server (1) helps to establish a connection between a smart device (2) of the user and the robot avatar (3); the connection between the smart device (2) and the robot avatar (3) is automatically disconnected after a certain period of time according to the reservation information, so that other users have chances to control the robot avatar (3). The method improves usage efficiency of multiple users on a robot avatar network, avoids conflicts between the multiple users, lowers a security risk, reduces traveling of people, and helps with low-carbon environmental protection.

Description

Method for remotely controlling a robot and a robotic avatar network

This application claims priority to Chinese Patent Application No. 201410129394.0, entitled "Method for Remote Control of Robots and Robotic Incarnation Network", filed on March 23, 2014, the entire contents of which are incorporated by reference. In this application.

Technical field

The present invention relates to a method for remotely controlling a robot and to a robotic avatar network.

Background technique

With the development and integration of power technology and information technology, the Cyber Physical System has been increasingly used. The mobile robot network is a representative information physical fusion system. When a plurality of movable robots are connected to the communication network as human avatars, a robot avatar network is formed. A plurality of users remotely control the robot avatar via the communication network via the smart device to explore the environment in which the remote robot is located. The smart device can be a smartphone, a tablet or a personal computer. The video information of the remote environment is transmitted to the user's smartphone by the avatar of the robot and displayed on the screen of the mobile phone. The user can send the control data through the smart phone to control the large-scale movement of the robot avatar, and the haptic data generated by the touch screen of the smart phone can be converted into Robot control data.

The robot avatar network can be used for art display, users can remotely control the robots located in the art gallery through the smartphone or tablet at home, and make the robot move around the artwork such as sculpture, the robot camera module will The acquired video information is transmitted to the user's smartphone or tablet in real time. As a result, the user can obtain more information than the artwork, thereby laying the foundation for subsequent artwork transactions. Compared to the 360-degree panoramic view of the artwork, the robot avatar can provide more realistic artwork information, and the user can pass By controlling the robot to observe the artwork from all angles, it is even possible to continuously observe multiple pieces of art in the art gallery. Here, a large number of users expect to remotely control a limited number of robots in the art gallery.

There are other possible applications for the robot avatar network. People can use robots to complete many things without leaving home. Due to the large reduction in people's travel, the robotic avatar network not only reduces traffic congestion, but also saves energy, and also reduces carbon dioxide emissions to protect the environment.

U.S. Patent Application Serial No. 20,100,076,600, the disclosure of which is incorporated herein by reference. The patent application utilizes existing point-to-point (P2P) VoIP technology to transmit video information and encode control information in the voice signal. Robots can be purchased by the family and controlled remotely by family members. In establishing the connection between the user and the robot, the server only verifies the identity of the user and whether the robot avatar is online. A single user can only remotely control a small number of robot avatars. Robot avatars are inefficient to use. Due to the existing VoIP protocol, although it is easier to implement remote control and video transmission, it is not easy to modify existing signaling during the connection establishment phase to improve efficiency.

US Patent Application Publication No. 20140071291 refers to a method for transmitting video data and haptic data and a video capture device as an avatar. The patent application utilizes a P2P technology to establish a first channel for transmitting video data and a second channel for transmitting haptic data between a computer connectable to the Internet and a movable video capture device connectable to the Internet. This is hereby incorporated by reference in its entirety. Since the P2P technology is implemented from the lower layer through the SOCKET, the signaling can be improved during the connection establishment phase to improve the efficiency of the robot avatar.

In reality, reservations are often used to improve the efficiency of using limited resources. The patient can call the clinic to make an appointment, but, unfavorably, the confirmation of the appointment is done manually by the clinic staff. Google's calendar online software also has the function of booking a conference room. Users who want to reserve a conference room can view the reservation of the conference room. If the conference room is free, the user can reserve the conference room at that time. Use rights. However, it is disadvantageous that during the actual use of the conference room, the person who reserved the conference room in the previous period often lengthens the time occupied by the conference room, so that the person who reserves the conference room in the next period cannot use the conference room, so it is difficult to avoid the resources. Use conflicts.

For the existing robot avatar network, there is a need to improve the efficiency of use, and should be as Reduce labor costs and avoid conflicts.

Summary of the invention

It is an object of the present invention to provide a method for remotely controlling a robot and a robotic avatar network to form a highly efficient, multi-user to multi-robot robotic avatar network and avoid conflicts as much as possible.

This task is solved by a method having the features of claim 1 and a robotic avatar network having the features of claim 9.

In terms of the method of the present invention, the idea of the present invention is that the robot is used as an avatar of the user, and the video data is transmitted from the robot avatar to the user's smart device via the communication network, and the user transmits the control data to the robot avatar via the communication network via the communication device. The user remotely controls the robot avatar to move in a wide range to explore the remote environment, and the server exists in the communication network. The method of the present invention includes the following steps: multiple users make reservations on the server to remotely control the avatar at different time periods; the user passes the smart device Sending connection invitation signaling to the avatar via the server; the server determines whether the user has reserved the current time period; in the case where the current time period has been reserved, the server helps establish a connection between the smart device and the avatar of the robot; after establishing the connection, the user The smart device and the robot avatar transmit video data and control data via the communication network; according to the reservation information, the connection between the smart device and the robot avatar is automatically interrupted after a period of time, so that other users have the opportunity to remotely control Robots in disguise.

The user's smart device may be a smart phone or tablet that the user can carry with him or a user's personal computer or laptop.

The reservation information may be a time period, such as a time period of 19 o'clock to 20 o'clock, or a duration of connection, such as 34 minutes. Since the user can reserve a plurality of consecutive time periods, and the user may issue a connection request after a few minutes after the start of the reservation period, the duration of the connection is not fixed.

Compared with a robot used in a home and a networkable robot that utilizes the VoIP protocol (the VoIP protocol is modeled on the use of a telephone, does not include a reservation mechanism for the device), the present invention The method improves the use efficiency of the robot as a user avatar, and multiple users can remotely control the robot at different time periods, and the previous user does not continue to occupy the robot after the appointment period expires, and the user who made the reservation in the next period can be remotely controlled. The robot thus avoids conflicts.

Compared to current clinic appointments or conference room appointments, the method of the present invention reduces manual intervention, thereby reducing costs. In addition, the method of the present invention effectively avoids resource occupation conflicts in the conventional reservation mode.

In a refinement of the present invention, the server transmits the reservation information to the smart device during the establishment of the connection process, and according to the reservation information, the smart device automatically disconnects after a period of time according to the criterion, which may be before the appointment period expires. Disconnected in one minute.

In another refinement of the invention, the server transmits the reservation information to the robot avatar during the establishment of the connection, and according to the reservation information, the robot avatar automatically disconnects after a period of time according to the criteria.

In still another embodiment of the present invention, after the user makes a reservation, the user is prompted to make an appointment. The prompt mode can be a short message or an instant message such as WeChat or WhatsApp.

Since the users are all users who have registered on the server. If the user information is stolen, when the unidentified person borrows the registered user information to make an appointment, the registered user can find out in time and cancel the appointment. This greatly reduces security risks. Moreover, the prompting mechanism allows the use of the robot to be accurately recorded in the server, and the person in the remote environment of the robot avatar can know which users have remotely controlled the robot avatar, so that the robot avatar is not to be abused.

In terms of the apparatus of the present invention, the present invention provides a robot avatar network comprising a plurality of robots as user avatars, wherein one robot in the robot avatar network can be selected by a plurality of users, as claimed The method described in 1 is reserved to enable a plurality of users to explore the environment in which the robot is located.

Compared to home-purchased robots for remote control, users have the opportunity to remotely control a large number of robots.

In a refinement of the invention, the robots in the robotic avatar network are arranged in a plurality of art galleries or museums.

Other advantages of the invention are obtained from the following specific examples.

DRAWINGS

The invention will be described in more detail below on the basis of embodiments with reference to the accompanying drawings. In the drawing:

Figure 1 shows the structure of a multi-user to multi-avatar robot avatar network;

Figure 2 shows the complete process of the method according to the invention.

detailed description

Figure 1 shows the structure of a multi-user to multi-avatar robot avatar network. The robot avatar network includes a plurality of robots 3 that can be connected to the Internet as a user avatar, a plurality of smart phones carried by the user, a server 1 having a public network IP address on the Internet, and a communication network 4. The communication network 4 may be a combination of the Internet and a wired local area network, a wireless local area network. In places where the Internet can be connected to the Internet, users registered on the server 4 can remotely control the robot avatar 3 large-scale motion through the smartphone 2 or tablet that is carried around, thereby exploring the environment in which the robot avatar 3 is located. For example, German users can remotely control the robot avatar 3 to see the fine porcelain vases in the Chinese handicraft showroom. Chinese users can remotely control the robot avatar 3 to visit the stone sculptures in the European Museum. Since there are a large number of robot avatars 3 in the robot avatar network and the robot avatar network involves a large number of users, the server 1 manages the connection of the user's smartphone 2 and the robot 3.

The robot 3 as a user avatar does not have to have a public network IP address. The robot 3 can have a private network IP address of the wireless local area network and is connected to the Internet through the wireless network. The robot 3 includes a video input unit 31, a calculation unit 32 for processing video data and control data, an electromechanical unit 33 for performing wide-range motion according to control data, and a communication unit for transmitting video data and control data (not shown) ).

The electromechanical unit 33 can be regarded as the "foot" of the robot avatar 3, which comprises a wheel, a transmission mechanism, a direct current motor for driving the wheel, a stepper motor or a servo motor, power electronics for controlling the motor, and A battery or the like as a power source. The power electronics can, for example, receive a PWM signal from computing unit 32 and determine DC based on the duty cycle of the PWM signal. The speed of the motor or the angle of the stepper motor controls the speed of the wheel. The PWM signal is generated based on control data.

The video input unit 31 can be regarded as an "eye" of the robot avatar 3, and the video input unit can include a CCD camera or a CMOS camera, a video capture card, a pan/tilt head, and the like. The video capture card can for example be connected to the computing unit via a PCI bus. The computing unit can send commands related to video capture to the video input unit, such as starting acquisition, zooming in, zooming out, and the like. The video input unit 31 may be equipped with a plurality of cameras for capturing video information of a plurality of angles or achieving a stereoscopic effect.

The computing unit 32 processes the video data and control data associated with the "foot" or "eye" of the robot avatar, which may be implemented using a computer or embedded computer. The communication unit transmits the video data and the control data, and can be implemented by using a wireless network card or a 3G network card. The video information from the video input unit 31 is compressed and divided into a plurality of data packets, which are then transmitted to the user's smartphone. A packet containing control data sent from the smartphone is received and processed. In conjunction with the state information acquired by the robot's own sensor, the calculation unit 31 generates a PWM signal and inputs it to the electromechanical unit of the robot to control the robot's wide range of movement.

In an embodiment of the invention, a plurality of robot avatars 3 may be arranged in an art gallery. The robot avatar 3 can observe the sculpture 360 degrees, and can observe from a distance and from a close view, and can even view the details of the sculpture by adjusting the focal length of the video input unit 31.

Multiple robot avatars 3 can be placed in the same art gallery. When one robot 3 in the exhibition hall is reserved by one user for a certain period of time, other users can reserve other robots 3 in the exhibition hall to observe the artwork.

The plurality of robot avatars 3 can also be arranged in a plurality of art galleries in a large area, for example, one robot avatar 3 is arranged in a gallery displaying wood carvings, and another robot avatar 3 can be arranged in a bronze sculpture. Inside the exhibition hall. Users can observe different artworks in different exhibition halls by booking different robots 3.

The plurality of robot avatars 3 can also be arranged in a plurality of art exhibition halls in different areas, for example, one robot avatar 3 is arranged in an exhibition hall displaying jade crafts in one city, and another robot avatar 3 can be arranged in another city. Exhibition hall for bronze sculptures. User can Observe different artworks by remotely controlling the robot avatars 3 located in different cities through the same server platform.

Figure 2 shows a sequence diagram of a method according to the invention, which depicts the flow of the method of the invention, the entire flow involving the robotic avatar 3 of Figure 1, a smartphone 21 carried by a user, and the intelligence carried by another user The mobile phone 22 and the server 1 located on the public network.

The robot avatar 3 communicates with the server 1 after power-on, so that the server learns the address information of the robot avatar 3. If the robot avatar 3 does not have a public IP address, it should maintain at least two User Datagram Protocol (UDP) channels between the avatar 3 and the nearest router with a public IP address for network operation. Network Address Translation (NAT). The first UDP channel can be used to transmit video data from the avatar 3 to the smartphone 2, and the second UDP channel can be used to transfer control data from the smartphone to the avatar. If the robot has an IPv6 address, there is no need to maintain a UDP channel for NAT.

To this end, at step 100, the robot avatar 3 registers on the server by transmitting a data packet containing registration signaling and robot information, which may include, for example, the ID number or nickname of the robot. The robot avatar 3 can periodically transmit registration signaling. When it is necessary to maintain a UDP channel, the transmission interval can be less than or equal to 3 minutes.

The server 1 responds to the avatar 3 for the acknowledgment at step 200 after receiving the registration signaling. The address information and other information of the robot avatar will be saved in the server. If the robot avatar has been registered before, the server can update the information of the robot avatar 3.

In fact, the plurality of robot avatars 3 in FIG. 1 will communicate with the server 1 after power-on, and the registration information of the plurality of robots is stored on the server 1.

In order to remotely control the robot avatar 3 via the communication network 4 via a smartphone 2, users around the world should make reservations several days or weeks in advance.

To this end, at step 300, a user can make a reservation for the robot 3 through the smartphone 21 at a time period of 19:00 to 20:00 next Monday. At step 400, another user may make a reservation for the robot 3 through the smartphone 22 during the time period of 20:00 to 21:00 on the next Monday. In addition to using a smartphone, users can also make an appointment by logging into a website located on the server.

During the appointment, the user can first see the selected plurality of robot avatars 3 (and the position where the robot is located if necessary) on the screen of the mobile phone or the computer screen. When the user selects a robot avatar 3, the user can see the robot. The appointment of the avatar 3 can be, for example, an appointment of twenty-four hours seven or seven days this week or next week. The reserved time period can be red, and the unscheduled time period can be green. The user can make a reservation by clicking on the unreserved time period, for example, the user can reserve a time period of one hour from 19:00 to 20:00 on the next Monday. The user can click on a plurality of consecutive unscheduled time periods so that the robot avatar 3 can be remotely controlled for a longer period of time.

Once within the time period reserved by the user carrying the smartphone 21, at step 500, the user can transmit connection invitation signaling including the robot information and the user information to the server 1 via the smartphone 21. Based on the user information and the robot information, the server 1 checks the stored reservation record and determines whether the user has reserved the current time period for the specified robot avatar. If the user has reserved the current time period, the server 1 forwards the connection invitation signaling of the user to the robot avatar 3 at step 510, if necessary, as in the case of applying NAT, the smart phone 21 carried by the user The address information is also forwarded to the robot avatar 3. Further, at step 600, the server 1 transmits back a data packet for confirming the request, the data packet including address information of the robot and reservation information.

The order of step 510 and step 600 described above can be interchanged.

A connection is established between the smartphone 21 and the robot avatar 3 based on the information from the server 1. For the case of applying NAT, the details of establishing a connection can be found in U.S. Patent Application Serial No. 2014007129, the entire disclosure of which is incorporated herein by reference. After the connection is established, at step 700, control data is transmitted from the smartphone 21 to the robot avatar 3, and at step 710, video data is transmitted from the robot avatar 3 to the smartphone 21, and the user remotely controls the robot 3 through the smartphone 21. Move to explore the distant environment.

In the process of the user remotely controlling the robot avatar 3, at step 800, based on the reservation information received from the server 1, the smartphone checks whether the appointment time expires at regular intervals. The time interval can be 30 seconds. When the appointment time is about to expire, at step 810, the smartphone sends a stop signaling to the server and sends a stop message to the avatar 3 at step 820. Therefore, the connection between the smartphone 21 and the robot avatar 3 is broken.

Of course, in another embodiment, the disconnection can be initiated by the robot 3, which acquires the reservation information from the server during the connection process.

In yet another embodiment, the disconnection may also be initiated by the server 1, which issues a stop signaling to the robot 3 and the smartphone 21.

After the robot avatar 3 is again in the idle state, the time is entered into the time period reserved by the user carrying the smartphone 22, and at step 900, the user can send a connection containing the robot information and the user information to the server 1 via the smartphone 22. Invite signaling. Based on the user information and the robot information, the server 1 checks the stored reservation record and determines whether the user has reserved the current time period for the specified robot avatar. If the user has reserved the current time period, the server 1 forwards the user's connection invitation signaling to the robot avatar 3 at step 910. Further, at step 920, the server returns a data packet for confirming the request, the data packet containing address information of the robot and reservation information. Based on the information from the server, a connection is established between the smartphone 22 and the robot 3 and data is transmitted, and a new round of remote control is started, at which time the robot 3 acts as an avatar of the user carrying the smartphone 22 in a different place.

If, for a robot avatar (3), multiple users want to reserve the same time period, then bidding is used to determine which user has the right to remotely control the robot avatar (3) during that time period. It is also possible to determine which user has the right to remotely control the robot avatar (3) during the time period based on the priority of the user.

It is to be understood that the above-described embodiments are not intended to limit the invention, but are intended to illustrate the invention. Numerous modifications and changes can be made by those skilled in the art without departing from the scope of the invention. The scope of protection of the present invention is defined in the appended claims.

List of reference signs

1 server

2 smart device

3 robot avatar

4 communication network

21 smartphone

22 smartphone

31 video input unit

32 calculation unit

33 Electromechanical unit

Claims

A method of remotely controlling a robot, wherein the robot is used as an avatar of a user, and the video data is transmitted from the robot avatar (3) to the user's smart device (2) via the communication network (4), and the user passes through the smart device (2) via The communication network (1) transmits control data to the robot avatar (3), and the user remotely controls the robot avatar (3) to move in a wide range to explore a remote environment in which the server (1) exists.

It is characterized in that

The method includes the following steps:

Multiple users make reservations on the server (1) to remotely control the robot avatar (3) at different time periods;

The user sends a connection invitation signaling to the robot avatar (3) via the server (1) via the smart device (2);

The server (1) determines whether the user has reserved a current time period;

In the case where the current time period has been reserved, the server (1) helps establish a connection between the smart device (2) and the robot avatar (3);

After establishing the connection, the video data and the control data are transmitted between the user's smart device (2) and the robot avatar (3) via the communication network (4);

According to the reservation information, the connection between the smart device (2) and the robot avatar (3) is automatically interrupted after a period of time, so that other users have the opportunity to remotely control the robot avatar (3).
The method of claim 1 wherein

The smart device (2) is a smart phone that a user can carry with him or her.
The method of claim 1 wherein

The method further includes the server (1) transmitting the reservation information to the smart device (2) during the establishment of the connection process, and the smart device automatically disconnects after a period of time according to the reservation information.
The method of claim 1 wherein

The method further includes the server (1) transmitting the reservation information to the robot avatar (3) during the establishment of the connection, and the robot avatar (3) automatically disconnecting after a period of time according to the reservation information.
The method of claim 1 wherein

After the user completes the appointment, the user is prompted to make an appointment by sending a text message to the user's mobile phone.
The method of claim 1 wherein

After the user makes a reservation, the user is prompted to make an appointment by sending an instant message to the user's smartphone.
The method of claim 1 wherein

If, for a robot avatar (3), multiple users want to reserve the same time period, then bidding is used to determine which user has the right to remotely control the robot avatar (3) during that time period.
The method of claim 1 wherein

The robot avatar (3) is a drone.
A robot avatar network comprising a plurality of robots as user avatars, wherein one robot avatar (3) in the robot avatar network can be selected by a plurality of users, and according to the method of claim 1. It is reserved to enable multiple users to explore the environment in which the robot avatar (3) is located.
The robot avatar network of claim 9 wherein:

The robot avatar (3) in the robot avatar network is arranged in a plurality of art exhibition halls, and the user controls the artwork in the exhibition hall by remotely controlling the robot avatar (3).