KR101009721B1 - Mobile image robot phone - Google Patents

Abstract

The present invention discloses a mobile video robot terminal that tracks a user's location and can perform AV communication smoothly with a heterogeneous terminal.

The robot phone according to the present invention includes a robot driver which performs a mechanism control having a user following function, maintains a short distance communication state with the robot driver, controls robot driving according to a surrounding environment, and photographs and transmits a surrounding image. And a system controller for transmitting and receiving a call voice signal, and a communication modem for transmitting or receiving a video and audio signal provided from the system controller to an external heterogeneous terminal.

Therefore, the present invention is to maintain the distance between the robot phone and the caller within the standard value, thereby increasing the ease of use with the natural atmosphere of the voice call or video call. In addition, by implementing a replaceable structure of the mobile communication terminal mounted as a robot phone, the communication with the heterogeneous terminal has an effect that the AV communication can be performed smoothly.

Robot Phone, Communication, Image, Sound, AV, Terminal, Detachable, Modem, LMS, AEC

Description

Mobile Visual Robot Terminal {MOBILE IMAGE ROBOT PHONE}

The present invention relates to a video communication robot phone, and more particularly, it is possible to make a call connection with the robot phone regardless of the type of mobile communication terminal of the callee, and to improve the sound quality at the time of the call connection, The present invention relates to a mobile visual robot terminal capable of providing a mechanism for maintaining a certain distance.

In recent years, a variety of information is provided so that a remote user can access the Internet to search for and talk with people, pets, and household appliances, etc. Service robots for controlling a plurality of furniture, appliances, etc., and security devices in the residential premises are being released in accordance with services and control signals.

This will be described with reference to the accompanying example drawings. First, Figure 1 is a block diagram showing a video communication service system of a conventional service robot, Figure 2 is a block diagram of a robot in the residential quarter. As shown, a user at an external remote site is authenticated through a service server 200 of a service center with a PC / laptop 10, a PDA 20, a mobile phone 30, etc., which is a remote user terminal 00. The service robot 100 moves through the wireless LAN repeater 140 of the elderly to look at the elderly 150 and pets 160, or furniture and appliances 170, gas valve 180, etc. Check the condition and instruct the opening and closing of the heating apparatus at each position.

In addition, the user in the living quarters is an image display 105 using the touch screen 104 connected to the USB1 / HUB 121 which is a USB port expansion device built in the main operation and control module 130 of the robot 100. Touch the robot UI menu prepared as the main menu, and can receive or reserve services such as after-sales service with the product customer center 310 and the moving image through the service server 200 in the service center. Touch) to watch the new product and listen to the explanation through video communication. In addition, by using the content provided by each type of service provider 330, real-time broadcasting or foreign native English lectures are also received 1: 1.

The service robot 100 in the residential quarter is largely divided into a power supply and voltage regulation module 106, a sensing acquisition module 107, a main operation and control module 120, and a drive control module 130. The regulation module 106 is equipped with a main power switch 101, a robot energy storage / source device 102, and a supply adapter 103 which is an energy supply device, so that the main operation and control module 120 has 12V 4A (123) and 5V. The TFT LCD 105, which supplies the power of 10A 124 and the image display, also supplies the power of 12V 4A, and supplies the power of the 12V 9A 132 to the drive unit by the power stored in the robot energy storage / source device 102. Supply.

The sensing acquisition module 107 is equipped with an obstacle detecting sensor 108, a temperature detecting sensor 109, a human body detecting sensor PIR 110, a gyro sensor 111 to keep the horizontal and an anti-tilt sensor, the main operation and The control module 120 is connected to a joystick 125 which is a wireless robot motion control device, a wireless LAN communication device 126, and a camera 127 which is an image and image input device. There is USB1 / HUB (121) which is a USB port expansion device, and there are two COM1 (115) and COM2 (116) which are RS232 serial communication devices in USB2 / HUB (122) which is a second USB port expansion device. RS232 communication is performed between the module 107 and the drive control module 130, and the USB2SERIAL 131, which is a device for converting USB to serial, connects the PAN / TILT 128, which is a motion control device for camera position control.

Here, the drive control module 130, which is a drive control device, controls two motor drive drivers 133 and 134. In addition, the service server 200 is configured to have its own unique IP on the Internet, the service server 200 may be installed and managed separately by having a service center, but requested by the ISP company and the robot 100 in the residential area The main operation and control module 120 in the robot are operated by exchanging messages with the robot.

In order to exchange video and voice messages with the main operation and control module 120 of the robot 100, the Java language prohibits direct access to hardware for system security. Is called.

As described above, the conventional service robot or a robot including the same has been establishing a communication system using a wireless communication infrastructure. In addition, an image of a remote location may be captured using such a communication system, or an image captured by a robot may be remotely transmitted. However, since the above-described communication system performs video transmission and reception based on a preset communication method, video communication using a robot is inevitably limited to a specific terminal.

The present invention has been made to solve the above problems, and an object of the present invention is to have a fluid mechanism to maintain the user's close position during a voice call or a video call, thereby inducing a natural call environment and ease of use It is to provide a mobile image robot terminal that can be increased.

Another object of the present invention is to provide a mobile video robot terminal capable of increasing the communication system function of a video communication robot by expanding a type of mobile communication terminal connected to a video communication robot by network connection.

Still another object of the present invention is to provide a mobile video robot terminal capable of increasing the efficiency of a call system by eliminating howling on a voice call, which may inevitably occur due to the structural characteristics of the robot.

Mobile image robot terminal according to an aspect of the present invention for achieving the above object, the robot driving unit for performing a mechanism control for driving the robot, maintains the near-field communication state with the robot driving unit, and controls the robot drive in accordance with the surrounding environment And a system controller for capturing and transmitting a surrounding image or transmitting and receiving a call voice signal, and a communication modem for performing a voice call or a video call based on the video and audio signals provided from the system controller. The robot driving unit includes a sensor driving unit including a motor for driving the robot, a battery for supplying system power, a plurality of sensors for detecting peripheral obstacles, a residual amount detection of the battery, and driving control of the motor. Detects a peripheral obstacle based on a signal detected from the sensor driver And a robot controller that performs user tracking control based on motion control and user position determination corresponding thereto; The system control unit includes a speaker and a microphone for inputting and outputting a voice signal according to a communication call connection of the communication modem, a camera module for capturing the surrounding image of the robot, a display module for displaying the expression of the robot, and a user's voice call. Call control to perform the video, Image control to transmit the image taken from the camera, Based on the ambient sound signal input from the microphone to determine the position of the user and from this to generate a user follow control signal for determining the direction and movement of the robot A system controller; The communication modem includes a mobile modem having a communication module of any one of CDMA-2000, WCDMA, GSM, WiMAX, and LTE, and a modem controller providing an operation algorithm of the communication module held by the mobile modem. It is done.

The present invention maintains the distance between the video communication robot and the caller within a standard value, thereby increasing the ease of use as well as creating a natural atmosphere of a voice call or a video call. In addition, by implementing a replaceable structure of the mobile communication terminal mounted as a robot, the communication with the heterogeneous terminal has an effect that the AV communication can be performed smoothly. In addition, regardless of the structure of the robot during the voice call of the user to facilitate the delivery of the voice, there is an effect that can increase the maturity of the video communication robot, and increase the activation of the robot market.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

3 is a perspective view showing a video communication robot phone according to the present invention. Here, the robot phone (ROBOT PHONE) is a combination of the function of the robot and the function of the mobile terminal, is a term used as a product name registered in the US-Korea joint technical task, it applies in the present invention. First, the front (a) and the back (b) of the video communication robot phone 300 is shown, and the emotion display lamp 301 and the speaker 305 in front, and the microphone 303 in both sides. In addition, the mobile communication terminal 307 is fastened to the back of the video communication robot phone 300 to be detachable, the terminal 307 includes a communication antenna 311 and the display panel 309, the front side of the emergency It has an emergency switch 315 for notifying the situation to the outside. In addition, the video communication robot phone 300 is equipped with a driving wheel 313 on the bottom.

The center of the emotion display lamp 301 is equipped with an infrared sensor, a digital camera, and transmits the image captured by the robot phone through the digital camera, and detects the user's position using the microphone and the infrared sensor.

The emotion display lamp 301 is a lamp for deforming the shape of the eye (circular) to correspond to the operating state of the system and has a plurality of LEDs therein. The microphone 303 and the speaker 301 are means for allowing a user in proximity to perform voice communication. The display panel 309 of the terminal 307 is a panel for a video call and is configured as a watch panel to provide convenience in AV communication.

As the basic operation of the video communication robot phone 300 configured as described above, the video call function of the remote mobile phone and the intelligent video communication robot phone enables the user to make a call without lifting the terminal, and the video call robot phone follows the calling party. AI terminal function. This is to detect the position of the user's voice during a voice or video call, and to determine the movement of the robot phone based on the detected position. Of course, if necessary, the user's voice position and the user's body detection, that is, the infrared sensor may be used to more accurately follow the user's location.

In addition, as an additional function of the robot phone, if you press the emergency button in an emergency situation for the disabled, the elderly living alone, or the child, the protection function delivers the image to a specific person, and monitors the movement inside the specific place. Surveillance function to transmit the image to a specific person, remote control function to move and control the intelligent robot phone remotely using mobile phone, robot function to follow the moving object to make video call, find the location of self, WLAN built-in It controls robots (cleaning robots) or systems (door cameras, air conditioners, boilers, etc.) or has monitoring and remote control functions.

Accordingly, the present invention provides a detachable structure of the mobile communication terminal 307 so as to be able to connect to a communication network with a heterogeneous terminal in connection with the basic operation of the video communication robot phone 300, and the mobile communication terminal 307 When the call is connected using the () to prevent howling due to the narrow space between the microphone 303 and the speaker 305. That is, the mobile communication terminal 307 of the detachable structure is capable of making a call connection or video transmission / reception with a terminal having a communication protocol such as CDMA-2000, WCDMA, GSM, WiMAX, LTE, etc. by replacing the terminal. .

In an exemplary embodiment of the present invention, the terminal 307 is replaced for a call call connection with a heterogeneous terminal. However, the terminal 307 has a plurality of heterogeneous communication modules inside the terminal 307, and is selectively used by a user or the type of the terminal. After recognizing, we will be able to automatically sort.

4 is a configuration diagram for explaining the main functions of the video communication robot phone 300 according to the present invention. As shown, the image communication robot phone 300 maintains a near field communication state with the robot driver 401 and the robot driver 401 performing a mechanism control having a user following function, and controls the robot drive to the surrounding environment. According to the control unit, and a system controller 403 for capturing and transmitting a surrounding image or transmitting and receiving a call voice signal, and a communication modem for transmitting or receiving an image and an audio signal provided from the system controller 403 to an external heterogeneous terminal. 405.

The communication modem 405 is a modem for performing communication with any one of terminals having a communication protocol such as CDMA-2000, WCDMA, GSM, WiMAX, LTE, and the like. The modem 405 can be replaced. Therefore, the system controller 403 has a connector for electrical connection with the communication modem 405, or uses a wireless connection with the communication modem 405, for example, Zigbee, Bluetooth, WIFI, UWB, infrared communication, or the like. Of course, data communication can be made.

The robot driver 401 includes a motor 419 for driving the robot, a battery 417 for supplying system power, and a sensor driver 413 including a plurality of sensors 415 for detecting peripheral obstacles. And detecting a residual amount of the battery 417 and driving control of the motor 419, and detecting peripheral obstacles based on a signal detected from the sensor driver 413 and corresponding to the motion control and user position determination. The robot controller 410 performs user tracking control.

The system controller 403 includes a speaker 305 and a microphone 303 for inputting and outputting a voice signal according to a call connection of the communication modem 405, a camera module 421 for capturing a peripheral image of the robot phone, and A display module 425 for displaying an expression of the robot phone, a call control for performing a voice call of a user, an image control for transmitting an image captured by a camera, and an ambient sound signal input from the microphone 303. And a system controller 427 for determining a user's position and generating a user following control signal for determining the direction and movement of the robot phone.

The communication modem 405 provides a mobile modem 430 having a communication module of any one of CDMA-2000, WCDMA, GSM, WiMAX, and LTE, and an operation algorithm of the communication module held by the mobile modem 430. It consists of a modem controller 431. Here, the modem controller 431 includes a SIM card 433, and determines the approval of the use of the mobile modem 430 according to the authentication result of the SIM card 433. The SIM card is provided for the purpose of user authentication, and can be applied as a universal subscriber identity module (USIM).

Meanwhile, the system controller 403 sets a communication protocol corresponding to the communication modem 405, performs AV communication with the communication modem 405, and performs short-range wired communication with the robot controller 410. It includes communication control function. In addition, the system controller 403 has a USB interface 423 for performing communication with an external terminal, and the display module 425 is composed of an LED or LCD.

In addition, the system controller 427 has an interference cancellation system of a voice signal. The voice signal output from the speaker 305 is input to the microphone 303 again so that the video communication robot phone is connected to the call sound. This function is to prevent the noise of the robot phone. This is because the video communication robot phone uses the microphone 303 to measure the volume level of the user, determines the user's position therefrom, and then determines the rotation direction and the moving direction of the robot phone. It is to prevent the output sound and the user's voice judgment error in advance.

Hereinafter, the operation of the present invention will be described in detail with reference to the accompanying drawings.

First, the video communication robot phone 300 may change the communication modem according to the type of the terminal 307 mounted to the rear. As shown in FIG. 5, as the terminal 307 is detachably configured, the communication modem 405 can be changed. The user can selectively mount a communication modem such as Wibro, WiMAX, WCDMA, LTE, etc. by changing the terminal 307, thereby widening the width of the voice or video call.

For example, the communication modem 405 may be changed to a CDMA modem for a voice call, a WCDMA modem for a video call, or a WiBro or WiMAX modem for transmitting and receiving wireless Internet video. As such, the replacement of the communication modem 405 is recognized by the system controller 427 of the system controller 403. The system controller 427 provides the power provided by the robot driver 401 to the communication modem 405 and determines the type of the communication modem 405.

As described above, the communication modem type is any one of modems for performing functions such as voice call, video call, wireless Internet, and the like. The system controller 427 is a camera module, an LCD module, a voice output module, etc. Enable control. That is, when the currently connected communication modem 405 is a Wibro modem, the system controller 427 of the system controller 403 operates the camera module 421 to perform wireless Internet or image transmission and reception. Of course, when the currently connected communication modem 405 is a CDMA modem, the system controller 427 enables the speaker 305 and the microphone 303 to enable voice calls.

Here, the call voice of the user is input to the microphone 303 of the robot phone spaced apart from the user. The robot phone determines the position of the user by following the input signal level of the microphone 303 and based on the determination result, Keep close proximity. Of course, the direction of the robot phone is also to face the user. That is, as shown in FIG. 8, the system controller 427 continuously converts the acoustic signals (user voice signals) input from the microphones 303 on the left and right sides in step S801, and then converts the digital signals. Measure the level.

In addition, the size of the input signal of each microphone 303 currently measured in step S803 is calculated. When the signal level input from the right microphone is greater than the signal level input from the left microphone as a result of the calculation, the steps S805 and S807 are performed. After storing the current sound level value, the robot phone is rotated to the right. Here, the rotation amount is designed to be operated at a predetermined angle, for example, 10 degrees. Of course, if the signal level input from the left microphone is greater than the signal level input from the right microphone, as determined in step S803, the rotation direction of the robot will be determined through steps S817 and S819.

As such, when the robot phone is rotated by a predetermined angle according to the user's sound, the robot phone may compare the currently received sound level with a previously stored sound level in operation S809. If the currently received sound level is less than or equal to the pre-stored sound level value, that is, the rotation amount of the robot phone is large and the operation of stopping the robot phone in operation S811 is stopped. On the other hand, if the user sound level after the rotation operation of the robot phone is larger than the pre-stored sound level value, that is, the amount of rotation of the robot phone is fed back to the step S803 to repeat the process. This is to follow the value of the largest user sound level, the robot phone detects the maximum value of the sound level, and determines the rotation direction of the robot phone based on this. However, in the present invention, if the user's sound is not input for a certain time, the system controller 427 performs the initialization at the current position of the robot phone.

As described above, when the rotation direction of the robot phone is determined, the system control path 427 determines whether the distance between the robot phone and the user is maintained within a reference value in step S813. This calculates the distance to the user using an infrared sensor, an ultrasonic sensor, an optical sensor or a thermal sensor mounted to one side of the robot phone, and determines whether the calculation result is within preset information, for example, within a range of 20 cm to 30 cm. . If the distance between the robot phone and the user exists within the set distance, the feedback to step S801.

On the other hand, when the distance between the robot phone and the user exists within the set distance, the system controller 427 controls the robot phone to move backward, and when the distance between the robot phone and the user is outside the set distance, the system controller 427 The robot phone will be controlled to move forward. Therefore, the system control furnace 427 controls the robot phone to maintain a constant distance from the user.

As described above, the following function of the robot phone is to bring the user and the robot phone closer to each other, so that the user can make a call at the current location when the user makes a voice call or a video call.

Next, the communication operation will be described. First, when the system controller 427 recognizes a connection state with the communication modem 405, the communication modem 405 receives power from the system controller 403, and the modem controller 431 of the communication modem 405. ) Performs a communication connection with the system controller 427. Here, when the communication modem 405 is a voice modem, voice input / output signals of the speaker 305 and the microphone 303 connected to the system controller 427 are provided to the modem controller 431, and the modem controller 431 is provided. Form a call with the base station through the mobile modem 430. Here, the modem controller 431 is responsible for operating the system according to the communication protocol of the mobile modem 430, and processes the voice signal provided from the system controller 427 according to the corresponding protocol to process the mobile modem ( 430).

In addition, the modem controller 431 demodulates the voice signal received from the mobile modem 430 according to a corresponding protocol, and then provides the demodulated signal to the system controller 427. The system controller 427 provides the received voice signal to the speaker 305 to enable a user call using the image communication robot phone 300.

On the other hand, the video communication robot phone 300 according to the present invention to remove the interference signal for the voice signal received from the communication modem 405 and the voice signal provided to the communication modem 405, the howling phenomenon of the call sound Remove This is because the distance between the microphone 303 and the speaker 305 is narrow, based on the shape of the video communication robot phone 300, the audio signal output from the speaker 305 is input to the microphone 303 This is to prevent the phenomenon. In particular, as described above, since the video communication robot phone 300 determines the rotation direction and the moving direction according to the user's voice, when the voice output from the speaker 305 is input to the microphone 303, Difficulties arise.

That is, the video communication robot phone 300 is input to the microphone 303 attached to both sides of the robot phone as a function of providing a stable communication service by directly finding and removing the interference signal generated by the robot phone except for the call sound in both calls. It is possible to accurately receive the phone call, and to determine the volume of the volume of the robot phone only by the original sound from which the sound is removed to determine the direction of rotation and the direction of the robot phone. To this end, the system control unit 403 of the video communication robot phone 300 adds an LMS function, preventing the remote call partner's pronunciation from entering the microphone of the video robot again, and transmitting the original sound of the user. In this way, the interference signal from which the transmission sound is re-introduced into the reception microphone is eliminated (oscillation prevention). That is, an interference cancellation system is provided.

Here, the system controller 427 includes an algorithm and an electronic circuit for implementing the above-described interference cancellation technique (ICS), or a separate LMS (Least Mean Square) module connected to an output port of the system controller 427. This may be implemented in a connected structure. 6 is an LMS configuration diagram illustrating an embodiment of the present invention. As shown, when a user's voice is input, high frequency noise is removed through a low pass filter (LPF), and a delay for a predetermined time input signal is provided by a delay circuit and a regressor buffer.

Then, the synthesized voice signal is synthesized again through the overlapping with the illustrated weight update circuit and then passed through the low pass filter LPF. Then, the sound signal output through the comparator is fed back to the delay circuit to perform a signal multiplication with the fading signal, and then overlaps with the user's call sound input signal, and repeats the above process until it matches the fading signal, thereby making the input / output signal. Eliminate the interference signal for. In the present invention, the LMS is set as an embodiment, but a plurality of types of interference cancellation techniques (ICS) may be applied.

Meanwhile, in the present invention, an acoustic echo canceller (AEC) circuit for canceling an echo signal may be added together with an interference cancellation technique (ICS). This is because the video communication robot phone 300 is used indoors, so as to exclude the echo effect caused by the indoor barrier. As shown in FIG. 7, the AEC circuit prevents a phenomenon in which the acoustic signal output from the speaker 305 is reflected on the barrier and re-enter the microphone 303. The AEC used in the present invention is known. As one of echo cancellation techniques, a detailed description thereof will be omitted.

As described above, the present invention enables the terminal 307 of the video communication robot phone 300 to be replaced, and at the same time, to perform noise removal on the audio signal transmitted and received from the terminal 307, thereby making the video communication robot phone 300. We aim at qualitative improvement for call tone of).

As described above, the mobile video robot terminal according to the present invention can follow the user's position, from which the user can make a phone call without the user's movement, thereby enabling efficient grafting of the home robot and the wireless phone, and the existing home robot. Overcoming the discontinuity of the network of heterogeneous terminals, which is one of the disadvantages of the video communication robot, and the activation of the home robot or service robot market by increasing the completeness of the voice call system, the possibility of the robot industry is high enough. Can be.

1 and 2 are views for explaining the main functions of the conventional service robot phone.

3 is a perspective view showing a video communication robot phone according to the present invention.

4 is a configuration diagram illustrating the main functions of FIG. 3.

5 is a diagram illustrating a terminal detachment function of FIG. 3.

6 is an equivalent circuit for explaining the LMS function applied to the present invention.

7 is a configuration diagram for explaining the AEC function applied to the present invention.

8 is a flowchart illustrating the determination of the direction and movement of the robot phone according to the present invention.

<Explanation of symbols for main drawings>

300: robot phone 301: emotion display lamp

303: microphone 305: speaker

307: terminal 309: display panel

311: communication antenna 401: robot drive unit

403: system control unit 405: communication modem

410: robot controller 413: sensor drive unit

415 sensor 417 battery

419: motor 427: system controller

421: Camera Module 423: USB Interface

425 display module 430 mobile modem

431 Modem Controller 433 SIM Card

Claims (8)

A system controller which maintains a near field communication state with the robot driver, which performs a mechanism control for driving the robot, controls the robot drive according to the surrounding environment, captures and transmits a surrounding image, and transmits and receives a call voice signal. And a communication modem configured to perform a voice call or a video call based on the video and audio signals provided from the system controller. The robot drive unit, a motor for driving the robot; A battery for powering the system; A sensor driver having an object detecting sensor for detecting a peripheral obstacle; And a robot controller including remaining battery level detection and driving control of a motor, detecting a peripheral obstacle based on a signal detected from the sensor driver, and performing a user tracking control based on motion control and user position determination corresponding thereto. Consisting of, The system control unit includes a speaker and a microphone for inputting and outputting a voice signal according to a call connection of the communication modem; A camera module for capturing the surrounding image of the robot phone; A display module for displaying an expression of the robot phone; And determining a user's position based on call control for performing a voice call of the user, image control for transmitting an image captured by the camera, ambient sound signals input from the microphone and the object detection sensor, and distance detection signal with the user. System controller for generating a user following control signal for determining the direction and movement of the robot phone from this, The communication modem may include a mobile modem having a communication module capable of a voice call or a video call; And a modem controller providing an operation algorithm of the communication module held by the mobile modem. Generation of the user following control signal in the system controller, Determining the rotation direction of the robot phone by detecting the maximum value of the user's sound level, if the rotation direction of the robot phone is determined, the distance between the robot phone and the user is calculated using a sensor mounted on the robot phone, Characterized in that it is generated by determining whether the calculated distance is within a preset range, And the communication modem has a detachable structure with the robot so that the communication modem can be replaced based on the terminal type according to the communication target. The method of claim 1, The communication module is a mobile video robot terminal, characterized in that any one of CDMA-2000, WCDMA, GSM, WiMAX, LTE. The method of claim 1, The modem controller includes a SIM card or a universal subscriber identity module (USIM), and determines the approval of the use of the mobile modem according to the authentication result of the SIM card or the SIM card. terminal. delete The method of claim 1, The system controller has a connector for electrical connection with the communication modem, or performs a near field communication of any one of Zigbee, Bluetooth, WIFI, UWB, infrared communication. The method of claim 1, The system controller has a USB interface for communicating with an external terminal, and the display module is a mobile image robot terminal, characterized in that consisting of LED or LCD. The method according to any one of claims 1, 5 and 6, The system controller is a mobile image robot terminal, characterized in that it comprises a Least Mean Square (LMS) module for interference cancellation on the call signal. The method of claim 7, wherein The system controller further comprises an acoustic echo canceller (AEC) module for canceling echo to an audio input / output terminal.

Images