JP2005246503A - Robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide robot capable of displaying an operation menu in a display region of a portable communication terminal for performing remote operation. <P>SOLUTION: The robot is provided with a imaging part 10 for imaging; a communication control part 4 making talking connection with the portable communication terminal 3; and an image composing part 13 for composing an image by pasting the command menu for the remote operation of the robot to image information imaged by the imaging part 8, during talking connection of the communication control part 4 with the portable communication terminal 3. Image information composed by the image composing part 13 is transmitted to the portable communication terminal 3 through the communication control part 4 and displayed in a first display region 3a of the portable communication terminal 3. The command menu is acquired from a command menu file storing a series of command menus. A DTMF (dual tone multi-frequency) signal generated by pressing a push button 3c of the portable communication terminal 3 is converted into a command by a DTMF signal analyzing part 7b and inputted to a timing control part 5a, and the timing control part 5a instructs a pasted information temporary holding part 12 to change the command menu. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a robot, and transmits information for remote operation simultaneously with image information captured by the robot to a mobile communication terminal for inputting remote operation, and displays the information for remote operation on the display unit of the mobile communication terminal. The present invention relates to a robot capable of displaying the information of the robot.

  As shown in Patent Documents 1 to 3, various types of robot systems for operating a mobile robot are considered. For example, a remote control software that is configured so that a control program for operating each joint of the robot is stored in the robot while a command can be transmitted to the control program via a wireless LAN is a CD (Compact Disc). The system prepared in is considered. In this system, a wireless LAN card is attached to a personal computer, and software for remote operation is loaded and started, and an operator inputs a command using a control panel displayed on the display screen of the personal computer. .

JP 2000-79587 A JP 2002-354139 A JP 2003-136455 A

However, in such a system, an operator who operates the robot is given software-related information by executing software for remote operation. Therefore, since the remote communication software is installed in the mobile communication terminal, the memory capacity increases. In addition, since a large load is applied to the control in the mobile communication terminal, the mobile communication terminal becomes large and the battery of the mobile communication terminal is exhausted. Therefore, the portability of the mobile communication terminal is impaired.
On the other hand, if the operation information peculiar to the robot is not stored in the mobile communication terminal and the memory of the operator is relied on, the number of commands to the robot inevitably decreases, and various commands cannot be given to the robot.

  In view of the above problems, an object of the present invention is to provide a robot that can display operation information on a screen of a mobile communication terminal that remotely controls the robot.

In order to solve the above-described problem, the present invention provides a robot that can be remotely operated with a mobile communication terminal, a communication control unit that performs a call connection with the mobile communication terminal, an imaging unit that performs imaging, and a communication control unit that An image compositing unit for compositing an image by pasting a command menu for remote control of the robot to image information captured by the image capturing unit during a call connection;
With this configuration, a remote operation command menu can be pasted on the image information captured by the image capturing unit, and the image can be synthesized and transmitted to the mobile communication terminal. The command menu can be displayed. Therefore, a command menu can be presented to a person who remotely operates the robot.

  In addition to the above configuration, preferably, a command menu file for storing a series of command menus to be pasted by the image composition unit, and paste information for temporarily holding a command menu to be pasted from the command menu file by the image composition unit in a switchable manner A temporary holding unit. With this configuration, since a series of command menus is stored in advance as a command menu file, it can be immediately pasted on the image information in the image composition unit.

  Preferably, the series of command menus displays an imaging unit control menu in which commands for the imaging unit are displayed corresponding to the buttons of the mobile communication terminal, and commands for the drive mechanism are displayed corresponding to the buttons of the mobile communication terminal. And a conversation communication menu in which commands related to voice output are displayed corresponding to the buttons of the mobile communication terminal. With this configuration, each command menu corresponding to the button of the mobile communication terminal can be displayed on the display unit of the mobile communication terminal, and the convenience of the operator is improved.

  In addition to the above-described configuration, the present invention preferably includes a suspicious object detection processing unit that performs suspicious object detection, and a suspicious object detection information received from the suspicious object detection processing unit, and makes a communication connection with the mobile communication terminal to the communication control unit A control unit for instructing to paste suspicious object detection information on the image information captured by the imaging unit when the communication control unit and the mobile communication terminal are in a call connection by this instruction, Is provided. With this configuration, the robot can detect a suspicious object, perform a call connection to a specific mobile communication terminal, and notify a person who has the mobile communication terminal that a suspicious object has been detected.

  The suspicious object detection processing unit preferably processes the image information captured by the imaging unit to perform suspicious object determination, and when determining that there is a suspicious object, notifies the control unit of suspicious object detection. A processing unit is provided. With this configuration, the suspicious object can be detected based on the image information obtained by the imaging unit of the robot, and when the suspicious object is detected, the mobile communication terminal can be notified of this.

  The suspicious object detection processing unit preferably processes the audio information input and converted by the sound input unit to perform suspicious object determination, and notifies the control unit of suspicious object detection when it is determined that there is a suspicious object. An image processing unit is provided. With this configuration, the suspicious object can be detected based on the voice information obtained by the sound input unit of the robot, and when the suspicious object is detected, the mobile communication terminal can be notified of this.

  The robot according to the present invention includes a communication control unit that establishes a call connection with a mobile communication terminal, an imaging unit that performs image capture, and a robot remote operation on image information captured by the image capture unit while the communication control unit is connected to the mobile communication terminal via a call. And an image composition unit for pasting a command menu for image composition. Therefore, the image composition unit pastes a command menu for remote operation on the image information captured by the image capturing unit of the robot and performs image composition. By transmitting the composite image information to the mobile communication terminal, A command menu for remote operation can be displayed on the display unit of the communication terminal. Therefore, a command menu can be presented to a person who remotely operates the robot, and the robot can be easily operated remotely.

  Also, the robot stores a series of command menus pasted by the image composition unit as a command menu file, and temporarily stores the command menu pasted by the image composition unit from the command menu file in the pasting information temporary storage unit. Let Therefore, the image composition unit can easily obtain information to be pasted from the pasting information temporary holding unit, and can be pasted immediately on the image information.

  The series of command menus includes command menus that display commands for the imaging unit, commands for the drive mechanism, and commands for audio output in correspondence with buttons of the mobile communication terminal. Therefore, it is possible for a person who operates the mobile communication terminal to easily understand which button corresponds to what command.

  In addition, the robot instructs the communication control unit to establish a call connection with the mobile communication terminal upon receiving the suspicious object detection processing unit that performs suspicious object detection and the suspicious object detection information from the suspicious object detection processing unit. I do. The mobile communication terminal that has started a call connection with the robot is also provided with a control unit that instructs the image composition unit to paste the suspicious object detection information on the image information captured by the imaging unit. Therefore, the robot detects the suspicious object, makes a call connection to the specific mobile communication terminal, and notifies the person who has the mobile communication terminal that the suspicious object is detected.

  The suspicious object detection processing unit processes image information captured by the image capturing unit to perform suspicious object determination, and when it is determined that there is a suspicious object, image processing for notifying the control unit of suspicious object detection. Therefore, the suspicious object can be detected based on the image information obtained by the imaging unit of the robot. Therefore, when a suspicious object is detected, it can be notified to the mobile communication terminal.

  The suspicious object detection processing unit processes the audio information input and converted by the sound input unit to perform suspicious object determination, and when it is determined that there is a suspicious object, notifies the control unit of suspicious object detection. Since the image processing unit is provided, the suspicious object can be detected based on the voice information obtained by the sound input unit of the robot. Therefore, when a suspicious object is detected, it can be notified to the mobile communication terminal.

The best mode for carrying out the present invention will be described below with reference to the drawings.
(Embodiment 1)
FIG. 1 is a functional block configuration diagram of a robot remote control system according to Embodiment 1 of the present invention. As shown in FIG. 1, a robot remote operation system 1 includes a robot 2 that is remotely operated, and a mobile communication terminal 3 to which a person who remotely operates the robot 2 (hereinafter referred to as “operator”) inputs a command. The robot 2 and the mobile communication terminal 3 can be connected by a telephone line including a wireless telephone line, for example.

  The robot 2 controls each drive mechanism according to an operation command from the outside and performs a predetermined movement. As the robot 2, for example, a desktop robot that is placed on a table or the like and operates by operating a drive mechanism provided at each joint part of the upper body, a drive provided at each joint part such as a shoulder or a leg. Animal-type robots that walk by actuating mechanisms, humanoid robots such as biped walking robots that actuate drive mechanisms provided at each joint such as shoulders and legs, and wheels and caterpillars by drive mechanisms There are mobile robots that can be moved by In the following description, as shown in FIG. 1, a robot having an actuator constituted by a motor and a potentiometer that measures a joint state as a drive mechanism is taken as an example in each joint portion 21.

  As shown in FIG. 1, the mobile communication terminal 3 transmits a first display area 3 a that displays image information transmitted from the counterpart mobile communication terminal 3 during communication, and transmits to the counterpart mobile communication terminal 3 during communication. In addition to the display section composed of the second display area 3b for displaying the displayed image information, a series of push buttons 3c of “0” to “9”, “*” and “#”, and an image to be transmitted to the other party An image pickup unit for picking up images, a push button necessary for causing the image pickup unit to function, and the like are provided. When the push button 3c is pressed, a signal conversion unit for converting into a DTMF (dual tone multi-frequency) signal is built in in order to establish a call session to the mobile communication terminal 3 of the other party.

The mobile communication terminal 3 is a terminal that can process an image obtained by an image sensor such as a CCD (Charge Couple Device) camera equipped in one mobile communication terminal 3 and transmit the processed image to the other mobile communication terminal 3. An example is a mobile phone. In all the following embodiments, a mobile videophone terminal that can transmit and receive a moving picture is described. However, a terminal that can transmit and receive a still image may be used.
In the following description, in a state where the mobile communication terminal 3 communicates with the robot 2, the image information captured by the image sensor of the mobile communication terminal 3, the voice information input by voice, and the push button 3 c being pressed down. The DTMF signal converted by the above can be compressed as one information and transmitted as compressed information, and the compressed information transmitted from another similar portable communication terminal 3 is decompressed to display the image information in the first display area 3a. It is assumed that the voice signal and the DTMF signal can be output by the voice output unit built in the mobile communication terminal 3. An example of the compression processing method is an MPEG (moving picture experts group) method.

  FIG. 2 is a functional block configuration diagram of the robot 2. The robot 2 includes a communication control unit 4 for communicating with the mobile communication terminal 3, and the communication control unit 4 communicates with the other party, for example, wirelessly via a relay station so as to be in a call state with the other party. Communication processing unit 4a that performs communication processing such as call processing, response processing to call signal and disconnection processing, and transmission / reception unit 4b that transmits / receives information to / from the other party through communication processing unit 4a in a call state . An example of the communication processing unit 4a is a communication card. In the robot 2 shown in FIG. 1, the communication card is built in the back. In addition, although not shown in figure, the transmission / reception part 4b is compressed by the receiving part which receives the compression information transmitted from the other party via the communication processing part 4a in the call state, and the other party via the communication processing part 4a in the call state. It is comprised with the transmission part which transmits information.

Moreover, the robot 2 includes a control unit 5 that performs overall control of the operation and sensors of the robot 2 as will be described later. A reception information processing unit 6 and a command analysis unit 7 are connected in series between the communication control unit 4 and the control unit 5.
The reception information processing unit 6 performs decompression processing corresponding to the compression method in the mobile communication terminal 3 on the compressed information received from the mobile communication terminal 3 and separates it into image information, audio information, and DTMF signal. .

The command analysis unit 7 sends the command information transmitted from the mobile communication terminal 3 as an audio signal, that is, the voice as a result of decompressing the compressed information when the mobile communication terminal 3 performs compression processing and transmits the compressed information. The signal is analyzed as command information.
The command analysis unit 7 is configured by connecting a speech recognition analysis unit 7a that recognizes and analyzes a speech signal and a DTMF signal analysis unit 7b that analyzes a DTFM signal in parallel. The voice recognition analysis unit 7a stores a first command correspondence table in which the recognition results in the voice recognition analysis unit 7a are associated with the commands to the control unit 5 corresponding to the respective recognition results. The DTMF signal analyzing unit 7b stores a second command correspondence table in which each DTMF signal is associated with a command to the control unit 5 for the DTMF signal. The first command correspondence table and the second command correspondence table may be stored in the command analysis unit 7 so that the speech recognition analysis unit 7a and the DTMF signal analysis unit 7b can refer to them. Thereby, both remote operation by voice input and remote operation by pressing the push button 3c of the mobile communication terminal 3 can be supported.

FIG. 3 is an example of the first command correspondence table T1. The speech recognition analysis unit 7a recognizes the input speech signal, compares the recognition result with the first command correspondence table T1, and outputs a command that matches the recognition result. In the first command correspondence table T1, as shown in FIG. 3, one command corresponds to a plurality of recognition results. For example, “zenshin”, “previous”, “straight” The recognition result corresponds to “Walk Forward”, which is convenient for the operator.
On the other hand, the second command correspondence table is a table in which the recognition result in the first command correspondence table T1 corresponds to the DTMF signal.

The control unit 5 includes a timing control unit 5a that receives an input of a command from the command analysis unit 7, and a transmission number confirmation processing unit 5b. Although implemented in one MPU (microprocessor unit) together with an operation control unit 5c, an utterance sentence generation unit 5d, an imaging control unit 5e, and an illumination control unit 5f as described later, it is implemented in a plurality of MPUs. May be realized.
The calling number confirmation processing unit 5b stores in advance a list of telephone numbers of the mobile communication terminal 3 that is permitted to make a call connection with the robot 2.
The communication processing unit 4a receives the call connection request from the mobile communication terminal 3 of the operator as a trigger, and the mobile number notified by the caller number notification from the mobile communication terminal 3 to the call number confirmation processing unit 5b. A request for determination as to whether or not to permit call connection is made together with the mobile number of the communication terminal 3.
Upon receiving this determination request, the calling number confirmation processing unit 5b determines whether or not the mobile number of the mobile communication terminal 3 is in the telephone number list, and returns the determination result to the communication processing unit 4a.
At this time, only when the determination result is that there is a number, the timing control unit 5a is notified that the telephone communication with the mobile communication terminal 3 is in progress. Further, when the communication processing unit 4a performs a call disconnection process by the mobile communication terminal 3 of the other party, the call disconnection confirmation processing unit 5b is notified of the call disconnection. The call number confirmation processing unit 5b that has received the call disconnect notification notifies the timing control unit 5a that the call has been disconnected.

The timing control unit 5a takes a timing corresponding to the command input from the command analysis unit 7 when the mobile communication terminal 3 is in a call connection, that is, based on a predetermined priority order related to control. Control the controlled part.
In the robot 2 shown in FIG. 2, as a controlled unit, from an actuator 8 as a drive mechanism provided in each joint unit 21 of the robot 2, or from an audio output unit 9 b such as a speaker arranged at the mouth of the robot 2. There are a voice synthesizer 9a for synthesizing a voice to be emitted, an imaging unit 10 such as a CCD camera attached to one eye, and an illumination unit 11 such as an LED attached to the other eye.

  In order to control each of these controlled parts, the control part 5 includes an action control part 5c composed of a walking pattern generation part 5c1 and each motor control part 5c2, and a speech sentence generated in the speech synthesis part 9a. An utterance sentence generation unit 5d that outputs, an imaging control unit 5e that controls the imaging unit 10, and an illumination control unit 5f that controls the illumination unit 11 are provided. The operation control unit 5c, the utterance sentence generation unit 5d, the imaging control unit 5e, and the illumination control unit 5f that control these controlled units are provided in the control unit 5, but may be provided separately from the control unit 5. Good.

  The timing control unit 5a of the control unit 5 determines the command input from the command input unit 7, and the input command is used by the operation control unit 5c, the utterance sentence generation unit 5d, the imaging control unit 5e, and the illumination control unit 5f. Determine which command is to be directed. Then, according to the priority order table stored in the timing control unit 5a, the controlled unit corresponding to the input command, that is, the operation control unit 5c, the utterance text generation unit 5d, An instruction corresponding to the command is transmitted to either the imaging control unit 5e or the illumination control unit 5f. Here, for the sake of explanation, it is assumed that the command input from the command analysis unit 7 also serves as a command for each controlled unit.

When the timing control unit 5a determines that the input command relates to the motion control unit 5c, the input command is input to the pedestrian pattern generation unit 5c1.
The pedestrian pattern generation unit 5c1 generates a corresponding walking pattern with reference to the walking pattern file 5c3 stored in advance according to the input command. The walking pattern file 5c3 is a file that generalizes which joint portion 21 the actuator 8 is to be operated in the operation of the robot 2, for example, a general operation (forward, leftward, rightward, etc.). In addition, the walking pattern is based on the result of the current joint state from the potentiometer built in each actuator 8, and the rotational speed and operating time of the motor with respect to the motor control unit 5c2 provided corresponding to each joint. It is calculated quantitatively.
The walking pattern generated by the pedestrian pattern generation unit 5c1 is input to each motor control unit 5c2, and each motor control unit 5c2 drives a motor built in the corresponding actuator 8 and is input to the timing control unit 5a. The robot 2 performs an operation corresponding to the command.

  If the timing control unit 5a determines that the input command relates to the utterance sentence generation unit 5d, the timing control unit 5a inputs the input command to the utterance sentence generation unit 5d. In response to the input, the utterance sentence generation unit 5d generates an utterance sentence from a language file stored in advance in response to the input command, and outputs the utterance sentence to the speech synthesis unit 9a. The voice synthesizer 9a that has received the input of the generated utterance text converts the input utterance text to be output as a sound wave by the voice output section 9b, and is output as a corresponding voice from the voice output section 9b.

  When the timing control unit 5a determines that the input command relates to the imaging control unit 5e, the timing control unit 5a inputs the input command to the imaging control unit 5e. Upon receiving this input, the imaging control unit 5e controls the imaging unit 10 so as to correspond to the input command. Thereby, zoom switching is performed, and the imaging unit 10 turns right or turns up.

  When the timing control unit 5a determines that the input command relates to the illumination control unit 5f, the timing control unit 5a inputs the input command to the illumination control unit 5f. Receiving this input, the illumination control unit 5f controls the illumination unit 11 so as to correspond to the input command. Thereby, illumination is turned on and off.

When the timing control unit 5a determines that the input command relates to the pasting information temporary holding unit 12, the timing control unit 5a inputs the input command to the pasting information temporary holding unit 10.
Here, the pasting information temporary holding unit 12 temporarily holds pasting information to be pasted on the image information picked up by the image pickup unit 10 in the imaging composition unit 13. The pasting information temporary holding unit 12 switches and temporarily holds a series of command menus stored in advance in the command menu file 14 by a command input to the timing control unit 5a. That is, the pasting information temporary holding unit 12 temporarily holds pasting information to be pasted on the image information captured by the imaging unit 10 from the command menu file 14. Therefore, when the pasting information temporary holding unit 12 receives an instruction to switch the pasting information from the timing control unit 5a of the control unit 5, the pasting information to be pasted from the command menu file 14 is acquired and held.

FIG. 4 shows image information showing an example of the command menu file 14. The series of command menus shown in FIG. 4 displays a list of the push buttons 3c and commands corresponding to the push buttons 3c.
As shown in FIG. 4, examples of the command menu include an imaging unit control menu M1 related to the imaging control unit 5e, a robot operation menu M2 related to the operation of the robot 2, and a conversation communication menu M3 that allows the robot 2 to talk. is there. Here, the imaging unit control menu M1 is a command menu that displays an imaging direction change and zoom switching command for the imaging unit 10 in association with the push button 3c of the mobile communication terminal 3. The robot operation menu M <b> 2 is a command menu that displays a robot operation command corresponding to the push button 3 c of the mobile communication terminal 3. The conversation communication menu M3 is a command menu in which a command related to voice output is associated with the push button 3c of the mobile communication terminal 3.

  The pasting information temporary holding unit 15 can switch each command menu stored in the command menu file 12 according to an instruction from the timing control unit 5a. FIG. 4 also shows how the command menu can be switched by the “*” or “0” push button 3 c of the mobile communication terminal 3. For example, when there is an input of “*” of the push button 3c from the initial menu M0, the robot operation menu M2 is switched by an instruction from the timing control unit 5a.

  Then, the image composition unit 13 performs image composition by pasting the paste information temporarily held in the paste information temporary holding unit 12 to the image information captured by the imaging unit 10. The pasting information temporary holding unit 12 and the image synthesizing unit 13 perform image synthesis by pasting pasting information on the image information captured by the imaging unit 10, and the transmission information processing unit 16 further uses a microphone or the like. Along with the voice information input from the sound input unit 15, the information is compressed as a single information in a predetermined format and transmitted to the mobile communication terminal 3 via the communication control unit 4.

As described above, by pressing the “*” or “0” of the push button 3c predetermined in the mobile communication terminal 3, for example, the push button 3c in the series of command menus shown in FIG. It is generated and compressed together with image information and audio information, and is sent to the robot 2 as call information.
Then, the reception information processing unit 6 that has received the call information via the communication control unit 4 decompresses the call information. The DTMF signal generated as a result of the decompression is converted into a corresponding command by the DTMF signal analysis unit 7b of the command analysis unit 7 and input to the timing control unit 5a.
Receiving this input, the timing control unit 5a determines that the input command is a command for switching the paste information, and instructs the paste information temporary holding unit 12 to paste the paste information temporarily held according to the command. Instruct to switch based on file 12.
As a result, a series of command menus are switched in response to the press of the push button 3c by the operator, and the image information is picked up by the image combining unit 13 on the image information picked up by the image pickup unit 10, and the sound input such as a microphone is input. The data is compressed together with the audio information from the unit 15 and transmitted to the mobile communication terminal 3 via the communication control unit 4.
Therefore, the mobile communication terminal 3 can decompress the received compressed information, display the image information with the instruction menu pasted on the first display area 3a, and present a series of instruction menus to the operator.

(Embodiment 2)
Next, a robot remote control system that improves the first embodiment and performs automatic transmission to a predetermined mobile communication terminal 3 when a suspicious object is found will be described.
FIG. 5 is a functional block configuration diagram of the robot 20 according to the second embodiment of the present invention. The mobile communication terminal 3 for remotely operating the robot is the same as that of the first embodiment, and the same or equivalent functional blocks of the robot 2 are denoted by the same reference numerals.

The robot 20 includes a suspicious object detection processing unit 17 that performs a suspicious object determination process. The suspicious object detection processing unit 17 includes a sound processing unit 17a that receives sound information from the sound input unit 15 such as a microphone and detects a suspicious sound.
When the sound processing unit 17a repeatedly performs a predetermined determination routine and detects a suspicious sound, the sound processing unit 17a notifies the timing control unit 5a of the control unit 5 of suspicious object detection information. Here, the predetermined determination routine determines, for example, whether or not a specific sound such as a sound with a certain intensity or a specific frequency is included in the sound information from the sound input unit 15 at an arbitrary time interval. If a specific sound includes audio information input from the sound input unit 15, it is determined that there is a suspicious sound.
The suspicious object detection processing unit 17 is provided with an image processing unit 17b that receives image information from the imaging unit 10 and determines a suspicious object. When the image processing unit 17b repeatedly performs a predetermined determination routine and detects a suspicious object, the image processing unit 17b notifies the timing control unit 5a of the control unit 5 of suspicious object detection information. Here, the predetermined determination routine determines whether or not there is a moving object in the image information, that is, whether or not there is a moving object in the captured image, and there is a moving object. If it is determined that there is a suspicious object.
The suspicious object detection processing unit 17 includes one or more sensors 17c. When the sensor 17c detects an abnormality, the suspicious object detection processing unit 17 notifies the timing control unit 5a of the control unit 5 of suspicious object detection information. Here, as the sensor 17c, there are an infrared human detection sensor, a temperature sensor, a gas sensor, and the like, and examples of abnormalities include human detection, temperature rise above a certain level, gas leakage, and the like. Although not shown, the robot 20 is provided with an infrared communication unit, and the infrared communication unit notifies that a predetermined abnormality is detected from a door open / close check sensor attached to the door or a window open / close check sensor attached to the window. The notification may be input to the timing control unit 5a.

  In addition, the control unit 5 includes a telephone call processing unit 5g. The timing control unit 5a that has received the notification of the suspicious object detection information from the suspicious object detection processing unit 17 instructs the telephone call processing unit 5g. In response to this instruction, the telephone call processing unit 5g instructs the communication processing unit 4a of the communication control unit 4 to automatically establish a call connection with the specific mobile communication terminal 3. In the telephone call processing unit 5g or the communication processing unit 4a, a mobile number of a specific mobile communication terminal 3 is stored in advance.

  When the robot 20 is connected to a specific mobile communication terminal 3 by calling, the timing control unit 5a acquires the transparent plain initial menu M0 from the command menu file 12 to the pasting information temporary holding unit 10, It is instructed to paste and hold suspicious object detection information such as text information “existence of suspicious object” at a predetermined position of the menu M0. Here, the suspicious object detection information is stored in advance in the suspicious object detection information file 18 so that the suspicious object detection information can be output to the pasting information temporary holding unit 12, for example. The text information pasted at a predetermined position of the initial menu M0 may be transferred from the suspicious object detection processing unit 15 attached to the suspicious object detection notification to the timing control unit 5a.

  As a result, the image composition unit 13 uses the image information from the imaging unit 10 to display the pasting information held by the pasting information temporary holding unit 12, for example, the image information of the initial menu M0 pasted with the text information of “suspicious”. Paste it into information and compose an image. And it compresses according to a predetermined | prescribed system with the audio | voice information from the sound input part 15, and transmits to the specific portable communication terminal 3 via the communication control part 4. FIG. Therefore, for example, “suspicious” is displayed in the first display area 3 a of the mobile communication terminal 3. In this way, a person possessing the mobile communication terminal 3 can receive suspicious object information by receiving automatic transmission from the robot 20 when suspicious object detection is performed in the suspicious object detection processing unit 15.

(Robot remote control method)
Hereinafter, the remote operation method of the robot 20 will be described with reference to FIG.
Note that the robot 20 is disposed at home, for example, and the power supply not shown in FIG. 5 is ON, and the control unit 5, the imaging unit 10, the sound input unit 15, and the suspicious object detection processing unit 17 are in an operating state. It shall be in

[Establish call status]
In order to remotely control the robot 20 with the mobile communication terminal 3, first, the mobile communication terminal 3 and the robot 20 are brought into a call state.
That is, the mobile communication terminal 3 calls the communication control unit 4 of the robot 20, specifically, the communication card as the communication processing unit 4 a of the communication control unit 4. Then, the caller number of the mobile communication terminal 3 that is the caller is notified from the communication processor 4a to the caller ID confirmation processor 5b of the controller 5. Upon receiving this notification, the calling number confirmation processing unit 5b confirms whether or not the call connection is permitted. If the call is permitted, the communication processing unit 4a is permitted to connect the call. In this case, a reply indicating that the call connection is not permitted is made. The communication processing unit 4a performs a call connection process when receiving a call connection permission response, and performs a disconnection process when receiving a call connection disapproval response. In the following, it is assumed that call connection processing has been performed.

[Initial state of calling]
Next, when the mobile communication terminal 3 and the robot 2 are in a call connection state, the calling number confirmation processing unit 5b notifies the timing control unit 5a that the call is in progress. Then, the timing control unit 5a instructs the pasting information temporary holding unit 12 to select and temporarily hold the transparent plain initial menu M0 (see FIG. 4) from the command menu file 14. Upon receiving this instruction, the pasting information temporary holding unit 12 acquires and holds the initial menu M0 from the command menu file 14 and outputs it to the image composition unit 13. The image composition unit 13 composes an image by pasting the initial menu M0 output from the pasting information partial holding unit 12 to the image information captured by the image capturing unit 10, and transmits the combined image information as a transmission information processing unit. 16 is output. Since the initial menu M0 is transparent and plain, image information captured by the imaging unit 10 is substantially output to the transmission information processing unit 16. Then, in the transmission information processing unit 16, the combined image information from the image combining unit 13 and the audio information input from the sound input unit 15 are compressed together, and the other party's mobile communication terminal is connected via the communication control unit 4. 3 is transmitted. Thus, the synthesized image information is displayed in the first display area 3a of the mobile communication terminal 3, and the audio information is output as a sound wave from the audio output unit of the mobile communication terminal 3.

[Display robot operation menu]
Next, in order to remotely operate the robot 20 from the mobile communication terminal 3, the robot operation menu M2 (see FIG. 4) is displayed in the first display area 3a of the mobile communication terminal 3.
That is, “*” of the push button 3 c of the mobile communication terminal 3 is pressed. Then, a DTMF signal corresponding to this is generated and transmitted to the robot 20. At this time, regardless of the presence or absence of image information captured by the mobile communication terminal 3 and audio input, the image information and audio information are compressed together and transmitted as compressed information.

  In the robot 20, the compression information is input to the reception information processing unit 6 via the communication control unit 4 and decompression processing is performed. Then, the DTMF signal is input to the command analysis unit 7. In the command analysis unit 7, the DTMF signal is input to the speech recognition analysis unit 7a and the DTMF signal analysis unit 7b. The speech recognition / analysis unit 7a outputs nothing to the input of the DTMF signal, but the DTMF signal analysis unit 7b converts the input of the DTMF signal into a corresponding command by referring to the second command correspondence table. Is done. The converted command is input to the timing control unit 5a.

  The timing control unit 5a recognizes the input command and instructs the pasting information temporary holding unit 12 to select and hold the robot operation menu M2 from the command menu file 14. Upon receiving this instruction, the pasting information temporary holding unit 12 selects and holds the robot operation menu M <b> 2 and outputs it to the image composition unit 13. Thereafter, in the same manner as described above, the image composition unit 13 performs image composition by pasting the robot operation menu M2 on the image information from the imaging unit 10, and outputs the synthesized image information to the transmission information processing unit 16. The transmission information processing unit 16 compresses the combined image information and audio information together and transmits the compressed information to the mobile communication terminal 3 via the communication control unit 4.

As a result, the mobile communication terminal 3 performs the decompression process and displays the image information with the robot operation menu M2 pasted on the first display area 3a, so that the operator can view the robot displayed on the first display area 3a. A command to the robot 20 can be input while looking at the operation menu M2.
If the robot operation menu M2 displayed in the first display area 3a of the mobile communication terminal 3c is difficult to see, the timing control unit can be operated by pressing the “0” push button 3c of the mobile communication terminal 3. Since the switching instruction is made from 5a to temporarily hold the initial menu M0 to the pasting information temporary holding unit 12, only the image information captured by the imaging unit 10 can be displayed in the first display area 3a of the mobile communication terminal 3. .

[Robot operation command (command by DTMF signal)]
Next, a case where a robot operation command is performed using a DTMF signal will be described. As an example, it is assumed that a command for moving the robot forward is given.
The operator refers to the operation menu M2 displayed in the first display area 3a of the mobile communication terminal 3 and presses the “2” push button 3c as a command for moving the robot 20 forward. Then, a DTMF signal corresponding to this is generated and transmitted to the robot 20. At this time, regardless of the presence or absence of image information or audio input captured by the mobile communication terminal 3, the image information and audio information are compressed together and transmitted as compressed information.
Further, the portable communication terminal 3 indicates that the instruction “forward” is issued to the robot 20 in the second display area 3b by pressing the “2” push button 3c. Can be displayed, so that it is possible to confirm what command the operator has given to the robot 20.

  In the robot 20, the compression information is input to the reception information processing unit 6 via the communication control unit 4 and decompression processing is performed. Then, the DTMF signal is input to the command analysis unit 7. In the command analysis unit 7, the DTMF signal is input to the speech recognition analysis unit 7a and the DTMF signal analysis unit 7b. The speech recognition analysis unit 7a does not output anything in response to the input of the DTMF signal, but the DTMF signal analysis unit 7b converts the command into a command corresponding to the input DTMF signal with reference to the second command correspondence table. The The converted command is input to the timing control unit 5a.

  The timing control unit 5a recognizes the input command and instructs the walking pattern generation unit 5c1 to generate a “forward” walking pattern. Then, the walking pattern generation unit 5c1 generates a “forward” walking pattern with reference to the walking pattern file 5c3, and a control signal is input to each motor control unit 5c2. Thereby, each motor control part 5c2 controls the motor of each actuator 8 according to the input control signal, and the robot 20 moves forward.

  Note that, for the robot operation commands other than “forward”, for example, “stop” and “right turn”, the operator presses the corresponding push button 3c of the mobile communication terminal 3 with reference to the robot operation menu M2. Similarly, the robot 20 can be remotely operated. At this time, the command given by the operator to the second display area 3b of the mobile communication terminal 3 can be displayed with characters, an illustrator, or the like.

[Robot operation command (command by voice input)]
Next, a case where a robot operation command is performed by voice input will be described. As an example, it is assumed that a command for moving the robot forward is given.
The operator inputs a voice “forward” to the mobile communication terminal 3. Then, an audio signal corresponding to the input is generated and transmitted to the robot 20. At this time, regardless of the presence or absence of image information captured by the mobile communication terminal 3, it is compressed together with the image information and transmitted as compressed information.

  In the robot 20, the compression information is input to the reception information processing unit 6 via the communication control unit 4 and decompression processing is performed. Then, an audio signal corresponding to “forward” is input to the command analysis unit 7. The input voice signal is input to the voice recognition analysis unit 7a and the DTMF signal analysis unit 7b. Then, the voice recognition analysis unit 7a recognizes the voice signal and inputs a command that matches the recognition result with reference to the first command correspondence table T1 to the timing control unit 5a. Note that the DTMF signal analysis unit 7b outputs nothing in response to the input of the audio signal.

  The timing control unit 5a recognizes the input command and instructs the walking pattern generation unit 5c1 to generate a “forward” walking pattern. Then, the walking pattern generation unit 5c1 generates a “forward” walking pattern with reference to the walking pattern file 5c3, and a control signal is input to each motor control unit 5c2. Thereby, each motor control part 5c2 controls the motor of each actuator 8 according to the input control signal, and the robot 20 moves forward. Note that robot operation commands other than “forward”, for example, “stop” and “right turn” can be performed in the same manner.

[Voice output command to robot]
Next, a case where the robot 20 is instructed to output a specific sound will be described. As an example, it is assumed that the robot 20 introduces itself.
The operator inputs a voice “self-introduction” to the mobile communication terminal 3. Then, an audio signal corresponding to the input is generated and transmitted to the robot 20. At this time, regardless of the presence or absence of image information captured by the mobile communication terminal 3, it is compressed together with the image information and transmitted as compressed information.

  In the robot 20, the compression information is input to the reception information processing unit 6 via the communication control unit 4 and decompression processing is performed. Then, an audio signal corresponding to “self-introduction” is input to the command analysis unit 7. In the command analysis unit 7, this voice signal is input to the voice recognition analysis unit 7a and the DTMF signal analysis unit 7b. The voice recognition analysis unit 7a inputs a command that recognizes the voice signal and matches the recognition result with reference to the first command correspondence table T1 to the timing control unit 5a.

  The timing control unit 5a recognizes the input command and instructs the utterance sentence generation unit 5d to generate a “self-introduction” sentence. Then, the utterance sentence generator 5d generates a self-introduction sentence with reference to the “self-introduction” language file, and outputs it to the speech synthesizer 9a. Thus, the voice synthesizer 9a controls the voice output unit 9b to output the self-introduction sentence as a sound wave. Therefore, the robot 20 introduces itself as if it were a self-introduction.

In addition, when transmitting a command to perform self-introduction with the push button 3c of the mobile communication terminal 3, it is as follows.
First, the command menu displayed in the first display area 3a of the mobile communication terminal 3 is changed to the conversation communication menu M3. That is, when the robot operation menu M2 is displayed in the first display area 3a, as shown in FIG. 4, by pressing the push button 3c of “*”, the above-described display of the robot operation menu has been described. Similarly to the above, the timing control unit 5a instructs the pasting information temporary holding unit 12 to switch to and hold the next command menu, that is, the conversation communication menu M3. As a result, the command menu displayed in the first display area 3a is switched to the conversation communication menu M3.

  Next, the operator confirms from the conversation communication menu M3 displayed in the first display area 3a that “self introduction” corresponds to the push button 3c of “2”, and the push button 3c of “2”. Press. Thus, a “2” DTMF signal is generated by the mobile communication terminal 3 and transmitted to the robot 20. At this time, regardless of the presence or absence of image information captured by the mobile communication terminal 3, it is compressed together with the image information and transmitted as compressed information.

  In the robot 20, the compression information is input to the reception information processing unit 6 via the communication control unit 4 and decompression processing is performed. Then, the DTMF signal “2” is input to the command analysis unit 7. In the command analysis unit 7, the DTMF signal is input to the speech recognition analysis unit 7a and the DTMF signal analysis unit 7b. Then, the DTMF signal analysis unit 7b refers to the second command correspondence table and inputs a corresponding command to the timing control unit 5a. Thereafter, through the same processing as when speech is input, the speech synthesizer 9a controls the speech output unit 9b to output a self-introduction sentence as a sound wave. Therefore, the robot 20 introduces itself as if it were a self-introduction.

[Command for imaging unit]
Next, a case where a command is given to the imaging unit 10 of the robot 20 will be described. As an example, it is assumed that the imaging direction of the imaging unit 10 is changed to the right (“→”).
First, the command menu displayed in the first display area 3a of the mobile communication terminal 3 is changed to the imaging unit control menu M1. That is, as shown in FIG. 4, when the conversation communication menu M3 is displayed in the first display area 3a, the robot operation menu M2 is displayed by pressing “*” once. If the initial menu M0 is displayed by pressing “*” twice, it can be changed by pressing “*” three times.

  Next, from the imaging unit control menu M1 displayed in the first display area 3a, the operator confirms that “→” corresponds to the push button 3c of “6”, and push button 3c of “6”. Press. Thus, a DTMF signal “6” is generated by the mobile communication terminal 3 and transmitted to the robot 20. At this time, regardless of the presence or absence of image information captured by the mobile communication terminal 3, it is compressed together with the image information and transmitted as compressed information.

  In the robot 20, the compression information is input to the reception information processing unit 6 via the communication control unit 4 and decompression processing is performed. Then, the DTMF signal “6” is input to the command analysis unit 7. The DTMF signal analysis unit 7b of the command analysis unit 7 converts the command into a corresponding command with reference to the second command correspondence table, and inputs this command to the timing control unit 5a.

The timing control unit 5a recognizes the input command and instructs the imaging control unit 5e to change the imaging direction of the imaging unit 10 to the right. Thereby, the imaging direction with respect to the imaging unit 10 can be changed to the right. The same operation can be performed by an operator's voice input.
In addition, since the imaging unit control menu M1 shown in FIG. 4 includes a command menu for the illumination unit 11, that is, a command for ON / OFF of the illumination unit 11, the illumination control unit 5f is similarly controlled. Thus, the light of the illumination unit 11 can be turned on and off.

[When the robot detects a suspicious object]
Next, the operation of the robot 20 when the robot 20 detects a suspicious object will be described on the assumption that the mobile communication terminal 3 and the robot 20 are not connected to each other.
When a suspicious object is detected in the suspicious object detection processing unit 17, a suspicious object detection notification is made to the timing control unit 5a. Then, the timing control unit 5a instructs the telephone call processing unit 5g to make a call connection with a specific telephone number stored in advance. In response to this, when receiving a call connection instruction, the telephone call processing unit 5g instructs the communication processing unit 4a of the communication control unit 4 to make a call connection with a specific telephone number.
Thus, the robot 20 and the mobile communication terminal 3 having a specific telephone number are in a call connection state.

  Then, the timing control unit 5a acquires the initial menu M0 (see FIG. 4) from the command menu file 14 to the pasting information temporary holding unit 12 in response to the start of the call connection from the telephone call processing unit 5g. At the same time, the suspicious object detection information is acquired from the suspicious object detection information file 18, and the suspicious object detection information is pasted and held at a predetermined position of the initial menu M0. Receiving this instruction, the pasting information temporary holding unit 12 acquires the initial menu M0 from the command menu file 14 and the suspicious object detection information from the suspicious object detection information file 18, respectively, and suspiciously moves to a predetermined position of the initial menu M0. Paste and hold object detection information. Then, the image synthesis unit 13 performs image synthesis by pasting the initial menu M0 with suspicious object detection information held in the pasting information temporary holding unit 12 on the image information captured by the imaging unit 10, and synthesized the image. The image information is output to the transmission information processing unit 16.

Therefore, the transmission information processing unit 16 compresses the combined image information and audio information together and transmits the compressed information to the mobile communication terminal 3 via the communication control unit 4.
Thereby, the mobile communication terminal 3 decompresses the received compressed information and displays the synthesized image information in the first display area 3a. The person holding the mobile communication terminal 3 sees the displayed composite image information, that is, the image information with the initial menu to which the suspicious object detection information is pasted.

As described above, in the robots 2 and 20, information related to the command menu of the robots 2 and 20 is incorporated into the information transmitted to the mobile communication terminal 3, so that the operation commands of the robots 2 and 20 can be given to the operator. You can teach the menu. Therefore, the operator can easily know which push button 3c corresponds to the desired operation command. Thereby, since the robots 2 and 20 can be remotely operated by the mobile communication terminal 3, the robots 2 and 20 are widely accepted in homes, and anyone can remotely operate the robots 2 and 20.
In addition, the robots 2 and 20 are placed in a room of a home or building and in a factory, and an image picked up by the imaging unit 10 of the robots 2 and 20 is obtained by making a call from the mobile communication terminal 3. 3 can be used as a tool for various security systems. In particular, it is useful in the case of the robot 20 including the suspicious object detection processing unit 17.

The command menu file 14 and the suspicious object detection information file 18 can be appropriately rewritten by connecting the robot 20 to a personal computer with a cable and transferring each new file created by the personal computer.
The pasting information temporary storage unit 12 generates not only pasting information from the command menu file 14 and the suspicious object detection information file 18 but also creates pasting information from a file storing general character information and image information. A pasting information generation unit that performs a pasting information generation instruction from the timing control unit 5a to the pasting information generation unit, and holds the pasting information generated by the pasting information generation unit in the pasting information temporary holding unit 12 Also good.
At this time, as information to be pasted on the image information captured by the image capturing unit 10, in addition to the information described above, the type of command actually input to the timing control unit 5 a, the internal state of the robot 2, 20, for example, an actuator The abnormalities of the motor No. 8 and the amount of decrease in the battery of the robots 2 and 20 can also be mentioned.

In the above description, it is assumed that the mobile communication terminal 3 is a mobile communication terminal such as a mobile videophone terminal. This is because it is possible to send and receive games and emails and is widely accepted by society. Therefore, the mobile communication terminal 3 is not limited to this, and the robots 2 and 20 and the communication control unit Any terminal can be used as long as it can be connected to the telephone 4. The mobile communication terminal 3 may be a voice communication device with a numeric keypad, for example, a cordless telephone base unit or handset that can be connected without using a communication company exchange.
In the first and second embodiments of the present invention, the robots 2 and 20 and the mobile communication terminal 3 are not connected using the Internet. I can't peek. Therefore, a safe security system can be constructed with the robots 2 and 2 and the mobile communication terminal 3.

  The embodiment described above is merely an example of the present invention, and various modifications can be made within the scope of the invention described in the claims.

It is a functional block block diagram of the robot remote control system by Embodiment 1 of this invention. It is a functional block diagram of the robot of FIG. It is an example of the 1st command corresponding table stored in the command analysis part of FIG. Fig. 5 schematically shows a series of command menus stored as a command menu file in Fig. 4. It is a functional block block diagram of the robot which comprises the robot remote control system by Embodiment 2 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Robot operation remote system 2,20 Robot 21 Joint part 3 Portable communication terminal 3a 1st display area 3b 2nd display area 3c Push button 4 Communication control part 4a Communication processing part 4b Transmission / reception part 5 Control part 5a Timing control part 5b Transmission number Confirmation processing unit 5c Operation control unit 5c1 Walking pattern generation unit 5c2 Motor control unit 5c3 Walking pattern file 5d Utterance sentence generation unit 5e Imaging control unit 5f Lighting control unit 5g Telephone transmission processing unit 6 Reception information processing unit 7 Command analysis unit 7a Voice recognition Analysis unit 7b DTMF signal analysis unit 8 Actuator 9a Speech synthesis unit 9b Audio output unit 10 Imaging unit 11 Illumination unit 12 Paste information temporary storage unit 13 Screen synthesis unit 14 Command menu file 15 Sound input unit 16 Transmission information processing unit 17 Suspicious object detection Processing unit 17a Audio processing unit 17b Image processing Part 17c sensor 18 suspicious object detection information file

Claims (6)

In a robot that can be remotely operated with a mobile communication terminal,
A communication control unit for making a call connection with the mobile communication terminal;
An imaging unit for imaging;
An image composition unit for compositing an image by pasting a command menu for remote control of a robot to image information captured by the image capture unit while the communication control unit is connected to the mobile communication terminal; Robot to do. A command menu file for storing a series of command menus to be pasted in the image composition unit;
The robot according to claim 1, further comprising: a pasting information temporary holding unit that temporarily holds a command menu pasted by the image composition unit from the command menu file in a switchable manner. In the series of command menus,
An imaging unit control menu that displays instructions for the imaging unit in correspondence with buttons of the mobile communication terminal;
A robot operation menu that displays instructions for the drive mechanism in correspondence with the buttons of the mobile communication terminal;
A conversation communication menu displaying instructions related to voice output in correspondence with the buttons of the mobile communication terminal;
The robot according to claim 2, comprising: A suspicious object detection processing unit for detecting a suspicious object;
Upon receiving suspicious object detection information from the suspicious object detection processing unit, the communication control unit is instructed to establish a call connection with the portable communication terminal, and the communication control unit and the portable communication terminal are instructed by the instruction. A control unit that instructs to paste suspicious object detection information on image information captured by the image capturing unit when a call is being connected. 5. Robot. The suspicious object detection processing unit
An image processing unit that processes image information captured by the imaging unit to perform suspicious object determination, and determines that there is a suspicious object, includes an image processing unit that notifies the control unit of suspicious object detection. Item 5. The robot according to item 4. The suspicious object detection processing unit
It comprises an image processing unit that processes the audio information input and converted by the sound input unit to perform suspicious object determination, and notifies the control unit of suspicious object detection when it is determined that there is a suspicious object. The robot according to claim 4 or 5.

Images