KR20010016048A - A home personal robot with multi-faculity - Google Patents

Abstract

PURPOSE: A multipurpose home personal robot is provided to enable everybody to easily contact with a computer for convenience by adding the function of the robot to the computer. CONSTITUTION: A multipurpose personal robot(50) comprises an image processor(11) converting image information to simple information; a sound processor(12) recognizing sound from a sound sensor(2) and outputting an answer through a speaker(4); and an obstacle detecting module(6) detecting an obstacle using supersonic waves. Further, the robot contains a main controller(13); a motion processor(14) driving a motor array(8) to move the robot by the control signal of the main controller; a display module(5) displaying all information; a storage unit(15); an operating unit(7) controlling the robot; and a wireless communicating module(16) communicating between the main controller and a personal computer or a medical sensor(3). Thus, the robot gives a user the best service by being wirelessly connected to the personal computer.

Description

A home personal robot with multi-faculity}

The present invention relates to a multi-functional home personal robot, and more particularly, by adding the concept of a robot to a computer to free it from input and output constraints and space constraints of a conventional computer, anyone can easily access the computer. It combines automation and internet technologies to perform user recognition, e-mail delivery, voice command recognition, internet search, internet phone, internet reservation / ordering, schedule management, health care, and security. The present invention relates to a multi-functional home personal robot that can maximize the user's satisfaction by installing the robot management server on-line and managing the robot to provide the best service at all times.

In general, personal computers (PCs) are now recognized as indispensable in human life, and the development of computers has made human life convenient and profitable, but many people still feel unfamiliar with computers. The reality is that the computer does not even use the computer at all, and even if it is used, it is limited to the space where the computer is located, and is limited to the use of paper work, entertainment, program development, and Internet search.

Looking at the problems of the existing computer, firstly, the current computer can be operated only through localized input devices such as a keyboard or a mouse, and the result can be obtained only by relying on local peripherals such as a monitor or a printer. Because of this, those who are not familiar with the keyboard or mouse are burdened with using the computer, and have no choice but to communicate with the computer.

Secondly, because the computer is not a movable object, it can be operated only at the place where it is installed, which means that the user has to move to the place where the computer is intentionally. Third, the existing computer has been transformed into a beautiful shape due to the advancement of many designs, but the basic framework is clerical and rigid, and it still does not fit with other elements in the home, making it easier for people unfamiliar to create discomfort. I can't. Fourth, if a person who is familiar with computers wants to be provided with services using the Internet, there is a hassle of having to find and visit a site that provides services.

In Japanese Patent Laid-Open No. 61-82777, a robot that is programmed through a personal computer and driven by the program is developed. In Japanese Patent Laid-Open No. 7-248823, a personal robot capable of speech recognition has been developed. Not only this limitation but also technology such as operating a personal computer by controlling a robot by a user's voice command has not been developed at present.

The present invention is provided to solve the above-mentioned conventional problems, by adding the concept of a robot to the computer to free from the input and output constraints and space constraints of the existing computer, anyone can easily access the computer, home The combination of automation and Internet technologies ensures a more convenient and comfortable life for humans. It also provides unique Internet search and Internet functions by strengthening Internet services as well as user recognition, email delivery, voice command recognition, and home automation. It has not only telephone function, internet reservation / order function, schedule management function, health management, active security function, but also the robot management server installed on-line so that the robot can always provide the best service for users' satisfaction. To help maximize do.

1 is a front view of a multifunctional home personal robot according to the present invention;

2 is a block diagram of a multifunctional home personal computer robot according to the present invention;

3 is a conceptual diagram of a multifunctional home personal computer robot according to the present invention;

4 is a functional diagram of a multifunctional home personal computer robot according to the present invention;

5A and 5B are views of the multifunctional home personal computer robot according to the present invention.

Operation flow chart of the person recognition function.

Figure 6a is a voice signal processing unit of the multifunctional home personal computer robot according to the present invention

Block diagram.

FIG. 6B is a detailed block diagram of the voice detector of FIG. 5A; FIG.

6C is an operation of voice recognition of a multifunctional home personal computer robot according to the present invention.

Flowchart.

7A illustrates an Internet mode of a multifunctional home personal computer robot in accordance with the present invention.

Connection diagram between robot and peripheral system to implement.

7B illustrates an Internet mode of a multifunctional home personal computer robot according to the present invention.

Flow chart for implementation.

8A is a home automation of a multifunctional home personal computer robot in accordance with the present invention.

Connection diagram for running the mode.

8B is a home automation of a multifunctional home personal computer robot according to the present invention.

Flowchart for executing the mode.

9 is a security mode implementation of the multi-functional home personal computer robot according to the present invention

Flowchart for.

10 is an environmental recognition function of a multifunctional home personal computer robot according to the present invention.

System block diagram showing.

11 is a self-diagnostic mode of the multifunctional home personal computer robot according to the present invention.

Flowchart for implementation.

12 is a learning mode implementation of the multifunctional home personal computer robot according to the present invention.

Flowchart for.

13 is a health management function of a multifunctional home personal computer robot according to the present invention.

Connection diagram between robot and external system to perform.

14 is an entertainment function of the multifunctional home personal computer robot according to the present invention.

Connection diagram with external system to perform the task.

15A is a server for providing a service to a robot through a robot management server;

Connection conceptual diagram.

15b is an operation for providing a service to a robot through the robot management server

Flowchart.

Hereinafter, the configuration and operation of the present invention will be described in detail with reference to the accompanying drawings.

1 is a front view of the multifunctional home personal robot according to the present invention, and FIG. 2 is a block diagram of the multifunctional home personal computer robot according to the present invention. The main control unit 13, the image processing unit 11, and the audio processing unit 12 are large. And a motion processor 14.

The image processor 11 converts the image information from the image sensor 1, such as a connected video camera, into simple information necessary for determination using an image processing technique, and transmits the image information to the main controller 13. For example, when the speaker recognizes the speaker's face and iris image information from the image sensor 1, the features are extracted and passed on to the main controller 13, which is established in the main controller 13. Based on the information, the speaker's identity is recognized.

The voice processor 12 classifies the sound input from the voice sensor 2 into a voice and a non-voice, and extracts and compares a feature using a voice recognition algorithm to recognize a command given by the voice, using a voice recognition algorithm. At the same time, it is possible to recognize the direction in which the voice is received by placing a plurality of voice sensors to face in multiple directions, and emits voice messages transmitted from the main control unit 13 to the outside through the speaker (4) It has a function.

The motion processor 14 receives the command from the main control unit 13 to control the motor array 8 to move the robot and rotate the head, and to pick up the object using the arm.

The main control unit 13 processes data based on the information input from the image processing unit 11 and the audio processing unit 12 and the information of the obstacle detecting sensor 6 to determine and execute a response to the user's request. The main control unit 13 is connected to a display module 5 including a monitor so that the requested information can be visually confirmed, while the main control unit 13 is responsible for wireless communication. It is built in so that it can transmit / receive data wirelessly with the home personal computer (Home PC) 20 installed in the home.

3 is a conceptual diagram of a multifunctional home personal computer robot according to the present invention, wherein the home personal computer 20 is connected to a common communication network through a LAN module 22, and a plurality of home appliances 30 are also wirelessly communicated. The module 21 is mounted and connected to the LAN module 22 through internet information appliances, and management offices, police stations, fire departments, and servers providing various services are connected to the LAN module 22 to meet the needs of the robot. Accordingly, each service can be supplied through the LAN module 22.

In addition, the robot management server 22 serves to help to smoothly upgrade the software of the robot online or to perform the service function of the robot, the wireless communication module 21 of the home personal computer 20 and the robot ( The wireless communication module 16 of 50 communicates with each other wirelessly.

When described in detail the operation of each function of the multi-functional home personal robot according to the invention configured as follows.

First, the home personal robot of the present invention can remember and recognize not only the owner but also the family and the people who are related to the owner, and the home personal robot is given a learning ability to give the owner a command to remember the shape of the face and The pattern of the iris is stored in the storage location by matching data such as name and phone number, and it is remembered the next time it is met again.

Also, if you remember that the owner should be alert when saving the information, the person does not open the door even if he visits the house when he does not have a master. Watch.

Such a person recognition function will be described in detail with reference to FIGS. 4A and 4B.

First, the human recognition process is described by FIG. 4A. When the power is turned on, the robot initializes the system (step 101), receives a command through the voice sensor in the standby mode, or prepares to respond to the sound. If it is determined that the sound comes through (step 102), the robot discovers the source of the sound and faces the place (step 103), and then operates the image sensor 1 to recognize the cause of the sound (step 104).

At this time, when the image is acquired from the image sensor 1 (step 105) and transmitted to the image processor 11, the noise such as unnecessary noise other than the voice is removed (step 106), converted into a binary code, and opened / crawled. Recognizing a person by obtaining an image obtained by extracting only a feature based on an opening / closing process (steps 107 and 108) (step 109), calculating the features, and comparing the stored data with the first stored data (step 110). )

When the data stored in the above step is large or when several data are compared at a time in order to accurately recognize the comparison processing time, it uses a GA-Neural Algorithm to search intelligently and efficiently.

Referring to FIG. 4B, the behavior of the robot in the state of extracting the features of a plurality of people through the process of FIG. 4A is recognized in the database. First, an object is recognized by acquiring an image from the image sensor 1. In step 112, it is determined whether a person is present (step 113), and if it is a person, a face shape and a pupil pattern are acquired (step 114) to extract a feature (step 115).

Thereafter, the extracted feature is compared with the information stored in the database (step 116), and if it is already stored, it is determined as a known person (step 117), and if it is determined that the person is not stored in the database, the owner Will be ordered (step 118) and behave differently by this order.

In other words, if the owner is ordered as a person to be alerted, it is stored in the database, followed by a border (step 119), etc. If there is no special command from the owner, the boundary is not stored in the database. In step 120), if a command is given to remember from the owner, it is stored in a database (step 121) so that the robot displays each behavior pattern for the person who should be alerted, the person who should not be alerted, and the first-time viewer. Determined by).

Next, the voice command recognition function will be described in detail.

First, the home personal robot 50 has an independent voice recognition processing part and a voice command database used in daily life, and when the owner commands by voice, it recognizes and performs as it is. For example, if you say "Please turn on the lights in the living room", you get "Living Room", "Light", "Turn on", etc., and you want to order food, "Food", "Order" You will hear the back and ask, "What would you like to order?"

This speech recognition process will be described with reference to FIGS. 6A to 6C.

First, for voice recognition, a human voice signal is first input through the microphone 24, and the voice signal output from the microphone 24 is converted into a voltage by the voltage detector 30 of FIG. 6A, and the converted voice signal is converted into a voltage. The audio signal is amplified by about 1000 times via the first and second amplification units 31 and 32, and then a high pass filter of 10 Hz to remove noise other than the human voice existing in the amplified signal. And the low pass filter 34 of 8 kHz through the analog signal processing to extract only the human voice signal.

The amplified and filtered voice signal as described above is converted into digital data through an A / D converter 26 having a resolution of 12 bits as shown in FIG. 6B, and the feature extractor 27 and the pattern recognition unit ( The meaning of the command is determined after comparing with the pattern of the database 29 prepared in advance through 28), and the command having the determined meaning is transmitted to the main control unit 13 through the CAN bus to operate the robot. Will affect.

That is, the main control unit 13 obtains a sentence from the processed speech data as shown in FIG. 6C (step 210), extracts a meaningful word (step 202), and confirms the command by combining the extracted words (step 203). If it is possible to execute the confirmed command (step 204), if it is possible to execute (step 206), otherwise, explain the reason (step 205) and enter the standby state. Figure 4.7 is a flowchart of the command recognition program.

Next, the Internet mode will be described with reference to FIGS. 7A and 7B.

Figure 7a is a connection diagram between the robot and the peripheral system for implementing the Internet mode of the multi-functional home personal computer robot according to the present invention, Figure 7b is a flow chart for implementing the Internet mode of the multi-functional home personal computer robot according to the present invention As the multi-functional home personal robot 50 of the present invention is connected to the network through the home personal computer 20 as shown in FIG. 7A, various things that can be done with the Internet can be performed using only voice.

That is, if there is a part that the owner wants to look for when reading a book or doing homework, it can be easily found by using the Internet search function of the home personal robot 50. The home personal robot 50 has a function of the existing web keeper, and when a target to be searched is given only to a target to be searched as shown in the search processing routine L1 of FIG. 7B, the home personal robot 50 is to be searched in a storage location of the home personal computer 20. Will find and keep online information, and the owner will be able to complete the desired work by organizing only the stored information.

When the owner's E-mail arrives, the home personal robot 50 receives a signal from the server and informs the owner, as in the e-mail processing routine L3 of FIG. On the other hand, when the owner writes the e-mail in voice or document and gives it to the home personal robot 50, the robot automatically transmits the e-mail.

In addition, the home personal robot 50 is connected to the Internet telephone service provider as shown in the automatic telephone processing routine (L2) of FIG. 7b, so that users can make a phone call without using an existing phone, and solve all problems online. When not available, for example, when you order something or want more information, you can use the Internet telephony feature to automatically call a representative.

In addition, it has an internet order and reservation function that makes reservations and orders for movies, railroads, travel, goods, etc. via voice. That is, when the robot 50 tells the item that it wants to reserve, the robot 50 approaches the server and performs the reservation process automatically. When performing the home shopping, the product is dynamically displayed through the display module 5 of the robot 50. By displaying the information of the owner, the owner can go directly to the shopping center and choose the effect.

Next, the home automation mode will be described with reference to FIGS. 8A and 8B.

8A is a system connection diagram for executing the home automation mode of the multifunctional home personal computer robot according to the present invention, and FIG. 8B is an operation flowchart for executing the home automation mode of the multifunctional home personal computer robot according to the present invention. Information can be manipulated by voice through a robot in conjunction with home appliances equipped with electric appliances and wireless communication modules.

That is, the TV, light, video tape recorder (VTR), washing machine and air conditioner in the house are turned on and off according to the owner's voice command. For example, if the owner asks the robot 50 to turn on the television by voice, the robot turns on the television, and if it asks to turn it off a few hours later, the robot turns off the television in time.

In addition, the vehicle diagnosis function can monitor the condition of the vehicle before traveling or driving, thereby predicting the failure of the vehicle in advance, preventing accidents in advance, and starting the vehicle by voice.

Next, the security mode will be described with reference to FIG. 9.

It is connected to gas leak sensor and fire detection sensor to notify management office and fire department in case of gas leakage and fire, and to owner to mobile phone, and it is connected to door camera. When you come, don't open the door and save the person's face.

In addition, if there is nobody in the house, if there is a strange sound, it is tracked and stored and reported to the owner. If it is judged to be a thief, the police department or security company is notified. Turn it off and make it look as if there are people in the house and protect the house by yourself.

9 is an operation flow chart for implementing a security mode of the multi-functional home personal computer robot according to the present invention, the robot is switched to the crime prevention mode by the owner's command when the house is empty, and when the security mode is switched to perform an alarm function and a crime prevention function Done. Therefore, when the danger signal comes from the dangerous goods sensor in the state of setting the security function, the robot detects an abnormality, contacts the guard room or the fire department, reports it to the owner's mobile phone, and authenticates the person through the door camera when the person comes in. To control.

Next, the environment recognition function will be described with reference to FIG. 10.

10 is a system block diagram showing an environmental awareness function of a multi-functional home personal computer robot according to the present invention, the robot 50 has a temperature sensor and a humidity sensor to operate an air conditioner if it is hot and to operate a heating facility when it is cold, In addition, during the rainy season, when the humidity increases, the heating system and the dehumidifier are operated to remove the humidity.

And because he has the ability to learn, he goes around every corner of the house and saves the structure of the house, so if the owner asks to come to the kitchen, he does not hesitate to go to the kitchen. In addition, it automatically searches for home appliances that can be operated by the home, stores them in a storage location, and uses them when the owner's orders are given.

Next, the self-diagnosis mode will be described with reference to FIG. 11.

The robot 50 performs a self-diagnosis function, and if the robot has an error or a communication error, the robot 50 checks and reports at the request of the user, and when the robot is repaired, the robot management server enters the robot's self-diagnosis mode. If possible, the necessary parts can be checked and replaced smoothly.

11 is a flowchart illustrating the operation of the self-diagnosis mode of the multi-functional home personal computer robot according to the present invention. When self-diagnosis is performed by a voice command or a request from the robot management server, a communication test is performed first to perform a quick diagnosis. It reports the status and asks the user for detailed diagnosis. If the detailed diagnosis is performed, all functions of the robot are tested to determine whether the robot is abnormal and report to the user.

Next, the learning mode will be described with reference to FIG. 12.

12 is an operation flowchart for implementing a learning mode of the multifunctional home personal computer robot according to the present invention. When the robot 50 does nothing in the standby mode, the robot 50 automatically learns the surrounding information by an automatic selection algorithm. Acquisition of image information and distance information as you walk around the house Creates a map of the house by yourself through a certain algorithm and stores it for use when necessary.

In addition, it communicates with the home appliance to generate a list of controllable home appliances, confirms and stores a location, and can also perform a learning function by a voice command, and in particular, switches to a learning mode to store a person's information. .

Next, the health management function will be described with reference to FIG. 13.

FIG. 13 is a connection diagram of a robot and an external system for performing a health care function of a multifunctional home personal computer robot according to the present invention, and monitoring a state of a user through a medical sensor 3 of the robot 50. You can check your health status from time to time by linking with a medical information service company. If you are sick, you can receive prescriptions through telemedicine and order medicines at pharmacies.

Next, the entertainment function will be described with reference to FIG. 14.

14 is a connection diagram with an external system for performing an entertainment function of a multifunctional home personal computer robot according to the present invention. The robot 50 may read a fairy tale to a child by voice and display a movie or video display module 5. Can also be shown through.

Next, a process of managing the robot 50 on-line by the robot management server 23 will be described with reference to FIGS. 15A and 15B.

The robot management server 23 is connected to the robot 50 on-line and plays a secondary role of software upgrade and service function of the robot 50. Data required for voice recognition or image recognition is delivered to the robot through on-line, and a company convenient to the user is selected from several service companies connected to the on-line and transmitted to the robot 50, and the robot is on-line according to the user's request. Even if you do not search on the top of the selected company will be able to access the service.

15A is a conceptual diagram of a connection to a server for providing a service to a robot through the robot management server, and FIG. 15B is an operation flowchart for providing a service to the robot through the robot management server. In the robot management server 23, a robot ( 50) always upgrades as soon as the program is developed so that the best performance can be achieved, and the robot management server 23 retrieves the data when the request of the robot 50 is received through the server in charge of Internet search and stores the robot. Store it in a place, connect with a mail server, send a signal to the robot when the user's mail arrives, send the contents of the mail to the robot, and send the mail to the destination server when the user sends the mail through the robot.

In cooperation with home shopping companies, restaurants, travel agencies, railway agencies, airlines, theaters, and so on, users can easily order and make reservations through robots and connect with companies or hospitals / pharmacies that perform online medical services. In addition to providing health care services, banks and telephone companies can search for billing schedules and amounts, and inform users of the payment schedule and amount. It provides a service for viewing through a robot, and provides a service that can be directly connected to a related server.

In addition, the robot can perform a self-diagnostic function to always monitor the status of the robot 50 and to immediately perform the after-sales service when there is a failure or abnormality, the robot management server 23 itself to the service information By establishing a database, the best information is always provided to users with the latest information.

In addition to the above-described functions, the multi-functional home personal robot according to the present invention, if the owner remembers the schedule through voice or home personal computer 20, electronic calendar, the robot informs you according to the time, and only memorizing anniversaries or important days It has a scheduling mode that reminds the owner without forgetting it, and an alarm function that wakes up like an alarm clock so that the robot can find it one by one at a specified time even if the set time is different, so as not to be late for morning work or school time.

In addition, many people forget the bill payment date and pay the late fee, but the bill payment schedule reminder to remind you of the bill payment date so as not to pay late, and the robot 50 has video and audio storage capability, so the owner goes out. When you store a message with the person you want to forward, the robot sends the message along with the video information when the robot meets the person you want to forward, while you receive a polite call when the phone is empty and you remember the message It provides a message delivery function that notifies you when it comes, and informs you of the weather or news collected through the Internet, and selects a meal menu, and provides financial and quote information and fortune telling if you want, so that you can advise the information you want. Has the advice function of acting as a partner.

As described above, the multi-functional home personal robot according to the present invention adds the concept of a robot to the computer, thereby releasing the input / output constraints and the space constraints of the existing computer so that anyone can easily access the computer. And internet technologies are combined to ensure a more convenient and comfortable life for humans.

In addition to adopting user recognition function, email delivery function, voice command recognition function, it also has unique internet search function, internet phone function, internet reservation / order function by strengthening home automation and internet service, and schedule management. It has the function to be able to fulfill the role of family secretary.

In addition, it has a health management function, and it is judged that it will be able to fulfill the role of the family doctor.The active security function not only makes it possible to leave the vacation and going out in peace, but also installs a robot management server online. By managing the robot to provide the best service, the user's satisfaction can be maximized.

Claims (7)

An image processing unit 11 for converting image information input from the image sensing sensor 1 installed on the face portion of the home personal robot 50 into simple information necessary for determination using an image processing technique, and outputting it; The sound received from the voice sensor 2 installed at the ear of the home personal robot 50 is extracted and compared using a voice recognition algorithm to recognize a command given by the voice, and the speaker answers the answer to the command. And a voice processing unit 12 output through An obstacle detection module 6 which detects an obstacle in front of the ultrasonic wave and outputs an obstacle signal; Recognizing the signal output from the obstacle detection module 6, and outputs a control signal of the movement direction of the robot 50, and recognizes the image information from the image detection sensor 1 based on the information established A main control unit 13 for recognizing the congestion, recognizing the sound from the voice sensor 2, recognizing the speaker, and outputting a predetermined control signal; A motion processor 14 for driving the motor array 8 for moving the robot 50 by the control signal of the main controller 13; A display module 5 for displaying all situations and image information by the control signal of the main controller 13; A storage unit 15 which transmits / receives data to / from the main control unit 13 and stores voice and image information and various data; An operation unit 7 operating for various control of the home personal robot 50, Multi-functional home personal robot, characterized in that the control signal of the main control unit (13) consisting of a wireless communication module (16) for performing wireless communication with the home personal computer (20) and the medical sensor (3). According to claim 1, wherein the voice processing unit 12, the display module 5, the obstacle detection module 6, the motion processing unit 14 is to communicate with the main control unit 13 by the CAN (CAN) method Featured multifunctional home personal robot. In the control method of the multifunctional home personal robot, The process of initializing the system and turning it on to identify the source of the sound when power is supplied, and recognizing the cause of the sound, Removing noise such as unnecessary noise other than the recognized voice in the above process, obtaining an image extracted only from features, calculating the features, and comparing the stored data with the stored data; Extracting features by acquiring a face shape and a pupil pattern when the object is a human in the above process; Multi-functional home personal robot, characterized in that the feature is extracted in the process compared to the information stored in the database is stored as a known person, and if not stored by the owner's command bound and stored Control method. In the control method of the multifunctional home personal robot, A method of controlling a multifunctional home personal robot, characterized in that the home personal computer and the multifunctional home personal robot are wirelessly connected to perform an internet search function, an e-mail transmission function, and an internet auto dialing function through voice. In the control method of the multifunctional home personal robot, A control method of a multi-functional home personal robot, characterized in that the home appliance equipped with a wireless communication module and a robot communicate wirelessly to control the home appliance by a robot through voice. In the control method of the multifunctional home personal robot, The control method of the multi-functional home personal robot that enables the medical sensor and robot to detect the blood pressure, body temperature, and pulse rate of the human body to wirelessly communicate with the user to monitor the user's health status and perform remote medical care through the Internet connection function. . In the control method of the multifunctional home personal robot, The control method of the multi-functional home personal robot that the software management of the robot is made by the robot management server connected on the Internet to manage the robot through online.