KR20210022394A - A moving robot for the blind and control method thereof - Google Patents

Abstract

The present invention relates to an autonomous moving robot for assisting shopping of the blind in a shopping space. More specifically, the moving robot for assisting shopping of the blind during autonomous driving comprises: a main body guiding a user to a destination; an input device mounted in an ear of the user to be wirelessly connected to the main body; a support installed in the main body; a handle connected to the support; a storage unit disposed in the support and movable along the support; at least one obstacle detection sensor disposed in the main body; a plurality of wheels disposed in a lower portion of the main body; a plurality of motors driving the wheels; a power supply device providing power; and a control unit controlling the autonomous driving of the moving robot.

Description

TECHNICAL FIELD A moving robot for the blind and control method thereof for the visually impaired

The present invention relates to a mobile robot for the visually impaired, and more particularly, to a mobile robot that runs autonomously to help the visually impaired to shop in a shopping space.

According to the World Health Organization (WHO), it is estimated that about 288 million people are blind, and about 36 million people are blind.

Blindness and blindness affect not only individuals but also families and communities, resulting in significant cost burdens. For example, visual impairment and blindness adversely affect not only quality of life, independence and mobility, but also mental health, social functioning, employment and education. Therefore, many methods to assist the life of the visually impaired are being studied.

Among them, robots have been developed for industrial use and have been in charge of a part of factory automation. In recent years, the field of application of robots has been further expanded, medical robots, aerospace robots, and the like have been developed, and home robots that can be used in general homes are also being made. Among these robots, those capable of driving by magnetic force are called mobile robots.

In the conventional technology as a mobile robot for the visually impaired (Korean Patent No. 10-1497096), a visually impaired person in a large mart or shopping mall guides the robot to the desired store while holding a handle at the back of the shopping assistant robot and maintaining a certain distance. Thus, a shopping assistant robot for the visually impaired to help shopping is disclosed.

In the prior art, when a robot detects a sensor attached to the floor through an infrared sensor, the name (or number) of the robot is confirmed after meeting the robot by transmitting a Bluetooth communication signal with a remote control with a built-in Bluetooth and a microphone. Thus, a technology that enables a visually impaired person and a robot to accurately guide to a store of a product to be purchased through a conversation by voice recognition has been disclosed.

However, the above technology has the problem of attaching a separate sensor to the floor, and because both hands are not free due to the structure of talking through the remote control, it is inconvenient for the visually impaired to support a shopping cart or select and purchase products. And there is a safety problem.

Meanwhile, in the field of mobile robots, interest in Simultaneous Localization And Mapping (SLAM) technology is increasing recently. SLAM is emerging as a core technology for autonomous driving as a technology that creates a map of the unknown environment while the robot moves around the unknown environment and recognizes its own location without external help with only the sensors attached to the robot.

Korean Patent Registration No. 10-1497096

In order to solve the above problems, an object of the present invention is to provide a mobile robot that assists the shopping of the visually impaired while autonomously driving through a minimum number of sensors using SLAM technology.

In addition, an object of the present invention is to provide a mobile robot that frees both hands of a visually impaired person to help safe shopping.

In addition, an object of the present invention is to provide a mobile robot that provides accurate information on a product so that the visually impaired can select a desired product.

The problems of the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the mobile robot for the visually impaired according to the present invention is a mobile robot that autonomously drives and helps the shopping of the visually impaired, comprising: a main body that guides a user to a destination; An input device mounted on the user and wirelessly connected to the main body; A support installed on the main body; A handle connected to the support; A storage unit disposed on the support and movable along the support; At least one obstacle detection sensor disposed on the main body; A plurality of wheels disposed under the main body; A plurality of motors driving the plurality of wheels; A power supply for providing electric power; And a control unit that controls the autonomous driving of the mobile robot.

The mobile robot for the visually impaired according to the present invention further includes a camera disposed on one side of the input device and photographing a user's gaze direction image, and generates map data by mapping a shopping space, and a user through the input device And a control unit configured to receive an input signal input by and set a destination, perform path-planning to the destination, and control a plurality of wheels to allow the mobile robot to autonomously travel along the planned route; .

The mobile robot for the visually impaired according to the present invention generates SLAM-based map data through an obstacle detection sensor, and when a destination is set, generates a first route that is an optimal route to the destination based on the map data, While driving along the first path, a cost value is calculated based on obstacle information detected by the obstacle detection sensor, a second path for avoiding the obstacle in a direction having a low cost value is generated, and the second path is It further includes a control unit for performing a route planning by correcting the first route through.

The mobile robot for the visually impaired according to the present invention further includes a user detection sensor disposed on a handle and detecting whether a user is holding the handle.

The mobile robot for the visually impaired according to the present invention acquires information on whether the user is holding the handle through a user detection sensor disposed on the handle, and when it is determined that the user is not holding the handle, driving a plurality of wheels is performed. It further includes a; control unit for generating a signal to stop.

The mobile robot for the visually impaired according to the present invention further includes a communication device that communicates with an external server.

The mobile robot for the visually impaired according to the present invention comprises: a wearer mounted on the ear of a user; MIC; speaker; And an input device including one or more buttons.

The mobile robot for the visually impaired according to the present invention further includes an input device including at least one of an on/off button of a power supply device or a communication start/end button with an external server through a communication device.

The mobile robot for the visually impaired according to the present invention further includes a control unit that receives an input signal input by a user through a microphone and transmits a signal guiding a planned route to the user to a speaker.

The mobile robot for the visually impaired according to the present invention, when receiving a communication start signal through a communication start/end button with an external server, streams an image captured by a camera to an external server through a communication device in real time, and the external server It further includes a; control unit for generating a signal to be output through the speaker by receiving information on the product from the.

Other specific matters of the present invention are included in the detailed description and drawings.

According to the present invention, there are one or more of the following effects.

First, by generating SLAM-based map data and performing route planning, there is an effect of enabling efficient autonomous driving to a destination with only a sensor attached to the robot, and helping the visually impaired with efficient shopping.

Second, there is an effect of improving autonomy in shopping by liberating both hands of the visually impaired through an input device attached to the ear of the visually impaired.

Third, there is an effect of improving safety in shopping by detecting and stopping when a blind person misses a handle through a user detection sensor disposed on the handle.

Fourth, through communication with an external server, stable guidance can be received even in an emergency situation, and accurate product information is obtained to improve satisfaction in shopping.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a diagram showing a guide system for a visually impaired according to an embodiment of the present invention.
2A to 2B are views showing the appearance of a mobile robot for the visually impaired according to an embodiment of the present invention.
3A to 3C are diagrams illustrating an input device of a mobile robot according to an embodiment of the present invention from various angles.
4 is a control block diagram of a mobile robot according to an embodiment of the present invention.
5 is a flow chart of a control unit according to an embodiment of the present invention.
6 is a diagram for explaining a route planning step according to an embodiment of the present invention.
7 is a diagram for explaining a first route and a second route according to an embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

The present invention improves the prior art by providing a mobile robot and a shopping guide system for the visually impaired to assist with shopping for the visually impaired. The present invention improves the prior art by providing insight into an environment that is impossible only with a cane or a guide dog, which is an existing technology that assists the blind person. Moreover, the present invention does not require training and can be operated by anyone.

1 is a diagram showing a guide system for a visually impaired according to an embodiment of the present invention.

Referring to FIG. 1, a guide system 10 for a visually impaired according to an embodiment of the present invention includes a mobile robot 100, an external server 300, and an input device 200 connecting the mobile robot 100 and a user. ) And a remote computing device 350 connecting the external server 300 and the guide.

The mobile robot 100, the input device 200, the external server 300, and the remote computing device 350 illustrated in FIG. 1 are not limited to those illustrated in FIG. 1. The mobile robot 100 and the input device 200 will be described later with reference to FIGS. 2A to 3C.

The guide system 10 may connect each component of FIG. 1 through a communication network. The communication network is one or more packet-switched networks such as the Internet, or any local area network, wide area network, corporate private network, cellular network including 4G LTE to 5G communication, a Wifi network, a telephone network, a mobile communication network, or a combination thereof. It may include.

In one embodiment of the invention, the communication network may be a cellular network.

The external server 300 may be communicatively connected to a communication network. The external server 300 includes a software engine that delivers applications, data, program codes, and other information to a device connected to a network. The software engine can perform a process such as streaming audio or video.

The external server 300 includes a database, and the database may include a SQL (Structured Query Language) database stored in the SQL server or a relational database including a database conforming to the NoSQL paradigm. The database may provide sensor data and related information used by the external server 300 and a device connected to the network during the operation process of the present invention.

The remote computing device 350 may be connected to a communication network. Remote computing devices may include any computing device such as, for example, integrated circuits, printed circuit boards, processors, ASICs, PCBs, mobile phones, smart phones, tablet computers, laptops and game consoles. The remote computing device 350 may be connected to a communication network in the form of wireless or wired or optical fibers.

In this embodiment, the remote computing device 350 may be a touch screen capable tablet computer.

2A to 2B are views showing the appearance of a mobile robot for the visually impaired according to an embodiment of the present invention.

2A to 2B, the mobile robot 100 for the visually impaired includes a main body 110, a support 120, a handle 130, a storage unit 140, a plurality of wheels 150, and an obstacle detection sensor ( 160), a controller 170, a user detection sensor 180, and an input device 200. The input device 200 will be described later in detail with reference to FIGS. 3A to 3C.

The body 110 may include a bottom panel, a front panel, a rear panel, and two side panels. In one embodiment of the present invention, the bottom panel, the front panel, the rear panel, and the two side panels are formed of a waterproof material, and each panel may have a substantially flat rectangular shape.

The support 120 may be disposed on one side of the main body 110. The support 120 may protrude upward from one side of the main body 110 to be connected to the handle 130. That is, the support 120 may connect the main body 110 and the handle 130.

The handle 130 may be disposed on one side of the support 120 to form a long cylindrical shape so that the user can grasp it while moving. In an embodiment of the present invention, the handle 130 has an elongated cylindrical body, but may have other shapes and sizes. The handle 130 may be made of a rubber material such as silicone, neoprene, and urethane.

The storage unit 140 may be used when a user selects an object and stores the selected object. The receiving unit 140 may be disposed on the support and be movable along the support.

The plurality of wheels 150 may be disposed under the main body 110. The plurality of wheels 150 allow the mobile robot 100 to move on a surface or along the ground. In one embodiment of the present invention, a total of four wheels are attached to each corner of the body 110 through an axle (not shown), and the axle and the wheel are adjustable to move back and forth.

The mobile robot 100 may include a plurality of motors that rotate at least one of the plurality of wheels 150 so that the body can move back and forth. The motor may be disposed on each of the plurality of wheels to drive the plurality of wheels.

The power supply device may provide power for the operation of the mobile robot 100. The power supply can provide power to the motor to move the body. In one embodiment of the present invention, the power supply may be a battery.

The obstacle detection sensor 160 may include at least one sensor capable of detecting an obstacle outside the mobile robot 100. The obstacle detection sensor 160 may include at least one of a camera, a radar, a lidar, an ultrasonic sensor, and an infrared sensor. The obstacle detection sensor 160 may provide obstacle information generated based on a sensing signal generated by the sensor to the controller 170.

The obstacles may be various objects related to the movement of the mobile robot 100. For example, it may include goods, product information boards, display stands, people, other mobile robots, roads, structures, landmarks, animals, and the like.

The camera may generate information about an obstacle outside the mobile robot 100 by using an image. The camera may include at least one lens, at least one image sensor, and at least one processor that is electrically connected to the image sensor and processes a received signal, and generates obstacle information based on the processed signal.

The camera may be at least one of a mono camera, a stereo camera, and an RGBD camera. The camera may use various image processing algorithms to obtain position information of an obstacle, information about a distance to an obstacle, or information about a relative speed with respect to the obstacle. For example, from the acquired image, the camera may acquire distance information and relative speed information with respect to the obstacle based on a change in the size of the obstacle over time.

The radar may use radio waves to generate information about obstacles outside the mobile robot 100. The radar may include an electromagnetic wave transmitter, an electromagnetic wave receiver, and at least one processor that is electrically connected to the electromagnetic wave transmitter and the electromagnetic wave receiver, processes a received signal, and generates obstacle information based on the processed signal.

The lidar may use laser light to generate information about obstacles outside the mobile robot 100. The radar may include at least one processor that is electrically connected to the optical transmitter, the optical receiver, and the optical transmitter and the optical receiver, processes a received signal, and generates obstacle information based on the processed signal.

The rider may be implemented in a Time of Flight (TOF) method or a phase-shift method. The lidar can be implemented either driven or non-driven. When implemented as a driving type, the radar is rotated by a motor, and obstacles around the mobile robot 100 may be detected. When implemented in a non-driven manner, the lidar may detect an obstacle located within a predetermined range with respect to the mobile robot 100 by optical steering.

The mobile robot 100 may include one or more non-driven lidars. The radar detects an obstacle, based on a time of flight (TOF) method or a phase-shift method, via laser light, and determines the position of the detected obstacle, the distance to the detected obstacle, and the relative speed. Can be detected.

In an embodiment of the present invention, the lidar may sense 270 degrees and may be a 2D lidar.

In an embodiment of the present invention, the obstacle detection sensor 160 may be disposed on the front side of the main body 110. Referring to FIG. 2B, the lidar 161, the RGBD camera 162, and the camera 163 may be disposed on the front side of the main body 110. The position and type of each sensor are not limited to FIG. 2B.

The communication device may exchange signals with devices located outside the mobile robot 100. The communication device may exchange signals with at least one of the external server 300 and other mobile robots. The communication device may include at least one of a transmission antenna, a reception antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

The controller 170 may generate SLAM-based map data by mapping the shopping space through the obstacle detection sensor 160. The controller 170 may receive a user input signal through the input device 200 and set a destination. The controller 170 may perform path-planning to a destination. The controller 170 may control the plurality of wheels 150 to allow the mobile robot 100 to autonomously travel along a planned route. A more detailed description will be given later through FIGS. 5 to 6.

The user detection sensor 180 is disposed on the handle 130 and may detect whether a user is holding the handle 130. The controller 170 may obtain information on whether a user is holding the handle 130 through the user detection sensor 180. When it is determined that the user is not holding the handle 130, the control unit 170 may generate a signal for stopping driving of the plurality of wheels 150.

Although not shown in the drawing, a user detection sensor may be disposed on the rear portion of the main body 110. In this case, the user detection sensor may be any one of a camera, an ultrasonic sensor, or an infrared sensor. The user detection sensor disposed on the rear portion of the main body 110 may measure a distance to the user and detect whether the distance to the user falls by a predetermined value or more. When the controller 170 acquires information that the distance to the user has fallen by a predetermined value or more through a user detection sensor disposed on the rear portion of the main body 110, the mobile robot 100 may stop driving.

Although not shown in the drawing, an audio sensor may be attached to the main body 110. The audio sensor may collect voice data formed in the surrounding environment of the mobile robot 100. The audio sensor may transmit the collected voice data to the external server 300 through a communication device.

3A to 3C are diagrams illustrating an input device of a mobile robot according to an exemplary embodiment of the present invention from various angles.

3A to 3C, the input device 200 may include a camera 210, a wearing unit 220, a button 230, a speaker 240, and a microphone 250.

The input device 200 may include a camera 210 disposed on one side of the input device 200 and capturing an image in a user's line of sight. The arrangement direction of the camera 210 may be changed according to the angle of view of the camera. The camera 210 may be disposed toward the opposite side of the user's face by a predetermined angle so that the user's face is not visible in the image captured by the camera 210.

The camera 210 may generate information about an obstacle outside the mobile robot 100 by using an image. The camera 210 may generate obstacle information in the direction of the user's gaze by using the image. The camera 210 may include at least one lens, at least one image sensor, and at least one processor that is electrically connected to the image sensor and processes a received signal, and generates obstacle information based on the processed signal. have.

The camera 210 may transmit an image captured to the external server 300 through a communication device. The camera 210 may provide a streaming image for reproducing an image captured by the external server 300 through a communication device in real time.

Together with the camera 210 of the input device 200, the camera of the obstacle detection sensor 160 may transmit the captured image to the external server 300 through a communication device. In this case, the camera 210 of the input device 200 may be referred to as a first camera, and the camera of the obstacle detection sensor 160 may be referred to as a second camera.

The guide may receive a streaming image captured by at least one of the first camera or the second camera through the external server 300 and may guide the user with information on the product. In this case, the remote computing device 350 may be used.

According to an embodiment of the present invention, the remote computing device 350 may include a display screen. The remote computing device 350 may include a display for viewing an image captured by at least one of the first camera and the second camera.

The remote computing device 350 may control the speed and steering of the mobile robot 100 through a communication network, and may adjust the direction of at least one of the first camera and the second camera and the magnification of the lens. .

The wearing unit 220 may be used to wear the input device 200 on the user's ear. The wearing part 220 may be formed in a curved shape so that it can be worn on a person's ear.

There may be one or more buttons 230. The button 230 may be an on/off button of a power supply device. The button 230 may be a start/end button for communication with the external server 300. One button 230 may be an on/off button of the power supply and a start/end button for communication with the external server 300. In this case, the on/off function of the power supply and the communication start/end function with the external server 300 can be distinguished through the pressing time of the button 230.

The speaker 240 may be for generating an audio-related output. The speaker 240 may convert an electrical signal provided from the control unit 170 into an audio signal and output it. The microphone 250 is for receiving information from a user, and data collected by the microphone 250 may be processed as a user's control command.

The controller 170 may receive a user input signal through the microphone 250 and transmit a signal guiding the user to the planned route to the speaker 240. The controller 170 may receive a guide signal from the external server 300 and output it to the speaker 240.

The control unit 170 may output a control signal of the mobile robot 100 to a user through the speaker 240 as a voice. The controller 170 may receive information on a product from the external server 300 and generate a signal that is output through the speaker 240. The information on the product may include detailed location information, image information, and price information on the product.

4 is a control block diagram of a mobile robot according to an embodiment of the present invention.

Referring to FIG. 4, the mobile robot 100 includes a control unit 170, a power supply unit 400, a communication unit 410, a memory 420, an interface 430, a sensing unit 440, and an input/output unit ( 450) and a driving unit 460 may be included.

The control unit 170 is electrically connected to the power supply unit 400, the communication unit 410, the memory 420, the interface 430, the sensing unit 440, the input/output unit 450, and the driving unit 460. You can exchange signals. The control unit 170 includes a mobile robot 100, a power supply unit 400, a communication unit 410, a memory 420, an interface 430, a sensing unit 440, an input/output unit 450, and a driving unit 460. By controlling the like, the overall operation of the robot 100 can be controlled.

The control unit 170 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, and controllers. It may be implemented using at least one of (controllers), micro-controllers, microprocessors, and electrical units for performing other functions.

The control unit 170 may be driven by power provided from the power supply unit 400. The control unit 170 may receive data, process data, generate a signal, and provide a signal while power is supplied by the power supply unit 400.

The controller 170 may receive information from another electronic device in the mobile robot 100 through the interface 430. The controller 170 may provide a control signal to another electronic device in the mobile robot 100 through the interface 430.

The power supply unit 400 may supply power to an electronic device including the control unit 170. The power supply unit 400 may receive power from a power source (eg, a battery) included in the mobile robot 100 and supply power to each unit of the electronic device. The power supply unit 400 may be implemented as a switched-mode power supply (SMPS).

The electronic device may include at least one printed circuit board (PCB). The control unit 170, the power supply unit 400, the communication unit 410, the memory 420, the interface 430, the sensing unit 440, the input/output unit 450 and the driving unit 460 are on a printed circuit board. Can be electrically connected.

The communication unit 410 may include at least one communication module to allow the mobile robot 100 to be connected to the Internet or a predetermined network, and to communicate with other devices. In addition, the communication unit 410 may process data transmission/reception between the mobile robot 100 and the external server 300 by connecting with a communication module provided in the external server 300.

The mobile robot 100 may include an internal communication system. A plurality of electronic devices included in the mobile robot 100 may exchange signals through an internal communication system. Signals may contain data. The internal communication system may use at least one communication protocol (eg, RS232, CAN, LIN, FlexRay, MOST, Ethernet).

The memory 420 is electrically connected to the control unit 170. The memory 420 may store basic data for a unit, control data for controlling the operation of the unit, and input/output data. The memory 420 may store data processed by the controller 170.

The memory 420 records various types of information necessary for control of the mobile robot 100 and may include a volatile or nonvolatile recording medium. A recording medium stores data that can be read by a micro processor, and is a hard disk drive (HDD), solid state disk (SSD), silicon disk drive (SDD), ROM, RAM, CD-ROM, magnetic It may include a tape, a floppy disk, an optical data storage device, and the like.

The interface 430 may exchange signals with at least one electronic device provided in the mobile robot 100 by wire or wirelessly. The interface 430 may exchange signals with at least one of an obstacle detection sensor 160, a user detection sensor 180, an input device 200, an output device, and a communication device by wire or wirelessly. The interface 430 may be composed of at least one of a communication module, a terminal, a pin, a cable, a port, a circuit, an element, and a device.

The sensing unit 440 may include at least one of an obstacle detection sensor 160, a user detection sensor 180, and a state detection sensor.

The obstacle detection sensor 160 may include at least one of a camera, a radar, a lidar, an ultrasonic sensor, and an infrared sensor. The obstacle detection sensor 160 may provide obstacle information generated based on a sensing signal generated by the sensor to the controller 170.

The obstacle detection sensor 160 may further include an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, a PSD (Position Sensitive Device) sensor, a cliff detection sensor that detects the presence of a cliff on the floor in the driving area. .

The obstacle detection sensor 160 detects an object, particularly an obstacle, which exists in the traveling (movement) direction of the mobile robot 100 and transmits obstacle information to the controller 170. In this case, the controller 170 may control the movement of the mobile robot 100 according to the detected position of the obstacle.

The state detection sensor may detect the internal state of the mobile robot 100. The state detection sensor includes an IMU (inertial navigation unit) sensor, a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight detection sensor, a heading sensor, a position module, and forward/reverse. It may include at least one of a sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor, an internal temperature sensor, an internal humidity sensor, an ultrasonic sensor, an illuminance sensor, an acceleration sensor, and a brake sensor. Meanwhile, the inertial navigation unit (IMU) sensor may include one or more of an acceleration sensor, a gyro sensor, and a magnetic sensor.

The sensing unit 440 may generate mobile robot state information based on sensing data generated by at least one sensor. The mobile robot status information may be information generated based on data sensed by various sensors included in the mobile robot.

For example, the mobile robot status information may include attitude information, speed information, tilt information, weight information, direction information, battery information, fuel information, tire pressure information, steering information, indoor temperature information, and indoor humidity information. .

The input/output unit 450 may include an input unit and an output unit. The input unit may include a microphone, a button or a camera, and the output unit may include a speaker or a display.

The mobile robot 100 according to an embodiment of the present invention may further include a voice input unit for receiving a user's voice input through a microphone. The voice input unit may include a processing unit that converts analog sound into digital data or is connected to the processing unit to convert a user input audio signal into data so that the control unit 170 or the external server 300 can recognize it.

The memory 420 may store data for voice recognition, and the controller 170 may process a user's voice input signal received through the input/output unit 450 and perform a voice recognition process.

In addition, the speech recognition process may not be performed by the mobile robot 100 itself, but may be performed by the external server 300. In this case, the controller 170 may control the communication unit 410 to transmit a user input voice signal to the external server 300.

There may be one or more buttons 230. The button 230 may be an on/off button of a power supply device. The button 230 may be a start/end button for communication with the external server 300. One button 230 may be an on/off button of the power supply and a start/end button for communication with the external server 300. In this case, the on/off function of the power supply and the communication start/end function with the external server 300 can be distinguished through the pressing time of the button 230.

According to an embodiment, the mobile robot 100 may further include an image acquisition unit capable of capturing a predetermined range. The image acquisition unit photographs the surroundings of the mobile robot 100, the external environment, and the like, and may include a camera. A number of cameras may be installed for each part for shooting efficiency.

The mobile robot 100 may display predetermined information as an image or output it as sound. The input/output unit 450 may include a speaker that outputs information corresponding to a user's command input, a processing result corresponding to the user's command input, an operation mode, an operation state, an error state, and the like as a voice.

The output unit may further include an audio output unit that outputs an audio signal. The sound output unit outputs a warning sound, an operation mode, a notification message such as an operation state, an error state, etc., information corresponding to the user's command input, a processing result corresponding to the user's command input, etc. can do. The sound output unit may convert the electrical signal from the control unit 170 into an audio signal and output it. To this end, a speaker or the like may be provided.

The driving unit 460 may include at least one wheel that moves the main body 110 of the mobile robot 100. The driving unit 460 may include a plurality of wheels 150 disposed under the main body 110 and a plurality of motors disposed on and driving each of the plurality of wheels 150.

The plurality of wheels 150 may be provided on the left and right sides of the main body 110, respectively, hereinafter referred to as left and right wheels, respectively. It is possible to change the driving direction of the main body 110 to the left or right by making a difference in the rotation speed of the left wheel and the right wheel.

5 is a flow chart of a control unit according to an embodiment of the present invention.

Referring to FIG. 5, the control unit 170 includes a map data generation step (S500), a destination input step (S505), a step of determining whether the autonomous driving mode is in the autonomous driving mode (S510), a route planning step (S515), and an autonomous driving step. The autonomous driving of the mobile robot 100 may be controlled through (S520) and a destination arrival determination step (S525).

The map data generation step S500 may include the step of mapping the shopping space by the mobile robot 100. The mobile robot 100 may generate map data based on sensor data acquired through the sensing unit 440 while driving in a shopping space in a manual mode. In the autonomous driving mode, the mobile robot 100 may drive a shopping space and update map data based on sensor data acquired through the sensing unit 440.

The mobile robot 100 can generate SLAM-based map data through a process of searching the surroundings through the sensing unit 440 while driving in a shopping space in a manual mode or an autonomous driving mode, and generating a current location. have. The mobile robot 100 may store the generated map data in the memory 420.

The destination input step S505 may be a step of receiving destination information input by a user through the input device 200. The destination information may include voice information. The controller 170 may receive voice information and determine a destination.

The destination information may be voice information of a user about a product or a product corner. For example, the destination information may be sushi input by a user by voice. For example, the destination information may be a fruit corner input by a user by voice.

Determining whether it is the autonomous driving mode (S510) may be a step of determining whether the control unit 170 is in an autonomous driving mode or a manual mode. The user can select an autonomous driving mode or a manual mode. The autonomous driving mode or the manual mode can be selected through voice input or button input.

If the controller 170 determines that the autonomous driving mode is in the autonomous driving mode, the destination may be reached through a route planning step (S515) and an autonomous driving control step (S520). If the control unit 170 determines that it is in the manual mode, the destination may be reached through the step of connecting to the external server 300 (S530), the real-time video streaming step (S535), and the remote control step (S540).

The step of performing route planning (S515) includes generating a first route based on the stored map data (S601), controlling autonomous driving along the first route (S602), and calculating a cost value (S603). ), generating a second path based on the cost value (S604), and correcting the first path based on the second path (S605). A more detailed description will be given later with reference to FIG. 6.

The autonomous driving control step S520 may be a step in which the controller 170 controls the traveling of the mobile robot 100 along the path generated through the step S515 of performing a path planning. The control unit 170 may acquire data on surrounding obstacles through the sensing unit 440 while autonomously driving along the planned route.

The controller 170 may control a plurality of wheels of the mobile robot 100 along a planned route. The controller 170 may control the moving speed and steering of the mobile robot 100 so that the mobile robot 100 can travel along a planned route. The controller 170 may accelerate, brake, or adjust a direction of the mobile robot by controlling a plurality of wheels.

The control unit 170 may transmit a signal guiding the user of the planned route to the speaker. The signal guiding the route may include a signal guiding straight forward, left turn, right turn, and the like. For example, the control unit 170 may output a guide signal “Turn left in front of 5m” through the speaker along a planned route.

When it is determined that the user is not holding the handle through the user detection sensor disposed on the handle, the control unit 170 may generate a signal for stopping driving of the plurality of wheels. Accordingly, it is possible to control the mobile robot 100 so that it does not fall away from the user while autonomously driving, thereby improving the safety of a user who is a blind person.

The destination arrival determination step S525 may be a step in which the controller 170 completes the driving of the mobile robot 100 along a planned route and determines whether or not the input destination has been reached. When determining that the controller 170 has arrived at the destination, the controller 170 may output a destination arrival completion signal through the speaker. In addition, a signal asking whether to communicate with an external server (eg, a guide server) may be output through a speaker.

The user may start communication through a communication start/end button with the external server 300. When receiving the communication start signal, the controller 170 may transmit an image captured by the camera of the input device to the external server 300 through the communication device. The controller 170 may stream an image captured by the camera in real time.

The controller 170 may receive information on a product from the external server 300 and output it to a user through a speaker. The information on the product may include detailed location information, image information, and price information on the product.

In the step of determining whether the autonomous driving mode is in the autonomous driving mode (S510), if the control unit 170 determines that the manual mode is in the manual mode, the connection with the external server 300 (S530), the real-time video streaming step (S535), and the remote control step (S540) You can reach your destination via.

The step of connecting with the external server 300 (S530) may be a step of communicating with the external server 300 through a communication start/end button when the autonomous driving mode is not. When the user selects the manual mode, the autonomous driving mode, but lacks destination information, the mobile robot 100 may connect with the external server 300 when the number of occurrences of an obstacle is greater than a certain value. That is, the mobile robot 100 may communicate with the external server 300 and exchange data.

The real-time image streaming step (S535) may be a step of streaming a user's gaze direction image captured by the camera 210 of the input device 200 in real time. The real-time image may be transmitted to the remote computing device 350 through the external server 300. The guide may check a real-time image through the remote computing device 350 and provide a guide service.

The real-time image may be an image captured by the camera of the obstacle detection sensor 160 together with the camera 210 of the input device 200. In this case, the camera 210 of the input device 200 may be referred to as a first camera, and the camera of the obstacle detection sensor 160 may be referred to as a second camera.

The guide may receive a streaming image captured by at least one of the first camera or the second camera through the external server 300 and may provide a guide service to a destination. In this case, the remote computing device 350 may be used.

According to an embodiment of the present invention, the remote computing device 350 may include a display. The remote computing device 350 may include a display for viewing an image captured by at least one of the first camera and the second camera.

The remote control step S540 may be a step of remotely controlling the driving of the mobile robot 100 while the guide sees the streaming image captured by the camera through the display. The guide may input a voice guidance signal for a driving direction and a driving speed while controlling the driving of the mobile robot through the remote computing device 350. The guide may adjust the direction of at least one of the first camera and the second camera and the magnification of the lens through the remote computing device 350.

6 is a diagram for explaining a route planning step according to an embodiment of the present invention.

Referring to FIG. 6, the step of performing route planning (S515) includes generating a first route based on the stored map data (S601), controlling autonomous driving along the first route (S602), and a cost value. It may include calculating (S603), generating a second path based on the cost value (S604), and correcting the first path based on the second path (S605).

The step of generating the first route (S601) may be a step of generating a first route, which is an optimal route to a destination, based on the stored map data. The first path may be a global path generated by the controller 170 based on map data generated through the sensing unit 440. The controller 170 may control the mobile robot 100 to autonomously travel along the first path (S602). In this case, the control unit 170 may determine the driving speed and the driving direction of the mobile robot 100 by controlling the driving unit 460.

The step of calculating the cost value (S603) may be a step of calculating a cost value based on a sensor data value obtained through the sensing unit 440 while the control unit 170 travels along the first route. The cost value may vary depending on the type of obstacle, a relative speed with the mobile robot 100, a relative distance with the mobile robot 100, and the like. The controller 170 may calculate a cost value based on obstacle information obtained through the obstacle detection sensor 160.

The step of generating the second path (S604) may be a step of generating a second path in a direction with a low cost value based on the cost value. The first path may be a local path generated by the controller 170 based on a cost value calculated through the obstacle detection sensor 160. The controller 170 may correct the first path based on the second path (S605).

The controller 170 may control autonomous driving of the mobile robot 100 along the first path corrected based on the second path (S520).

7 is a diagram for explaining a first route and a second route according to an embodiment of the present invention.

Referring to FIG. 7, the controller 170 may correct the first path 702 based on the second path 703.

The controller 170 may control driving along the first route 702 from the current position 701 of the mobile robot 100 to the destination. The controller 170 may generate a second path based on obstacle information while driving the first path. The sensing unit 440 may detect an obstacle around the mobile robot 100 while driving the first path and generate obstacle information.

In the case of FIG. 7, a situation in which an obstacle 704 is detected while moving along the first path 702 may be indicated. The control unit 170 may calculate a cost value according to the obstacle information and generate a second path 703 based on the cost value. The controller 170 may modify the first path 702 through the second path 703. The modified first path may be a path merged with the previously generated first path 702 through the second path avoiding the obstacle 704.

As can be seen from FIG. 7, the mobile robot 100 autonomously travels through the optimal route to the destination based on the first route 702 based on map data, while driving the second route 703 based on sensor data. Obstacles can be avoided as a basis.

At least one electronic device included in the mobile robot 100 may include a software module or a hardware module (hereinafter, referred to as an artificial intelligence module) that implements artificial intelligence (AI). At least one electronic device may input acquired data to an artificial intelligence module and use data output from the artificial intelligence module.

The artificial intelligence module may perform machine learning on input data using at least one artificial neural network (ANN). The artificial intelligence module may output driving plan data through machine learning on input data.

At least one electronic device may generate a control signal based on data output from the artificial intelligence module.

At least one electronic device included in the mobile robot 100 may receive data processed by artificial intelligence from an external server through a communication device. At least one electronic device included in the mobile robot 100 may generate a control signal based on data processed by artificial intelligence.

The present invention described above can be implemented as a computer-readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices storing data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. There is also a carrier wave (for example, transmission over the Internet) also includes the implementation of the form. In addition, the computer may include a processor or a control unit. Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: mobile robot
110: main body
120: support
130: handle
140: storage unit
150: multiple wheels
160: obstacle detection sensor
170: control unit
180: user detection sensor
200: input device
210: camera
220: wearing part
230: button
240: speaker
250: microphone
300: external server
350: remote computing device

Claims (5)

In a mobile robot that drives autonomously and helps the visually impaired to shop,
A body that guides the user to a destination;
An input device mounted on the user and wirelessly connected to the main body;
A support installed on the main body;
A handle connected to the support;
A storage unit disposed on the support and movable along the support;
At least one obstacle detection sensor disposed on the main body;
A plurality of wheels disposed under the main body;
A plurality of motors driving the plurality of wheels;
A power supply for providing electric power; And
Including; a control unit for controlling the autonomous driving of the mobile robot,
The input device further includes a camera for photographing an image of a user's gaze direction,
The control unit maps a shopping space through the obstacle detection sensor to generate SLAM-based map data, receives an input signal input by a user through the input device, sets a destination, and plans a route to the destination. -Planning) and controlling the plurality of wheels to allow the mobile robot to autonomously travel along a planned route. The method of claim 1,
The control unit,
When the destination is set, a first route that is an optimal route to the destination is generated based on the map data,
A cost value is calculated based on obstacle information detected by the obstacle detection sensor while driving along the first path,
Create a second path to avoid the obstacle in the direction of the low cost value,
A mobile robot for the visually impaired, which performs the route planning by correcting the first route through the second route. The method of claim 1,
The mobile robot,
A user detection sensor disposed on the handle and detecting whether a user is holding the handle; further comprising,
The control unit,
Obtaining information on whether a user is holding the handle through a user detection sensor disposed on the handle,
If it is determined that the user is not holding the handle, generating a signal for stopping driving of the plurality of wheels, a mobile robot for the visually impaired. The method of claim 1,
The mobile robot.
A communication device that communicates with an external server; further includes,
The input device,
Wearing parts mounted on the user's ears; MIC; speaker; And one or more buttons; further comprising,
Any one button of the one or more buttons is at least one of an on/off button of the power supply device or a communication start/end button of the external server through the communication device,
The control unit,
A mobile robot for the visually impaired, which receives the input signal of the user through the microphone and transmits a signal guiding the user to the planned route to the speaker. The method of claim 4,
The control unit,
When a communication start signal is received through a communication start/end button with the external server, a signal for real-time streaming of the image captured by the camera is transmitted to the external server through the communication device,
Generates a signal to receive information on the product from the external server and output through the speaker,
Information on the above products,
A mobile robot for the visually impaired, including detailed location information, image information, and price information on the product.

Images