KR102508073B1 - A moving-robot and control method thereof - Google Patents

Abstract

The present invention includes a sensing unit for obtaining image information around the mobile robot; a control unit generating a control signal using the image information; and a communication unit for transmitting the control signal to a terminal connected to the mobile robot through a wired/wireless communication method and receiving a terminal signal from the terminal, wherein the control unit extracts characteristic information of an unidentified user from the image information, It is characterized in that the intruder is determined based on the characteristic information.

Description

A moving robot and its control method {A MOVING-ROBOT AND CONTROL METHOD THEREOF}

The present invention relates to security-related technologies for mobile robots.

Robots have been developed for industrial use and have been in charge of a part of factory automation. In recent years, the field of application of robots has been further expanded, and medical robots, space robots, etc. have been developed, and household robots that can be used at home are also being made. Among these robots, those capable of driving on their own are called mobile robots. A typical example of a mobile robot used at home is a robot vacuum cleaner.

Various technologies are known for sensing the environment around the mobile robot and the user through various sensors provided in the mobile robot. In addition, technologies are known in which a mobile robot learns and maps a driving area by itself and identifies a current location on a map.

In particular, patent literature discloses that a robot cleaner captures a surrounding environment with a camera, recognizes the surrounding environment with a sensor, avoids surrounding obstacles, and performs cleaning at a scheduled time.

However, in the prior art, there are problems in that the robot cleaner simply performs functions limited to cleaning, does not have a function to detect external intrusion while the user is absent, and does not have a means for informing the user of external intrusion.

Korean Patent Publication No. 20190015930

An object to be solved by the present invention is that a robot cleaner detects an outside intrusion in a user's house while the user is absent, and informs the user of this.

Another problem to be solved by the present invention is that, when there is an outside intrusion in the user's house during the user's absence, the user recognizes this and transmits a command for generating an alarm to each home appliance in the user's home.

Another problem to be solved by the present invention is to provide a robot cleaner that recognizes a person's face to determine whether the person is a registered user, accurately determines the intrusion of an outsider, and provides face information of the intruder to the user.

The present invention is characterized in that the robot cleaner recognizes a person's face and determines whether it is a legitimate user or an intruder.

Specifically, a sensing unit for obtaining image information around the mobile robot; a control unit generating a control signal using the image information; and a communication unit for transmitting the control signal to a terminal connected to the mobile robot through a wired/wireless communication method and receiving a terminal signal from the terminal, wherein the control unit extracts characteristic information of an unidentified user from the image information, It is possible to determine whether an intruder is based on the characteristic information.

The controller detects a change in the image information, detects whether a person exists through the image information when the amount of change is equal to or greater than a predetermined reference value, and extracts the characteristic information when it is determined that a person exists.

The control unit may detect the existence of a person through the image information and extract the characteristic information when it is determined that a person exists.

The control unit may extract the unconfirmed user characteristic information through the characteristic information and compare the extracted unconfirmed user characteristic information with pre-registered user characteristic information to determine the intruder.

If the similarity between the unverified user characteristic information and pre-registered user characteristic information is less than a preset reference value, the control unit determines the unidentified user as the intruder, and determines the similarity between the unverified user characteristic information and pre-registered user characteristic information. If is equal to or greater than a preset reference value, the unidentified user may be determined as a registered user.

If the controller determines that the unidentified user is an intruder, the controller may control the terminal to output a screen notifying the intruder.

When it is determined that the unidentified user is an intruder, the controller may control the terminal to output a control screen including characteristic information of the unidentified user.

The control unit may control an icon for receiving the alarm generating command to be output together with the image information on the control screen.

When it is determined that the unidentified user is an intruder, the control unit may extract an alarm generation command from the terminal signal and control the mobile robot according to the alarm generation command.

The present invention further includes an output unit that outputs a visually or audibly perceptible alarm, and the control unit may control the output unit to output an alarm when it is determined that the unidentified user is an intruder.

The communication unit transmits the control signal to a home appliance connected to the mobile robot through a wired/wireless communication method, and the control unit extracts an alarm generating command from the terminal signal when it is determined that the unidentified user is an intruder, and generates the alarm. At least one of the mobile robot and the home appliance may be controlled according to a command.

The control unit may control the home appliance to output an alarm when receiving the alarm generation command.

The present invention further includes a storage unit for storing data, and the control unit may receive pre-registered user characteristic information from the terminal and store it in the storage unit.

The present invention may further include a storage unit for storing data, and the control unit may extract characteristic information from the image information when a user input command is input, and store the user's characteristic information in the storage unit.

When a home guard command is input, the control unit may extract characteristic information of an unidentified user from the video information and determine whether the user is an intruder based on the characteristic information.

The characteristic information may include at least one of gender information, age information, and face information of the user.

Through the above solution, the present invention can detect a dangerous situation in the house by utilizing various sensors of the robot cleaner without adding other separate equipment when the user is absent, and the user can recognize the house state. There are advantages to being able to.

In addition, according to the present invention, the robot cleaner can report to the user when an outsider intrudes into the home, and provides an image of the situation when the outsider intrudes to the user, so that the user can see the situation and recognize the dangerous situation. There is an advantage of recognizing and issuing a command corresponding to a dangerous situation to the home appliance.

In addition, the present invention has an advantage of effectively distinguishing an intruder because it is possible to determine whether the intruder is an intruder by detecting the face of an intruder entering the home in the absence of a user.

In addition, the present invention transmits an image capable of identifying the intruder's face to the user's terminal in a situation where someone intrudes from outside in the absence of the user, so the user can determine whether to generate an alert by looking at the intruder's face. this exists

In addition, since the present invention identifies an intruder by recognizing a person's face, there is an advantage of registering a plurality of users to reduce unnecessary warnings or notifications to users when family members and their acquaintances enter the home.

1 is a perspective view showing a mobile robot 100 and a charging station 200 for charging the mobile robot according to an embodiment of the present invention.
FIG. 2 is an elevational view of the mobile robot 100 of FIG. 1 viewed from above.
FIG. 3 is an elevational view of the mobile robot 100 of FIG. 1 viewed from the front.
4 is an elevational view of the mobile robot 100 of FIG. 1 viewed from the lower side.
FIG. 5 is a block diagram showing a control relationship between major elements of the mobile robot 100 of FIG. 1 .
FIG. 6 is a conceptual diagram illustrating a network of the mobile robot 100 and the terminal 300 of FIG. 1 .
7 is a conceptual diagram illustrating an example of the network of FIG. 6 .
8 is a conceptual diagram illustrating a control method of a mobile robot according to an embodiment of the present invention.
9 is a diagram illustrating a control screen of a terminal according to an embodiment of the present invention.
10 is a flowchart illustrating a control method of a mobile robot according to another embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

The spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe components and their correlations with other components. Spatially relative terms should be understood as encompassing different orientations of elements in use or operation in addition to the orientations shown in the drawings. For example, if you flip a component shown in the drawing, a component described as "below" or "beneath" another component will be placed "above" the other component. can Thus, the exemplary term “below” may include directions of both below and above. Elements may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" means that a stated component, step and/or operation excludes the presence or addition of one or more other components, steps and/or operations. I never do that.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used in a meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

In the drawings, the thickness or size of each component is exaggerated, omitted, or schematically illustrated for convenience and clarity of explanation. Also, the size and area of each component do not entirely reflect the actual size or area.

Hereinafter, with reference to the accompanying drawings, the preferred embodiments of the present invention are described as follows.

The mobile robot 100 according to the present invention refers to a robot capable of moving by itself using wheels and the like, and may be a home helper robot and a robot vacuum cleaner.

Hereinafter, with reference to FIGS. 1 to 5, the robot cleaner 100 among mobile robots will be described as an example, but it is not necessarily limited thereto.

The mobile robot 100 includes a body 110. Hereinafter, in defining each part of the main body 110, the part facing the ceiling in the driving area is defined as the upper part (see FIG. 2), and the part facing the floor in the driving area is defined as the bottom part (see FIG. 4). And, of the part forming the circumference of the body 110 between the upper and lower surfaces, the part facing the driving direction is defined as the front part (see FIG. 3). In addition, a portion facing the opposite direction to the front portion of the main body 110 may be defined as a rear portion. The main body 110 may include a case 111 forming a space in which various parts constituting the mobile robot 100 are accommodated.

The mobile robot 100 includes a sensing unit 130 that senses external information of the mobile robot 100 .

The sensing unit 130 may detect information about a driving area. The sensing unit 130 may detect obstacles such as walls, furniture, and cliffs on the driving surface. The sensing unit 130 may detect information about the ceiling. The sensing unit 130 may detect an object placed on the driving surface. Through information sensed by the sensing unit 130, the mobile robot 100 may map a driving area. The sensing unit 130 may be implemented with at least one sensor. The sensing unit 130 detects intrusion detection information using a sensor. Specifically, the sensing unit 130 detects image information around the mobile robot.

Here, the intrusion detection information may include all information capable of detecting an external intrusion. For example, the intrusion detection information may include image information and sound information around the mobile robot, image information and sound information around the front door and windows. Of course, the intrusion detection information may include image information and sound information around home appliances connected to the mobile robot through a network.

For example, the sensing unit 130 may include an image sensing unit 135 that detects (acquires) an external image (surroundings of the mobile robot). The image detector 135 captures the driving area and may include a digital camera. The digital camera is an image sensor (eg, CMOS image sensor) including at least one optical lens and a plurality of photodiodes (eg, pixels) forming images by light passing through the optical lens. and a digital signal processor (DSP) that composes an image based on the signals output from the photodiodes. The digital signal processor can generate not only still images, but also moving images composed of frames composed of still images.

The sensing unit 130 may include a distance sensing unit 131 that detects a distance to an obstacle. The distance sensor 131 may be disposed on the front of the main body 110 . The distance detector 131 may detect an obstacle in front. A plurality of distance detectors 131 may be provided. The distance sensor 131 may be implemented using ultrasonic waves or infrared rays. The distance detection unit 131 may be implemented using a camera.

The sensing unit 130 may include a cliff detection unit 132 that detects whether a cliff exists on the floor in the driving area. A plurality of cliff detection units 132 may be provided.

In addition, the sensing unit 130 may include an illuminance sensor (not shown) that detects the illuminance (level of brightness) of the driving area. A plurality of cliff detection units 132 may be provided.

The sensing unit 130 may further include a lower image sensor 137 that acquires an image of the floor.

The mobile robot 100 includes a driving unit 160 that moves the body 110. The driving part 160 moves the main body 110 with respect to the floor. The driving unit 160 may include at least one driving wheel 166 for moving the main body 110 . The driving unit 160 may include a driving motor. The driving wheels 166 may be provided on the left and right sides of the main body 110, respectively, and are hereinafter referred to as a left wheel 166(L) and a right wheel 166(R), respectively.

The left wheel 166(L) and the right wheel 166(R) may be driven by a single drive motor, but the left wheel drive motor and the right wheel 166(R) driving the left wheel 166(L) as needed ) may be provided respectively with a right wheel drive motor for driving. The driving direction of the main body 110 can be switched to the left or right by making a difference between the rotation speeds of the left wheel 166 (L) and the right wheel 166 (R).

The mobile robot 100 includes a work unit 180 that performs a predetermined task. The work unit 180 generates noise and performs a predetermined task. For example, noise is generated to rotate the brush for sweeping, and noise is generated to operate a suction motor for vacuum suction.

For example, the work unit 180 may be provided to perform household tasks such as cleaning (sweeping, suction cleaning, mopping, etc.), dishwashing, cooking, laundry, garbage disposal, and the like. As another example, the work unit 180 may be provided to perform tasks such as manufacturing or repairing tools. As another example, the work unit 180 may perform tasks such as finding objects or expelling insects.

As another example, the work unit 180 may perform a security function of detecting a surrounding external intruder or dangerous situation. The work unit 180 may include a camera for performing such a security function, and in this case, it is also possible for the image detector 135 to perform the function of a camera that performs a security function.

Hereinafter, in this embodiment, it is described that the work unit 180 performs cleaning work, but the type of work of the work unit 180 may have various examples, and there is no need to be limited to the example of this description.

The mobile robot 100 may clean the floor by the work unit 180 while moving in a driving area. The work unit 180 includes a suction device for sucking foreign substances, brushes 184 and 185 for sweeping, a dust bin (not shown) for storing foreign substances collected by the suction device or brush, and/or a mop for wiping. A part (not shown) may be included.

A suction hole 180h through which air is sucked may be formed on the bottom of the main body 110 . In the main body 110, a suction device (not shown) that provides suction power so that air can be sucked through the suction hole 180h, and a dust bin (not shown) that collects dust sucked together with the air through the suction hole 180h. may be provided.

An opening for insertion and removal of the dust bin may be formed in the case 111 , and a dust box cover 112 that opens and closes the opening may be rotatably provided with respect to the case 111 .

A roll-type main brush 184 having brushes exposed through the suction port 180h, and an auxiliary brush 185 located on the front side of the bottom surface of the main body 110 and having a brush made of a plurality of wings extending radially may be provided. The rotation of the brushes 184 and 185 removes dust from the floor in the driving area, and the dust separated from the floor is sucked through the suction port 180h and collected in the dust bin.

The battery 177 may supply not only the driving motor, but also power necessary for the overall operation of the mobile robot 100.

The battery 177 is provided to be rechargeable. The mobile robot 100 may be docked to the charging station 200 and the battery 177 may be charged through connection between the charging terminal 210 and the corresponding terminal 190 . When the charge amount of the battery 177 is less than or equal to a predetermined value, the mobile robot 100 may return to the charging stand 200 for charging. 200) can be detected.

The charging station 200 may include a signal transmission unit (not shown) that transmits a predetermined return signal. The return signal may be an ultrasonic signal or an infrared signal, but is not necessarily limited thereto.

The mobile robot 100 includes a communication module 170 that receives or receives information. The communication module 170 may output or transmit information. The communication module 170 may include a communication unit 175 that transmits and receives information with other external devices. The communication module 170 may include an input unit 171 for inputting information. The communication module 170 may include an output unit 173 that outputs information.

For example, the mobile robot 100 may directly receive information from the input unit 171. As another example, the mobile robot 100 may receive information input to a separate terminal 300 through the communication unit 175 .

For example, the mobile robot 100 may directly output information to the output unit 173. As another example, the mobile robot 100 may transmit information to a separate terminal 300 through the communication unit 175 so that the terminal 300 outputs the information.

The input unit 171 may receive On/Off or various commands. The input unit 171 may include a button, key, or touch type display. The input unit 171 may include a microphone for voice recognition. Although not shown, information on whether the home guard mode to be described later is activated, home guard mode setting information, whether a specific function to be described later is activated, and/or various setting information of the specific function is input through the input unit 171. may receive

The output unit 173 may inform the user of various types of information. The output unit 173 outputs a visually or audibly perceptible alarm. The output unit 173 may include at least one of a speaker, a light source, and a display. Of course, such an output unit may also be installed in a home appliance to be described later.

The communication unit 175 may transmit/receive data by being connected to a terminal and/or another electronic device located in a specific area through one of wired, wireless, and satellite communication methods.

The communication unit 175 may be provided to communicate with other electronic devices 300 , 310 , 320 such as the terminal 300 , the wireless router 400 and/or the server 500 . The communication unit 175 can communicate with other devices located within a specific area. The communication unit 175 may communicate with the wireless sharer 400 . The communication unit 175 may communicate with the mobile terminal 300 . The communication unit 175 may communicate with the server 500 . The communication unit 175 may transmit a control signal to a terminal or home appliance.

The communication unit 175 may receive various command signals (terminal signals) from an external device such as the terminal 300 . The communication unit 175 may transmit information to be output to an external device such as the terminal 300 . The terminal 300 may output information received from the communication unit 175 .

Referring to Ta of FIG. 7 , the communication unit 175 may perform wireless communication with the wireless sharer 400 . Referring to Tc of FIG. 7 , the communication unit 175 may perform wireless communication with the mobile terminal 300a. Although not shown, the communication unit 175 may directly communicate with the server 500 wirelessly. For example, the communication unit 175 may be implemented for wireless communication using wireless communication technologies such as IEEE 802.11 WLAN, IEEE 802.15 WPAN, UWB, Wi-Fi, Zigbee, Z-wave, and Blue-Tooth. The communication unit 175 may vary depending on the communication method of another device or server to be communicated with.

Information may be received from the mobile robot 100 on the network through the communication unit 175, and the mobile robot 100 may be controlled based on the received information. For example, the communication unit 175 may receive various types of information from the terminal 300 . The communication unit 175 may receive information input from the terminal 300 such as a smartphone or a computer.

Information on the mobile robot 100 may be transmitted over a network through the communication unit 175 . For example, the communication unit 175 may transmit information to be output to the terminal 300 . The terminal 300 may output information received from the communication unit 175 .

Referring to FIG. 6 , the mobile robot 100 may communicate with the terminal 300 through a predetermined network. The communication unit 175 communicates with the terminal 300 through a predetermined network. A predetermined network means a communication network connected directly or indirectly by wire and/or wirelessly. That is, 'the communication unit 175 communicates with the terminal 300 through a predetermined network' means that the communication unit 175 and the terminal ( 300) is meant to cover the case of indirect communication via the wireless sharing device 400 and the like.

The network may be built based on technologies such as Wi-Fi, Ethernet, zigbee, z-wave, and Bluetooth.

The communication unit 175 may transmit information to be output to the terminal 300 through a predetermined network. The terminal 300 may transmit various types of information (eg home guard mode setting information) to the communication unit 175 through a predetermined network.

The mobile robot 100 may be connected to a plurality of electronic devices through a network or directly. Here, the plurality of electronic devices may include at least one of home appliances such as a washing machine 310, an air conditioner 320, a clothes handling device, a door lock, a refrigerator, a dryer, and a water purifier.

7 is a conceptual diagram illustrating an example of a predetermined network. The mobile robot 100, the wireless sharing device 400, the server 500, and the mobile terminals 300a and 300b are connected by a network and can transmit and receive information to each other. Among them, the mobile robot 100, the wireless router 400, the mobile terminal 300a, and the like may be disposed in a building 10 such as a house. The server 500 may be implemented within the building 10, but may also be implemented outside the building 10 as a more extensive network.

The wireless router 400 and the server 500 may include a communication module capable of accessing a network according to a predetermined communication protocol. The communication unit 175 of the mobile robot 100 is provided to be able to connect to a network according to a predetermined communication protocol.

The mobile robot 100 may exchange data with the terminal 300 through a network. The communication unit 175 may exchange data with the wireless sharing device 400 wired or wirelessly, and consequently exchange data with the terminal 300 .

The mobile robot 100 may exchange data with the server 500 through a network. The communication unit 175 may exchange data with the wireless router 400 wired or wirelessly, and consequently exchange data with the server 500 .

Referring to Ta of FIG. 7 , the wireless sharer 400 may be wirelessly connected to the mobile robot 100 . Referring to Tb of FIG. 7 , the wireless router 400 may be connected to the server 8 through wired or wireless communication. Through Td of FIG. 7 , the wireless router 400 may be wirelessly connected to the mobile terminal 300a.

Meanwhile, the wireless router 400 may allocate a wireless channel according to a predetermined communication method to electronic devices within a predetermined area, and perform wireless data communication through the corresponding channel. Here, the predetermined communication method may be a WiFi communication method.

The wireless router 400 may communicate with the mobile robot 100 located within a predetermined range. The wireless router 400 can communicate with the mobile terminal 300a located within a predetermined area range. The wireless sharer 400 may communicate with the server 500 .

The server 500 may be provided to be accessible through the Internet. It is possible to communicate with the server 500 through various terminals 300b connected to the Internet. The terminal 300b may be a mobile terminal such as a personal computer (PC) or a smart phone.

Referring to Tb of FIG. 7 , the server 500 may be connected to the wireless router 400 wired or wirelessly. Referring to Tf of FIG. 7 , the server 500 may be directly wirelessly connected to the mobile terminal 300b. Although not shown, the server 500 may communicate directly with the mobile robot 100.

The server 500 includes a processor capable of processing programs. The functions of the server 500 may be performed by a central computer (cloud) or may be performed by a user's computer or a mobile terminal.

For example, the server 500 may be a server operated by a manufacturer of the mobile robot 100. As another example, the server 500 may be a server operated by a public application store operator. As another example, the server 500 may be a home server that is provided in the home and stores state information about home appliances or content shared by the home appliances.

The server 500 may store information and driving information (course information, etc.) on the mobile robot 100 and register product information on the mobile robot 100.

Referring to Td of FIG. 7 , the mobile terminal 300a may be wirelessly connected to the wireless router 400 through wi-fi or the like. Referring to Tc of FIG. 7 , the mobile terminal 300a may be directly wirelessly connected to the mobile robot 100 through Bluetooth or the like. Referring to Tf of FIG. 7 , the mobile terminal 300b may be directly wirelessly connected to the server 500 through a mobile communication service.

The terminal 300 may store information and driving information (course information, etc.) on the mobile robot 100 and register product information on the mobile robot 100.

The network may further include a gateway (not shown). The gateway may mediate communication between the mobile robot 100 and the wireless sharer 400 . The gateway may communicate with the mobile robot 100 wirelessly. The gateway may communicate with the wireless sharer 400 . For example, communication between the gateway and the wireless router 400 may be based on Ethernet or Wi-Fi.

Referring to FIG. 5 , the communication module 170 may receive or receive information on whether the home guard mode is active (whether On/Off). The communication module 170 may receive or receive home guard mode setting information. The communication module 170 may receive or receive information about the setting time of the home guard mode. The communication module 170 may receive or receive information about a specific function to be restricted in the home guard mode. The communication module 170 may receive or receive information about a mode function to be activated in the home guard mode. The communication module 170 may receive or receive information about a home guard mode setting area.

In addition, the communication module 170 may receive or receive input of whether a specific function is activated and various setting information of the specific function, which will be described later.

The mobile robot 100 includes a control unit 140 that processes and determines various information such as mapping and/or recognizing a current location. The control unit 140 may control the overall operation of the mobile robot 100 through control of various elements constituting the mobile robot 100 . The controller 140 may map a driving area through an image and recognize a current location on a map. That is, the controller 140 may perform a Simultaneous Localization and Mapping (SLAM) function.

The controller 140 may receive and process information from the communication module 170 . The controller 140 may receive and process information from the input unit 171 . The control unit 140 may receive and process information from the communication unit 175 . The control unit 140 may receive and process information detected by the sensing unit 130 .

The control unit 140 may give information to the communication module 170 for output. The controller 140 may give information to the communication unit 175 . The control unit 140 may control the output of the output unit 173 . The controller 140 may control driving of the driving unit 160 . The controller 140 may control the operation of the work unit 180 .

The mobile robot 100 includes a storage unit 150 for storing various types of information. The storage unit 150 records various information necessary for controlling the mobile robot 100, and may include a volatile or non-volatile recording medium.

The storage unit 150 stores home guard mode setting information input or received from the communication module 170 (eg, setting time information, information on specific functions to be restricted, information on mode functions to be activated, and/or or information on a set area) may be stored. The storage unit 150 may store setting information of a specific function input or received from the communication module 170 .

A map for a driving area may be stored in the storage unit 150 . The map may be input by an external terminal capable of exchanging information with the mobile robot 100 through the communication unit 175, or may be generated by the mobile robot 100 through self-learning. In the former case, the terminal 300 may include, for example, a remote controller, a PDA, a laptop, a smart phone, a tablet, and the like loaded with an application for setting a map.

The actual driving area may correspond to the driving area on the map. The traveling area may be defined as the sum of areas on all planes in which the mobile robot 100 has driving experience and areas on the plane currently traveling.

The controller 140 may determine the movement path of the mobile robot 100 based on the operation of the driving unit 160 . For example, the controller 140 can determine the current or past moving speed of the mobile robot 100, the distance traveled, etc. based on the rotational speed of the driving wheels 166, and each driving wheel 166 (L) , 166(R)), the current or past direction change process can also be grasped. Based on the driving information of the mobile robot 100 identified in this way, the location of the mobile robot 100 on the map may be updated. Also, the location of the mobile robot 100 on the map may be updated using image information.

The user can change On/Off of the home guard mode through the input unit 171 or the terminal 300 or the like. The controller 140 may determine whether the home guard mode is active (whether On/Off) through the communication module 170 .

The home guard mode is a mode for detecting an intrusion occurring in a home according to a user's need or within a set time. In the home guard mode, the mobile robot may collect surrounding images and/or sounds while waiting at a designated location (eg, near a front door or near a window). As another example, in the home guard mode, the mobile robot may sequentially cycle through designated locations or collect surrounding images and/or sounds while patrolling designated areas.

Referring to FIG. 8 , the controller 140 may generate a control signal using intrusion detection information. The controller 140 may generate a control signal using intrusion detection information when the home guard mode is turned on or when there is no user in the home for a certain period of time.

The controller 140 may determine whether there is a dangerous situation through intrusion detection information. Specifically, the controller 140 may determine a dangerous situation by analyzing image information. For example, the controller 140 may extract characteristic information of an unidentified user from video information and determine whether the user is an intruder based on the characteristic information. Here, the intruder means a human except for a registered user described later.

The control unit 140 detects a change in image information, detects whether a person exists through the image information when the amount of change in the image information is greater than a predetermined reference value, and extracts characteristic information when it is determined that a person exists. . Specifically, the controller 140 determines that there is motion in the home when the amount of change in the image information is greater than or equal to the reference value, and detects the presence of a person through the image information. Various methods may be used to detect a person, and for example, a method of detecting a face and a shoulder may be used.

More specifically, the controller 140 analyzes the image information by time, and when the image information changes according to time, the controller 140 can detect the presence of a person by analyzing the image information. Alternatively, the control unit 140 determines the image after the home guard mode is executed when the amount of change between the image information after a predetermined time has elapsed after the home guard mode is executed and the image information collected for each time period exceeds a predetermined reference value. By analyzing the information, it is possible to determine whether a person exists or not. If there is a change in image information, movement can be detected when the user is absent, so an intrusion can be easily detected.

Since an error may occur in determining that a dangerous situation is caused by simply changing the image information, the control unit 140 detects the change in the image information and detects the presence of a human through the image information when the amount of change is greater than a certain reference value. And if it is determined that a human exists, the characteristic information of the human can be extracted. The controller 140 may determine whether a human exists by extracting a pixel or the like from image information and analyzing it.

As another example, the controller 140 detects whether a person exists through image information, and when it is determined that a person exists, the controller 140 may extract characteristic information of the detected person.

When there is a change in the image and the change in the image is triggered by a person, the controller 140 determines the intruder based on the person's characteristic information, thereby reducing errors in determining the intruder.

The control unit 140 may extract unconfirmed user characteristic information through the characteristic information and compare the extracted unconfirmed user characteristic information with pre-registered user characteristic information to determine an intruder.

Here, the characteristic information is data for identifying the identity of the user. For example, the characteristic information may include at least one of gender information, age information, and face information of the user. Such user characteristic information may be stored in the storage unit 150 of the mobile robot and stored in the network 50 . The controller 140 identifies the user by comparing the extracted characteristic information with previously stored characteristic information.

A method of registering user characteristic information may be input through the image sensor 135 of the mobile robot or through the camera of the terminal 300 . Specifically, the controller 140 acquires image information by operating the image sensor 135 when a user input command requesting user registration is input to the mobile robot, extracts characteristic information from the acquired image information, and stores In (150), user's characteristic information may be stored. Of course, the control unit 140 may transmit the user's characteristic information stored in the storage unit 150 to the server or the terminal 300 .

As another example, when a user input command requesting user registration is input, the terminal 300 obtains image information by operating a camera of the terminal 300, extracts characteristic information from the obtained image information, and moves the extracted characteristic information. can be transmitted to the robot. The control unit 140 may store pre-registered user characteristic information received from the terminal 300 in the storage unit 150 . Of course, the control unit 140 may transmit the user's characteristic information stored in the storage unit 150 to the server or other terminal 300 .

To this end, the image detector 135 combines the polynomial support vector machine proposed by Papgeorgiou and Poggio to extract the user's image ( Haar)-based method can be used, the method using the chamfer distance between the boundary image and the example database proposed by Gavrila and Philomin, or the extended method proposed by Viola A technique such as a similar Harr wavelet method may be used.

The control unit 140 extracts unconfirmed user characteristic information through the characteristic information and compares the extracted unconfirmed user characteristic information with pre-registered user characteristic information to determine an intruder.

Specifically, if the similarity between the unverified user characteristic information and the pre-registered user characteristic information is less than a preset reference value, the control unit 140 determines the unidentified user as an intruder, and determines the similarity between the unverified user characteristic information and the pre-registered user characteristic information. If is equal to or greater than a preset reference value, it is possible to determine an unidentified user as a registered user.

When the unidentified user is determined to be an intruder, the mobile robot may notify the user or recognize the face of the user intruder.

When it is determined that the unidentified user is an intruder, the controller 140 may control the terminal 300 to output a screen notifying the intruder. Here, the screen notifying the intruder may be at least one of a part of a control screen, a pop-up window, and a message.

Specifically, the controller 140 may control the terminal 300 to output a control screen 301 for determining a dangerous situation when it is determined that an unidentified user is an intruder. When it is determined that the unidentified user is an intruder, the controller 140 may output a control signal for outputting a control screen 301 including characteristic information of the unidentified user. More specifically, when it is determined that the unidentified user is an intruder, the controller 140 may control an image including face information of the unidentified user to be displayed on the control screen 301 .

The control screen 301 of the terminal 300 may include a dangerous situation output window 303 outputting dangerous situation information (information on the characteristics of an intruder) as an image and an alarm generating icon 305 for receiving an alarm generating command. .

As another example, the control unit 140 may control to output the control screen 301 including the image information of the unidentified user when the amount of change in the image information is equal to or greater than a predetermined reference value and an unidentified user is detected. Here, the control screen 301 including image information may include at least one image or video when the amount of change in image information is greater than or equal to a predetermined reference value.

Of course, the control unit 140 may control an icon for receiving an alarm generating command to be output together with image information on the control screen 301 . That is, when a control signal is input from the control unit 140, the terminal 300 may display an icon for receiving image information and an alarm generation command on the control screen 301. The user can check whether the dangerous situation is external intrusion with a single screen on the terminal 300, and in the case of an outside intrusion, the user can input a command to generate an alarm to the mobile robot and home appliances through an alarm generating command icon.

When it is determined that the unidentified user is an intruder, the controller 140 may output a control signal to control at least one of the home appliance and the mobile robot to output an alarm. That is, the control unit 140 may determine a dangerous situation by itself and output an alarm to an intruder by itself or/and together with home appliances in the home. Here, the home appliance means that it is connected to the mobile robot through a network, but is located at a short distance from the mobile robot.

Specifically, the control unit 140 may control the output unit to output an alarm when it is determined that an unidentified user is an intruder. The alarm output from the output unit may include at least one of sound and light. In addition, when the controller 140 determines that an unidentified user is an intruder, the control unit 140 outputs a control signal to the home appliance so that the home appliance outputs an alarm, and each home appliance receiving the control signal extracts a control command from the control signal, An alarm corresponding to the control signal may be output. The alarm of the home appliance may include at least one of sound and light. Each home appliance may include a speaker and a light source for outputting sound.

When it is determined that the unidentified user is an intruder, the controller 140 may extract an alarm generating command from a signal of the terminal 300 and control at least one of a mobile robot and a home appliance according to the alarm generating command. That is, when it is determined that the unidentified user is an intruder, the control unit 140 transmits information about the intruder to the terminal 300 so that the user can determine a dangerous situation by looking at the terminal 300, and the terminal 300 Through this, it is possible to extract a command for controlling at least one of the home appliance and the mobile robot.

When the mobile robot determines that the unidentified user is an intruder, the mobile robot may deactivate the control authority of the mobile robot or the user registration request command. Here, inactivating the suggestion right means not executing the user command even if the intruder inputs a user command or a user registration request command through the input unit 171 of the mobile robot. Accordingly, it is possible to prevent the mobile robot from being controlled by an intruder or from being registered as a user by an intruder. Specifically, when it is determined that an unidentified user is an intruder, the control unit 140 does not extract a control command from the user command and does not execute the user command even if a user command is input.

In addition, when the mobile robot determines that the unidentified user is a registered user, it may activate a control authority of the mobile robot or a user registration request command. Here, activating the suggestion right means that the mobile robot executes the user's command when a user command or a user registration request command is input through the input unit 171 of the mobile robot.

Hereinafter, a control method of the mobile robot 100 according to embodiments of the present invention will be described with reference to FIG. 10 . In each flowchart, overlapping contents are denoted by the same reference numerals, and overlapping descriptions are omitted.

The control method may be performed by the control unit 140. The present invention may be a control method of the mobile robot 100 or a mobile robot 100 including a control unit 140 that performs the control method. The present invention may be a computer program including each step of the control method, or a recording medium in which a program for implementing the control method in a computer is recorded. 'Recording medium' means a computer-readable recording medium. The present invention may be a mobile robot 100 control system including both hardware and software.

Each step of the flowchart drawings of the control method and combinations of the flowchart drawings may be performed by computer program instructions. The instructions may be embodied in a general purpose computer or special purpose computer or the like, and the instructions create means for performing the functions described in the flowchart step(s).

Also, in some embodiments, it is possible for the functions mentioned in the steps to occur out of order. For example, two steps shown in succession may in fact be performed substantially concurrently, or the steps may sometimes be performed in reverse order depending on the function in question.

Referring to FIG. 10 , the control method of the mobile robot includes a mode input step (not shown) for setting a home guard mode. In the mode input step, whether to turn on/off the home guard mode may be selected.

Of course, the control method of the mobile robot may further include a command input step (not shown) of receiving a task execution command. In the command input step, the user may command the mobile robot 100 to perform the first task.

The control method of the mobile robot 100 is performed only when the home guard mode is turned on. The mobile robot 100 may determine whether the home guard mode is On, and execute the following steps only when the home guard mode is On (S111).

In the step of detecting intrusion detection information ( S113 ), the mobile robot 100 may detect intrusion detection information while patrolling a predetermined area or detect intrusion detection information at a predetermined location. For example, the mobile robot may collect image and sound information around the front door while waiting near the front door, and store them in the storage unit 150 . Here, the intrusion detection information means all information when an external intrusion is suspected. Intrusion detection information may include characteristic information of an unidentified user.

Specifically, in the step of detecting intrusion detection information (S113), the mobile robot may extract characteristic information of an unidentified user from image information and determine whether the user is an intruder based on the characteristic information. The mobile robot detects a change in image information, detects the existence of a person through the image information when the amount of change in the image information is greater than a certain reference value, and extracts characteristic information when it is determined that a person exists.

In the step of transmitting the intrusion detection information (S115), the mobile robot determines whether there is a dangerous situation based on the intrusion detection information, and if it is a dangerous situation, the intrusion detection information (control signal) corresponding to the dangerous situation is sent to the terminal 300. send. Specifically, if the similarity between the unidentified user characteristic information and pre-registered user characteristic information is less than a preset reference value, the mobile robot determines the unidentified user as an intruder, and if the unidentified user is determined to be an intruder, it can be determined as a dangerous situation. .

The terminal 300 receiving the intrusion detection information may generate a control screen 301 based on the intrusion detection information. At this time, as described above, the control screen 301 of the terminal may display characteristic information including face information of an unidentified user and an icon for receiving an alarm generation command.

Thereafter, the terminal 300 may receive a user's control command (S119). The user's control command can be input through various input units, but it is preferable to receive the input through the control screen 301 .

Thereafter, the terminal 300 may convert the control command into a terminal 300 signal (S121) and transmit the terminal 300 signal to at least one of a mobile robot and a home appliance.

The mobile robot receiving the terminal 300 signal from the terminal 300 extracts an alarm generation command from the terminal 300 signal transmitted from the terminal 300 (S125, S127).

The mobile robot may generate a control signal so that at least one of the mobile robot and the home appliance outputs an alarm according to the alarm generation command (S126 and S128). If the mobile robot and home appliances together output an alarm, it is possible to effectively give an alarm to an intruder from the outside, and there is an advantage of expelling the intruder.

100: mobile robot 120: sensing unit
140: control unit 160: driving unit
170: communication module 171: input unit
173: output unit 175: communication unit
180: work unit 200: charging stand
300: terminal

Claims (16)

a sensing unit that acquires image information around the mobile robot;
a control unit generating a control signal using the image information; and
A communication unit configured to transmit the control signal to a terminal connected to the mobile robot through a wired/wireless communication method and to receive a terminal signal from the terminal;
The control unit,
Extracting characteristic information of an unidentified user from the image information, and determining whether the user is an intruder based on the characteristic information;
When it is determined that the unidentified user is an intruder, the control unit controls the terminal to output a control screen including characteristic information of the unidentified user, and displays an icon for receiving an alarm generating command on the control screen along with the image information. Control to output together, and control to output an image including face information of an unidentified user on the control screen,
The communication unit transmits the control signal to a home appliance connected to the mobile robot through a wired or wireless communication method,
The control unit,
When it is determined that the unidentified user is an intruder, an alarm generating command is extracted from the terminal signal, and at least one of the mobile robot and the home appliance is controlled according to the alarm generating command;
The control unit,
When receiving the alarm generation command, control the home appliance to output an alarm;
Wherein the controller determines that the unidentified user is an intruder, even if a user command is input, does not extract a control command from the user command and does not execute the user command. According to claim 1,
The control unit,
The mobile robot detects a change in the image information, detects whether a person exists through the image information when the amount of change is greater than a predetermined reference value, and extracts the characteristic information when it is determined that a person exists. According to claim 1,
The control unit,
A mobile robot detecting the presence of a person through the image information and extracting the characteristic information when it is determined that a person exists. According to claim 1,
The control unit,
The mobile robot extracts the unconfirmed user characteristic information through the characteristic information and compares the extracted unconfirmed user characteristic information with pre-registered user characteristic information to determine the intruder. According to claim 4,
The control unit,
If the similarity between the unverified user characteristic information and the pre-registered user characteristic information is less than a preset reference value, determining the unidentified user as the intruder;
The mobile robot determining that the unidentified user is a registered user if the degree of similarity between the unidentified user characteristic information and the pre-registered user characteristic information is equal to or greater than a predetermined reference value. According to claim 1,
The control unit,
If it is determined that the unidentified user is an intruder, the mobile robot controls the terminal to output a screen notifying the intruder. delete delete According to claim 1,
The control unit,
When it is determined that the unidentified user is an intruder, the mobile robot extracts an alarm generating command from the terminal signal and controls the mobile robot according to the alarm generating command. According to claim 1,
Further comprising an output unit for outputting a visually or audibly perceptible alarm,
The control unit,
The mobile robot controlling the output unit to output an alarm when it is determined that the unidentified user is an intruder. delete delete According to claim 1,
Further comprising a storage unit for storing data,
The control unit,
A mobile robot that receives pre-registered user characteristic information from the terminal and stores it in the storage unit. According to claim 1,
Further comprising a storage unit for storing data,
The control unit,
A mobile robot that extracts characteristic information from the image information when a user input command is input and stores the user's characteristic information in the storage unit. According to claim 1,
The control unit,
A mobile robot that extracts characteristic information of an unidentified user from the image information when a home guard command is input, and determines whether the user is an intruder based on the characteristic information. According to claim 1,
The characteristic information is
A mobile robot including at least one of the user's gender information, age information, and face information.

Images