KR20200063355A - Personalized arrangement robot and arrangement method using the same - Google Patents

Abstract

The present invention relates to a personalized organizing robot which identifies personal tendency to organize domestic articles, thereby reducing the sense of difference on organizing articles and helping a user to easily find articles. To this end, according to the present invention, the personalized organizing robot comprises: an image shooting unit which shoots images of articles; an article recognition unit which recognizes the articles whose images are shot through the image shooting unit; a movement performing unit which is to move the articles; an articles location map unit which stores the predicted location of the articles therein; and a control unit which controls the movement performing unit by comparing the predicted location of the articles location map with an actually measured location of the articles whose images are shot through the image shooting unit. Accordingly, according to the present invention, the personalized organizing robot capable of organizing articles used by a person can be provided, and the tendency of an individual can be identified such that articles can be organized at the most preferred location.

Description

Personalized arrangement robot and arrangement method using the same}

The present invention relates to a personalized organizing robot and a method of arranging objects using the personalized robot, and specifically, by identifying personal preferences and arranging the housework to reduce the heterogeneity of organizing objects and helping the user to easily find the object. The present invention relates to a custom arrangement robot and a method for arranging objects using the same.

With the development of robot technology, many studies have been conducted not only for industrial robots, but also for household robots. In particular, the household robot is the area where the cleaning robot is most actively commercialized. These cleaning robots store the indoor map to understand the indoor structure, and they move around themselves and use vacuum absorbers to absorb dust or wipe the floor.

As a prior art, referring to the registered patent No. 10-0877072 shown in FIG. 1, the mobile robot for cleaning uses a feature 뱁 generating unit for generating a feature map for location recognition of the mobile robot, and the feature map Based on the posture information of the obtained mobile robot, each cell includes a plurality of cells, and each cell includes a path map generator for generating a path map having obstacle information and path information, and the mobile robot according to the obstacle information and path information It consists of a movement control unit for moving.

According to the patent, since the cleaning path is generated using the feature map and the route map, the cleaning time is significantly reduced.

However, these prior arts only serve to clean the floor according to a predetermined route, and there is no method or configuration for arranging objects according to personal preferences.

Therefore, there is a need for research on a robot capable of grasping personal tendencies rather than a predetermined route and arranging personal objects.

The present invention is to solve the above problems, and it is an object of the present invention to provide a personalized organizing robot for arranging and arranging personal objects in a preferred position by learning personal tendencies by themselves.

In addition, it is another object of the present invention to provide information that can help a user to find an object by knowing the location of the personal object.

In order to achieve the above object, the present invention, an image photographing unit for photographing an object; An object recognition unit recognizing an object photographed through the image photographing unit; A moving performance unit for moving the object; An object location map unit for storing the predicted location of the object; And a control unit that controls the moving unit by comparing the predicted position by the object location map unit with the actual position of the object photographed by the image photographing unit.

It is preferable that the object location map unit includes the position and recognition time of the object recognized through the object recognition unit as input data, and performs machine learning by including the predicted position or discard of each object as output data.

The control unit determines the presence or absence of a corresponding predicted position by referring to the object position map unit with respect to the object recognized by the object recognition unit, and calculates an error distance between the predicted position and the actual position if the predicted position exists. When the error distance is more than a certain distance, it is preferable to move the object to the predicted position by controlling the moving performance unit.

The personalized rearranging robot further includes a personal propensity determination unit, and the personal propensity determination unit calculates the number of movements in which the error distance is greater than or equal to a predetermined distance, and the number of movements is greater than or equal to a certain number of times, and the actual position is within a certain range The actual position can be changed to the predicted position.

The personal propensity determination unit may receive feedback on the object moved by the movement execution unit and update the object location map unit.

The personalized organizing robot further includes a display unit for giving information to the user, and the controller searches the object location map unit and displays the location information of the object through the display unit when the location information of a specific object is requested. desirable.

Preferably, the object recognition unit classifies the types of objects photographed by the image photographing unit, and the object location map unit stores the predicted positions for each type of object recognized by the object recognition unit.

The object recognition unit stores the shooting date of the object photographed by the image photographing unit, and the control unit compares the setting date and the shooting date for the object recognized by the object recognition unit, and the shooting date is equal to or equal to the setting date After that, it is preferable to move the object to the predicted position by controlling the moving performance unit.

According to the present invention, it is also possible to photograph an object by the image capturing unit; Recognizing an object photographed through the image photographing unit by an object recognition unit; Storing the predicted position of the object in the object location map unit; And the control unit compares the predicted position by the object location map unit with the actual position of the object photographed by the image photographing unit, and controls the moving execution unit to move the object. Gives

It is preferable that the object location map unit includes the position and recognition time of the object recognized through the object recognition unit as input data, and performs machine learning by including the predicted position or discard of each object as output data.

According to the present invention, it is possible to provide a rearranging robot capable of rearranging objects used by an individual, and there is an advantage of arranging the objects in the most preferred location by grasping the personal preferences.

In addition, according to the present invention, it is possible to efficiently organize objects, and when a user finds a specific object, there is an advantage that can be conveniently used by displaying information about the object.

1 is a configuration diagram showing the configuration of a cleaning robot according to the prior art;
Figure 2 is a block diagram showing the configuration of a personalized organizer robot according to the present invention;
Figure 3 is a block diagram showing the configuration of the object recognition unit in Figure 2;
4 is an exemplary view showing an example of a personalized organizing robot according to the present invention;
5 is an exemplary view showing a learning method of an object location map unit according to the present invention;
Figure 6 is an explanatory diagram showing a method of organizing a personalized organizing robot according to the present invention.

The configuration and operation of the embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 and 6, the personalized organizing robot according to the present invention includes an image photographing unit 110, an object recognition unit 120, a moving performance unit 130, an object location map unit 140, and a personal propensity determination unit. It includes 150, the control unit 160, the communication unit 170, the driving unit 180.

The image photographing unit 110 may include one or more sensors capable of photographing a front image, and may include an RGB camera, an RGBD camera, a lidar, an infrared scanner, an ultrasonic sensor, and an LRF sensor. Can be.

The object recognition unit 120 recognizes an object included in the image using the image captured by the image photographing unit 110 to generate a label.

Specifically, referring to FIG. 3, the object recognition unit 120 includes an image receiving unit 121, a feature point extraction unit 122, a contour extraction unit 123, and a label matching unit 124.

The image receiving unit 121 may receive an image photographed by the image photographing unit 110 or may receive an image photographed from the outside through the communication unit 160. In the image received through the image receiving unit 121, feature points are extracted from the feature point extraction unit 122. The feature points may be points in which a change in pixel (color or contrast) is large, for example, an edge or a vertex of an object.

In addition, the outline extraction unit 123 serves to extract the appearance of the photographed object. A segmentation technique may be used as a method of extracting contours.

The image information of the object separated by the contour extraction unit 123 is classified by the label matching unit 124 to generate a label.

The label matching unit 124 may be composed of an artificial intelligence neural network such as a deep learning including an input layer, a hidden layer, and an output layer, and weight information is stored in the hidden layer using the learned data.

The object recognition unit may classify the type of the object photographed by the image photographing unit in this way, and may store a photographing date of the object photographed by the image photographing unit.

The moving performance unit 130 is formed in front of the main body as shown in FIG. 4 and serves to grip an object photographed from the image photographing unit 110, and is preferably provided with a soft gripper. That is, when the object photographed from the image photographing unit 110 can be gripped under the control of the control unit, the moving performance unit 130 grips the object and releases the gripping of the object after moving to a specific place by the driving unit 180 can do.

The moving unit 130 may be provided in various forms, such as a robot hand mimicking five fingers, a vacuum absorber capable of vacuum adsorption, and robot tongs provided in two directions.

The control unit may determine whether or not gripping is possible through an image of an object photographed from the image photographing unit. That is, the controller classifies the type of the photographed object by matching the image information of the object stored in the storage unit with the database of the type of the object stored in the storage unit, and classifies the type of the photographed object, or the weight of the classified object or Depending on the shape, it is determined whether or not gripping is possible through the moving performance unit.

In addition, when it is determined that the control unit can grip the mobile unit, the control unit can learn the object. Here, the control unit can learn the object through the GAN-based picking learning policy network. For example, the controller learns about an object to be actually gripped through a generator and generates false data based on this. In addition, the controller performs learning so that it is possible to determine whether the data provided by the constructor is real or false through a discriminator. In this way, the control unit can process a large amount of data without a purification process, and can solve a problem in which it is not easy to secure learning data by learning by itself.

The present invention can manage a vast amount of data by setting aside a management server and building a database on the management server. That is, the management server receives an image of the object detected in front from the personalized organizing robot, or transmits the weight or shape according to the kind of the object or the kind determined by the server control unit back to the personalized organizing robot, or grips it. Data about whether it is possible or not can be transmitted.

In addition, the server control unit can identify the type of the object included in the image by matching the image of the object received from the personalized organizing robot with the image information stored in the database.

The object location map unit 140 serves to store and update the predicted location of the user object. The predicted location may be a user's preferred location for each object, and the location where the object should be located may be viewed as data corresponding to a predetermined location in the room. In particular, the object location map unit 140 may store the predicted location for each type of object recognized by the object recognition unit.

The object location map unit 140 stores the indoor map including the indoor structure, and also learns the location and pose of the object by machine learning such as artificial neural network or deep learning as shown in FIG. 5 to predict the location of each object. Information will be saved. For example, when the cup is recognized by the object recognition unit, the predicted position of the cup may be stored as a kitchen, and when the clothes are recognized, the predicted position of the clothes may be stored as a laundry room.

In the indoor map, the location and structure information of a room, a living room, a kitchen, and a toilet may be stored, and an indoor map may be generated based on image information obtained through pre-made map information or a driving unit. In addition, the predicted position of the object may correspond to 2D or 3D coordinates of each object based on the indoor map, or may correspond to functional compartments such as a room, a kitchen, and a toilet.

When learning the predicted location, the input layer may include the location and recognition time of the object acquired through the image capturing unit 110 as input data, and the output layer may include the predicted location or discard of each object as output data. have.

For example, when it is said that the bottle is recognized through the imaging unit, it may be determined that the volume of the contents is estimated by analyzing the color of the bottle and discarded when the volume of the contents is determined to be less than or equal to a predetermined volume. In addition, in the case of a cookie box, if the box recognized by the image capturing unit is open, it may be determined whether there is a cookie or not, and if the cookie box is not present, it may be discarded.

As described above, in the case of the object judged to be discarded, the predicted position is stored in the recycle bin, and the moving execution unit can grasp the object and move it to the recycle bin.

In addition, even if the same object, the predicted position may vary according to the state of the object recognized by the object recognition unit 120. For example, when an object is recognized as a cola bottle in the object recognition unit, the predicted location may be a refrigerator when it is recognized that the cola bottle is full of coke, and discarded when the object is recognized as empty in the coke bottle. Judge.

The driving unit 180 is configured to move the main body, and may be composed of a wheel installed on a lower portion of the main body and a motor for driving the wheel or a leg that can walk.

The main body is equipped with a device or software, such as an image capturing unit, an object recognition unit, and a moving performance unit, and the main body is capable of height adjustment or maintains a constant height to recognize and move objects in a certain height. It is preferably configured.

In addition, the communication unit 170 provided in the main body is provided to perform wireless communication such as Bluetooth, Wi-Fi, LTE, and can communicate with a management server through a communication network. In particular, the communication unit 170 transmits an image of an object in front taken by the image photographing unit 110 to the management server, or the type of object corresponding to the image transmitted from the management server or whether the object can be gripped. Data can be received.

In addition, the main body may be provided with an input unit 190 and a display unit 195. The input unit 190 may include one or more buttons. For example, the input unit 190 may include a button for inputting a command for confirming a predicted location, image information, object location, etc., or a button for resetting the command, a delete button, a cleanup start button, a stop button, and the like. Also, the input unit 190 may include a button for setting or deleting reservation information. The input unit 190 may include at least one of a keypad and a touch pad that generate an input signal according to a user's touch or manipulation. At this time, the input unit may be configured in the form of one touch panel together with the display unit to simultaneously perform input and display functions.

The display unit 195 displays reservation information, battery status, arrangement method, driving method, predicted location, and object location on the screen. The display unit may output the current state and the organized state of each unit of the main body.

The control unit 160 serves to control the operation of each component unit of the main body. In addition, the main body moves through the indoor space through the driving unit to scan the indoor objects periodically or at a preset time.

The controller 160 recognizes an object by analyzing an image captured by the object recognition unit based on the image taken through the image capturing unit, and determines whether there is a corresponding predicted position with respect to the recognized object by referring to the object location map unit. .

At this time, if there is a predicted position corresponding to the recognized object, an error distance between the predicted position and the actual position photographed through the image capturing unit is calculated. The error distance may be calculated as a straight line distance with reference to the indoor map, but is not limited thereto, and may be calculated in a manner suitable for the situation. For example, it may be calculated as a distance between functional compartments such as a room, a kitchen, and a bathroom, not the shortest distance considering a wall or door in the interior, or a distance of coordinates. The functional compartment space can be divided into functional objects such as a bed, a desk, and a chair in a room. If the predicted position and the measured position of the recognized object are the same, the error distance is calculated as 0.

The control unit calculates the error distance for each object in the same way as above and stores it in the storage unit. If the error distance is greater than or equal to a certain distance, the control unit may move the object to the predicted position after holding the object.

At this time, the error distance may be different from the general distance concept. For example, if there is a desk next to a bed in the same room, even if the measured distance measured with a ruler is a short distance, the error distance can be stored as a very large distance if the functional compartment space changes. That is, the error distance can be calculated by multiplying the measured distance measured with a ruler by the weight of the functional compartment space. If the functional compartment space is changed, the weight is set to be large. That is, if the functional partition space of the predicted position and the actual position is different, the weight may be multiplied by a value greater than 1, and if the functional partition space of the predicted position and the actual position are the same, the weight may be multiplied by a value less than 1.

After completing the entire indoor scan, the control unit may arrange the objects in order of the largest error distance, and then start sorting from the object with the largest error distance.

Meanwhile, a set date or a set number of times may be stored according to the recognized object. The set date or the set number of times is a date or number of times to take action on the product. For example, when a specific coat is recognized, the controller may compare the set date with the picked-up date recognized by the object recognition unit and perform a set operation if the picked-up date is equal to or later than the set date. The set operation may be an operation such as moving to the laundry room using the moving unit. In addition, when a specific food is recognized through the object recognition unit, when the set date is reached, an operation of moving the food to the trash can be performed through the movement performing unit.

The set number of times represents the number of times recognized by the object recognition unit. At this time, it is preferable that the number of settings increases so that the number of settings increases when the shooting date is changed.

The set date or the set number of times may be input by the user or may be set automatically. For example, when the recognized object is a specific shirt, the set number of times is automatically determined, and when the number of times the shirt is recognized reaches the set number, the control unit moves the shirt to the laundry room using a moving unit.

Unlike the general environment, the personal propensity determination unit 150 is used to arrange the household items by reflecting personal propensities. The number of times an error distance is greater than or equal to a certain distance by analyzing an image captured by the image taking unit That is, when the number of movements is calculated for a certain period and the number of movements is greater than or equal to a certain number, and the actual location is within the actual error range, the actual location is changed to the predicted location in the object location map unit through the control unit.

The measurement error range may mean a range within a predetermined distance from the stored final measurement position, and the weight may be multiplied according to the functional partition space as well as the error distance. That is, if the final measured position and the current measured position are the same functional partition space, the weight may be less than 1, and if the final measured position and the current measured position are the same functional partition space, the weight may be greater than 1.

The personal preference determining unit 150 may receive an individual's preferred location through an external terminal. For example, a preferred location of a specific object may be input through a terminal such as a smartphone, and at this time, the personal preference determination unit changes the preferred location received through the external terminal to the predicted location of the corresponding object.

In addition, the personal preference determining unit 150 may update the object location map unit by receiving feedback on the object moved by the movement performing unit 130 and using it as input data for machine learning. In this case, it is preferable to use artificial intelligence learning based on GAN (Generative Adversarial Networks) for machine learning.

Meanwhile, the display unit 195 displays various information to the user, and the control unit may display information corresponding to the request received through the input unit 190 or an external terminal through the display unit.

The object location map unit 140 stores the final location of the scanned object or the object moved through the movement performing unit. At this time, when the location information of a specific object is requested, the control unit may search the object location map unit and display the location information of the object through the display unit.

Although described above with reference to the embodiments of the present invention, those skilled in the art can variously modify the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can change it.

110: video recording unit 120: object recognition unit
130: mobile execution unit 140: object location map unit
150: personal propensity determination unit 160: control unit
170: communication unit 180: driving unit
190: input unit 195: display unit

Claims (10)

An image photographing unit for photographing objects;
An object recognition unit recognizing an object photographed through the image photographing unit;
A moving performance unit for moving the object;
An object location map unit for storing the predicted location of the object; And
And a control unit that controls the moving unit by comparing the predicted position by the object location map unit with the actual position of the object photographed by the image photographing unit.
According to claim 1,
The object location map unit includes the location and recognition time of the object recognized through the object recognition unit as input data, and includes a predicted location or discard of each object as output data to personalize the personalized theorem. robot.
According to claim 1,
The control unit determines the presence or absence of a corresponding predicted position by referring to the object position map unit with respect to the object recognized by the object recognition unit, and calculates an error distance between the predicted position and the actual position if the predicted position exists. If the error distance is more than a certain distance, the personalized organizing robot characterized in that the object is moved to the predicted position by controlling the moving part.
According to claim 3,
The personalized organizing robot further includes a personal propensity determination unit,
The personal propensity determination unit personalizes the personality characterized by changing the measured position to a predicted position by calculating the number of movements in which the error distance is greater than or equal to a certain distance and when the number of movements is greater than or equal to a predetermined number and the measured position is within a predetermined range. Cleanup robot.
According to claim 3,
The personal propensity determination unit is personalized organizing robot, characterized in that receiving the feedback about the object moved by the moving unit to update the object location map.
According to claim 1,
The personalized organizing robot further includes a display unit for giving information to the user,
The control unit searches for the location map of the object when the location information of a specific object is requested, and displays the location information of the object through the display unit.
According to claim 1,
The object recognition unit classifies the types of objects photographed by the image capturing unit,
The object location map unit stores a personalized arrangement robot characterized in that it stores a prediction location for each type of object recognized by the object recognition unit.
According to claim 3,
The object recognition unit stores the shooting date of the object photographed by the image photographing unit, and the control unit compares the setting date and the shooting date for the object recognized by the object recognition unit, and the shooting date is equal to or equal to the setting date Afterwards, the personalized organizing robot characterized in that the object is moved to the predicted position by controlling the moving part.
Photographing an object by the image photographing unit;
Recognizing an object photographed through the image photographing unit by an object recognition unit;
Storing the predicted position of the object in the object location map unit; And
The control unit compares the predicted position by the object location map unit with the actual position of the object photographed by the image photographing unit, and controls the moving unit to move the object by performing a personalized organizing robot.
The method of claim 9,
The object location map unit includes the location and recognition time of the object recognized through the object recognition unit as input data, and includes a predicted location or discard of each object as output data to personalize the personalized theorem. How to clean things up with a robot.

Images