KR101862545B1 - Method and system for providing rescue service using robot - Google Patents

Abstract

The present invention relates to a method and a system for providing a rescue service using a robot. The method for providing a rescue service using a robot according to an embodiment of the present invention comprises: a step of obtaining a time series image of a surrounding situation of a robot; a step of analyzing the time series image to obtain motion data of a dynamic object; a step of analyzing a set of motion data acquired according to the time series on a learning basis to determine whether the dynamic object is a living body or a living body; a step of extracting a partial image corresponding to the dynamic object among sources of the time series images related to the surrounding situation when the dynamic object is the living body; and a step of transmitting the extracted partial image to a user remote apparatus.

Description

 [0001] METHOD AND SYSTEM FOR PROVIDING RESCUE SERVICE USING ROBOT [0002]

The present invention relates to a method and system for providing a rescue service using a robot, and more particularly, to a method and system for providing a rescue service using a robot, in which the robot first recognizes a dynamic body object in an environment of a bad condition requiring rescue, And sends a confirmation request to the remote situation room. In this way, the remote situation room provides a rescue service using a robot that can quickly rescue life by separating the biological objects and promptly delivering the structure orders ≪ / RTI >

In general, disaster relief work is carried out by rescue personnel and is carried out under the control of the progress of the rescue work through the wireless communication equipment carried by the rescue personnel.

At this time, a structure in which the space is narrow or the risk of additional collapse is great, or in a situation where the boat is sunk in the sea as shown in Fig. 1, the environment requiring the structure is dark, the temperature is low, There is a problem that accessibility of these is difficult.

In the case where the rescue personnel are unable to carry out the rescue operation directly, the rescue robot is searched for the rescue person. In the past, the remote control of the user has enabled the robot to acquire all of the surroundings without selecting the high- It is difficult to collect and judge real-time information because it takes a long time to transmit and receive data because it requires a long transmission time because the data size is large and the image received from the user remote device is analyzed and the search command is continuously sent to the robot.

Korean Patent Registration No. 10-1123189 " Remote control of the resident dog and handler monitoring system and method thereof "

SUMMARY OF THE INVENTION It is an object of the present invention to provide a dynamic bio-object recognition system in which a robot recognizes a dynamic bio-object first in an environment of a bad condition requiring a structural retrieval, extracts an image of the recognized dynamic bio- To provide a rescue service providing method and system using a robot capable of quickly and easily distinguishing a living body object and promptly delivering a rescue command in a remote situation room so that life can be smoothly rescued.

According to an embodiment of the present invention, there is provided a method for providing a rescue service using a robot, including: obtaining a time series image of a surrounding situation of a robot; analyzing the time series image to obtain operation data of the dynamic object; Determining whether the dynamic object is a living body or a living body by analyzing the acquired set of the operation data by learning based on the learning object; Extracting the corresponding partial image, and transmitting the extracted partial image to the user remote device.

A system for providing a rescue service using a robot according to an embodiment of the present invention includes a sensor device for acquiring operation data of a dynamic object by analyzing a time series image of a surrounding situation acquired through a sensor device, A robot for extracting a partial image corresponding to the dynamic object from an original of the time series image with respect to a surrounding situation when the dynamic object is a living body and a partial image corresponding to the dynamic object from the robot, And a user remote device for controlling the robot according to a structure command received from a user.

According to the present invention, in the event of a disaster, the robot quickly detects living things irrespective of a poor situation such as dark vision, cold or hot temperature, lack of oxygen, etc., sends concentrated information about the detected living organisms to the remote control room, , The real-time information transmitted by the robot is analyzed and the structure command is immediately transmitted to the robot, so that the life structure can be executed quickly.

Here, the rescue order can be a guide to how a member of the remote control room moves from the disaster situation to the person through the speaker of the robot through the speaker, and the robot transmits the voice of the person to the remote control room through the receiver, By enabling real-time communication to the situation, the efficiency of lifesaving can be increased.

In addition, according to the embodiment of the present invention, since the robot extracts only the image of the dynamic bio object and transmits the extracted image, the amount of data transmission and transmission time can be reduced, and a plan for dealing with a disaster situation can be quickly established.

FIG. 1 illustrates an example of a rescue service using a robot according to an embodiment of the present invention.
2 is a schematic block diagram of a system for providing a rescue service using a robot according to an embodiment of the present invention.
Fig. 3 shows an example of the general operation of a dynamic bio-object.
4 is a view showing a distribution of a moving part according to a kind of an operation of a human.
5 shows an example of a 3D mask according to an embodiment of the present invention.
FIG. 6 shows an example of overlapping a 3D mask with a dynamic object.
FIG. 7 is a view for explaining an example of obtaining a 3D mask and a covariance matrix therefor according to an embodiment of the present invention.
Figure 8 shows a point cloud corresponding to one mask according to an embodiment of the present invention.
9 to 11 are flowcharts for explaining a method of providing a rescue service using a robot according to an embodiment of the present invention.

Prior to the description of the concrete contents of the present invention, for the sake of understanding, the outline of the solution of the problem to be solved by the present invention or the core of the technical idea is first given.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It will be apparent to those skilled in the art, however, that these examples are provided to further illustrate the present invention, and the scope of the present invention is not limited thereto.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, in which: It is to be noted that components are denoted by the same reference numerals even though they are shown in different drawings, and components of different drawings can be cited when necessary in describing the drawings. In the following detailed description of the principles of operation of the preferred embodiments of the present invention, it is to be understood that the present invention is not limited to the details of the known functions and configurations, and other matters may be unnecessarily obscured, A detailed description thereof will be omitted.

FIG. 1 is a block diagram of a system for providing a rescue service using a robot according to an embodiment of the present invention.

Referring to FIG. 2, a system 100 for providing a rescue service using a robot according to an embodiment of the present invention includes a robot 110 and a user remote device 130.

The robot 110 analyzes the time series image of the surrounding situation acquired through the sensor device to acquire the operation data of the dynamic object, analyzes the set of the operation data acquired according to the time series on the learning basis, Extracts a partial image corresponding to the dynamic object from the source of the time-series image with respect to the surrounding situation, and transmits the extracted partial image to the user remote device 130. And executes the drive in accordance with the structure command received from the user remote device 130. [

A brief description will be given of an embodiment of the present invention with reference to FIG. For example, in the event of a catastrophic event where a boat sinks in the sea, the rescue team is mobilized to carry out rescue operations, but it is not easy to grasp life by penetrating into the boat where the rescue crew sinks.

When the robot 110 according to the present invention is put into such a situation, the living organism is detected quickly regardless of a poor situation such as a dark field, a cold temperature, a lack of oxygen, and the collected information about the detected living organism is transmitted to the remote control room 130 Lt; / RTI > In the remote control room 130, the real-time information transmitted by the robot 110 is analyzed and the structure command is immediately transmitted to the robot 110, so that the robot 110 can rapidly execute the lifesaving structure. Here, the rescue command can be a guide for how a member of the remote control room 130 moves through the speaker of the robot 110 in the direction of the person's life in order to escape from the disaster situation. In addition, the robot 110 transmits a voice of a person's name to the remote control room 130 through the receiver, thereby enabling real-time communication in a disaster situation, thereby enhancing the efficiency of the life structure.

The robot 110 according to the embodiment of the present invention includes a sensor device 111, an analysis module 112, an image editing module 113, a learning engine 114, a communication module 115, a speaker module 117, And a control module 118.

The sensor device 111 acquires a time-series image of the surrounding environment and the surrounding environment. As the sensor device 111, for example, at least one of an input depth image camera, an infrared camera, a motion sensor, a motion recognition sensor, and a kinect may be used.

The analysis module 112 analyzes the time series image obtained from the sensor device 111 to obtain operation data of the dynamic object, analyzes the set of the operation data acquired according to the time series on a learning basis, It is judged whether or not it is microorganism.

Specifically, the analysis module 112 estimates the existence of a dynamic object through analysis between neighboring frames of the time series image, and tracks the motion pattern, the type of motion, the viewing direction, and the angle information of the estimated dynamic object.

When a human being or an animal performs a specific operation, each entity can be divided into a high frequency portion and a low frequency portion. Fig. 3 (a) shows the human walking, (b) shows the human running, (c) shows the running of the four-footed animal, and (d) shows the birding. 4 (A) to 4 (C) show distributions in which different characteristics are displayed for different parts of the body according to different human action patterns and different kinds of motions. When you look at these behaviors, most humans and four-footed animals usually use their arms and legs to perform some actions, while birds usually use their wings. On the other hand, dynamic objects, such as automobiles, machines, or natural phenomena, are much different from dynamic motion patterns of humans and animals. Accordingly, the present invention can analyze dynamic behavior of a dynamic object by analyzing inherent operation characteristics of only individual / individual dynamic objects, and can classify objects / sorts.

In the present invention, a 3D mask is used to extract characteristics of a dynamic object. Here, a process for extracting the 3D mask is as follows.

For this, the analysis module 112 receives the depth image, for example, obtained from the sensor device 111, and models the dynamic object in the depth image. That is, 3D body modeling of the dynamic object is performed.

In this process, the spatial coordinate values of each pixel of the depth image can be changed through feature extraction and transformation matrix calculation, and the resolution difference can be solved by interpolation or the like.

The 3D body modeling is to extract a 3D silhouette image using a depth image. For example, pixel segmentation is performed on each pixel of the depth image to divide the input depth image into a plurality of segments. Then, a segment corresponding to a dynamic bio-object, which is a living body, is detected through learning-based biometric detection among a plurality of segmented segments. When the 3D silhouette image is generated, a part of the characteristic body of the living body is detected from the 3D silhouette image, and based on the detected body part, the arcs of the dynamic object, for example, The position of joints can be calculated, and the motion pattern, motion type, direction of view, and angle information of the dynamic object can be tracked by using calculated arcs and positions of joints. Then, a 3D mask matching the tracked dynamic object is extracted from the database.

Using the extracted 3D mask, the 3D point cloud of the dynamic object received from the real sensor is analyzed. For this, the analysis module 112 extracts a 3D mask matching the operation data from the database, overlaps the extracted 3D mask to cover the dynamic object of the time-series image, and at least one joint of the overlapped dynamic object And a point cloud including an arthrosis, and analyzing the 3D point cloud as a learning basis to judge whether the living body of the dynamic object is living or non-living.

Here, the analysis module 112 obtains the covariance matrix of the 3D point cloud corresponding to the dynamic object overlapped by the 3D mask in order to analyze the 3D point cloud based on the learning, Analyze the change of the covariance matrix of the point cloud and grasp the movement information of the 3D point cloud according to the analysis.

The covariance matrix changes with the change of the frame of the time series image. Particularly, in the case of arms and legs, it moves repeatedly in the front and back direction. Therefore, by analyzing the change of the covariance matrix, the 3D point cloud in this partial mask can know which direction it is moving repeatedly. In this way, the training data can be defined as the change state of the covariance matrix in the corresponding mask.

In the case of recognition, the direction of movement of the dynamic object must first be grasped. FIG. 9 is a diagram for explaining a method of obtaining the moving direction of the dynamic object by the robot. It can be calculated by grasping the moving direction of the point cloud between two neighboring frames of the dynamic object and the moving direction and speed of the robot 110 and the like. The angular value between the sensor mounted on the robot 110 and the dynamic object can be calculated. When a sensor receives a point cloud of a dynamic object, it can acquire only the face in which the dynamic object is seen by the robot and the portion existing in the sensing range at that angle, rather than acquiring it from every face of the dynamic object. Therefore, when the robot recognizes a dynamic object, only a part of the features extracted from the view plane and the angle can be used. FIG. 6 shows an example of overlapping a 3D mask with a dynamic object. If only the mask corresponding to the visible angle of the dynamic object is applied and the motion pattern of several frames is calculated, the dynamic object can recognize whether it is a biological object. For a typical automobile and machine, the covariance matrix in all masks is almost unchanged.

Hereinafter, a method of obtaining the covariance matrix will be schematically described. As shown in Fig. 7, the point cloud for the dynamic object corresponding to the mask is input using the four masks for the walking action of the person. At this time, the point cloud can be received as time series data on a frame basis. One covariance matrix can be expressed as C in the point cloud change between two frames, and t ₀ , t ₁ The covariance matrix between the two is expressed as C ₁ .

Figure 8 shows a point cloud corresponding to one mask at time t ₀ , t ₁ . Represent the coordinates of the point clouds in each set as _{(x0 i, y0 i, z0} i), (x1 i, y1 i, z1 i). Specify the center position of the mask as (0,0,0) for the reference setting of the coordinates used for calculation.

The dispersion formula of the coordinate X of t ₀ , t ₁ is as shown in Equation (1). Through the distribution, we can see the geometric difference between the current two. When performing the calculation, if the data sets on both sides do not match, the data of the remaining sets are randomly selected based on the approximate one set of data quantity.

[Equation 1]

The covariance matrix used in Equation (2) differs from the general covariance matrix. In general covariance matrices, it is necessary to distribute both the x, y, and z axis corresponding values in order to obtain the geometric characteristics. However, since the present invention does not require a process of grasping the relation between all the data in order to grasp the relation between the coordinates, The covariance matrix is obtained from the equation of Equation (2).

&Quot; (2) "

The temporal distribution characteristic corresponding to one mask is Mask ₁ = {C ₀ , C _{1 ...} C _{n - 1} }. The feature in the entire 3D mask is Mask _Feature = {Mask ₁ , Mask ₂ , ... Mask _n }. This feature enables learning and recognition through machine learning algorithms.

When performing HMM (Hidden Markov Model) algorithm, time unit should be expressed. Therefore, the data in each mask is classified into Mask _{Feature t} = {Mask ₁ (C _i ) Mask ₂ (C _i ), ... Mask _n (C _i )}. Frame Mask _Feature All masks in _t have the same C _i Only one feature value is owned.

Applying SVM (support vector machine) algorithm can use data representation as it is. The SVM (support vector machine) must fix the input data size, so it should limit the operation time. For example, when 10 frames are calculated, n = 10, and if data is collected every 10 frames, it can be recognized as SVM.

Then, the analysis module 112 grasps at least one of a plurality of artefacts in the 3D point cloud and at least one of a position, a length, an angle value and an interlocking relationship of at least one joint connecting the arthrosis,

Comparing at least one of a plurality of artefacts of the 3D point cloud and at least one of a position, a length, an angular value and an interlocking relationship of at least one joint with predefined motion data for each of a plurality of dynamic objects, As a result, an object corresponding to similar operation data is recognized as the dynamic object.

5 shows an example of a 3D mask according to an embodiment of the present invention. In the 3D mask of Fig. 5, the red portion indicates a portion where the degree of movement of the living body is less than or equal to a threshold value, and the yellow and blue portions indicate a portion where the degree of movement is equal to or greater than a threshold.

Analyzing the dynamic object for the walking motion of the human object, a large motion is observed in the upper body and lower body parts, and the mask is defined according to the object shown on the front, side, upper face and lower face with respect to each upper body and lower body. Masks for other operations of other dynamic objects can also be measured and defined in the same way. In the case of a 3D mask corresponding to the walking motion of a human object, the motion directions in the yellow and green boxes are mostly opposite, and the fly mask can be defined as the same direction of motion of the dynamic objects in both boxes. Swim masks and climb masks differ in human and animal behavior. These parts are discerned and information is stored in advance based on learning.

 The 3D mask includes a plurality of boxes in which the degree of motion of the dynamic object is divided according to the degree of motion that is discriminated on the basis of the threshold, and a segmentation block that expresses the plurality of boxes in different colors according to the degree of motion .

Further, the 3D mask is provided in the inside of the segmentation block in a translucent or transparent manner so as to cover the dynamic object to be viewed through, and the edge of the segmentation block is provided with a solid line or a dotted line which is non-transparent or translucent.

These 3D masks are provided in different forms according to various directions and angles of the dynamic bio-objects according to operation patterns or operation types for each individual / each type of dynamic bio-object.

(A), a climbing mask (b), a fly mask (c), and a swimming mask for each dynamic type of object, for example, as described with reference to Fig. 5, Different types of 3D masks may be provided depending on the viewing direction and angle of the front, side (left, right), top and bottom, and the like.

Particularly, the 3D mask of the present invention is characterized in that it is a mask generated by reflecting characteristics of dynamic objects according to directions and angles of each object / type through simulation of occurrence of various disasters and rescue needed. That is, the 3D mask of the present invention can be said to be masks previously generated and stored in the database in order to reflect the normal actions taken by the organism in various structural rescue situations.

When the dynamic object analyzed by the analysis module 111 is determined to be living body, the image editing module 113 extracts a partial image corresponding to the dynamic object from the original of the time series image with respect to the surrounding situation.

The communication module 115 transmits the partial image corresponding to the dynamic object extracted by the image editing module 113 to the user remote device 130 when the analysis module 111 determines that the dynamic object is a living body.

At this time, when the partial image is transmitted to the user remote device 130, the communication module 115 may transmit the current position coordinate information of the dynamic object acquired through the GPS module (not shown).

The communication module 115 then receives the rescue relief command from the user remote device 130. Here, the rescue command refers to a command for executing a rescue operation through at least one of a speaker module, a receiver module, a display, an arm, and a wheel mounted on the robot.

The speaker module 117 may output the voice guidance of the user received from the user remote device so that the dynamic bio object recognizes it. For example, a user's voice guide may include a guide that provides real-time notification of a disaster situation, how it can escape from the environment in which it occurs, and its path.

The control module 118 executes a rescue operation through at least one of a speaker, a receiver, a display, an arm, and a wheel mounted on the robot according to a structure command received from a user remote device. Specifically, the control module 118 outputs the voice guidance of the user received from the user remote device 130 to a speaker so that the dynamic bio object recognizes it, or outputs the user's character guide to the display, And at least one of the arm and the wheel may be driven according to an operation command.

The user remote device 130 refers to devices in the remote control situation room provided for rescue in the event of a disaster. The user remote device 130 receives positional coordinate information of the robot and / or the detected dynamic bio-object from the robot 110 connected via the wireless communication network and outputs the positional coordinate information to the display unit 131, Generates voice command information capable of wireless transmission and transmits the generated voice command information to the robot 110 and executes a structural action through at least one of an arm, a wheel, and a beacon provided in the robot 110 Command can be transmitted. For example, the operation for causing the robot to perform the rescue operation may include outputting a voice guide of the user received from the user remote device to a speaker so that the dynamic living body object recognizes the voice guidance, outputting the user's character guide to the display, And a command to drive at least one of the arm and the wheel according to an operation command by the user.

Conventionally, since the robot captures all of the surrounding environment without selecting the surroundings by the remote control of the user, it transmits the image to the user remote device. Therefore, it takes a long time to transmit data because the size of the data is large and the image received from the user remote device is analyzed, There is a problem that the time required for data transmission and reception takes a long time to structure,

 According to the embodiment of the present invention, the robot recognizes a dynamic bio-object in a surrounding environment requiring a rescue, and sends a confirmation request to the user's remote device, thereby significantly shortening the time for rescuing the creature in the scene in the event of an accident or an event In addition, since only the image for the dynamic bio object is extracted and transmitted, the data transmission amount and the transmission time can be reduced.

9 to 11 are flowcharts illustrating a method of providing a rescue service using a robot according to an embodiment of the present invention. The detailed description of each step is the same as the one described with reference to FIGS. 1 to 8, so that detailed description will be omitted.

Referring to FIG. 9, a method for providing a rescue service using a robot according to an embodiment of the present invention will be described. In operation S510, the robot obtains a time-series image of the surrounding environment. For this step, the robot estimates a dynamic object through analysis between neighboring frames of the time-series image, and tracks the motion pattern, motion type, viewing direction, and angle information of the estimated dynamic object.

Next, the motion data of the dynamic object is obtained by analyzing the time series image (S520).

Next, the set of the operation data obtained according to the time series is analyzed on the basis of learning (S530), and it is determined whether the dynamic object is living or non-living (S540).

This step includes extracting a 3D mask matching the operation data from the database (S531), overlapping the extracted 3D mask to cover the dynamic object of the time series image (S532) (S533) a 3D point cloud including at least one joint and artefact of the dynamic object overlapped by the 3D mask, and analyzing the 3D point cloud based on the learning (S534) (S535) of determining whether the living body of the dynamic object is living body or not.

Here, the step (S534) of analyzing the 3D point cloud based on learning may include a step S5341 of obtaining a covariance matrix of the 3D point cloud corresponding to the dynamic object overlapped by the 3D mask (S5341) (S5342) of analyzing the change of the covariance matrix of the 3D point cloud according to the change of the 3D point cloud (S5342) and determining the movement information of the 3D point cloud according to the analysis (S5343).

 In addition, the step (S535) of analyzing the 3D point cloud as a learning basis (S534) and determining whether the dynamic object is a living body or a living body (S535) may include a step of selecting a plurality of artefacts and at least one joint Determining at least one of position, length, angle value, and interlocking relationship; and

Comparing at least one of a plurality of artefacts of the 3D point cloud and at least one of a position, a length, an angular value and an interlocking relationship of at least one joint with predefined motion data for each of a plurality of dynamic objects, And recognizing, as the dynamic object, an entity corresponding to the resultant similar operation data.

Here, the 3D mask includes a plurality of boxes in which the degree of motion of the dynamic object is divided according to the degree of motion that is discriminated on the basis of the threshold, and a segmentation block that expresses the plurality of boxes in different colors according to the degree of motion .

Further, the 3D mask is provided in the inside of the segmentation block in a translucent or transparent manner so as to cover the dynamic object to be viewed through, and the edge of the segmentation block is provided with a solid line or a dotted line which is non-transparent or translucent.

These 3D masks are provided in different forms according to various directions and angles of the dynamic bio-objects according to operation patterns or operation types for each individual / each type of dynamic bio-object.

For example, for each type of action, such as running, swinging, and swimming, for a dynamic bio object of an entity such as a person, Type 3D mask is provided.

Next, if it is determined that the dynamic object is a living body, a partial image corresponding to the dynamic object is extracted from an original of the time series image with respect to the surrounding situation (S550).

Then, the extracted partial image is transmitted to the user remote device (S560), and then the corresponding function is performed (S570).

Here, when transmitting the partial image to the user remote device, the current position coordinate information of the dynamic object acquired through the GPS module may be transmitted together.

The function performing step S570 may include receiving a configuration command from the user remote device and transmitting a configuration action through at least one of a speaker, a display, an arm, a wheel, and a beacon mounted on the robot according to the configuration command And executing the program.

For example, an operation for causing the robot to perform a rescue operation may include outputting a voice guide of a user received from a user remote device to a speaker so that the dynamic living body object recognizes the voice guide, outputting the user's character guide to the display, And may drive at least one of the arm and the wheel according to an operation command by the control unit.

Conventionally, since the robot captures all of the surrounding environment without selecting the surroundings by the remote control of the user, it transmits the image to the user remote device. Therefore, it takes a long time to transmit data because the size of the data is large and the image received from the user remote device is analyzed, There is a problem that the time required for data transmission and reception takes a long time to structure,

 According to the embodiment of the present invention, the robot recognizes a dynamic bio-object in a surrounding environment requiring a rescue, and sends a confirmation request to the user's remote device, thereby significantly shortening the time for rescuing the creature in the scene in the event of an accident or an event In addition, since only the image for the dynamic bio object is extracted and transmitted, the data transmission amount and the transmission time can be reduced.

 According to the structure retrieval system and method using the robot according to the embodiment of the present invention, the robot recognizes the dynamic living body object first in an environment of a bad condition requiring rescue, extracts the image of the recognized living body body, By sending only the aggregated information to the remote situation room and sending a confirmation request, the remote situation room can quickly and easily distinguish the biological object and instantly deliver the structure order, so that the life can be smoothly rescued.

Embodiments of the present invention may be implemented in the form of program instructions that can be executed on various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains.

Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

100: Rescue service system using robot
110: robot 111: sensor device
112: Analysis module 113: Image editing module
114: learning engine 115: communication module
117: Speaker module 118: Control module
130: User remote device 131:
132: command input unit 133: communication module

Claims (19)

Obtaining a time-series image of a surrounding situation of the robot;
Analyzing the time series image to obtain motion data of the dynamic object;
Analyzing the set of the operation data acquired according to the time series on a learning basis to determine whether the dynamic object is a living body or a living body;
If the dynamic object is a living body, extracting a partial image corresponding to the dynamic object from an original of the time series image with respect to a surrounding situation; And
And transmitting the extracted partial image to a user remote device,
Analyzing the time series image to obtain motion data of the dynamic object,
And estimating a dynamic object through analysis between neighboring frames of the time series image and tracking an operation pattern, an operation type, a viewing direction, and angle information of the estimated dynamic object. delete The method according to claim 1,
Wherein the step of analyzing the set of operation data obtained according to the time series on a learning basis and determining whether the dynamic object is a living body or a living body comprises:
Extracting a 3D mask matching the operation data from a database;
Overlapping the extracted 3D mask to cover the dynamic object of the time series image;
Creating a 3D point cloud comprising at least one joint and artefact of the dynamic object overlapped by the 3D mask; And
And analyzing the 3D point cloud based on learning to determine whether the dynamic object is a living body or a living body. The method of claim 3,
Analyzing the 3D point cloud based on learning,
Obtaining a covariance matrix of a 3D point cloud corresponding to the dynamic object overlapped by the 3D mask;
Analyzing a change in the covariance matrix of the 3D point cloud according to a change of a frame corresponding to the 3D mask; And
And determining movement information of the 3D point cloud according to the analysis. The method of claim 3,
The step of analyzing the 3D point cloud based on learning and determining whether the dynamic object is a living body or a living body includes:
Determining at least one of a plurality of artefacts in the 3D point cloud and at least one of a position, a length, an angular value, and an interlocking relationship of at least one joint connecting the arthrosis; And
Comparing at least one of a plurality of artefacts of the 3D point cloud and at least one of a position, a length, an angular value and an interlocking relationship of at least one joint with predefined motion data for each of a plurality of dynamic objects, And recognizing the object corresponding to the similar operation data as the dynamic object. 6. The method of claim 5,
Further comprising the step of the robot learning the 3D point clouds of a plurality of objects / types of dynamic objects in advance and storing them in a database,
Wherein learning in advance for the 3D point cloud comprises:
Wherein the robot matches at least one of an operation pattern, an operation type, a viewing direction, and an angle of the dynamic object of the dynamic object with an interlocking relationship and an angle change of motion between the arthroscope and the joint in operation, A method of providing rescue service using robots that learns class recognition. The method according to claim 1,
After the step of transmitting the extracted partial image to the user remote device,
Receiving a rescue command from the user remote device; And
And performing a rescue operation through at least one of a speaker, a display, an arm, a wheel, and a beacon mounted on the robot according to the rescue command. 8. The method of claim 7,
Wherein the act of performing the structuring action comprises:
And outputting the user's character guide to the display or displaying at least one of the arm and the wheel according to an operation instruction by the user, And providing the rescue service using the robot. Analyzing a time series image of a surrounding situation acquired through a sensor device to acquire operation data of the dynamic object and analyzing the set of operation data acquired according to the time series on a learning basis, A robot for extracting a partial image corresponding to the dynamic object from an original of the time series image with respect to a surrounding situation; And
And a user remote device receiving and displaying a partial image corresponding to the dynamic object from the robot and controlling the robot according to a structure command received from a user,
The robot includes:
And a controller for analyzing the time series image to acquire operation data of the dynamic object, analyzing the set of the operation data acquired according to the time series on a learning basis to determine whether the dynamic object is a living body or a living body, And an analysis module that estimates a dynamic object through analysis between frames and tracks an operation pattern, an operation type, a viewing direction, and angle information of the estimated dynamic object. 10. The method of claim 9,
The robot includes:
An image editing module for extracting a partial image corresponding to the dynamic object from an original of the time series image with respect to a surrounding situation when the dynamic object is a living body;
A learning engine for learning and defining at least one of a plurality of artefacts of the dynamic object and at least one of a position, a length, an angular value and an interlocking relationship of at least one joint in advance for each of a plurality of dynamic objects by object / type; And
And a communication module for transmitting a partial image corresponding to the dynamic object extracted by the image editing module to the user remote device when the dynamic object is determined to be living body by the analysis module Rescue service provision system. delete 11. The method of claim 10,
Wherein the analysis module comprises:
Extracting from the database a 3D mask that matches the motion data, overlapping the extracted 3D mask to cover the dynamic object of the time series image, interpolating at least one joint of the dynamic object overlapped by the 3D mask, Wherein the 3D point cloud is analyzed based on a learning basis to determine whether the dynamic object is a living body or a living body by using a 3D point cloud. 13. The method of claim 12,
Wherein the analysis module comprises:
Calculating a covariance matrix of the 3D point cloud corresponding to the dynamic object overlapped by the 3D mask to analyze the 3D point cloud on a learning basis and calculating a covariance matrix of the 3D point cloud according to the change of the frame corresponding to the 3D mask, Analyzing the change of the matrix and grasping the movement information of the 3D point cloud according to the analysis. 13. The method of claim 12,
Wherein the analysis module comprises:
Determining at least one of a plurality of artefacts in the 3D point cloud and at least one of a position, a length, an angle value, and an interlocking relationship of at least one joint connecting the arthrosis;
Comparing at least one of a plurality of artefacts of the 3D point cloud and at least one of a position, a length, an angular value and an interlocking relationship of at least one joint with predefined motion data for each of a plurality of dynamic objects, And obtains an object corresponding to similar operation data as the dynamic object. 13. The method of claim 12,
In the 3D mask,
A plurality of boxes in which the degree of motion of the dynamic object is divided according to a degree of motion separated based on a threshold value and a segmentation block for representing the plurality of boxes in different colors according to the degree of motion, Rescue service provision system. 16. The method of claim 15,
In the 3D mask,
Wherein the inside of the segmentation block is semi-transparent or transparent so as to cover the dynamic object to be viewed, and the edge of the segmentation block is provided with a solid or dotted line of non-transparent or translucent. 13. The method of claim 12,
In the 3D mask,
Which is generated in advance by reflecting movements of dynamic objects according to directions and angles of each object / species through simulations of various disasters and rescue needed situations, and stored in a database. 10. The method of claim 9,
The robot includes:
Further comprising a control module for executing a rescue operation through at least one of a speaker, a display, an arm, and wheels mounted on the robot according to a rescue command received from the user remote device, Delivery system. 10. The method of claim 9,
Wherein the robot simultaneously transmits current position coordinate information of the dynamic object obtained through GPS when the partial image is transmitted to the user remote device.

Images