KR101989461B1 - Safety system and Method for Robot working environment using vision system - Google Patents

Abstract

The present invention provides a system and method for safety enhancement of a robot workplace using a vision system that allows a person to enter the robotic workplace to safely exit the robotic workplace and inform the outside of the workplace accurately.

Description

Technical Field [0001] The present invention relates to a robotic workplace safety enhancement system and method using a vision system,

The present invention relates to a safety enhancement system and method for a robot workplace using a vision system.

Recently, an automatic manufacturing system using a multi-axis driving robot has been used at a work site. In the workshop, a plurality of robots corresponding to the respective component parts are provided for handling the component parts of the component parts and the respective component parts supplied in the preceding step, and a plurality of press machines are provided for component molding.

According to Korean Patent No. 10-1487166, as shown in Fig. 1, the press processing system includes a de-stacker 10 on which a material is loaded, a material transfer robot 20 for transferring the material loaded on the de-stacker 10 A positioner 30 on which the material transferred by the material transfer robot 20 is seated, a material supply robot 40 for picking up the material seated on the positioner 30 and supplying the material to the press machine 51, A press machine 51 for pressurizing and molding the mold by receiving a material from the press machine 40, 40 for supplying the molded product to the press machine 52, 53, 54 of the next step, 71 of the product take-out robot (70).

The molded product molded in the press machine 51 is supplied to the next press machine 52 through the molded product supply robot 61 and moves the slide of the press machine 51 downward to exchange the mold of the press machine 51 At the same time, after the molded product supply robot 61 is returned to the home position, the tool supply robot 61 is replaced with a tool for producing the next product. The entire equipment contained in the press system is continuously operated without interruption to improve the productivity of the product.

On the other hand, in order to inspect or repair a press machine, a robot, or a conveyor included in the press processing system, a person is divided by the fence 1 and the doors 2, 3, 4, 5, 6 , 7) into the robot work place. However, the person entering the robotic workplace is not well identifiable from outside. The press machining system operator could not identify the person who entered the work robot and operated the press processing system, thereby causing unexpected human injury.

Korean Registered Patent No. 10-1487166 (Registered on Feb. 21, 2015)

The present invention has been proposed in order to solve the problems of the prior art, and provides a system and method for safety enhancement of a robot workplace using a vision system that allows a person to enter the robot work place to safely finish work and inform the outside that the robot has left the work place precisely do.

In addition, the present invention provides a system and method for safety enhancement of a robot workplace using a vision system that a central manager can create while checking the management log of the entire robot work place through an actual image.

Other objects of the present invention will become readily apparent from the following description of the embodiments.

According to an aspect of the present invention, there is provided a system for safely enhancing a robot workplace using a vision system, including: a plurality of camera modules for photographing a plurality of surveillance regions set around a press; A person tracking device for detecting and tracking a person present in the vicinity of the press from the images input from the plurality of camera modules and generating motion information of a person including camera identification information and images; And a central management device for receiving and storing movement information of the person from the person tracking device, converting the information into screen information that can be recognized by a person, and outputting the information.

The safety work system for a robot workplace using a vision system according to the present invention is characterized in that a plurality of press lines including a press machine and a robot are divided by a fence and a plurality of gates are provided so as to be able to enter and exit the respective press lines As a safety enhancement system for a robot workplace,

Wherein each of the plurality of press lines includes a first camera module installed near a door, a plurality of second camera modules capturing a plurality of surveillance regions set around the press machine, Detects and tracks a person from the images input from the first and second camera modules, determines whether a person enters the robot work place, whether the robot is in operation in the robot work place, or leaves the robot, And a person tracking device for generating movement information of a person including any one of the exit information,

A line monitor device for receiving human tracker identification information and human motion information from each of the plurality of press lines and outputting the received information to an external device; A line monitor device identification information, a person tracking device identification information, and a person's motion information from the line monitor device, stores the received information, and converts the screen information to human-readable screen information.

According to the present invention, when a person fails to track a person in a plurality of surveillance regions set around the press before a person leaves the robot workplace, the person tracking device estimates the position of the person using the existing accumulated motion information Pan, Tilt, Zoom (Pan, Tilt, Zoom) control signal to the camera module.

According to the present invention, the central management apparatus receives target information (e.g., trouble-shooting, periodic inspection, part replacement) in which a person enters the robot workplace in response to an administrator operation input, and generates and stores a robot workplace management log.

The robot workplace safety enhancement system using the vision system according to the present invention includes an alarm unit that receives an alarm signal corresponding to one of a human entry, a work, and a departure from a human tracking device and converts the alarm signal into a signal recognizable by a human being .

According to another aspect of the present invention, there is provided a method for enhancing robots workplace safety using a vision system, comprising the steps of: receiving an image from a camera module for photographing a surveillance region around a press and a camera module installed near a door, Judging whether or not a detection signal has been detected; Tracking a position of a person in a plurality of surveillance regions when a person is detected;

Determining whether human detection and tracking information is input from a camera module installed near a door; If the result of the determination is positive, it is determined whether the person detection and tracking information is input from the camera module for photographing the surveillance area around the press. If the result is positive, it is estimated that the person entered the robot work place and the person entrance information is transmitted to the external device Wow;

Determining whether a person leaves the robot workplace while tracking a position of a person by inputting an image from a camera module for photographing a surveillance area around the press and a camera module installed near the door; And transmitting the person exit information to the external device when the person leaves the robot workplace as a result of the determination.

According to the present invention, it is possible to accurately inform the outside that a person enters the robotic work place and completes the work safely and leaves the robotic work place.

In addition, the central manager can check the management log of the whole robot work place while confirming through the actual image.

1 is an exemplary view for explaining an arrangement state of a press processing system.
FIGS. 2 and 3 are illustrations for explaining the arrangement and operation of the camera module for monitoring the periphery of the press machine.
4 is an exemplary view for explaining a vision system installed in a press processing system.
5 is a view for explaining a configuration of a safety system for enhancing robots workplace safety using a vision system according to the present invention.
FIG. 6 is a view for explaining a configuration of a human tracking device included in a safety system for enhancing robots workplace safety using a vision system according to the present invention.
FIG. 7 is a flowchart of a method for enhancing safety of a robot workplace using a vision system according to the present invention.

While the invention is susceptible to various modifications and its various embodiments, it is intended that the specific embodiments be described in detail in the drawings and will herein be described in detail, but it is not intended to limit the invention to the specific embodiments, It is to be understood that the invention includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout.

FIGS. 2 and 3 are illustrations for explaining the arrangement and operation of the camera modules 21, 22, 23 and 24 for monitoring the periphery of the press machine 50. FIG.

The press machine 50 installed in the robot work place is a mechanical device for molding or cutting an iron plate by injecting an iron plate or the like as a working material between the upper and lower molds. The robot 60 installed in the periphery of the press machine 50 is a material supplying robot for sucking the material seated on the positioner and supplying it to the press machine 50 or a molding machine for supplying the molded article molded in the press machine 50 to the press machine in the next step May be a supply robot. The press machine 50 and the robot 60 installed in the robot work site continuously operate without interruption to improve the productivity of the product.

The camera modules 21, 22, 23 and 24 are installed to monitor the periphery of the press machine 50. A plurality of surveillance regions A1, A2, A3, A41, and A42 photographed by the camera modules 21, 22, 23, and 24 in FIGS. 2 and 3 may overlap each other. However, the camera modules 21, 22, 23, and 24 are controlled so as to face the periphery of the press machine 50, that is, the space where the probability of existence of a person is high.

The camera modules 21, 22, 23, and 24 may be operated only when the human presses or checks the press machine 50 and the robot 60, or is always operated to operate. Referring to FIGS. 2 and 3, the camera modules 21, 22, 23 and 24 are connected to a plurality of surveillance areas A1, A2, and A3 according to a pan, tilt, zoom, A3, A41, and A42, as shown in Fig. For example, the vertical rotation range includes -90 ° to + 40 °, and the horizontal rotation range includes 0.1 ° to 100 °, and can be automatically adjusted according to the degree of zoom. For example, only the camera modules 21 and 24 for photographing the surveillance areas A41 and A42 provided with the robot 60 around the press machine 50 can be rotated vertically or horizontally.

The camera modules 21, 22, 23, and 24 may be implemented by an analog camera, a digital camera, or an IP camera. The analog camera uses a coaxial cable to generate and output a CVBS (composite video baseband signal) signal that can be transmitted up to 300 meters.

Digital cameras support a high-definition serial digital interface (HD-SDI) that can be used to transmit uncompressed high-definition digital video over a coaxial cable at a 1.5Gbps rate. Cable distances up to 100 meters are supported at these high transfer rates. IP cameras transmit compressed high-definition digital video.

A signal frequency band corresponding to a plurality of video signals output from the IP camera includes a composite video baseband signal (CVBS), a downstream stream, and an upstream stream. The CVBS (composite video baseband signal) signal uses SD analog video up to 5MHz band and the downstream has 8MHz bandwidth around 18.75MHz and transmits HD IP video data at a transmission rate of 36Mbps. Upstream has a 2.5MHz bandwidth around 39MHz and transmits data such as camera control signals at a transmission rate of up to 11Mbps.

4 is an exemplary view for explaining a vision system installed in a press processing system. The press processing system comprises a deformer 10 on which a material is loaded, a material transfer robot 20 for transferring the material loaded on the deformer 10, a positioner 30 on which the material transferred by the material transfer robot 20 is seated A material supply robot 40 for picking up a material placed on the positioner 30 and supplying the material to the press machine 51, a press machine 51 for supplying the material from the material supply robot 40 and press-molding the metal mold, 62 and 63 to be supplied to the press machines 52, 53 and 54 of the next step and a product carry-out robot 70 for carrying out the molded products to the conveyor 71. [

The press processing system is partitioned by a fence 1, and doors 2, 3, 4, 5, 6, 7 are provided in the fence 1. [ The vision system installed in the press processing system includes a plurality of camera modules 311, 321, 331, 341, and 351 installed near the respective doors 2, 3, 4, 5, 6, A plurality of camera modules 312 to 315, 322 to 325, 332 to 335 and 342 to 345 installed around the conveyor 71 and a plurality of camera modules 352 to 355 installed around the conveyor 71 can do.

A plurality of camera modules 311, 321, 331, 341, and 351 installed near each of the doors 2, 3, 4, 5, 6, and 7 are used to determine whether a person enters and exits the inside of the press processing system And provides the image information so that it can detect and track the information. A plurality of camera modules 312 to 315, 322 to 325, 332 to 335, and 342 to 345 installed around the respective press machines 51, 52, 53 and 54 are disposed in the periphery of the press machine 50, And is controlled to face a high space. A plurality of camera modules 312 to 315, 322 to 325, 332 to 335 and 342 to 345 installed around the respective press machines 51, 52, 53 and 54 are connected to the press machine and the robots 40, 61, 62, 63 and 70 ), Or may only be operated when a person enters for repair, or may be implemented to operate at all times.

Although not shown in FIG. 4, a network video recorder (NVR), which is connected to a plurality of camera modules through an analog coaxial cable or a twisted pair cable, may be installed. The network video recorder (NVR) is a device that automatically sets the IP address and performs communication with the camera module when the camera module is connected. The network video recorder (NVR) 300 includes a plurality of ports PORT for inserting headers of analog coaxial cables connected to the camera module, a SLOC-R modem connected to the port PORT, an Ethernet switch, A video controller, and an analog video decoder.

When a camera identification number and an output signal including a video signal are input from a camera module, the SLOC-R modem separates the baseband video signal and the digital video signal included in the output signal by frequency to remove distortion from the digital video signal And compensates for the attenuation of the baseband video signal and outputs it. The Ethernet switch detects whether the header of the analog coaxial cable connected to the camera module is inserted into the port. The host processor receives the digital video signal from the SLOC-R modem and stores it in a storage unit (HDD, USB), provides the digital video signal to the digital video decoder, and provides the upstream data to the SLOC-R modem.

FIG. 5 is a view for explaining a configuration of a safety system for enhancing robots workplace safety using a vision system according to the present invention, FIG. 6 is a view for explaining a configuration of a human being tracking system Fig.

The robot workplace safety enhancement system using the vision system according to the present invention is implemented by including a plurality of camera modules 411 to 415 and a human tracking device 416. The camera modules 411 to 415 are installed so that there are no dead zones around the press machine and the robot. The camera modules 411 to 415 may be operated only when the human being enters and working for checking or repairing the press machine and the robot, or may be implemented to always operate. The camera modules 411 to 415 may be implemented by an analog camera, a digital camera, or an IP camera. The person tracking device 416 detects a person around the press and the robot using the image information photographed from the camera modules 411 to 415, and tracks the movement of the detected person.

A plurality of press lines (S1 to S9) are partitioned by the fence (1) and managed for each production process. As shown in FIG. 5, for example, a robot workplace safety enhancement system using a vision system according to the present invention may be implemented by including a plurality of line monitoring apparatuses 401, 402, 403 and a central management apparatus 500 . Each of the line monitoring devices 401, 402, and 403 may be configured to receive human motion information and image information from a plurality of human tracking devices and display them on a separate monitor screen or to transmit them to the central management device 500. For example, the first line monitoring apparatus 401 may be configured to monitor human movement information (e.g., entering, working, leaving, etc.) at each person tracking device 416, 426, 436 installed in a plurality of press lines S1, S2, Is input, the time at which the motion information is generated and the image information can be displayed on the monitor screen.

The central management apparatus 500 receives information on the time at which the motion information of the person is generated from the plurality of line monitoring apparatuses 401, 402, 403, the motion information of the person including the image information and the person tracking device identification information (e.g., Exits) is received and stored, and can be displayed on the surveillance screen. The manager can use the information of the stored person's motion to input the entry information (eg, troubleshooting, periodic inspection, part replacement) to the robot workplace. Accordingly, the central management apparatus 500 can generate and store management journals including the press machine of each robot work place, the state of the robot, parts replacement, and repair contents.

Hereinafter, a person tracking system according to the present invention will be described with reference to FIG. 6, the human tracking device 416 includes an interface 4161 connected to a plurality of camera modules 411 through 415, an image processing unit 4162, an image storage unit 4163, a safety control unit 4164, a communication unit 4165, And an alarm unit 4166.

The interface 4161 may be implemented with a plurality of ports PORT into which the headers of the analog coaxial cable or twisted pair cable are inserted, a SLOC-R modem connected with the port PORT, and an Ethernet switch. When an output signal is input from the IP camera 100, the SLOC-R modem separates the baseband video signal and the digital video signal included in the output signal by frequency to remove distortion from the digital video signal, And outputs the compensated signal. The Ethernet switch detects whether the header of the cable connected to the camera module is inserted into the port.

The image processing unit 4162 stores the HD compressed image for recording among the data received via the interface 4161 in the image storage unit 4163. The image processing unit 4162 includes a human detection unit for detecting a person from the reconstructed image after restoring the data received through the compressed image decoder and a position tracking unit for tracking the position of the detected person.

The human detection unit detects a motion region, which is a candidate region of human detection, from the image, and recognizes whether or not the detected motion region is a human region. For example, the motion region detection may be performed by calculating statistical characteristics of a background region from scenes having a predetermined length, and comparing the statistical characteristics of the background region with the statistical characteristics of a newly input image, A moving object region can be divided into pixels using a connected component labeling technique, and the motion region can be detected by performing such an operation on each screen. In addition, the motion area detection algorithm may use the color information of the object. Various known techniques such as a Histogram of Oriented Gradients (HOG) algorithm can be applied to the motion region detection algorithm.

The method of recognizing whether the detected motion region is a human region is performed by generating a slope histogram of a region from the head to the shoulder, which is a candidate region of human detection, to obtain the direction and size of the pixel unit slope, Value is normalized in units of cells to obtain a feature vector, and the feature vector is compared with a predefined person area feature vector to determine whether the feature vector is similar to a certain probability value or more.

The position tracking unit tracks the recognized human region by calculating the center coordinates of the human region as the locus of the human region over time. The position locator can estimate the position of the person using the accumulated motion information if the person does not move or the occlusion occurs in the workplace.

The safety control unit 4164 transmits a camera control signal to the camera modules 411 to 415. When the camera identification information and the human tracking information are inputted in the image processing unit, the safety control unit 4164 generates human motion information (e.g., entering, And transmits it to the first line monitoring apparatus 401 via the communication unit 4165. [ In addition, the safety control unit 4164 outputs the motion information (e.g., entry, work, and exit) of the person through the alarm unit 4166. The alarm unit 4166 is installed near the door, and receives an alarm signal corresponding to one of the entrance, the operation, and the exit from the safety control unit 4164 and converts it into a signal that can be recognized by a person. For example, an LED diode or an LCD display.

The safety controller 4164 generates a control signal for controlling the operation of the camera modules 411 to 415 and transmits the control signals to the camera modules 411 to 415 while tracking the position of a person entering through the door.

FIG. 7 is a flowchart of a method for enhancing safety of a robot workplace using a vision system according to the present invention. For example, as shown in FIG. 4, the vision system installed in the press processing system includes a plurality of camera modules 311, 321, 331, and 332 installed near the respective doors 2, 3, 4, 5, And a plurality of camera modules 312 to 315, 322 to 325, 332 to 335, and 342 to 345 installed around the press machines 51, 52, 53 and 54 and around the conveyor 71 It is assumed that a plurality of camera modules 352 to 355 are included.

First, the method for safety enhancement of a robot workplace using a vision system drives a human tracking device connected to a plurality of camera modules 352 to 355, and then drives a camera module (S711). For example, the person tracking device can control the camera module to change from the idle mode to the active mode when the door open signal is input. As another example, the camera module may be implemented to always operate.

Thereafter, the person tracking device receives the image from the camera module and stores the image in the image storage unit (S712), and determines whether a person is detected from the image (S713). As a result of the determination, if a person is detected, the position of the person is tracked in a plurality of surveillance regions (S714).

Thereafter, the person tracking device determines whether a person enters the robot worksite or exits the workplace from the image transmitted from the camera module installed near the door (S715, S721). The person tracking device transmits the motion information (e.g., entry and exit) of the person to the external device in steps S715 and S721 (S716 and S722). For example, the external device may be an LED diode or LCD display installed near the door. As another example, the external device may be a central management device.

After step S716, the person tracking device determines whether human tracking has failed (S717). For example, when a person is not moving or a blind phenomenon occurs in the workplace, the human region in the current screen is compared with a previously defined human region feature vector, and the probability is below a certain probability value.

If the person tracking device fails to track the person, the position of the person is estimated within the plurality of audit areas (S718). For example, the position of a person can be estimated using existing accumulated motion information. As another example, the human tracking device may transmit a pan, tilt, zoom (Pan, Tilt, Zoom) control signal to the camera module to change the photographing position of each camera module to detect and track the position of the person.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention as defined by the appended claims. Accordingly, the true scope of the present invention should be determined only by the appended claims.

50: Pressing machine
60: Mold supply robot
21, 22, 23, 24: camera module

Claims (6)

delete A plurality of press lines including a press machine and a robot are divided by a fence and a plurality of gates are provided in the fence so as to be able to enter and exit the respective press lines,
Wherein each of the plurality of press lines includes:
A first camera module installed near the door, a plurality of second camera modules capturing a plurality of surveillance areas set around the press machine;
Detects and tracks a person from the images input from the first and second camera modules, determines whether a person enters the robot work place, whether the robot is in operation in the robot work place, or leaves the robot, And a person tracking device for generating movement information of a person including any one of the exit information,
A line monitor device for receiving human tracker identification information and human motion information from each of the plurality of press lines and outputting the received information to an external device;
And a central management unit for receiving the line monitoring device identification information, the human tracking device identification information, and the human motion information from the line monitoring device, storing the inputted information, converting the information into screen information that can be recognized by a human being,
The person tracking device comprises:
When a person fails to track a person within a plurality of surveillance areas set around the press before a person enters the robot work site after exiting the robot, the position of the person is estimated using the existing accumulated motion information,
The central management apparatus comprising:
A robot control unit for generating and storing a robot work center management log by receiving a target information of a person entering a robot work center according to an administrator operation input,
Robot Workplace Safety Enhancement System Using Vision System. delete delete The method of claim 2,
An alarm unit receiving an alarm signal corresponding to any one of a human entry, a work, and an exit from the human tracking device and converting the alarm signal into a signal recognizable by a person and outputting the signal;
A safety system for robotic workplace using a vision system. Receiving an image from a camera module for photographing a surveillance area around a press and a camera module installed near a door, storing the image in an image storage unit, and determining whether a person is detected from the image;
When a person is detected, the position of a person is tracked in a plurality of surveillance areas. If a person is not tracked within a plurality of surveillance areas set around the press before a person enters the robot work area, Estimating a position of a person using the position information;
Determining whether human detection and tracking information is input from a camera module installed near a door;
If the result of the determination is positive, it is determined whether the person detection and tracking information is input from the camera module for photographing the surveillance area around the press. If the result is positive, it is estimated that the person entered the robot work place and the person entrance information is transmitted to the external device Wow;
Determining whether a person leaves the robot workplace while tracking a position of a person by inputting an image from a camera module for photographing a surveillance area around the press and a camera module installed near the door;
Transmitting the person exit information to an external device when the person leaves the robot work site as a result of the determination; And
Generating and storing a robot workplace management logbook by inputting objective information of a person entering a robot workplace according to an administrator operation input;
A method for safety enhancement of a robot workplace using a vision system including a robot.

Images