KR20230163055A - System, apparatus and method for detecting object - Google Patents

Abstract

An object detection system relates to an object detection system, an object detection device, and an object detection method. The object detection system includes a photographing unit that captures an image of a monitoring space and an object within the monitoring space, and obtains depth information about the object, and extract the object from the image, obtain relative coordinates for the object on a three-dimensional map corresponding to the monitoring space, create a virtual zone for the object using the relative coordinates, and It may include an object detection device that determines whether zones overlap, and a warning unit that outputs a warning when the virtual zone overlaps with the danger zone.

Description

Object detection system, apparatus and method {SYSTEM, APPARATUS AND METHOD FOR DETECTING OBJECT}

It relates to object detection systems, devices, and methods.

Workers at work sites such as factories or construction sites are exposed to various risks. In particular, at these sites, there are many areas where accidents are likely to occur when handled by low-skilled workers or approached by inexperienced people, such as machine tools, temporary structures, or excavation slopes. To prevent safety accidents in these areas, access to these areas must be controlled and workers must be properly supervised. Conventionally, proximity sensors, infrared sensors, etc. have been used to prevent safety accidents by outputting an audio-visual warning in response when these sensors detect a worker's approach to a hazardous area. However, these proximity sensors and infrared sensors have narrow measurement angles and short measurement distances, making it difficult to detect all movements of workers, and thus have the problem of not being able to sufficiently detect workers' approaches to dangerous areas. In addition, proximity sensors and infrared sensors cannot identify individual workers, so a warning is sometimes output even when an authorized worker approaches. This has been an impediment to proper management supervision at the work site.

The problem sought to be solved is to provide an object detection system, device, and method that can provide a warning when an object, such as a worker operating within at least one area, approaches or enters a hazardous area.

In order to solve the above-mentioned problems, an object detection system, an object detection device, and an object detection method are provided.

The object detection system includes a monitoring space and an image of an object within the monitoring space, a photographing unit that acquires depth information about the object, extracts the object from the image of the object, and 3 devices corresponding to the monitoring space. An object detection device that obtains relative coordinates for the object on a dimensional map, creates a virtual zone for the object using the relative coordinates, and determines whether the virtual zone and the danger zone overlap, and the virtual zone and the danger zone. If areas overlap, a warning section that outputs a warning may be included.

The object detection device includes coordinates in the image for at least one point of the object, coordinates of the width direction center of the image, coordinates of the height direction center of the image, the width of the image, and the height of the image. Relative coordinates with respect to the object may be obtained using the width angle of view of the photographing unit, the height direction angle of view of the photographing unit, and a size adjustment variable for adjusting the size of the virtual zone.

The object detection device acquires the depth of at least one point of the object, and selects one of a first depth obtained by subtracting a depth determination value from the depth and a second depth obtained by subtracting the depth determination value from the depth. One more method may be used to obtain relative coordinates for the object.

The virtual area may include a virtual hexahedron with the relative coordinates obtained using the first depth as the coordinates of the four corners of the front and the relative coordinates obtained using the second depth as the coordinates of the four corners of the back. It may be possible.

The object detection device may further determine the class of the object, and the warning unit may be provided not to output the warning when the class of the object can approach or enter the danger zone.

The object detection device includes a monitoring space and an image of an object within the monitoring space, a photographing unit for acquiring depth information about the object, extracting the object from the image of the object, and a device corresponding to the monitoring space. A processor that acquires relative coordinates for the object on a three-dimensional map, creates a virtual zone for the object using the relative coordinates, and determines whether the virtual zone and the danger zone overlap, and the virtual zone and the danger zone. If this overlaps, it may include a warning section that outputs a warning.

The object detection method includes: capturing a monitoring space and an image of an object within the monitoring space; acquiring depth information about the object; extracting the object from an image of the object; and corresponding to the monitoring space. Obtaining relative coordinates for the object on a three-dimensional map, generating a virtual zone for the object using the relative coordinates, determining whether the virtual zone overlaps with the risk zone, and the virtual zone If the and danger zones overlap each other, it may include the step of outputting a warning.

According to the above-described object detection system, device, and method, when an object such as a worker operating within at least one area approaches or enters a hazardous area, the effect of being able to appropriately warn of the approach or entry into the hazardous area is achieved. You can get it.

According to the above-described object detection system, device, and method, it is possible to accurately and quickly recognize whether a worker working in a workspace with a hazardous area is approaching or entering the hazardous area, and based on this, warns the worker to prevent the worker from Safety-promoting effects can also be achieved.

According to the above-described object detection system, device, and method, it is possible to eliminate or minimize the undetectable section that occurs due to the narrow measurement angle or measurement distance of existing detection sensors.

According to the above-described object detection system, device, and method, it is possible to selectively output a warning about approaching or entering a dangerous area depending on whether each worker can enter, access, or is permitted.

1 is a schematic diagram of one embodiment of an object detection system.
Figure 2 is a block diagram of an embodiment of an object detection device.
FIG. 3 is a diagram illustrating a three-dimensional map of a dangerous area and an example of a detected object block and a target area block.
Figure 4 is a diagram showing an example of relative coordinates on a 3D map.
Figure 5 is a flowchart of an embodiment of an object detection method.

Throughout the specification below, the same reference signs refer to the same components unless there are special circumstances. Terms with the addition of 'unit' used below may be implemented as software and/or hardware, and depending on the embodiment, one 'unit' may be implemented as one physical or logical part, or a plurality of 'units' may be implemented. It is also possible to be implemented with one physical or logical part, or one 'part' to be implemented with a plurality of physical or logical parts. When a part is said to be connected to another part throughout the specification, this may mean that the part and the other part are physically connected to each other and/or electrically connected. In addition, when a part includes another part, this does not mean excluding another part other than the other part unless specifically stated to the contrary, and means that another part may be included depending on the designer's choice. do. Expressions such as the first to Nth (N is a natural number greater than or equal to 1) are intended to distinguish at least one part(s) from other part(s), and do not necessarily mean that they are sequential unless otherwise specified. Additionally, singular expressions may include plural expressions, unless the context clearly makes an exception.

Hereinafter, an embodiment of an object detection device and an object detection system including the same will be described with reference to FIGS. 1 to 4.

1 is a schematic diagram of one embodiment of an object detection system.

As shown in FIG. 1, in one embodiment, the object detection system 10 may include at least one image (at least one of a still image and a moving image) for at least one space (1, hereinafter referred to as monitoring space). Based on the image acquired by the imaging unit 11 and the imaging unit 11 (the same applies hereinafter), the object 9 in the monitoring space 1 approaches the danger zone (z2 in FIG. 3) Alternatively, it may include an object detection device 100 that determines whether entry has occurred, and a warning unit 19 that outputs a warning signal according to the detection result of the object detection device 100.

At least two of the photographing unit 11, the object detection device 100, and the warning unit 19 may be connected through a wired or wireless communication network. Here, the wireless communication network is a short-range communication network such as Wi-Fi, Wi-Fi Direct, Bluetooth, or Near Field Communication (NFC), or 3GPP, 3GPP2, WiBro, or It may also include a mobile communication network implemented based on mobile communication standards such as the WiMAX series.

The imaging unit 11 is installed at least one of the inside and outside of the monitoring space 1, and may be installed one or more than one for one monitoring space 1 as needed. The photographing unit 11 may include a camera device. The camera device includes a lens for focusing light (visible light or infrared light, etc.), and an image sensor (which may be implemented including a plurality of imaging devices) that outputs an electrical signal corresponding to the image based on the focused light. can do. According to one embodiment, the camera device may include, for example, an RGB-Depth camera (hereinafter referred to as an RGB-D camera). The RGB-D camera recognizes three-dimensional depth (for example, it can include the distance between the imaging unit 11 and the object 9 or the distance between the boundary of the monitoring space 1 and the object 9). Additional modules may be included. Depending on the embodiment, the RGB-D camera includes a plurality of lenses and an image sensor to obtain a plurality of image data for at least one object 9, thereby measuring the depth of the object 9 based on the difference between the plurality of images. It may be performed (stereo camera), or the depth measurement of the object 9 may be performed based on the travel time of the infrared light by irradiating infrared light through an infrared light source and then receiving the infrared light reflected from the object 9. (Tof camera). If the photographing unit 11 is implemented using such an RGB-D camera, not only information about the shape or color of the object 9 inside the monitoring space 1 but also depth information of the object 9 can be obtained. There will be.

The photographing unit 11 may transmit the acquired image to the object detection device 100 through a wired or wireless communication network. Depending on the embodiment, the photographing unit 11 may obtain information about depth (for example, a plurality of images captured at different angles at the same time, the delay time of a received optical signal, or an object obtained by calculating based on at least one of these). Depth information (9), etc.) may be further transmitted to the object detection device 100.

The object detection device 100 uses the image acquired by the photographing unit 11 or, depending on the embodiment, further uses information about depth to detect objects such as humans (workers, outsiders, etc.) within the monitoring space 1 (9). ) is present and whether the object 9 has approached or entered the danger zone z2.

If necessary, the object detection device 100 may generate a corresponding signal based on the determination result and transmit it to the warning unit 19, thereby allowing the warning unit 19 to perform a warning operation. Depending on the embodiment, the object detection device 100 may include two or more physically separate information processing devices, for example, a first processing device 100a and a second processing device 100b. These processing devices 100a and 100b may be implemented using the same or different devices, and may include, for example, server hardware devices, desktop computers, smart phones, or programmable network switches. Two or more information processing devices 100a and 100b are connected to each other so that they can communicate with each other, and are each capable of performing some or all of the operations performed by the object detection device 100. For example, the first processing device 100a collects images captured by the photographing unit 11, detects the object 9, determines whether the danger zone z2 is approached, and/or controls the warning unit 19. It is arranged to perform operations such as, and the second processing device 100b stores a virtual space, for example, a 3D map (z1), for determining whether the danger zone (z2) is accessible, setting the danger zone (z2), It may be provided to perform functions such as monitoring or management of each device 11, 19, and 100a and/or sending and receiving data or commands to and from the user terminal device 90. Depending on the embodiment, the object detection device 100 may also be implemented using a single physical device. Detailed operations of the object detection device 100 will be described later.

The warning unit 19 displays a warning signal visually and/or in response to an object (9, for example, a worker) not permitted to approach or enter the danger zone (z2). Alternatively, it can be output audibly. Accordingly, the worker (9) must determine whether he or she is approaching or entering the hazardous area (z2), or a person responsible for managing the worker (9) (workplace manager, supervisor, etc.) You can immediately know whether you are approaching or entering z2). Depending on the embodiment, the warning unit 19 may be placed within the monitoring space 1 and/or installed in an area other than the monitoring space 1 (e.g., the workplace of a workplace manager or supervisor, etc.). It could be. The warning unit 19 may include a sound output device such as a siren, a warning light, a speaker device, a smartphone, a smart watch, a smart band, a headset, a head mounted display (HMD), a tablet PC, a desktop computer, or a laptop computer. However, it is not limited to this.

Depending on the embodiment, the object detection system 1 is a user terminal device 90 that can visually or audibly check an image (for example, an image captured by an RGB-D camera) or a detection result of the object detection device 100. ) may further be included.

The user terminal device 90 is communicatively connected to the object detection device 100, connects to the object detection device 100, receives images captured from the object detection device 100, and outputs them to the outside to the user. It can be provided to (for example, workers or managers). The user terminal device 90 may include, for example, a smartphone, tablet PC, desktop computer, or laptop computer.

Hereinafter, with reference to FIGS. 2 to 4 , the object detection device 100 will be described in more detail.

FIG. 2 is a block diagram of an embodiment of an object detection device, and FIG. 3 is a diagram illustrating a three-dimensional map of a dangerous area, a detected object block, and an example of a target area block.

As shown in FIG. 2, in one embodiment, the object detection device 100 connects to a wired or wireless communication network, receives information about the image or depth from the photographing unit 11, and sends the determination result to the warning unit 19. A communication unit 101 that transmits a signal corresponding to a communication unit 101, a storage unit 102 that temporarily or non-temporarily stores data or programs (apps, applications, or software, etc.) necessary for the operation of the object detection device 100, It may include a processor 110 that performs operations such as detecting an object based on information about the image and depth.

Here, at least two of the communication unit 101, the storage unit 102, and the processor 110 are electrically connected to each other and can transmit data or commands in one or both directions. The communication unit 101 may include a LAN card or may be implemented using an antenna and a wireless communication chip.

The storage unit 102 may include at least one of a main memory and an auxiliary memory, and these can be implemented based on a semiconductor chip or magnetic disk. The processor 110 includes a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller unit (MCU), an application processor (AP), and an electronic control unit. It may be implemented using a unit (ECU: Electronic Controlling Unit) and/or at least one device capable of performing various calculations and control processing. If necessary, the object detection device 100 includes an input unit (for example, a keyboard, touch screen, microphone, or data input/output terminal, etc.) for receiving commands or data from a user, etc., and provides information visually or audibly to the user, etc. It may further include an output unit (for example, a display, a speaker device, a video output terminal, or a data input/output terminal, etc.) for provision.

In one embodiment, the storage unit 102 may store one or more 3D maps (z1) corresponding to the monitoring space (1). In this case, the 3D map z1 stored in the storage unit 102 may be prepared to correspond to each installed imaging unit 11. Accordingly, the storage unit 102 may store the number of three-dimensional maps z1 corresponding to the number of the photographing units 11 installed for photographing the monitoring space 1 (for example, the same number).

The 3D map (z1) is a virtual space constructed by reflecting the monitoring space (1), and the overall shape of the 3D map (z1) is given in the form of a hexahedron (cube, cuboid, etc.) as shown in FIG. 3. It may be possible. However, the shape of the 3D map (z1) is not limited to this and may be defined in various ways depending on the overall shape of the monitoring space (1). Each of at least one point(s) within the 3D map (z1) may be expressed as relative coordinates based on a predetermined reference point (e.g., the center or one corner point of the 3D map (z1), etc.) , for example, may be expressed using a Cartesian coordinate system based on three-dimensional axes (x-axis, y-axis, and z-axis). Depending on the embodiment, each point of the 3D map z1 may be expressed in a spherical coordinate system or a cylindrical coordinate system.

In one embodiment, as shown in FIG. 2, the processor 110 includes an object detection unit 111, a depth extraction unit 112, a virtual area creation unit 113, an overlap confirmation unit 114, and a class It may include a determination unit 115 and a warning control unit 116. These (111 to 116) may be logically or physically distinct from each other. If necessary, at least one of these (111 to 116) can be omitted.

The object detection unit 111 may detect the object 9 from the image acquired by the photographing unit 11 and extract the object 9 from the image. In one embodiment, detection and extraction of the object 9 in the image may be performed using at least one learning algorithm. Here, the learning algorithm is, for example, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), and a convolutional recurrent neural network (CRNN). Neural Network, Deep Belief Network (DBN), Deep Q-Networks, Long short term memory (LSTM), Multi-layer Perceptron, Support Vector Machine ( It may include, but is not limited to, at least one learning model such as a support vector machine (SVM), a generative adversarial network (GAN), and/or a conditional generative adversarial network (cGAN). no.

When the object 9 is extracted from the image, the depth extraction unit 112 may determine the depth of the object 9 (for example, the distance between the photographing unit 11 and the object 9) in response. Determination of depth may be performed using information about depth acquired by the imaging unit 11.

For example, the depth extraction unit 112 determines the depth of the object 9 using a triangulation method based on the positions of the plurality of photographing units 11 and the plurality of images acquired by each photographing unit 11. Alternatively, the depth of the object 9 may be obtained using the difference between the point of time at which infrared light is irradiated and the point at which reflected infrared light is received. The acquired depth may be transmitted to the virtual zone generator 113.

As shown in FIG. 3, the virtual zone generator 113 generates information about the subject in the image acquired by the photographing unit 11, for example, the extracted object 9, to the corresponding photographing unit 11. At least one virtual zone (z3) corresponding to the object 9 may be created on the corresponding 3D map (z1).

As described above, each point in the three-dimensional map (z1) may have relative coordinates (x, y, z), and accordingly, all or part of the object (9) in the monitoring space (1) also has the corresponding relative coordinates. has The virtual zone generator 113 can use this to obtain relative coordinates for the object 9. That is, when the virtual zone generator 113 extracts the object 9 from the image acquired by the imaging unit 11, the virtual zone generator 113 uses the coordinates for the object 9 on the image to create the corresponding object ( 9) At least one relative coordinate can be obtained.

According to one embodiment, the virtual zone generator 113 may use the depth extracted by the depth extractor 112 to obtain at least one relative coordinate of the object 9 on the 3D map z1. . More specifically, for example, the virtual zone generator 113 uses a combination of the depth of at least one point of the object 9, the coordinates on the image of the point, and the angle of view of the photographing unit 11 to determine the object ( One or more relative coordinates for all or part of 9) may be obtained. For example, when the monitoring space 1 is expressed as a three-dimensional map (z1) in the shape of a rectangular parallelepiped (i.e., when using a Cartesian coordinate system), the x value ( That is, the relative coordinate value for the left and right directions) is determined, for example, using Equation 1 below, and the y value (relative coordinate value for the up and down directions) is calculated and obtained using Equation 2 below. It could be.

[Equation 1]

[Equation 2]

In Equations 1 and 2, x and y mean relative coordinates in the three-dimensional map (z1) for at least one point of the object 9, and pos_x and pos_y are the imaging unit ( 11) refers to the coordinates within the acquired image. center_x and center_y refer to the coordinates of the width-direction center and height-direction center of the image, respectively, and Width and height refer to the width and height of the image, respectively. Depth is the depth extracted by the depth extraction unit 112. fov_w and fov_h are the field of view (FOV) in the width direction and the angle of view in the height direction of the photographing unit 11, respectively. Meanwhile, α is a size control variable for adjusting the size of the virtual zone (z3) to be created, and may vary depending on the settings of the user, etc. The size adjustment variable (α) may be set to, for example, 1, or may be set to a value exceeding 1. When the scaling variable (α) is 1, the size of the virtual zone (z3) for the object 9 is given as the same as the area of the actually detected object 9, and the scaling variable (α) exceeds 1. In this case, the virtual zone (z3) is formed to be relatively larger.

Figure 4 is a diagram showing an example of relative coordinates on a 3D map.

As described in Equation 1 and Equation 2, the x value and y value of the relative coordinates of at least one point of the object 9 on the 3D map z1 are determined by the depth extracted by the depth extractor 112. It can be determined proportionally. In more detail, as shown in Figure 4, when the width direction view angle (fov_w) and the height direction view angle (fov_h) are both 90 degrees and the depth is given a default value (for example, 1 m), the upper left corner of the square The coordinates of are, for example, (-0.5, -0.5, 1), the coordinates of the upper right corner are (0.5, -0.5, 1), and the coordinates of the lower left corner are (-0.5, 0.5, 1). , the coordinates of the lower right corner may be (0.5, 0.5, 1). In this case, if the depth increases to twice the default value (e.g. 2m), the coordinates of the upper left corner of the corresponding rectangle are given as (-1, -1, 2), and the coordinates of the upper right corner are It is given as (1, -1, 2), the coordinates of the lower left corner are given as (-1, 1, 2), and the coordinates of the lower right corner are given as (1, 1, 2). That is, in proportion to the increase in depth, the value of each relative coordinate also increases proportionally. Therefore, it is possible to determine relative coordinate values in the x-axis direction and y-axis direction of a specific point based on the depth to the specific point.

When at least one relative coordinate is generated as described above, the virtual zone generator 113 creates a virtual zone (z3) corresponding to the object 9 using a method such as combining one or more relative coordinates, The object 9 can be converted into a virtual area (z3) on the 3D map (z1). In this case, the created virtual zone z3 may have a shape corresponding to the object 9. Here, the shape corresponding to the object 9 may be, depending on the embodiment, the same shape as the object 9, the shape of a virtual cube (for example, a cube or a rectangular parallelepiped) including all or part of the object 9, or the object ( It may include an ellipsoid shape including all or part of 9), but is not limited thereto.

If the virtual zone (z3) is created in the form of a hexahedron containing all or part of the object 9, the virtual zone generator 113 uses the above-mentioned equations 1 and 2 to generate the object 9. To obtain the coordinate value in the x-axis direction and the coordinate value in the y-axis direction for at least one point (for example, four points such as the upper end, lower end, and left and right end of the object 9) on the end boundary of (9), respectively. can do. In this case, the virtual zone generator 113 determines two different depths (hereinafter referred to as first depth and second depth), and uses the two depths to determine the coordinate values of the four corners of the front of the hexahedron and the four corners of the back of the hexahedron. By obtaining the coordinate values of the corners, the coordinate values for the eight corners of the hexahedron corresponding to the virtual zone (z3) can also be obtained. In this case, the first depth may be obtained by subtracting a predetermined value (hereinafter referred to as a depth determination value) from the depth extracted by the depth extraction unit 112, and the obtained first depth can be calculated using Equation 1 and Equation 2. By substituting into the depth, the coordinates of the four corners of the corresponding rectangular parallelepiped are obtained.

In addition, the second depth may be obtained by adding the depth determination value to the depth extracted by the depth extractor 112, and when the second depth is substituted for the depth of Equation 1 and Equation 2, the rear of the corresponding rectangular parallelepiped The coordinates of the four corners of the direction are obtained. Accordingly, a virtual zone z3 in the shape of a rectangular parallelepiped corresponding to the detected object 9 can be obtained. In this case, the front-to-back length of the rectangular parallelepiped corresponding to the virtual zone (z3) is given as twice the depth determination value. Here, the depth determination value may be half of the above-described basic value (for example, 0.5 m), but is not limited thereto. Designers, users, etc. can determine the front and rear lengths in various ways depending on their wishes.

When the virtual zone z3 of a predetermined shape (for example, a rectangular parallelepiped shape) is created as described above, the created virtual zone z3 may be transmitted to the overlap confirmation unit 114.

The overlap confirmation unit 114 receives the generated virtual zone (z3), and determines that the overlap zone (z4) is between the generated virtual zone (z3) and the predefined risk zone (z2) in the three-dimensional map (z1). It is determined whether it exists or not. Here, the danger zone (z2) may be predefined by a user, etc., and may have a shape corresponding to the area corresponding to the danger zone (z2) or the external shape of the device. For example, the danger zone z2 may have the shape of a rectangular parallelepiped, as shown in FIG. 3, or may have the shape of an ellipsoid or a sphere. The danger zone (z3) may also be defined by at least one relative coordinate on the 3D map (z1).

In this case, the overlap checker 114 may compare the relative coordinates of the danger zone (z2) and the relative coordinates of the virtual zone (z3) to determine whether these (z2, z3) overlap. The determination result may be transmitted to the warning control unit 116.

The warning control unit 116 determines that the overlap confirmation unit 114 overlaps the danger zone z2 and the virtual zone z3 corresponding to the object 9, so that an overlap zone z4 exists between them z2 and z3. If it is determined that a warning operation of the warning unit 19 is necessary, at least one signal (can be expressed as a notification signal or a guidance signal, etc.) is provided so that the warning unit 19 can perform a corresponding warning operation. ) can be created. At least one generated signal is transmitted to the warning unit 19 through a wired or wireless communication network connected to the communication unit 101, and the warning unit 19 performs a corresponding warning operation in response to reception of the signal. For example, the warning unit 19 may output a specific pattern of light or sound to notify the object 9 or its manager that the object 9 (e.g., a worker) has approached or entered the danger zone z2. there is. Conversely, if the overlap confirmation unit 114 determines that the danger zone z2 and the virtual zone z3 corresponding to the object 9 do not overlap each other and thus the overlap zone z4 does not exist, the warning control unit 116 It is judged that warning action is unnecessary. Accordingly, actions related to warnings are no longer performed.

When the object detection unit 111 extracts the object 9 from the image, the class determination unit 115 may determine the class to which the object 9 belongs based on the extracted object 9. The class may include, for example, information or a category indicating whether a specific object 9 can approach or enter the danger zone z2, but is not limited thereto. More specifically, for example, a class may include general workers and special workers. Here, a general worker may be defined as a worker who is not permitted to access the danger zone (z2), and a special worker may be defined as a worker who is permitted to approach or enter the danger zone (z2). Classes may be defined in various ways depending on the purpose or state of the monitoring space 1, the nature of the work performed, and/or the degree of risk, etc. Determination of which class the object 9 belongs to may be performed based on the appearance (appearance, appearance) of the object 9. For example, the class determination unit 115 is based on the clothing of the object 9 (for example, whether special clothing is worn or a safety helmet is worn) or a mark (epaulette or tag, etc.) mounted on the object 9. It may also be determined whether the object 9 is a general worker or a special worker.

Depending on the embodiment, the class determination unit 115 may perform this determination by executing a predetermined learning algorithm. In this case, for the learning algorithm, information about the appearance of the object 9 (image, etc.) becomes an input value, and whether it is a general worker or a special worker becomes an output value. The determination result of the class determination unit 115 may be transmitted to the warning control unit 116.

According to one embodiment, the warning control unit 116 may further use the determination result of the class determination unit 115 to determine whether a warning operation is necessary. Specifically, for example, if the class determination unit 115 determines that the object 9 is a class (for example, a special worker) that can approach or enter the danger zone (z2), the warning control unit 116 determines that the object 9 is a class that can approach or enter the danger zone (z2). Even when (z2) and the virtual zone (z3) corresponding to the object 9 overlap each other and an overlapping zone (z4) exists, it may be determined that the warning operation of the warning unit 19 is not necessary. If the class determination unit 115 determines that the object 9 is a class (for example, a general worker) that cannot approach or enter the danger zone (z2), the warning control unit 116 determines that the danger zone (z2) and the virtual It is determined that a warning operation of the warning unit 19 is necessary only when an overlapping area (z4) exists between the zones (z3), and at least one signal or data for the warning unit 19 is generated in response. You can. In addition, if no overlap zone (z4) exists between the virtual zone (z3) and the danger zone (z2) corresponding to the object 9, the warning control unit 116 issues a warning regardless of the class of the object 9. The action is judged unnecessary. Accordingly, it is possible to differentially control the warning output depending on whether the object 9 is allowed or permitted to access the danger zone z2.

Regarding the abnormal object detection system 10, an embodiment in which the imaging unit 11, the warning unit 19, and the object detection device 100 are all physically separated has been described, but depending on the embodiment, the object detection The system 10 may be implemented based on a single physical device (for example, the object detection device 100, etc.). That is, a physical device such as the object detection device 100 (e.g., a smartphone, tablet PC, desktop computer, or a device specifically designed to detect the object 9, etc.) includes the above-described photographing unit 11 and the warning unit. (19), a storage unit 102, and a processor 110, etc. may be included, each of which performs the above-described shooting, warning, and determination operation of whether the object 9 approaches or enters the danger zone z2, The same purpose and effect as described above can also be achieved.

Hereinafter, an embodiment of an object detection method will be described with reference to FIG. 5.

Figure 5 is a flowchart of an embodiment of an object detection method.

Referring to FIG. 5 , first, one or more imaging units captured inside or outside the monitoring space capture images of the inside of the monitoring space where a risk zone exists (400). Here, the photographing unit may include a camera device, for example, an RGB-D camera. If the photographing unit is an RGB-D camera, the recording unit may capture an image as well as obtain depth information about the photographed object (for example, the distance between the photographing unit and the object or the distance between a boundary of the monitoring space and the object). The image captured by the photographing unit may be transmitted to the object detection device through a wired or wireless communication network, or may be transmitted to the processor through a cable or circuit line.

The object detection device or processor may detect an object from an image and extract the object (402). In this case, at least one of detecting and extracting the object may be performed using at least one learning algorithm. Accordingly, the area of the object within the image can be determined.

According to one embodiment, the depth of the object may be extracted along with or sequentially with the extraction of the object (404). The depth of the object may be acquired based on information acquired by the imaging unit. For example, it may be obtained by applying triangulation to the positions of a plurality of imaging units and the images acquired by them, or the irradiation point of infrared light and the reflected It may also be obtained by calculating based on the time difference between the reception points of infrared light.

At least one virtual area corresponding to the object may be created on the 3D map based on the image of the object and the acquired depth (406). A virtual area may be created by calculating relative coordinates on a 3D map for an object and combining the calculated relative coordinates. Relative coordinates can be performed using calculated depth. More specifically, as described in Equation 1 and Equation 2 above, the relative coordinates for one point of the object include the depth of at least one point of the object, the coordinates of at least one point in the image, the coordinates of the center of the image, It may be calculated using the width of the image, the height of the image, the calculated depth, the angle of view in the width direction of the photographing unit, the angle of view in the height direction of the photographing unit, and the size adjustment variable. Here, the size control variable may have a value of 1 or more and is a variable provided to determine the size of the virtual area corresponding to the object. On the other hand, when the virtual area is, for example, in the shape of a rectangular parallelepiped, the relative coordinates of each corner of the rectangular parallelepiped are the value obtained by applying the depth determination value to the calculated depth instead of the depth calculated in Equation 1 and Equation 2 above. It can also be calculated by applying . In this case, the x-axis coordinate value and y-axis coordinate value (x, y) of at least one of the four front corners of the rectangular parallelepiped are the first depth obtained by subtracting the depth determination value from the calculated depth in Equation 1 and Equation 2. are respectively calculated by applying to the Each can be calculated by applying Equation 1 and Equation 2. In this case, the coordinates of at least one point in the image used in Equation 1 and Equation 2 are at least one point on the end boundary of the object (for example, four points such as the upper end, lower end, and left and right end of the object). It may include any one of the coordinates of a point).

When a virtual zone is created, it can be determined whether the virtual zone and the risk zone overlap and an overlap zone exists between them (408). Here, the risk area may be defined in advance by a user or designer, may exist in a 3D map corresponding to the monitoring area, and may have relative coordinates within the 3D map. The danger zone may have the shape of a cuboid, ellipsoid, or sphere.

If it is determined that the danger zone and the virtual zone overlap (example of 408), the warning unit may respond and output a warning that the object (for example, a worker) has approached or entered the danger zone (410). Depending on the embodiment, the warning unit may include a sound output device, a warning light, a speaker device, a smartphone, a smart watch, a smart band, a headset, a head-mounted display, a tablet PC, a desktop computer, or a laptop computer. Warnings may be output in visual or audible form. For example, if the warning unit is a sound output device such as a siren, a warning may be output in the form of sound, and if the warning unit is a warning light, the warning may be output in the form of a flashing light. If the warning unit is a smartphone or desktop computer, the warning may be output in the form of a message. Conversely, if it is determined that the danger zone and the virtual zone do not overlap (No in 408), the warning unit does not perform a warning operation.

According to one embodiment, a warning may not be output even if the danger zone overlaps. Specifically, the class of the object may be determined simultaneously or sequentially with the process of determining whether the danger zone overlaps (408), and whether or not to output a warning may be determined according to the determined class. A class is information or a category regarding a specific subject's permission to access or enter a hazardous area, for example, if the subject is a worker, the class is a general worker who is not authorized to access or enter the hazardous area and access to the hazardous area. Alternatively, it may include special workers who are permitted to enter. Class judgment may be performed based on the object's appearance, etc., and a predetermined learning algorithm may be used for this purpose. As a result of the class's judgment, if the object is permitted to approach or enter the danger zone, the warning unit does not perform a warning output operation even if the danger zones overlap. Conversely, if the object is not permitted to approach or enter the danger zone and the object and the danger zone overlap each other, the warning unit performs a warning output operation as described above (410).

The above-described object detection process may be continuously and repeatedly performed (412).

The object detection method according to the above-described embodiment may be implemented in the form of a program that can be driven by a computer device. A program may include instructions, libraries, data files, and/or data structures, etc., singly or in combination, and may be designed and produced using machine code or high-level language code. The program may be specially designed to implement the above-described object detection method, or may be implemented using various functions or definitions known and available to those skilled in the art in the field of computer software. In addition, here, the computer device may be implemented by including a processor or memory that enables the function of the program, and may further include a communication device if necessary.

A program for implementing the above-described object detection method may be recorded on a recording medium readable by a device such as a computer. Recording media that can be read by a computer include, for example, semiconductor storage media such as ROM, RAM, SD cards, or flash memory (eg, solid state drives (SSD), etc.), or magnetic disk storage such as hard disks or floppy disks. At least one medium capable of temporarily or non-temporarily storing one or more programs that are executed upon call from a device such as a computer, such as an optical recording medium such as a compact disk or DVD, or a magneto-optical recording medium such as a floptical disk. It may include any type of physical storage medium.

Although several embodiments of the abnormal object detection system, object detection device, and object detection method have been described, the object detection system, object detection device, or object detection method are not limited to the above-described embodiments. Various other systems, devices or methods that can be implemented by those skilled in the art by modifying and modifying the above-described embodiments may also be an embodiment of the above-described object detection system, object detection device or object detection method. You can. For example, the described method(s) may be performed in an order other than as described, and/or component(s) of the described system, structure, device, circuit, etc. may be combined, connected, or otherwise in a form other than as described. Even if combined or replaced or replaced by other components or equivalents, it can be an embodiment of the above-described object detection system, object detection device, and/or object detection method.

1: Monitoring space 9: Object
11: Filming unit 19: Warning unit
90: terminal device 100: object detection device
101: communication unit 102: storage unit
110: Processor 111: Object detection unit
112: Depth extraction unit 113: Virtual zone creation unit
114: Overlap confirmation unit 115: Class determination unit
116: warning control unit

Claims (7)

a photographing unit that captures images of a monitoring space and an object within the monitoring space, and acquires depth information about the object;
Extract the object from the image of the object, obtain relative coordinates for the object on a three-dimensional map corresponding to the monitoring space, create a virtual zone for the object using the relative coordinates, and an object detection device that determines whether a zone overlaps with a danger zone; and
An object detection system comprising a warning unit that outputs a warning when the virtual area overlaps with the danger area. According to paragraph 1,
The object detection device includes coordinates in the image for at least one point of the object, coordinates of the width direction center of the image, coordinates of the height direction center of the image, the width of the image, and the image. An object detection system that obtains relative coordinates for the object using the height of the photographing unit, the width direction angle of view of the photographing unit, the height direction angle of view of the photographing unit, and a size adjustment variable for adjusting the size of the virtual zone. According to paragraph 2,
The object detection device acquires the depth of at least one point of the object, and selects one of a first depth obtained by subtracting a depth determination value from the depth and a second depth obtained by subtracting the depth determination value from the depth. An object detection system that obtains relative coordinates for the object using one more object. According to paragraph 3,
The virtual area includes a virtual hexahedron in which the relative coordinates obtained using the first depth are the coordinates of the four corners of the front and the relative coordinates obtained using the second depth are the coordinates of the four corners of the back. Object detection system. According to paragraph 1,
The object detection device further determines the class of the object,
The warning unit is an object detection system that does not output the warning when the class of the object can approach or enter the danger zone. a photographing unit that captures images of a monitoring space and an object within the monitoring space, and acquires depth information about the object;
Extract the object from the image of the object, obtain relative coordinates for the object on a three-dimensional map corresponding to the monitoring space, create a virtual zone for the object using the relative coordinates, and a processor that determines whether zones overlap with hazardous zones; and
Object detection device comprising: a warning unit that outputs a warning when the virtual area and the danger area overlap. Taking images of a monitoring space and an object within the monitoring space;
Obtaining depth information about the object;
extracting the object from an image of the object;
Obtaining relative coordinates for the object on a three-dimensional map corresponding to the monitoring space;
generating a virtual zone for the object using the relative coordinates;
determining whether the virtual zone overlaps with the danger zone; and
An object detection method comprising: outputting a warning when the virtual zone and the danger zone overlap each other.