KR20230108239A - device for management classifying objects and acquiring multi object big data - Google Patents

Abstract

According to one embodiment, a transceiver for acquiring a plurality of images from a video monitoring device for monitoring crime prevention, disaster, safety or traffic conditions; an upscaling image acquisition unit acquiring the plurality of images by performing interpolation based on colors of pixels to be upscaled to filter out noise included in the initial image; an object analyzer configured to determine at least one of a speed of an object included in the plurality of images, a type of the object, a characteristic of the object, a situation of the object, and a number of the object included in the plurality of images; And a control unit for controlling at least one of a shutter, gain, and lighting based on at least one of whether the object is included in the plurality of images, the size of the object, and the size of an area where the object is detected per frame. An apparatus for object classification and multi-object big data acquisition management is provided.

Description

Device for management classifying objects and acquiring multi object big data}

The present disclosure relates to technologies related to image quality enhancement of an image obtained from a video monitoring device, object classification accordingly, and a multi-object big data acquisition management device.

A technology for performing object classification and object recognition based on an image acquired from a conventional video surveillance device such as CCTV has been provided, and accordingly, the technology is being utilized in various fields.

However, according to the current situation of most domestic CCTVs, despite acquiring high-quality images, there is a problem in that low-quality images that have lowered the quality of high-quality images can only be linked to the center due to the bandwidth limitation of the communication network.

These problems can be partially resolved by upgrading or replacing the communication network to improve performance, but in areas where network lines and equipment need to be replaced by spending a lot of money, high-definition images obtained from video surveillance devices can be obtained without deterioration in image quality. An effective way to do this is lacking.

Korean Patent Publication No. 10-2021-0038280 (2021.04.07)

An embodiment of the present disclosure is to solve the above-mentioned problems of the prior art, to provide a method for upgrading a low-quality image to a high-definition image and an algorithm-based object classification and multi-object big data acquisition management device for transmitting high-definition image in real time. do.

The purpose of the present disclosure is not limited to the purpose mentioned above, and other objects not mentioned will be clearly understood from the description below.

According to a first aspect of the present disclosure, a transceiver for acquiring a plurality of images from a video monitoring device for monitoring crime prevention, disaster, safety or traffic conditions; an upscaling image acquisition unit acquiring the plurality of images by performing interpolation based on colors of pixels to be upscaled to filter out noise included in the initial image; an object analyzer configured to determine at least one of a speed of an object included in the plurality of images, a type of the object, a characteristic of the object, a situation of the object, and a number of the object included in the plurality of images; and a control unit controlling at least one of a shutter, a gain, and illumination based on at least one of whether the object is included in the plurality of images, a size of the object, and a size of a region in which the object is detected per frame; Including, it is possible to provide an object classification and multi-object big data acquisition management device.

In addition, in the process of obtaining the plurality of images from the video monitoring device, the transmitting and receiving unit obtains an image including noise generated in the process of compression transmission by a camera included in the video monitoring device through RTSP communication, and then FTP. Alternatively, the plurality of images may be transmitted in a socket communication method.

In addition, the upscaling image acquiring unit performs first filtering according to a non-local means method and second filtering performed based on a temporal window size on the plurality of images, A filtered image may be obtained, and upscaling of the filtered image may be performed using a Conv function.

In addition, the object analysis unit based on a comparison result between the first image and the second image or the third image obtained first among the first image, the second image, and the third image sequentially obtained over time Change coordinates of the object, size of the object, and text indicating the object may be determined.

In addition, the object analysis unit scale-up algorithm according to ESRGAN, background difference (absdiff) algorithm, dilate algorithm, changed object discrimination (findContours) algorithm, detected object bounding and coordinate value extraction (boundingRect) algorithm, and RGB extraction algorithm Object analysis may be performed using at least one of the above.

In addition, the control unit controls the shutter, gain, and illumination by using at least one of the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object determined based on the plurality of images. At least one of the operation timings may be controlled.

Also, the controller may be included in the video monitoring device through at least one of HTTP communication and ONVIF.

In addition, the control unit may control the MCU of the lighting using at least one of serial communication or TCP communication in order to adjust the brightness.

In addition, the control unit may control an operation sync between the shutter and the light when the video monitoring device acquires the plurality of images by controlling the MCU of the lighting and the video monitoring device.

In addition, the controller converts the plurality of images including the object into the object by performing deep learning on the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object. It can be named as a character representing .

In addition, the control unit determines whether or not the object is included in the plurality of images, the size of the object, and the size of the area where the object is detected per frame based on the speed of the object, and the pulse value of the video monitoring device and lighting. can decide

In addition, when the camera included in the video monitoring device operates in dual streaming, the camera transmits the plurality of images according to single streaming, which is one of the dual streaming, to the object analysis unit, and according to the other one, sub-streaming. It can be transmitted to a center that stores the plurality of images.

In addition, the object analyzer compares the plurality of images according to the single streaming with preset object information using a pre-stored algorithm to determine whether the object is included in the plurality of images, and if the object is included , At least one of the speed of the object, the type of the object, the characteristic of the object, the situation of the object, and the number of the object may be determined based on the image including the object.

In addition, the control unit transmits whether or not the object is included in the plurality of images, the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object through the transceiver through FTP or socket communication. may be transmitted to the center in this manner.

Also, when the object is a vehicle, the controller may update at least one of the shutter, the gain, and the lighting to a preset value.

In addition, a frame-by-frame image receiving unit for obtaining a plurality of images from the video monitoring device for monitoring crime prevention, disaster, safety or traffic conditions; A coordinate value extractor for each frame that performs interpolation based on the color of a pixel to be upscaled to filter noise included in the initial image, extracts RGB values for each frame as a character string, and transforms the plurality of images into a character string. ; an object analyzer configured to determine at least one of a speed of an object included in the plurality of images, a type of the object, a characteristic of the object, a situation of the object, and a number of the object included in the plurality of images; A comparison analysis unit that classifies fixed objects and moving objects through the recognition values of the object analysis unit and sets an average value of about 50-100 pix based on the center value to determine I-frames and P-frames through three images; a controller for controlling at least one of a shutter, a gain, and illumination based on at least one of whether the object is included in the plurality of images, a size of the object, and a size of a region in which the object is detected per frame; and a selection transmission unit that transmits only images selected by the comparison and analysis unit.

In addition to this, other specific details are included in the detailed description and drawings.

According to an embodiment of the present disclosure, it is possible to effectively solve the conventional problems by providing a method for upgrading a low-quality image to a high-definition image and an algorithm-based object classification and multi-object big data acquisition management apparatus for transmitting high-quality image in real time.

The effects of the present disclosure are not limited to the above effects, and should be understood to include all effects that can be inferred from the configuration of the invention described in the detailed description or claims of the present disclosure.

1 is a diagram schematically illustrating the configuration of an apparatus for object classification and multi-object big data acquisition management according to an embodiment.
2 is a diagram schematically illustrating an embodiment in which an apparatus for object classification and multi-object big data acquisition management operates according to an embodiment.
3 is a diagram schematically illustrating another embodiment in which an apparatus for object classification and multi-object big data acquisition management according to an embodiment operates.
4 is a diagram schematically illustrating another embodiment in which an apparatus for object classification and multi-object big data acquisition management operates according to an embodiment.
5 is a diagram illustrating an example of an object recognition operation for a high-definition image acquired by an apparatus for object classification and multi-object big data acquisition and management according to an embodiment.
6 is a flowchart illustrating an operation of an apparatus for object classification and multi-object big data acquisition management according to an embodiment.
7 is a diagram illustrating an example in which an apparatus for object classification and multi-object big data acquisition management operates in relation to encoding according to an embodiment.

The terms used in the embodiments have been selected as general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, terms used in the present disclosure should be defined based on the meaning of the term and the general content of the present disclosure, not simply the name of the term.

In the entire specification, when a part is said to "include" a certain component, it means that it may further include other components, not excluding other components unless otherwise stated. In addition, as described in the specification, "... wealth", "… A term such as “module” refers to a unit that processes at least one function or operation, and may be implemented as hardware or software or a combination of hardware and software.

Throughout the specification, 'support' may be broadly interpreted as meaning including the performance of a related operation to achieve a specific purpose, and is not construed as limiting. For example, if server C supports server A so that server A performs operation B, server C's support for server A comprehensively performs related operations required in the process of server A performing operation B. can be interpreted as including For example, server C provides relevant information necessary for server A to perform operation B, a preliminary operation expected to be necessary in the process of server A performing operation B, and a message requested by server A to perform operation B. It is possible to perform reception and the like for .

In addition, throughout the specification, 'providing' includes a process in which an object acquires specific information or directly or indirectly transmits/receives it to a specific object, and can be interpreted as comprehensively including the performance of related operations required in this process.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily carry out the present disclosure. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

1 is a diagram schematically illustrating the configuration of an apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment.

Referring to FIG. 1 , an apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment includes a video monitoring apparatus 110, an upscaling image acquisition unit 120, a transceiver 130, and an object analysis unit. 140 and a control unit 150.

The transceiver 130 according to an embodiment may obtain a plurality of images from the video monitoring device 110 for monitoring crime prevention, disaster, safety, or traffic conditions.

In addition, in the process of obtaining a plurality of images from the video monitoring device 110, the transceiver 130 acquires an image including noise generated in the process of compression and transmission by the video monitoring device 110 through RTSP communication. Afterwards, multiple images can be transmitted via FTP or socket communication.

For example, when video is transmitted according to RTSP, a lot of dicing occurs, but if a method of transmitting changed coordinate values and changed RGB values is used, data loss due to RTSP can be overcome to a large extent.

The upscaling image obtaining unit 120 according to an embodiment may acquire a plurality of images by performing interpolation based on the color of a pixel to be upscaled in order to filter out noise included in the initial image.

In addition, the upscaling image obtaining unit 120 performs first filtering based on a non-local means method and second filtering performed based on a temporal window size on a plurality of images. It is possible to obtain a filtered image and perform upscaling on the filtered image using the Conv function.

The object analyzer 140 according to an embodiment may determine at least one of object speed, object type, object characteristic, object situation, and object number included in the plurality of images by using a plurality of images. .

In addition, the object analyzer 140 based on a comparison result between the first image, the second image, or the third image obtained first among the first image, the second image, and the third image sequentially obtained over time. By doing so, the change coordinates of the object, the size of the object, and the text referring to the object can be determined.

In addition, the object analyzer 140 is a scale-up algorithm according to ESRGAN, a background difference (absdiff) algorithm, a dilate algorithm, a changed object discrimination (findContours) algorithm, a detected object bounding and coordinate value extraction (boundingRect) algorithm, and Object analysis may be performed using at least one of RGB extraction algorithms.

The controller 150 according to an embodiment may control at least one of shutter, gain, and lighting based on at least one of whether objects are included in a plurality of images, the size of the object, and the size of the area where the object is detected per frame. can

In addition, the controller 150 may use at least one of the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object determined based on the plurality of images, and at least one of the shutter, gain, and lighting operation times. You can control one.

In addition, the control unit 150 may be included in the video monitoring device 110 through at least one of HTTP communication and ONVIF.

In addition, the controller 150 may control the MCU of the lighting using at least one of serial communication and TCP communication to adjust the brightness.

In addition, the control unit 150 may control operation synchronization between shutters and lights when the video monitoring device 110 acquires a plurality of images by controlling the lighting MCU and the video monitoring device 110 .

In addition, the controller 150 names a plurality of images including the object as characters representing the object by performing deep learning on the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object. can do.

In addition, the control unit 150 determines whether or not an object is included in a plurality of images, the size of the object, and the size of the area where the object is detected per frame determine the pulse value of the video monitoring device 110 and lighting based on the speed of the object. can

The video monitoring device 110 according to an embodiment may include a first video monitoring device 210 and a second video monitoring device 410 .

A camera according to an embodiment may be included in the video monitoring device 110 .

In addition, those skilled in the art can understand that other general-purpose components other than the components shown in FIG. 1 may be further included in the apparatus 100 for object classification and multi-object big data acquisition management. . For example, the object classification and multi-object big data acquisition management apparatus 100 may further include a memory (not shown) for storing a plurality of images. Alternatively, in the case of other embodiments, those skilled in the art may understand that some of the components shown in FIG. 1 may be omitted.

The apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment may be used by a user or a worker, and may include a mobile phone, a smart phone, a personal digital assistant (PDA), a portable multimedia player (PMP), and a tablet PC. It can be interlocked with all types of handheld-based wireless communication devices equipped with touch screen panels, such as desktop PCs, tablet PCs, laptop PCs, and IPTVs including set-top boxes. It can be included in or interlocked with a device that has a base that can be executed.

The object classification and multi-object big data acquisition management device 100 may be implemented as a terminal such as a computer that operates through a computer program for realizing the functions described in this specification.

The apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment may include a center 220 and a server (not shown), but is not limited thereto. The center 220 and the server according to an embodiment may support an application providing an image of improved quality and a service using the same. In addition, the center 220 and the server may be integrated into one configuration or may operate in two or more separate forms.

Hereinafter, an embodiment in which the object classification and multi-object big data acquisition management device 100 according to an embodiment operates independently will be described, but as described above, it operates through linkage with the center 220 and the server. can also be performed. That is, the object classification and multi-object big data acquisition management apparatus 100, center 220, and server according to an embodiment may be integrated and implemented in terms of their functions, and the center 220 and server may be omitted. , It can be seen that it is not limited to any one embodiment.

2 is a diagram schematically illustrating an embodiment in which the apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment operates.

Referring to the drawing, the object classification and multi-object big data acquisition management device 100 may obtain a plurality of images from the first video monitoring device 210 .

The plurality of images may be obtained by performing interpolation based on the color of a pixel to be upscaled in order to filter out noise included in the initial image. The object classification and multi-object big data acquisition management apparatus 100 transmits the acquired plurality of initial images and the plurality of images to the center 220 so that the center 220 stores and manages each image. can

Specifically, the object classification and multi-object big data acquisition management apparatus 100 is based on the first filtering and temporal window size method according to the non-local means method for a plurality of images The filtered image may be obtained by performing the second filtering performed by the above, and upscaling of the filtered image may be performed using a Conv function.

Since the non-local means method and the temporal window size method described above and mentioned in the present disclosure are conventionally disclosed and provided technologies in the field of image correction, detailed descriptions in the present disclosure will be omitted.

The object classification and multi-object big data acquisition management apparatus 100 determines at least one of the speed of an object included in a plurality of images, the type of the object, the characteristic of the object, the situation of the object, and the number of the object by using a plurality of images. can

Specifically, the apparatus 100 for object classification and multi-object big data acquisition management may first acquire a first image and a second image among a first image, a second image, and a third image sequentially obtained over time, or Based on the comparison result between the third images, the change coordinates of the object, the size of the object, and text indicating the object may be determined.

In addition, the apparatus 100 for object classification and multi-object big data acquisition management includes a scale-up algorithm according to ESRGAN, a background difference (absdiff) algorithm, a dilate algorithm, a changed object division (findContours) algorithm, and detected object bounding and coordinates. Object analysis may be performed using at least one of a value extraction (boundingRect) algorithm and an RGB extraction algorithm. Technologies used in the above-described algorithm are also disclosed and provided prior art, and detailed descriptions in the present disclosure will be omitted.

Thereafter, the apparatus 100 for object classification and multi-object big data acquisition management monitors images based on at least one of whether or not the object is included in a plurality of images, the size of the object, and the size of the area where the object is detected per frame. By controlling at least one of the shutter, gain, and lighting of the device 110, and performing deep learning on the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object for the image obtained accordingly. Accordingly, a plurality of images including objects may be named as characters representing the objects and transmitted to the center 220 .

The center 220 transmits information and a plurality of images indicating whether an object is included or not, the size of an object, and the size of an area in which an object is detected per frame, obtained from the apparatus 100 for object classification and multi-object big data acquisition management, are transmitted to the user terminal 230. ) can be sent.

By doing this, it is possible to effectively solve the conventional problem of replacing a high-quality video with a low-quality video having a lower quality due to a bandwidth limitation of a communication network.

3 is a diagram schematically illustrating another embodiment in which the apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment operates.

Referring to the drawing, the center 220 may receive a plurality of initial images obtained from the first video monitoring device 210, store and manage the received plurality of initial images, or separate user input or object classification and When an image provision request for the center 220 is transmitted from the multi-object big data acquisition management device 100, the center 220 may classify the object and transmit the multi-object big data acquisition management device 100.

For example, when a plurality of initial images obtained from the first video monitoring device 210 according to the order and description shown in FIG. 2 are directly transmitted to the object classification and multi-object big data acquisition management device 100, the object Classification and multi-object big data acquisition management device 100 may be overloaded and performance degradation may occur accordingly.

In this way, when the amount of analysis for a plurality of images provided to the object classification and multi-object big data acquisition management device 100 is greater than a predetermined value, in an embodiment, in order to solve this problem, the first video monitoring device 210 A plurality of acquired images are preferentially stored in the center 220, and when the amount of analysis decreases below a preset value, the center 220 analyzes the plurality of images with the object classification and multi-object big data acquisition management device 100. The object classification and multi-object big data acquisition management device 100 transmits the analysis result to the center 220 so that it can be provided to the user terminal 230 or object classification and multi-object It can be transmitted through the transmission/reception unit 130 included in the big data acquisition management device 100 .

2 and 3, the object classification and multi-object big data acquisition management device 100, the first video monitoring device 210, the center 220, and the user terminal 230 are separated from each other, but , This is for explaining an embodiment, and is not limited to that shown in the drawing, and according to another embodiment, the object classification and multi-object big data acquisition management device 100, the first video monitoring device 210, and the center ( 220) and the user terminal 230 may be implemented in an integrated state with one of ordinary skill in the art.

4 is a diagram schematically illustrating another embodiment in which the apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment operates.

Referring to the drawing, the object classification and multi-object big data acquisition management device 100 may acquire a plurality of images from the second video monitoring device 410 .

In one embodiment, the plurality of images obtained from the second video monitoring device 410 may be high-quality images having higher quality and resolution than the plurality of images obtained from the first video monitoring device 210 .

When acquiring a plurality of images from the high-performance second video monitoring device 410, the apparatus 100 for object classification and multi-object big data acquisition management omits some of a series of steps required for image analysis or performs each step separately. By performing all of them, it is possible to obtain an analysis result of more improved image quality.

In one embodiment, the object classification and multi-object big data acquisition management apparatus 100 detects the brightness of a plurality of images, whether an object is included in the plurality of images, the size of the object, and the object per frame based on the plurality of images. It is possible to determine the size of the area to be used, and based on this, at least one of a shutter, a gain, and illumination may be controlled.

Specifically, shutter, gain, and illumination control may be performed to obtain an image in which an object is captured more clearly. For example, an instantaneous scene of a moving object may be captured according to a shutter speed value according to shutter control, and a brighter image may be acquired according to a gain value according to gain control when the shooting environment is in a dark low-light state, A clear image of an object can be obtained even in a bright or dark shooting environment according to the lighting value according to the lighting control.

Also, it is preferable to control the shutter speed value according to the shutter control to be proportional to the brightness of the photographing environment, and to determine the gain value according to the gain control to be inversely proportional to the brightness of the photographing environment.

In some embodiments according to this, when the brightness value of the shooting environment is darker than the preset value or during the preset time period (according to the season), the brightness value of the shooting environment is expected to decrease to the preset value or less (e.g., 7:00 pm to 6:00 am), the object classification and multi-object big data acquisition management device 100 determines the brightness of a plurality of images, whether an object is included in the plurality of images, and the size of an area where an object is detected per frame. And it is possible to assign a high weight in the order of the size of the object, and control the shutter, gain, and lighting based on the assigned weight.

For example, in an image acquired in a dark environment, it may be difficult to easily determine an object even if the object is included in the image, and even if the object is detected on the plurality of images obtained from the object analyzer 140, the plurality of images When the brightness of is less than a preset value, it may be difficult to obtain a clear and accurate object analysis image.

Accordingly, the apparatus 100 for object classification and multi-object big data acquisition management prioritizes brightness of a plurality of acquired images and controls shutter, gain, and lighting, so that it is easy to determine whether an object is included in the image, object size, etc. The highest weight can be given to the brightness of a plurality of images in that it can be detected in a sequentially large way with respect to whether objects are included in the plurality of images, the size of the area where the object is detected per frame, and the size of the object. By controlling at least one of shutter, gain, and lighting by determining each element based on a weighted value, it is possible to effectively analyze an object on an image even when the brightness of a plurality of images is low.

In another embodiment, when performing image analysis on a moving object, the object classification and multi-object big data acquisition management apparatus 100 determines whether an object is included in a plurality of images, the size of an area in which an object is detected per frame, High weights may be assigned in order of the object size and brightness of a plurality of images, and shutter, gain, and lighting may be controlled based on the assigned weights.

For example, a case of performing image analysis on a moving object may include a case of performing analysis on the number, vehicle model, moving direction, speed, and the like of a moving vehicle.

In this way, when performing image analysis on a moving object, first of all, whether to perform the subsequent image analysis step is determined according to the result of determining whether the object is included among a plurality of images, so object classification and multi-object big data The acquisition management device 100 may assign the highest weight to whether an object is included in a plurality of images.

In addition, the size of an area where an object is detected per frame can be determined according to whether an object is included in a plurality of images, and more detailed analysis such as vehicle type, speed, license plate location analysis, etc. can be determined according to the size of the detection area per frame. In that image analysis is possible, a second high weight may be assigned to the size of an area where an object is detected per frame.

In addition, depending on the case where the size of the object determined based on the plurality of images is large or small, it is possible to determine what type of moving object corresponds to, such as a vehicle, pedestrian, or animal. In that the type can be determined, the size of the object can be given a high weight in the third place.

In addition, it may be easy to detect a moving object on an image according to the brightness of a plurality of images determined based on the plurality of images, but the environment for acquiring an image of a moving object is various environments such as street lights and other lighting adjacent to the image acquisition device. In terms of a high possibility that the level is appropriate for detecting a moving object on an image due to an enemy element, the 4th highest weight may be assigned to the brightness of the plurality of images.

The apparatus 100 for object classification and multi-object big data acquisition management effectively analyzes, detects, and discriminates moving objects among a plurality of images by controlling at least one of shutter, gain, and lighting based on the weights given in different sizes. be able to perform

In another embodiment, the apparatus 100 for object classification and multi-object big data acquisition management includes an average brightness of a plurality of images, a size of an area in which the brightness of a plurality of images is equal to or greater than a preset value, the brightness of a shooting target region, and a shooting target. A history of the brightness of the region may be acquired, and a gain value applied to gain control may be determined based on the obtained information.

In one embodiment, as the gain value increases, the overall brightness of the image may increase, but unnecessary noise also increases in proportion to the gain value. Accordingly, control of the gain value is generally required in a specific situation, such as when the brightness value of an image cannot be increased any more or when a certain amount of noise is generated, which is not particularly a problem.

For example, the apparatus 100 for object classification and multi-object big data acquisition management determines whether an object is included in a plurality of images, the average brightness of the plurality of images, the brightness of the target area to be captured, and the order of the brightness history of the target area to be captured. It is possible to determine a gain value used for gain control based on a weight given high to .

For example, if an object is included in a plurality of images depending on whether the object is included in the plurality of images, whether or not the object is included can be determined without controlling the gain value, so that the object is included in the plurality of images. The highest weight may be given to whether or not it is included.

In addition, by controlling the gain value according to the average brightness of a plurality of images, it is possible to more accurately determine and analyze objects included in the image, and to substantially improve the image quality and to indicate the brightness of the target image itself for object analysis. At this point, a second-highest weight may be assigned to the average brightness of a plurality of images.

In addition, the brightness of a plurality of images can be controlled by controlling the gain value according to the brightness of the region to be captured, and the brightness of the plurality of images can be determined according to the difference between the average brightness of the plurality of images and the brightness value of the region to be captured. Reliability or the reliability of the sensing information of the video monitoring device 110 that senses the brightness of the region to be captured is third in the brightness of the region to be captured in that it acts as an additional factor in gain control. Weights may be assigned.

In addition, according to the brightness history of the shooting target area, it is possible to predict the brightness of the shooting target area in a specific time period, a specific climate, or a specific situation that can affect the brightness of the shooting target area or a situation where the brightness of the shooting target area cannot be sensed. However, in that it is not real-time information on the region to be captured, a high 4th weight may be assigned to the brightness history of the region to be captured.

The apparatus 100 for object classification and multi-object big data acquisition management can more efficiently recognize and analyze objects on an image by determining a gain value for gain control based on the weights differently assigned as described above, and can use the same for high definition Images can also be acquired.

In one embodiment, the object classification and multi-object big data acquisition management apparatus 100 determines whether or not an object is included, which is determined according to a plurality of images, and the speed of the object based on the size of the object and the size of the area where the object is detected per frame. , the type of object (e.g. vehicle, person, animal, other object, etc.), the characteristics of the object (e.g. the photographed object is part of the whole, etc.), the situation of the object (e.g. stationary state, moving state, accident situation) etc.) and an object number (eg, vehicle number, identification number, etc.) may be acquired, and at least one of the shutter, gain, and lighting operation timing may be determined using at least one of them.

For example, operation points of shutter, gain, and lighting can be used to more accurately and easily identify objects included in a plurality of images obtained time-sequentially. Image characteristics representing characteristics such as brightness, contrast, sharpness, etc. of a plurality of images may be determined based on the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object corresponding to each image, and the determined image Depending on the characteristics, operating timings of the shutter, gain, and lighting may be determined. Thereafter, the control of the shutter, gain, and lighting is performed based on the determined operating timing of the shutter, gain, and lighting, so that a plurality of images capable of more easily and accurately identifying objects can be obtained.

In addition, the object classification and multi-object big data acquisition management device 100 compares the similarity between a specific area (eg, license plate, emblem, etc.) of the same type of object (eg, a specific vehicle) existing in the previous frame and the current frame. It is possible to detect a specific area for an object to be analyzed and tracked.

5 is a diagram illustrating an example of an object recognition operation for a high-definition image acquired by the apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment.

Referring to the drawing, the apparatus 100 for object classification and multi-object big data acquisition management determines whether an object 510 is included in a high-definition image obtained by the above-described methods, the size of the object 510, and the size of the object 510. ) may indicate the size of a region detected per frame, and based on this, the object 510 in the image may be recognized and analyzed.

In this way, by recognizing and analyzing the moving object 510 in the image based on whether the object 510 is included in the plurality of images, size, area, etc., the conventional problems are effectively solved, and at the same time, unnecessary calculations are minimized while high-quality analysis result images are obtained. be able to obtain

6 is a flowchart illustrating the operation of the apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment.

Referring to the drawing, in step S610, the object classification and multi-object big data acquisition management device 100 may obtain a plurality of images from the video monitoring device 110 for monitoring crime prevention, disaster, safety, or traffic conditions.

In one example, the video monitoring device 110 may be implemented in the form of a CCTV in an area adjacent to the area to be photographed so as to easily obtain a plurality of images for crime prevention, disaster, safety, or traffic conditions, object classification, It can be used to perform object classification operations in the overall image, such as object analysis and discrimination.

In addition, in the process of acquiring a plurality of images from the video monitoring device 110 by the object classification and multi-object big data acquisition management device 100, the camera included in the video monitoring device 110 compresses and transmits through RTSP communication. After obtaining an image containing noise generated in the process, a plurality of images can be transmitted through FTP or socket communication.

In step S620, the object classification and multi-object big data acquisition management apparatus 100 acquires a plurality of images by performing interpolation based on the color of a pixel to be upscaled in order to filter noise included in the initial image. can

For example, the apparatus 100 for object classification and multi-object big data acquisition management includes first filtering and temporal window size method according to a non-local means method for a plurality of images. A filtered image may be obtained by performing second filtering based on , and upscaling of the filtered image may be performed using a Conv function.

In step S630, the apparatus 100 for object classification and multi-object big data acquisition management uses a plurality of images to determine the speed of the object 510 included in the plurality of images, the type of the object 510, and the characteristics of the object 510. , at least one of the situation of the object 510 and the number of the object 510 may be determined.

For example, the object classification and multi-object big data acquisition management apparatus 100 determines whether an object 510 is included in a plurality of obtained images, and sets a predetermined analysis algorithm for the included object 510. (e.g. scale-up algorithm according to ESRGAN, background difference (absdiff) algorithm, dilate algorithm, changed object distinction (findContours) algorithm, detected object bounding and coordinate value extraction (boundingRect) algorithm, RGB extraction algorithm, etc.) Object analysis may be performed using at least one.

In addition, the object classification and multi-object big data acquisition management apparatus 100 uses the object analysis result according to the object 510 speed, the type of the object 510, the characteristics of the object 510, and the quality of the object 510. It is possible to determine at least one of the situation and the number of the object 510, and the determined object classification and multi-object big data acquisition management apparatus 100 determines the speed of the object 510, the type of the object 510, and the object 510. At least one of the shutter, gain, and lighting operation timing may be controlled using at least one of the characteristics of the object 510, the situation of the object 510, and the number of the object 510. Details on this will be continuously described in step S640 below.

In addition, the apparatus 100 for object classification and multi-object big data acquisition management includes a first image obtained first and a second image among first images, second images, and third images sequentially obtained over time. Alternatively, change coordinates of the object 510 , a size of the object 510 , and text indicating the object 510 may be determined based on a comparison result between the third images.

In step S640, the object classification and multi-object big data acquisition management apparatus 100 determines whether or not the object 510 is included in a plurality of images, the size of the object 510, and the size of the area where the object 510 is detected per frame. At least one of the shutter, gain, and lighting may be controlled based on at least one of the above.

Specifically, the object classification and multi-object big data acquisition management device 100 may acquire a plurality of images from the video monitoring device 110 using at least one of HTTP communication and ONVIF, and serial communication to adjust brightness. Alternatively, the MCU of the lighting of the video monitoring device 110 may be controlled using at least one of TCP communication.

In addition, the object classification and multi-object big data acquisition management device 100 controls the lighting MCU and the video monitoring device 110 to synchronize operation between shutters and lights when the video monitoring device 110 acquires a plurality of images. can be controlled.

In addition, the object classification and multi-object big data acquisition management apparatus 100 includes the speed of the object 510 acquired in step S630, the type of the object 510, the characteristics of the object 510, the situation of the object 510, and As deep learning is performed on the number of the object 510 , a plurality of images including the object 510 may be named as characters representing the object 510 .

In addition, the object classification and multi-object big data acquisition management apparatus 100 determines whether or not the object 510 is included in a plurality of images, the size of the object 510, and the size of the area where the object 510 is detected per frame. , It can be determined by determining the pulse value of the video monitoring device 110 and lighting based on the speed of the object 510 .

Accordingly, the object classification and multi-object big data acquisition management apparatus 100 solves the chronic problem of the prior art, upgrades a low-quality image obtained in real time to a high-quality image, and simultaneously transmits a high-quality image in real time.

In another embodiment, when the camera included in the video surveillance device 110 operates in dual streaming, the camera transmits a plurality of images according to single streaming, which is one of dual streaming, to the object analyzer 140, and the remaining images are transmitted to the object analyzer 140. A plurality of images according to one sub-streaming may be transmitted to the center 220 that stores them.

For example, dual streaming may include a function of acquiring a plurality of images by dividing a streaming channel into single streaming and sub-streaming, and the plurality of images acquired when the camera operates in dual streaming may include the same information. can include

In addition, the object analyzer 140 compares a plurality of images according to single streaming with preset object information using a pre-stored algorithm (eg, a YOLO-based algorithm) to determine whether the object 510 is included in the plurality of images. can decide

For example, the object analyzer 140 uses a pre-stored algorithm to determine whether a plurality of images and an object 510 are defined in advance and used for comparison to determine whether the object is included (e.g., a function according to the YOLO algorithm). A comparison result with preset object information including a labeling class obtained based on ) may be obtained, and based on the comparison result, whether the object 510 is included in the plurality of images may be determined.

When the object 510 is included in a plurality of images, the object analyzer 140 performs at least one of the speed of the object, the type of the object, the characteristic of the object, the situation of the object, and the number of the object based on the image including the object. can decide

The control unit 150 determines whether or not the object 510 is included in the plurality of images, the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object to the center 220 through FTP or socket communication. can be sent

In one embodiment, when the type of object 510 determined by the object analyzer 140 is a vehicle, the controller 150 may update at least one of a shutter, a gain, and lighting to a preset value.

For example, the object classification and multi-object big data acquisition management device 100 detects an object 510 using a pre-stored algorithm for each frame of a plurality of images obtained from the video monitoring device 110, and When 510 is detected, the values of the shutter, gain, and lighting may be updated through the controller 150 to be higher than the existing values.

According to embodiments of the present disclosure, by providing a method for upgrading a low-quality image to a high-definition image and an algorithm-based object classification and multi-object big data acquisition management apparatus 100 for transmitting high-quality image in real time, high-definition image due to bandwidth limitations It is possible to effectively solve the conventional problem of transmitting a low-quality video.

In addition, the apparatus 100 for object classification and multi-object big data acquisition management may use technology methods such as SPAT (Spatial Partition analysis tracking) and STF (SignTelecom Features) algorithms in the process of image processing.

7 is a diagram illustrating an example in which the apparatus 100 for object classification and multi-object big data acquisition management according to an embodiment operates in relation to encoding.

An object classification and multi-object big data acquisition management device 100 according to an embodiment includes a frame-by-frame image receiving unit 710 that acquires a plurality of images from a video monitoring device that monitors crime prevention, disaster, safety, or traffic conditions, and an initial image. A frame-by-frame coordinate value extraction unit 720 that performs interpolation based on the color of a pixel to be upscaled to filter out noise included in, extracts RGB values for each frame as a string, and transforms a plurality of images into a string. , an object analysis unit 730 that determines at least one of the speed of an object included in a plurality of images, the type of an object, the characteristics of an object, the situation of an object, and the number of an object included in a plurality of images, and a recognition value of the object analysis unit. Classify stationary objects (e.g. noise, trees, stones, houses, etc.) and moving objects (e.g. people, vehicles, etc.) through I At least one of shutter, gain, and illumination based on at least one of a comparative analysis unit 740 that determines a frame and a P frame, whether or not an object is included in a plurality of images, the size of the object, and the size of the area where the object is detected per frame. a controller 750 controlling one; and a selection transmission unit 760 that transmits only images selected by the comparison and analysis unit.

The order and combination of the steps shown above is an embodiment, and the order, combination, branch, function, and subject of execution thereof may be added, omitted, or modified within the scope that does not deviate from the essential characteristics of each component described in the specification. It can be seen that this can be done.

Various embodiments of the present disclosure are implemented as software including one or more instructions stored in a storage medium (eg, memory) readable by a machine (eg, a display device or a computer). It can be. For example, the processor of the device may call at least one of one or more instructions stored from a storage medium and execute it. This enables the device to be operated to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term refers to the case where data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in the present disclosure may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store™) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smartphones. In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium such as a manufacturer's server, an application store server, or a relay server's memory.

The description of the present disclosure described above is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present disclosure. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present disclosure is indicated by the following claims, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present disclosure.

100: object classification and multi-object big data acquisition management device
110: video monitoring device
120: upscaling image acquisition unit
130: transceiver
140: object analysis unit
150: control unit
210: first video monitoring device
220: center
230: user terminal
410: second video monitoring device
510: object
710: frame-by-frame video receiver
720: coordinate value extraction unit for each frame
730: object analysis unit
740: comparison analysis unit
750: control unit
760: selective transmitter

Claims (16)

Transmitting and receiving unit for obtaining a plurality of images from the video monitoring device for monitoring crime prevention, disaster, safety or traffic conditions;
an upscaling image acquisition unit acquiring the plurality of images by performing interpolation based on colors of pixels to be upscaled to filter out noise included in the initial image;
an object analyzer configured to determine at least one of a speed of an object included in the plurality of images, a type of the object, a characteristic of the object, a situation of the object, and a number of the object included in the plurality of images; and
a controller for controlling at least one of a shutter, a gain, and illumination based on at least one of whether the object is included in the plurality of images, a size of the object, and a size of a region in which the object is detected per frame;
Containing, object classification and multi-object big data acquisition management device.
According to claim 1,
In the process of acquiring the plurality of images from the video monitoring device, the transmitting/receiving unit obtains an image including noise generated in the process of compressing and transmitting a camera included in the video monitoring device through RTSP communication, and then FTP or socket. An object classification and multi-object big data acquisition management device for transmitting the plurality of images in a communication manner.
According to claim 1,
The upscaling image acquiring unit
Obtaining a filtered image by performing first filtering according to a non-local means method and second filtering performed based on a temporal window size on the plurality of images,
Object classification and multi-object big data acquisition management apparatus for performing upscaling on the filtered image using a Conv function.
According to claim 1,
The object analysis unit
Change coordinates of the object based on a comparison result between the first image obtained first among the first image, the second image, and the third image sequentially acquired over time and the second image or the third image, Object classification and multi-object big data acquisition management apparatus for determining the size of the object and the text referring to the object.
According to claim 4,
The object analysis unit
Using at least one of a scale-up algorithm according to ESRGAN, a background difference (absdiff) algorithm, a dilate algorithm, a changed object distinction (findContours) algorithm, a detected object bounding and coordinate value extraction (boundingRect) algorithm, and an RGB extraction algorithm An object classification and multi-object big data acquisition management device that performs object analysis.
According to claim 1,
The control unit
At least one of the shutter, gain, and lighting operation timings using at least one of the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object determined based on the plurality of images. Object classification and multi-object big data acquisition management device that controls
According to claim 6,
The control unit
An object classification and multi-object big data acquisition management device included in the video monitoring device through at least one of HTTP communication and ONVIF.
According to claim 6,
The control unit
An object classification and multi-object big data acquisition management device for controlling a lighting MCU using at least one of serial communication or TCP communication to adjust brightness.
According to claim 6,
The control unit
An object classification and multi-object big data acquisition management device that controls the MCU of the light and the video monitoring device to control operation synchronization between the shutter and the light when the video monitoring device acquires the plurality of images.
According to claim 6,
The control unit
As deep learning is performed on the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object, the plurality of images including the object are named as characters representing the object. , object classification and multi-object big data acquisition management device.
According to claim 1,
The control unit
Whether or not the object is included in the plurality of images, the size of the object and the size of the area where the object is detected per frame determines the pulse value of the video monitoring device and lighting based on the speed of the object, object classification and a multi-object big data acquisition management device.
According to claim 1,
When the camera included in the video surveillance device operates in dual streaming,
The camera transmits the plurality of images according to single streaming, which is one of the dual streaming, to an object analysis unit and transmits the plurality of images according to the other one, sub-streaming, to a center for storing object classification and multi-object big data. Data Acquisition Management Unit.
According to claim 12,
The object analysis unit
Using a pre-stored algorithm, comparing the plurality of images according to the single streaming with preset object information to determine whether the object is included in the plurality of images;
When the object is included, determining at least one of the speed of the object, the type of the object, the characteristic of the object, the situation of the object, and the number of the object based on an image including the object, object classification and Multi-object big data acquisition management device.
According to claim 13,
The control unit
Whether or not the object is included in the plurality of images, the speed of the object, the type of the object, the characteristics of the object, the situation of the object, and the number of the object are transmitted to the center through the transmission/reception unit through FTP or socket communication method. Transmitting, object classification and multi-object big data acquisition management device.
According to claim 13,
The control unit
An object classification and multi-object big data acquisition management apparatus for updating at least one of the shutter, the gain, and the lighting to a preset value when the object is a vehicle.
According to claim 1,
a frame-by-frame image receiving unit that acquires a plurality of images from a video monitoring device that monitors crime prevention, disaster, safety, or traffic conditions;
A coordinate value extractor for each frame that performs interpolation based on the color of a pixel to be upscaled to filter noise included in the initial image, extracts RGB values for each frame as a character string, and transforms the plurality of images into a character string. ;
an object analyzer configured to determine at least one of a speed of an object included in the plurality of images, a type of the object, a characteristic of the object, a situation of the object, and a number of the object included in the plurality of images;
A comparison analysis unit that classifies fixed objects and moving objects through the recognition values of the object analysis unit and sets an average value of about 50-100 pix based on the center value to determine I-frames and P-frames through three images;
a controller for controlling at least one of a shutter, a gain, and illumination based on at least one of whether the object is included in the plurality of images, a size of the object, and a size of a region in which the object is detected per frame; and
Object classification and multi-object big data acquisition management apparatus further comprising; a selection transmission unit for transmitting only the images selected by the comparison analysis unit.