KR102497939B1 - object identification system by using image processing and operating method thereof - Google Patents

Abstract

An object recognition system using image processing according to an embodiment of the present invention includes at least one image sensor for collecting 360-degree image data, an image divider for generating and processing partial image data by segmenting image data, and partial image data At least one image sensor module including an object detection unit that detects an object and generates image information by confirming a change using an optical module, an optical module that collects pre-specified interest object information using an optical function and transmits it to a control module, and interest It includes a control module that recognizes objects and performs learning on them.

Description

Object identification system by using image processing and operating method thereof}

The present invention relates to an object recognition system using image processing and an operating method thereof. More specifically, it relates to an image processing technique using a rotational camera and an optical camera and an object recognition system and method using the same.

Research related to the present invention was carried out with support from the Korea Electric Power Corporation's 2018 energy base university cluster project. lim)

An object recognition system that recognizes an object identifies an object through a fixed position and angle and provides image information about the object to the user. An area desired by a user is observed through a closed circuit television (CCTV) or a digital video recorder (DVR), and a person recognizes an observed object through the naked eye. In addition, by observing an object or region that is indistinguishable from the naked eye through an optical camera, the user can check image information. In order to obtain such image information, a 360-degree field of view is implemented using a plurality of cameras in a 360-degree space according to the user's command or request. fix the direction. Image data output from each camera is transmitted to a system that users can check. The system configured in this way has a problem in that the price is high because a plurality of cameras are used, and the system is complicated and power consumption is high in order to transmit a large amount of image data. In addition, since there is a large amount of image data to be processed, image processing techniques are complicated and require a lot of processing time. In order to observe 360-degree images, image data is displayed using a panoramic display method on a user interface such as a monitor. Such a panoramic image is displayed on a user interface by connecting each input image data using warping and blending. This conventional method requires a lot of memory usage and delay time to collect image data and combine them into a panoramic image, and has a problem in that a lot of power is consumed to perform this. In addition, the existing method has difficulty in providing real-time changes to the shape of a distant object or region of interest.

Accordingly, an object of the present invention is to provide an image processing method capable of recognizing the shape of an object in an entire 360-degree space, using a low-capacity memory for image information in a region of interest, and processing image data quickly in order to solve the above problems. is in providing In addition, it is to provide a system capable of operating low-power real-time image information using the image processing method.

In addition, an object of the present invention is to provide a user control device capable of observing such image information and changes in a region of interest in real time.

In order to solve the above problems, an object recognition system using image processing according to the present invention and an operating method thereof are provided.

An object recognition system using image processing according to an embodiment of the present invention includes at least one image sensor that collects 360-degree image data, and an image divider that divides the image data to generate partial image data and processes the partial image data. , and at least one or more image sensor modules including an object detection unit that detects an object using the partial image data, checks a change in the object, and generates image information about the image data, and a predetermined interest using an optical function. It includes an optical module that collects object information and transmits it to the control module, and a control module that recognizes the object of interest and performs learning on it.

An object recognition method using image processing according to an embodiment of the present invention is an object recognition method using image processing in an object recognition system including at least one image sensor module, an optical module, a control module, and an interface module, wherein the image sensor module The step of collecting the 360-degree image data, the image sensor module dividing the image data to generate partial image data and processing it, the image sensor module detecting an object using the partial image data and detecting a change The method may include generating image information by checking, generating information on a predetermined object of interest based on the image information by the optical module, and recognizing and learning the object of interest by the control module.

According to an embodiment of the present invention, the processing speed for 360-degree image data is increased by using the partial image processing technique, and the amount of buffer usage and power consumption can be reduced.

In addition, according to one embodiment of the present invention, the weight and shape can be minimized, so it is suitable for an embedded system.

In addition, according to an embodiment of the present invention, it is possible to recognize a distant object and observe a region of interest in real time.

1 is a block diagram of an object recognition system according to an embodiment.
2 is a block diagram of an image sensor module in an object recognition system according to an embodiment.
3 is a diagram illustrating an example of an image sensor module in an object recognition system according to an embodiment.
4 illustrates a method of transmitting an image using one image sensor module according to an exemplary embodiment.
5 illustrates a method of transmitting an image using four image sensor modules according to an embodiment.
6 is a diagram illustrating an example of a partial image processing technique of an object recognition system according to an embodiment.
7 is a diagram illustrating an example of an optical module in an object recognition system according to an embodiment.
8 is a diagram illustrating an example of a control module in an object recognition system according to an embodiment.
9 is a flowchart of an object recognition method in an object recognition system according to an embodiment.

The features and effects of the present invention described above will become more apparent through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs can easily practice the technical idea of the present invention. You will be able to.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this application, it should not be interpreted in an ideal or excessively formal meaning. Should not be.

The suffixes module, block, and unit for the components used in the following description are assigned or used interchangeably in consideration of ease of specification preparation, and do not have meanings or roles that are distinguished from each other by themselves.

Hereinafter, a preferred embodiment of the present invention will be described so that those skilled in the art can easily implement it with reference to the accompanying drawings. In the following description of embodiments of the present invention, if it is determined that a detailed description of a related known function or known configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

Hereinafter, an object recognition system according to the present invention will be described.

1 is a block diagram of an object recognition system 10 according to an embodiment. The object recognition system 10 according to the present invention may include an image sensor module 100, an optical module 200, a control module 300 and an interface module 400.

The image sensor module 100 may collect image data using an image sensor, divide the image data to generate partial image data, and detect an object using the partial image data.

2 is a block diagram of an image sensor module 100 in an object recognition system according to an embodiment. The image sensor module 100 may include at least one image sensor 110 , an image division unit 120 , an object detection unit 130 , a communication unit 140 and a storage unit 150 .

The image sensor module 100 collects and collects 360 degree image data using the at least one image sensor 110 .

3 is a diagram showing an example of the image sensor module 100 in the object recognition system 10 according to an embodiment. The image sensor module 100 according to an embodiment of the present invention may be an image sensor embedded platform in which at least one or more image sensors are mounted.

Referring to FIGS. 2 and 3 , an image divider 120 of the image sensor module 100 according to an embodiment of the present invention sequentially transmits image data according to a partial image transmission technique. Since partial images are extracted, transmitted, and processed for the image data collected through the image sensor, it is possible to process quickly, operate with low power, and use less memory on the platform. Real-time processing of the image data is possible.

4 illustrates a method of transmitting an image using one image sensor module according to an exemplary embodiment.

Specifically, when using one image sensor module, it rotates 6 degrees per 1/60 second and can rotate 360 degrees per second. The embedded platform equipped with a general image sensor (CCD, CMOS, IR) has a viewing angle of 60 degrees, and moves left and right with a deviation of 192 pixels horizontally per 1/60 second in 1920 x 1280 image data. In a general image sensor, 192 pixels of input image data, that is, image data of 192 x 1280 are processed and executed 60 times to construct a 360-degree image data panorama. In addition, in the case of using one image embedded platform, 360-degree image data can be collected using a step motor that rotates 20 times per second.

5 illustrates a method of transmitting an image using four image sensor modules according to an embodiment.

When using four image sensor modules, 360-degree image data can be collected using a step motor that rotates five times per second. Partial image data according to the rotation of the stepper motor may vary according to embodiments.

6 is a diagram illustrating an example of a partial image processing technique of an object recognition system according to an embodiment.

The image dividing unit 120 of the image sensor module 100 according to an embodiment of the present invention sequentially transmits image data according to a partial image processing technique. A partial image processing technique according to an embodiment of the present invention may be a sub cascade technique.

Specifically, the image segmentation unit 120 divides image data to generate partial image data. At this time, partial image data may be generated in consideration of the rotation angle of the stepper motor. Referring to FIG. 6 , partial image data may be n pieces of partial image data when moving from left to right in a sub-cascade manner. That is, partial image data of 1 to n portions may be included in one image data.

Next, the image division unit 120 transmits partial image data to the object detection unit 130 using a partial image processing technique.

It processes the newly input partial image data, compares the existing image data of 1 to n with the existing image data, and transmits only the differences.

The object detector 130 corrects the received partial image data, compares the received partial image data with the partial image data of the previous frame, checks the changed portion, and generates image information. The object detector 130 may classify an object and a background from the received partial image data through edge detection and FFT (Fast Fourier Transforms), and perform a correction algorithm according to size and location changes. The object detection unit 130 checks the changed part by comparing it with the partial image data of one frame before.

The object detector 130 may be implemented as a Field Programmable Gate Array (FPGA). The FPGA is hardware-optimized and has a fast processing speed, and can process object detection for the partial image data in real time.

Using the partial image processing method according to an embodiment of the present invention increases the processing speed and reduces the amount of buffer usage and power consumption. In addition, it can minimize weight and shape, so it is suitable for embedded systems.

7 is a diagram showing an example of an optical module 200 in the object recognition system 10 according to an embodiment. The optical module 200 according to an embodiment of the present invention may be an optical embedded platform equipped with an optical function.

The optical module 200 collects information on an object of interest by using an optical function such as optical zoom or autofocus for a previously designated object of interest based on the image information generated by the image sensor module 100, and controls the module 300. ) can be transmitted. The object of interest may be an object of interest or a region of interest. The optical module 200 may operate a zoom lens, a focus lens, and rotation using a servo motor. In addition, by using the optical module 200, information of up to 20 km of a previously designated object of interest may be recognized and displayed on the interface module 400 such as a monitor.

8 is a diagram showing an example of a control module 300 in the object recognition system 10 according to an embodiment.

Referring to FIG. 8 , the control module 300 performs object recognition and learning. Object recognition and learning results can be stored in a database. That is, the control module may perform object recognition and learning on an object of interest, and may store the object recognition result and learning result in a database. Object recognition and learning results stored in the database can be used as learning data for input image information. Changes in the object of interest can be identified through the object recognition data and learning data, and information on the object of interest and changes in the object can be transmitted to the user through the interface module 400 .

Hereinafter, with reference to FIG. 9, an embodiment of an object recognition method will be described.

9 is a flowchart of an object recognition method in an object recognition system according to an embodiment of the present invention.

As shown in FIG. 9, the object recognition method according to an embodiment of the present invention is an object recognition method using image processing in an object recognition system composed of at least one image sensor module, an optical module, a control module, and an interface module. The image sensor module collects 360-degree image data, the image sensor module divides the image data to generate partial image data and processes it, and the image sensor module detects an object using the partial image data and generating image information by checking the change, generating information on a predetermined object of interest based on the image information by the optical module, and recognizing and learning the object of interest by the control module. .

The object recognition system includes an image sensor module, an optical module, a control module and an interface module. The image sensor module may collect image data using an image sensor, divide the image data to generate partial image data, and detect an object using the partial image data.

Specifically, the image sensor module collects 360 degree image data using at least one image sensor (S110).

Next, the image segmentation unit divides the image data to generate partial image data, which is processed and transmitted to the object detector (S120). The image segmentation unit may sequentially transmit image data according to a partial image transmission technique. For the image data collected through the image sensor, partial images are extracted, transmitted, and processed, so memory usage is small and real-time processing is possible. The partial image processing technique may be a sub cascade technique.

The object detector corrects the received partial image data and compares it with the partial image data of the previous frame to check the changed part (S130). That is, the object detector detects an object using the partial image data, checks a change in the object, and generates image information. The object detector may classify an object and a background from the received partial image data through edge detection and FFT (Fast Fourier Transforms) and perform a correction algorithm according to size and positional changes. The object detection unit may check the changed part by comparing with partial image data of one frame before.

The optical module may collect information on an object of interest by using optical zoom or autofocus for a previously designated object of interest based on image information observed by the image sensor module and transmit the information to the control module (S140). The object of interest may be an object of interest or a region of interest.

The control module may perform object recognition and learning on the object of interest (S150). In addition, the object recognition result and learning result may be stored in a database. Object recognition and learning results stored in the database can be used as learning data for input image information. Changes in the object of interest can be distinguished through the object recognition data and learning data, and information on the object of interest and changes in the object can be delivered to the user through the interface module.

In the above, various embodiments of an image sensor processing module using a partial image processing technique according to the present invention, an object recognition system including the image sensor processing module, and an object recognition method using the object recognition system have been described.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

According to the software implementation, not only the procedures and functions described in this specification, but also the design and parameter optimization of each component may be implemented as a separate software module. The software code may be implemented as a software application written in any suitable programming language. The software code may be stored in a memory and executed by a controller or processor.

Claims (16)

At least one image sensor that collects 360-degree image data, an image division unit that divides the image data to generate partial image data and processes the partial image data, and detects an object using the partial image data and detects the object At least one image sensor module including an object detector for generating image information for the image data by checking a change in
An optical module that collects information on a predetermined object of interest using an optical function and transmits it to the control module; and
A control module recognizing the object of interest and learning about it;
The image division part
Processing partial image data using a sub cascade technique,
The control module stores object recognition results and learning results in a database, object recognition and learning results stored in the database are used as learning data for input image information, and changes in the object of interest are monitored through the object recognition data and learning data. An object recognition system using image processing that classifies and transmits information on an object of interest and its change to an interface module. delete According to claim 1,
Further comprising an interface module displaying information about the object of interest and change of the object of interest
Object recognition system using image processing. According to claim 1,
The object recognition system using image processing implemented by the object detection unit with a field programmable gate array (FPGA). According to claim 1,
The object detection unit uses image processing to distinguish objects and backgrounds using edge detection and FFT (Fast Fourier Transforms) and perform correction algorithms according to size and position changes. According to claim 1,
The optical module generates information on the object of interest using optical zoom or autofocus for a previously designated object of interest based on the image information processed by the image sensor module and recognizes the object using image processing to transmit the information to the control module. system. An object recognition method using image processing in an object recognition system consisting of at least one image sensor module, an optical module, a control module, and an interface module,
Collecting, by the image sensor module, 360-degree image data;
generating, by the image sensor module, partial image data by dividing the image data and processing it;
generating image information by using the image sensor module to detect an object and confirm a change using the partial image data;
generating, by the optical module, information on a predetermined object of interest based on the image information; and
Recognizing and learning the object of interest by the control module;
The step of generating and processing the partial image data,
Processing the partial image data using a sub cascade technique;
The step of recognizing and learning the object of interest by the control module stores object recognition results and learning results in a database, and object recognition and learning results stored in the database are used as learning data for input image information, and object recognition data and classifying changes in the object of interest through the learning data and transmitting the object of interest and information on the change in the object of interest to an interface module. delete According to claim 7,
The method of recognizing an object using image processing further comprising displaying information on the object of interest and a change of the object of interest on an interface module. According to claim 7,
Generating the image information,
Object recognition method using image processing implemented with FPGA (Field Programmable Gate Array). According to claim 7,
Generating the image information,
An object recognition method using image processing that distinguishes objects and backgrounds using edge detection and FFT (Fast Fourier Transforms) and performs correction algorithms according to size and position changes. According to claim 7,
Generating, by the optical module, information on the object of interest
An object recognition method using image processing for generating information on an object of interest using optical zoom or autofocus for a previously designated object of interest based on the image information processed by the image sensor module and transmitting the information to the control module. delete delete delete delete

Images