KR101268391B1 - Image processing apparatus and method thereof - Google Patents

Abstract

The present specification relates to an image processing apparatus and a method for improving an image compression ratio by removing overlapping image information by comparing image frames that are periodically repeated instead of comparing overlapping image frames by comparing adjacent image frames. will be. To this end, an image processing apparatus according to an embodiment of the present disclosure includes a camera module for photographing an image of a region of interest while periodically moving a camera repeatedly in a predetermined movement path; An image processor may be configured to detect image frames that are periodically repeated based on the captured images, and to perform image compression to remove overlapping image information among the periodically repeated image frames.

Description

Image processing apparatus and its method {IMAGE PROCESSING APPARATUS AND METHOD THEREOF}

The present specification relates to an image processing apparatus and a method thereof.

In general, the image processing method applied to the ground operation surveillance equipment, to install a surveillance camera at the main point of interest to monitor a wide area such as coastal and mountainous area. Looking at the operational characteristics of the ground operation monitoring equipment, it is possible to repeatedly monitor the region of interest at a fixed period from a fixed position, and to process the acquired image to detect / recognize the target and synthesize the image. Such detection / cognition results and composite images are transmitted to the rear control station or the superior unit using wired / wireless communication as necessary.

In the above-mentioned ground-based surveillance equipment, cameras for monitoring a wide area of interest may use wide-angle cameras, but wide-angle cameras have a problem that it is difficult to detect / recognize the periphery of the lens and remote objects. The application is very limited in the ground operation surveillance equipment. Therefore, ground-based surveillance equipment monitors the entire area of interest by repeatedly moving one or more cameras that employ a telephoto lens with a field of view (FOV) of 10 ° or less as an alternative to wide-angle lenses. At this time, in the repetitive movement of the camera from side to side, in the classic surveillance equipment that observers directly monitor the image, a method of continuously driving the camera was applied.

The continuous driving method of the camera has an advantage of relatively easy control of camera movement, but when the time required for collecting the amount of light required for image acquisition is relatively long, such as an infrared sensor, the acquired image is blurred due to the movement of the camera for precise monitoring. Makes it difficult. In addition, it is necessary to use a high frame rate in order to ensure the high quality of the image according to the continuous driving of the camera, the continuous image having such a high frame rate is very large data load, signal processing, image synthesis for detection / recognition And making image processing such as image transmission through wired / wireless communication difficult. Due to this problem, the camera operation in the ground operation monitoring equipment stops the image acquisition during movement, and uses a step driving to acquire the image by stopping for a short time at a specified angle.

1 is an exemplary view illustrating an image processing concept according to a camera step driving according to the related art. That is, FIG. 1 is a diagram illustrating a concept of operating an image such as a camera acquired image, a synthesized image, a compression, and a transmission of the ground operation monitoring equipment using the step driving.

As shown in FIG. 1, the acquired camera image is obtained by overlapping a partial region for synthesis, and performs a synthesis operation such as continuously connecting the acquired camera image based on data of the overlapped region. By applying such step driving, it is possible to acquire high quality images such as still images, and due to the reduction of the image frame rate according to the step driving, it is possible to greatly reduce the data load for signal processing for detection / recognition and image synthesis. In addition, the acquired image or the synthesized image due to the step driving has a different characteristic from that of a general video. That is, the general video has a high correlation between successive image frames by applying a high image frame rate, but the image and the synthesized image obtained by the step driving show little spatial correlation between adjacent frames. Therefore, since a periodically repeated image does not have a high correlation between adjacent image frames, a high compression ratio is used as a general compression method in which duplicate information between image frames is deleted by setting one of the consecutive image frames as a reference image frame. Difficult to achieve

An object of the present specification is to compare image frames adjacent to each other and not to delete overlapping information between comparative image frames, and to compare image frames that are periodically repeated, thereby improving image compression rate by using overlapping image information between frames. An image processing apparatus and a method thereof are provided.

Another object of the present specification is to compare video frames adjacent to each other and not to delete overlapping information between comparative image frames, and to reduce video traffic load by comparing overlapping image frames periodically and using overlapping image information between frames. An image processing apparatus and a method thereof are provided.

Yet another object of the present specification is to compare wired / wireless communication by using overlapping image information between frames by periodically comparing image frames rather than deleting overlapping information between comparative image frames by comparing adjacent image frames. The present invention provides an image processing apparatus and a method for improving the image transmission performance.

An image processing apparatus according to an embodiment of the present disclosure includes a camera module for photographing an image of a region of interest while periodically moving a camera repeatedly in a predetermined movement path; An image processor which detects image frames that are periodically repeated based on the captured images, and compares image frames having the same azimuth angle among the periodically repeated image frames and compresses the image by using overlapping image information It may include.

As an example related to the present invention, the camera module may photograph the periodically repeated images according to a direction azimuth of the camera while periodically moving the camera repeatedly in a predetermined movement path.

As an example related to the present invention, each of the periodically repeated image frames may include a direction azimuth of the camera.

As an example related to the present invention, the image processor detects the directed azimuth angles and compares image frames having the same directed azimuth angle among the detected directed azimuth angles with each other so that the image information overlaps between the periodically repeated image frames. Image frames may be compressed using.

As an example related to the present invention, the camera module may photograph the periodically repeated images by repeatedly moving the camera from side to side.

As an example related to the present invention, the image processing unit receives image frames including the azimuth information of the camera from the camera module, detects the periodically repeated image frames among the image frames, and periodically Repeated image frames may be compared with each other to compress the image frames to have a high image compression rate by using overlapping image information in the image frame.

As an example related to the present invention, the apparatus may further include a communication unit configured to transmit the compressed image frame.

As an example related to the present invention, the camera module may capture the image while moving the camera at a predetermined step movement angle.

As an example related to the present invention, the preset step movement angle may be changeable.

An image processing apparatus according to an embodiment of the present disclosure includes a camera module for photographing images of a region of interest according to a direction azimuth of the camera while periodically moving a camera repeatedly in a predetermined movement path; An image processor which detects periodically repeated image frames having the directed azimuth from the photographed images, compares them, and compresses the image frames using overlapping image information, wherein the image processor periodically By detecting the azimuths included in each of the repeated image frames and comparing the image frames having the same direction azimuth among the detected direction azimuths, the image information overlapped between the periodically repeated image frames may be used. .

An image processing method according to an exemplary embodiment of the present disclosure includes: photographing an image of a region of interest while periodically moving a camera repeatedly in a predetermined movement path; Detecting image frames that are periodically repeated based on the captured images; The method may include using overlapping image information among the periodically repeated image frames.

An image processing method according to an exemplary embodiment of the present disclosure includes: capturing images of a region of interest according to a direction azimuth of the camera while periodically moving a camera repeatedly in a predetermined movement path; Detecting periodically repeated image frames having the directed azimuth from the captured images; Using image information overlapping the periodically repeated image frames; Compressing periodic image frames, wherein using the overlapping image frame comprises: detecting a directed azimuth included in each of the periodically repeated image frames; The method may include compressing the image frames having the same direction azimuth angle among the detected direction azimuth angles by using the image information overlapping the periodically repeated image frames.

In the image processing apparatus and the method according to the embodiments of the present invention, the image compression ratio is increased by comparing the image frames adjacent to each other and using the overlapping image information by comparing the repeated image frames. Can be improved.

The image processing apparatus and the method according to the embodiments of the present invention compare image frames adjacent to each other and delete the duplicate image information, and use the image information by comparing the repeated image frames and using the overlapped image information. The load can be reduced.

The image processing apparatus and the method according to the embodiments of the present invention may not overlap the image frames adjacent to each other and delete the duplicate image information, but use the duplicate image information by comparing the repeated image frames. Image transmission performance using wireless communication can be improved.

1 is an exemplary view illustrating an image processing concept according to a camera step driving according to the related art.
2 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present invention.
3 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present invention.
FIG. 4 is an exemplary view illustrating an operation of an image compression method using correlation of periodic repetitive images applied to terrestrial operating surveillance equipment according to an embodiment of the present invention.
5A and 5B are exemplary views illustrating two step driving methods of a camera for acquiring a composite image frame of the terrestrial surveillance equipment according to an exemplary embodiment of the present invention.
FIG. 6 is an exemplary diagram for describing a limited image (synthetic image) compression system for a case where L image compression modules can be implemented.

Recently, the interest in the ground operation surveillance equipment for unmanned / remote monitoring of barbed wire and coastline has been increasing for the modernization of the military. The basic operation of such ground operation monitoring equipment is to install a camera at a fixed position, and step-by-drive the camera to repeatedly monitor the region of interest. At this time, the image obtained through the camera is converted to the composite image for the display and detection / recognition of the target to the user, and the composite image is transmitted to another area by using the display, detection / recognition signal processing or wired / wireless communication For this purpose, various image signal processing such as image compression is performed.

Currently widely used video compression technology supports high video compression rate by using the correlation between adjacent spaces in the video frame and the correlation between successive video frames. Since the correlation between adjacent image frames is not large, it is difficult to achieve a high compression rate even when a compression method of deleting image information overlapping between image frames by setting one of the consecutive image frames as a reference frame is difficult. . Therefore, periodic image frames directed to the same region than adjacent image frames have a high correlation with each other. That is, the general video has a high correlation between successive image frames by applying a high image frame rate, but the image and the synthesized image obtained by the step driving show little spatial correlation between adjacent frames. On the other hand, images directed to the same area show characteristics having a high correlation with each other according to the periodic repetitive movement of the camera.

Hereinafter, referring to FIGS. 1 to 6, an image processing apparatus and a method for deleting overlapping image information by comparing image frames that are periodically repeated instead of deleting duplicate image information by comparing image frames adjacent to each other are described with reference to FIGS. 1 to 6. Will be explained.

2 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present invention.

As illustrated in FIG. 2, the image processing apparatus 100 according to an exemplary embodiment of the present invention may be configured to capture an image of a region of interest while periodically moving a camera (not shown) in a predetermined movement path. 101; The image processing unit 102 detects periodically repeated image frames based on the captured images, and deletes overlapping image information (s) among the periodically repeated image frames.

The camera module 101 captures images that are periodically repeated according to a direction azimuth of the camera while periodically moving the camera repeatedly in a predetermined movement path.

The image processor 102 compares the image frames having the same direction azimuthal angle among the periodically repeated image frames, and uses the image information overlapped between the periodically repeated image frames and compresses the image frame. .

The image processor 102 compresses image frames by using the overlapped image information, and transmits the compressed image frames to the server 200 through a communication network.

The storage unit 103 of the image processing apparatus 100 according to an embodiment of the present invention stores the images acquired by the camera module 101.

The camera module 101 may move the camera at a predetermined step movement angle (for example, 6 degrees).

3 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present invention.

First, the camera module 101 periodically moves the camera (not shown) repeatedly in a predetermined movement path (S11). For example, the camera module 101 captures the preset region of interest by repeatedly moving the camera left and right to capture a preset region of interest. In this case, the camera module 101 may repeatedly move the camera left and right or move the camera continuously left and right repeatedly by the step driving method. Here, images (image frames) that are periodically repeated are generated by taking an image while repeatedly moving the camera left and right.

The camera module 101 detects an azimuth (direction azimuth) facing the camera when the preset region of interest is captured by the camera, and records the detected azimuth in the header of each of the captured image frames. . For example, the camera module 101 records the first direction azimuth in the header of the image frame photographed at the first direction azimuth through a sensor such as a gyro sensor or a resolver, and records the image frame at the second direction azimuth. The second directed azimuth is recorded in the header. That is, each of the image frames includes direction azimuth information of the camera.

The camera module 101 outputs image frames including the azimuth information of the camera to the image processor 102.

The image processor 102 receives image frames including the azimuth information of the camera from the camera module 101, and synthesizes the received image frames (S12). For example, the image processing unit 102 may sequentially connect the image frames photographed by the first movement of the camera among the received image frames in the order in which the images are taken. Frames), and the second composite image frame (periodically repeated image frames) is generated by successively connecting the image frames photographed by the second movement of the camera in the photographed order. That is, the image processor 102 periodically repeats the image frames photographed while moving the second position from the first position, assuming that the camera repeatedly moves the second position from the first position. Determine the image frames.

The image processor 102 detects the periodically repeated image frames among the image frames received from the camera module 101 (S13).

The image processor 102 compresses an image by using overlapping image information between the periodically repeated image frames (S14). For example, the image processor 102 detects a direction azimuth included in each of the periodically repeated image frames, and compares the image frames including the same direction azimuth from each other among the detected direction azimuths. The video is compressed by using overlapping image information between periodically repeated image frames.

The image processor 102 deletes (excludes) the overlapped image information when the overlapped image information exists between the periodically repeated image frames.

The image processor 102 compresses the image frames based on the image information from which the duplicated image information is deleted (excluded) (S14). For example, the image processor 102 may improve an image compression ratio by compressing image frames based on the overlapped image information.

The image processor 102 compresses the image frames from which the duplicate image information is deleted, and transmits the compressed image frames to the server 200 through a communication network.

On the other hand, the image processing apparatus 100 according to an embodiment of the present invention may be applied to the ground operation monitoring equipment. Hereinafter, the ground operation monitoring equipment to which the image processing apparatus according to the embodiment of the present invention is applied will be described with reference to FIGS. 4 to 6.

FIG. 4 is an exemplary view illustrating an operation of an image compression method using correlation of periodic repetitive images applied to terrestrial operating surveillance equipment according to an embodiment of the present invention.

Typical image compression methods are continuous frames adjacent to the first frame, which is a reference frame, 2, 3,... The duplicated information is deleted by comparing the M-th frames with each other, and the compression efficiency is lowered because the correlation between successive image frames is low. Here, M is the number of frames to be compared used for deleting redundant information (overlapped image frame information) between image frames. However, the image compression according to the embodiment of the present invention utilizes the first frame as a reference frame in the same way as general image compression, but the image is repeated periodically in units of N periods instead of consecutively adjacent image frames. Use frames. Here, N denotes the number of frames of the composite image that may represent an entire region of interest as a period of the composite image frame. That is, the first image frame and N + 1, 2N + 1,... Compared with the MN + 1st image frames, these image frames are all local images having the same or similar directed azimuth angles, and thus have high correlations between image frames, unlike successive image frames. Also, the second frame includes N + 2, 2N + 2,... Compared to the MN + 2th image frames, a correlation between high image frames can be guaranteed to support high image compression rates. As such, by processing the composite image frames into N independent images and comparing the composite image frames in units of periods, image compression between image frames may be performed to achieve a high image compression ratio for the entire image.

Hereinafter, a step driving (for example, step movement angle) scheme and system design of a camera for increasing an image compression ratio in the directed azimuth based ground operation monitoring equipment will be described. The ground-operated surveillance equipment drives the camera to obtain a clear image. In addition, in order to further increase the video compression rate of the terrestrial surveillance equipment, it is necessary to operate the camera so that the k th frame (for example, the first frame) and the N + k th image frame always point to the same area.

5A and 5B are exemplary views illustrating two step driving methods of a camera for acquiring a composite image frame of the terrestrial surveillance equipment according to an exemplary embodiment of the present invention.

As shown in FIGS. 5A and 5B, it is assumed that a field of view (FOV) of the camera is 8 °, the surveillance area is 360 °, and the step driving angle (step movement angle) of the camera is 7 ° and 6 °, respectively. . When the camera step driving angle is 7 °, that is, when the step driving is performed at an angle that is not a divisor of the entire surveillance area 360 °, the first image frame and the N + 1th image frame do not face the same azimuth angle. However, when the step driving angle (step movement angle) of the camera is 6 ° and is a divisor of 360 ° of the entire surveillance area, the first image frame and the N + 1th image frame are directed at the same azimuth angle so that the correlation between the two image frames is achieved. The relationship is greatest. Thus, as a sub-multiple of the step driving angle of the surveillance camera entire surveillance area to a high image compression rate of the composite video frame, so that the overlap angle is seamless to the image combining θ _min ≤ θ _step <θ _fov May be determined as follows. Where θ _step is a divisor of θ _syn , θ _min is the minimum overlap angle for image synthesis, θ _step is the camera's step driving angle, θ _fov is the camera's field of view (FOV), and θ _syn is the The difference between the start and end range angles.

The orientation angle (azimuth angle) of the camera can be obtained through various sensors such as a gyro and a resolver, and the orientation direction of the camera applied to the ground operating surveillance equipment corresponds to the image orientation area to use the orientation angle of the camera. This makes it easy to predict the correlation between images that are periodically repeated. That is, if the directed azimuth angles are the same, it may be determined as a periodic repetitive image having a high correlation and used as a comparison image frame for compressing the composite image.

In order to implement an image processing apparatus applied to the above-mentioned ground operating surveillance equipment, a means for independently compressing N images, which are a composite image frame period, is required. Application of hardware / software for parallel processing is essential for the plurality of image compression, but it may be difficult to design and implement a system capable of compressing N independent images due to limitations in processing power of the applied hardware and the central processing unit. Therefore, when a maximum of L compression modules can be implemented within a limited range, a composite video compression system design of the ground operating monitoring equipment will be described with reference to FIG. 6.

FIG. 6 is an exemplary diagram for describing a limited image (synthetic image) compression system for a case where L image compression modules can be implemented. For example, in an environment in which L compression modules are implemented, the proposed composite image compression may apply L-1 image compression modules and the other one image compression module may apply a general image compression module. Where L is a natural number. The L-1 image compression modules perform the proposed composite image compression method on the composite image frames having M co-orientation azimuths, and the other image compression module performs general image compression on all remaining composite image frames. This allows compression and reconstruction of entire image frames. Such a system design method may be designed to support a maximum image compression ratio by designing a system with L = N if system resources are available, and to implement a system at the expense of an image compression ratio when system resources are insufficient. In order to apply the proposed composite image compression method, it is necessary to include the azimuth information in the header of the image so that an appropriate image compression module according to the azimuth information of the camera is selected.

As described above, the image processing apparatus and the method according to the embodiments of the present invention do not remove overlapping image information by comparing image frames adjacent to each other, and compare overlapping image frames by periodically repeating image frames. By removing the information, the video compression ratio can be improved.

The image processing apparatus and the method according to the embodiments of the present invention do not remove duplicate image information by comparing image frames adjacent to each other, and remove image data by comparing the repeated image frames and removing the duplicate image information. The load can be reduced.

The image processing apparatus and its method according to the embodiments of the present invention are not to remove duplicate image information by comparing image frames adjacent to each other, and to remove duplicate image information by comparing image frames that are periodically repeated. Image transmission performance using wireless communication can be improved.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

101: camera module 102: image processing unit
103: storage unit 104: communication unit

Claims (16)

A camera module for generating a plurality of image frames from left to right by periodically moving the camera from the left to the right in order to monitor the region of interest;
The first composite image frame is generated by sequentially connecting the image frames photographed while the camera moves first from the left to the right, and the image frames photographed by the second movement of the camera from the left to the right. Generating a second composite image frame by successively connecting the images in the photographed order, comparing the first composite image frame with the second composite image frame, and as a result of the comparison, the first composite image frame and the second composite image frame An image processor which deletes any one of the first composite image frame and the second composite image frame and compresses the deleted composite image frame when the overlapping is performed;
And a communication unit which transmits the compressed composite image frame. The method of claim 1, wherein the camera module,
And periodically photographing the periodically repeated images according to a direction azimuth of the camera while repeatedly moving the camera periodically. The image processing apparatus of claim 2, wherein each of the periodically repeated image frames includes a direction azimuth of the camera. delete delete delete delete delete delete delete delete delete delete delete delete delete

Images