KR20190026147A - Region of interest changeable and manifiable image processing system and method - Google Patents

Abstract

The present invention relates to an image processing system capable of moving a region of interest and magnifying an image, and to a method thereof. The image processing system comprises: a camera photographing an image; a foothold switch setting a region of interest inside the image photographed from the camera, and setting a magnification ratio or a reduction ratio of the image; and an image processing apparatus receiving the image from the camera, acquiring the image of the region of interest among the received image in accordance with the region of interest set by the foothold switch, and magnifying or reducing the image of the region of interest in accordance with the magnification ratio or the reduction ratio set by the foothold switch.

Description

TECHNICAL FIELD [0001] The present invention relates to an image processing system and method capable of moving a region of interest and enlarging an image,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing technique, and more particularly, to an image processing system and method capable of real-time central movement of an image captured from a fixed camera, that is, movement of an ROI and enlargement or reduction of an image.

The endoscopic camera has been developed many times, and now it is not only a plane image (2D) but also a stereoscopic image (3D), and a 4K camera is introduced in resolution. However, in spite of this remarkable development of the camera, the center of the camera is always fixed to only one place, and it is inconvenient to re-adjust the direction of the camera in order to observe the peripheral image. When an endoscope is used in a narrow space such as an ear canal or nasal cavity, there is a problem that an endoscope and a surgical instrument collide with each other and hinder smooth operation. In this case, since the use of the surgical instrument can be facilitated by fixing the endoscope, an endoscope holder for fixing the endoscope has been developed. However, even if the endoscope holder is used, the position of the endoscope during surgery is required to perform a smooth operation. Therefore, since the operation of moving and fixing the endoscope is repeated several times, the operation time becomes longer and the use of the endoscope holder becomes meaningless . Therefore, in order to maximize the advantage of the endoscopic surgery using the endoscopic holder, a function of automatically changing the center of the screen to be photographed in a state in which the endoscope is fixed is required.

Korean Patent No. 10-1707113

An object of the present invention is to provide an image processing system and method capable of freely changing the center of an image photographed by a camera to move a region of interest in real time.

It is an object of the present invention to provide an image processing system and method capable of moving a region of interest to a specific portion within an image without moving and fixing the endoscope fixed by the endoscope holder several times.

According to a first aspect of the present invention, there is provided an image processing system comprising: a camera for capturing an image; A foot switch for setting an area of interest in an image photographed by the camera and setting an enlargement or reduction ratio of the image; And a control unit for receiving an image from the camera and obtaining a region of interest image of the image according to a region of interest set by the foot switch, and enlarging or reducing the region of interest image according to an enlargement or reduction ratio set by the foot switch And an image processing device.

Preferably, the camera may correspond to an endoscope camera that captures a high-resolution image or a camera that is fixed at a specific position and includes a lens having an auto focus function and a zoom function.

Preferably, the foot switch includes a first footstool for setting the area of interest and a second footstep for setting the enlargement or reduction ratio, wherein the footstep switch comprises a footstep switch, And a region of a certain range from the position of the mark may be set as the region of interest.

Preferably, the image processing apparatus is provided with a continuous image photographed from the camera in real time, and can reproduce or stop the image according to a user's setting, and capture the image or capture a moving image of the image.

Preferably, the image processing apparatus filters an image corresponding to the set region of interest, the image is converted into a gray scale, filtered with a specific reference color, and image noise can be removed through a Gaussian filter.

Preferably, the image processing apparatus sets a center point of the ROI in the filtered image, captures an object in the filtered image, and a vertex of the largest object among the captured objects may be set as a center point.

Preferably, the image processing apparatus obtains and outputs a ROI image based on the center point, and when there is an enlargement or reduction ratio set by the foot switch, Enlarge or reduce the enlarged or reduced region of interest image, and re-output the enlarged or reduced region of interest image based on the center point.

According to a second aspect of the present invention, there is provided an image processing method performed in an image processing apparatus connected to a camera and a foot switch, the method comprising: (a) receiving an image photographed by the camera; (b) receiving setting information for setting a region of interest in an image photographed by the camera and setting an enlargement or reduction ratio of the image from the foot switch; And (c) acquiring a region of interest image of the image according to the region of interest set based on the setting information, and enlarging or reducing the region of interest image according to the set enlargement or reduction ratio.

Preferably, the camera may correspond to an endoscope camera that captures a high-resolution image or a camera that is fixed at a specific position and includes a lens having an auto focus function and a zoom function.

Preferably, the foot switch further comprises a first footrest for setting the area of interest and a second footrest for setting the magnification or reduction ratio, wherein step (c) comprises: Displaying a moving marker in the image according to an operation; And setting a region of a range from the position of the marker displayed in the image to the region of interest.

Preferably, in the step (c), when the continuous image photographed by the camera is received in real time, the image is reproduced or stopped according to the setting of the user, and the capturing of the image, And performing a capturing operation.

Preferably, the step (c) comprises filtering an image corresponding to the set region of interest, wherein the step of filtering the image comprises: converting the image into gray scale; Filtering the gray-scale-converted image with a specific reference color; And removing the image noise by applying a Gaussian filter to the image filtered with the specific reference color.

Preferably, the step (c) includes the step of setting a center point of the ROI in the filtered image, wherein the setting of the center point comprises: capturing an object in the filtered image; And setting a vertex of the largest object among the captured objects as a center point.

Preferably, the step (c) includes: obtaining and outputting a region of interest image based on the center point; And if the enlargement or reduction ratio is set according to the setting information, enlarges or reduces the region of interest image according to the enlargement or reduction ratio, and re-outputs the enlarged or reduced region of interest image based on the center point Step < / RTI >

As described above, according to the present invention, since the center of the camera can be moved to a desired region without moving the position of the camera to adjust the center of the camera to be observed, Since the region can be moved, the position of the endoscope can be fixed to one side of the very narrow ear canal such as ear surgery using an endoscope, so that the use of the surgical instrument is much easier.

1 is a block diagram of an image processing system according to a preferred embodiment of the present invention.
2 is a block diagram of an image processing apparatus according to an embodiment.
3 is a flowchart illustrating an image processing method according to an exemplary embodiment.
4 and 5 are illustrations for explaining an image processing method.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will be more apparent from the following detailed description taken in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification. "And / or" include each and every combination of one or more of the mentioned items.

Although the first, second, etc. are used to describe various elements, components and / or sections, it is needless to say that these elements, components and / or sections are not limited by these terms. These terms are only used to distinguish one element, element or section from another element, element or section. Therefore, it goes without saying that the first element, the first element or the first section mentioned below may be the second element, the second element or the second section within the technical spirit of the present invention.

Also, in each step, the identification code (e.g., a, b, c, etc.) is used for convenience of explanation, and the identification code does not describe the order of each step, Unless the order is described, it may happen differently from the stated order. That is, each step may occur in the same order as described, may be performed substantially concurrently, or may be performed in reverse order.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. It is noted that the terms "comprises" and / or "comprising" used in the specification are intended to be inclusive in a manner similar to the components, steps, operations, and / Or additions.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense commonly understood by one of ordinary skill in the art to which this invention belongs. Also, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions in the embodiments of the present invention, which may vary depending on the intention of the user, the intention or the custom of the operator. Therefore, the definition should be based on the contents throughout this specification.

1 is a block diagram of an image processing system according to a preferred embodiment of the present invention.

Referring to FIG. 1, an image processing system 100 includes a camera 110, a foot switch 120, and an image processing device 130.

The camera 110 is an apparatus for photographing an image. Preferably, the camera 110 may be an endoscope camera for photographing a high-resolution continuous image or a camera fixed at a specific position and equipped with an auto focus function and a zoom function lens. For example, the camera 110 may be a 12M global shutter CMOS camera and may take images in the region of 1920X1080 (2M) using USB 3.0. That is, if an ultra-high resolution camera is used, movement can be smooth by reducing the region of interest. Preferably, when the camera 110 is a camera used for endoscopic surgery, the camera 110 may be mounted on a coupler having a lens function for fixing the endoscope. In the case of a camera used for general surgery, Can be installed. According to the image processing method according to the present invention to be described below, the user is required to move the region of interest without adjusting the physical position of the camera, Lt; / RTI >

The foot switch 120 is a device for setting a region of interest in an image photographed from the camera 110 and setting an enlargement or reduction ratio of the image, And a second scaffold for setting an enlargement or reduction ratio. Preferably, the user uses the first footrest to move a marker of a specific color located in the image and to manipulate the switch of the first footrest (e.g., pressing the footrest with the foot) Can be set. In addition, the user can use the second scaffold to extract the image of the region of interest 1x, 1.5x, 2x, ... , Or 10 times or more, or the enlarged image may be reduced again. For example, if the user presses the second foot plate once, it can return 1.5 times, and if it is pressed twice, it can return to the original magnification by pressing twice or three times.

The image processing apparatus 130 is connected to the camera 110 and the foot switch 120 to perform an image processing method. Preferably, the image processing apparatus 130 receives an image from the camera 110, acquires a region of interest image according to the region of interest set by the foot switch 120, Or enlarge or reduce the region of interest image according to the reduction ratio. In addition, the image processing apparatus 130 may be connected to a separate display device for outputting an image, or may output an image processed through an output unit provided therein.

In one embodiment, the image processing system 100 may include a keyboard instead of the foot switch 120, and the first foot and second foot of the footswitch 120, and the switches provided in each of them, And may be implemented in the form of a specific key or button, and descriptions of the foot switch 120 to be described below may all be applied to the keyboard.

2 is a block diagram of an image processing apparatus according to an embodiment.

2, the image processing apparatus 130 includes an image receiving unit 131, a setting information receiving unit 132, an image processing unit 133, an image output unit 134, and a control unit 135. The control unit 135 controls the operations of the image receiving unit 131, the setting information receiving unit 132, the image processing unit 133, and the image sending unit 134 and the flow of data. Hereinafter, the operation performed in each configuration of the image processing apparatus 130 will be described in detail with reference to FIG. 2 and FIG.

Referring to FIG. 3, the image receiving unit 131 receives the image captured by the camera 110 (step S310). Preferably, the camera 110 may generate an image having a lower resolution than the photographable resolution so that the image having the lower resolution may be moved according to the movement of the ROI described below to acquire the ROI image For example, in the case of a camera that captures an image of UHD (3842X2160), a FULL HD (1920X1080) image may be generated and the FULL HD image itself may be moved according to the movement of the ROI to obtain a changed ROI image .

The setting information receiving unit 132 receives setting information for setting an area of interest in the captured image from the camera 110 and setting an enlargement or reduction ratio of the area from the foot switch 120 (step S320). The image received from the image receiving unit 131 is output in real time through the image output unit 134. When the foot switch 120 is operated, A mark that can indicate the center of the area can be output. At this time, the user can operate the first foot plate of the foot switch 120 to move the cover to a position to set as a region of interest while viewing the image on which the cover is being output, The position of the cover can be transmitted to the setting information receiving unit 132 through the operation of pressing the first foot plate. Here, the position of the cover corresponds to setting information for setting a region of interest. When the user operates the second footrest, that is, the second footrest is pressed a certain number of times, the corresponding number of times is transmitted from the foot switch 120 to the setting information receiving unit 132, The number of times of pressing corresponds to the setting information for setting the enlargement or reduction ratio.

The image processing unit 133 acquires the ROI image according to the ROI set on the basis of the setting information, and enlarges or reduces the ROI according to the set enlargement or reduction ratio (step S330). More specifically, when the position of the mark is received through the setting information receiving unit 132, the image processing unit 133 can set the region of interest within a certain range from the position of the mark, as shown in Fig. 4 (a) The image of the region of interest may be output through the image output unit 134 as shown in FIG. Hereinafter, the process of acquiring the ROI image from the ROI image shown in FIG. 4A will be described.

Preferably, the image processing unit 133 may filter the image corresponding to the set region of interest (FIG. 4A). More specifically, the image processing unit 133 converts an image into a gray scale, removes unnecessary parts by filtering an image converted to a gray scale with a specific reference color (RGB Threshold color filter) A Gaussian filter is applied to the removed image to remove the image noise. The filtered image is shown in FIG. 4 (b).

Preferably, the image processing unit 133 can set the center point of the ROI in the filtered image. More specifically, the image processor 133 detects the outline of the object using the Sobel filter in the filtered image shown in FIG. 4B to acquire an image as shown in FIG. 4C . The image processing unit 133 can capture and detect an object on the basis of the detected contour, and set the vertex (the red point shown in (c) of FIG. 4) of the largest object among the captured and detected objects as the center point.

Preferably, the image processing unit 133 obtains the ROI image based on the center point, and the image output unit 134 outputs the ROI image through the image processing unit 133. That is, the image processing unit 133 obtains the ROI image based on the center point indicated by the red dot in (c) of FIG. 4, and outputs the ROI image of interest, as shown in (d) . 4 (a) and 4 (d), the image obtained from the camera 110 is processed through the image processing unit 133 based on the region of interest.

The method performed through the image processing unit 133 described above processes an image such that an image based on a region of interest is acquired in accordance with movement of the center of the ROI within one image captured through the camera 110 When the camera 110 generates an image having a resolution lower than the photographable resolution, that is, for example, when the UHD camera generates a full HD image, the setting information receiving unit 132 The camera 110 may generate a low resolution image centering on the position of the cover 110 and provide the image to the image processing apparatus 130 according to the position of the cover of the received setting information. That is, the image captured through the camera 110 is not subjected to the image processing according to the setting information, but the image generated by the camera 110 is moved to acquire the region of interest image.

If there is an enlargement or reduction ratio set according to the setting information received by the setting information receiving unit 132, the image processing unit 133 processes the ROI image according to the enlargement or reduction ratio, Can be re-output through the control unit (134). More specifically, when the setting information receiving unit 132 receives the number of times the second footstep has been pressed, the image processing unit 133 enlarges the interest region image according to the number of times the second footstep has been pressed and information about the enlargement or reduction ratio The enlarged or reduced region of interest image is re-acquired on the basis of the reference point, and then the re-acquired region of interest image is output through the image output unit 134. FIG. For example, if the region of interest image is obtained as shown in FIG. 5A and the number of times the second footstep received through the setting information receiving unit 132 is pressed once, 5A is enlarged 1.5 times using the B-spline algorithm according to the enlargement ratio 1.5 times corresponding to one time, and the center-point is enlarged 1.5 times And then the region of interest image re-acquired based on the center point after being enlarged as shown in FIG. 5 (b) may be output again through the image output unit 134.

In one embodiment, when a continuous image photographed from the camera 110 is received in real time via the image receiving unit 131, the continuous image is output through the image output unit 134 in accordance with a user's setting or a command input by the user The image may be reproduced or stopped, the image may be captured (.jpg), or the image may be saved as a video capture (.avi file format).

Meanwhile, the image processing method according to an embodiment of the present invention can also be implemented as a computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a hard disk, a floppy disk, a removable storage device, a flash memory, Optical data storage devices, and the like.

In addition, the computer readable recording medium may be distributed and executed in a computer system connected to a computer communication network, and may be stored and executed as a code readable in a distributed manner.

Although the preferred embodiments of the image processing system and method capable of moving and zooming in the ROI according to the present invention have been described above, the present invention is not limited thereto. And the present invention also belongs to the present invention.

100: Image processing system
110: camera
120: foot switch
130: Image processing device
131:
132: setting information receiving section
133:
134: Video output unit
135:

Claims (15)

A camera for capturing an image;
A foot switch for setting an area of interest in an image photographed by the camera and setting an enlargement or reduction ratio of the image; And
A camera which receives an image from the camera and acquires a region of interest image of the image according to a region of interest set by the foot switch, and enlarges or reduces the region of interest image according to an enlargement or reduction ratio set by the foot switch, Processing system.
The camera according to claim 1,
An endoscopic camera for photographing a high-resolution image, or a camera fixed to a specific position and equipped with a lens having an auto focus function and a zoom function.
The foot switch according to claim 1,
A first scaffold for setting the region of interest and a second scaffold for setting an enlargement or reduction ratio,
Wherein a mark displayed in an image outputted through the image processing apparatus is moved by the first footstep and a region within a certain range from the position of the mark is set as the region of interest.
The image processing apparatus according to claim 1,
Wherein the image processing apparatus is provided with a continuous image photographed from the camera in real time and reproduces or stops the image according to a user's setting and captures the image or captures a moving image of the image.
The image processing apparatus according to claim 1,
Filtering an image corresponding to the set region of interest,
Wherein the image is converted to a gray scale, filtered with a specific reference color, and image noise is removed through a Gaussian filter.
6. The image processing apparatus according to claim 5,
Setting a center point of the ROI in the filtered image,
Wherein an object is captured in the filtered image and a vertex of a largest object among the captured objects is set as a center point.
7. The image processing apparatus according to claim 6,
Acquiring and outputting a region of interest image based on the center point,
When the enlargement or reduction ratio set by the foot switch is present, enlarging or reducing the region of interest image according to the enlargement or reduction ratio and re-outputting the enlarged or reduced region of interest image based on the center point Image processing system characterized by.
An image processing method performed in an image processing apparatus connected to a camera and a foot switch,
(a) receiving an image photographed by the camera;
(b) receiving setting information for setting a region of interest in an image photographed by the camera and setting an enlargement or reduction ratio of the image from the foot switch; And
(c) acquiring a region of interest image of the image according to the region of interest set on the basis of the setting information, and enlarging or reducing the region of interest image according to the set enlargement or reduction ratio.
9. The camera according to claim 8,
An endoscopic camera for photographing a high-resolution image, or a camera fixed to a specific position and equipped with a lens having an auto focus function and a zoom function.
9. The method of claim 8,
Wherein the foot switch has a first footrest for setting the area of interest and a second footrest for setting the enlargement or reduction ratio,
The step (c) may include displaying a mark in the image moving according to an operation of the user with respect to the first footstool; And
And setting the area of interest as a region of interest from a position of a mark displayed in the image.
9. The method of claim 8, wherein the step (c)
The method of claim 1, further comprising the steps of: (a) receiving a continuous image captured by the camera in real time; and (c) playing or stopping the image according to a user's setting and capturing the image or capturing a moving image of the image. / RTI >
9. The method of claim 8, wherein the step (c)
And filtering the image corresponding to the set region of interest, wherein the step of filtering the image includes:
Converting the image into grayscale;
Filtering the gray-scale-converted image with a specific reference color; And
And removing the image noise by applying a Gaussian filter to the image filtered with the specific reference color.
13. The method of claim 12, wherein step (c)
And setting a center point of the ROI in the filtered image, wherein the setting of the center point comprises:
Capturing an object in the filtered image; And
And setting a vertex of a largest object among the captured objects as a center point.
14. The method of claim 13, wherein step (c)
Acquiring and outputting a region of interest image based on the center point; And
Enlarging or reducing the ROI image according to the enlargement or reduction ratio when the enlargement or reduction ratio is set according to the setting information and outputting the enlarged or reduced ROI image based on the center point Wherein the image processing method comprises the steps of:
A computer-readable recording medium having recorded thereon a computer program for executing the method according to any one of claims 8 to 14.

Images