KR102203272B1 - Apparatus and method for processing object of interest - Google Patents

Abstract

The present invention relates to an apparatus and a method for processing an object of interest when displaying content.
An object-of-interest processing apparatus according to the present invention includes an image receiving unit that receives raw content, a memory storing a conversion program for the raw content, and a processor that executes the program, and the processor sets a region of interest in the raw content, and It is characterized in that the size conversion is performed according to the size conversion ratio, and the final image is synthesized and displayed.

Description

Object processing apparatus of interest and its method {APPARATUS AND METHOD FOR PROCESSING OBJECT OF INTEREST}

The present invention relates to an apparatus and a method for processing an object of interest when displaying content.

According to the prior art, when content is displayed on a screen display, if the aspect ratio of the original content is different from the aspect ratio of the screen display device, information to be displayed is distorted and displayed.

According to the prior art, by converting the size of the content through a simple proportional expression without considering the difference between the aspect ratio of the original content and the aspect ratio of the display device, the shape of the part corresponding to the area of interest inside the content is originally intended. Unlike this, it may be severely distorted, and there is a problem in that information transmission is difficult.

The present invention has been proposed to solve the above-described problem, and provides an apparatus and method for processing an object of interest capable of solving a distortion problem for an important area and highlighting an important area in displaying content on various screen display devices. It has its purpose.

An object-of-interest processing apparatus according to the present invention includes an image receiving unit that receives raw content, a memory storing a conversion program for the raw content, and a processor that executes the program, and the processor sets a region of interest in the raw content, and It is characterized in that the size conversion is performed according to the size conversion ratio, and the final image is synthesized and displayed.

The method of processing an object of interest according to the present invention includes the steps of receiving raw content, setting a region of interest in the original content, transforming the size of the original image as a whole, and performing size conversion according to the size conversion ratio for the region of interest. And synthesizing and displaying the final image by overlaying the ROI on which the size conversion has been performed on the overall size-converted image.

According to an embodiment of the present invention, when various screen display devices are used for performances, advertisements, exhibitions, and promotions, it is possible to reduce distortion of displayed information.

The effects of the present invention are not limited to those mentioned above, and other effects that are not mentioned will be clearly understood by those skilled in the art from the following description.

1 shows raw content.
FIG. 2 shows a screen in which the original content shown in FIG. 1 is displayed on a screen display device having a different aspect ratio.
3 shows a content display method according to the prior art.
4 is a block diagram illustrating an apparatus for processing an object of interest according to an embodiment of the present invention.
5 shows the result of maintaining an aspect ratio for an ROI according to an embodiment of the present invention.
6 shows an ROI setting according to an embodiment of the present invention.
7 shows an offset position and a horizontal and vertical length of a bounding box with respect to an ROI according to an embodiment of the present invention.
8 shows a final image display according to an embodiment of the present invention.
9 is a flowchart illustrating a method of processing an object of interest according to an embodiment of the present invention.
10 is a flowchart illustrating a method of processing an object of interest according to another embodiment of the present invention.
11 illustrates merging of regions of interest according to another embodiment of the present invention.
12 shows a partial area setting according to another embodiment of the present invention.
13 illustrates a bounding box area and a peripheral area according to another embodiment of the present invention.
14 is a diagram illustrating a determination of a size conversion ratio for a surrounding area according to another embodiment of the present invention.
15 illustrates an example of using a screen display device according to another embodiment of the present invention.

The above-described objects and other objects, advantages, and features of the present invention, and methods of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, and only the following embodiments are for the purpose of the invention to those of ordinary skill in the art, It is only provided to easily inform the composition and effect, and the scope of the present invention is defined by the description of the claims.

Meanwhile, terms used in the present specification are for explaining embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, actions and/or elements in which the recited component, step, operation and/or element is Or does not preclude addition.

Hereinafter, in order to help those skilled in the art understand, the background to which the present invention is proposed is first described, and then an embodiment of the present invention will be described.

, 세로길이는

라고 가정한다. Assuming that the original content as shown in FIG. 1 is given, the horizontal length of the original content is

, The vertical length is

Is assumed.

Here, as the unit of length, any unified length unit such as the number of pixels or defined in the metric system can be used, but it is not necessary to use it limited to a specific length unit, and it is a problem if the predetermined unit is consistently used. There is no

Nevertheless, in a very special display environment, the aspect ratio (aspect ratio) at which pixels are expressed may not be 1:1. In this case, raw contents and finally displayed through the display device Contents (displayed contents) are consistently converted into a predetermined length unit and processed.

Hereinafter, in describing a preferred embodiment of the present invention with reference to FIGS. 4 to 15, it is assumed that the aspect ratio of elements representing pixels in the screen display device is equal to 1:1, and the number of pixels The embodiment will be described without making any difference between the and length units.

However, it goes without saying that the object processing method proposed in the present invention can be easily applied even when the aspect ratio is not the same.

Referring back to FIG. 1, the letters “Kanada...” and “ABC...” are written, and it is preferable that such character information is expressed as less distortion as possible from the viewpoint of information transmission.

However, as illustrated in FIG. 2, when the original content is displayed on a screen display device having a different aspect ratio, the displayed content is stretched or shrunk in a specific direction to be displayed.

In order to more clearly describe the problem according to the prior art, it will be described using an equation.

라고 할 때, 도 2에 도시한 화면표시장치에 표출하기 위한 목표콘텐츠를

의 크기로 변환하여 모델링할 수 있다.Raw content in the form of a two-dimensional image as shown in FIG. 1

In the case of, target content to be displayed on the screen display device shown in FIG. 2

It can be modeled by converting it to the size of.

로써 임의 픽셀의 가로 위치를 나타내는 인덱스를 의미하고, y는

는 임의의 픽셀의 세로 위치를 나타내는 인덱스를 의미한다. Where x is

Means the index representing the horizontal position of an arbitrary pixel, and y is

Denotes an index indicating the vertical position of an arbitrary pixel.

Regarding the size conversion, according to the prior art, in order to fill the original content in the display device and display it, a function in the form of [Equation 1] below may be called and processed.

,

)인 입력된 영상

를 가로세로 크기 (

,

)로 변환 처리된 영상

를 생성한다는 의미를 포함하고 있다.The function of [Equation 1] has a size of (

,

)

The horizontal and vertical size (

,

) Converted to

It contains the meaning of creating.

In [Equation 1], the resize() function is expressed as a matrix expression as shown in [Equation 2] below.

를 출력 영상의 좌표위치

로 매핑한다. In other words, the coordinate position of the input image expressed as a vector

The coordinate position of the output image

Maps to

는 가로방향으로의 크기변환비율(resize ratio)을 나타내고,

는 세로방향으로의 크기변환비율을 나타낸다.In [Equation 2]

Represents the resize ratio in the horizontal direction,

Represents the ratio of size conversion in the vertical direction.

,

,

,

가 된다. Here, the coordinate index values representing the coordinates of the pixels are each

,

,

,

Becomes.

The expression of [Equation 2] represented by the multiplication of the matrix represents the mathematical concept for the resize function of [Equation 1] described above, and more complex processing is additionally required when implemented using a computer program.

For example, if forward mapping is performed through matrix operation during image size transformation or shape transformation, a decimal point may be entered in the coordinate index of the target image. In the process of converting this to an integer, the coordinates are not calculated in the target image. As a result, there may be a part where the image is not expressed.

To solve this problem, it is common to calculate and process in the reverse direction based on the final image.

In addition, an aliasing problem due to under sampling may occur in a change in image reduction, and additional processing such as low-pass filtering may be added to solve this problem.

These additional processes may be used with the embodiments of the present invention described below with reference to FIGS. 4 to 15.

,

이 되어 종횡비 1:1의 원시콘텐츠를 종횡비 2:1로 바뀌어 표출되도록 처리하게 된다. When [Equation 2] is interpreted from the point of view of FIG. 2, for example, assuming that an image of 10x10 pixels is displayed by converting the size linearly with respect to width and length on a display device of 20x10 size,

,

As a result, the original content with an aspect ratio of 1:1 is changed to an aspect ratio of 2:1 and processed to be displayed.

Accordingly, as shown in FIG. 2, as a result, a distorted image is viewed in which the horizontal is doubled compared to the vertical.

According to the prior art, in order to solve this problem, as shown in FIG. 3, when the aspect ratio of the screen display device is different from the original content, the original content is not forcibly stretched and the ratio is maintained and displayed on the screen. The method was also used.

In other words, this is an expression in which the original content is maximized and displayed on the screen without overflowing the screen display device.

However, in the conventional technology, as shown in FIG. 3, an empty space that does not display content may occur.

There may be cases where it is desirable to display content without empty space, and there is a need to maintain a predefined aspect ratio only for some important areas of interest within the content, and the present invention focuses on this point and proposes an object of interest. It provides a processing apparatus and a method thereof.

4 is a block diagram illustrating an apparatus for processing an object of interest according to an embodiment of the present invention.

An object of interest processing apparatus according to an embodiment of the present invention includes an image receiving unit 100 for receiving raw content, a memory 200 in which a conversion program for raw content is stored, and a processor 300 for executing the program. In step 300, a region of interest in the original content is set, size conversion according to a size conversion ratio is performed on the region of interest, and a final image is synthesized and displayed.

The processor 300 sets the region of interest as a bounding box using a user input or an object detection algorithm.

At this time, the processor 300 sets the region of interest through the offset position, the horizontal length, and the vertical length of the bounding box with respect to the region of interest, determines a position in which the region of interest is to be expressed within the target content, Size transformation is performed on the region of interest.

The processor 300 merges the regions of interest when common regions exist, applies a size conversion ratio for the merged bounding box region, and determines a size conversion ratio for the remaining regions other than the merged bounding box region. do.

The processor 300 may perform shape transformation on the ROI by using a parameter for homography transformation on the ROI.

According to an embodiment of the present invention, as shown in FIG. 5, while displaying content without empty space, the aspect ratio set according to the purpose is maintained or the size can be adjusted for the regions of interest set in a predetermined method. do.

As shown in FIG. 5, two separate regions of "Kanada..." and "ABC..." are set as regions of interest. Through a specific embodiment described below, the aspect ratio of the region of interest is It is possible to maintain or adjust.

Hereinafter, an embodiment in which an ROI is set and image processing is performed in a case where it is not relevant even if it is assumed that the shape of the original content finally expressed through the screen display device is rectangular.

가 있을 때, 관심 영역을 설정한다. 6, the input raw image

When there is, set the area of interest.

As shown in FIG. 6, three regions of interest set as a rectangular bounding box exist for the regions "Kanada...", "ABC...", and "123...".

For setting the regions of interest, the user can set each region using a predetermined input device such as a mouse or a keyboard.

Accordingly, according to an embodiment of the present invention, an input device capable of receiving a user input may be included.

In the example of FIG. 6, the text is set as the region of interest, but it is also possible to set an arbitrary picture area, more generally, an arbitrary bounding box area.

Even if the user does not explicitly set the region of interest, it can be performed using a predetermined image processing method or an object detection method using machine learning technology.

For example, in advance, “Finding a person's face area as a region of interest”, “Finding a person's whole body as a region of interest”, “Finding a vehicle object as a region of interest”, “Finding a building region as a region of interest”, “English alphabet region As the examples of “Find a text sentence as a region of interest”, “Find a Korean sentence region as a region of interest”, “Find a number as a region of interest”, and “Find a text sentence as a region of interest regardless of language”. Methods can be used to find areas of interest.

At this time, it is preferable to set each region of interest to be a bounding box having an arbitrary rectangular shape.

개라고 가정한다. The number of regions of interest separated from each other is

Suppose it is a dog.

According to an embodiment of the present invention, a region of interest setting unit using image processing or deep learning technology may be included.

에는 서로 구분되는

개의 관심 영역

들이 존재한다. As shown in Fig. 6, the input raw image

There are distinct

Areas of interest

They exist.

로써

번째 관심 영역을 의미하고, 각각의 관심 영역을 기술하는 파라미터가 존재하게 되는데,

번째 관심 영역은

,

,

,

라는 4개의 파라미터로 나타낼 수 있다.

As

It means the second region of interest, and there is a parameter describing each region of interest.

Area of interest

,

,

,

It can be represented by four parameters:

Referring to FIG. 7, an offset position of a bounding box with respect to an ROI and a length of each of the bounding boxes are shown.

8 shows a shape in which regions of interest set through the above-described process are finally expressed through a screen display device.

번째 관심 영역은

,

,

,

라는 4개의 수치로 나타낼 수 있다.in this case

Area of interest

,

,

,

It can be represented by four numbers.

번째 관심 영역에 대해서 {

,

,

,

}과 {

,

,

,

}사이의 관계를 결정하여야 한다. random

For the first area of interest {

,

,

,

} And {

,

,

,

} The relationship between them must be determined.

In order to express this relationship, [Equation 1] and [Equation 2] described above are reorganized for each region of interest as shown in [Equation 3] and [Equation 4] below.

,

의 관계를 가지고 있다. In [Equation 3] and [Equation 4],

,

Have a relationship.

와

가 종속변수가 아니라 제어에 활용하는 독립변수라고 가정하면,

,

로 나타낼 수 있다.

Wow

Assuming that is not a dependent variable but an independent variable used for control,

,

It can be expressed as

According to an embodiment of the present invention, it is technically characterized in that the original content is size-converted and displayed on a screen display device, and the corresponding area is displayed by receiving size conversion ratio information for regions of interest independently.

9 is a flowchart illustrating a method of processing an object of interest according to an embodiment of the present invention.

An object of interest processing method according to an embodiment of the present invention includes receiving raw content, setting an ROI in the original content (S910), transforming the size of the original image as a whole, and performing a size conversion ratio for the ROI. And performing size conversion (S920 to S940) and synthesizing and displaying a final image by overlaying a region of interest on which the size conversion has been performed on the overall size-converted image (S950).

According to an embodiment of the present invention, if there are common regions between regions of interest, they are merged, a size conversion ratio is applied to the merged bounding box region, and size conversion is performed on the remaining regions other than the merged bounding box region. Determine the ratio.

Hereinafter, a method of processing an object of interest according to an embodiment of the present invention will be described in more detail.

라고 가정하면, 크기는

,

이다.(i-1) the raw content to be expressed

Assuming that the size is

,

to be.

,

라고 가정한다.(i-2) The size of the target content to be converted corresponding to the display device is

,

Is assumed.

,

대신에 전체적인 크기를 변환하기 위해 필요한 가로 및 세로의 크기변환비율

와

를 명시적으로 입력 받는 것이 가능하다. At this time

,

Instead, the ratio of horizontal and vertical size conversion required to convert the overall size

Wow

It is possible to explicitly receive input.

에 대해서

개의 관심 영역을 설정한다.(i-3) raw content

about

Set the area of interest for the dog.

개의 관심 영역들 중에서

번째 관심 영역 정보는 {

,

,

,

},

로 표현할 수 있다. Set

Among the areas of interest

The first region of interest information is {

,

,

,

},

It can be expressed as

As described above, the region of interest may be explicitly set by the user through a user interface, or may be automatically set using image processing or machine learning technology.

번째 관심 영역의 가로 및 세로 크기변환

과

를 입력 받는다. (i-4)

Horizontal and vertical scaling of the first region of interest

and

Is entered.

If there is no input size conversion ratio, the size conversion ratio is determined according to a predetermined value or a predetermined calculation procedure.

=

=min(

,

) 또는

=

=max(

,

) 또는

=

=function(

,

) 과 같은 계산절차 방식을 사용할 수 있다. for example,

=

=min(

,

) or

=

=max(

,

) or

=

=function(

,

) Can be used.

Here, min() is a function that calculates the minimum value among the input values, max() is a function that calculates the maximum value, and function() is an arbitrary function.

를

= resize(

,

,

,

,

)와 같이 크기변환한다. (i-5)

To

= resize(

,

,

,

,

) And size conversion.

= resize(

,

,

,

,

)에 따라 크기변환을 수행할 수도 있다. At this time, as described above,

= resize(

,

,

,

,

), you can also perform size conversion.

,

,

,

}를 매개변수로 사용하면 크기변환비율을 입력 크기와 출력 크기의 비로써 계산한다는 묵시적 의미가 담겨있는데 반하여, {

,

,

,

}를 매개변수로 사용한다면 명시적으로 가로와 세로의 크기변환비율을 입력해준다는 것이다. Explaining the difference, the {

,

,

,

} Is used as a parameter to contain the implicit meaning that the size conversion ratio is calculated as the ratio of the input size and the output size, whereas {

,

,

,

If} is used as a parameter, the ratio of the horizontal and vertical size conversion is explicitly entered.

번째 관심 영역이 표출될 위치 {

,

}를 결정하며, 이는 아래 [수학식 5]와 같이 표현할 수 있다. (i-6)

The location where the first region of interest will be expressed {

,

} Is determined, which can be expressed as [Equation 5] below.

For the (i-7)-th region of interest, size conversion is performed according to the given size conversion ratio.

(i-8) On the result image output after size conversion in the above-described (i-5) process, the N regions of interest performed by the size conversion in (i-7) are overlaid on the position determined in (i-6) for final expression. The video to be played is synthesized and displayed on the screen display device.

According to the above-described embodiment, there occurs a case in which regions of interest and regions other than the region of interest may not be connected continuously.

In another embodiment of the present invention, a method is disclosed that is similar to the above-described embodiment, but allows regions of interest and regions other than the region of interest to be successively connected as constraints are strengthened.

10 is a flowchart illustrating a method of processing an object of interest according to another embodiment of the present invention.

According to another embodiment of the present invention, a region of interest is set in the original content (S1010), regions of interest that are common to each other are merged (S1020), a partial region for regions of interest and non-interest regions is set, and A location and a size conversion ratio are determined for each area (S1030), and a final image is synthesized by resizing each of the partial areas and displayed on the screen display device (S1040).

In the process of a method of processing an object of interest according to another embodiment of the present invention, (ii-1) to (ii-3) are the same as those of (i-1) to (i-3) described above.

(ii-4) If a common area exists between the regions of interest in the horizontal or vertical direction, it is merged to determine one bounding box.

Referring to FIG. 11, an example of merging a plurality of regions of interest when they are set is shown.

In order to help those skilled in the art understand, it is assumed that 6 regions of interest are initially set.

If there is an area common to each other in a horizontal direction or a vertical direction between any two areas of interest for the set areas of interest, they are merged into one bounding box.

The bounding box that has already been merged is merged again if there is a common portion in the horizontal and vertical directions with the remaining regions of interest that have not been merged.

That is, according to the example shown in FIG. 11, the "region of interest 1" and the "region of interest 2" have a common region in the vertical direction and are merged into one bounding box.

It is preferable to determine the merged bounding box as a bounding box having a minimum size including two regions of interest.

The bounding box including “region of interest 1” and “region of interest 2” no longer exists in common with other regions of interest, and thus becomes an independent bounding box.

Next, looking for a common region of interest for the “region of interest 3”, the “region of interest 5” and the “region of interest 6” are merged into one bounding box because a common region exists in the horizontal direction.

In the merged new bounding box, a common area occurs vertically with “area of interest 4”, and “area of interest 4” is also merged.

As a result, in the example shown in FIG. 11, two independent bounding boxes are generated.

As described above, when multiple regions of interest are set through the process of merging regions of interest, there is an effect of removing the problem that the content is very different from the original content, and contents are mutually interposed between the region of interest and the regions other than the region of interest. It can provide a function to be connected continuously.

(ii-5) Set partial areas for areas of interest and non-interest areas.

The partial regions are created by extending each corner of each bounding box to the horizontal and vertical axes of the image.

Referring to FIG. 12, this is an example of setting partial regions in the example shown in FIG. 11.

That is, referring to FIG. 12, a total of 25 partial regions are generated, and regions 7 and 19 correspond to the two merged bounding boxes in FIG. 11.

개라고 한다면

번째 바운딩 영역의 가로 및 세로 크기변환비율

과

를 설정파일이나 사용자로부터 입력 받는다.(ii-6) The number of bounding boxes newly created by merging

A dog

Horizontal and vertical size conversion ratio of the second bounding area

and

Is input from a configuration file or user.

이고, 만약 별도로 입력된 크기변환비율이 없다면 사전에 내정된 값(default value) 또는 소정의 계산절차에 따라 크기변환비율을 결정한다. here

And, if there is no separately inputted size conversion ratio, the size conversion ratio is determined according to a preset value or a predetermined calculation procedure.

=

=min(

,

) 또는

=

=max(

,

) 또는

=

=function(

,

) 과 같은 계산절차 방식을 사용할 수 있다. for example,

=

=min(

,

) or

=

=max(

,

) or

=

=function(

,

) Can be used.

Here, min() is a function that calculates the minimum value among the input values, max() is a function that calculates the maximum value, and function() is an arbitrary function.

However, in this process, it is desirable to impose a constraint that prevents overlapping or affecting each bounding box when the size is changed.

(ii-7) The size conversion ratio is also determined for the remaining areas other than the merged bounding box area.

Through the above-described process, as shown in FIG. 12, since the horizontal and vertical size conversion ratios for regions 7 and 19 are determined, the remaining regions are preferably determined by reflecting them.

와

이라고 가정한다면 {2, 12, 17, 22}번 영역의 가로 크기변환비율은

이어야 하고, {6, 8, 9, 10}번 영역의 세로 크기변환비율은

가 되도록 변환되어야 한다. For example, the horizontal and vertical ratio of area 7 of

Wow

Assuming that, the horizontal size conversion ratio of the {2, 12, 17, 22} area is

Should be, and the vertical size conversion ratio of the {6, 8, 9, 10} area is

Should be converted to

와

라고 가정한다면, {4, 9, 14, 24}번 영역의 가로 크기변환비율은

이고, {16, 17, 18, 20}번 영역의 세로 크기변환비율은

가 되도록 변환되어야 한다. Likewise, the horizontal and vertical size conversion ratio of area 19

Wow

Assuming that, the horizontal size conversion ratio of the {4, 9, 14, 24} area is

And the vertical size conversion ratio of the {16, 17, 18, 20} area is

Should be converted to

For each area, there are parts whose ratio is determined and parts that are not.

As a result, areas {7, 9, 17, 19} have both the horizontal and vertical size conversion ratios determined.

For areas {2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24}, only one horizontal or vertical size conversion ratio is determined.

For areas {1, 3, 5, 11, 13, 15, 21, 23, 25}, the size conversion ratio is not determined at all.

As described above, there exist areas for which the size conversion ratio is not determined, and according to an embodiment of the present invention, the size conversion ratio thereof is determined.

FIG. 13 illustrates a bounding box area and a peripheral area according to another embodiment of the present invention, and FIG. 13 illustrates a case where there is only one bounding box area including regions of interest to aid understanding of those skilled in the art. .

The left drawing of FIG. 13 is an example in which one bounding box area exists.

The right diagram of FIG. 13 is an example of dividing this into partial regions, and region 5 corresponds to a bounding box region in which regions of interest are merged.

와

이라고 가정한다. Adjust the horizontal and vertical size conversion ratio of area 5 respectively

Wow

Is assumed.

으로 동일하게 결정된다. Considering the horizontal conversion ratio, the aspect ratio of 2, 5, 8 is

Is determined in the same way.

On the other hand, {1, 4, 7} and {3, 6, 9} were not determined. A method of determining this will be described with reference to FIG. 14.

14 shows the relationship of size conversion in the horizontal direction.

=

이다. As shown in the left figure of FIG. 14, if the size conversion is performed in a manner according to the prior art without special processing, it is expressed as a linear mapping function in the form of a straight line, and the slope is

=

to be.

개의 바운딩 박스가 결정되면,

번째 크기변환비율이 결정되며, 이는 도 14의 우측 도면에 도시한 바와 같다. On the other hand, according to an embodiment of the present invention, a region of interest is set and merged

When two bounding boxes are determined,

The th size conversion ratio is determined, as shown in the right figure of FIG. 14.

번째 바운딩 박스는 가로축에서

부터

+

까지 영역을 차지하게 된다.

The first bounding box is on the horizontal axis

from

+

To occupy the area.

+

가 된다. So the middle position of this area is

+

Becomes.

+

가 된다. Therefore, the position on the vertical axis corresponding to the middle position of the bounding box on the horizontal axis is

+

Becomes.

번째 바운딩 영역의 가로 크기변환비율이

로 이미 주어졌으므로, 도 14의 우측 도면의 2차원 좌표계에서의 점 B(

+

,

+

)를 지나고 기울기가

인 직선을 그린다. In the above (ii-6) process

The horizontal size conversion ratio of the second bounding area is

Since it has already been given as, point B (

+

,

+

) And the slope is

Draw a straight line.

, 0)을 지나는 수직선과 만나는 점 A(

,

)가 되고, 이 직선이 세로축 점(

+

, 0)을 지나는 수직선과 만나는 점은 C(

+

,

+

)가 된다. Then this line is the horizontal axis point (

, Point A(

,

), and this line is the vertical axis point (

+

, The point where it meets the vertical line passing through 0) is C(

+

,

+

).

=

로 결정할 수 있고, {3, 6, 9}번 영역들의 가로 크기변환비율은

=

로 결정할 수 있다.Accordingly, the horizontal size conversion ratio of the areas {1, 4, 7} in FIG. 13 is as shown in the right figure of FIG. 14,

=

Can be determined as, and the horizontal size conversion ratio of areas {3, 6, 9} is

=

Can be determined by

By applying this process to the vertical axis as well, it is possible to determine the horizontal and vertical size conversion ratios for all partial regions.

According to an embodiment of the present invention, even when there are several bounding boxes, the size conversion ratio of the undecided partial regions can be automatically determined by first setting a given size conversion ratio for the bounding box and obtaining each intersection point through this. .

(ii-7) Each image of the partial regions is size-converted at the determined size conversion ratio of the partial regions, and the images of the partial regions are combined into one and displayed on the screen display.

In the above-described embodiment, a method for processing the aspect ratio of the region of interest to a value set in advance is described in the case where it is safe to assume that the displayed screen display device has a rectangular shape.

Hereinafter, as another embodiment of the present invention, a method of setting a region of interest and an image processing method when raw content is transformed into a polygonal shape having four arbitrary vertices through a screen display device will be described.

According to another embodiment of the present invention, as shown in FIG. 15, when a screen display device such as a projector is used, even when the shape of the final displayed image is distorted such as homography conversion, the region of interest is Carry out processing.

In yet another embodiment of the present invention, parameters for homography conversion are provided.

Hereinafter, a process according to another embodiment of the present invention will be described.

라고 가정하며, 그 크기는

,

이다.(iii-1) the raw content to be displayed

And the size is

,

to be.

라고 주어졌다고 가정한다.(iii-2) A matrix for homography conversion to be displayed on the display device to be displayed

Is given.

에 대해서

개의 관심 영역을 설정한다. (iii-3) Raw content

about

Set the area of interest for the dog.

개의 관심 영역들 중에서

번째 관심 영역 정보는 {

,

,

,

},

로 표현할 수 있다. Set

Among the areas of interest

The first region of interest information is {

,

,

,

},

It can be expressed as

As described above, the region of interest may be explicitly set by the user through a user interface, or may be automatically set using image processing or machine learning technology.

번째 관심 영역에 대한 호모그래피 변환행렬

의 계수값들을 입력 받는다. (iii-4)

Homography transformation matrix for the first region of interest

The coefficient values of are input.

The homography transformation matrix serves to transform the shape of the input image into the output image as shown in [Equation 6] below.

If there is no input homography conversion matrix, it is determined according to a preset value or a predetermined calculation procedure.

를 (iii-2)에서 주어진 호모그래피 변환행렬

를 적용하여 모양변환시킨다. (iii-5) The input content video

The homography transformation matrix given in (iii-2)

Apply to transform the shape.

번째 관심 영역이 표출될 위치 {

,

}를 결정하며, 이는 아래 [수학식 7]과 같이 표현할 수 있다. (iii-6)

The location where the first region of interest will be expressed {

,

} Is determined, which can be expressed as [Equation 7] below.

개의 관심 영역들에 대해서 주어진 호모그래피 변환행렬

,

를 이용하여 각각 모양변환을 수행한다.(iii-7)

Homography transformation matrix given for two regions of interest

,

Each shape transformation is performed using.

(iii-8) In the result image output after shape conversion in step (iii-5) above, the N ROI images converted in shape in (iii-7) are located at the location set in (iii-6). The image to be displayed is synthesized by overlaying it and displayed on the screen display device.

Meanwhile, the method of processing an object of interest according to an embodiment of the present invention may be implemented in a computer system or recorded on a recording medium. The computer system may include at least one processor, memory, user input device, data communication bus, user output device, and storage. Each of the above-described components communicates data through a data communication bus.

The computer system may further include a network interface coupled to the network. The processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in a memory and/or storage.

The memory and storage may include various types of volatile or nonvolatile storage media. For example, the memory may include ROM and RAM.

Accordingly, the method of processing an object of interest according to an embodiment of the present invention may be implemented in a computer-executable method. When the method of processing an object of interest according to an embodiment of the present invention is performed in a computer device, computer-readable instructions may perform the processing method according to the present invention.

Meanwhile, the method of processing an object of interest according to the present invention may be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording media in which data that can be decoded by a computer system is stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like. In addition, the computer-readable recording medium can be distributed to a computer system connected through a computer communication network, and stored and executed as code that can be read in a distributed manner.

So far, it looked at the center of the embodiments of the present invention. Those of ordinary skill in the art to which the present invention pertains will be able to understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

100: image receiving unit 200: memory
300: processor

Claims (7)

An image receiving unit for receiving raw content;
A memory storing a conversion program for the original content; And
Including a processor for executing the program,
The processor sets a region of interest in the original content, performs size conversion according to a size conversion ratio for the region of interest, synthesizes and displays a final image,
The processor sets the region of interest as a bounding box using a user input or object detection algorithm, merges the regions of interest when common regions exist in the horizontal or vertical direction, and merges the regions of the merged bounding box. Applying a conversion ratio and determining a size conversion ratio for the remaining areas other than the merged bounding box area
Object of interest processing device.
delete The method of claim 1,
The processor sets a region of interest through an offset position, a horizontal length, and a vertical length of the bounding box with respect to the region of interest, determines a position in which the region of interest is to be expressed within the target content, and applies to the region of interest according to a size conversion ratio. Performing a size conversion for
Object of interest processing device.
delete The method of claim 1,
The processor performs shape transformation on the region of interest by using a parameter for homography transformation on the region of interest.
Object of interest processing device.
(a) receiving the original content and setting a region of interest in the original content;
(b) performing a size conversion on the original image as a whole and performing size conversion on the ROI according to a size conversion ratio; And
(c) synthesizing and displaying a final image by overlaying the region of interest on which the size conversion has been performed on the overall size-converted image,
In the step (a), the region of interest is set as a bounding box using a user input or an object detection algorithm,
In the step (b), if there are common regions between regions of interest in the horizontal or vertical direction, they are merged, a size conversion ratio is applied to the merged bounding box region, and the remaining regions other than the merged bounding box region To determine the size conversion ratio for
How to handle objects of interest. delete

Images