KR101169786B1 - Video carving method and apparatus thereof - Google Patents

Abstract

The present invention relates to a method and apparatus for resizing a video based on content, and more particularly, to extracting a seam by dividing a seam extraction method based on whether a current frame is changed in a video resizing method using a seam carving method. By using independent seam extraction method only when changing scenes, the amount of computation is reduced and processing speed is increased. When scene change does not occur, the seam of the current frame is extracted by using the relevant information of seams extracted from the previous frame. Simultaneous frames in the same scene with similar distribution can be extracted from the current frame without the information of the entire frame of the scene, and can be applied to a video provided in the real-time streaming form. Without shim It is effective in enabling the normal playback of the video scaling to prevent deliberate redundancy coordinates by extracting plant material of the frame.

Description

Content-based video resizing method and device {VIDEO CARVING METHOD AND APPARATUS THEREOF}

The present invention relates to a method and apparatus for resizing a video based on content, and in particular, an algorithm for detecting a seam of a current frame using seam related information detected in a previous frame in a video resizing method using a seam carving method. The present invention relates to a content-based video resizing method and apparatus that can be applied to a video of a real-time streaming method.

Recently, as the utilization of wireless communication and mobile devices increases, the frequency of transmitting and receiving various images through a communication network or playing videos on a mobile device is gradually increasing. As a result, video has become an important information medium.

Since the screen size of the mobile device is limited, in order to use various images more effectively in the corresponding device, it is very important to resize and play the video. For example, a thumbnail technology for reducing the size of an image to simultaneously search a large number of images and simultaneously displaying a plurality of pictures on the screen is one example.

There are various ways to adjust the size of the image, but one of the most recent spotlights is to adjust the size of the image based on its importance based on the content of the image so that the original image can be adjusted in any direction. It is an image resizing method based on the video contents that can contain all the image information of the original image without significantly distorting the original ratio.

Among the content-based image resizing methods, a high performance recognition method based on a still image is a seam carving method, and a deep carbing method analyzes the contents of an original image and is determined to be insignificant. Image resizing is made by increasing or decreasing the size of images by copying or removing them.

That is, the simcarving method obtains energy values of each pixel defined by a preset method, and uses the correlation that the lower importance portion of the original image has a lower energy value. Adjust the size of the image by deleting or duplicating the small value.

In this case, when the size is deleted by deleting or duplicating the pixels in a general manner, the final image is distorted. Therefore, in order to prevent this, the energy of the horizontal and vertical directions of the image is obtained, and the line where the sum of the connected energy values in each start coordinate is the minimum is found and the seam is adjacent to the neighboring pixels in the horizontal or vertical direction. It is used as a candidate). The line having the smallest sum among these seam candidates is selected as the final seam, and the image size is adjusted by duplicating or removing pixels of the seam.

Using the sim carding method, there is a loss of information of certain parts of the original image, but there is an advantage that the size of the image can be naturally adjusted by reducing the data of the uninterested region such as the background region while maintaining the content as a whole.

This simcarving method can be applied to a video to adjust the size of the video. FIG. 1 is an exemplary view of a video using a deep carving method, and video carving uses a 3D cube 1 connecting frames 3, 4, and 5 on a time axis 6.

When adjusting the size of a video, unlike the still image, the energy correlation of the energy in the frame and the continuous frames is used for the entire frame included in the video. Therefore, in order to find the optimal seam set, which is an accurate and low loss seam set, not only the energy value of the current frame but also the energy of each position in subsequent frames is important. Find the lowest seam set (including 2: 11,12,13).

This optimal set of shims has an optimal value in spatial and temporal coordinates, and if a plurality of shims is obtained, it is necessary to repeatedly calculate the number of shims.

As an example of a method of applying seam carving to a video, there is a method of extracting an optimal set of seams by searching all frames belonging to a scene. However, since this method must have the entire video in advance, there is no problem in converting the embedded video to the desired size by sim carving and playing it in advance, but there is a problem in applying it to a real-time broadcasting video.

In addition, in order to obtain an optimal shim set from a video, an optimal shim is obtained from a frame at which a scene starts, and then the number of all cases having connectivity in a subsequent frame must be recalculated. This has a problem that the computational complexity increases exponentially with the number of shims increasing by the number of cases.

In addition, in order to be an optimal shim set in a video, it must have both simplicity (monotonic) and connected (connected). In other words, in the absence of simplicity, important image information may be damaged. If the connectivity is not guaranteed, a plurality of shims in each spatial coordinate may exist independently of each other, and thus, using such seams that are not guaranteed to be connected, There is a problem that severe jitter occurs when playing.

An object of an embodiment of the present invention for improving the above-described problem is to extract the seam by dividing the seam extraction method based on whether or not the scene transition of the current frame to use an independent seam extraction method only when the scene change And to provide an apparatus.

Another object of the embodiment of the present invention for improving the above-described problem is to extract the seam of the current frame using the seam information extracted from the previous frame when the scene change does not occur, the scene for the continuous frame in the same scene with similar energy distribution The present invention provides a method and apparatus for resizing a content-based video that extracts a seam of a current frame without information on the entire frame.

Another object of an embodiment of the present invention for improving the above-described problem is to adjust the content-based video size to prevent the overlap of the seam coordinates by extracting the seam of the current frame with the exception of the seam already extracted by referring to the order of seams extracted from the previous frame And to provide an apparatus.

Another object of the present invention for improving the above-described problem is to extract simplicity by extracting the energy value and the cumulative energy value of each pixel in the extraction of the seam of the current frame in the scene by comparing the energy value and the cumulative energy value of the seam of the previous frame. To provide a content-based video resizing method and apparatus to satisfy.

Another object of the present invention for improving the above-described problem is to extract the seam of the current frame in the scene in consideration of the spatial connectivity of each pixel and the temporal connectivity with the seam of the previous frame to extract the content-based content A method and apparatus for resizing a video are provided.

Another object of the present invention for improving the above-described problem is to synchronize the coordinates of the video and the seam map by inserting or deleting the coordinates of the seam map corresponding to the pixels of the image to be copied or removed when the video is resized One aspect of the present invention is to provide a method and an apparatus for resizing video.

Content-based video resizing method according to an embodiment of the present invention for achieving the above object comprises the steps of: a) detecting whether or not to change the scene of the current frame; b) extracting a seam of a current frame corresponding to the seam using information related to seams extracted from a previous frame if the current frame is not a scene change frame in step a); c) storing the shim related information of the current frame extracted in step b); And d) adjusting the size of the current frame using the seams extracted in step b).

The step b) further includes extracting the seam of the current frame without using the seam-related information extracted from the previous frame if the current frame is a scene change frame in step a).

Step c) stores the order of seams extracted from the current frame, and step b) repeats the extraction of seams corresponding to the seams extracted from the previous frame with the number of seam orders except for the seams already extracted.

Step c) stores the coordinates of the pixels included in the seam extracted from the current frame, and step b) starts the coordinates within a predetermined range of the same line based on the coordinates of the start pixel of the seam extracted from the corresponding previous frame. Extract the seam with the coordinates of.

The step b) further includes e) selecting one or more seam candidates having one of the coordinates within the predetermined range as starting pixel coordinates to extract the seams of the current frame.

In step e), the pixels constituting the shim candidate are adjacent to the pixels of the adjacent line, and have pixel conditions that the coordinates of the corresponding pixel and the coordinates of the corresponding line in the same line of the shim of the corresponding previous frame are adjacent or the same.

In step c), the energy value of the pixel included in the seam extracted from the current frame is stored. In step e), the energy value of the pixels satisfying the pixel condition is calculated to correspond to the same line of the seam of the previous frame. In FIG. 5, pixels having a minimum difference from an energy value of a pixel corresponding to the pixel are selected as the shim candidates as pixels constituting the shim candidate.

In step c), a cumulative value of energy values of pixels included in a seam extracted from a current frame is stored, and in step b), a cumulative value of a previous frame corresponding to the cumulative value of energy values of pixels among the one or more seam candidates is determined. A seam candidate having a minimum difference from an accumulated value of energy values of pixels is extracted as a seam of the current frame corresponding to the seam of the previous frame.

In step e), the pixels constituting the shim candidate are within a predetermined range with the pixels of the adjacent line, and the pixel condition is that the coordinates of the corresponding pixel and the coordinates are within the predetermined range in the same line of the shim of the corresponding previous frame. Have

The step d) may include generating a seam map corresponding to the video; And inserting or deleting a pixel of the seam map corresponding to a pixel that is copied or removed when the video is resized for synchronization with coordinates.

In addition, a content-based video resizing apparatus according to an embodiment of the present invention for achieving the above object includes a scene change detection unit for detecting whether the current frame scene change; If the current frame is not a transition frame, the seam of the current frame corresponding to the seam extracted from the previous frame is extracted. If the current frame is a transition frame, the seam of the current frame is extracted without using the seam information of the previous frame. A seam extractor for extracting; A shim information storage unit for storing shim related information of the extracted current frame; And a resizing unit for adjusting the size of the current frame by using the extracted shim.

The shim information storage unit stores an order of shims extracted from the current frame,

The shim extractor repeats extracting the shims corresponding to the shims extracted from the previous frame in a state other than the pre-extracted shims by the number of shims.

Content-based video resizing method and apparatus according to an embodiment of the present invention is to extract the seam by dividing the seam extraction method based on whether the current frame scene change or not, so that the independent seam extraction method is used only when the scene change This reduces the amount of computation and improves the processing speed.

Content-based video resizing method and apparatus according to an embodiment of the present invention extracts the seam of the current frame by using the seam-related information extracted from the previous frame when no scene change occurs in the continuous frame in the same scene with similar energy distribution By extracting the seam of the current frame without the information of the entire frame of the scene, there is an effect that can be applied to the video provided in the real-time streaming form.

Content-based video resizing method and apparatus according to an embodiment of the present invention to extract the seam of the current frame in the state except the pre-extracted seam with reference to the order of seams extracted from the previous frame to prevent overlapping coordinates of the seam It has the effect of enabling normal reproduction.

Content-based video resizing method and apparatus according to an embodiment of the present invention by extracting the energy value and the cumulative energy value of each pixel in the extraction of the seam of the current frame in the scene by comparing the energy value and cumulative energy value of the seam of the previous frame The simplicity is satisfied to enable smooth playback of the resized video.

Content-based video resizing method and apparatus according to an embodiment of the present invention to extract the seam of the current frame in the scene in consideration of the spatial connectivity of each pixel and the temporal connectivity with the seam of the previous frame to extract the size to satisfy the connectivity It is effective to prevent the screen shaking of the adjusted video.

Content-based video resizing method and apparatus according to an embodiment of the present invention by inserting or deleting the coordinates of the seam map corresponding to the pixel of the image to be copied or removed when the size of the video is adjusted to adjust the coordinates of the video and seam map By synchronizing, it is easy to resize the video.

1 is an exemplary diagram of a video to which a video carving method is applied.
2 is a flow chart of a content-based video scaling method according to an embodiment of the present invention.
3 is an exemplary diagram of seam related information of a previous frame according to an embodiment of the present invention.
4 is a flow chart of a seam extraction step according to an embodiment of the present invention.
5 is an exemplary view of a shim candidate according to an embodiment of the present invention.
6 is an exemplary diagram of pixel selection of a shim candidate according to an embodiment of the present invention.
7 is an illustration of shim candidate selection according to an embodiment of the present invention.
8 is an exemplary view of shim extraction according to an embodiment of the present invention.
9 is an exemplary diagram of frame size adjustment according to an embodiment of the present invention.
10 is a flowchart illustrating frame size adjustment according to an embodiment of the present invention.
11 is a block diagram of a video size control device according to an embodiment of the present invention.
12 is a block diagram of a heart extraction unit according to an embodiment of the present invention.

The present invention as described above will be described in detail with reference to the accompanying drawings and embodiments.

2 is a flowchart of a content-based video resizing method according to an embodiment of the present invention, the content-based video resizing method according to an embodiment of the present invention step (S110), the current frame information is input; Detecting whether a frame has a scene change (S120), and if the current frame is not a scene change frame in the scene change detection step (S120), by using the relevant information of a seam extracted from a previous frame, Extracting a seam (S130), storing the seam-related information of the current frame extracted in the seam extraction step (S130) (S150) and the seam extracted in the seam extraction step (S130) of the current frame Adjusting the size (S160).

The seam extraction step S130 further includes extracting a seam of the current frame without using seam-related information extracted from a previous frame if the current frame is a scene change frame in the scene change detection step S120.

The scene change detection step (S120) receives a frame of a video as an input and determines whether a new scene has started through scene change detection. In this case, the scene change may be detected using various methods, and thus its detailed description will be omitted.

When the new scene starts, the shim extraction step ignores the shim related information in the previous scene and extracts the shim through a seam carving method in the still image (S140). On the other hand, if no scene change occurs, that is, belonging to the same scene as the previous frame, the seam is searched for in the current frame using the seam related information of the previous frame (S130). Detailed description of the shim related information will be given below with reference to FIG. 3.

After the above step, after updating the information on the searched seam, the size of the image is adjusted (S160).

In this way, since the seam extraction method is used to extract seams based on whether the current frame is switched in the seam extraction step (S130, S140), an independent seam extraction method is used only when the scene changes, so that the target video is compared with the existing method. You can reduce the amount of computation and increase the processing speed when resizing.

In addition, if no scene transition occurs, the seam of the current frame is extracted using the seam information extracted from the previous frame, and the seam of the current frame is extracted without information of the entire frame of the scene for successive frames in the same scene having similar energy distribution. It can also be applied to videos provided in the form of real-time streaming.

3 is a diagram illustrating seam related information of a previous frame according to an exemplary embodiment of the present invention, and illustrates seam related information of a previous frame used for seam extraction in a frame in which no scene change occurs.

If a scene change occurs, the first frame of the scene is extracted using a seam carving method of a general still image. At this time, the coordinates of the extracted seam and the energy value obtained for each coordinate corresponding to the pixel included in the seam are stored and used to extract the seam in the next frame. As shown as the shim related information of the previous frame, the order of seams (1, 2, 3, 4), the pixel coordinates of the seams, the energy value per coordinate (energy value of pixels) and the total cumulative energy value of the seams in the order of seam extraction. (A cumulative value of pixel energy values included in the shim) is stored.

Extracting the seam from the current frame using the seam related information of the previous frame will be described in detail with reference to FIG. 4.

4 is a flowchart of a seam extraction step according to an embodiment of the present invention. When a scene change does not occur and the current frame belongs to the same scene as the previous frame, the seam of the current frame is extracted using the seam-related information stored in the previous frame. The steps are shown.

In the seam extraction step, the seam extracted from the previous frame is an optimal seam in the frame, and two consecutive frames in the same scene, that is, the previous frame and the current frame have similarities. Similarity of images means that the energy distribution is similar, which means that the depths of the two images are extracted in a similar form. Accordingly, the seam of the current frame may be extracted within a predetermined range from the coordinates of the seam extracted from the previous frame.

If the seams are extracted independently every frame without this process, jitter, which is a phenomenon in which the main content is shaken, occurs because the number of seams determined around the content varies from frame to frame.

For example, suppose that three shims were extracted to the left of the content and five shims to the right from the previous frame, and five shims were extracted to the left of the content and three shims to the right from the current frame. The position moves two pixels between frames.

In a video image, it is important to find an optimal shim and to avoid shaking, and by using a seam extraction step according to an exemplary embodiment of the present invention, a shim can be found to easily adjust the size of an image while preventing a shake.

In the shim extraction step, the seam-related information is extracted using a sequence of shims in a previous frame stored in the step of storing the shim-related information (S131), except for seams that have been previously extracted in the order of shims extracted from the previous frame. The number of steps is repeated (S131 to S138).

As described above, the reason for proceeding in order using the order of shims is to remove the first shim and then extract the next shim in order to eliminate the redundancy of coordinates. Therefore, the order of the shims in the current frame is also the same as the order of the shims in the previous frame by updating with reference to the order of the shims calculated in the previous frame.

If the seam is extracted while ignoring the redundancy of the coordinates, when the size of the corresponding frame is actually adjusted, one pixel may be insufficient at the overlapping portion of the coordinates, resulting in the entire image being distorted.

As described above, the content-based video resizing method according to an embodiment of the present invention extracts the shim of the current frame with the exception of the pre-extracted shim with reference to the order of shims extracted from the previous frame, thereby preventing overlapping coordinates of the seam. Enable playback.

In addition, the current frame extracts a shim having coordinates within a predetermined range of the same line as starting pixel coordinates based on the coordinates of the starting pixel of the shim extracted from the previous frame corresponding to the order of the shims of the current frame. In order to extract the seam of the current frame, one or more seam candidates having one of the coordinates within the predetermined range as the start pixel coordinates are selected to extract seam candidates that satisfy a predetermined condition as seams (S132 to S137).

In this case, the pixels constituting the shim candidate are adjacent to the pixels of the adjacent line, and have pixel conditions that the coordinates of the corresponding pixel and the coordinates of the corresponding line in the same line of the shim of the corresponding previous frame are adjacent or the same (S133).

Alternatively, in the selecting of the shim candidate, the pixels constituting the shim candidate are within a predetermined range with the pixels of the adjacent line, and the coordinates of the corresponding pixels and the coordinates of the corresponding lines in the same line of the shim of the corresponding previous frame are within the predetermined range. It may have to be a pixel condition (S133).

Due to such pixel condition, when extracting the seam of the current frame in the scene, it can be extracted in consideration of the spatial connectivity of each pixel and the temporal connectivity with the seam of the previous frame, thereby satisfying the inter-connectivity in the video. The phenomenon can be prevented.

The pixel condition of the shim candidate will be described in detail with reference to FIGS. 5 to 6.

The selecting of the shim candidate may include calculating an energy value of pixels satisfying the pixel condition, and selecting a shim candidate to have a minimum difference from an energy value of a pixel corresponding to the pixel in the same line of the shim of the previous frame. Are selected as the pixels forming the shim candidate (S134, S135).

Subsequently, the process of selecting each shim candidate with respect to the one or more starting coordinates is repeated (S136), and an energy value of the pixel of the shim of the previous frame corresponding to the cumulative value of energy values of pixels among the selected one or more shim candidates is repeated. A shim candidate having a minimum difference with a cumulative value of is extracted as a shim of the current frame corresponding to the shim of the previous frame (S137).

A detailed description of the comparison method and the shim extraction method of the energy value and the cumulative energy value will be described in detail with reference to FIGS. 7 to 8.

As described above, the content-based video resizing method according to an embodiment of the present invention compares the energy value and the cumulative energy value of each pixel with the energy value and the cumulative energy value of the previous frame at the time of extracting the seam of the current frame in the scene. Extraction satisfies the simplicity of the seam, thus enabling smooth playback of the target video.

FIG. 5 is an exemplary diagram of a shim candidate according to an exemplary embodiment of the present invention, and illustrates an example of a shim candidate path depending on the start coordinates (1, 2, 3) of the shim candidate. The starting coordinates (1, 2, 3) of the shim candidates do not simply use the same coordinates as the starting coordinate (2) of the shim 50 of the previous frame, but consider all N coordinates connected to the left and right.

The reason for considering all of the connected coordinates is that the following coordinates satisfying the pixel condition vary according to the starting coordinates of the seam as shown in FIG. 5 (51, 52, 53). 5 illustrates a case in which three coordinates are considered.

6 is an exemplary diagram of pixel selection of a shim candidate according to an exemplary embodiment of the present invention, wherein a pixel having a coordinate satisfying the predetermined condition is determined using a coordinate of a currently selected pixel and a coordinate of a pixel of a shim calculated in a previous frame. To obtain the next pixel of the seam.

The pixel condition includes a spatial connection and a temporal connection. The spatial connection requires that the coordinate 62 of the pixel to be selected currently is connected to the coordinate 61 of the pixel selected in the previous line. The corresponding coordinates 63 of the pixel of the seam of the seam to be calculated are connected to the time axis 64.

That is, the spatial connection means that adjacent pixels 61 and 62 in the current frame should be connected as shown in FIG. 6A, and the temporal connection indicates that the pixel 64 to be selected in the current frame as shown in FIG. 6B is determined by the previous frame. To be associated with the pixel coordinate 63. This temporal connection can prevent the shaking. 6c shows a pixel 65 that satisfies both the spatial and temporal connections.

FIG. 7 illustrates an example of shim candidate selection according to an exemplary embodiment of the present disclosure, and calculates an energy value of pixels satisfying the pixel condition, compares the energy value of a corresponding shim pixel of a previous frame, and matches a shim that meets the condition; FIG. Selecting candidates is shown.

First, an energy value is obtained for the coordinates 71 of pixels satisfying the pixel condition described in FIG. 6 (72, 73). The energy calculation method used in an embodiment of the present invention can be implemented in various ways, but it is preferable to obtain an energy value using a forward energy method.

In the step of obtaining the energy value of the pixel, at least one next pixel that satisfies the pixel condition with respect to the currently selected coordinate may exist (72,73), and thus, the energy value (72,73) of each pixel for the plurality of pixels. ) Is compared with the energy value 78 of the seam obtained in the previous frame to obtain the most similar value.

That is, among the plurality of pixels that satisfy the condition of the pixels constituting the shim candidate, the pixels 73, 74, 75, and 76 having the minimum difference from the energy value of the corresponding pixel in the previous frame are included in the corresponding case (that is, the start of the shim). Coordinates of the seam candidates in one of the number of cases distinguishable according to the coordinates are determined and stored in the buffer. Since pixels are selected one by one on the same line, a single shim candidate 77 can be extracted for one start coordinate through this process.

Of course, when the minimum difference of pixel energy values is the same, it is natural that a single shim candidate can be extracted by selecting one pixel by adding different conditions. The energy value of the corresponding pixel in the previous frame is shown on the right.

This process is repeated as many as a predetermined number (number of start coordinates for selecting the seam candidate) to extract one or more seam candidates.

8 is an exemplary diagram of seam extraction according to an exemplary embodiment of the present invention, wherein a step of obtaining a seam of a current frame corresponding to a seam of a previous frame is finally obtained from one or more seam candidates classified according to respective start coordinates in the current frame. It is shown.

At least one shim candidate classified according to the start coordinates of the shim was extracted through the steps described with reference to FIG. 7. Then, the cumulative value 84 of the extracted shim candidates is compared with the cumulative value 84 of pixel energy values of the corresponding shim of the previous frame and the cumulative values 81, 82, 83 of the pixel energy values of the extracted shim candidates. The seam candidate 83 having the selected seam is selected as the seam of the current frame corresponding to the seam of the previous frame.

In this example, a process of selecting a shim, for example, of three cases among a plurality of cases is illustrated. For the seam candidate selected according to the position of each start coordinate, the energy value of the corresponding seam candidate pixel is accumulated and compared with the cumulative value of the pixel energy values of the corresponding seam of the previous frame to find the case where the difference is the smallest. Decide on The determined shims are stored in the buffer in the order of the corresponding shims for operation in the next frame (85). In addition, this process is repeated as many times as the number of shims stored in the previous frame to obtain a plurality of shims necessary to adjust the size of the image in the current frame.

FIG. 9 is an exemplary view of adjusting a frame size according to an embodiment of the present invention. In the process of FIGS. 5 to 8, the process of extracting a seam of a current frame is repeated as many as the number of seams to adjust a plurality of seams required for size adjustment in the current frame. After extraction, it shows how to adjust the size of the current frame using this.

Since the order of shims is stored in the process of extracting seams, the corresponding coordinates are deleted for each order of seam extraction when the image size is reduced. On the other hand, when increasing the size of the image, values are put in the corresponding coordinates in the order of extracting the seams. In this case, it is preferable to use an average value of pixel energy values corresponding to the coordinates.

In this embodiment, a seam map is generated using the stored seam related information, and the size of the image is adjusted using the seam map. The seam map is the same as the original image size and is data in which the seam number (or order) is stored in the coordinates of the seam.

First, as shown in FIG. 9B, when the size of an image is reduced, the seam map is searched to remove pixels of the image in order of seams, and neighboring pixels are connected. After the image is reduced by one seam, the seam map is also reduced by one seam to keep the coordinates synchronized.

On the other hand, when the size of the image is enlarged as shown in FIG. 9C, pixels of the image are moved one space to the right in the order of shims, and the average of the neighboring pixel energy values is filled in the space. In addition, after one seam is enlarged, the seam map is also enlarged to synchronize the coordinates. By repeating this process, the final image is obtained.

FIG. 10 is a flowchart illustrating frame size adjustment according to an embodiment of the present invention, and illustrates a procedure of adjusting an image size using a shim map in the image size adjusting step described with reference to FIG. 9.

If a seam map is not generated (S161), a seam map corresponding to the image is generated (S162) and corresponds to a pixel that is duplicated or removed during the size adjustment (S163) of the image for synchronization with the coordinates of the image. By inserting or deleting the pixel of the seam map (S164).

Using this method, the coordinates of the video and the seam map can be synchronized by inserting or deleting the coordinates of the seam map corresponding to the pixels of the image to be copied or removed when the video is resized. The operation can be performed easily.

11 is a configuration diagram of a video size adjusting apparatus according to an embodiment of the present invention, wherein the video size adjusting apparatus includes an image input / output unit 110 that is responsible for input / output of a video and a scene change detection unit that detects whether a scene is changed in a current frame. If the current frame is not a scene change frame, the seam of the current frame corresponding to the scene is extracted using the shim related information extracted from the previous frame. If the current frame is a scene change frame, the shim related information extracted from the previous frame is not used. A seam extractor 140 for extracting a seam of a current frame, a seam information storage unit 130 for storing seam related information of the extracted current frame, a resizing unit 150 for adjusting a size of a current frame by using the extracted seam; It includes a control unit 160 for controlling each unit.

The shim extractor 140 repeats extracting the shims corresponding to the shims extracted from the previous frame in a state other than the shims that have been previously extracted as many as the number of shims.

As such, when no scene transition occurs, the seam of the current frame is extracted by using the seam-related information extracted from the previous frame, and the seam of the current frame is extracted without information of the entire frame of the scene for successive frames in the same scene having similar energy distributions. It can also be applied to videos provided in the form of real-time streaming.

12 is a block diagram of a seam extractor according to an exemplary embodiment of the present invention, wherein the seam extractor 140 is a seam information use unit 141 using seam-related information of a previous frame, and uses the seam information to start coordinates using the seam information. And a seam candidate selecting unit 142 for selecting one or more seam candidates classified according to the above, and a seam selection unit 143 for selecting seams of the current frame from the at least one seam candidate.

The shim candidate selector 142 extracts the shim candidate in consideration of the spatial connectivity of each pixel and the temporal connectivity of the previous frame of the seam, and the seam selector 143 performs the seam of the previous frame among the seam candidates. The seam with the smallest difference between and is selected as the seam of the current frame.

The foregoing embodiments and advantages are merely exemplary and are not to be construed as limiting the present invention. However, the present invention is not limited to the above-described embodiments, and various changes and modifications may be made by those skilled in the art without departing from the scope of the present invention. .

100: video resize device 110: video input and output unit
120: scene change detection unit 130: shim information storage unit
140: shim extracting unit 141: shim information using unit
142: Sim candidate selection unit 143: Sim selection unit
150: resizing unit 160: control unit

Claims (12)

In the content-based video resizing method carried out in the device that receives the video and adjusts the size of the video,
a) detecting whether a scene of the current frame is changed;
b) extracting a seam of a current frame corresponding to the seam using information related to a seam extracted from a previous frame if the current frame is not a scene change frame in step a);
c) storing the shim related information of the current frame extracted in step b); And
and d) adjusting the size of the current frame using the seams extracted in step b).
The method of claim 1,
And b) extracting a seam of the current frame without using seam-related information extracted from the previous frame if the current frame is a scene change frame in step a).
The method of claim 1,
Step c) stores the order of seams extracted from the current frame,
The step b) is the content-based video resizing method characterized in that it is repeated in the order of the seams extracted from the previous frame to extract the number of seams extracted from the previous frame as the number of the order of seams extracted from the previous frame.
The method of claim 3, wherein
Step c) stores the coordinates of the pixels included in the seam extracted from the current frame,
And b) extracting a seam having a coordinate within a predetermined range of the same line as the start pixel coordinates based on the coordinates of the start pixel of the seam extracted from the corresponding previous frame.
The method of claim 4, wherein b)
and e) selecting at least one shim candidate having one of the coordinates within the predetermined range as a start pixel coordinate to extract the shim of the current frame.
The method of claim 5, wherein in step e)
Pixels constituting the shim candidate are adjacent to pixels of an adjacent line, and have a pixel condition that the coordinates of the corresponding pixel in the same line of the shim of the corresponding previous frame and the coordinates are adjacent or identical to each other. How to resize your video.
The method according to claim 6,
Step c) stores the energy value of the pixel included in the seam extracted from the current frame,
In step e), the energy values of the pixels satisfying the pixel condition are calculated, and the seam candidates having the minimum difference with the energy value of the pixel corresponding to the pixel in the same line of the seam of the previous frame corresponding thereto are determined. Content-based video resizing method characterized in that it is selected as the candidate for the seam as a pixel constituting.
8. The method of claim 7,
Step c) stores the cumulative value of the energy values of the pixels included in the seam extracted from the current frame,
In step b), a seam candidate having a minimum difference between an energy value of a pixel of the corresponding previous frame and an energy value of the pixel of the corresponding previous frame is presently corresponding to the seam of the previous frame. Content-based video resizing method characterized in that the extraction to the seam of the frame.
The method of claim 5, wherein in step e)
The pixel constituting the shim candidate is within a predetermined range with the pixels of the adjacent line, and the pixel condition is that the coordinates of the corresponding pixel in the same line of the shim of the corresponding previous frame and the coordinates are within the predetermined range. How to resize content based videos.
The method of claim 1, wherein step d)
Generating a shim map corresponding to the video; And
And inserting or deleting a pixel of the seam map corresponding to a pixel that is duplicated or removed when resizing the video for synchronization with coordinates.
In the device that receives a video and adjusts the size of the video based on the content,
A scene change detector detecting whether a scene of the current frame is changed;
If the current frame is not a transition frame, the seam of the current frame corresponding to the seam extracted from the previous frame is extracted. If the current frame is a transition frame, the seam of the current frame is extracted without using the seam information of the previous frame. Extracting seam extracting unit;
A shim information storage unit for storing shim related information of the extracted current frame; And a resizing unit configured to adjust a size of a current frame by using the extracted shim.
12. The method of claim 11,
The shim information storage unit stores an order of shims extracted from the current frame,
And the seam extractor repeats extracting seams corresponding to seams extracted from a previous frame in a state other than seams extracted from the previous frame as many as the number of seams extracted from the previous frame.

Images