KR20210083524A - Method And Apparatus for Retargeting Video - Google Patents

Abstract

Disclosed are a method and device for retargeting an image for expanding or reducing an image ratio. In this embodiment, the image ratio is expanded by acquiring an image in which a line extracted from an energy map for an area other than an area to be preserved in the input image is inserted, and the image ratio is reduced by acquiring a line deletion image extracted from the energy map for the area excluding the area to be preserved in a correct answer image, in the provided method and device for retargeting an image.

Description

Image retargeting method and apparatus for expanding or reducing image ratio {Method And Apparatus for Retargeting Video}

The present embodiment relates to an image retargeting method and apparatus for expanding or reducing an image ratio.

The content described below merely provides background information related to the present embodiment and does not constitute the prior art.

In order to form a new image according to the user's convenience, image editing/synthesis technology for correcting or synthesizing image fragments has been developed in various ways. In general, video editing/compositing technology is being developed for the purpose of erasing unwanted video parts, separating desired parts, and synthesizing new parts.

In order to synthesize an image, it is required to extract an object within an image. As an image segmentation technology for extracting an object, technologies such as a Watershed algorithm and a Graph Cut algorithm have been developed and used in various ways. have.

A technique developed for image resizing includes a seam carving/insertion algorithm. The seam carving algorithm calculates the importance between each pixel in the original image, constructs a line that connects pixels of low importance by dynamic programming, and removes the lines in the order of low importance. (Removal) or insert (Insertion) to reduce or expand the size of the image. The seam carving technology was originally designed for the purpose of displaying images without distortion in devices having various resolutions.

The prior art used a method of synthesizing a background image by simply overlaying an object image. Even in the process of image enlargement and reduction, there is a limitation in image editing/compositing because the entire image is simply enlarged or reduced regardless of the ratio of the object image and the background image, and thus, there is a problem in that it is difficult to provide a natural synthetic image.

In this embodiment, a region of interest to be preserved in the input image is set, and an image in which blank lines are inserted in an order of low interest in the remaining regions except for the region is constructed, and the corresponding line is based on machine learning. An object of the present invention is to provide an image retargeting method and apparatus aimed at filling or correcting the .

According to one aspect of the present embodiment, the process of receiving an input image (Input Image) (I); setting a partial region of the input image (I) as a Preservation Mask (M); calculating an importance for all pixels in an area except for the preservation mask M from the input image I, and generating an energy map composed of the importance of each pixel; a process of searching for and extracting a line composed of a connection of pixels of relatively low importance in the energy map; acquiring _{the line insertion image (I +} ) while preserving the area of the preservation mask (M); Image retargeting method comprising the step of generating the line-inserted image (I ₊ ^{) as a retarget image (I ^} _R ) with an expanded image ratio using an artificial intelligence model trained in advance based on machine learning provides

According to another aspect of the present embodiment, the input unit for receiving an input image (Input Image) (I); a mask setting unit for setting a partial region of the input image (I) as a preservation mask (M); an energy map generator for calculating the importance of all pixels in an area except for the preservation mask M from the input image I, and generating an energy map composed of the importance of each pixel; a line extracting unit that searches for and extracts a line formed by connecting pixels of relatively low importance in the energy map; a line insertion unit which acquires _{the line insertion image (I +} ) while preserving the area of the preservation mask (M); _{An image ratio extension unit for generating the line-inserted image (I +} ) as a retarget image (I ^{^} _R ) with an expanded image ratio using an artificial intelligence model learned in advance based on machine learning; characterized in that it comprises a; An image retargeting device is provided.

As described above, according to this embodiment, an image in which a line extracted from an energy map is obtained for a region excluding a region to be preserved in the input image is obtained to expand the image ratio or correct the correct answer. There is an effect that the image ratio can be reduced by obtaining an image in which the line extracted from the energy map for the area except for the area to be preserved in the image is obtained.

1A, 1B, and 1C are block diagrams schematically illustrating an image retargeting apparatus for expanding or reducing an image ratio according to the present embodiment.
2A, 2B, and 2C are block diagrams schematically illustrating an image retargeting program mounted in an image retargeting apparatus according to the present embodiment.
3 is a diagram illustrating a method of expanding or reducing an image ratio according to the present embodiment.
4 is a flowchart illustrating a method of extending an image ratio according to the present embodiment.
5 is a flowchart illustrating a method of reducing an image ratio according to the present embodiment.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings.

1A, 1B, and 1C are block diagrams schematically illustrating an image retargeting apparatus for expanding or reducing an image ratio according to the present embodiment.

The image retargeting apparatus 110 according to the present embodiment expands or reduces the ratio of the image by using the mounted image retargeting program 120 . The image retargeting program 120 expands or reduces the ratio of the image by using line carving, infating technology, and artificial intelligence model.

The image retargeting device 110 uses the image retargeting program 120 to set a partial region of the input image I as a preservation mask (M), and after generating an energy map (Energy Map) After acquiring the image (I ₊ ) with the extracted line inserted, the image ratio is extended by using the pre-trained artificial intelligence model.

The image retargeting device 110 uses the image retargeting program 120 to set _{a partial area of the correct image I R} as a preservation mask (M), and generate an energy map (Energy Map). _{After obtaining the image (I -} ) from which the extracted line is deleted, the image ratio is reduced by using the pre-trained artificial intelligence model.

The image retargeting device 110 may be a user terminal or an application server, a service server, or a server terminal on which the image retargeting program 120 is mounted.

The image retargeting device 110 is each (i) a communication device such as a communication modem for performing communication with various devices or wired and wireless communication networks, (ii) a memory for storing various programs and data for expanding the image ratio, ( iii) It may refer to various devices including a microprocessor for executing and controlling the image retargeting program 120 . According to at least one embodiment, the memory is a computer such as random access memory (RAM), read only memory (ROM), flash memory, optical disk, magnetic disk, solid state disk (SSD), etc. It may be a readable recording/storage medium. According to at least one embodiment, a microprocessor may be programmed to selectively perform one or more of the operations and functions described herein. According to at least one embodiment, the microprocessor may be fully or partially implemented as hardware such as an Application Specific Integrated Circuit (ASIC) of a specific configuration.

In general, in the process of creating a data set required for learning, if a line is created and then filled with a general infading technique, the image has a sense of heterogeneity because the statistical characteristics of the image filled in the line are different. . For example, when a person's face is stretched from side to side, the data set itself needs to be adapted to the environment.

Therefore, the image retargeting program 120 according to the present embodiment sets the preservation mask M in the region to be preserved based on machine learning, and then finds a line in the remaining region except for the preservation mask region. Thereafter, the image retargeting program 120 may create a reduced image by removing the line, and _{in the process of creating a reduced image from the correct answer image I R} , the region in which the preservation mask M will be set may be found by machine learning. . The image retargeting program 120 may create a data set (input image I, correct answer image I _R ) by removing the line after finding the region where the preservation mask M is to be set.

The image retargeting program 120 expands the ratio of the image by using line carving and inpainting technology. The image retargeting program 120 expands the image ratio by using line carving technology, which has fewer artifacts and excellent generalization performance compared to machine learning technology, and fills the extended lines through inpainting technology.

_{The image retargeting program 120 acquires the line insertion image (I +} ) while preserving the area of the preservation mask (M) in the input image (I), and the area of the preservation mask (M) in the correct answer image ( _IR ) A line deletion image (I ₋ ) can be acquired while preserving. The image retargeting program 120 may use an inpainting technique when generating the _{line inserted image I + .}

Therefore, when creating a dataset required for learning, if a correct answer image ( _IR ) exists, the line deletion image (I _- ) and the line insertion image (I ₊ ) can be used as a pair for the training data. Here, the correct answer image I _R means an image required for learning.

When creating training data, the image retargeting program 120 creates an image line deletion image (I _{- ) in which lines are removed from the correct answer image (I R} ) (original image), and then the input image (I), the image ratio is expanded It is used for learning together with the target image (I ^{^} _{R ).} The image retargeting program 120 inputs the line-inserted image (I ₊ ) to the CNN to create the correct answer image ( _{IR ).}

^{The image retargeting program 120 creates a retargeting image (I ^} _R ) with an expanded image ratio when an input image (I) to be increased based on the learning data is input. Video retargeting program 120 when learning artificial intelligence model, delete the line image (I _-) enter, answer the image (I _R). The image retargeting program 120 learns the artificial intelligence model by receiving and learning the _{line inserted image (I +} ) and the line deleted image (I _{- ) as a pair.}

2A, 2B, and 2C are block diagrams schematically illustrating an image retargeting program mounted in an image retargeting apparatus according to the present embodiment.

The image retargeting program 120 according to the present embodiment includes an input unit 210 , a mask setting unit 220 , an energy map generation unit 230 , a line extraction unit 240 , a line insertion unit 252 , and a line deletion unit 254 , an image ratio extension unit 262 , an image ratio reduction unit 264 , and a learning unit 270 . Components included in the image retargeting program 120 are not necessarily limited thereto.

Each component included in the image retargeting program 120 may be connected to a communication path that connects a software module or a hardware module inside the device to operate organically with each other. These components communicate using one or more communication buses or signal lines.

Each component of the image retargeting program 120 shown in FIG. 2 means a unit for processing at least one function or operation, and may be implemented as a software module, a hardware module, or a combination of software and hardware.

The input unit 210 receives an input image (I). The input unit 210 receives the correct answer image I _R .

The mask setting unit 220 sets a partial region of the input image I as a Preservation Mask (M). The mask setting unit 220 performs instance segmentation on the input image I, and combines the results of performing the instance segmentation with the score given to each segmentation area (Saliency Map) for an important area. Set the maintenance mask (M) where it is determined as .

A mask setting section 220 sets a partial region of the correct image (I _R) to preserve the mask (Mask Preservation) (M). A mask setting section 220 using a scheme that combines the correct image (I _R) instance segmentation (Instance Segmentation) or object detection (object detection) frozen when map (Saliency Map) Sally to about the preserve mask (M) set A mask setting section 220 may correct the image (I _R) instance for the segment (Instance Segmentation) perform, and combining the results of the score (Saliency map) and instances segment (Instance Segmentation) assigned to each segmentation area relevant A preservation mask (M) is set at a place determined as an area.

The energy map generator 230 calculates the importance of all pixels in the region except for the preservation mask M from the input image I, and generates an energy map composed of the importance of each pixel. do. Energy map generation unit 230 may correct the image (I _R), calculate the importance, and the importance of energy map (Energy Map), consisting of each pixel for all the pixels (Pixel) for an area other than the preservation mask (M) from the create

The line extractor 240 searches for and extracts a line formed by connecting pixels of relatively low importance in the energy map for the input image I. Line extracting section 240 extracts the search line (Line) made of a relative importance is attached to a lower pixel in the energy map of the correct images (I _R).

The line inserter 252 acquires _{the line insert image I +} while preserving the region of the preservation mask M in the input image I. The line deletion unit 254 acquires _{the line deletion image I -} while preserving the area of the preservation mask M in the correct answer image I _{R .}

The image ratio extension unit 262 generates a line-inserted image (I ₊ ) as a retarget image (I ^{^} _R ) with an extended image ratio using an artificial intelligence model trained in advance based on machine learning. The image ratio reduction unit 264 generates a line deletion image (I _- ) as a retarget image with a reduced image ratio using an artificial intelligence model.

The learning unit 270 preserves the area of the preservation mask M in the input image I while preserving the area of the preservation mask M in the line-inserted image I ₊ and the correct answer image I _R The artificial intelligence model is trained by pairwise learning the line deletion image (I _{- ) where the line has been deleted.}

Learning unit 270 is preserved line removed image (I _-) delete lines, while preserving the areas of the mask (M) in the correct answer image (I _R), generating an original input image (I) as obtained by a sequential to learn _{The learning unit 270 sequentially acquires line-inserted images (I +} ) in which lines are inserted while preserving the area of the preservation mask (M) in the input image (I) to generate a correct answer image ( _{IR ).}

3 is a diagram illustrating a method of expanding or reducing an image ratio according to the present embodiment.

The image retargeting program 120 performs segmentation on the input image I.

The image retargeting program 120 calculates a score (Saliency Map) for each segmentation. At this time, the image retargeting program 120 performs an instance segment (if there is a person as an object, segmentation is performed for each person), a score is given to each segment area.

The image retargeting program 120 generates a preservation mask (M) by combining the score (Saliency map) given to each segment area and the result of performing the instance segmentation (Instance Segmentation).

The image retargeting program 120 predicts a region to set the preservation mask (M) from the input image (I). The image retargeting program 120 may predict an area to set the preservation mask M by combining instance segmentation or object detection with a saliency map.

The image retargeting program 120 generates a preservation mask M for identifying and maintaining an important area. That is, the image retargeting program 120 does not perform general seam carving, but combines the score (Saliency map) given to each segment area and the result of performing the instance segmentation (Instance Segmentation) to determine the area as an important area. (M) is set.

A mask is also set in the general seam carving method, but in this embodiment, when the image retargeting program 120 sets the preservation mask (M), the preservation mask (M) area is segmented to give a score, and only the meaningful area is the preservation mask (M). Areas with low scores among segmentation areas are not preserved. In other words, in the general seam carving method, a person directly selects an area or object when making a mask, but in this embodiment, the area or object is automatically converted into a preservation mask (M) using the image retargeting program 120 artificial intelligence. set

The seam carving algorithm is a method of expanding the image by filling in the line with a line after finding an area that a person is not interested in. The energy map means that a score value is given to each pixel value, and the score value is spread out in the form of a map. The seam carving algorithm computes the score as the gradient magnitude in the pixel.

The gradient magnitude refers to the sum of, for example, the absolute value of the difference between the pixel value on the left and the difference between the pixel value on the left and the upper pixel value based on the pixel value. Since the area where there is a person has a large amount of change in the area with a large amount of change in the image, the line can be avoided even without a mask.

The seam carving method refers to a method of filling a line to be filled when extending an image, and may expand or reduce an image. When reducing the image, find the line and then remove the line as it is. It judges that the line is unnecessary, finds the line, and reduces it by merging it. When expanding the image, it finds a line and then expands the line twice. Expand the line and fill in the appropriate value (interpolation).

In the seam carving method, if masking is necessary, a mask may be applied to a specific area. Before finding a line, an energy map is constructed, and the energy map is calculated by gradient magnitude. For example, a red mask makes the energy maximum, and a red mask over the area to be preserved avoids the line. For example, the green mask minimizes energy, and the green mask over the area to be erased passes through the line.

In a general seam carving method, a person directly masked, but in this embodiment, an area of interest to the person is selected and masking is performed.

The image retargeting program 120 performs seam carving only on the remainder of the input image (I) except for the preservation mask (M), and preserves the area of the preservation mask (M) while preserving the line insertion image (I _{+ ).} ) is obtained.

When performing a seam carving on the image retargeting program 120 is an input image (I), for example, 8k correct original and image (I _R) a going and reduced while sim Carving removed after creating an energy map in sequence information made an input image (I), and then learn the process of creating an image of the correct answer (I _R) is based on the re-input image (I).

That is, the image retargeting program 120 may learn again based on the result of performing the reverse seam carving. The image retargeting program 120 generates a reduced image by inversely seam carving the extended original region (the correct answer image I _{R ).} Video retargeting program 120 has a reduced image seam carving to be done to make learning the correct answer image (I _R).

The image retargeting program 120 predicts the ^{retarget image (I ^} _R ) in which the image ratio is extended by using the inserted line of the line inserted image (I _{+ ) as a mask.} That is, the image retargeting program 120 expands the ratio of the image by using the seam carving and inpainting technology.

Image retargeting program 120 predicts the preservation mask (M) from the correct image (I _R) to perform the learning. The image retargeting program 120 acquires a _{line deletion image (I -} ) while preserving the area of the preservation mask (M). The image retargeting program 120 configures the training data by pairing the _{line deletion image (I -} ) and the retarget image ( _{IR ).}

4 is a flowchart illustrating a method of extending an image ratio according to the present embodiment.

The image retargeting program 120 receives an input image (I) (S410). The image retargeting program 120 sets a partial region of the input image I as a Preservation Mask (M) (S420).

In step S420, the image retargeting program 120 performs instance segmentation on the input image I, and performs an instance segmentation and a score given to each segmentation area (Saliency Map). Combining the results, a preservation mask (M) is set where it is determined as an important area.

The image retargeting program 120 calculates the importance of all pixels in the region except the preservation mask M from the input image I, and generates an energy map composed of the importance of each pixel. do (S430).

The image retargeting program 120 searches for and extracts a line consisting of a connection of pixels of relatively low importance in the energy map for the input image I (S440).

_{The image retargeting program 120 acquires the line insertion image (I +} ) while preserving the area of the preservation mask (M) in the input image (I) (S450).

The image retargeting program 120 generates a line-inserted image (I ₊ ) as a retarget image (I ^{^} _R ) with an extended image ratio using an artificial intelligence model trained in advance based on machine learning (S460).

Although it is described that steps S410 to S460 are sequentially executed in FIG. 4 , the present invention is not limited thereto. In other words, since it may be applicable to changing and executing the steps described in FIG. 4 or executing one or more steps in parallel, FIG. 4 is not limited to a time-series order.

As described above, the method of extending the image ratio according to the present embodiment illustrated in FIG. 4 may be implemented as a program and recorded in a computer-readable recording medium. The computer-readable recording medium in which the program for implementing the method of extending the image ratio according to the present embodiment is recorded includes all kinds of recording devices in which data readable by the computer system is stored.

5 is a flowchart illustrating a method of reducing an image ratio according to the present embodiment.

The image retargeting program 120 receives the correct answer image I _R (S510). Image retargeting program 120 sets a partial region of the correct image (I _R) to preserve the mask (Mask Preservation) (M) (S520).

In step S520, image retargeting program 120 is preserved by a method in combination with the correct answer image (I _R) instance segmentation (Instance Segmentation) or object detection (object detection) frozen when map (Saliency Map) Sally to about Set the mask (M).

Image retargeting program 120 is combining the result of performing the instance of the segment (Instance Segmentation) for a correct answer image (I _R), and carry out the score (Saliency map) and instances segment (Instance Segmentation) assigned to each segmentation area A preservation mask (M) is set where it is determined as an important area.

The image retargeting program 120 calculates the importance for all pixels in the area except the preservation mask M from the _{correct answer image I R , and generates an energy map composed of the importance of each pixel.} generated (S530).

Image retargeting program 120 is extracted by searching the lines (Line) made of a relative importance is attached to a lower pixel in the energy map of the correct images (I _R) (S540).

_{The image retargeting program 120 acquires the line deletion image I -} while preserving the area of the preservation mask M in the correct answer image I _R (S550). The image retargeting program 120 generates a line deletion image (I _- ) as a retargeting image with a reduced image ratio using an artificial intelligence model (S560).

Although it is described that steps S510 to S560 are sequentially executed in FIG. 5 , it is not necessarily limited thereto. In other words, since it may be applicable to changing and executing the steps described in FIG. 5 or executing one or more steps in parallel, FIG. 5 is not limited to a chronological order.

As described above, the method for reducing the image ratio according to the present embodiment illustrated in FIG. 5 may be implemented as a program and recorded in a computer-readable recording medium. The computer-readable recording medium in which the program for implementing the method of reducing the image ratio according to the present embodiment is recorded includes all kinds of recording devices in which data readable by the computer system is stored.

The above description is merely illustrative of the technical idea of this embodiment, and various modifications and variations will be possible by those skilled in the art to which this embodiment belongs without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain rather than limit the technical spirit of the present embodiment, and the scope of the technical spirit of the present embodiment is not limited by these embodiments. The protection scope of the present embodiment should be interpreted by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present embodiment.

110: video retargeting device
120: video retargeting program
210: input unit 220: mask setting unit
230: energy map generation unit 240: line extraction unit
252: line insertion unit 254: line deletion unit
262: image ratio expansion unit 264: image ratio reduction unit
270: study unit
I : Input Image
M: Preservation Mask
I ₊ : Seam Inserted Image
I _- : Seam Removed Image
I ^{^} _R : Retargeted image with extended image ratio ((Predicted) Retargeted Image)
I _R : (Ground-Truth) Retargeted Image

Claims (15)

a process of receiving an input image (I);
setting a partial region of the input image (I) as a Preservation Mask (M);
calculating an importance for all pixels in an area except for the preservation mask M from the input image I, and generating an energy map composed of the importance of each pixel;
a process of searching for and extracting a line composed of a connection of pixels of relatively low importance in the energy map;
acquiring _{the line insertion image (I +} ) while preserving the area of the preservation mask (M);
generating the line-inserted image (I ₊ ^{) as a retarget image (I ^} _R ) with an extended image ratio using an artificial intelligence model trained in advance based on machine learning;
Image retargeting method comprising a. According to claim 1,
The process of setting the preservation mask (M) is,
Instance segmentation is performed on the input image (I), and a score given to each segmentation area (Saliency map) and a result of performing the instance segmentation are combined to determine an important area. Image retargeting method, characterized in that setting the preservation mask (M). According to claim 1,
The process of receiving the correct answer image (I _{R );}
The process of setting a partial area of the correct images (I _R) to the preservation mask (Preservation Mask) (M);
calculating the importance for all pixels in the area except the preservation mask M from the correct answer image I _{R and generating an energy map composed of the importance of each pixel;}
a process of searching for and extracting a line composed of a connection of pixels of relatively low importance in the energy map;
acquiring _{the line deletion image (I -} ) while preserving the area of the preservation mask (M);
The process of generating the line deletion image (I _- ) as a retarget image with reduced image ratio using the artificial intelligence model
Image retargeting method, characterized in that it further comprises. 4. The method of claim 3,
While preserving the area of the preservation mask M in the input image I, while preserving the area of the preservation mask M in the line-inserted image I ₊ and the correct answer image I _R A process of learning the artificial intelligence model by learning the line deleted image (I _{− ) from which the line has been deleted in pairs;}
Image retargeting method, characterized in that it further comprises. 5. The method of claim 4,
The process of learning the artificial intelligence model is,
The correct image the preservation mask (M) the line removes the image (I _-), while preserving the area, delete the line in the (I _R) and obtaining in sequence, and learning the step of generating the input image (I),
An image characterized in that learning the process of generating the correct answer image by sequentially acquiring _{the line-inserted image (I +} ) in which lines are inserted while preserving the area of the preservation mask (M) in the input image (I) How to retarget. 4. The method of claim 3,
The process of setting the preservation mask (M) is
Image retargeting method characterized by using the method in combination with the correct images (I _R) instance segmentation (Segmentation Instance) or object detection (object detection) when unloading map (Map Saliency) Sally to about. 7. The method of claim 6,
The process of setting the preservation mask (M) is,
The instance segmentation is performed on the correct image I _R , and the result of performing the instance segmentation with the score given to each segmentation area is combined to determine an important area. Image retargeting method, characterized in that setting the preservation mask (M) in the place. an input unit for receiving an input image (I);
a mask setting unit for setting a partial region of the input image (I) as a preservation mask (M);
an energy map generator for calculating the importance of all pixels in an area except for the preservation mask M from the input image I, and generating an energy map composed of the importance of each pixel;
a line extracting unit that searches for and extracts a line formed by connecting pixels of relatively low importance in the energy map;
a line insertion unit which acquires _{the line insertion image (I +} ) while preserving the area of the preservation mask (M);
an image ratio extension unit for generating _{the line-inserted image (I +} ) as a retarget image (I ^{^} _R ) with an extended image ratio using an artificial intelligence model learned in advance based on machine learning;
Image retargeting device comprising a. 9. The method of claim 8,
The mask setting unit,
Instance Segmentation is performed on the input image I, and a score (Saliency Map) given to each segmentation area and a result of performing the Instance Segmentation are combined to determine an important area. Image retargeting device, characterized in that setting the preservation mask (M). 9. The method of claim 8,
When the correct answer image (I _R ) is input from the input unit,
From the mask setting unit process of setting a partial region of the correct image (I _R) to the preservation mask (Preservation Mask) (M);
Wherein the correct answer image (I _R) Energy Map (Energy Map) calculates the priority for all the pixels (Pixel) for the region excluding the preservation mask (M) and consisting of the importance of each pixel from the energy map generator for create and
An image retargeting apparatus, characterized in that the line extractor searches for and extracts a line consisting of a connection of pixels of relatively low importance in the energy map. 11. The method of claim 10,
a line erasing unit for acquiring _{the line erasing image (I -} ) while preserving the area of the preserving mask (M);
Image ratio reduction unit for generating the line deletion image (I _- ) as a retarget image with reduced image ratio using the artificial intelligence model
Image retargeting device, characterized in that it further comprises. 12. The method of claim 11,
While preserving the area of the preservation mask M in the input image I, while preserving the area of the preservation mask M in the line-inserted image I ₊ and the correct answer image I _R A learning unit that learns the artificial intelligence model by learning the line deleted image (I _{- ) from which the line has been deleted in pairs}
Image retargeting device, characterized in that it further comprises. 13. The method of claim 12,
The learning unit,
Learning the process of generating the original of the input image (I) and obtained by sequentially _- the correct image (I _R) the preservation mask (M) the line removes the image (I), while preserving the area, delete the line in and,
To obtain the line insert images (I _+), insert the lines, while preserving the portion of the retention mask (M) in the input image (I) in order to study the process of generating the correct image (I _R) A video retargeting device characterized by. 13. The method of claim 12,
The mask setting unit,
Characterized in that for setting the preservation mask (M) by using a method in combination with the correct images (I _R) instance segmentation (Instance Segmentation) or object detection (object detection) frozen when map (Saliency Map) Sally to about video retargeting device. 14. The method of claim 13,
The mask setting unit,
The instance segmentation is performed on the correct image I _R , and the result of performing the instance segmentation with the score given to each segmentation area is combined to determine an important area. Image retargeting device, characterized in that setting the preservation mask (M) in the place.

Images