KR102095814B1 - System and Method for Analyzing Video Data Using Change Pattern of Shot Size - Google Patents

Abstract

According to one aspect of the present invention, an image data analysis system using a shot change pattern capable of analyzing image data using a shot size change pattern of the image data comprises: a shot boundary detection unit which detects a boundary between shots from image data using a shot boundary detection algorithm; a shot size determination unit which determines a shot size of a corresponding shot based on a size of a representative object extracted from at least one of images constituting each shot; a climax section determination unit for determining a climax section of the image data using a change pattern of the shot size; and a database generation unit which maps and stores the determined shot size change pattern of the climax section with identification information of the image data.

Description

System and Method for Analyzing Video Data Using Change Pattern of Shot Size}

The present application relates to an image data analysis system and method.

Recently, video data such as Korean dramas and movies have gained popularity in Asia and Europe. This popularity is due to the fact that scenario planning powers such as dramas and movies, and various technologies such as shooting techniques, editing techniques, and special effects are combined and expressed in a comprehensive manner. In particular, climax in dramas and movies is an important part of enhancing immersion, and can be seen as a decisive factor for the success of dramas and movies.

Therefore, in order to maintain this popularity, it is necessary to analyze the characteristics of the video data that has been successful, and generate the image data based on the analyzed characteristics. At this time, to analyze the characteristics of the video data, attempts have been made to select a climax of the video data from the video data, and to confirm a shooting technique or editing technique applied to the climax.

However, since the selection of the climax of the image data is generally performed by the experience of the analyst, different climaxes may be selected for each analyst even for the same image data, and analysis of the image data may not be accurately performed. There is a problem that it can take a lot of time.

An object of the present invention is to provide an image data analysis system and method using a shot size change pattern capable of analyzing image data using a shot size change pattern of image data.

In addition, the present invention provides a system and method for analyzing image data using a shot size change pattern that can generate a database for image analysis by mapping and storing the shot size change pattern in a climax section with identification information of corresponding image data. Another technical task.

In addition, the present invention provides an image data analysis system and method using a shot size change pattern capable of determining in advance whether or not the success of specific image data is previously performed using a shot size change pattern of image data stored in a database for image analysis. Another technical task.

Another object of the present invention is to provide an image data analysis system and method using a shot size change pattern capable of generating a target image using a database for image analysis.

An image data analysis system using a shot change pattern according to an aspect of the present invention for achieving the above-described technical problem includes: a shot boundary detection unit that detects a boundary between shots from image data using a shot boundary detection algorithm; A shot size determination unit that determines a shot size of a corresponding shot based on a size of a representative object extracted from at least one of the images constituting each shot; A climax section determination unit for determining a climax section of the image data using the change pattern of the shot size; And a database generation unit that maps and stores the determined shot size change pattern of the climax section with the identification information of the image data.

An image data analysis method using a shot change pattern according to an aspect of the present invention for achieving the above-described technical problem includes: detecting a boundary between shots from image data using a shot boundary detection algorithm; Determining a shot size of a corresponding shot based on a size of a representative object extracted from at least one image among images constituting each shot; And determining a climax section of the image data using the change pattern of the shot size.

According to the present invention, since the climax can be automatically extracted from the image data by using the shot size change pattern of the image data, there is an effect that the climax can be accurately extracted from the image data in a short time.

In addition, according to the present invention, since it is possible to create a database for analyzing image data using a shot size change pattern corresponding to a climax of each image data, there is an effect that analysis of image data is facilitated.

In addition, according to the present invention, it is possible to predict whether or not the success of specific video data can be predicted by using the shot size change pattern of the video data in the box office stored in the database for image analysis. have.

In addition, according to the present invention, when the target image data is generated, a shot size change pattern of image data of a specific genre stored in a database for image analysis can be used, thereby increasing the ease of generating the target image data and increasing the likelihood of the success of the target image data. It has the effect of being able to.

1 is a block diagram showing the configuration of an image data analysis system using a shot size change pattern according to an embodiment of the present invention.
2A to 2C are diagrams showing inaccuracies in shot boundaries detected using one shot boundary detection algorithm.
3 is a block diagram specifically showing the configuration of the shot size determination unit illustrated in FIG. 1.
4A is a diagram showing an example of a plurality of target areas detected in an image.
4B is a diagram showing an example of a representative object determined among a plurality of target areas.
4C is a view showing an example of a representative object whose shot size is a close-up shot.
5A is a view showing an example of an image including a target area that does not include a person photographed from the front.
5B is a diagram showing an example of a representative object determined from the image shown in FIG. 5A.
6 is a diagram showing an example of an image in which a person is not included.
7 is a graph showing an example of a climax section determined according to a shot size change pattern.
8 is a flowchart illustrating an image data analysis method using a shot size change pattern according to an embodiment of the present invention.

Advantages and features of the present application, and a method of achieving them will be clarified with reference to examples described below in detail together with the accompanying drawings. However, the present application is not limited to the examples disclosed below, but will be implemented in various different forms, and only the examples of the present application make the disclosure of the present application complete, and common knowledge in the art to which this application belongs It is provided to fully inform the person having the scope of the invention, and the present application is only defined by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining an example of the present application are exemplary, and the present application is not limited to the illustrated matters. The same reference numerals refer to the same components throughout the specification. In addition, in the description of the present application, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present application, the detailed description will be omitted.

When 'include', 'have', 'consist of' and the like mentioned in this specification are used, other parts may be added unless '~ man' is used. When a component is expressed as a singular number, the plural number is included unless otherwise specified.

In analyzing the components, it is interpreted as including the error range even if there is no explicit description.

In the case of the description of the positional relationship, for example, when the positional relationship of two parts is described as '~ top', '~ upper', '~ bottom', '~ side', etc., 'right' Alternatively, one or more other parts may be located between the two parts unless 'direct' is used.

In the case of a description of a time relationship, for example, 'after', 'following', '~ after', '~ before', etc., when the temporal sequential relationship is described, 'right' or 'direct' It may also include cases that are not continuous unless it is used.

The first, second, etc. are used to describe various components, but these components are not limited by these terms. These terms are only used to distinguish one component from another component. Accordingly, the first component mentioned below may be the second component within the technical spirit of the present application.

"First horizontal axis direction", "Second horizontal axis direction" and "Vertical axis direction" should not be interpreted only as a geometric relationship in which the relationship between each other is vertical, and the scope in which the configuration of the present application can function functionally It can mean having a wider directionality than within.

It should be understood that the term “at least one” includes all possible combinations from one or more related items. For example, the meaning of “at least one of the first item, the second item, and the third item” means 2 of the first item, the second item, and the third item, as well as the first item, the second item, and the third item, respectively. It can mean any combination of items that can be presented from more than one dog.

Each of the features of the various examples of the present application may be partially or wholly combined with or combined with each other, technically various interlocking and driving may be possible, and each of the examples may be independently implemented with respect to each other or may be implemented together in an associative relationship. .

Hereinafter, an image data analysis system using a shot size change pattern according to the present invention will be described in detail with reference to the drawings. In adding reference numerals to the components of each drawing, the same components may have the same reference numerals as possible even though they are displayed on different drawings.

1 is a block diagram schematically showing a configuration of an image data analysis system using a shot size change pattern according to an embodiment of the present invention. As shown in FIG. 1, the image data analysis system (hereinafter referred to as 'analysis system') using the shot size change pattern according to the present invention includes an image data receiving unit 110, a shot boundary detection unit 120, It includes a shot size determining unit 130, a climax section determining unit 140, and a database generating unit 150.

The image data receiving unit 110 receives image data to be analyzed. In one embodiment, the image data received through the image data receiving unit 110 may be scenario-based image data such as a movie, drama, or animation. As another example, the image data received through the image data receiving unit 110 may include image data such as a music video.

The image data receiving unit 110 may receive image data from the outside through a network such as the Internet. As another example, the image data receiving unit 110 receives image data from an image database (not shown) included in the analysis system 100, or a separate storage medium such as a mobile device, hard disk, or Universal Serial Bus (USB). It may be possible to receive image data from.

The image data receiving unit 110 transmits the received image data to the shot boundary detection unit 120.

The shot boundary detection unit 120 detects a shot boundary from the image data received through the image data reception unit 110. Here, the shot refers to image data generated by continuous movement of the camera, and may be composed of a plurality of consecutive images. A plurality of shots are gathered to form a scene, and a plurality of scenes are gathered to form a sequence.

The shot chance detection unit 120 may detect a shot boundary from image data using a predetermined shot boundary detection algorithm. The shot boundary detection unit 120 according to the present invention detects a shot boundary by mixing a plurality of shot boundary detection algorithms.

The reason why the shot boundary detection unit 120 according to the present invention detects a shot boundary by mixing a plurality of shot boundary detection algorithms is that in the case of video data such as a movie, a scene transition technique is used for the connection between the shot and the shot (Video Transition Effects). It is used a lot, and even though it is the same scene due to various uses of lighting, the brightness of the lighting can change dramatically, and when the object moves while in extreme close-up, the entire image of the screen changes. Because there is a characteristic that can be judged, if a shot boundary is detected using only one shot boundary algorithm, an accurate shot boundary is not detected.

For example, when the shot boundary detection unit 120 detects a shot boundary using a single shot boundary detection algorithm, as shown in FIG. 2A, the brightness of the lighting changes rapidly even though the background and the actor are not changed. In case, it can be detected as a shot boundary.

As another example, when the shot boundary detection unit 120 detects a shot boundary using a single shot boundary detection algorithm, as shown in FIG. 2B, such as Dissove or Fade in / out. Even if two frames are continuously mixed for more than 1 second by applying the scene change technique, it can be detected as a shut boundary.

As another example, when the shot boundary detection unit 120 detects a shot boundary using a single shot boundary detection algorithm, even when a large object moves in front of the camera in an excessive close-up state as illustrated in FIG. 2C, the shot is shot. It can be detected as a boundary.

Therefore, the shot boundary detection unit 120 according to the present invention detects the shot boundary by mixing and applying a plurality of shot boundary detection algorithms when detecting the shot boundary in order to solve the inaccuracy of the shot boundary detection as described above. In one embodiment, the shot boundary detection unit 120 may detect a shot boundary by mixing a color histogram algorithm, an edge histrogram algorithm, and an optical flow tracking algorithm.

Specifically, the shot boundary detection unit 120 according to the present invention is used by mixing a color histogram algorithm and an edge histogram algorithm to prevent the shot boundary from being incorrectly detected due to a sudden change in the brightness of the lighting. do. This is because the color histogram change is large when the brightness of the light changes rapidly, but the edge histogram does not change.

Particularly, the shot boundary detection unit 120 according to the present invention additionally uses KCF filters (Kernelized Correlation Filters) used in the optical flow tracking algorithm, so that even when scene transition techniques such as dissolve or fade in / out are applied, the shot boundary is applied. Will accurately detect. At this time, the KCF filter is a specific classification model that is mostly used to distinguish the target from the surrounding environment as one of the tracker functions.For the shots of gradual change applied with a scene change technique such as dissolve or fade in / out due to the use of the KCF filter It is possible to limit overlapping images and overlapping pixels.

Referring back to FIG. 1, when the shot boundary is detected by the shot boundary detection unit 120, the shot sizing determiner 130 determines the shot size of the shot based on at least one image among images constituting each shot. Decide. In one embodiment, the shot size determination unit 130 determines the shot size of the shot based on a shot boundary image (eg, a beginning image of a shot or an end image of a shot) positioned at a shot boundary among each shot You can. Hereinafter, for convenience of description, the shot size determination unit 130 will be described as determining the shot size based on the shot boundary image.

The shot size determination unit 130 may extract the representative object from the shot boundary image and determine the shot size of the corresponding shot based on the size of the extracted representative object. The shot size determination unit 130 stores the shot size determined for each shot in the shot size database 135 together with the shot boundary image of the corresponding shot.

Hereinafter, the configuration of the shot size determination unit 130 according to the present invention will be described in more detail with reference to FIG. 3.

3 is a block diagram showing the configuration of a shot size determination unit according to an embodiment of the present invention. 3, the shot size determination unit 130 according to an embodiment of the present invention includes a target area detection unit 310, a representative object determination unit 320, and a shot size determination unit 330 . In addition, the shot size determination unit 130 according to the present invention may further include a candidate target area determination unit 340.

The target region detection unit 310 extracts at least one target region corresponding to a person from the shot boundary image using a bounding box image having a predetermined size in the shot boundary image. For example, the target area detection unit 310 determines the upper left vertex and lower right vertex of the bounding box within the shot boundary image as shown in FIG. 4A, draws bounding boxes close to each person's movement direction, and targets each person. Regions 410 to 440 are extracted. 4A, a plurality of target areas 410 to 440 are extracted because a plurality of people exist in the shot boundary image.

The representative object determination unit 320 determines a target region having the largest size among at least one target region extracted by the target region extraction unit 310 as a representative object. For example, as illustrated in FIG. 4B, when a plurality of target regions 410 to 440 are extracted, the representative object determining unit 320 is a target region having the largest width among the detected target regions 410 to 440. 410 is determined as a representative object.

The shot size determination unit 330 detects the shot size of the representative object determined by the representative object determination unit 320 and determines the detected shot size of the representative object as the shot size of the corresponding shot. For example, the shot size is Extreme Close up, Close up, Bust Shot, West Shot, Medium Shot, Knee Shot, Full Shot ), Long Shot (Lone Shot), and Extreme Long Shot (Extreme Long Shot).

In one embodiment, the shot size determination unit 330 may detect the shot size of the representative object by comparing the object in the reference image set for each shot size with the representative object using a deep learning-based learning model. For example, in the case of the shot boundary images illustrated in FIGS. 4A and 4B, since the shot size of the representative object is a full shot, the shot size determination unit 330 may determine the shot size of the corresponding shot as a full shot. As another example, in the case of the shot boundary image illustrated in FIG. 4C, since the shot size of the representative object is close-up, the shot size determination unit 330 may determine the shot size of the corresponding shot as a close-up shot.

Meanwhile, the target area having the largest size in the flow of the movie may not correspond to the representative object. For example, as illustrated in FIGS. 5A and 5B, when the target area 510 having the largest size does not include the front of the person, the second largest target area 520 including the front of the person is representative. It can correspond to an object. Accordingly, the shot size determining unit 130 according to the present invention may further include a candidate target region determining unit 340 as illustrated in FIG. 3 to accurately determine a representative object.

When a plurality of target areas are detected by the target area detection unit 310, the candidate target area determination unit 340 determines candidate target areas including a person photographed from the front among the detected target areas. In one embodiment, the candidate target area determining unit 340 may select target areas including a person photographed from the front of the target areas using an indexing technique.

For example, the candidate target area determining unit 340 does not determine the candidate target area in FIG. 5A and 5B because the first target area 510 does not include a person photographed from the front, and the person photographed from the front is not determined. Only the included second target area 520 is determined as a candidate target area. Accordingly, the first target area 510 having the largest size but not including the person photographed from the front is not determined as a representative object, and the size is the second largest but includes the person including the person photographed from the front. 2 The target area 520 is determined as a representative object.

According to this embodiment, if the representative object determining unit 320 has a plurality of candidate target regions, the candidate target region having the largest size among the plurality of candidate target regions is determined as the representative object. For example, in FIGS. 5A and 5B, since there is one candidate target area, the second target area 520, the representative object determination unit 320 determines the second target area 520 as a representative object, and the shot size determination unit 330 Determines the medium shot, which is the shot size of the determined representative object, as the shot size of the shot.

Meanwhile, in the above-described embodiment, it has been described that the target area detection unit 310 detects only the area including the person in the shot boundary image as the target area. However, in a modified embodiment, as illustrated in FIG. 6, when an area corresponding to a person is not detected in the shot boundary image, the target area detection unit 310 sets the area including the predetermined object as the target area. It may be detectable.

Referring back to FIG. 1, the climax section determination unit 140 extracts a shot size change pattern of each shot determined by the shot size determination unit 130 from the shot size database 135, and the shot size of each extracted shot The climax section of the image data is determined based on the change pattern.

Specifically, the climax section determiner 140 calculates a shot size change pattern of consecutive shots constituting a scene or a sequence based on the shot size of each shot. To this end, the climax section determiner 140 may perform scoring for each shot based on the shot size. As an example, the climax section determining unit 140 may reduce the order of shot sizes (eg, extreme close up, close up, bust shot, bust shot, west shot, medium shot) Scoring for each shot may be performed to increase the score according to (Midium Shot), Knee Shot, Full Shot, Long Shot, and Extreme Long Shot order.

For example, the climax section determiner 140 assigns 1 point to the shot when the shot size is extreme close-up, and 2 points to the shot when the shot size is close-up, and the shot when the shot size is the bust shot Assign 3 points to, if the shot size is West Shot, assign 4 points to the shot, if the shot size is medium shot, assign 5 points to the shot, and if the shot size is knee shot, 6 to the shot You can assign 7 points to the shot if the shot size is full shot, 8 points to the shot if the shot size is long shot, and 9 points to the shot if the shot size is extreme long shot. have.

The climax section determiner 140 may generate a change pattern of the shot size based on the score allocated for each shot. The change pattern of the shot size generated by the climax section determining unit 140 is illustrated in FIG. 7. In FIG. 7, the shot size is shown to include only extreme close-up shot, close-up shot, medium shot, full shot, long shot, and extreme long shot, but this is only one example, and the shot size is bust shot, west shot, knee shot, etc. It may include.

In addition, in the above-described embodiment, the climax section determination unit 140 has been described as performing scoring for each shot so that the score increases in the order in which the shot size decreases, but unlike this, the climax section determination unit 140 Scoring may be performed for each shot so that the score increases in the order in which the shot sizes increase.

Meanwhile, when the shot size change pattern is calculated, the climax section determiner 140 determines the climax section based on the shot size change pattern. Specifically, the climax section determiner 140 may determine the point where the change in the score allocated for each shot is the largest as the starting point of the climax based on the shot size change pattern. In the example shown in FIG. 7, the climax section determiner 140 may determine a point A having the largest change in score assigned to each shot as a starting point of the climax.

On the other hand, if there are a plurality of points having the largest change in the score allocated for each shot, the climax section determiner 140 may further select the starting point of the climax by additionally using a change in camera angle in each shot.

Specifically, the climax section determiner 140 may perform scoring for each shot based on the camera angle, similar to scoring of each shot based on the shot size. In accordance with this embodiment, the climax section determining unit 140 starts the climax of the points having the largest change in the score of each shot determined based on the camera angle among the plurality of points having the largest change in the score allocated for each shot. You can decide by time.

Meanwhile, the climax section determiner 140 may determine a point at which the scene change occurs as the end point of the climax. For example, the climax section determiner 140 may determine a point where the shot size is maintained for a predetermined time as an extreme long shot as a point where a scene change occurs. For example, in FIG. 7, the climax section determiner 140 may determine point B as the end point of the climax. According to this example, a section between points A and B is determined as a climax section.

The database generating unit 150 generates a database 160 for image analysis by mapping and storing the shot size change pattern of the climax section determined by the climax section determining unit 140 with identification information of the image data. For example, the database generating unit 150 may map and store identification information of the corresponding shot and shot size information for each shot included in the climax section. At this time, the identification information of each shot may be sequentially assigned according to the time sequence of each shot, and the shot size information may be a score given to the shot based on the shot size.

In one embodiment, the database generating unit 150 may store the shot size change pattern of the climax section together with genre information of the corresponding image data. In the present invention, the reason why the database generation unit 150 stores the shot size change pattern of the climax section together with the genre information of the corresponding image data is that the image producer easily provides only the image analysis information for the genre that the image producer wants to generate. It is to do.

As described above, in the present invention, since the database generation unit 150 matches the shot size change pattern of the climax section with genre information of the corresponding image data and generates the stored image analysis database 160, the image producer analyzes the image. By comparing the shot size change pattern of the image data stored in the usage database 160 with the shot size change pattern of the specific image data, it is possible to predict whether or not the success of the specific image data is successful.

 In addition, when an image producer wishes to generate a video of a specific genre, it is possible to easily generate video data by extracting a shot size change pattern of the video data of the genre from the video analysis database 160 and using the same as it is. In addition, when the shot size change pattern of the image data that has been successful is extracted and used, it is possible to increase the likelihood of the image that the image producer wants to generate.

Hereinafter, an image data analysis method using a shot size change pattern according to the present invention will be described with reference to FIG. 8.

8 is a flowchart illustrating an image data analysis method using a shot size change pattern according to an embodiment of the present invention. The image data analysis method using the shot size change pattern illustrated in FIG. 8 may be performed by an image data analysis system (hereinafter referred to as an 'analysis system') using the shot size change pattern illustrated in FIG. 1.

First, the analysis system detects a boundary between shots from image data using a shot boundary detection algorithm (S800). In this case, the image data subject to the detection of the shot boundary may be scenario-based image data such as a movie, drama, or animation, or image data such as a music video. The image data may be received from the outside through a network such as the Internet or directly from an image database included in the analysis system, or may be received from a separate storage medium such as a mobile device, hard disk, or USB (Universal Serial Bus).

In one embodiment, the analysis system may detect a shot boundary using a color histogram algorithm, an edge histogram algorithm, and an optical flow tracking algorithm. The reason why the analysis system detects a shot boundary by mixing a plurality of shot boundary detection algorithms is that in the case of image data such as a movie, a scene change technique is often used in the connection portion between a shot and a shot, and it is the same scene due to various uses of lighting. In spite of this, the brightness of the lighting can change dramatically, and when the object moves while in extreme close-up, it can be judged that the entire image of the screen is changed. This is because the correct shot boundary is not detected when is detected.

Thereafter, the analysis system extracts at least one target area corresponding to the person from at least one of the images constituting each shot (S810). In one embodiment, the analysis system may extract a target area from a shot boundary image located at a shot boundary among each shot.

Specifically, the analysis system extracts at least one target area corresponding to a person from the shot boundary image using a bounding box image having a predetermined size in the shot boundary image. When the target area corresponding to the person is not detected in the shot boundary image, the analysis system may detect the area including the predetermined object as the target area.

Thereafter, the analysis system determines the target area having the largest size among the at least one target areas extracted in S810 as the representative object (S820), and then determines the determined shot size of the representative object as the shot size of the corresponding shot (S830). .

Thereafter, the analysis system determines a climax section of the image data using the change pattern of the shot size determined in S830 (S840). In one embodiment, the analysis system may assign a score for the shot based on the shot size of each shot to determine the climax interval. At this time, the analysis system may allocate a score for each shot so that the score increases in the order in which the shot size decreases or a score for each shot so that the score increases in the order in which the shot size increases.

Based on the score assigned to each shot, the analysis system may determine the point at which the largest score change among the plurality of shots occurs as the start point of the climax, and determine the point at which the scene change occurs as the end point of the climax. have. With regard to determining the point where the scene change occurs, the analysis system may determine the point where the shot size is maintained as an extreme long shot for a predetermined time as the point where the scene change occurs.

On the other hand, when there are a plurality of points having the largest change in the score allocated for each shot, the analysis system may further select the starting point of the climax by additionally using a change in camera angle in each shot. On the other hand, if there are a plurality of points having the largest change in the score allocated for each shot, the climax section determiner 140 may further select the starting point of the climax by additionally using a change in camera angle in each shot.

For example, the analysis system performs scoring for each shot based on the camera angle, similar to scoring of each shot based on the shot size, and the camera angle among a plurality of points having the largest change in score allocated for each shot The point where the change in the score of each shot determined based on is the largest may be determined as the starting point of the climax.

Meanwhile, the analysis system may generate a change pattern of the shot size based on the score allocated for each shot.

Thereafter, the analysis system maps and stores the determined shot size change pattern of the climax section with the identification information of the image data (S850).

For example, the analysis system may map and store identification information of the corresponding shot and shot size information for each shot included in the climax section. At this time, the identification information of each shot may be sequentially assigned according to the time sequence of each shot, and the shot size information may be a score given to the shot based on the shot size.

Meanwhile, although not illustrated in FIG. 8, the analysis system may further include determining candidate target areas including a person photographed from the front among at least one extracted target areas. The reason the analysis system according to the present invention determines candidate target areas among the target areas is that the target area having the largest size in the flow of the movie may not correspond to the representative object.

According to this embodiment, when a plurality of target areas detected through S830 are extracted, the analysis system determines candidate target areas including a person photographed from the front among the extracted plurality of target areas, and determines the determined candidate target areas The target area with the largest size is determined as the representative object. At this time, the analysis system may select target regions including a person photographed from the front of the target regions using an indexing technique.

Those skilled in the art to which the present invention pertains will understand that the above-described present invention can be implemented in other specific forms without changing its technical spirit or essential features.

For example, the image data analysis method using the shot size change pattern illustrated in FIG. 8 may be implemented in a program form.

Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modifications derived from the meaning and scope of the claims and equivalent concepts are included in the scope of the present invention. do.

100: image data analysis system using a shot size change pattern
110: image data receiving unit 120: shot boundary detection unit
130: shot size determining unit 140: climax section determining unit
150: database generation unit 160: database for image analysis
310: target area detection unit 320: representative object determination unit
330: shot size determination unit 340: candidate target area determination unit

Claims (16)

A shot boundary detection unit that detects a boundary between shots from image data using a shot boundary detection algorithm;
A shot size determination unit that determines a shot size of a corresponding shot based on a size of a representative object extracted from at least one of the images constituting each shot;
A climax section determination unit for determining a climax section of the image data using the change pattern of the shot size; And
And a database generator configured to map and store the determined shot size change pattern of the climax section with identification information of the image data,
The shot size determination unit,
Within the image, at least one target region corresponding to a person is extracted using a bounding box image, and candidate target regions including a person photographed in front of the extracted at least one target regions are determined and determined The target area having the largest size among candidate target areas is determined as the representative object, and the shot size of the representative object is determined as the shot size of the corresponding shot,
An image data analysis system using a shot size change pattern, characterized in that the shot size of the representative object is obtained by comparing an object in a reference image set for each shot size with the representative object based on a deep learning-based learning model. According to claim 1,
The climax section determining unit,
Based on the shot size of each shot, a score is assigned to the shot, and the point at which the largest score change occurs among the plurality of shots is determined as the start point of the climax, and the point at which the scene change occurs is the end of the climax An image data analysis system using a shot size change pattern characterized by determining the viewpoint. According to claim 2,
The climax section determining unit,
When there are a plurality of points where the largest score change occurs, an image data analysis system using a shot size change pattern is characterized in that the starting point of the climax is determined by additionally using a camera angle change in each shot. According to claim 3,
The climax section determining unit,
A score is assigned to a corresponding shot based on the camera angle, and a point in which a change in score allocated based on the camera angle is the greatest among a plurality of points where the largest score change occurs is a starting point of the climax. Video data analysis system using a shot size change pattern, characterized in that to determine. delete delete delete According to claim 1,
The shot size determination unit,
An image data analysis system using a shot size change pattern, characterized in that the target target regions including a person photographed in front of the target regions are determined using the indexing technique, and the candidate target regions are determined. According to claim 1,
The shot size determination unit,
If an area corresponding to a person is not detected in the image, at least one target area including a predetermined object is detected, and a target area having the largest size among the extracted at least one target areas is determined as a representative object. , The image data analysis system using the shot size change pattern characterized in that the determined shot size of the representative object is determined as the shot size of the corresponding shot. According to claim 1,
The shot boundary detection unit,
An image data analysis system using a shot size change pattern, characterized in that the shot boundary is detected using a color histogram algorithm, an edge histogram algorithm, and an optical flow tracking algorithm. Detecting a boundary between shots from image data using a shot boundary detection algorithm;
Determining a shot size of a corresponding shot based on a size of a representative object extracted from at least one image among images constituting each shot; And
And determining a climax section of the image data using the change pattern of the shot size,
Determining the shot size,
Extracting at least one target area corresponding to a person from the image using a bounding box image;
Determining candidate target areas including a person photographed from the front among the extracted at least one target areas;
Determining a target area having the largest size among the determined candidate target areas as the representative object; And
And determining a shot size of the determined representative object as a shot size of the corresponding shot,
The shot size of the representative object is obtained by comparing an object in a reference image set for each shot size with the representative object based on a deep learning-based learning model, and analyzing the image data using the shot size change pattern. The method of claim 11,
In the step of determining the climax section,
Based on the shot size of each shot, a score is assigned to the shot, and the point at which the largest score change occurs among the plurality of shots is determined as the start point of the climax, and the point at which the scene change occurs is the end of the climax Video data analysis method using a shot size change pattern, characterized in that determined by the viewpoint. delete delete The method of claim 11,
In the step of detecting the shot boundary,
A method for analyzing image data using a shot size change pattern, characterized in that the shot boundary is detected using a color histogram algorithm, an edge histogram algorithm, and an optical flow tracking algorithm. The method of claim 11,
And mapping and storing the determined shot size change pattern of the climax section with identification information of the image data.

Images