KR102131751B1 - Method for processing interval division information based on recognition meta information and service device supporting the same - Google Patents

Abstract

The service device according to an embodiment of the present invention may include a memory and a processor functionally connected to the memory. The processor acquires image content stored in the memory, extracts recognition meta information from the acquired image content, clusters detection time information of the extracted recognition meta information, and divides a section according to the distribution of clustered clusters. After that, it may be set to generate section classification information.

Description

Method for processing interval division information based on recognition meta information and service device supporting the same}

The present invention relates to section division information processing of video content, and more particularly, to a method for obtaining section operation information using recognition meta information about image content, a method of operating the same, and a service device supporting the section.

The division of a certain section within the video content, in particular, the division by sequence is a task required for the purpose of advertisement insertion and the like. Conventionally, a method of using audio information in classifying a section of video content has been proposed. For example, a point where the previous sound source ends and a new sound source starts may be found and used as section classification information. However, since the end and start points of the sound source are often different from the point where the contents of the corresponding video content are diverged, there is a problem in that the sequence classification using the sound source is less accurate.

As described above, a technique of searching for sequence information corresponding to a section divided into a certain story unit or a section divided into a context unit is the most basic and core technology in service of media. A sequence is a division unit of content that can be divided into the concept of context in content, and can be used when an intermediate advertisement or the like needs to be inserted at a natural point during content playback in a media service. Accordingly, the sequence should be selected as a main point in which an intermediate advertisement can be naturally inserted without being awkward in context. However, in order to detect a sequence, since a person needs to recognize contextual content and distinguish natural flows in content, it is necessary for a person to directly watch the content and artificially classify the sequence. That is, unlike a shot that is sensed by a change in the camera, the sequence is difficult to define simply by visual classification, and must be able to be dynamically classified according to the context, the flow of content, and the requirements of the service. It should be done naturally in context. Therefore, there is a problem in that it is difficult to distinguish a sequence without performing direct division as many as the number of sequences that a person wants to manually divide.

Korean Patent Publication No. 10-2016-0110433, registered on September 21, 2016 (name: content analysis method and device)

The present invention is to satisfy the above-described needs, and performs natural sequence classification by using various recognition meta information detected from the video content, and based on this, segment division using recognition meta information capable of providing various sequence-based services It is to provide an information processing method and a service device supporting the information.

The service device according to an embodiment of the present invention may include a memory and a processor functionally connected to the memory. The processor acquires image content stored in the memory, extracts recognition meta information from the acquired image content, clusters detection time information of the extracted recognition meta information, and divides a section according to the distribution of clustered clusters. Thereafter, it may be set to generate section classification information.

In particular, the processor may be configured to generate recognition meta information by integrating recognition meta information repeatedly appearing within a specified time range, and to perform clustering on the integrated recognition meta information.

Alternatively, the processor extracts the recognition meta information using the plurality of recognition technologies, and after performing integrated recognition meta information generation for each extracted recognition meta information, the start and end points of the recognition meta are It may be configured to process similar input metadata of a cluster within a specified range and process input values of one cluster.

Meanwhile, the processor may be configured to determine the type of recognition meta to be extracted by checking content-related information of the video content.

In this case, the processor may be configured to extract a sound source recognition meta and a person recognition meta when the image content is movie content.

Alternatively, the processor may be configured to extract a recognition meta and a sound source recognition meta for a specific object or a particular animal when the video content is a documentary.

Meanwhile, the service device may further include a communication circuit forming a communication channel with an external electronic device, and the processor receives the sequence number from the external electronic device through the communication circuit, and according to the sequence number, It can be set to determine the number of clusters to be classified through clustering.

In this case, the processor assigns a ranking of clusters according to the degree of change according to the appearance of the recognition meta based on the clustering distribution, and selects clusters within a certain ranking according to the number of sequences as sequences to be used for generating section classification information Can be set.

According to an embodiment of the present invention, a method for processing section classification information using a recognition meta includes: a service device extracting recognition meta information from image content stored in a memory, and integrating the recognition meta information according to a specified criterion to integrate the recognition meta. The method may include generating information, clustering detection time information of the integrated recognition meta information, classifying sections according to the distribution of the clustered clusters, and generating section classification information.

Here, the method may further include assigning a ranking of clusters according to the degree of change according to the appearance of the recognition meta based on the clustering distribution.

In this case, the step of generating the section classification information includes receiving a sequence number from an external electronic device through a communication circuit, and selecting clusters within a certain ranking according to the sequence number as sequences to be used to generate section classification information. It may further include a step.

A computer recording medium storing at least one instruction executed by a processor according to an embodiment of the present invention stores the at least one instruction, and the at least one instruction is recognized by the service device in the image content stored in the memory. Extracting meta information, integrating the recognition meta information according to a specified criterion to generate integrated recognition meta information, clustering detection time information of the integrated recognition meta information, and distribution of the clustered clusters Accordingly, after classifying the sections, it may be set to perform an operation for generating section classifying information.

The present invention can extract the recognition meta information within the video content, integrate the extracted recognition meta information, and then allow the specified number of sequences to be properly classified through clustering, so that the video content is faster and faster without artificial separation of persons. It is possible to provide a section division function and to provide a section classification capability of good quality by providing a uniform criteria for classifying video contents.

Other effects of the present invention will be mentioned in the following description.

1 is a diagram illustrating an example of a network environment related to image content section classification information processing according to an embodiment of the present invention.
2 is a view showing a schematic form of a service device according to an embodiment of the present invention.
3 is a diagram illustrating an example of a processor of a service device according to an embodiment of the present invention.
4 is a graph showing an example of recognition meta information recognized by a person in a specific video content.
FIG. 5 shows the integration result of the recognition meta information shown in FIG. 4 according to a certain criterion.
FIG. 6 shows the values obtained by manually classifying and tagging the combined recognition meta information described in FIG. 5, and FIG. 7 shows values obtained by combining the tagged information according to a specified rule.
FIG. 8 shows a value configured by configuring a sequence candidate based on the divided clusters and dynamically designating a median value (k) of clustering according to the requested number of sequences.
9 shows an example of a method for classifying sequences based on clustering maximum/minimum values.
10 is a diagram showing an example of a case where the start of a sequence is set to a minimum value.
11 is a diagram illustrating an example of a method for processing section classification information using recognition meta information according to an embodiment of the present invention.

In order to clarify the features and advantages of the problem solving means of the present invention, the present invention will be described in more detail with reference to specific embodiments of the present invention shown in the accompanying drawings.

However, in the following description and attached drawings, detailed descriptions of well-known functions or configurations that may obscure the subject matter of the present invention are omitted. In addition, it should be noted that the same components throughout the drawings are denoted by the same reference numerals as much as possible.

The terms or words used in the following description and drawings should not be interpreted as being limited to ordinary or dictionary meanings, and the inventor can appropriately define the concept of terms for explaining his or her invention in the best way. Based on the principle of being present, it should be interpreted as meanings and concepts consistent with the technical spirit of the present invention. Therefore, the embodiments shown in the embodiments and the drawings described in this specification are only the most preferred embodiments of the present invention, and do not represent all of the technical spirit of the present invention, and can replace them at the time of this application. It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers such as first and second are used to describe various components, and are used only for the purpose of distinguishing one component from other components, and to limit the components It is not used. For example, the second component may be referred to as a first component without departing from the scope of the present invention, and similarly, the first component may also be referred to as a second component.

In addition, the terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, terms such as "comprises" or "have" described herein are intended to indicate that there are features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, or one or more thereof. It should be understood that the above or other features or numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

In addition, terms such as “part”, “group”, and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. In addition, "a (a or an)", "one (one)", "the (the)" and similar related terms in the context of describing the present invention (especially in the context of the following claims) is different herein. It may be used in a sense including both singular and plural unless indicated or clearly contradicted by context.

In addition to the above-mentioned terms, specific terms used in the following description are provided to help understanding of the present invention, and the use of these specific terms may be changed into other forms without departing from the technical spirit of the present invention.

In addition, embodiments within the scope of the present invention include computer readable media having or carrying computer-executable instructions or data structures stored on computer-readable media. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. By way of example, such computer readable media may be in the form of RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or computer executable instructions, computer readable instructions or data structures. Physical storage media, such as any other media that may be used to store or transfer certain program code means in, and can be accessed by, general purpose or special purpose computer systems. .

1 is a diagram illustrating an example of a network environment related to image content section classification information processing according to an embodiment of the present invention.

Referring to FIG. 1, a network environment 10 related to image content section classification information processing according to an embodiment of the present invention includes a network 50, a terminal device 100, a content providing server 300, and a service device 200 It may include.

In the above-mentioned network environment 10 of the present invention, when the terminal device 100 accesses the service device 200 through the network 50, the service device 200 uses the content designated on the terminal device 100. In response to the search request information requested by the terminal device 100, the service device 200 may search for content corresponding to the search request information and provide it to the terminal device 100. In this process, the service device 200 may provide the terminal device 100 with content having a specified number of sequences using a recognition meta. Alternatively, the service device 200 may provide the terminal device 100 with content in which a specified advertisement or additional information is inserted in each of the specified number of sequences.

The network 50 serves to transmit data for data transmission and reception between the terminal device 100 and the service device 200, or transmits data for data transmission and reception between the content providing server 300 and the service device 200 It can play a role. These networks 50 are wired communication such as Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coaxial Cable), FTTC (Fiber to The Curb), FTTH (Fiber To The Home), depending on the system implementation method. You can also use the method, WLAN (Wireless LAN), Wi-Fi (Wi-Fi), Wibro (Wibro), Wimax (Wimax), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution Advanced) may be used, and in addition to the above-described communication methods, other widely known or future communication methods may be included. Alternatively, when the terminal device 100 receives a broadcast signal transmitted from the service device 200 using a broadcast antenna or based on a set-top box, the network 50 includes an example of a broadcast network can do.

The terminal device 100 refers to a user device capable of transmitting and receiving various data with the service device 200 through the network 50 according to a user's operation. The terminal device 100 may perform video content transmission and reception through the network 50 (Internet network or broadcast network), execute a memory and various programs for storing and processing programs and protocols for video content transmission and reception, and calculate and It may be provided with a processor for controlling. In addition, the terminal device 100 of the present invention may be implemented in various forms. For example, the terminal device 100 described in the present specification is a mobile phone such as a smart phone, a tablet PC, a personal digital assistants (PDA), a portable multimedia player (PMP), or an MP3 player. Of course, a fixed terminal such as a smart TV, IPTV, IPTV set-top, laptop computer, desktop computer, etc. may be used, and if it is a device capable of receiving data related to image content according to the present invention, Any device can be used as the terminal device 100 according to an embodiment of the present invention.

Meanwhile, the terminal device 100 according to an embodiment of the present invention includes a configuration of a communication circuit, a display, an input device, an audio device, a processor, a memory, and the network 50 ( For example: it may be connected to the service device 200 through an Internet network or a broadcast network. For example, the terminal device 100 executes a web browser or an application corresponding to the user input, and accesses the service device 200 based on pre-entered Internet address information or address information input by the user. Can. The terminal device 100 may receive a specified web page from the service device 200 and output it on the display. For example, the web page may include a virtual page capable of searching for at least one video content. As another example, the terminal device 100 includes an IPTV or an IPTV set top, and may receive and output video content transmitted by the service device 200 through an antenna.

The content providing server 300 may provide at least one content to the service device 200. In this regard, the content providing server 300 may receive content from a content producer or the like, and store and manage it. When the new content is collected, the content providing server 300 may provide the collected content to the service device 200. The content providing server 300 may provide video content and content-related information to the service device 200. For example, the content-related information may include information such as content type, genre, director, actor, storyline, and highlight scene.

The service device 200 may have a communication standby state so that the terminal device 100 can access through the network 50. When the terminal device 100 accesses, the service device 200 may provide the terminal device 100 with a content use screen (or virtual page) capable of using at least one content. The service device 200 may receive content search request information from the terminal device 100, search for content corresponding to the content search request information, and provide the searched content to the terminal device 100. In this process, the service device 200 may provide the terminal device 100 with the video content including the clustering section classification information or the video content with the intermediate advertisement inserted thereon according to the embodiment of the present invention. Can. Meanwhile, the service device 200 may form a communication channel with the content providing server 300 in connection with content collection, and receive and store video content and content related information from the content providing server 300.

The above-described technology related to image content section classification information processing of the present invention collects recognition meta information of video content using at least one recognition technology, and then integrates, clusters (or groups) a plurality of section classification information ( Or, by generating a plurality of sequence information, it is possible to perform the sequence classification of the video content on a good basis, and according to the required number (for example, the request of the advertisement provider or the request of the content provider, the administrator of the service device, etc.) Number) section can be easily obtained. Accordingly, the present invention can provide viewers with an intermediate advertisement at a point in time that can minimize the influence on viewing content, thereby reducing viewers' reluctance to additional information such as an intermediate advertisement and maximizing the recognition rate or advertisement effect of the additional information. have. In addition, the present invention increases the reliability of the sequence classification time, and provides various information at the time, thereby maximizing the effect of delivering information to viewers.

2 is a view showing a schematic form of a service device according to an embodiment of the present invention.

Referring to FIG. 2, the service device 200 of the present invention may include a communication circuit 210, a memory 240 and a processor 250.

The communication circuit 210 may form a communication channel of the service device 200. For example, the communication circuit 210 may form a communication channel with the terminal device 100 in response to a request for access to the terminal device 100. The communication circuit 210 may provide a virtual page (eg, a content usage screen) designated in the terminal device 100 in response to the control of the processor 250. The communication circuit 210 may receive search request information related to content search from the terminal device 100. The communication circuit 210 may provide the terminal device 100 with specified content corresponding to the search request information. The communication circuit 210 may transmit the content selected by the terminal device 100 request to the terminal device 100 in a streaming method or a download method. The communication circuit 210 may form a communication channel between the service device 200 and the content providing server 300. The communication circuit 210 is video content or content-related information stored in the content providing server 300 (for example, information describing video content, such as video content title, genre, director, actor, storyline, highlight, etc.) Can receive.

The memory 240 may store various data, programs, and algorithms related to the operation of the service device 200. For example, the memory 240 may include image content DB 241 and sequence information 243.

The video content DB 241 may include at least one video content. For example, the image content DB 241 may store various image contents such as a movie, a drama, a music video, and a documentary. The image contents stored in the image content DB 241 may be mapped to section division information stored in the sequence information 243.

The sequence information 243 may include section classification information related to image contents stored in the image content DB 241. The section classification information may include viewpoint information for classifying sequences applied to specific video content. Section classification information included in the sequence information 243 may be used for advertisement insertion of video content. In this regard, the memory 240 may include at least one virtual advertisement that can be inserted in the middle of playing the video content.

The processor 250 may perform data transfer or data processing related to the operation of the service device 200. For example, the processor 250, in relation to the section classification information processing and content providing function of the present invention, forms a communication channel with the content providing server 300, receives video content or content related information from the content providing server 300 , It is possible to process the connection of the terminal device 100, provision of a content use screen, reception of search request information from the terminal device 100, and provision of content corresponding to the received search request information. In particular, the processor 250 of the present invention extracts recognition meta information from scene analysis of image content, integrates the extracted recognition meta information, and clusters the integrated recognition meta information (or related to the generation of section classification information). Grouping). The processor 250 may include a configuration as shown in FIG. 3.

3 is a diagram illustrating an example of a processor of a service device according to an embodiment of the present invention.

Referring to FIG. 3, the processor 250 includes a content collection unit 251, a meta information extraction unit 253, a meta information integration unit 255, a sequence extraction unit 257, and a service provision unit 259 can do.

The content collection unit 251 may collect video content, store the collected video content in the memory 240, and process the original file to be recognized by at least one recognition technology. For example, the content collection unit 251 may form a communication channel with the content providing server 300 providing specific video content, and collect at least one video content from the content providing server 300. When the video content is collected, the content collection unit 251 extracts an image in which a specific object (eg, actor, animal, specific object, etc.) is recognized with respect to the collected video content, or a specific sound source (eg, background music) is The detected image can be extracted. When extracting an image in which a specific object is recognized, the content collection unit 251 converts the video content into a frame image for each unit required, or when recognizing music, extracts and divides only the sound source WAV file from the VOD file of the video content Basic operations for extracting recognition meta information, such as may be performed. The content collection unit 251 may collect video content from the content providing server 300 in real time or at regular intervals, and perform the above-described basic operation on the collected video content.

The meta information extraction unit 253 extracts recognition meta information for each image content stored in the image content DB 241 based on at least one recognition technology. The meta information extraction unit 253 is a part necessary for extracting various recognition meta information from the image content, and extracts various recognition meta information such as a person or a sound source from the image content for each recognition unit according to a designated recognition technology. The recognition unit may be a certain number of frame intervals in the case of a person, and in the case of a sound source, it may be a minimum time unit capable of detecting the type of the sound source. The recognition unit may vary depending on the recognition method. For example, the meta information extraction unit 253 may use face recognition technology as a recognition technology for extracting the meta information. The meta information extraction unit 253 detects an image in which people or actors appear from the video content using face recognition technology, and extracts recognition meta information (eg, type and extraction time information) of each image. can do. Alternatively, the meta information extraction unit 253 may use a sound source title extraction technology. In this regard, the meta-information extraction unit 253 collects at least a portion of a sound source that can be compared with a previously stored whole sound source among a part of the sound source included in the video content as a recognition unit, and uses the collected partial sound source to generate the entire sound source. You can extract the title. The meta information extraction unit 253 may combine and operate various deep learning-based recognition engines to extract recognition meta information. Recognition meta information extracted as a unique unit time for each recognition meta information may be integrated in a time unit having a significant length in an integration unit of the next step.

The meta-information integration unit 255 performs an integration operation to make the recognition meta-information divided by the recognition unit time extracted by the meta-information extraction unit 253 into a single meaningful section. For example, in the case of person recognition, when a person is recognized at an interval of 10 frames in an image having 29.97 fps per second, and repeatedly appears within 10 seconds, the meta-information integration unit 255 is configured as one continuous appearance for the person. You can connect. This process can be used to change the level of the recognized meta information in the future by matching different meta units to similar levels and splitting them too finely.

Recognition meta information such as person/sound source/situation information in a specific video content work is different depending on the recognition unit of the recognition engine, and when it is expressed in the form of a whole graph, it can represent the recognition result in a detailed unit as shown in FIG. 4. 4 is a graph showing an example of recognition meta information recognized by a person in a specific video content. That is, when recognizing a specific video content in units of 10 frames and integrating every 100 frames, detailed recognition meta information may appear as shown in FIG. 4. Source raw data resulting from the actual recognition may be better divided than the graph shown in FIG. 4 according to each recognition meta feature. If specific information is displayed using the results shown in FIG. 4, it is possible to obtain a result of repeating that the specific information repeatedly appears and disappears at each time point whether or not a specific person appears.

On the other hand, when the recognition meta information shown in FIG. 4 is integrated at a significant interval, a distribution as shown in FIG. 5 may appear. FIG. 5 shows the integration result of the recognition meta information shown in FIG. 4 according to a certain criterion. For example, the graph illustrated in FIG. 5 shows the result of merging into one continuous section when a specific person appears during a designated time interval (for example, during a 25 second or 30 second interval) based on the recognition meta information of FIG. 4. May be In this way, it is possible to define the start and end points of a character in a meaningful form by tying them up in meaningful units, and refine the recognition metadata in the form of appearing and disappearing repeatedly insignificantly.

The sequence extracting unit 257 is a part that finds a position to be a sequence point by using each integrated recognition meta information. The sequence extraction unit 257 detects the start and end points of various recognition meta appearance times for the recognition meta information that the meta information integration unit 255 previously integrated between the certain recognition meta intervals. The start and end points of the integrated recognition meta information may serve as the most important input values of sequence information extraction. That is, a new appearance of a character, a change of a background sound source, or other meta-information value of a situation may be an important parameter for classifying a sequence. The sequence extraction unit 257 of the present invention can extract the ranking of the point where the overall recognition meta changes frequently. The sequence extracting unit 257 may select a ranking that can be a candidate for a sequence that is contextually changed, and may determine a contextual change in order of severe change.

The sequence extraction unit 257 identifies the transformation points of various recognition meta, that is, start and end points, and performs k-means nearest clustering by using these points. Through the K-means nearest clustering, the sequence extraction unit 257 may perform a task of collecting each input value into K clusters to be divided. Since the recognition meta information used for extracting the section classification information applied to the present invention is a time value of the time code, the one-dimensional clustering method of the x-axis without the y-axis may be applied. The sequence extracting unit 257 may dynamically generate k sequence clusters according to the number of contexts to be divided, that is, the length of video content or service requirements. In this regard, the sequence extraction unit 257 may assign rankings to change values such as start and end points of various types of recognition meta information, and generate section classification information according to the number of clusters. In this operation, the ranking may have a higher ranking value when the change values of the start and end points of the various recognition meta are relatively high, and when the change values of the start and end points of the various recognition meta are relatively low, the ranking As a result, a low ranking value may be given.

In addition, when generating section classification information, input values used may also be dynamically designated. For example, in the case of content such as a documentary in which a person does not appear, the sequence extraction unit 257 recognizes only animal recognition data or sound source information, except for the cast information meta, as the values of inputting the start and end points of the meta information Decide, and based on this, k clusters can be generated. Alternatively, in the case of movie content having a significant sound source, the sequence extraction unit 257 removes object information such as animals or objects, determines only the information of the sound source and the cast as recognition meta information, and a specified number based on the determined recognition meta information Clustering may be performed to calculate a cluster of. As described above, the sequence cluster operated by the sequence extraction unit 257 of the present invention can dynamically reflect the dynamic application according to the service requirements to be shared and meta values according to VOD content characteristics. In order to apply the above-described function, the sequence extracting unit 257 may check content-related information, extract what type of recognition meta-information according to the content-related information, and determine what to use for sequence detection.

FIG. 6 shows the values obtained by manually classifying and tagging the combined recognition meta information described in FIG. 5, and FIG. 7 shows values obtained by combining the tagged information according to a specified rule. For example, among the recognition meta information acquired through a plurality of recognition technologies, the start and end points of the actor and music meta may overlap or be located within a certain range, and after using these features, the start and end points of various recognition meta are recorded. If these points are within a specified proximity distance, the corresponding values are collected and k-means nearest clustering is performed to find a sequence cluster with frequent changes.

As the clustering process is performed, the sequence extracting unit 257 may classify the distribution of clustered clusters based on where each meta change of metamorphosis changes more frequently than a specified value. The sequence extraction unit 257 may divide the clusters (or sequence sections) differently according to the above-described distribution of clustered clusters. The cluster points collected in this way can be directly used as a sequence division entity. FIG. 8 shows a value configured by configuring a sequence candidate based on the divided clusters and dynamically designating a median (k) value of clustering according to the requested number of sequences. The biggest advantage of dynamically specifying the number of clusters using the above-described clustering technique is that it can cope with variable sequence segmentation requirements in one implementation according to actual service requirements, and frequent appearances and disappearance of recognition meta. That is, it supports to effectively divide the sequence by the number of requested sequence clusters regardless of characteristics such as content with relatively frequent changes or content with relatively little change in recognition meta.

In addition, the sequence extraction unit 257 may apply the importance of each of the various recognition meta to the sequence classification. For example, in the genre of entertainment, the sequence extraction unit 257 determines that the sound source acts as a main element according to a predefined setting, and doubles the input point of the sound source recognition meta among the recognition meta to distinguish the cluster distribution. have. Alternatively, the sequence extracting unit 257 increases the detection of the starring actor's appearance point to about twice when the main actor is determined to be an important case in a specific drama or movie according to a predefined setting. Sequences are extracted and stored in a form that has more influence on cluster classification.

The method of classifying the extracted result as shown in FIG. 8 into an actual sequence is a method of classifying a sequence based on the maximum or minimum value of divided clustering as shown in FIG. 9. 9 shows an example of a method for classifying sequences based on clustering maximum/minimum values. Alternatively, the sequence may be divided based on the point closest to the median (asterisk) without using the maximum or minimum values for the sequence classification, and used according to the needs of the service.

For example, when the actual clustering result is shown as shown in FIG. 10, the sequence extracting unit 257 is related to the division of the sequence, the minimum, maximum, or median value of each point of the corresponding cluster (ie, corresponding The average point of the cluster) can be selected. For example, the sequence extracting unit 257 divides the cluster based on the minimum value when classifying a sequence as the starting point of change of the recognition meta, or, when classifying a sequence by the point where the change is most frequent, determines the point value closest to the central value. Clusters can be partitioned by reference.

10 is a diagram showing an example of a case where the start of a sequence is set to a minimum value. It is assumed that each number shown in FIG. 10 is a timecode for the start and end points of a certain recognition meta (person appearance, sound source appearance, situation meta appearance, etc.). When the starting point of the sequence is divided in this way, the sequence may be split as shown in FIG. 10.

The service providing unit 259 may provide a designated service based on section classification information stored in the sequence information 243. For example, the service provider 259 provides the API so that the sequence information 243 can be used by other application services with the sequence information 243 defined by the sequence extraction unit 257 as a comprehensive orchestration of various recognition meta information. And other services. For example, the service providing unit 259 may provide input values and result values that can be utilized by the sequence extraction unit 257 in the form of a screen or data. Alternatively, the service providing unit 259 inserts the intermediate advertisement into the specific video content using the sequence information 243, inserts additional information, and inserts the additional advertisement, and the intermediate content is inserted in response to the request of the terminal device 100 Can be provided to the terminal device 100.

11 is a diagram illustrating an example of a method for processing section classification information using recognition meta information according to an embodiment of the present invention.

Referring to FIG. 11, in relation to a method for processing section classification information according to an embodiment of the present invention, the processor 250 of the service device 200 may determine whether image content is collected in step 1101. If the video content is not collected, the processor 250 may process the function execution in step 1103. For example, the processor 250 supports access to the terminal device 100 and may provide the terminal device 100 with video content designated according to the request of the terminal device 100. In this case, the video content provided to the terminal device 100 may include video content in which at least one additional information is inserted according to the sequence information 243. The additional information may include, for example, a designated video advertisement. With regard to video content collection, the processor 250 forms a communication channel with the content providing server 300 according to a predetermined cycle or a notification from the content providing server 300 (for example, a new video content registration guide), and provides the content Video content may be collected from the server 300.

When the image content is collected in step 1101, the processor 250 may extract meta information in step 1105. In this regard, the processor 250 may extract various recognition meta information from image content using a plurality of recognition technologies. For example, the processor 250 may extract a recognition meta associated with at least one specific person using face recognition technology. Alternatively, the processor 250 may extract the recognition meta associated with at least one specific object or a specific animal using object recognition technology. Alternatively, the processor 250 may extract the recognition meta associated with the sound source applied to at least a portion of the video content using the sound source recognition technology. For the above-described operation, the processor 250 may divide frames of image content into recognition units according to respective recognition technologies.

In operation 1107, the processor 250 may process meta information integration. For example, the processor 250 may combine continuous recognition meta information within a designated time range for each extracted recognition meta and process it as one integrated recognition meta. For example, when a specific person repeatedly appears and exits within a certain time range, the corresponding time range may be treated as one integrated recognition meta.

In operation 1109, the processor 250 may perform clustering and sequence extraction. The processor 250 may perform a clustering operation according to the requested number of sequences. In this operation, the processor 250 detects changes in recognition meta and may assign a ranking according to the degree of change in recognition meta. The processor 250 may check the ranking and perform sequence extraction according to the requested number of sequences. For example, when three sequence numbers are requested, the processor 250 may select clusters in the third rank. The number of sequences may be received, for example, from the content providing server 300 providing video content or from an advertiser device.

In step 1111, the processor 250 may generate section classification information. For example, the processor 250 may obtain section viewpoint information for classifying the clusters based on the selected clusters according to the requested number of sequences as section segmentation information. If the section classification information is obtained, in step 1113, the processor 250 may store the section classification information.

In step 1115, the processor 250 may check whether there is an event related to the end of the section classification information processing. If there is no event related to the termination of section division information processing, the processor 250 may branch to step 1101 and perform the following operation again. When an event related to the termination of section division information processing occurs, the processor 250 may end the sequence extraction of the image content and the section division information generation operation.

As another example, the processor 250 of the service device 200 may receive the number of sections and advertisement materials from an external electronic device (eg, the content providing server 300 or the advertisement providing server). When an advertisement is received, the processor 250 may insert the received advertisement into video content, but may insert it according to sequence information.

As described above, the present invention can provide a method of automatically detecting a sequence of finding a natural point in context by utilizing various meta recognized in video content. The above-described present invention configures a natural sequence detection system in context, and supports to provide a service suitable for natural interstitial advertisement insertion or segmentation without requiring a person to find a sequence by hand.

If the conventional automatic detection technology of a sequence is provided as a bundle of scene/shot information using only image recognition, or if it is a technology that relies on this by applying a change amount and a change detection in an image, the present invention efficiently utilizes multiple metadata. It is used in combination, but it is not directly used as information to distinguish a specific scene/shot, but a method for constructing a service for natural sequence detection by dynamically detecting a point where input values frequently change and configuring it as a sequence cluster. Device can be provided.

The method according to the present invention may be implemented in a form of software readable through various computer means and recorded on a computer-readable recording medium. Here, the recording medium may include program instructions, data files, data structures, or the like, alone or in combination. The program instructions recorded on the recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in computer software. For example, the recording medium includes magnetic media such as hard disks, floppy disks and magnetic tapes (Magnetic Media), compact disk read only memory (CD-ROM), optical media such as DVD (Digital Video Disk), and optical media. Includes magneto-optical media, such as a floppy disk, and hardware devices specifically configured to store and execute program instructions such as ROM, random access memory (RAM), flash memory, etc. do. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. Such a hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

In addition, implementations of the functional operation and subject matter described in this specification may be implemented in other types of digital electronic circuits, or computer software, firmware, or hardware including the structure and its structural equivalents disclosed herein, or these It can be implemented in combination with one or more of. Implementations of the subject matter described herein are one or more computer program products, that is, one for computer program instructions encoded on a tangible program storage medium to control or thereby execute the operation of a device according to the invention. It can be implemented as the above modules. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of materials affecting a machine-readable propagated signal, or a combination of one or more of these.

In addition, this specification includes details of a number of specific implementations, but these should not be understood as limiting on the scope of any invention or claim, but rather on features that may be specific to a particular embodiment of a particular invention. It should be understood as an explanation. Certain features that are described in this specification in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. Further, although features may operate in a particular combination and may be initially depicted as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, and the claimed combination subcombined. Or sub-combinations.

Likewise, although the operations are depicted in the drawings in a particular order, it should not be understood that such operations should be performed in the particular order shown or in sequential order, or that all shown actions should be performed in order to obtain desirable results. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems are generally integrated together into a single software product or packaged in multiple software products. You should understand that you can.

Specific embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, the operations recited in the claims may be performed in different orders while still achieving desirable results. As an example, the process illustrated in the accompanying drawings does not necessarily require that particular illustrated order or sequential order to obtain desirable results. In certain implementations, multitasking and parallel processing can be advantageous.

The described description presents the best mode of the present invention, and provides examples for explaining the present invention and for those skilled in the art to make and use the present invention. This written specification is not intended to limit the invention to the specific terms presented. Therefore, although the present invention has been described in detail with reference to the above-mentioned examples, those skilled in the art can make modifications, alterations and modifications to these examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be determined by the described embodiments, but should be determined by the claims.

The present invention is applied to the segmentation field of video content.

Particularly, the present invention increases the reliability of the sequence classification by performing section division using the recognition meta of the video content, and by providing various additional information by inserting it into the video content based on this, a person directly performs the section division of the content It can help you do your job more quickly and easily. Accordingly, the present invention can quickly and accurately satisfy the needs of advertisers, etc., which need to insert additional information, and can maximize the advertising effect by minimizing the refusal to the additional information of the viewer watching the video content.

10: network environment
100: terminal device
150: network
200: service device
210: communication circuit
240: memory
250: processor
251: content collection unit
253: meta information extraction unit
255: meta information integration unit
257: sequence extraction unit
259: service provider

Claims (12)

◈ Claim 1 was abandoned when payment of the set registration fee was made.◈ Memory;
A communication circuit forming a communication channel with an external electronic device; And
And a processor functionally connected to the memory,
The processor
Acquiring the video content stored in the memory,
Recognizing meta information is extracted from the acquired image content,
Clustering detection time information of the extracted recognition meta information,
After classifying sections according to the distribution of clustered clusters, section classification information is generated,
When the required number of sequences is received from the external electronic device through the communication circuit, a sequence candidate is configured based on the clustered clusters, and the median of the clustering is dynamically designated according to the required number of sequences to classify the sections. Service device characterized in that. ◈ Claim 2 was abandoned when payment of the registration fee was set.◈ According to claim 1,
The processor
Recognized meta information that repeatedly appears within a specified time range is integrated to generate integrated recognition meta information,
And a service device configured to perform clustering on the integrated recognition meta information. ◈ Claim 3 was abandoned when payment of the set registration fee was made.◈ According to claim 1,
The processor
The recognition meta information is extracted using a plurality of recognition technologies,
After performing integrated recognition meta information generation for each extracted recognition meta information,
A service device characterized in that the start and end points of the recognition meta are set to process similar input metadata of a cluster within a specified range and process input values of one cluster. ◈ Claim 4 was abandoned when payment of the set registration fee was made.◈ According to claim 1,
The processor
And checking the content-related information of the video content, and determining the type of the extracted meta-information. ◈ Claim 5 was abandoned when payment of the set registration fee was made.◈ According to claim 4,
The processor
If the video content is a movie content, characterized in that the service device is configured to extract the sound source recognition meta and person recognition meta. ◈ Claim 6 was abandoned when payment of the set registration fee was made.◈ According to claim 4,
The processor
When the video content is a documentary, a service device characterized in that it is set to extract a recognition meta and a sound source recognition meta for a specific object or a specific animal. ◈ Claim 7 was abandoned when payment of the set registration fee was made.◈ According to claim 1,
The processor
A service device characterized by classifying the section by assigning a weight according to the importance of each of the recognition meta information. ◈ Claim 8 was abandoned when payment of the set registration fee was made.◈ According to claim 1,
The processor
Based on the distribution of the clustered clusters, ranking of clusters according to the degree of change according to the appearance of recognition meta is given,
A service device configured to select clusters within a certain ranking as sequences to be used to generate section classification information according to the number of sequences. Service device,
Extracting recognition meta information from the video content stored in the memory;
Generating integrated recognition meta information by integrating the recognition meta information according to a specified criterion;
Clustering detection time information of the integrated recognition meta information; And
After classifying the sections according to the distribution of the clustered clusters, generating section classification information; includes,
When receiving the required number of sequences from the external electronic device,
The step of generating the section classification information
A method of generating section classification information using recognition meta information, comprising configuring a sequence candidate based on the clustered clusters and dynamically specifying the median of the clustering according to the required number of sequences. The method of claim 9,
Method of generating section classification information using recognition meta information further comprising; ranking the clusters according to the degree of change according to the appearance of recognition meta based on the distribution of the clustered clusters. delete A computer recording medium storing at least one instruction executed by a processor, comprising:
The at least one command
Service device,
Extracting recognition meta information from the video content stored in the memory;
Generating integrated recognition meta information by integrating the recognition meta information according to a specified criterion;
Clustering detection time information of the integrated recognition meta information; And
After classifying the sections according to the distribution of the clustered clusters, generating section classifying information;
The operation of generating the section classification information is
A computer recording medium characterized by configuring a sequence candidate based on the clustered clusters and classifying the sections by dynamically designating a median value of the clustering according to the required number of sequences.

Images