KR101745370B1 - Method and system for providing image information with main frame - Google Patents

Abstract

The present invention relates to a method and system for providing information on a video image included in a moving picture, and more particularly, to a method and system for providing information on a video image included in a moving picture using a main video frame set for the moving picture content ≪ / RTI >
A method of providing image information using a main image frame includes the steps of: (a) extracting one or more main image frames based on a scene change from one or more image frames included in the moving image content; (b) (C) obtaining a main image frame matched with the information of the image frame, (d) acquiring a main image frame corresponding to the information of the image frame, And (e) transmitting attribute information matched with the acquired image image to the user terminal. According to another aspect of the present invention,

Description

TECHNICAL FIELD [0001] The present invention relates to a method and system for providing image information using a main image frame,

The present invention relates to a method and system for providing information on a video image included in a moving picture, and more particularly, to a method and system for providing information on a video image included in a moving picture using a main video frame set for the moving picture content ≪ / RTI >

With the development of the Internet technology, the kinds of content provided through the Internet network are increasing, and in particular, the services provided by the video content through the Internet network such as the IPTV service are becoming widespread.

In addition, unlike a conventional service that provides video content to a user unilaterally, it is possible to provide a bidirectional service for exchanging signals with a user via an Internet network and providing specific information according to a request of a user, In addition, services for providing information related to video contents to users are gradually being used.

In the related art, in order to provide information related to video contents, in particular, information about objects, actors, buildings, etc. appearing in moving pictures, related information matching information on objects or persons included in video frames of moving picture contents, detailed information Is set for all video frames included in the video contents.

Recently, video content provided to a user has a short playback time from several minutes to several hours, so that the video content may include a very large amount of image frames.

Accordingly, in order to provide attribute information for a video image included in moving image contents according to the related art, attribute information must be set for a large amount of image frames. Therefore, resource consumption such as time and cost There was a problem of being large.

In the embodiment of the present invention, not the attribute information is set for all image frames, but a main image frame in which a scene is rapidly switched among image frames is extracted, and when a click event is generated by a user, A method and system for searching for a video image in a main video frame matching a frame and providing attribute information of the retrieved video image to minimize resources consumed for providing attribution information.

According to a first aspect of the present invention, there is provided a method for processing a moving image, comprising the steps of: (a) extracting one or more main image frames based on scene change from one or more image frames included in moving image content; (C) obtaining a main image frame matched with the information of the image frame, (d) acquiring a main image frame corresponding to the information of the image frame, And (e) transmitting attribute information matched with the obtained image to the user terminal, wherein the image information providing server uses the main image frame Thereby providing a method of providing image information.

In the first aspect of the present invention, the step (a) includes: (a1) a first image frame included in the moving image content and a second image frame temporally continuous with respect to the first image frame, (A2) a block of the first image frame and a block of the second image frame based on the pixels of the block of the first image frame and the block of the second image frame corresponding to the block of the first image frame, (A3) calculating a number of blocks having a difference of the corresponding block equal to or greater than a predetermined first threshold value, and (a4) calculating the number of blocks having a difference of at least a predetermined second threshold , And setting the second image frame as a main image frame.

In addition, in the first aspect of the present invention, the step (a2) includes the steps of (a21) calculating a histogram including a distribution of pixel values of each of the blocks of the first image frame and the blocks of the second image frame, and a22) calculating a difference between the block of the first image frame and the block of the second image frame based on the calculated histogram.

(A2) comparing pixel values of a block of the first image frame and pixel values of a block of the second image frame, respectively, and (a24) comparing the pixel values of the block of the first image frame with the pixel values of the block of the second image frame, And calculating a pixel difference between the first image frame and the second image frame based on the comparison result.

In the first aspect of the present invention, the step (a) includes the steps of: (a5) extracting a video image matched with an image previously stored in a video image database among video images included in the extracted main video frame; Mapping the extracted video image to an image matched to the extracted video image, and registering the position coordinates of the extracted video image to match the main video frame.

According to a second aspect of the present invention, there is provided a mobile communication system including a main image frame management unit for extracting a main image frame matched with one or more image frames based on a scene change from a moving image content and extracting a video image from the main image frame, An image image matching unit for obtaining a video image matched with a positional coordinate at which a click event included in the click event information is generated from a main video frame matched with video frame information generated by a click event included in one click event information, And an attribute information providing unit for searching attribute information matching the video image and transmitting the retrieved attribute information to the user terminal.

According to an embodiment of the present invention, an image frame in which a scene is rapidly switched based on scene transition among image frames constituting a moving image content is set as a main frame, and a main frame, It is possible to determine whether there is a video image corresponding to the position coordinates at which a click event occurs in the frame, and provide attribute information of the video image corresponding to the clicked position to the user.

According to the present invention, attribute information is provided only for a main image frame representing at least one image frame corresponding to a similar scene, not for a whole image frame of the moving image content, The main image frame representative of the generated image frame is used to acquire the image of the clicked position to provide the attribute image so that the same attribute information can be provided to the user while minimizing the resources required for assigning the attribute information to the image frame .

1 is a view illustrating a configuration of an image information providing system using a main image frame according to an embodiment of the present invention;
2 is a flowchart illustrating a method of providing image information to a user terminal in a method of providing image information using a main image frame according to an embodiment of the present invention.
3 is a flowchart illustrating a method of extracting a main image frame from a moving image content in a method of providing image information using a main image frame according to an exemplary embodiment of the present invention.
FIG. 4 illustrates a main image frame and an image included in a main image frame, which are matched with one or more image frames according to an exemplary embodiment of the present invention.
5 is a diagram illustrating a configuration of an image information providing server in an image information providing system using a main image frame according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

1 is a block diagram of a system for providing image information using a main image frame according to an embodiment of the present invention.

The user terminal 100 receives the video content from the video content providing server 300 via the network and outputs the received video content to the user. The user terminal 100 receives the click event of the video content being output from the user, transmits the information of the video frame in which the click event occurs and the clicked position coordinate information to the video information providing server 200, Request attribute information.

The user terminal 100 may output the video content in the video play mode and provide the video content to the user. In response to the user's request, the user terminal 100 switches from the video play mode to the information providing mode, .

The user terminal 100 may include a terminal capable of performing a communication function and a video providing function such as a desktop, a notebook, and a set-top box.

The image information providing server 200 receives the moving image content, that is, the image frame information of the moving image from the moving image content providing server 300, and extracts a main image frame that can represent at least one image frame among the plurality of image frames.

The main image frame is a frame representing one or more image frames for a specific scene and may be set as a main image frame for one or more image frames included in the scene in which the first image frame is switched when the scene is switched.

The image information providing server 200 extracts a video image included in the main video frame, maps the coordinate information of the extracted video image and attribute information matched to the extracted video image, and stores the mapped information in a database.

The attribute information matching the video image may include detailed information of the video image. For example, if the video image is a specific actor or a person, the attribute information may include a detailed profile of the actor, etc. If the video image is an image for a car, the attribution information includes the name, Manufacturer, and the like.

The image information providing server 200 receives the attribute information request from the user terminal 100, searches the attribute information matching the image frame information and the position coordinate information received together with the attribute information request, 100).

As described above, according to the present invention, a main image frame that can represent a corresponding scene is extracted for each scene, instead of extracting a video image and attribute information for the entire video frame included in the video content, Only the video image is extracted and the attribute information is given. When a click event occurs from the user, the attribute information is provided using the main image frame, so that the database for providing attribute information can be constructed more easily.

2 is a flowchart illustrating a method of providing image information to a user terminal in a method of providing image information using a main image frame according to an exemplary embodiment of the present invention.

In step S105, the image providing server extracts one or more main image frames from the moving image contents received from the moving image contents providing server. As described above, the main image frame may represent an image frame group including one or more image frames. The video frame group may be set based on scene transition according to the time, and the first video frame of the video frame group may be set as the main video frame.

For example, when moving from a scene including a distance to a scene including a room in the video content, one or more image frames forming a scene including a room may form one image frame group, The first video frame to be switched can be set as the main video frame of the video frame group.

In step S110, the image providing server extracts the image image from the main image frame extracted in step S105, and stores the coordinate information of the extracted image image in the main image frame database (not shown).

That is, the image providing server extracts a video image matching the image stored in the video image database among the video images included in the main video frame, and maps the image of the video image database matching the extracted video image to the extracted video image.

The image providing server stores the positional coordinates of the image image of the main image frame mapped to the image of the image image database in the main image frame database.

The image providing server can compare the image extracted from the main image frame with the image image stored in the built-up image database (not shown) to acquire a matching image.

For example, the image providing server may extract an image of the bag from the main image frame, and compare the extracted image with one or more image images stored in the image image database to obtain a matching bag image.

The image providing server acquires the coordinate information of the outer frame forming the video image of the main image frame from which the matching image is obtained from the image image database, and acquires the image of the main image frame, the matching image of the image image database, Can be mapped and stored in the main image frame database.

The image providing server can configure an outline forming a video image in a preset polygonal shape, and can store coordinate information of the outline of the polygon as coordinate information of the outline of the video image.

4, the image providing server may classify one or more image frames of the moving image content into one or more image frame groups 410, 420 and 430 for each scene, and each of the image frame groups 410 and 420 And 430, main image frames 411, 421, and 431 that can represent image frame groups can be set.

The image providing server can extract a video image 412 for each main video frame 411, 421 and 431 and output the extracted video image 412 to the image 261 of the video image database 260. [ And store the matching information in the main frame database.

As described above, the image providing server extracts a main image frame having a similar position of an object or a character appearing in the image with one or more image frames, instead of extracting the image image matched with the attribute information of all the image frames, The resource required to construct the database for providing the attribute information can be minimized.

In step S115, the image providing server receives the information of the click event generated by the user from the user terminal.

A click event at the user terminal may occur in a specific mode. That is, in the video play mode for video output, the user terminal does not recognize such a click as a click event for providing the video attribute information, even if the user clicks the video, Only clicks performed can be recognized as click events for providing image attribute information.

Accordingly, even when the user accidentally clicks a part of the moving picture in the moving picture play mode, attribute information of the moving image can be prevented from being provided against the intention of the user.

In this way, the image providing server monitors whether or not the user terminal has converted the video play mode for providing the video contents into the information providing mode, monitors whether or not the user's click event has occurred in the information providing mode state, .

The click event information may include information of the image frame output at the time of the occurrence of the click event by the user, coordinate information of the clicked position, and the like.

In step S120, the image providing server obtains the information of the image frame outputted at the time of the occurrence of the click event, and the coordinate information of the clicked position, from the click event information received from the user terminal.

In step S125, the image providing server searches for a main image frame matched with the image frame information acquired in step S120. That is, the image providing server searches the image frame group to which the image frame in which the click event occurred by the user belongs, and searches for the main image frame matched with the searched image frame group.

In step S130, the image providing server determines whether a video image corresponding to the position coordinate information acquired in step S120 exists in the main image frame retrieved in step S125.

That is, the image providing server determines whether a video image set in step S110 exists in a position corresponding to the position coordinate information acquired in step S120 in the main image frame retrieved in step S125.

As described above, in step S105, the image providing server classifies the video content into at least one video frame group including at least one video frame based on the sudden scene change, and sets a main video frame for each video frame group do.

Accordingly, the main image frame includes scenes similar to other image frames in the corresponding image frame group, and positions of objects, people, and the like in the image may be similar.

Accordingly, the image providing server searches for a video image in which attribute information matched with the position where the click event occurs in the main video frame matched with the video frame in which the click event occurs, and a video image capable of providing attribute information to the user exists Or not.

If it is determined in step S135 that there is a video image corresponding to the clicked position in the main video frame in step S130, the video providing server searches for the attribute information matched with the searched video image, And transmits it to the user terminal and provides it to the user.

Accordingly, the user can receive the attribute information about the object of interest or the related information, that is, related information, by clicking on the object or the person appearing in the video content during the viewing of the video content.

3 is a flowchart illustrating a method of extracting a main image frame from moving image contents in a method of providing image information using a main image frame according to an exemplary embodiment of the present invention.

In step S205, the image providing server divides the kth image frame and the (k + 1) th image frame included in the moving image content into N blocks. That is, the image providing server divides two consecutive image frames among the image frames constituting the moving image content into a predetermined number of blocks.

In step S210, the image providing server sets the value of n to 1, and in step S215, the image providing server obtains the value of n in each of the n-th block of the k-th image frame and the n-th block of the And a histogram representing a distribution of pixel values are respectively calculated.

That is, the image providing server calculates the histogram of the distribution of the pixel values of the blocks in the same position of two consecutive image frames.

In step S220, the image providing server determines whether the difference between the histogram of the n-th block of the k-th image frame and the histogram of the n-th block of the (k + 1) -th image frame calculated in step S215 is greater than or equal to a preset threshold value .

That is, the image providing server determines whether the distribution of the pixel values of the n-th block of the k-th image frame and the distribution of the pixel values of the n-th block of the (k + 1) th image frame are equal to or greater than a preset threshold value.

If it is determined in step S225 that the difference between the histogram of the n-th block of the k-th image frame and the histogram of the n-th block of the (k + 1) -th image frame is equal to or greater than a preset threshold value in step S220, Calculates the difference between the sum of the pixel values of the nth block of the kth image frame and the sum of the pixel values of the nth block of the (k + 1) th image frame.

In step S230, the image providing server calculates the difference between the sum of the pixel values of the n-th block of the k-th image frame calculated in step S225 and the sum of the pixel values of the n-th block of the (k + 1) Is equal to or greater than a preset threshold value.

If the difference between the sum of the pixel values of the n-th block of the k-th image frame and the sum of the pixel values of the n-th block of the (k + 1) -th image frame is greater than or equal to a preset threshold value in step S225 The image providing server increases the value of m indicating the number of different blocks between the kth image frame and the (k + 1) th image frame by one.

That is, when the difference between the histograms showing the distribution of the pixel values of the n-th block of the k-th image frame and the n-th block of the k + 1-th image frame is equal to or more than the threshold value, And the n-th block of the (k + 1) -th image frame is equal to or greater than a preset threshold value, the image providing server determines that the n-th block of the k- .

In step S240, the image providing server determines whether all of the N blocks of the k-th image frame and the (k + 1) -th image frame are compared. That is, the image providing server determines whether all the N blocks of the kth image frame and the N blocks of the (k + 1) th image frame are different or not.

If it is determined in step S245 that all of the N blocks of the k-th image frame and the (k + 1) -th image frame are not compared in step S240, the image providing server transmits the k- The value of the identification value n of the block of the (k + 1) -th image frame is incremented by 1, and the steps (S215) to (S235) are performed for the n-th block of the k-th image frame and the (k + 1) th image frame.

If it is determined in step S250 that all of the N blocks of the k-th image frame and the (k + 1) -th image frame have been compared in step S240, it is determined whether the value of m, which is the value of the number of different blocks between k + 1 image frames, is equal to or greater than a predetermined threshold value.

That is, the image providing server determines whether the kth image frame and the (k + 1) th image frame have a predetermined number or more of different blocks.

If it is determined in step S255 that the value of m, which is the value of the number of different blocks between the k-th image frame and the (k + 1) -th image frame, is equal to or greater than a preset threshold value, Set to the main video frame.

That is, when the value of m, which is the value of the number of different blocks between the k-th image frame and the (k + 1) -th image frame, is equal to or greater than a preset threshold value, It is determined that the scene change has occurred, and the (k + 1) -th image frame, which is the first image frame that has been scene-switched, can be set as the main image frame.

In step S260, the image providing server determines whether or not block segmentation and block comparison of forward and backward image frames are performed on all image frames of the moving image contents. That is, the image providing server determines whether steps S205 to S255 are applied to all the image frames of the moving image contents.

If it is determined that the block division and the block comparison of the previous and next image frames are performed for all the image frames of the moving image content, the image providing server ends the main image frame setting.

In step S265, if it is determined in step S260 that the block segmentation and the block comparison of the forward and backward image frames are performed only on a part of the image frame of the moving image content, The value is incremented by 1, and steps S205 to S255 are performed.

5 is a diagram illustrating a configuration of an image information providing server in an image information providing system using a main image frame according to an embodiment of the present invention.

The image information providing server 200 according to an embodiment of the present invention includes a main image frame managing unit 210, an image image matching unit 220, an attribute information providing unit 230, an event information receiving unit 240, A database 250, a video image database 260, and an attribute information database 270.

The main image frame management unit 210 analyzes a moving image content provided from a content providing server (not shown) and extracts a main image frame. The main image frame managing unit 210 extracts a main image frame, And stores it in the main image frame database 250. The main image frame database 250 includes a plurality of image frames.

The main image frame management unit 210 divides the two consecutive image frames, that is, the previous image frame and the current image frame into N blocks, compares the pixel values of the corresponding blocks of the respective image frames, If it is judged that the scene has been switched from the previous image frame to the current image frame and the current image frame is set as the main image frame if the number of blocks having the difference of the pixel value of the corresponding block is equal to or greater than the predetermined number have.

The main image frame management unit 210 extracts a video image from the main video frame and matches the video image with an image stored in the video image database 260. The main video frame management unit 210 maps the extracted video image, And stores it in the database 250.

The main image frame management unit 210 receives the click event information from the event information receiving unit 240 and generates a video image by matching the video frame information included in the click event information and the coordinate information of the clicked position The main image frame is searched and information of the searched main image frame is transmitted to the image image matching unit 220.

The image image matching unit 220 includes coordinates of the clicked position included in the click event information among the image images stored in the main image frame matching the main image frame using the main image frame information received from the main image frame management unit 210 Search the video image.

The image image matching unit 220 acquires the image information including the coordinates of the clicked position from the image image database 260 and transmits the identification information of the acquired image image to the attribute information providing unit 230.

The attribute information providing unit 230 acquires attribute information matched with the identification information from the attribute information database 270 using the image image identification information acquired from the image matching unit 220, To a user terminal (not shown) that has generated the data and provides the data to the user.

The attribute information providing unit 230 may obtain the identification information of the user terminal that generated the click event from the event information receiving unit 240 and may transmit the attribute information to the user terminal.

The event information receiving unit 240 receives the click event information generated in the user terminal, and outputs the information of the image frame where the click event included in the click event information and the clicked position coordinate information to the main image frame management unit 210 and the image image matching (220), and transmits the information of the user terminal that generated the click event to the attribute information providing unit (230).

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

While the methods and systems of the present invention have been described in connection with specific embodiments, some or all of those elements or operations may be implemented using a computer system having a general purpose hardware architecture.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: user terminal
200: a video information providing server
300: Video content providing server
410, 420 and 430: image frame group
411, 421 and 431: main picture frame

Claims (9)

A method of providing image information using a main image frame,
(a) extracting one or more main image frames based on scene change from one or more image frames included in the moving image content,
(b) extracting a video image from the main video frame, storing coordinate information of the extracted video image and attribute information matched to the extracted video image in a database,
(c) receiving information of an image frame in which a click event occurs in which a user clicks an image frame other than the main image frame and position coordinate information in the image frame,
(d) acquiring a main image frame matched with the information of the image frame,
(e) determining whether a main image frame has a video image corresponding to the received position coordinate information,
(f) acquiring an image image matching the position coordinate information from the acquired main image frame when it is determined that there is a video image corresponding to the received position coordinate information; and
(g) transmitting attribute information matching the obtained image to the user terminal
And providing the image information.
The method according to claim 1,
The step (a)
(a1) dividing a first image frame included in the moving image content and a second image frame temporally continuous with respect to the first image frame into a predetermined number of blocks,
(a2) calculating a difference between a block of the first image frame and a block of the second image frame based on a pixel of the block of the first image frame and a pixel of the block of the second image frame corresponding to the block of the first image frame, Calculating,
(a3) calculating a number of blocks in which the difference between the corresponding blocks is equal to or greater than a predetermined first threshold value, and
(a4) setting the second image frame as a main image frame when the number of the calculated blocks is equal to or larger than a preset second threshold value
The method comprising the steps of:
3. The method of claim 2,
The step (a2)
(a21) calculating a histogram including a distribution of pixel values of a block of the first image frame and a block of the second image frame, and
(a22) calculating a difference between the block of the first image frame and the block of the second image frame based on the calculated histogram
The method comprising the steps of:
3. The method of claim 2,
The step (a2)
(a23) comparing pixel values of a block of the first image frame and pixel values of a block of the second image frame, and
(a24) calculating a pixel difference between the block of the first image frame and the block of the second image frame based on the comparison result
The method comprising the steps of:
The method according to claim 1,
The step (a)
(a5) extracting a video image matched with an image stored in the video image database among the video images included in the extracted main video frame,
(a6) mapping the extracted video image to an image matched with the extracted video image, and registering the position coordinates of the extracted video image with the main video frame and registering
The method comprising the steps of:
A video information providing system using a main video frame,
Extracting a main image frame matching one or more image frames based on a scene change from the video content, extracting a video image from the main video frame, extracting a video image from the main video frame, A main image frame management unit for storing information and attribute information matched to the extracted image image in a database,
An event information receiving unit for receiving information of an image frame in which a click event occurs in which a user clicks an image frame other than the main image frame from the user terminal and position coordinate information in the image frame,
An image image matching unit for acquiring an image image matched with the position coordinate information from a main image frame matched with the image frame information received from the user terminal;
Searching attribute information matched with the acquired image image, and transmitting the searched attribute information to the user terminal,
Lt; / RTI >
Wherein the video information providing system determines whether a video image corresponding to the received position coordinate information exists in the main video frame,
And acquires a video image matched with the position coordinate information when it is determined that there is a video image corresponding to the received position coordinate information.
The method according to claim 6,
Wherein the main image frame management unit divides the temporally successive first and second image frames into a predetermined number of blocks and generates a second image frame corresponding to a block of the first image frame and a second image corresponding to a block of the first image frame And sets the second image frame as a main frame when the difference of the blocks of the frame is equal to or greater than a preset threshold value.
8. The method of claim 7,
Wherein the main image frame management unit includes a first histogram including a distribution map of pixel values of the first image frame and a second histogram including a distribution of pixel values of blocks of the second image frame, Value, the second image frame is set as a main frame.
8. The method of claim 7,
Wherein the main image frame management unit manages the second image frame in the main frame if the difference between the sum of the pixel values of the blocks of the first image frame and the sum of the pixel values of the blocks of the second image frame is equal to or greater than a preset threshold value, Is set to " 0 ".

Images