KR102414211B1 - Method and system for providing video - Google Patents

Abstract

The present invention discloses a video providing method and apparatus. The present invention extracts an object for each scene included in a plurality of moving images, generates indexing information indicating the characteristics of the object, and then includes, according to the indexing information, video identification information and time information at which the scene is displayed Classifies and stores object identification information. After that, when the user terminal sends a request image requesting video provision, it extracts objects from the requested image, generates search information in the same way as indexing information, and compares the search information with the indexing information to find objects included in the request image. Check the object identification information of the object corresponding to , and transmit the video data to the user terminal according to the video identification information included in the corresponding object identification information, and play the video data from a time corresponding to the time information. Therefore, as described above, the present invention stores object identification information by classifying according to the indexing information indicating the characteristics of the object extracted from the video, so that the object included in the user's request image and the video in which the object appears are very quickly User convenience can be maximized by allowing the user to play from the scene in which the requested image received from the user terminal appears in the video as well as the user's ability to search.

Description

Method and system for providing video

The present invention relates to a method and apparatus for providing a video, and more particularly, to a method and apparatus for providing a video by searching for a video including an image provided from a user terminal.

Recently, as the spread of smartphones has expanded and high-speed communication networks have become common, the way consumers consume video content is shifting from TVs to smartphones.

In addition, as well as news content provided through Internet portal sites, various contents are provided with a video inserted or an image captured from a video is inserted and provided, and consumers who have viewed it respond to the video or image The method of retrieving and consuming videos that are used is becoming common.

However, it takes a lot of effort and time for the consumer to search for the entire video corresponding to the video or image, and even if the video is found, it is very difficult for the consumer to search for the scene desired by the consumer from the entire video. .

For example, if a user encounters an image of a scene in a movie starring his/her favorite actor through an Internet article and wants to see a scene in which the image is displayed, the user downloads the movie through the OTT service that provides the movie, or You need to run streaming. In addition, when movie content is executed, the user has to either watch the movie from the beginning or search for a corresponding scene in the movie while skipping a predetermined time period.

However, if the user knows the title of the video he is looking for and a service provider that can download or stream the video, video search is possible, but it is more difficult to find a video only with a simple image.

In this way, in order to reduce the effort of the consumer to find a video, a service is required to find a video including the corresponding image when the user transmits a specific image, but such a service is not yet provided.

This seems to be due to the limitation of the prior art for determining whether the images match by comparing the hash value of the user's query image with the hash value of the images stored in advance in a general image search process. That is, in the case of an image that is displayed to the user and the user requests a search, various modifications are applied, such as subtitles added, brightness adjusted, or a part of the image cut off by several people, so that the hash value is different from the original video. , since it is difficult to accurately search for it, there is a limitation in that the image search result is not accurate.

The problem to be solved by the present invention is not only to quickly and accurately search for and provide a user's desired video, but also to improve the user's convenience by allowing the user to play the video from the desired scene when the user plays the video using the terminal. It is to provide a method and apparatus for providing.

In a video providing method according to a preferred embodiment of the present invention for solving the above-described problems, (a) the video providing apparatus for a plurality of videos, object identification information for each of the objects displayed for each scene of each video and generating indexing information; (b) storing, by the video providing device, the object identification information and the indexing information in correlation with video data; (c) when the video providing device receives the requested image from the user terminal, extracting an object from the requested image, and then generating search information for each extracted object; (d) checking, by the video providing apparatus, indexing information corresponding to the search information and object identification information corresponding to the indexing information, and checking time information at which the video corresponding to the requested image and the requested image are reproduced; and (e) providing, by the video providing apparatus, video data to the user terminal so that the video corresponding to the requested image can be played from the time corresponding to the time information.

In addition, the indexing information in step (a) is obtained by obtaining the color histogram for the object image with the codes generated by the following ① to ④, and combining the numbers indicating the color components sequentially according to the size of the color components. Generated code, ② Code assigned according to the inclination of the object image, ③ Code assigned according to the horizontal to vertical ratio of the object image, ④ Assigned according to the area ratio of the object image to the circumscribed rectangle touching the object image code], and may include first indexing information generated by combining two or more.

In addition, the first indexing information is a code given according to the area ratio of the object image to the circumscribed ellipse touching the object image, ⑥ the code generated by the ① to ④ and the following ⑤ to ⑦ It can be generated by combining two or more of a code assigned using the gradient between the most distal points and a code assigned using the object recognition result].

In addition, the indexing information further includes second indexing information implemented as a Fourier descriptor generated for the object image, and in step (b), the video providing apparatus converts the object identification information into the first indexing information may be classified primarily according to , and secondarily classified and stored according to the second indexing information.

Also, in step (c), the video providing device generates first search information for the object extracted from the requested image in the same manner as the first indexing information, and uses the second indexing information in the same manner Generates second search information for the object extracted from the request image, and in step (d), the video providing device includes a first search information corresponding to the first search information from among the object identification information stored in step (b). 1 According to the video identification information included in the object identification information classified primarily according to the indexing information and secondarily classified according to the second indexing information corresponding to the second search information, check the video corresponding to the requested image, Time information for displaying the requested image may be checked.

In addition, the indexing information further includes third indexing information indicating a relationship between each pixel value included in the object image and neighboring pixel values, and in step (b), the video providing apparatus provides the object identification information may be primarily classified according to the first indexing information and secondarily classified according to the second indexing information and stored together with the third indexing information.

In addition, in step (c), the video providing apparatus provides first search information, first search information, Generates second search information and third search information, respectively, and in step (d), the video providing apparatus performs a first indexing corresponding to the first search information among the object identification information stored in step (b). Objects classified primarily according to information, classified secondarily according to second indexing information corresponding to the second search information, and in which the third search information and the third indexing information match more than a predefined threshold ratio According to the video identification information included in the identification information, a video corresponding to the requested image may be checked, and information about a time at which the requested image is displayed may be checked.

On the other hand, a video providing apparatus according to a preferred embodiment of the present invention for solving the above-described problems is a video providing apparatus including a processor and a storage device for storing predetermined instructions, wherein the instructions stored in the storage device are executed. The processor may include: (a) generating object identification information and indexing information for each of the objects displayed for each scene of each moving picture with respect to a plurality of moving images; (b) storing the object identification information and the indexing information in association with video data; (c) upon receiving the requested image from the user terminal, extracting an object from the requested image, and then generating search information for each extracted object; (d) searching for indexing information corresponding to the search information and object identification information corresponding to the indexing information, and checking time information at which a video corresponding to the requested image and the requested image are reproduced; and (e) providing video data to the user terminal so that the video corresponding to the requested image can be played from the time corresponding to the time information to perform the video providing method.

In addition, the indexing information in step (a) is obtained by obtaining the color histogram for the object image with the codes generated by the following ① to ④, and combining the numbers indicating the color components sequentially according to the size of the color components. Generated code, ② Code assigned according to the inclination of the object image, ③ Code assigned according to the horizontal to vertical ratio of the object image, ④ Assigned according to the area ratio of the object image to the circumscribed rectangle touching the object image code], including first indexing information generated by combining two or more.

In addition, the first indexing information is a code given according to the area ratio of the object image to the circumscribed ellipse touching the object image, ⑥ the code generated by the ① to ④ and the following ⑤ to ⑦. It can be generated by combining two or more of a code assigned using the gradient between the most distal points and a code assigned using the object recognition result].

In addition, the indexing information further includes second indexing information implemented as a Fourier descriptor generated for the object image, and in step (b), the object identification information is primarily classified according to the first indexing information. and secondarily classified and stored according to the second indexing information.

In addition, in step (c), first search information is generated for the object extracted to the request image in the same manner as the first indexing information, and extracted to the requested image in the same manner as the second indexing information Second search information is generated for an object, and in step (d), from among the object identification information stored in step (b), the first search information is classified according to first indexing information corresponding to the first search information, Check the video corresponding to the requested image according to the video identification information included in the object identification information classified secondly according to the second indexing information corresponding to the second search information, and check the time information at which the requested image is displayed can

In addition, the indexing information further includes third indexing information indicating a relationship between each pixel value included in the object image and neighboring pixel values, and the step (b) includes converting the object identification information into the first indexing information is classified primarily according to First search information, second search information, and third search information are respectively generated for the object extracted from the request image in the same manner as the third indexing information, wherein step (d) is performed in step (b) Among the stored object identification information, primary classification according to first indexing information corresponding to the first search information, secondary classification according to second indexing information corresponding to the second search information, and the third search information and the third indexing information check the video corresponding to the requested image according to the video identification information included in the object identification information that matches by more than a predefined threshold ratio, and check the time information at which the requested image is displayed have.

The present invention extracts an object for each scene included in a plurality of moving images, generates indexing information indicating the characteristics of the object, and then includes, according to the indexing information, video identification information and time information at which the scene is displayed Classifies and stores object identification information.

After that, when the user terminal sends a request image requesting to provide a video, the object included in the request image is extracted by extracting the objects from the request image to generate search information in the same manner as the indexing information, and by comparing the search information with the indexing information. Check the object identification information of the object corresponding to , and transmit the video data to the user terminal according to the video identification information included in the corresponding object identification information, and play the video data from a time corresponding to the time information.

Therefore, as described above, the present invention stores object identification information by classifying according to the indexing information indicating the characteristics of the object extracted from the video, so that the object included in the user's request image and the video in which the object appears are very quickly User convenience can be maximized by allowing the user to play from the scene in which the requested image received from the user terminal appears in the video as well as the user's ability to search.

1 is a diagram showing the overall configuration of a video providing apparatus according to a preferred embodiment of the present invention.
2 is a flowchart illustrating a video providing method according to a preferred embodiment of the present invention.
3 is a view for explaining a process of extracting an object from a video according to a preferred embodiment of the present invention.
4 is a view for explaining a process of generating first indexing information according to a preferred embodiment of the present invention.
5 is a diagram for explaining a process of generating second indexing information according to a preferred embodiment of the present invention.
6 is a diagram for explaining a process of generating third indexing information according to a preferred embodiment of the present invention.
7A and 7B are diagrams for explaining a process of comparing whether the third indexing information matches according to a preferred embodiment of the present invention.
8 is a view for explaining a method in which object identification information is classified according to indexing information and stored in a storage device according to a preferred embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a diagram illustrating the overall configuration of a video providing apparatus according to a preferred embodiment of the present invention and a connection relationship with a user terminal.

Referring to FIG. 1 , an apparatus 200 for providing a video according to a preferred embodiment of the present invention includes a processor 210 , a storage device 220 , an input unit 230 , and an output unit 240 .

The storage device 220 according to the preferred embodiment of the present invention may store instructions executable by the processor 210 and programs executed by the processor 210 , and may also store input/output data.

Also, the storage device 220 may store video data, and may store object identification information and video indexing information associated with the video data. Here, the object identification information is information indicating whether each object shown in the video is included in a scene reproduced at what time of a certain video, and the video indexing information is for quickly searching video data in which the object is stored. The contents will be described later.

In addition, the storage device 220 may be alternatively operated as a web storage or a cloud server performing a function of a storage medium on the Internet, or may be independently operated as a separate database server. Also, the storage device 220 may be operated separately with a configuration for driving the processor 210 and a configuration for storing video data and video indexing information.

The processor 210 according to the preferred embodiment of the present invention executes instructions stored in the storage device 220, thereby performing each step of the method for providing a video to be described later with reference to FIG. 2 . A detailed operation performed by the processor 210 will be described later with reference to FIG. 2 .

The input unit 230 may be implemented as a typical input means such as a mouse and a keyboard, and may receive setting information and selection information from an administrator and output it to the processor 210 .

The output unit 240 may be implemented as a monitor or the like to display data and information generated by the processor 210 to the user.

The video providing apparatus 200 of the present invention is connected to a plurality of user terminals 100 through a wired/wireless communication network, and when receiving an image of a specific scene included in a video from the plurality of user terminals 100, the image is Search for a video included and provide video data in a streaming method so that the user terminal 100 can play from the corresponding image of the video, or provide a video file and time information of the scene together so that the user terminal 100 can Make it playable from the scene.

2 is a flowchart illustrating a video providing method according to a preferred embodiment of the present invention. Hereinafter, an operation of the video providing apparatus 200 and a video providing method will be described with further reference to FIG. 2 .

First, the video providing apparatus 200 stores a plurality of video data in advance in order to provide a video desired by a user to the user terminals 100 , and object identification information for objects displayed for each scene in each video and generating indexing information (S210), and storing each object identification information and indexing information in association with video data (S220).

3 to 6 are diagrams for explaining a process of generating indexing information according to a preferred embodiment of the present invention. Hereinafter, a process of generating the indexing information in step S210 will be described with reference to FIGS. 3 to 6 .

The object identification information and indexing information of the present invention are generated for each scene of a moving picture and for each object included in an image representing the scene. Here, the image representing each scene may be selected according to a predefined rule, and may be a key frame image included in the corresponding scene. In addition, the indexing information is configured to include first indexing information, second indexing information, and third indexing information for one object.

In order to generate indexing information, first, the video providing apparatus 200 extracts an object from a representative image selected for each scene of the video.

Referring to FIG. 3 , the video providing apparatus 200 extracts an object from an image representing a scene, and identifies an outline of each extracted object.

Meanwhile, the video providing apparatus 200 generates a series of codes for each object by using the color and geometric characteristics of the object, and combines the codes to generate first indexing information.

4 is a view for explaining a method of generating first indexing information according to a preferred embodiment of the present invention. Referring to FIG. 4 , the video providing apparatus 200 may generate various code values representing characteristics of an object image using the following methods ① to ⑦, and use at least two methods among the methods ① to ⑦. The first indexing information may be generated by combining the generated code values.

① Code generation using color information

The video providing device 200 obtains a color histogram for the object image extracted as an object, and sequentially combines the numbers representing the RGB components from the largest (or smallest) color component to generate a 3-digit code. do. For example, if a color histogram is obtained for the object image shown in Fig. 4, (R, G, B) = (35.75, 81.77, 45.81), and (R, G, B) code numbers of each are (1,2) ,3) and arranging the codes sequentially according to the size of the color components of the color histogram, it becomes 2(G), 3(B), 1(R). Accordingly, 231 is generated as a code using color information.

② Code generation using object gradient

The video providing apparatus 200 generates a one-digit code according to the inclination of the object image extracted as an object. For example, when the gradient of the object image shown in FIG. 4(a) is obtained, it can be seen that the object image moves downward, and a corresponding code 0 is generated. For reference, code 1 is generated when the gradient of the object is upward.

③ Code generation according to the aspect ratio of the object

The video providing apparatus 200 generates a one-digit code according to the aspect ratio of the object image extracted as an object. For example, with respect to the object image shown in (b) of FIG. 4 , a circumscribed rectangle is drawn in contact with the object, and since the vertical length is longer than the horizontal length, code 1 is generated. If the vertical length is shorter than the horizontal length, code 0 is generated.

④ Code generation according to the ratio of the area to the circumscribed rectangle of the object

The video providing apparatus 200 generates a one-digit code according to the ratio of the area occupied by the object image extracted as an object to the circumscribed rectangle. For example, when a circumscribed rectangle adjoining the object is drawn with respect to the object image shown in FIG.

⑤ Code generation according to the ratio of area to circumscribed ellipse of object

In a manner similar to ④ above, the video providing apparatus 200 generates a one-digit code according to the area ratio occupied by the object image extracted as an object with respect to the circumscribed ellipse. For example, when a circumscribed ellipse is drawn with respect to the object shown in FIG. 4 and the area of the circumscribed ellipse is 10, if the proportion of the object is 6, code 6 is generated.

⑥ Code generation using the slope between the extreme points

The video providing apparatus 200 generates a 4-digit code by using the inclination between the upper, lower, left, and rightmost points of the extracted object image. For example, for the object image shown in (c) of FIG. 4, the most distal points (A, B, C, D) of the object are obtained, and the slope of the line segment (AB) connecting the points A and B is set to 1 because the slope is upward. It is given as a code value. Similarly, since the slope of the line segment BC connecting the points B and C is downward sloping, 0 is assigned as a code value. In the same way, CD = 1 and DA = 0 are assigned as code values. By concatenating these code values, a 4-digit code 1010 is generated.

⑦ Code generation using object recognition result

The video providing apparatus 200 inputs the extracted object image to the artificial intelligence engine and generates a code using the recognition result value of the image. For example, if the object image of FIG. 4 is input into the PreTrained model capable of recognizing 80 objects, the result is recognized as a person, and accordingly, 00, which is a recognition result value, is generated as a code. For reference, the model outputs a recognition result value of 09 for a puppy and 73 for a chair. However, the recognition model used at this time should use the same when recognizing the stored video data and when recognizing the image received from the user terminal 100 .

As described above, the video providing apparatus 200 may generate the first indexing information by combining at least two or more of the code values generated using ① to ⑦. For example, a preferred embodiment of the present invention In the example, [231 0 1 4 6 1010 00] was generated as the first indexing information by sequentially combining all the codes generated using the above-described methods ① to ⑦. In this case, it goes without saying that the first indexing information may be generated by combining the code values generated by using 3, 4, 5, etc. among the methods of ① to ⑦ in a predefined order.

Meanwhile, the video providing apparatus 200 generates second indexing information indicating the shape characteristics of the object for each extracted object as shown in FIG. 3 . As shown in FIG. 5 , in a preferred embodiment of the present invention, the video providing apparatus 200 generates a Fourier descriptor of each object as the second indexing information. The Fourier descriptor converts a curve on a plane in the frequency domain. As a description method, since the Fourier descriptor itself is a known technology, a detailed description thereof will be omitted.

Finally, in operation S210 , the video providing apparatus 200 generates third indexing information indicating a relationship between each pixel value included in the object image and neighboring pixel values.

6 is a diagram for explaining a process in which the video providing apparatus 200 generates third indexing information according to a preferred embodiment of the present invention.

Referring to FIG. 6 as an example, the video providing apparatus 200 normalizes each object extracted from the video image to an image of a predefined size. 6 , in a preferred embodiment of the present invention, the video providing apparatus 200 normalized the extracted object image to an image having a size of 32*40, but the size of the normalized image can be changed according to embodiments. do.

Thereafter, the video providing apparatus 200 compares the pixel value of each pixel of the normalized object image 601 with the values of eight surrounding pixels, and assigns an indexing pixel value to the corresponding pixel according to the comparison result. Create indexing information (indexing image).

In the example shown in FIG. 6 , when the original pixel value of the pixel to which the indexing pixel value is to be assigned is 163 (refer to reference numeral 602), the pixel value is sequentially set with the pixel values 194,183,173,142,114,167,191,191 of 8 surrounding pixels. , if the value of the neighboring pixel is larger, 1 is assigned to the neighboring pixel, and if the value of the neighboring pixel is smaller, 0 is assigned to the neighboring pixel (refer to reference numeral 603). In the example shown in FIG. 6 , the values given to the surrounding 8 pixels are 1,1,1,0,0,1,1,1, and when expressed in 8 bits by connecting them, 11100111=233. The video providing apparatus 200 assigns the thus obtained value 233 as an indexing pixel value of the corresponding pixel (refer to reference numeral 604).

When indexing pixel values are assigned to all pixels normalized in the same way, third indexing information (indexing image) as shown in reference numeral 605 of FIG. 6 is generated. FIG. 6 illustrates third indexing information generated with respect to the object image used to generate the first indexing information in FIG. 4 .

Meanwhile, with reference to FIGS. 7A and 7B, in step S240 to be described later, the third indexing information (indexing image) is compared with other third indexing information (indexing image) or third search information to measure the degree of matching First, explaining the process, since the sizes of the third indexing information (indexing image) (A) and the third indexing information (indexing image) (B) are normalized, there are pixels corresponding to each other, and the pixel value of each pixel is composed of a predefined number of bits (8 bits in the case of FIG. 7A).

Accordingly, the video providing apparatus 200 compares the pixel values corresponding to each bit, and if a predefined number of pixel values match, it is determined that the corresponding pixels match each other (refer to FIG. 7A ), and in the total number of pixels If the number of pixels matched with each other is greater than or equal to a predefined threshold ratio, it is determined that the images compared to each other are the same image, and if the number of pixels is less than the threshold ratio, it is determined that they are different images (see FIG. 7B ).

In a preferred embodiment of the present invention, since the pixel is determined to be a mutually identical pixel when 6 or more bit values of 8 bits representing a pixel value match, the example shown in FIG. 7A corresponds to a pixel that does not match with each other.

In addition, in the preferred embodiment of the present invention, since 90% or more of all pixels are judged to be the same image, images with a degree of agreement of 30% in the example shown in FIG. 7B do not match each other, and images with a degree of agreement of 100% are judged to be consistent with each other.

Referring back to FIG. 2 , when indexing information is generated for an object included in a main scene of a video by performing the above-described process, object identification information is classified according to the indexing information and stored in correlation with video data (S220) .

8 is a diagram illustrating an example in which object identification information is stored according to indexing information according to a preferred embodiment of the present invention. Referring to FIG. 8 , object identification information for each object is first classified according to first indexing information. The video providing apparatus 200 may define object identification information having the same first indexing information as one classification, and may define information having similar values of the first indexing information as one classification. In the example shown in FIG. 8 , in a preferred embodiment of the present invention, object identification information having the same first indexing information is primarily classified.

Thereafter, the video providing apparatus 200 classifies the object identification information first classified according to the first indexing information, and secondarily classifies the object identification information according to the second indexing information. In a preferred embodiment of the present invention, a Fourier descriptor is used as the second indexing information, and the object identification information is grouped and classified so that information having a similar waveform of the Fourier descriptor among the object identification information belongs to the same group.

Finally, for each group classified according to the second indexing information, third indexing information is stored, and object identification information corresponding thereto is stored. The object identification information stored in association with the third indexing information includes a plurality of fields. Basically, the video identification information (eg, video file name, video code, video serial number, etc.) including the corresponding object and the corresponding object are displayed. Include time information in the video. In addition, the object identification information may further include first indexing information and object information (person, car, chair, etc.) in addition to video identification information and time information.

In the example shown in FIG. 8 , the object identification information is stored in association with the third indexing information by sequentially combining first indexing information, object information, video identification information, and time information.

Steps S210 and S220 described so far are performed whenever a new video is additionally stored in the storage device.

On the other hand, when the user checks the image captured from the video using the user terminal 100 and transmits the image (search image) to the video providing device 200 to view the video corresponding thereto, it is transmitted to the video providing device ( 200), the video providing apparatus 200 extracts an object from the search image, and then generates search information for each extracted object (S230).

The search information for each object is substantially the same as the indexing information generated in step S210, and the search information consists of first to third search information, which are the first to third indexing information described in step S210. Each of the indexing information is generated in the same manner, and the configuration thereof is also the same, so a detailed description thereof will be omitted.

After the search information for each object is generated, the video providing apparatus 200 searches for indexing information corresponding to each search information, and checks the video corresponding to the image requested by the user and the time information at which the image is played ( S240).

To describe step S240 in more detail, first, the video providing device 200 compares the first search information with the first indexing information stored in the storage device 220, and is the same as or corresponding to the first search information. The object identification information classified into the first indexing information is selected.

For example, in the example shown in FIG. 8 , if the first search information generated in step S230 is 0100010221057, the video providing apparatus 200 is an object in which the first indexing information among the object identification information stored in the storage device is classified as 0100010221057 Identification information is selected.

Thereafter, the video providing apparatus 200 searches for second indexing information indicating the same or most similar waveform to the Fourier descriptor as the second search information, and secondarily selects object identification information classified as the searched second indexing information.

Finally, the video providing apparatus 200 searches for third indexing information that matches or corresponds to the third search information from among the secondarily selected object identification information. As described above, the third indexing information and the third search information are implemented as images of a predefined size, and the third indexing information and the third search information are compared between corresponding pixels to determine whether they match.

In a preferred embodiment of the present invention, the third indexing information and the third search information are images of a size of 32*40, each pixel is implemented with 8 bits, and 6 or more of the 8 bits are compared between the corresponding bits of each pixel value. If the bits match, it is determined as the same bit, and when the number of pixels determined as the same bit exceeds a predefined ratio (eg, 90%), the third search information and the third indexing information are determined to be the same. A method of determining whether the third search information and the third indexing information correspond is the same as described with reference to FIGS. 7A and 7B .

If indexing information corresponding to the search information does not exist by performing step S240, the video providing apparatus 200 notifies the user terminal 100 that the video requested by the user terminal 100 does not exist (S250, S260).

On the other hand, if the third indexing information corresponding to the third search information exists, the video providing apparatus 200 checks the video data included in the object identification information corresponding to the indexing information, and starts the corresponding video from the time indicated by the time information. A video is provided to the user terminal 100 in a streaming method or a download method so that the data can be reproduced in the user terminal 100 (S270).

When there are a plurality of objects included in the image requested by the user terminal 100, the above-described step S240 is performed for each object, in most cases, the video and time information indicated by the video identification information searched for each object match do. However, when the video and time information searched for a plurality of objects are different from each other, a video corresponding to a search result of more objects is selected. For example, if two of the three objects extracted from the requested image correspond to the video A and one object corresponds to the video B, it is determined that the video A is the video requested by the user terminal 100 . If it is determined that all of the plurality of objects correspond to different videos, the video providing apparatus 200 determines the matching video based on the object having the highest matching rate between the third search information and the third indexing information.

The video providing method according to the preferred embodiment of the present invention described so far may be implemented as a computer program stored in a non-transitory storage medium by being implemented as instructions executable by a computer.

The storage medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable storage media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable storage medium is distributed in network-connected computer systems, and computer-readable codes can be stored and executed in a distributed manner.

So far, with respect to the present invention, the preferred embodiments have been looked at. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

100: user terminal
200: video providing device
210: processor
220: storage device
230: input unit
240: output unit

Claims (14)

delete (a) object identification information for each of the objects displayed for each scene of each video by the video providing device for a plurality of videos (including video identification information including the corresponding object and time information at which the object is displayed) and generating indexing information;
(b) storing, by the video providing device, the object identification information and the indexing information in correlation with video data;
(c) when the video providing apparatus receives the requested image from the user terminal, extracting an object from the requested image, and then generating search information for each extracted object in the same manner as the indexing information;
(d) checking, by the video providing apparatus, indexing information corresponding to the search information and object identification information corresponding to the indexing information, and checking time information at which the video corresponding to the requested image and the requested image are reproduced; and
(e) providing, by the video providing device, video data to the user terminal so that the video corresponding to the requested image can be played from the time corresponding to the time information,
The indexing information of step (a) is among the codes generated by the following ① to ④
① A code generated by obtaining a color histogram for an object image and sequentially combining numbers representing color components according to the size of the color components.
② Code assigned according to the ratio of the area of the object image to the circumscribed ellipse tangent to the object image
③ Code assigned according to the aspect ratio of the object image
④ Code assigned according to the ratio of the area of the object image to the circumscribed rectangle touching the object image
A method for providing a video, characterized in that it includes first indexing information generated by combining two or more. 3. The method of claim 2,
The first indexing information is among the codes generated by ① to ④ and the following ⑤ to ⑦
⑤ Code assigned according to the gradient of the object image
⑥ Code assigned using the gradient between the extreme points of the object image
⑦ Code assigned using the object recognition result
A method of providing a video, characterized in that it is generated by combining two or more. 4. The method of claim 3, wherein the indexing information is
Further comprising second indexing information implemented as a Fourier descriptor generated for the object image,
In step (b), the video providing device provides a video characterized in that the object identification information is primarily classified according to the first indexing information, and then classified and stored secondarily according to the second indexing information. Way. 5. The method of claim 4,
In step (c), the video providing device generates first search information for the object extracted from the request image in the same manner as the first indexing information, and the requested image in the same manner as the second indexing information Creates second search information for the extracted object,
In step (d), the video providing apparatus is classified primarily according to the first indexing information corresponding to the first search information, among the object identification information stored in the step (b), and added to the second search information. A video characterized in that the video corresponding to the requested image is checked according to video identification information included in the object identification information classified secondly according to the corresponding second indexing information, and information about the time at which the requested image is displayed is checked How to provide. 5. The method of claim 4,
The indexing information further includes third indexing information indicating a relationship between each pixel value included in the object image and neighboring pixel values,
In the step (b), the video providing device primarily classifies the object identification information according to the first indexing information, and secondarily classifies it according to the second indexing information and stores it together with the third indexing information A method of providing a video, characterized in that. 7. The method of claim 6,
In step (c), the video providing apparatus provides first search information, second search for the object extracted from the requested image in the same manner as the first indexing information, the second indexing information, and the third indexing information. information, and third search information, respectively;
In step (d), the video providing apparatus is classified primarily according to the first indexing information corresponding to the first search information, among the object identification information stored in the step (b), and added to the second search information. Secondary classification is made according to the corresponding second indexing information, and according to the video identification information included in the object identification information in which the third search information and the third indexing information match more than a predefined threshold ratio, the requested image is added to the image. A method of providing a video, characterized in that by checking a corresponding video and checking time information for displaying the requested image.
A computer program that is stored in a non-transitory storage medium and is executed in a computer including a processor to perform the method for providing a video according to any one of claims 2 to 7.
delete A video providing device comprising a processor and a storage device for storing predetermined instructions,
The processor executing the instructions stored in the storage device
(a) For a plurality of videos, object identification information (including video identification information including the object and time information at which the object is displayed) and indexing information for each of the objects displayed for each scene of each video generating;
(b) storing the object identification information and the indexing information in association with video data;
(c) upon receiving the requested image from the user terminal, extracting an object from the requested image, and then generating search information for each extracted object in the same manner as the indexing information;
(d) searching for indexing information corresponding to the search information and object identification information corresponding to the indexing information, and checking time information at which a video corresponding to the requested image and the requested image are reproduced; and
(e) performing a video providing method by performing the step of providing video data to the user terminal so that the video corresponding to the requested image can be played from the time corresponding to the time information,
The indexing information of step (a) is among the codes generated by the following ① to ④
① A code generated by obtaining a color histogram for an object image and sequentially combining numbers representing color components according to the size of the color components.
② Code assigned according to the ratio of the area of the object image to the circumscribed ellipse tangent to the object image
③ Code assigned according to the aspect ratio of the object image
④ Code assigned according to the ratio of the area of the object image to the circumscribed rectangle touching the object image
A video providing apparatus, characterized in that it includes first indexing information generated by combining two or more. 11. The method of claim 10,
The first indexing information is among the codes generated by ① to ④ and the following ⑤ to ⑦
⑤ Code assigned according to the gradient of the object image
⑥ Code assigned using the gradient between the extreme points of the object image
⑦ Code assigned using the object recognition result
A video providing device, characterized in that generated by combining two or more. 12. The method of claim 11,
The indexing information further includes second indexing information implemented as a Fourier descriptor generated for the object image,
In the step (b), the object identification information is primarily classified according to the first indexing information, and is secondarily classified and stored according to the second indexing information. 13. The method of claim 12,
In the step (c), first search information is generated for the object extracted to the request image in the same manner as the first indexing information, and in the same way as the second indexing information, it is added to the object extracted from the request image. to create second search information for
In step (d), from among the object identification information stored in step (b), first classification is performed according to first indexing information corresponding to the first search information, and second indexing corresponding to the second search information is performed. A video providing apparatus, characterized in that the video corresponding to the requested image is checked according to video identification information included in the object identification information classified secondarily according to the information, and information about a time at which the requested image is displayed is checked. 13. The method of claim 12,
The indexing information further includes third indexing information indicating a relationship between each pixel value included in the object image and neighboring pixel values,
In the step (b), the object identification information is primarily classified according to the first indexing information, and is secondarily classified according to the second indexing information and stored together with the third indexing information,
In step (c), first search information, second search information, and third Each of the search information is created,
In step (d), from among the object identification information stored in step (b), first classification is performed according to first indexing information corresponding to the first search information, and second indexing corresponding to the second search information is performed. The video is classified according to the information, and the video corresponding to the requested image is identified according to the video identification information included in the object identification information in which the third search information and the third indexing information match more than a predefined threshold ratio. And, the video providing device, characterized in that for checking the information on the display time of the requested image.

Images