KR20120036494A - Method and system for providing real-time broadcasting information - Google Patents

Abstract

PURPOSE: A method for providing real time broadcasting information and a system thereof are provided to offer basic information and additional information for a broadcasting program by photographing a broadcasting video with a terminal while watching the broadcasting program. CONSTITUTION: A terminal(200) photographs a broadcasting video of a real time broadcasting program. A broadcasting information providing device(100) generates a plurality of video search information corresponding to a plurality of broadcasting programs. The broadcasting information providing device compares video query information with the plurality of video search information when requested for broadcasting program information by the terminal. The broadcasting information providing device transmits the broadcasting program information with high similarity to the video search information to the terminal.

Description

Method and system for providing real-time broadcasting information

The present invention relates to a method and system for providing real-time broadcast information, and more particularly, to provide real-time broadcast information that can be simply provided with various information of a corresponding broadcast program by using a broadcast image of a broadcast program that is currently being broadcast without complicated search work. It relates to a method and a system.

Recently, smart phones equipped with a high-performance processor such as a PC and a large memory and allowing a user to freely install and use an application are rapidly spreading.

A smart phone is an intelligent terminal that integrates mobile phone functions with data communication functions such as schedule management, fax transmission, and Internet access. It is not only able to directly access the Internet using the wireless Internet, but also can be accessed in various ways using various browsing programs. Support for an open platform, create your own applications, install a variety of applications to implement your own interface, and share applications among smartphones with the same operating system (OS). Due to the advantages that existing mobile phones do not have, such as can do, the mobile phone market is rapidly encroaching.

In addition, attempts have been made to provide content or services to which various ideas are applied based on such smartphones.

On the other hand, real-time broadcasting services are provided through various communication media such as airwaves, satellite signals, cables, and the Internet. However, in order to receive various additional information related to a broadcasting program that is being broadcasted, a user directly accesses the Internet through a terminal such as a PC or a smartphone, and then inputs a keyword or the like to search for desired information or connects to a specific server. You must request service.

However, the conventional method has a problem in that it is difficult to obtain accurate information when the name of the broadcasting program is not well known. In particular, the existing method of searching by keyword through a separate terminal distributes the concentration of the user. , There is a problem that disturbs the immersion to the broadcast program, increases the user's inconvenience.

The present invention is proposed in order to solve the conventional problems, a method and system for providing real-time broadcast information that can be simply provided with a variety of information of the broadcast program by using the broadcast image of the broadcast program currently being broadcast without complicated search work To provide.

According to an aspect of the present invention, there is provided a means for solving a problem, comprising: a terminal for capturing a broadcast image of a real-time broadcast program that is being broadcast through a video device, extracting image query information from the captured broadcast image, and transmitting the image query information together with a broadcast program information request; And generating a plurality of image search information corresponding to a plurality of broadcast programs by collecting broadcast images of a real time broadcast program in real time, and receiving a broadcast program information request, the image query information and the plurality of images included in the broadcast program information request. And a broadcast information providing apparatus for comparing search information and transmitting information of a broadcast program corresponding to image search information having a high similarity.

In the real-time broadcast information providing system according to the present invention, the broadcast information providing device is one or more of a PC, laptop, tablet PC, mobile phone, smart phone, PDA (Personal Digital Assistant), PMP (Portable Multimedia Player), digital broadcast receiver Characterized in that it is implemented as a user terminal device or a server device comprising.

In addition, the present invention provides another means for solving the problem, a broadcast receiving unit for receiving a broadcast signal of a plurality of real-time broadcast programs provided for each of a plurality of communication media and channels; An information collecting unit collecting basic information on a plurality of broadcast programs broadcast in real time and image search information on broadcast images of the plurality of broadcast programs in real time; A storage unit for matching and storing basic information and image search information of a plurality of broadcast programs collected by the information collection unit; A service for receiving a broadcast program information request transmitted from a terminal and comparing the video query information included in the broadcast program information request with a plurality of video search information to transmit information of a broadcast program corresponding to the video search information with high similarity to the terminal. It provides a broadcast information providing apparatus comprising a providing unit.

In the broadcast information providing apparatus according to the present invention, the basic information of a broadcast program may include at least one of a channel name, a region-specific channel number, a broadcast program name, a broadcast date, a broadcast start time, and a broadcast end time.

In the broadcast information providing apparatus according to the present invention, the image retrieval information includes at least one of image characteristic information on the broadcast image and visual information on which the broadcast image from which the image characteristic information is extracted is collected.

In the broadcast information providing apparatus according to the present invention, the video query information includes at least one of video feature information on the broadcast video, time information from which the broadcast video from which the video feature information is extracted, and local information from which the broadcast video is collected. It may include.

In the broadcast information providing apparatus according to the present invention, broadcast program information includes a channel name of a broadcast program, a program name, a time difference between an actual broadcast time and a video collection time of a terminal, a sequence number of a broadcast video, and a broadcast time of a corresponding broadcast video. It may include one or more.

In the apparatus for providing broadcast information according to the present invention, the information collector extracts program guide information included in a broadcast signal received by the broadcast receiver, or receives broadcast program information for each communication medium or channel, and collects basic information of a broadcast program. Characterized in that.

In the apparatus for providing broadcast information according to the present invention, the information collecting unit captures a broadcast image from a broadcast signal received by the broadcast receiving unit at a predetermined period, analyzes the captured broadcast image, and extracts image feature information to collect the image search information. It is characterized by.

In the apparatus for providing broadcast information according to the present invention, the information collecting unit converts the captured broadcast image into one or more resolutions applied to the terminal before extracting the image characteristic information so as to extract the image characteristic information in an environment similar to the terminal. Can be.

In the broadcast information providing apparatus according to the present invention, the storage unit may store a plurality of image search information collected within a predetermined time on the basis of the current time.

In addition, the present invention is a means for solving the problem, a communication unit for transmitting and receiving data through a communication network; An input unit configured to generate a user input signal according to a user's manipulation; A camera unit for capturing a real-time broadcast image reproduced through an image device; And a controller for analyzing the real-time broadcast image captured by the camera unit to extract image query information according to a user input signal, and requesting broadcast program information by transmitting the image query information to the broadcast information providing apparatus through a communication unit. And a display unit which displays an operation result of the terminal under the control of the controller.

In the terminal according to the present invention, the control unit receives the broadcast program information transmitted from the broadcast information providing apparatus in response to the request and outputs it to the display unit, or compares the image query information and the plurality of image search information, the image search information with high similarity Information of the broadcast program corresponding to may be extracted and output to the display unit.

In the terminal according to the present invention, the image query information may include at least one of image feature information on the broadcast image, time information from which the broadcast image from which the image feature information is extracted, and region information from which the broadcast image is collected. have.

In the terminal according to the present invention, the broadcast program information is one of a channel name of the broadcast program, a program name, the time difference between the actual broadcast time and the video collection time of the terminal, the sequence number of the corresponding broadcast video, and the broadcast time of the broadcast video. It may contain the above.

The terminal according to the present invention may further include a storage unit for storing one or more of the broadcast image photographed by the camera unit, the image query information extracted by the controller, a plurality of image search information, and broadcast program information.

The terminal according to the present invention further includes a GPS unit for receiving a GPS satellite signal and calculating a position of the terminal, wherein the controller extracts collection area information of the real-time broadcast image through the GPS unit.

In the terminal according to the present invention, the control unit extracts the image query information by combining the analysis results of the two or more broadcast images taken continuously or a predetermined number of times for a predetermined time.

The terminal according to the present invention further includes a motion detecting unit for detecting a movement of the terminal, and when the controller checks the movement of the terminal while the broadcast image is being photographed through the motion detecting unit, a movement outside the reference range occurs. The broadcast image captured at the viewpoint may be excluded from the extraction target of the image query information.

In the terminal according to the present invention, the control unit excludes the captured broadcast image from the extraction target of the image query information when a difference in the reference range or more occurs by comparing the broadcast image photographed by the camera unit with the previously captured broadcast image. Can be.

In addition, the present invention is another means for solving the problem, the step of collecting the basic information of a plurality of broadcast programs broadcast in real time; And collecting image broadcast information of a plurality of broadcast programs currently being broadcast, and extracting image search information. The present invention provides a method for providing real-time broadcast information including matching and storing a plurality of basic information and image search information.

According to an aspect of the present invention, there is provided a method of providing real-time broadcast information, the method comprising: receiving a broadcast program information request message including image query information extracted from a broadcast video being viewed; Searching for the requested broadcast program information by comparing the image query information with the image search information; And transmitting the searched broadcast program information.

In the real-time broadcast information providing method according to the present invention, the step of collecting the basic information of the broadcast program is a step of receiving program guide information transmitted with the broadcast signal for each communication medium or channel, or broadcast programming information for each communication medium or channel It can be made, including the step of inputting.

In the real-time broadcast information providing method according to the present invention, the step of extracting the image search information, receiving a broadcast signal broadcast in real time for each communication medium or channel; Capturing a broadcast image at a predetermined period from the received broadcast signal; Obtaining visual information capturing the broadcast image; Extracting image feature information from the captured broadcast image; And matching the image characteristic information with the captured time information and storing the image retrieval information.

In the real-time broadcast information providing method according to the present invention, the step of extracting the image search information, before extracting the image feature information, further comprises the step of converting each of the captured broadcast video to one or more resolutions applied to the terminal; The method may extract image feature information in an environment similar to that of a terminal using a broadcast image whose resolution is converted.

In the real-time broadcast information providing method according to the present invention, the step of matching and storing a plurality of basic information and the image search information, characterized in that for storing the video search information within a predetermined time or a predetermined number based on the current time point. do.

In the real-time broadcast information providing method according to the present invention, the step of searching for broadcast program information, the method comprising: comparing the video feature information included in the video query information and the video feature information included in the plurality of video search information; As a result of the comparison, the method may include identifying image search information having the highest similarity and basic information of the broadcast program matched with the image search information.

In the real-time broadcast information providing method according to the present invention, the step of searching for broadcast program information further comprises the step of checking the collection time of the image query information, with reference to the confirmed collection time, the image search information and the image query It is characterized by comparing the information.

In the real-time broadcast information providing method according to the present invention, the step of retrieving the broadcast program information, may include the step of extracting the channel information of the region where the image query information is collected by identifying the region where the image query information is collected. have.

In addition, the present invention as another means for solving the problem, collecting the broadcast image of the real-time broadcast program to be reproduced through the video device; Analyzing the collected broadcast video and extracting video query information for identifying the broadcast; Requesting information on a broadcast program corresponding to the image query information by transmitting the extracted image query information; And receiving the retrieved broadcast program information by using the image query information in response to the request.

In the real-time broadcast information providing method according to the present invention, the step of collecting the broadcast image of the real-time broadcast program, may include the step of continuously photographing the broadcast image of the real-time broadcast program for a predetermined period or a predetermined number of times for a predetermined time. have.

In the real-time broadcast information providing method according to the present invention, the step of collecting the broadcast image of the real-time broadcast program, the step of sensing the movement of the terminal while photographing the broadcast image; And if the detected movement of the terminal is out of the reference range, removing the broadcast image photographed at the corresponding time point, and excluding the image query information from the target.

In the real-time broadcast information providing method according to the present invention, the step of collecting the broadcast image of the real-time broadcast program, Comparing the previously recorded broadcast image of the captured broadcast image; And if the difference between the previously captured broadcast image and the reference range is greater, removing the captured broadcast image and excluding the image query information from the target to extract the image query information.

In the real-time broadcast information providing method according to the present invention, the step of extracting the image query information, the step of extracting the image characteristic information by analyzing the collected broadcast image; Extracting collection time information of the collected broadcast image; And acquiring at least one region information from which the broadcast image is collected, and generating at least one combination of the image feature information, the collection time information, and the region information.

The present invention collects the image search information extracted from the broadcast image of the program broadcast in real time in the broadcast information providing apparatus in real time, and photographs the broadcast image aired through the video device in the terminal to extract the image query information therefrom broadcast information When a request is made to the providing apparatus, the broadcasting information providing apparatus compares the image query information and the image search information to identify a broadcasting program matched based on the similarity level and provides related information so that the user is watching through the terminal while watching the broadcasting program. With a simple operation of capturing a broadcast image, there is an excellent effect of providing basic information and various additional information about a broadcast program being viewed.

In addition, the present invention searches for a matching broadcast program using feature information of the broadcast video, so that the user can be provided with information on the broadcast program even if the user does not know the name of the broadcast program or a related keyword. Has an excellent effect of improving.

1 is a block diagram showing the overall configuration of a system for providing real-time broadcast information according to the present invention.
2 is a block diagram showing a detailed configuration of a broadcast information providing apparatus in a real-time broadcast information providing system according to the present invention.
3 is a block diagram showing a detailed configuration of a terminal in a system for providing real-time broadcast information according to the present invention.
4 is a message flow diagram illustrating the overall flow of a method for providing real-time broadcast information according to the present invention.
5 is a flowchart illustrating a process of collecting basic program information of a broadcast program in a method for providing real-time broadcast information according to the present invention.
6 is a flowchart illustrating a process of collecting image search information of a broadcast information providing apparatus in a method for providing real time broadcast information according to the present invention.
7 is a flowchart illustrating in detail a process of collecting broadcast images of a terminal in a method for providing real-time broadcast information according to the present invention.
8 is a flowchart illustrating a process of generating video query information of a terminal in a method for providing real-time broadcast information according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, in the following description and the accompanying drawings, detailed descriptions of well-known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that like elements are denoted by the same reference numerals as much as possible throughout the drawings.

1 is a block diagram showing the overall configuration of a system for providing real-time broadcast information according to the present invention.

Referring to FIG. 1, the real-time broadcast information providing system according to the present invention is implemented by the broadcast information providing apparatus 100 and the terminal 200 connected through the communication network 300. In FIG. 1, reference numeral 400 denotes a video device that receives and reproduces a real-time broadcast signal.

The broadcast information providing apparatus 100 collects broadcast images of a real-time broadcast program in real time to generate a plurality of image search information corresponding to a plurality of broadcast programs, and generates the plurality of image search information with the basic information of each broadcast program. When matching and storing, and receiving a broadcast program information request, and compares the video query information and the plurality of video search information contained in the broadcast program information request, and transmits the information of the broadcast program corresponding to the video search information with high similarity .

Here, the basic information of the broadcast program is information directly related to each broadcast program broadcast in real time, and includes one or more of a channel name, a region-specific channel number, a broadcast program name, a broadcast date, a broadcast start time, and a broadcast end time. Include.

In addition, the image search information is information provided as a reference for a broadcast program search through an image provided in the broadcast information providing apparatus 100, and the image query information is information indicating a broadcast image that the user is watching, and the terminal 200 Is extracted from. The image search information and the image query information may include at least one of image feature information extracted from a broadcast image captured by a broadcast signal of a real-time broadcast program and visual information from which a broadcast image from which the image feature information is extracted is collected. Can be. In particular, the image query information may further include region information from which the broadcast image is collected. In addition, the video feature information included in the video search information and the video query information, which represents the feature of the corresponding broadcast video, may include channel information or channel name exposed to any one of up, down, left, and right portions of the broadcast video. .

The broadcast program information provided from the broadcast information providing apparatus 100 to the terminal 200 includes information directly related to the broadcast program, and various additional information provided in association with the broadcast program. It may include. In particular, the broadcast program information may include at least one of a channel name of the corresponding broadcast program, a program name, a time difference between an actual broadcast time and a video collection time of the terminal, a sequence number of the corresponding broadcast video, and a broadcast time of the broadcast video. Can be. The broadcast time of the broadcast video may be expressed as a time when the corresponding broadcast video is broadcast based on the broadcast start time of the corresponding broadcast program.

The broadcast information providing apparatus 100 may be implemented through one server device or a plurality of server devices, and may include a PC, a notebook computer, a tablet PC, a mobile phone, a smartphone, a personal digital assistant (PDA), and a portable multimedia player (PMP). ) May be implemented as a user terminal device or a server device including one or more of the digital broadcast receivers.

The terminal 200 is a user-side device, which captures a broadcast image of a live broadcast program that is being broadcast through an arbitrary video device 400, extracts image query information from the broadcast image, and provides broadcast information together with a broadcast program information request. After transmitting to the device 100, and receives the information of the broadcast program corresponding to the captured broadcast image provided from the broadcast information providing apparatus 100 to output to the user.

The terminal 200 may be a user device of various types, for example, a personal computer (PC), a notebook computer, a mobile phone, a tablet PC, navigation, a smart phone, It may include a personal digital assistant (PDA), a portable multimedia player (PMP), or a digital broadcasting receiver. Of course, this is only an example, and in addition to the above-described example, it may include any device capable of communication that is currently developed and commercialized or will be developed in the future. However, the terminal 200 preferably has a camera function.

The communication network 300 is for transmitting and receiving data between the broadcast information providing apparatus 100 and the terminal 200, and transmits a broadcast program information request of the terminal 200 to the broadcast information providing apparatus 100, and provides broadcast information. The broadcast program information of the device 100 is transmitted to the terminal 200. The communication network 300 is, for example, an IP network for providing a data transmission / reception service and a disconnected data service through an Internet protocol, an IP network structure in which different networks are integrated based on IP, Various known methods such as Wibro (Wireless Broadband) network, Wireless LAN including Wi-Fi network, WPAN, wired communication network, mobile communication network, High Speed Downlink Packet Access (HSDPA) network and satellite communication network It can be made by including one or more networks of, in addition, it may include a next-generation communication network to be developed in the future.

In addition, the video device 400 is a video device that receives a real-time broadcast signal and plays a broadcast program. For example, the video device 400 may include a TV, a PC including a TV reception card, and the like.

Next, the configuration and operation of the broadcast information providing apparatus 100 and the terminal 200 will be described in more detail with reference to FIGS. 2 and 3.

2 is a block diagram showing the detailed configuration of the broadcast information providing apparatus 100 in the real-time broadcast information providing system according to the present invention.

Referring to FIG. 2, the broadcast information providing apparatus 100 may include a broadcast receiving unit 110, an information collecting unit 120, a storage unit 130, and a service providing unit 140.

The broadcast receiver 110 receives broadcast signals of a plurality of real time broadcast programs provided for each of a plurality of communication media and channels. To this end, the broadcast receiving unit 110 may include a plurality of broadcast receiving modules for receiving and processing broadcast signals transmitted through corresponding communication media in correspondence with a plurality of communication media and channels.

The information collector 120 collects basic information about a plurality of broadcast programs broadcast in real time and image search information about broadcast images of the plurality of broadcast programs in real time.

Here, the basic information of the broadcast program may include one or more of a channel name, a channel number for each region, a broadcast program name, a broadcast date, a broadcast start time, and a broadcast end time. Here, the channel number for each region is intended to reflect the difference in the channel number for which broadcasting is provided depending on the region.

In order to collect such basic information, the information collector 120 extracts program guide information (EPG) included in the broadcast signal received by the broadcast receiver 110 or inputs broadcast program information for each communication medium or channel. In response, basic information of the broadcast program may be collected. That is, in the case of a channel providing program guide information together, program guide information is extracted from the broadcast signal received by the broadcast receiver 110, and basic information is collected from the extracted program guide information. On the other hand, in the case of a channel that does not provide program guide information, the program receives the program information distributed through a newspaper or an Internet site, and collects basic information from the input program information.

The image search information may include at least one of image feature information of a broadcast image and visual information of a broadcast image from which the image feature information is extracted. In order to collect such image search information, the information collecting unit 120 captures a broadcast image from a broadcast signal received by the broadcast receiving unit 110 at regular intervals, and extracts image feature information from each captured broadcast image. Collect by video search information. In order to extract the image feature information, the information collector 120 may use a feature point extraction algorithm. In addition, the image feature information of the extracted image search information is compared with the image feature information included in the image query information transmitted from the terminal 200 and used to identify a corresponding broadcast program. Therefore, the information collecting unit 120, before extracting the image characteristic information in order to extract the image characteristic information in the environment similar to the terminal 200, at least one applied to the terminal 200 to the captured broadcast image The resolution may be converted and image characteristic information may be extracted for each resolution. Accordingly, the accuracy of the search result may be improved by increasing the similarity with the image characteristic information transmitted from the terminal 200. The image characteristic information may be information that is a feature of the corresponding broadcast image, and may include, for example, channel information or a channel name or a program name exposed to any one of up, down, left, and right portions of the broadcast video.

The storage unit 130 matches and stores basic information and image search information of a plurality of broadcast programs collected by the information collecting unit 120. On the other hand, even in the case of real-time broadcasting, there may be a delay in signal transmission depending on a communication medium or a transmission method. Therefore, the time point when the broadcast information providing apparatus 100 receives the same broadcast program and the video device 400 Some time difference may occur between the points of time of reception and playback. Accordingly, the broadcast information providing apparatus 100 may compare the plurality of image search information continuously collected within a predetermined time range with the image query information of the terminal 200 to overcome such a time difference. To this end, the storage unit 130 stores image search information continuously collected within a predetermined time or a predetermined number of times based on the current time. Therefore, the storage unit 130 may manage data in a first in first out (FIFO) manner. That is, when new image search information is collected by the government collecting unit 120, the oldest image search information is deleted and the newly collected image search information is stored.

The service provider 140 receives the broadcast program information request transmitted from the terminal 200, compares the image query information included in the broadcast program information request with the plurality of image search information, and then applies the image search information with high similarity. Information on the corresponding broadcast program is transmitted to the terminal 200. In addition, in checking the corresponding broadcast program from the image query information, the service providing unit 140 is further provided with region information from which the image query information is collected, that is, region information where the terminal 200 is located. Based on the information, the broadcast program or channel number broadcasted in the region may be checked. The service provider 140 provides the broadcast program information corresponding to the broadcast image photographed by the terminal 200 to the terminal 200 through the above. In this case, the broadcast program information may include a channel name, a program name, a real name of a broadcast program. One or more of a time difference between a broadcast time and a video collection time of the terminal, a sequence number of the corresponding broadcast video, and a broadcast time of the broadcast video. The broadcast program information may be used by the user terminal 200 to request additional information or a service related to the broadcast program. In addition, the broadcast program information may further include various additional service information related to the broadcast program. The service provider 140 may further execute a user authentication function to provide a service only to an authenticated user.

3 is a block diagram showing a detailed configuration of a terminal 200 in the real-time broadcast information providing system according to the present invention.

Referring to FIG. 3, the terminal 200 according to the present invention includes an input unit 210, a camera unit 220, a GPS unit 230, a communication unit 240, a display unit 250, and a storage unit. 260, a controller 270, and a motion detector 280 may be included.

The input unit 210 generates a user input signal according to a user's operation. For example, a key input means such as a keyboard or a keypad, a touch input means such as a touch screen or a touch pad, a specific movement of the terminal 200 It may include one or more of a gesture input means for generating a user input signal by recognizing a specific movement of the user, and a voice input means for generating a user input signal by discriminating a specific voice signal of the user.

The camera unit 220 is a means for capturing a picture or a video. In the present invention, the camera unit 220 is used for capturing a broadcast video of a real-time broadcast program broadcast by the video device 400.

The GPS unit 230 receives a GPS satellite signal and calculates a location of the terminal 200.

The communication unit 240 transmits and receives data through the communication network 300, and more specifically, transmits a broadcast program information request to the broadcast information providing apparatus 100 and receives broadcast program information from the broadcast information providing apparatus 100. do. The communication unit 240 may be implemented differently according to the communication network 300, and may include a plurality of wired and wireless communication modules to transmit and receive data through various communication networks.

The display unit 250 is for displaying broadcast program information received under the control of the controller 270. The display unit 250 may be implemented by various types of expression means. For example, a liquid crystal display (LCD) and a TFT-LCD ( Thin Film Transistor-Liquid Crystal Display (LED), Light Emitting Diodes (LED), Organic Light Emitting Diodes (OLED), Active Matrix Organic Light Emitting Diodes (AMOLED), Flexible Displays, 3D Displays It may include a means of expressing, in addition to any present means that can be included in the present development and commercialization or can be implemented according to the future technology development.

The storage unit 260 is for storing an operation program and various application programs and data necessary for the operation of the terminal 200. In particular, in the present invention, the broadcast image captured by the camera unit 220 and the control unit 270 are stored in the storage unit 260. The extracted image query information may be stored. The storage unit 260 is, for example, all, such as RAM (Read Access Memory), ROM (Read Only Memory), hard disk (HDD, Hard Disk Drive), flash memory, CD-ROM, DVD It should be interpreted as a concept involving any kind of storage medium.

The controller 270 operates according to a set operating program or an application program to control the operation and state of the terminal 200 as a whole. In the present invention, in particular, the camera unit 200 captures images according to a user input signal. Extract the video query information by analyzing the real-time broadcast video, and transmit the video query information to the broadcast information providing apparatus 100 through the communication unit 240 to request broadcast program information, and in response to the request, broadcast information. Control to receive the broadcast program information transmitted from the providing device 100. The received program information is output to the display unit 250. In another embodiment, the controller 270 does not transmit the extracted image query information to the broadcast information providing apparatus 100, and directly compares the extracted image query information with a plurality of image search information to obtain high similarity image search information. The broadcast program information corresponding to may be directly searched for. In this case, the storage unit 260 stores a plurality of image search information and broadcast program information to be searched.

The image query information extracted by the controller 270 may include image feature information of the broadcast image captured by the camera unit 220, time information in which the broadcast image from which the image feature information is extracted, and the broadcast image are collected. It may include one or more of the area information collected the image. To this end, the controller 270 collects time information of the time point at which the camera unit 220 captures the broadcast image from a built-in clock or the GPS unit 230, and collects position information of the terminal 200. 230).

As described above, the video feature information included in the video query information is feature information for identifying the corresponding broadcast video and includes channel information, channel name, or program name exposed to any one of up, down, left, and right portions of the broadcast video. It may include. Such image feature information may be extracted through a feature point extraction algorithm in the same manner as the broadcast information providing apparatus 100.

In addition, the controller 270 may extract the image query information by combining analysis results of two or more broadcast images continuously photographed at a predetermined period or a predetermined number of times for a predetermined time. Through this, the terminal 200 may configure more accurate image query information for the broadcast program to receive the information.

In addition, the motion detector 280 detects the degree of movement of the terminal 200. The detection result may be used to filter out inappropriate video from the broadcast video captured by the controller 270.

In more detail, the motion detector 280 detects the movement of the terminal 200 while the camera 220 is photographed. Such movement may be expressed by an angle, a direction, a speed, or the like. The controller 270 checks the movement of the terminal 200 detected by the motion detector 280 before moving to the reference value before generating the image query information from the broadcast image captured by the camera 220. In this case, the broadcast image photographed at the corresponding time point is determined to be an inappropriate image and excluded from the extraction target of the image query information. For example, when the terminal 200 is turned upside down or moves more than a certain angle during shooting, the broadcast image is not accurately captured, and thus the captured image is removed. In addition, the controller 270 compares the broadcast video currently shot with the broadcast video previously shot, and determines that the video is inappropriate when the difference is greater than the normal range based on the previous video. . Through the above process, the control unit 270 may generate image query information by collecting only the broadcast image photographed in an appropriate state.

Next, a method of providing real time broadcast information according to the present invention will be described.

4 is a message flow diagram illustrating the overall flow of a method for providing real-time broadcast information according to the present invention.

Referring to FIG. 4, the broadcast information providing apparatus 100 collects basic information of a plurality of broadcast programs broadcast in real time (S105). The basic information is information describing each broadcast program, and may include one or more of a channel name, a region-specific channel number, a broadcast program name, a broadcast date, a broadcast start time, and a broadcast end time. Collection of such basic information may be performed whenever a broadcast schedule, a schedule, or a change is made.

In addition, the broadcast information providing apparatus 100 collects broadcast images of a plurality of broadcast programs currently being broadcast in real time, and collects image search information (S110). The image search information is information for identifying which broadcast image of a broadcast program, and may include one or more of image feature information of a broadcast image and visual information of a broadcast image from which the image feature information is extracted. have. In this case, the image retrieval information is collected at regular intervals. Preferably, the predetermined period is set such that image search information is collected for all scenes changed in the broadcast program.

When the basic information and the image search information of each of the real-time broadcast programs are collected, the broadcast information providing apparatus 100 matches and stores a plurality of basic information and a plurality of image search information (S115).

On the other hand, the terminal 200 collects the broadcast image of the real-time broadcast program broadcast in the video device 400 at the user's request (S120). For example, when a viewer who sees a specific broadcast program in the middle through a channel change is curious about the corresponding broadcast program, when the user requests a collection of broadcast images by operating the input unit 210 of the terminal 200, the camera unit 220 Captures a broadcast image of a broadcast program of a broadcast program that the user is watching by continuously photographing a broadcast image of a scene broadcast from the video device 400 for a predetermined time.

The terminal 200 extracts video query information for querying the broadcast program from the broadcast video thus collected (S125). The image query information includes image feature information of the captured broadcast image, collection time information of the broadcast image from which the image feature information is extracted, that is, time information of the broadcast image is captured, and the region in which the broadcast image is collected. That is, the terminal 200 may include one or more of the location information of the terminal 200. Meanwhile, in step S120, a plurality of broadcast images may be collected at a predetermined period or a predetermined number of times for a predetermined time, and in step S125. Through the plurality of broadcast images collected continuously, more accurate and detailed video query information may be extracted.

When the video query information is extracted, the terminal 200 transmits a broadcast program information request message including the extracted video query information to the broadcast information providing apparatus 100 to provide information about the broadcast program related to the captured broadcast video. Request (S130).

In response, the broadcast information providing apparatus 100 receiving the broadcast program information request includes a plurality of broadcast query information included in the broadcast program request message and a plurality of broadcast programs collected in real time through step S110. Image search information is compared (S135).

In operation S140, a broadcast program corresponding to the image search information having the highest similarity is identified through the comparison. That is, the broadcast program associated with the broadcast image photographed by the terminal 200 is compared with the image query information and the image search information of the plurality of broadcast programs as a reference to search for a corresponding broadcast program.

In operation S145 ˜ S150, the terminal searches for information on the identified broadcast program and provides the received information to the terminal 200 (S145 ˜ S150). The broadcast program information provided includes one or more of a channel name of the corresponding broadcast program, a program name, a time difference between an actual broadcast time and a video collection time of the terminal, a sequence number of the corresponding broadcast video, and a broadcast time of the broadcast video. .

In the step S135, the broadcast information providing apparatus 100 according to the present invention can further confirm the collection time of the image query information, and with reference to the confirmed collection time, the image search information and the image query information Can be compared. In addition, the broadcast information providing apparatus 100 may further confirm a region where the image query information is collected, and may extract channel information of the region where the image query information is collected.

FIG. 5 is a flowchart illustrating a detailed step (S105) of collecting basic information of a broadcast program in the broadcast information providing apparatus 100 in the above-described real-time broadcast information providing method.

Referring to FIG. 5, in operation S105, the broadcast information providing apparatus 100 receives program guide information (EPG) transmitted with a broadcast signal for each communication medium or channel (205).

In addition, the broadcast information providing apparatus 100 receives broadcast programming information distributed for each communication medium or channel (S210).

Then, the database is extracted by extracting basic information (channel name, channel number broadcasted by region, broadcast program name, broadcast start time, broadcast end time, etc.) for each broadcast program from the received program guide information or the received broadcast program information. (S215, S220).

Through the above process, the broadcast information providing apparatus 100 may collect basic information about a plurality of real time broadcast programs broadcast through a plurality of communication media and a plurality of channels.

6 is a flowchart illustrating a process of collecting image search information of a broadcast information providing apparatus in a method for providing real time broadcast information according to the present invention.

Referring to FIG. 6, in operation S110, the broadcast information providing apparatus 100 first receives a broadcast signal broadcast in real time for each communication medium or channel in operation S305.

In operation S310, a broadcast image is captured from the received broadcast signal at regular intervals. The capture of the broadcast video is preferably made in units of frames.

At about the same time, the broadcast information providing apparatus 100 obtains captured visual information about each captured broadcast image (S315).

In addition, the broadcast information providing apparatus 100 converts the captured broadcast image into one or more resolutions and sizes applied to the terminal 200 in order to obtain image feature information similar to the image feature information extracted from the terminal 200. (S320). Image characteristic information may be extracted in an environment similar to a terminal using the broadcast image having the converted resolution and size.

Subsequently, image characteristics information is extracted by analyzing the broadcast image whose resolution and size are converted after the capturing through the feature point extraction algorithm (S325). The feature extraction algorithm used here is preferably the same as the feature extraction algorithm used in the terminal 200.

The extracted image feature information and capture time information are matched and stored as image search information (S330).

In the above, steps S310 to S330 are repeated at regular intervals, and the image search information stored in step S330 is managed in a predetermined window size for each broadcast program. That is, image retrieval information for the broadcast image up to a certain range is stored based on the current time.

The broadcast information providing apparatus 100 may check the broadcast program by comparing the plurality of image search information stored as described above with the image query information of the terminal 200.

Next, FIG. 7 is a flowchart illustrating in more detail a step (S120) of collecting a broadcast image of a broadcast program to be queried by a user in the terminal 200 in the real-time broadcast information providing method according to the present invention.

Referring to FIG. 7, the terminal 200 captures a broadcast image displayed on the screen of the imaging apparatus 400 according to a user's request (S405). Although the broadcast image may be used as it is, the present invention determines the appropriateness of the captured image in order to generate more accurate image query information.

That is, the terminal 200 detects the movement of the terminal 200 while taking a picture, and checks whether a movement above the set reference range is detected (S410). The movement above the reference range refers to the movement to the extent that the entire broadcast video displayed on the imaging device 400 cannot be stably photographed. For example, the terminal 200 is turned upside down or forwards or backwards, or the terminal 200 It may include movements such as twisting more than a predetermined angle with respect to the front. This reference range may be expressed as a movement angle, speed, etc. of the terminal 200.

When the terminal 200 is moved beyond the reference range in step S410, the broadcast image may not be obtained reliably in many cases. Therefore, the broadcast image photographed at the corresponding point of time is determined to be inappropriate and removed (S425). .

In addition, the terminal 200 may further determine whether the captured broadcast image is an appropriate image by analyzing the broadcast image photographed in operation S405.

In more detail, the terminal 200 compares the broadcast image photographed at the present time with the broadcast image photographed immediately before (S415).

As a result of the comparison, when a difference in the reference range or more occurs between the currently captured broadcast image and the previously captured broadcast image, the captured broadcast image is determined to be an inappropriate image and removed (S420 and S425). Here, when a difference in the reference range or more occurs, for example, when the camera moves up, down, left, and right, and only a part of the broadcast screen of the image device 400 is photographed, a specific object existing in the previously captured broadcast image is currently present. And disappeared from the recorded broadcast image.

When the terminal 200 does not move beyond the reference range as a result of the checking of the step S410 and / or step S420, and the difference between the previous broadcast image and the reference range does not occur, the image is determined as an appropriate broadcast image. The query information is stored for generation (S430).

Steps S405 to S430 may be repeated for a predetermined period or a predetermined number of times for a set time, thereby continuously collecting one or more appropriate broadcast images.

Finally, FIG. 8 is a flowchart illustrating a process of extracting image query information of the terminal 200 in the method of providing real-time broadcast information according to the present invention.

Referring to FIG. 8, in step S125, the terminal 200 may extract image feature information by analyzing the collected broadcast image (S505).

In addition, the terminal 200 may obtain region information on which the captured broadcast image is collected (S510).

In addition, the terminal 200 may extract the collection time information of the collected broadcast image (S515).

The terminal 200 may perform one or more of steps S505 to S515 to extract one or more of image feature information, collection time information, and collection region information. The image query information is generated by combining at least one of the visual information and the local information (S520).

The generated image query information is transmitted to the broadcast information providing apparatus 100 and used to search for a corresponding broadcast program.

The method for providing real-time broadcast information according to the present invention may be implemented in software form readable through various computer means and recorded in a computer-readable recording medium. Here, the recording medium may include a program command, a data file, a data structure, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and constructed for the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. For example, the recording medium may be magnetic media such as hard disks, floppy disks and magnetic tapes, optical disks such as Compact Disk Read Only Memory (CD-ROM), digital video disks (DVD), Magnetic-Optical Media, such as floppy disks, and hardware devices specially configured to store and execute program instructions, such as ROM, random access memory (RAM), flash memory, and the like. do. Examples of program instructions may include high-level language code that can be executed by a computer using an interpreter as well as machine code such as produced by a compiler. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be apparent to those skilled in the art. In addition, although specific terms are used in the specification and the drawings, they are only used in a general sense to easily explain the technical contents of the present invention and to help the understanding of the present invention, and are not intended to limit the scope of the present invention.

The present invention provides the image search information extracted from the broadcast image of the program broadcast in real time in the broadcast information providing apparatus, the terminal to take a video broadcasted through the video device to extract the image query information therefrom to provide broadcast information When requesting to the device, the broadcast information providing device compares the image query information and the image search information to identify a broadcast program matched based on the similarity level and provides related information, so that the user is watching through the terminal while watching the broadcast program. The user can be provided with basic information and various additional information about the broadcast program being watched by a simple operation of photographing an image. As a result, there is an excellent effect of improving user convenience.

100: broadcast information providing apparatus
200: terminal
300: network
400: video equipment
110: broadcast receiving unit
120: information collector
130: storage unit
140: service provider
210: input unit
220: camera unit
230: GPS unit
240: communication unit
250: display unit
260: storage unit
270: control unit

Claims (36)

A terminal for capturing a broadcast image of a real-time broadcast program that is being broadcast through a video device, extracting image query information from the broadcast image, and transmitting the image query information together with a broadcast program information request; And
Collects a broadcast image of a real time broadcast program in real time to generate a plurality of image search information corresponding to a plurality of broadcast programs, receives a request for broadcast program information transmitted from the terminal, and queries the video included in the broadcast program information request. And a broadcast information providing apparatus for comparing the plurality of image search information with the plurality of image search information and transmitting information of a broadcast program corresponding to the image search information with high similarity to the terminal. The apparatus of claim 1, wherein the broadcast information providing apparatus
Real time broadcasting, which is implemented as a user terminal device or a server device including at least one of a PC, a notebook computer, a tablet PC, a mobile phone, a smart phone, a personal digital assistant (PDA), a portable multimedia player (PMP), and a digital broadcast receiver. Informational system. A broadcast receiver configured to receive broadcast signals of a plurality of real-time broadcast programs provided for a plurality of communication media and channels;
An information collection unit collecting basic information on a plurality of broadcast programs broadcast in real time and image search information on broadcast images of the plurality of broadcast programs in real time;
A storage unit for matching and storing basic information and image search information of a plurality of broadcast programs collected by the information collection unit; And
And a service providing unit configured to receive a broadcast program information request, compare the video query information included in the broadcast program information request with the plurality of video search information, and transmit the information of the broadcast program corresponding to the video search information with high similarity. Broadcast information providing apparatus, characterized in that. The method of claim 3, wherein the basic information of the broadcast program
And at least one of a channel name, a regional channel number, a broadcast program name, a broadcast date, a broadcast start time, and a broadcast end time. The method of claim 3, wherein the image search information is
And at least one of video feature information on the broadcast video and visual information on which the broadcast video from which the video feature information has been extracted is collected. The method of claim 3, wherein the image query information is
And at least one of video feature information on the broadcast video, visual information on which the broadcast video from which the video feature information has been extracted, and local information on which the broadcast video is collected. The method of claim 3, wherein the broadcast program information
Broadcasting information providing apparatus comprising at least one of a channel name, a program name, the actual broadcast time of the corresponding broadcast program, the time difference between the video collection time of the terminal, the sequence number of the broadcast image, the broadcast time of the broadcast image . The method of claim 3, wherein the information collecting unit
And extracting program guide information included in the broadcast signal received by the broadcast receiving unit, receiving broadcast program information for each communication medium or channel, and collecting basic information of the broadcast program. The method of claim 3, wherein the information collecting unit
And capturing a broadcast image from a broadcast signal received by the broadcast receiver at a predetermined period, analyzing the captured broadcast image, extracting image feature information, and collecting the captured image information as image search information. The method of claim 9, wherein the information collecting unit
And extracting the captured broadcast image into one or more resolutions applied to the terminal before extracting the image feature information so as to extract the image feature information in an environment similar to that of the terminal. The method of claim 3, wherein the storage unit
And a plurality of image retrieval information collected within a predetermined time from the current time point. Communication unit for transmitting and receiving data through a communication network;
An input unit configured to generate a user input signal according to a user's manipulation;
A camera unit for capturing a real-time broadcast image reproduced through an image device; And
A controller configured to analyze the real-time broadcast image captured by the camera unit to extract image query information according to a user input signal, and to request broadcast program information by transmitting the image query information to a broadcast information providing device through the communication unit; And
And a display unit for displaying an operation result of the terminal under the control of the controller. The method of claim 12, wherein the control unit
And responding to the request, receiving the broadcast program information transmitted from the broadcast information providing apparatus and outputting the broadcast program information to the display unit. The method of claim 12, wherein the control unit
And comparing the image query information with a plurality of image search information, extracting information of the broadcast program corresponding to the image search information with high similarity, and outputting the extracted information to the display unit. The method of claim 12, wherein the image query information is
And at least one of video feature information on the broadcast video, visual information on which the broadcast video from which the video feature information is extracted, and local information on which the broadcast video is collected. The method of claim 12, wherein the broadcast program information
And at least one of a channel name of a broadcast program, a program name, a time difference between an actual broadcast time and a video collection time of the terminal, a sequence number of the corresponding broadcast video, and a broadcast time of the broadcast video. The method according to claim 13 or 14,
And a storage unit which stores at least one of a broadcast image photographed by the camera unit, image query information extracted by the controller unit, a plurality of image search information units, and broadcast program information unit. The method of claim 12,
Further comprising a GPS unit for receiving a GPS satellite signal to calculate the position of the terminal,
The controller is characterized in that for extracting the area information of the collection of the real-time broadcast image through the GPS unit. The method of claim 12, wherein the control unit
And extracting the image query information by combining analysis results of two or more broadcast images continuously photographed at a predetermined period or a predetermined number of times for a preset time. The method of claim 12,
Further comprising a motion detection unit for detecting the movement of the terminal,
The controller checks the movement of the terminal while the broadcast image is being photographed through the motion detection unit, and when a motion outside the reference range occurs, the broadcast image photographed at the corresponding time point is excluded from the extraction target of the image query information. Terminal. The method of claim 12,
The controller may be configured to exclude the captured broadcast image from an object to be extracted from the image query information when a difference in a reference range or more is generated by comparing the broadcast image photographed by the camera unit with a previously captured broadcast image. . Collecting basic information of a plurality of broadcast programs broadcast in real time;
Extracting image search information by collecting broadcast images of a plurality of broadcast programs currently being broadcast in real time; And
And matching and storing the plurality of pieces of basic information and image search information. The method of claim 22,
Receiving a broadcast program information request message including video query information extracted from the broadcast video being viewed;
Searching for the requested broadcast program information by comparing the image query information with the image search information; And
And transmitting the found broadcast program information. The method of claim 22, wherein the collecting of basic information about the broadcast program is performed.
And receiving program guide information transmitted together with the broadcast signal for each communication medium or channel. The method of claim 22, wherein the collecting of basic information about the broadcast program is performed.
And receiving broadcast programming information for each communication medium or channel. The method of claim 22, wherein extracting the image search information comprises:
Receiving a broadcast signal broadcast in real time for each communication medium or channel;
Capturing a broadcast image at a predetermined period from the received broadcast signal;
Obtaining visual information capturing the broadcast image;
Extracting image feature information from the captured broadcast image;
And matching the image feature information with capture time information and storing the image feature as image retrieval information. The method of claim 22, wherein extracting the image search information comprises:
Before extracting the video characteristic information, converting the captured broadcast video into one or more resolutions applied to a terminal;
And extracting image feature information in an environment similar to a terminal by using the broadcast image having the converted resolution. The method of claim 22,
Matching and storing the plurality of basic information and image search information may include:
Method for providing real-time broadcast information, characterized in that for storing the image search information within a predetermined time or a predetermined number based on the current time. The method of claim 23, wherein the searching for the program information comprises:
Comparing the image feature information included in the image query information with the image feature information included in the plurality of image search information;
And comparing the image search information with the highest similarity and basic information of the broadcast program matched with the image search information. The method of claim 23, wherein the searching for the program information comprises:
Checking the collection time of the image query information;
The method for providing real time broadcasting information comprising comparing the image search information and the image query information with reference to the identified collection time. The method of claim 23, wherein the searching for the program information comprises:
And checking channel information of the region where the image query information is collected, and extracting channel information of the region where the image query information is collected. Collecting a broadcast image of a real time broadcast program reproduced through a video device;
Analyzing the collected broadcast video to extract video query information for identifying the broadcast;
Requesting information about a broadcast program corresponding to the image query information by transmitting the extracted image query information; And
And in response to the request, receiving searched broadcast program information by using the video query information. 33. The method of claim 32, wherein collecting the broadcast image of the real-time broadcast program
And continuously photographing the broadcast image of the real-time broadcast program at a predetermined period or a predetermined number of times for a predetermined time. 33. The method of claim 32, wherein collecting the broadcast image of the real-time broadcast program
Sensing the movement of the terminal while capturing the broadcast image; And
If the detected movement of the terminal is out of the reference range, the method comprising the step of removing the broadcast image photographed at the time point, and excludes the image query information from the target to extract the real-time broadcast information providing method. 33. The method of claim 32, wherein collecting the broadcast image of the real-time broadcast program
Comparing the captured broadcast image with the captured broadcast image; And
And removing the captured broadcast image and excluding the captured broadcast image from an object for extracting image query information when a difference between the previously captured broadcast image and the reference range is exceeded. 33. The method of claim 32, wherein extracting the image query information comprises:
Extracting image feature information by analyzing the collected broadcast image;
Extracting collection time information of the collected broadcast image; And
At least one of acquiring local information from which the broadcast image is collected;
And generating one or more of the video feature information, collection time information, region information, and sequence number to generate video query information.

Images