CN110073667A

CN110073667A - Display device, content identification method and non-transitory computer readable recording medium

Info

Publication number: CN110073667A
Application number: CN201780076820.0A
Authority: CN
Inventors: 吕海东
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2016-12-21
Filing date: 2017-12-20
Publication date: 2019-07-30
Anticipated expiration: 2037-12-20
Also published as: KR20180072522A; CN110073667B; KR102468258B1

Abstract

The present invention provides a kind of display device, its content identification method and non-transitory computer readable recording mediums.The display device includes: display, is configured as storing the memory of the information about the fingerprint generated by the characteristic for extracting content, and the content corresponding to the fingerprint；It is configured as the communication equipment with server communication；And at least one processor, it is configured as extracting the characteristic of the content screen currently reproduced on the display and generates fingerprint, with search in memory with the fingerprint of fingerprint matching generated in the presence/absence of, and the result based on search, determine whether to send server for the inquiry including fingerprint generated, to request the information about the content currently reproduced.

Description

Display device, content identification method and non-transitory computer readable recording medium

Technical field

This disclosure relates to display device, content identification method, and the computer-readable record of at least one non-transitory Medium.More specifically, this disclosure relates to the display device of the content of user's viewing, its content identification method can be efficiently identified And non-transitory computer readable recording medium.

In addition, being related to for example with the consistent device and method of various embodiments through the engineering using such as deep learning Practise artificial intelligence (AI) system of the human brain function of algorithm and its application technology simulation identification, determining etc..

Background technique

In recent years, the display device of such as TV (TV) uses set-top box more and more rather than directly receives broadcast letter Number.In this case, display device can not know what content user is currently viewed.

If display device knows what content user is currently viewing, such as targeted ads, content can be provided Recommend, the intelligent Service of relevant information service.For this purpose, developing as identifying the content currently shown at display device Technology automatation content identification (ACR).

In the method for the relevant technologies, display device periodically captures the screen being currently viewed, and extracts for knowing The characteristic of other screen, and current screen is identified periodically by inquiry request server.

However, display device is had no option, can only inquiry continually be sent to server quickly to detect and watch The variation of content, therefore, ACR need many resources and great amount of cost.

With the development of computer technology, data traffic is increased in the form of exponential function, and artificial intelligence (AI), which becomes, draws The important trend of the following innovation of neck.Due to the mode of AI simulation mankind's thinking, it can ad infinitum be applied to all industries.

AI system, which refers to, is embodied as the computer system of human intelligence for high intelligence, and is that one kind learns machine itself With determine and become smarter system, and be different from existing rule-based intelligence system.AI system can be when in use Improve discrimination and can accurately understand the taste of user, thus existing rule-based intelligence system more and more by Replaced AI system based on deep learning.

AI technology includes machine learning (for example, deep learning) and the Element Technology using machine learning.

Machine learning is that itself carries out the algorithmic technique classified // learnt to the feature of input data, and Element Technology is For simulating the skill of the human brain function of identification, determining etc. by the machine learning algorithm using deep learning Art, and may include the technical fields such as language understanding, visual analysis, reasoning/prediction, the representation of knowledge, operation control.

It is as follows using the various fields of AI technology.Language understanding is that human language/character and application/handles it for identification Technology, and may include natural language processing, machine translation, conversational system, problem and answer, speech recognition/synthesis. Visual analysis be for by with the mankind using eyes it is identical in a manner of identify the technology of things, and may include Object identifying, To image tracing, picture search, person recognition, scene understanding, space understands and image enhancement.Deduction/prediction is for by true The technology for determining information logically to infer and predict, and may include the deduction of knowledge based/probability, Optimization Prediction, be based on Plan, recommendation of preference etc..The representation of knowledge be for being the technology of knowledge data by human experience's information automation, and can be with (data generation/classification), information management (data utilization) etc. are established including knowledge.Operation control is for controlling the automatic of vehicle The technology with the movement of robot is driven, and may include motion control (navigation, drives collision), manipulation and control (behavior control System) etc..

Information above is used as background information to present only to help to understand the disclosure.Any determination is not made, and is not had It asserts about any of the above prior arts whether being suitable for about the disclosure.

Summary of the invention

All aspects of this disclosure aim to solve the problem that at least the above problems and/or disadvantage, and provide at least the following advantages.Therefore, An aspect of this disclosure is to provide a kind of information for being able to use content and carrys out the display device of Suitable content identification period, in it Hold recognition methods and non-transitory computer readable recording medium.

According to the one side of the disclosure, a kind of display device is provided.The display device includes display；Memory, quilt It is configured to store the information about by the characteristic fingerprint generated and content corresponding with the fingerprint that extract content；It is logical Believe equipment, is configured as and server communication；And at least one processor, it is configured as extraction and currently reproduces over the display Content screen characteristic and generate fingerprint, to search in memory and the presence of the fingerprint of fingerprint matching generated/no In the presence of, and the result based on search, it is determined whether server is sent by the inquiry including fingerprint generated to request to close In the information of the content currently reproduced.

At least one processor can be configured as, in response to searching in memory and fingerprint matching generated Fingerprint, the content currently reproduced is identified based on the information about the content for corresponding to searched for fingerprint, and this is at least One processor can be configured as in response to not searching the fingerprint with fingerprint matching generated in memory, and control is logical Letter equipment sends server for the inquiry including fingerprint to request the information about the content currently reproduced.

At least one processor can be configured as in response to not searching in memory and fingerprint generated The fingerprint matched, control communication equipment from the server in response to inquiry receive about the content currently reproduced information and it is current again The fingerprint of existing content.

At least one processor, which can be configured as based on the information about the content currently reproduced, determines content Type, and change the content recognition period according to the type of identified content.

In addition, it is ad content and in each first period that at least one processor, which can be configured as in response to content, Middle identification content, and be that broadcast program contents each of segment length when than first identifies content in second period in response to content.

At least one processor, which can be configured as based on the information about the content currently reproduced, determines content Type, and changed according to the type of identified content by the quantity of the fingerprint of the received content currently reproduced.

At least one processor can be configured as based on about the content currently reproduced information and viewing history come The probability that the content reproduced is changed is calculated, and changes the content recognition period according to probability calculated.

At least one processor can be configured as based on viewing history the content of predicting down secondary reproduction, and from clothes Business device requests the information about the content predicted.

At least one processor can be configured as from server and receive additional letter related with the content currently reproduced Breath, and control display to show the received additional information with the content currently reproduced.

According to another aspect of the present disclosure, a kind of method of content of display device for identification is provided.This method packet It includes the characteristic for extracting the content screen currently reproduced and generates fingerprint, search for whether the fingerprint of fingerprint matching generated stores In a display device, and the result based on search, it is determined whether sending includes that fingerprint generated inquires external service Device is to request the information about the content currently reproduced.

Determine whether that sending a query to external server may include in response to searching matching institute in a display device The fingerprint of the fingerprint of generation, the information based on the content for corresponding to the fingerprint searched identify the content currently reproduced, and, it rings Ying Yuyu is not searched the fingerprint with fingerprint matching generated in a display device, sends clothes for the inquiry including fingerprint Device be engaged in request the information about the content currently reproduced.

In addition, this method may be responsive to and not search in a display device and fingerprint matching generated Fingerprint receives the information and the current fingerprint of content that reproduces about the content currently reproduced from the server in response to inquiry.

This method can also include that the type of content is determined based on the information about the content currently reproduced, and according to institute The type of determining content changes the content recognition period.

It is ad content that the change content recognition period, which may include in response to content, in each first period in identification Hold, and in response to content be broadcast program contents, each of segment length identifies content in the second period when than first.

This method can also include that the type of content is determined based on the information about the content currently reproduced, and according to institute The type of determining content changes the quantity of the fingerprint of the received content currently reproduced.

This method can also include calculating reproduction content with viewing history based on the information about the content currently reproduced The probability being changed, and change the content recognition period according to calculated probability.

This method can also include the content of secondary reproduction is predicted down based on viewing history, and from server request about The information for the content predicted.

This method can also include receiving additional information related with the content currently reproduced from server, and show and connect The additional information of receipts and the content currently reproduced.

According to another aspect of the present disclosure, at least one non-transitory computer readable recording medium is provided.This is at least A kind of non-transitory computer readable recording medium includes the program for running the method for the content of display device for identification, This method includes extracting the characteristic of the screen of the content currently reproduced and generating fingerprint, the finger of search and fingerprint matching generated Whether line is stored in display device, and being determined whether based on the result of search will be including the inquiry of fingerprint generated External server is sent to request the information about the content currently reproduced.

According to above-mentioned various embodiments, display device dynamically adjusts server automatation content identification (ACR) and local ACR Between ratio and run the period, to reduce to the load of server and improve the accuracy of identification content.

By disclosing the detailed description of the various embodiments of the disclosure below in conjunction with attached drawing, other aspects of the disclosure, Advantage and notable feature will become obvious those skilled in the art.

Detailed description of the invention

Pass through following description with reference to the accompanying drawings, above and other aspects, the feature and advantage of some embodiments of the disclosure It will be apparent from, in which:

Fig. 1 shows display system according to an embodiment of the present disclosure；

Fig. 2 a and Fig. 2 b are the schematic block diagrams for showing the configuration of display device according to an embodiment of the present disclosure；

Fig. 3 is the block diagram of processor according to an embodiment of the present disclosure；

Fig. 4 a is the block diagram of data unit according to an embodiment of the present disclosure；

Fig. 4 b is the block diagram of data identification unit according to an embodiment of the present disclosure；

Fig. 5 is the block diagram for showing the configuration of display device according to an embodiment of the present disclosure；

Fig. 6 is the view for showing mixing automatation content identification (ACR) according to an embodiment of the present disclosure；

Fig. 7 is to show the view according to an embodiment of the present disclosure with varigrained finger print information；

Fig. 8 is the view for showing viewing historical information according to an embodiment of the present disclosure；

Fig. 9 is the view for showing the display of the additional information according to an embodiment of the present disclosure about content；

Figure 10, Figure 11, Figure 12 a, Figure 12 b, Figure 13 a, Figure 13 b, Figure 14 a, Figure 14 b, Figure 15 a and Figure 15 b are to show basis The flow chart of the content identification method of the display device of the various embodiments of the disclosure.

Figure 16 is to show according to an embodiment of the present disclosure learnt by display device interlocked with one another and server and identification Data view；

Figure 17 is the flow chart for showing the content identification method of display system according to an embodiment of the present disclosure；

Figure 18 is the flow chart for showing the content identification method of display system according to an embodiment of the present disclosure；

Figure 19 is to show wherein display device according to an embodiment of the present disclosure to change according to and interlocking with server The probability of content is come view the case where changing the content recognition period；

Figure 20 be show under display device according to an embodiment of the present disclosure prediction the content of secondary reproduction and by with Server interlocks and receives in advance the view of the method for the information about predictive content；And

Figure 21 be show under display device according to an embodiment of the present disclosure prediction the content of secondary reproduction and by with Multiple servers interlock and receive in advance the view of the method for the information about predictive content.

Throughout the drawings, identical appended drawing reference will be understood to refer to identical component, component and structure.

Specific embodiment

The disclosure being described below to help comprehensive understanding to be defined by the claims and their equivalents for referring to attached drawing is provided Various embodiments.It includes various details to help to understand, but these are considered only as illustratively.Therefore, this field Skilled artisan will realize that in the case where not departing from the scope of the present disclosure and spirit, it can be to described herein various Embodiment makes various changes and modifications.In addition, for clarity and conciseness, it is convenient to omit the description to known function and structure.

The term and word used in the following description and claims is not limited to dictionary meanings, but is only made by inventor With enabling to clear and consistently understand the disclosure.Therefore, it will be apparent to one skilled in the art that mentioning Be for illustration purposes only for being described below for various embodiments of the disclosure, rather than in order to limit by appended claims and its The purpose for the disclosure that equivalent limits.

It should be understood that unless the context clearly determines otherwise, otherwise singular " one ", "one" and "the" refer to including plural number Show object.Thus, for example, including the reference to surface as one or more to the reference of " assembly surface ".

Term " substantially " refers to that the feature, parameter or value do not need accurately to realize, but including such as tolerance, survey The deviation or variation for measuring error, measurement accuracy limitation and other factors well known by persons skilled in the art, can be to be not excluded for spy The amount for levying the effect being intended to provide occurs.

The term of " first " and " second " that such as uses in various embodiments can be used for illustrating various elements, but unlimited Make corresponding element.These terms can be used for the purpose for distinguishing an element and another element.For example, not departing from this In the case where disclosed interest field, first element can be named as second element, and similarly, and second element can be by It is named as first element.Term "and/or" includes combination or any one of the multiple relevant items of multiple relevant items.

The term used in the various embodiments of the disclosure is used only for the purpose of describing specific embodiments, and is not intended to Limitation and/or the restriction disclosure.As used herein, unless the context is clearly stated, otherwise singular be also intended to including Plural form.Term " includes " or " having " indicate feature, number, operation, element and component or its group described in specification The presence of conjunction, and do not preclude the presence or addition of other one or more features, number, operation, element or component or its group It closes.

In addition, " module " used in embodiment or " unit " executes one or more functions or operation, and can lead to It crosses using the combination of hardware or software or hardware and software and realizes.In addition, in addition to " module " that needs to be realized by specific hardware Or except " unit ", multiple " modules " or multiple " units " can be integrated into one or more modules, and can be by reality It is now one or more processors.

Hereinafter, the disclosure will be described below with reference to attached drawing.

Fig. 1 shows display system according to an embodiment of the present disclosure.

With reference to Fig. 1, display system 1000 includes display device 100 and server 200.

Display device 100 can be smart television (TV), but this is only example, and display device 100 can be by using each The device of seed type realizes, such as projection TV, monitor, self-service terminal, laptop, computer (PC), flat Plate computer, smart phone, personal digital assistant (PDA), digital photo frame, tabletop display etc..

Display device 100 can be from the content screen extraction property currently reproduced, and fingerprint can be generated.In addition, aobvious Showing device 100 can execute sheet by searching for fingerprint generated in the fingerprint database being stored in display device 100 Ground automatation content identification (ACR), and identify the content that currently reproduces, and can be by that will include inquiry that the institute generates fingerprint It is sent to server 200 and identifies that the content carrys out execute server ACR.More specifically, display device 100 can be by using institute The content information of identification checks the Suitable contents such as the history identification period, to fit from quantity of the received fingerprint of server 200 etc. The ratio of operation between locality adjustment local ACR and server ACR.

Server 200 can by using can send including for distinguishing specific image and other images identification (or Identification (ID)) device of information of information realizes.For example, server 200 can send fingerprint to display device 100.Refer to Line is a kind of identification information that can distinguish image Yu other images.

Specifically, fingerprint is from including the performance data extracted in video and audio signal in frame.Be based on text Metadata it is different, fingerprint can reflect the specific characteristic of signal.For example, fingerprint can when audio signal is included in frame To be the data for representing the characteristic of audio signal, frequency, amplitude etc..When video (or static image) signal is included in frame When middle, fingerprint can be the data for representing feature, motion vector, color etc..

It can be taken the fingerprint by various algorithms.For example, display device 100 or server 200 can according to rule when Between interval to divide audio signal, and the size of the signal for the frequency for including in each time interval can be calculated.In addition, aobvious Showing device 100 or server 200 can calculate frequency slope by the size difference between the signal in acquisition side frequency section. It can be by being arranged 1 when frequency slope calculated is positive value and being arranged for 0 next life when the frequency slope of calculating is negative value At the fingerprint in audio signal.

Fingerprint can be stored on specific image by server 200.Server 200 can be deposited on chartered image Store up one or more fingerprints, and when storing two or more fingerprints about specific image, can using fingerprint as The fingerprint list of specific image manages.

Term used in the disclosure " fingerprint " can refer to a fingerprint on specific image, or according to circumstances, can be with Refer to the fingerprint list formed by multiple fingerprints on specific image.

Term used in the disclosure " frame " refers to the volume of data about audio or the information of image.Frame can be During the scheduled time about audio or the data of image.In the case where digital image content, frame can be by 30-60 per second Image data is formed, and these 30-60 image data can be referred to as frame.For example, when being used together present frame and image When the next frame of content, which can refer to be included in content in and each picture screen for continuously displaying.

Although Fig. 1, which depicts display system 1000, includes a display device 100 and a server 200, multiple Display device 100 can be connect with a server 200 or multiple servers 200 can connect with a display device 100 It connects.Other combinations are also possible.

Fig. 2 a is the block diagram for showing the configuration of display device according to an embodiment of the present disclosure.

With reference to Fig. 2 a, display device 100 may include display 110, memory 120, communication unit 130 and processor 140.

Display 110 can show the various picture materials provided by display device 100, information, user interface (UI) etc.. For example, display device 100 can show the picture material provided by set-top box (not shown), broadcast program image, user interface Window.

Memory 120 can store various modules, software and the data for driving display device 100.For example, memory 120 can store multiple fingerprints, the information about the content for corresponding to fingerprint, viewing historical information etc..It is stored in memory 120 In fingerprint can be the fingerprint generated by display device 100 itself, or can be from the received fingerprint of server 200.It deposits The index information for being used for local ACR can be attached to fingerprint and store fingerprint by reservoir 120.

In addition, fingerprint can be with corresponding content when the reproduction of display device 100 stores the content in memory 120 It matches and can be stored in memory 120.For example, fingerprint can be added in each frame of memory 120 and It can be stored in the form of being combined with the new file of content and fingerprint.In another example, in fingerprint may also include and be mapped to The information of the correspondence frame of appearance.

The external device communication of wire/wireless communication method Yu such as server 200 can be used in communication unit 130.Example Such as, communication unit 130 can exchange data with server 200, such as fingerprint, content information, viewing historical information, with content phase The additional information and control signal of pass (such as identification period changes control signal).

Processor 140 can identify the content being currently reproduced by, and can be controlled based on recognition result with appropriate Precision executes ACR.For example, processor 140 can identify the period based on content information come Suitable content, and can determine will be from The content that server 200 receives and stores in advance and the fingerprint quantity about content.

Processor 140 can extract the characteristic of the content screen currently reproduced, and fingerprint can be generated.In addition, processing Device 140 can be searched for from the multiple fingerprints being stored in memory 120 with the presence or absence of the fingerprint with fingerprint matching generated. It is inquired in addition, processor 140 can be sent according to search result to server 200, and may determine whether to attempt to execute clothes Be engaged in device ACR.For example, processor 140 can first attempt to execute local ACR, to reduce the load on server 200.

In response to searching the fingerprint with fingerprint matching generated, processor 140 can be searched based on corresponding to The information of the content of fingerprint identifies the content currently reproduced.For example, may include working as about the information for corresponding to fingerprint content Position, recovery time etc. of the previous frame in total frame, are the information about present frame.In addition, about the content for corresponding to fingerprint Information may include content name, content ID, content supplier, content series information, type, about content whether be real-time The information of broadcast and about content whether be pay content at least one of information.

On the other hand, in response to not searching the fingerprint with fingerprint matching generated, processor 140 can control logical Letter unit 130 will be used to that the inquiry about the information of the content currently reproduced to be requested to be sent to server 200.For example, inquiry can To include the fingerprint generated, watch history, about information of display device 100 etc..

It is received from the server 200 in response to inquiry about current in addition, processor 140 can control communication unit 130 The fingerprint of the information of the content of reproduction and the content currently reproduced.Here, the fingerprint of the content currently reproduced can be about It is located at the fingerprint of the frame after present frame in entire content.Since processor 140 is known currently based on the fingerprint for including in inquiry Time indicated by position of the frame in entire content, thus processor 140 can from server receive it is contemplated that present frame it The fingerprint on frame reproduced afterwards.

As described above, processor 140 can identify content by being appropriately combined local ACR and server ACR.Pass through It does so, processor 140 can identify the content currently reproduced on display 110, while minimize negative on server 200 It carries.

In order to be appropriately combined local ACR and server ACR, processor 140 can result and sight based on content recognition At least one of history is seen to predefine the received fingerprint of slave server 200 for local ACR, and can be determined that The no change content recognition period.For example, processor 140, which can determine, considers which content to receive fingerprint, and will once connect How many fingerprint received.

In accordance with an embodiment of the present disclosure, in processor 140 can be determined based on the information about the content currently reproduced The type of appearance.In addition, processor 140 can change the content recognition period according to the type of content.It can be according to such as content Details, type, about content whether be that the standard of information, importance of real-time broadcast etc. classifies to the type of content.

For example, being identified as ad content in response to the content currently reproduced, processor 140 can be by the content recognition period It is adjusted to short (for example, being adjusted to identify the content screen currently shown in every frame).It is identified in response to the content currently reproduced For movie contents or broadcast program contents, the content recognition period can be adjusted to long (for example, being adjusted to every 30 by processor 140 The content screen that second identification is currently shown).

The each type of identification period can be predetermined period.It can be according to above-mentioned various standards and viewing history come individual character Change and be arranged each period.

In the case where ad content, processor 140 needs continually to identify content, because advertisement is usually in a short time Change into another advertisement.On the contrary, processor 140 does not need continually to identify content in the case where movie contents, because only really It is fixed whether continuously to watch film.

In the above examples, classified according to type of the type of content to content.Such as in the above example, wide In the case where accusing content, content can be identified in each frame, but can be according to other of such as importance or viewing history Standard infrequently identifies content.

The quantity of the frame received and stored in advance from server 200 can change according to the content type of identification.Due to In the presence of fingerprint corresponding with each frame, therefore the quantity of the fingerprint received and stored in advance can also change.For example, in video point In the case where broadcasting (VOD) or digital video recorder (DVR), server 200 can have all image informations, but be broadcast live In the case where, server 200 can be in former seconds reception image informations of display device 100.Let us act one is per second aobvious Show the example of the 60Hz image of 60 frames.In the case where one hour VOD, server 200, which can possess, corresponds approximately to 200, The fingerprint of 000 frame, but in the case of a live broadcast, server 200 can only possess the fingerprint for corresponding approximately to hundreds of frames.

Therefore, processor 140 can be according to the quantity of the fingerprint to be requested of content determination of identification.In addition, processor 140 The content recognition period can be changed based on the quantity of primary received fingerprint.

In accordance with an embodiment of the present disclosure, processor 140 can information based on the content about identification and viewing history come Change the content recognition period.Viewing history may include user watched content, viewing time, viewing when run Additional application.

For example, can determine whether by comparing the content that currently reproduce and viewing history will continuously again for processor 140 The content that now currently reproduces will reproduce another content.In addition, processor 140 can be requested from server 200 about correspondence The information of the fingerprint of the content of secondary reproduction under expection.

In accordance with an embodiment of the present disclosure, except when the fingerprint of the content of preceding reproduction and the content reproduced corresponding to expected next time Fingerprint except, processor 140 can also receive additional information relevant to content.For example, additional information may include content Title, content reproduction time, content supplier, occur PPL product information in the content, it is related with PPL product information extensively Announcement and the additional application etc. run.

In addition, processor 140 can control display 110 to show received additional information and the current content that reproduces.

In the examples described above, display device 100 requests the information of the content of such as fingerprint from server 200.However, service Device 200 can send service for information needed for display device 100 (such as fingerprint) in the case where being not received by request Device 200.

According to various embodiments of the present disclosure, data identification model can be used to estimate the class of content in display device 100 Type.Data identification model can be, for example, generated using statistical machine learning using the information of content and/or from content Fingerprint estimates one group of algorithm of the type of content.

In addition, data identification model can be used to calculate the probability that content is changed in display device 100.Data identify mould Type can be, for example, for the information (for example, content reproduction time, content reproduce channel, type of content etc.) using content Estimation reproduces one group of algorithm of the probability that content is changed.

Can by using for running this group of algorithm software or engine realize data identification model.By using soft The data identification model of part or engine implementation can by display device 100 processor or server (for example, the clothes in Fig. 1 Business device 200) processor run.

In accordance with an embodiment of the present disclosure, server 200 may include the configuration of common server equipment.For example, server 200 may include memory 210, communication unit 220, broadcast signal receiver 230 and processor 240.

Server 200 can capture the video/audio information of multiple contents.For example, server 200 can be collected based on frame Image.For example, various division of teaching contents can be the data of frame unit by server 200 in advance, and data can be collected.Separately Outside, server 200 can generate fingerprint by analyzing collected frame.In another example, server 200 can be from broadcast It stands and receives broadcast singal, and video/audio information can be captured from received signal.Server 200 can be filled in display Broadcast singal is received before setting 100 receptions.

For example, can be the information for distinguishing screen and audio in specific time by the fingerprint that server 200 generates.Or Person may include the information about scene change mode by the fingerprint that server 200 generates, and it is continuous to can be instruction Watch the information of what content.Server 200 can establish database, wherein the fingerprint generated and the content corresponding to fingerprint Information is indexed to search for.For example, the information about the content for corresponding to fingerprint may include present frame in entire content In position, recovery time etc..In addition, the information about the content for corresponding to fingerprint may include content name, it is content ID, interior Hold provider, content series information, type, whether be the information of real-time broadcast, whether be paid for about content about content in At least one of information of appearance.

In response to receiving inquiry from display device 100, server 200 can extract at least one fingerprint from inquiry. In addition, server 200 can receive information in the display device 100 for having had sent inquiry.

Server 200 can match extracted fingerprint with the information of storage in the database, and can be true Determine what content is display device 100 be currently viewing.Server 200 can will send the response of identified content information To display device 100.

It is managed in addition, the received information in display device 100 can be used in server 200 with the content information determined The viewing history of each in display device 100.By doing so, server 200 can be provided in display device 100 The personalized service of each.

The information of the content currently shown in display device 100 can be used in server 200 and viewing historical information is come The secondary content shown in display device 100 under prediction.In addition, the finger that server 200 can will be extracted from predictive content Line is sent to display device 100.It is located at after present frame in entire content for example, extracted fingerprint can be to correspond to The fingerprint of frame.In another example, extracted fingerprint can be another broadcasting channel based on viewing historical information prediction Fingerprint in content.

In addition, server 200 can be interior by analyzing content images or be determined using electronic program guides (EPG) information Hold the identification period.According to the identified content recognition period, server 200, which can be determined, executes local at display device 100 The quantity of fingerprint needed for ACR.Display device 100 can by analyzed in each content recognition period the frame currently shown come Generate fingerprint.In addition, display device 100 can be generated searching for from the fingerprint database that server 200 receives and stores Fingerprint.Therefore, server 200 can only send the fingerprint for corresponding to for display device 100 and identifying the frame of content.Due to The fingerprint of necessary amount is only had sent, therefore even if it is negative that server 200 also can be minimized communication in execute server ACR It carries.

Fig. 2 b shows the configuration of display device according to an embodiment of the present disclosure.

With reference to Fig. 2 b, display device 100 may include first processor 140-1, second processor 140-2, display 110, Memory 120 and communication unit 130.However, be not all elements shown in figure being all necessary component.

First processor 140-1 can control the operation for being mounted on the application of at least one of display device 100.For example, First processor 140-1 can generate fingerprint by the image of capture display on display 110, and can execute ACR. First processor 140-1 can be to be integrated with central processing unit (CPU), graphics processing unit (GPU), communication chip and sensing The form of the system on chip (SoC) of the function of device is realized.Alternatively, first processor 140-1 can be application processor (AP).

Data identification model can be used to estimate the type of content in second processor 140-2.Data identification model can be with It is, for example, for being estimated using the result of statistical machine learning using the information of content and/or the fingerprint generated from content One group of algorithm of the type of content.

In addition, data identification model can be used to calculate the probability that content is changed in second processor 140-2.Data are known Other model can be, for example, for the information using content (for example, content reproduction time, content reproduce the class of channel, content Type etc.) and viewing history estimate to reproduce one group of algorithm of probability that content is changed.

In addition, second processor 140-2 data identification model can be used come under estimating in after reproducing the contents it is secondary again Existing content.Data identification model can be, for example, for the information using content (for example, content reproduction time, content are again Existing channel, type of content etc.) and viewing history estimate one group of algorithm of probability that reproduction content is changed.

Second processor 140-2 can be to execute in type and the estimation of estimating content for using data identification model Hold the form of the proprietary hardware chip of the AI of the function of the probability changed to manufacture.

In accordance with an embodiment of the present disclosure, first processor 140-1 and second processor 140-2 can be interlocked with one another to execute A series of processing as processor 140, to generate fingerprint from content using ACR above with reference to described in Fig. 2 a and identify Content.

Display 110, memory 120 and communication unit 130 correspond respectively to display 110, memory 120 in Fig. 2 a With communication unit 130, therefore the explanation of its redundancy is omitted.

Fig. 3 is the block diagram of processor according to an embodiment of the present disclosure.

With reference to Fig. 3, processor 140 according to the embodiment may include data unit 141 and data identification unit 142.

Data unit 141 can learn so that data identification model has for analyzing predetermined video/audio data Characteristic standard.Processor 140 can be by the characteristic of each of the frame of analysis capture (for example, the frequency of audio data Variation, every frame video data color variation or motion vector variation) Lai Shengcheng fingerprint.

Data unit 141 can determination what learning data will determine the spy of capture content screen (or frame) using Property.In addition, identified learning data can be used to learn the characteristic of the content for extracting capture in data unit 141 Standard.

According to various embodiments of the present disclosure, data unit 141 can learn so that data identification model has use In estimated based on learning data related with the type of information and video/audio data about predetermined video/audio video/ The standard of the type of audio data.

Information about video/audio data may include, for example, position of the present frame in entire video/audio and The position of recovery time, they are the information about present frame.In addition, the information about video/audio may include video/sound Frequency title, video/audio ID, video/audio provider, video/audio series information, type, about video/audio whether be Whether the information of real-time broadcast is paid at least one of information of content about video/audio.

The type of video data may include such as drama, advertisement, film, news.The type of audio data can wrap It includes such as music, news, advertisement.However, the type of audio/video data is without being limited thereto.

According to various embodiments of the present disclosure, data unit 141 can learn so that data identification model has use In viewing history/sound based on type and video with information, video/audio data about predetermined video/audio data Frequency is being reproduced according to (for example, having changed as the history of another video/audio data to be checked) related learning data, estimation Period video/audio data is changed to the standard of the probability of another video/audio data, or for estimating reproducing completion The standard of the video/audio data of secondary reproduction under afterwards.

The data identification model that acquistion can be used in data identification unit 142 is based on predetermined identification data identification situation.Number Predetermined identification data can be obtained according to the preassigned obtained according to study according to recognition unit 142, and institute can be used The identification data of acquisition are come as input value using data identification model.

For example, data identification unit 142, which can be extracted to be included in, identifies that data are (all using the feature extraction model of acquistion Such as the content of capture) in each frame on characteristic, and fingerprint can be generated.In addition, data identification unit 142 can be used Carry out more new data identification model as the output data for being applied to the result of data identification model as input value again and obtaining.

According to various embodiments of the present disclosure, data identification unit 142 can by by with about predetermined video/audio Information-related identification data are applied to data identification model as input value to obtain the type of determining video/audio data As a result.

According to various embodiments of the present disclosure, by by the information-related identification data with predetermined video/audio data It is applied to data identification model as input value with the type of video/audio data, data identification unit 142 can be regarded reproducing Frequently/audio data while obtain estimation video/audio data be changed to another video/audio data probability as a result, Or estimate the result for the video/audio data that next time reproduces after the completion of reproduction.

At least one of data unit 141 and data identification unit 142 can be with a hardware chips or multiple The form of hardware chip manufactures, and can be installed in display device 100.For example, at least one data unit 141 and data identification unit 142 can be manufactured in the form of the proprietary hardware chip for AI, or can be manufactured to existing There is a part of general processor (for example, CPU or AP) or figure application specific processor (for example, GPU, ISP), and can be pacified In above-mentioned various display devices 100.

In this case, it can be the application specific processor for being exclusively used in calculating probability for the proprietary hardware chip of AI, and And it can have parallel processing performance more higher than existing general processor, therefore the operation that can quickly handle the field AI is appointed Business, such as machine learning.When by using software module (or program module including instruction) Lai Shixian data unit 141 When with data identification unit 142, software module can be stored in non-transitory computer readable recording medium.In this feelings Under condition, software module can be provided by operating system (OS), or can be provided by scheduled application.Alternatively, the one of software module Part can be provided by OS, and another part can be provided by scheduled application.

Although Fig. 3 depicts data unit 141 and data identification unit 142 is all installed in display device 100, But they can also be mounted in a separate device.For example, one in data unit 141 and data identification unit 142 It is a to be included in display device 100, and another can be included in server 200.In addition, data unit 141 and data identification unit 142 can be connected to each other in wired or wireless method, and can will be by data unit 141 The model information established is supplied to data identification unit 142, and the data that can will enter into data identification unit 142 are made Data unit 141 is supplied to for accretion learning data.

Fig. 4 A is the block diagram of data unit according to an embodiment of the present disclosure.

With reference to Fig. 4 A, data unit 141 according to the embodiment may include data capture unit 141-1 and model learning Unit 141-4.In addition, data unit 141 includes pretreatment unit 141-2, learning data selection with being also an option that property At least one of unit 141-3 and model evaluation unit 141-5.

Learning data needed for data capture unit 141-1 can obtain certain situation.For example, data capture unit 141- 1 can obtain picture frame by capturing the screen reproduced on display 110.In addition, data capture unit 141-1 can be from External equipment, such as set-top box receive image data.Image data can be formed by multiple images frame.In addition, data acquisition list First 141-1 can receive study image data from the network of server 200 or such as internet.

Model learning unit 141-4 can learn so that data identification model has for based on learning data certain situation Standard.In addition, model learning unit 141-4 can learn so that data identification model has for selecting anything will be used to learn Practise the standard that data carry out certain situation.

For example, model learning unit 141-4 can learn the physics for distinguishing image by comparing multiple images frame Characteristic.Model learning unit 141-4 can by extract picture frame in foreground and background between ratio, the size of object, Position and arrangement and characteristic point learn the standard for distinguishing picture frame.

In addition, model learning unit 141-4 can learn the standard of the type of the content including picture frame for identification.Example Such as, model learning unit 141-4 can learn have the frame of text box as one in the upper end of picture frame or lower end for identification The standard of seed type.This is because the image of news content has text box to show in news in the upper end in left side or lower end Hold.

According to various embodiments of the present disclosure, model learning unit 141 can learn so that data identification model has use In estimated based on the type of learning data relevant to the information of predetermined video/audio data and video/audio data video/ The standard of the type of audio data.

The information of video/audio data may include, for example, the position of the present frame in entire video/audio and reproduction Time, they are the information about present frame.In addition, the information about video/audio may include video/audio title, view It frequently/audio ID, video/audio provider, video/audio series information, type, about video/audio whether is real-time broadcast Information, about video/audio whether be paid at least one of information of content.

According to various embodiments of the present disclosure, model learning unit 141 can learn so that data identification model has base In the viewing history (example with the information of predetermined video/audio data, the type of video/audio data and video/audio data Such as, it has been changed to the history of another audio/video data or has been completed select after watching audio/video data The history of another audio/video data) related data, for estimating that video/audio data is changed during reproduction For the standard of the probability of another video/audio data, or video/sound for estimating the secondary reproduction in the case where reproducing after completion The standard of frequency evidence.

Data identification model can be the model having built up.For example, data identification model can be reception basic studies The model they data (for example, sample image) and had built up.

Data unit 141 can also include pretreatment unit 141-2, learning data selecting unit 141-3 and model Assessment unit 141-5, to improve the result of data identification model identification or to save needed for generating data identification model Resource or the time.

Pretreatment unit 141-2 can pre-process learning data obtained, and learning data obtained is used In study with certain situation.Pretreatment unit 141-2 can handle data obtained in a predetermined format, so that model learning list First 141-4 can be used data obtained and be learnt with certain situation.

For example, pretreatment unit 141-2 can be by executing decoding to the image data of input, scaling, noise filtering, dividing Resolution conversion etc. generates the picture frame of same format.In addition, pretreatment unit 141-2 can only be cut including in the more of input The specific region of each of a picture frame.Only in response to the specific region of cutting, display device 100 can consume less Resource distinguishes one in frame and other frames.

In another example, pretreatment unit 141-2 can be extracted including text filed in the picture frame of input.Separately Outside, pretreatment unit 141-2 can be by extracted text filed execution optical character identification (OCR) Lai Shengcheng textual data According to.Pretreated text data as described above can be used for distinguishing picture frame.

Data needed for learning data selecting unit 141-3 can select study from pretreated data.It can incite somebody to action Selected data is supplied to model learning unit 141-4.Learning data selecting unit 141-3 can be according to for the pre- of certain situation The quasi- data needed for selection study from preprocessed data of calibration.In addition, learning data selecting unit 141-3 can be according to passing through Model learning data selection unit 141-3's learns determining preassigned to select data, this will be described below.

For example, learning data selecting unit 141-3 can be from pretreated picture frame in the initial time of study Except the picture frame with high similitude.For example, learning data selecting unit 141-3 can choose the use of the data with low similarity In initial study, allow to the standard for learning to be easy to learn.

In addition, learning data selecting unit 141-3 can choose one usually met by the determining standard of study Pretreatment image frame.By doing so, model learning unit 141-4 can learn different marks from the standard of acquistion It is quasi-.

Assessment data can be input in data identification model by model evaluation unit 141-5, and in response to from discontented The recognition result of the assessment data output of sufficient preassigned, can be such that model learning unit 141-4 learns again.In such case Under, assessment data can be the tentation data for assessing data identification model.

In the configuration operation of initial identification model, assessment data can be the picture frame for indicating different content type.Hereafter, Assessment data can be replaced with one group of picture frame with higher similitude.By doing so, model evaluation unit 141-5 can With the performance of verify data identification model by stages.

For example, in response to inaccurately being identified from the recognition result of the learning data identification model about assessment data As a result the quantity or ratio of assessment data are more than predetermined threshold, and model evaluation unit 141-5, which can be assessed, is unsatisfactory for pre- calibration It is quasi-.For example, preassigned can be defined as 2% ratio.In this case, it is directed in response to learning data identification model 20 or more assessment data output error recognition results in 1000 overall evaluation data, model evaluation unit 141-5 can It is inappropriate to assess learning data identification model.

In response to there are multiple learning data identification models, model evaluation unit 141-5 can assess learning data identification Whether each of model meets preassigned, and the model for meeting preassigned can be determined as to final data identification Model.In this case, in response to meeting multiple models of preassigned, model evaluation unit 141-5 can be from most higher assessment Estimating score starts successively to determine the model of any pre-determined model or predetermined quantity as final data identification model.

Data identification mould can be established based on the computer performance of the application field of identification model, the aim of learning or equipment Type.Data identification model can be based on such as neural network.It is, for example, possible to use such as deep neural networks (DNN), recurrence mind Through network (RNN), forward-backward recutrnce deep neural network (BRDNN) model as data identification model, but data identify mould Type is without being limited thereto.

According to various embodiments of the present disclosure, in response to having been set up multiple data identification models, model learning unit The learning data and basic studies data that 141-4 can be determined and be inputted have the data identification model of high correlation as data Learn identification model.In such a case, it is possible to be classified according to data type to basic studies data, and can basis Data type establishes data identification model.For example, can be classified according to various criteria to basic studies data, example It such as generates the region of learning data, generate the time of learning data, the size of learning data, the type of learning data, study number According to generator, the object type in learning data etc..

In addition, model learning unit 141-4 can be by using for example including error back propagation or gradient descent method Learning algorithm carry out learning data identification model.

The learning data for being used to learn is considered as example, model learning unit 141-4 can be such that data identification model passes through Learn with the standard of determination as the supervised learning of input value.In another example, model learning unit 141-4 can not have Data type needed for learning itself certain situation in the case where having individually supervision, to make data identification model by finding use Learn in the unsupervised learning of the standard of certain situation.In another example, model learning unit 141-4 can be such that data know Other model learns by using about according to the intensified learning that whether correctly feeds back of result of study certain situation.

In addition, model learning unit 141-4 can store learning data identification mould in response to acquistion data identification model Type.In this case, learning data identification model can be stored in depositing for display device 100 by model learning unit 141-4 In reservoir 120.Alternatively, model learning unit 141-4 can be stored in learning data identification model and wired or wireless network In the memory of server 200 that is connected of electronic equipment in.

In this case, wherein storing the memory 120 of the data identification model of acquistion can also store and show The related instruction of at least one other element of device 100 or data.In addition, memory 120 can store software and/or journey Sequence.For example, program may include kernel, middleware, Application Programming Interface (API) and/or application (or application).

Including the data capture unit 141-1 in data unit 141, pretreatment unit 141-2, learning data choosing Select in unit 141-3, model learning unit 141-4 and model evaluation unit 141-5 at least can be at least one hardware core The form of piece manufactures, and can be mounted in the electronic device.For example, data capture unit 141-1, pretreatment unit 141- 2, at least one of learning data selecting unit 141-3, model learning unit 141-4 and model evaluation unit 141-5 can be with In the form of the proprietary hardware chip for AI manufacture, or can be fabricated to existing general processor (for example, CPU or AP) or A part of figure application specific processor (for example, GPU, ISP), and may be mounted in above-mentioned various display devices 100.

In addition, data capture unit 141-1, pretreatment unit 141-2, learning data selecting unit 141-3, model learning Unit 141-4 and model evaluation unit 141-5 can be installed in an electronic equipment, or can be separately mounted on list In only electronic equipment.For example, data capture unit 141-1, pretreatment unit 141-2, learning data selecting unit 141-3, Model learning unit 141-4 and a part of of model evaluation unit 141-5 can be included in display device 100, and its He can partially be included in server 200.

Data capture unit 141-1, pretreatment unit 141-2, learning data selecting unit 141-3, model learning unit At least one of 141-4 and model evaluation unit 141-5 can be realized by using software module.Work as data capture unit 141-1, pretreatment unit 141-2, learning data selecting unit 141-3, model learning unit 141-4 and model evaluation unit When at least one of 141-5 is by using software module (or program module including instruction) Lai Shixian, software module can be with It is stored in non-transitory computer readable recording medium.It can be provided by OS or at least one can be provided by scheduled application A software module.Alternatively, a part of of at least one software module can be provided by OS, and other parts can be answered by predetermined With offer.

Fig. 4 b is the block diagram of data identification unit according to an embodiment of the present disclosure.

With reference to Fig. 4 b, data identification unit 142 according to the embodiment may include data capture unit 142-1 and identification knot Fruit provides unit 142-4.In addition, data identification unit 142 includes pretreatment unit 142-2, identification number with being also an option that property According at least one of selecting unit 142-3 and model modification unit 142-5.

Identification data needed for data capture unit 142-1 can obtain certain situation.

Recognition result provide unit 142-4 can by by selected identification data application in data identification model come true Pledge love condition.Recognition result, which provides unit 142-4, can provide recognition result according to the purpose of identification input data.Recognition result There is provided unit 142-4 can by using the identification data selected by identification data selection unit 142-3 as input value by institute Select data application in data identification model.Furthermore it is possible to determine recognition result by data identification model.

For example, recognition result offer unit 142-4 can be according to the classification standard determined at data identification model to institute The picture frame of selection is classified.In addition, recognition result offer unit 142-4 can be with the characteristic value of output category, so that processing Device 140 generates fingerprint.In another example, recognition result, which provides unit 142-4, can be applied to number for selected picture frame According to identification model, and it can determine the type of content belonging to picture frame.In response to the type of identified content, processor 140 can request the finger print data of the granularity corresponding to content type from server 200.

According to various embodiments of the present disclosure, recognition result provide unit 142-4 can by using with predetermined video/sound The information-related identification data of frequency evidence obtain the result of the type of determining video/audio data as input value.

The information of video/audio data may include, for example, the position of the present frame in entire video/audio and reproduction Time, these are the information of present frame.In addition, the information about video/audio may include video/audio title, video/sound Frequency ID, video/audio provider, video/audio series information, type, about video/audio whether be real-time broadcast letter Whether breath is paid at least one of information of content about video/audio.

According to various embodiments of the present disclosure, by using with the information and video/sound about predetermined video/audio data As input value, recognition result provides unit 142-4 and can obtain to be estimated making the related identification data of the type of frequency evidence Period video/audio data be changed to the probability of another video/audio data as a result, estimation video/audio data The result of secondary reproduction in the case where reproducing after completion.

Data identification unit 142 can also include pretreatment unit 142-2, identification data selection unit 142-3 and model Updating unit 142-5, so as to improve identification data identification model result or so as to save provide recognition result needed for money Source or time.

Pretreatment unit 142-2 can pre-process data obtained and identification data obtained are determined for Situation.Pretreatment unit 142-2 can handle identification data obtained in a predetermined format, so that recognition result provides unit 142-4 can be used identification data obtained and carry out certain situation.

Identify that data selection unit 142-3 can the identification data needed for selecting certain situation in preprocessed data.It can Unit 142-4 is provided so that selected identification data are supplied to recognition result.Identify that data selection unit 142-3 can basis Predetermined selection criteria identification data needed for selecting certain situation in preprocessed data for certain situation.In addition, identification Data selection unit 142-3 can be selected according to according to the predetermined selection criteria by above-mentioned model learning unit 141-4 acquistion Data.

Model modification unit 142-5 can be based on the assessment for the recognition result for providing recognition result unit 142-4 offer To control more new data identification model.It is provided for example, recognition result can be provided unit 142-4 by model modification unit 142-5 Recognition result be supplied to model learning unit 141-4 so that model learning unit 141-4 controlled with more new data identification Model.

Including the data capture unit 142-1 in data identification unit 142, pretreatment unit 142-2, identification data choosing Select unit 142-3, recognition result provide at least one of unit 142-4 and model modification unit 142-5 can be at least one The form of a hardware chip manufactures, and can be mounted in the electronic device.For example, data capture unit 142-1, pretreatment Unit 142-2, identification data selection unit 142-3, recognition result are provided in unit 142-4 and model modification unit 142-5 At least one can be manufactured in the form of the proprietary hardware chip for AI, or can be manufactured to existing general processor A part in (for example, CPU or AP) or figure application specific processor (for example, GPU, ISP), and can be installed in above-mentioned each In kind display device 100.

In addition, data capture unit 142-1, pretreatment unit 142-2, identification data selection unit 142-3, recognition result An electronic equipment can be installed in by providing unit 142-4 and model modification unit 142-5, or can be separately mounted on In individual electronic equipment.For example, data capture unit 142-1, pretreatment unit 142-2, identification data selection unit 142- 3, recognition result provides unit 142-4 and a part of of model modification unit 142-5 and can be included in display device 100, And other parts can be included in server 200.

Data capture unit 142-1, pretreatment unit 142-2, identification data can be realized by using software module Selecting unit 142-3, recognition result provide at least one of unit 142-4 and model modification unit 142-5.When by using Software module (or program module including instruction) Lai Shixian data capture unit 142-1, pretreatment unit 142-2, identification number When providing at least one in unit 142-4 and model modification unit 142-5 according to selecting unit 142-3, recognition result, software mould Block can be stored in non-transitory computer readable recording medium.It can be provided, or can be mentioned by scheduled application by OS For at least one software module.Alternatively, a part of at least one software module can be provided by OS, and can be answered by predetermined With offer other parts.

Fig. 5 is the block diagram for showing the configuration of display device according to an embodiment of the present disclosure.

With reference to Fig. 5, display device 100 may include display 110, memory 120, communication unit 130, processor 140, Picture receiver 150, image processor 160, audio processor 170 and audio output device 180.

Display 110 can show various picture materials, information, the UI etc. provided by display device 100.Specifically, it shows Show that device 110 can show the picture material provided by external equipment (for example, set-top box) and UI window.For example, UI window can be with Menu, content correlated information, additional application including EPG, for selecting the content to be reproduced run button, guidance message, lead to Know message, function setting menu, calibration setting menu, operation executive button etc..Display 110 may be realized in various forms, The various forms liquid crystal display (LCD), Organic Light Emitting Diode (OLED), Activematric OLED (AM-OLED), etc. Gas ions display panel (PDP) etc..

Memory 120 can store various programs and data needed for the operation of display device 100.Memory 120 can be with It is realized in the form of flash memory, hard disk etc..For example, memory 120 may include for storing the behaviour for executing display device 100 The read-only memory (ROM) of the program of work, and the random of data of the operation for temporarily storing adjoint display device 100 are deposited Access to memory (RAM).In addition, memory 120 can also include the electrically erasable ROM for storing various reference datas (EEPROM)。

Memory 120 can store the program and data for configuring the various screens to show on display 110.This Outside, memory 120 can store the program and data for executing special services.For example, memory 120 can store multiple fingers Line, viewing history, content information etc..Fingerprint can be generated by processor 140, or can be received from server 200.

Communication unit 130 can be communicated according to various types of communication means with server 200.Communication unit 130 can be with In a wired or wireless fashion be connected thereto server 200 and exchange finger print data.In addition, communication unit 130 can be from server 200 reception content information control signal, additional information, about the production occurred in the content for changing the content recognition period The information etc. of product.In addition, communication unit 130 can transmit the image data of (stream) from external server as a stream.Communication Unit 130 may include the various communication chips for supporting wire/wireless communication.For example, communication unit 130 may include The chip operated in cable LAN (LAN), Wireless LAN (WLAN), Wi-Fi, bluetooth (BT) or near-field communication (NFC) method.

Picture receiver 150 can receive picture contents data by each provenance.For example, picture receiver 150 can be from External broadcasting station receives broadcast data.In another example, picture receiver 150 can from external equipment (for example, set-top box, Digital versatile disc (DVD) player) image data is received, or can be received by communication unit 130 from external server The image data of stream transmission.

Image processor 160 executes image procossing to from the received image data of picture receiver 150.Image processor 160 can execute various image processing operations, such as decoding, scaling, noise filtering, frame-rate conversion or resolution to image data Rate conversion.

Audio processor 170 can execute the processing to audio data.For example, audio processor 170 can be to audio number According to executing decoding, amplification, noise filtering etc..

Audio output device 180 can not only export the various audio datas in audio processor processing, can also export each Kind notification voice or speech message.

Processor 140 can control the said elements of display device 100.For example, processor 140 can pass through communication unit 130 receive fingerprint or content information.In addition, institute's received content information can be used in processor 140 when carrying out Suitable content identification Section.It can realize that processor 140, can also be by using with executive control operation, search operation etc. by using single cpu Multiple processors are realized with the IP for executing specific function.

Hereinafter, reference attached drawing is described below to the operation of processor 140.

Fig. 6 is the view for showing mixing ACR according to an embodiment of the present disclosure.

With reference to Fig. 6, mixes ACR and refer to (being stored in memory 120 according to its 140 use of processor using local ACR Finger print information identify the content of reproduction) and server ACR (according to it by identifying content from the received information of server) Combined method.When combining local ACR and server ACR, it is possible to reduce the load on server 200, and show dress Setting 100 can accurately identify what content reproduced.

Processor 140 can identify what content be currently reproduced by, and can based on recognition result be adjusted with ACR is executed with precision appropriate.For example, processor 140 can identify the period based on content information Suitable content, and can be pre- The content that first determination will be received and stored from server 200, and the fingerprint quantity about content.

With reference to Fig. 6, processor 140 can extract the characteristic of the content screen currently reproduced, and fingerprint can be generated.Separately Outside, processor 140 can be searched for from the multiple fingerprints being stored in memory 120 whether there is and fingerprint matching generated Fingerprint (1. local ACR).

In response to searching the fingerprint with fingerprint matching generated in memory 120, processor 140 can based on pair The information of the content for the fingerprint that Ying Yu is searched identifies the content currently reproduced.For example, about the content for corresponding to fingerprint Information may include position, recovery time etc. of the present frame in entire content, these are the information about present frame.In addition, Information about the content for corresponding to fingerprint may include content name, content ID, content supplier, content series information, class Whether whether type be the information of real-time broadcast about content and be at least one of information of pay content in relation to content,.

In response to local ACR as described above success, processor 140 need not attempt execute server ACR, therefore can subtract Load on few server 200.For local ACR appropriate, memory 120 should store necessary finger print information and content letter Breath.This will be described again below.

On the other hand, in response to not searching the fingerprint with fingerprint matching generated, processor in memory 120 140, which can control communication unit 130, will be used to that the inquiry about the information of the content currently reproduced to be requested to be sent to server 200 (2. server A CR).For example, inquiry may include the fingerprint generated, viewing history, about information of display device 100 etc..

From display device 100 receive server A CR request in the case where, server 200 can pre-establish about The fingerprint database of various picture materials.Server 200 can analyze picture material and can extract the spy of all picture frames Property.In addition, extracted characteristic can be used to generate the fingerprint for the picture frame that is distinguished from each other in server 200.Server 200 Fingerprint generated can be used to establish database.

Server 200 can extract at least one fingerprint from requested inquiry.In addition, server 200 can built Extracted fingerprint is searched in vertical database, and what content can be currently reproduced by with identification display device 100.Service Device 200 can send display device 100 for the content information identified.In addition, the content that server 200 can will be identified Information is added to the viewing history of each display device, and can manage viewing history.

In addition, processor 140 can control communication unit 130 and receive from server 200 about current in response to the inquiry The fingerprint of the information of the content of reproduction and the content currently reproduced.Here, the fingerprint of the content currently reproduced can be about The time, upper (in time) was located at the fingerprint of the frame after the frame currently shown in entire content.Since processor 140 is based on inquiry In include fingerprint know the time indicated by position of the frame currently shown in entire content, therefore processor 140 can be with The fingerprint it is contemplated that on the frame reproduced after the frame currently shown is received from server 200.

In order to be appropriately combined local ACR and server ACR, processor 140 can result and sight based on content recognition At least one of history is seen to determine being local ACR from the received fingerprint of server 200, and may determine whether to change The content recognition period.

In accordance with an embodiment of the present disclosure, in processor 140 can be determined based on the information about the content currently reproduced The type of appearance.In addition, processor 140 can change the content recognition period according to the type of identified content.It can basis The details of content, type, about content whether be information, importance of real-time broadcast etc. standard to the type of content into Row classification.

According to another embodiment of the present disclosure, processor 140 can be used be arranged to based on the information about content come The data identification model of the type of content is estimated to estimate the type of content.For estimating that the data identification model of content type can To be, for example, being set to based on the information and content with content (for example, video/audio data) (for example, video/audio number According to) the relevant learning data of type learn to have standard for estimating content (for example, video/audio data) type Data identification model.

Fig. 7 is to show the view according to an embodiment of the present disclosure with varigrained finger print information.

With reference to Fig. 7 (a) and Fig. 7 (b), for example, reproducing news content or ad content, processor in response to identifying 140 can reduce the content recognition period.Since news content or ad content often change its details, processor 140 can It can need accurately to identify the content of reproduction.For example, the content recognition period can be set in processor 140, so that processor 140 is tasted Examination identifies the content in every frame.In this case, as shown in Fig. 7 (a), processor 140 can request to close from server 200 The finger print information for all frames to be reproduced after the frame currently shown.

In another example, in response to identifying just in reproducing broadcasting program perhaps movie contents, processor 140 can be with Increase the content recognition period.About broadcast program contents or movie contents, processor 140 be not required in each frame of to master include Details, and can only grasp and whether continuously reproduce identical content.Therefore, processor 140 can request only to show currently Frame after in the frame for the time showing for corresponding to the content recognition period include finger print information in the frame to be reproduced.Fig. 7's (b) In example, since every four frame only needs a finger print information, the granularity of fingerprint is lower than the granularity of Fig. 7 (a).

In accordance with an embodiment of the present disclosure, processor 140 can be by analyzing in the user of display device 100 is watching The application run when appearance determines the content recognition period.For example, processor 140 can usually incite somebody to action in the case where theatrical content The content recognition period is set as long.However, having been running for shopping application simultaneously in response to determining that there are users when watching drama And the history of PPL product is had purchased, processor 140 can reduce the content recognition period about theatrical content.Processor 140 can To determine which frame of theatrical content shows PPL product, and it can control the correlation that the display of display 110 has theatrical content Advertisement.In addition, processor 140 can control the display of display 110 for running the UI of shopping application immediately.

As described above, processor 140 can be learned based on the information of viewing historical information and the additional application about operation It commonly uses in the standard for determining the content recognition period.By doing so, processor 140 can be personalized for determining each user's The standard of content recognition period.When learning the standard for determining the content recognition period, AI study is can be used in processor 140 Scheme, such as above-mentioned unsupervised learning.

In accordance with an embodiment of the present disclosure, processor 140 can differently determination will be from clothes in advance according to the content identified The quantity of the fingerprint of business device request.In addition, processor 140 can be according to the content type and identified content recognition identified Period determines the fingerprint quantity of the subsequent frame about the content currently reproduced.

The quantity of the frame received and stored in advance from server 200 can change according to the content type of identification.For example, In the case where VOD or DVR, server 200 can have the information about all picture frames, but in the case of a live broadcast, Server 200 can only receive several seconds image informations (for example, hundreds of figures in the case where 60Hz before display device 100 As the information of frame).Due to there is fingerprint corresponding with each frame, so display device 100 is in advance from the received finger of server 200 The quantity of line and storage can also change.

For example, being theatrical content in response to determining identified content type and being therefore to know for every 30 seconds by curriculum offering It is not primary, processor 140 can the quantity of the fingerprint to be requested of determination be 0.It to be broadcast live since server 200 does not correspond to In the fingerprint of frame that is reproduced after 30 seconds, processor 140 can be omitted unnecessary communication process.

In accordance with an embodiment of the present disclosure, processor 140 can be based on the information and viewing history about the content identified To change the content recognition period.Viewing history may include content, the viewing time and in viewing time that user has watched The additional application that period has run.

Fig. 8 is the view for showing viewing historical information according to an embodiment of the present disclosure.

With reference to Fig. 8, processor 140 can determine whether connect by comparing the Current Content and viewing history that are identified It is continuous to reproduce the content currently reproduced or another content be reproduced.In addition, processor 140 can be according to the general of another content of reproduction Rate changes the content recognition period.In addition, in processor 140 can request to be reproduced about next expection from server 200 The information of appearance, and can in advance from server 200 receive local ACR needed for information.By doing so, execution can be reduced The probability of server A CR, therefore the load on server 200 can be reduced, and display device 100 can also be identified accurately Content.

According to various embodiments of the present disclosure, data identification model can be used to estimate to reproduce in another in processor 140 The probability of appearance estimates the next expection content to be reproduced, the data identification model be arranged to based on the information for reproducing content and The type estimation of content reproduces another content during reproduction, alternatively, what estimation next expection after reproducing completion to be reproduced Content.

Type based on information, content (for example, video/audio data) with content (for example, video/audio data) and Viewing history (for example, the history for having been changed to another video/audio data) phase of content (for example, video/audio data) The learning data of pass is arranged to estimate to reproduce the probability of another content or estimation during reproduction in the case where reproducing after completion The data identification model of the content of reproduction can be estimated the probability that another content is reproduced during reproduction or estimation phase by secondary expection The content for hoping reproduce next time.

For example, the user of display device 100 is usually between 17:00 to 18:00 in channel 3 with reference to the viewing history of Fig. 8 Upper viewing news content.When the content currently identified in 17:30 is the music content on channel 2, processor 140 can be determined A possibility that channel is changed is very high.Therefore, the content recognition period can be adjusted to shorter and can be frequent by processor 140 Whether the content that ground inspection reproduces is changed.

In accordance with an embodiment of the present disclosure, it is relevant to content attached from the reception of server 200 to can use fingerprint for processor 140 Add information.For example, additional information may include content name, content reproduction time, content supplier, about occur in the content PPL product information, with PPL generate information-related advertisement, the additional application that can be run etc..

Fig. 9 is the view for showing the display of the substantial additional information of tool according to an embodiment of the present disclosure.

With reference to Fig. 9, processor 140 can receive the additional letter that instruction PPL product 910 is included in particular image frame Breath.In addition, processor 140 can control display 110 display include received additional information and content UI 920.UI 920 may include photo, guide message, additional application operation button of PPL product 910 etc..

According to above-described embodiment of the disclosure, display device 100 can be subtracted by dynamically adjusting the content recognition period Load on few server 200, while accurately carrying out ACR.

Figure 10 is the flow chart for showing the content identification method of display device according to an embodiment of the present disclosure.

With reference to Figure 10, display device 100 can capture the content screen currently reproduced first.In addition, display device 100 can be with From the screen extraction property captured, and fingerprint can be generated using extracted characteristic at operation S1010.Fingerprint is to use In the identification information for distinguishing an image and other images.Specifically, fingerprint is from including the video or audio signal in frame The performance data of middle extraction.

According to various embodiments of the present disclosure, the data described above with reference to Fig. 3 to Fig. 4 b can be used in display device 100 Identification model generates fingerprint.

At operation S1020, display device 100 may search for whether being stored in the fingerprint of fingerprint matching generated In display device 100.For example, local ACR can be first carried out in display device 100.In response to local ACR success, display device 100 can identify be currently reproduced by what content without sending inquiry to server 200.Due to the inside of display device 100 Limited storage space, display device 100, which should properly select, to be wanted received finger print information in advance and stores.

Display device 100 can be according to search result, that is, the result of the local ACR at operation S1030, it is determined whether External server 200 is sent by the inquiry including fingerprint generated.

Figure 11 is the flow chart for showing the content identification method of display device according to an embodiment of the present disclosure.

Repeated explanation is omitted since operation S1110 and S1120 corresponds to operation S1010 and S1020 with reference to Figure 11.

In response to searching the fingerprint with fingerprint matching generated in display device 100 at operation S1130-Y, show Showing device 100 can use stored fingerprint at operation S1140 to identify the content currently reproduced.

On the other hand, in response to not searching the fingerprint with fingerprint matching generated, display dress at operation S1130-N Setting 100 can send to external server 200 for requesting looking into for the information about the content currently reproduced at operation S1150 It askes.At operation S1160, display device 100 can be from 200 reception content information of server.In addition, display device 100 may be used also To receive the fingerprint of the content about reproduction secondary under expection from server 200.For example, display device 100 may be received in it is whole It is located at the fingerprint of the frame after current rendering frame, and the finger of the frame about another content reproduced expected next time in a content Line.

As described above, display device 100 can identify the content currently reproduced by local ACR or server A CR.Under Wen Zhong, by the operation of display device of the description after identifying content.

Figure 12 a is the method for showing the content recognition period according to an embodiment of the present disclosure for changing display device View.

With reference to Figure 12 a, at operation S1210, display device 100 can identify the content currently reproduced.In addition, operating At S1220, the information of identified content is can be used to determine the type of content in display device 100.For example, can be based on all Information, the standard of importance whether details, type, the content of such as content are real-time broadcast divide the type of content Class.The standard classified for the type to content can be learnt by display device 100 self by AI is used (for example, Data identification model described in Fig. 3 to Fig. 4 b).

In addition, display device 100 can change content knowledge according to the type of identified content at operation S1230 The other period.For example, in the case where the news content or ad content of the frequent details for changing reproduction content, display device 100 It can set short for the content recognition period.In addition, only needing about whether the decision for continuously reproducing the content currently reproduced VOD content in the case where, display device 100 can set long for the content recognition period.

The standard can change according to individual's viewing taste.It is a to be arranged that viewing history can be used in display device 100 Property standard.Display device 100 can be used unsupervised learning method oneself to learn the standard.

Figure 12 b is the method for showing the content recognition period according to an embodiment of the present disclosure for changing display device View.

With reference to Figure 12 b, first processor 140-1 can be controlled to run and be mounted in display device 100 at least One application.For example, first processor 140-1 can capture display image over the display and generate fingerprint, and can be with Execute ACR.

Data identification model can be used to estimate the type of content in second processor 140-2.Data identification model can be with It is, for example, the result using statistical machine learning estimates content using the information of content and/or the fingerprint generated from content One group of algorithm of type.

With reference to Figure 12 b, at operation S1240, first processor 140-1 can identify the content currently reproduced.For example, the Local ACR or server A CR can be used to identify content in one processor 140-1.

At operation S1245, first processor 140-1 can send second processor 140- for the result of content recognition 2.For example, first processor 140-1 can send second processor 140-2 for the result of content recognition, to request at second Manage the type that device 140-2 estimation reproduces content.

At operation S1250, data identification model is can be used to estimate the class of reproduction content in second processor 140-2 Type.For example, data identification model can based on the information and content of content (for example, video/audio data) (for example, video/ Audio data) the related learning data of type estimate the type of content (for example, video/audio data).

At operation S1255, when second processor 140-2 can be according to the identification of the content type export content of estimation Section.For example, it is frequent change the news content or ad content that reproduce the details of content in the case where, display device 100 can be with Set short for the content recognition period.In addition, only needing the VOD about whether the decision for continuously reproducing the content currently reproduced In the case where content, display device 100 can set long for the content recognition period.

At operation S1260, second processor 140-2 can send first processor for the derived content recognition period 140-1.At operation S1270, first processor 140-1 can change content recognition based on institute's received content recognition period Period.

According to various embodiments of the present disclosure, first processor 140-1 can receive estimation from second processor 140-2 Content type, and can be executed at operation S1255.

Figure 13 a is the quantity for showing the fingerprint according to an embodiment of the present disclosure that requested by display device for determination The view of method.

With reference to Figure 13 a, display device 100 can identify the content currently reproduced at operation S1310.In addition, operating At S1320, the information of identified content is can be used to determine the type of content in display device 100.For example, display device 100 can be used data identification model (for example, data identification model described in Fig. 3 to Fig. 4 b) to determine (or estimation) content Type.

Similar to the method for changing the content recognition period according to identified content type, display device 100 can grasped Making determination at S1330 will be from the quantity of the received fingerprint of server 200.Due to the number for the picture frame being present in server 200 Amount changes according to the type of content, therefore the quantity for the fingerprint for corresponding to each frame and being present in server 200 can also Changed with the type according to content.

Display device 100 can by consider the type of content, viewing history, about content whether be live streaming information etc. To determine the quantity of received fingerprint.In response to received determination fingerprint quantity, display device 100 can execute optimization Local ACR, while minimizing the quantity of fingerprint being stored in display device 100.

Figure 13 b is the amount for showing the fingerprint that requested by display device for determination according to another embodiment of the disclosure The view of method.

With reference to Figure 13 b, first processor 140-1 can be controlled to run and be mounted in display device 100 at least One application.For example, first processor 140-1 can capture display image over the display and generate fingerprint, and can be with Execute ACR.

Data identification model can be used to estimate the type of content in second processor 140-2.Data identification model can be with It is, for example, using statistical machine learning as a result, estimating the class of content using the information of content and the fingerprint generated from content One group of algorithm of type.

With reference to Figure 13 b, first processor 140-1 can identify the content currently reproduced at operation S1340.

At operation S1345, first processor 140-1 can send second processor 140- for the result of content recognition 2。

At operation S1350, data identification model is can be used to estimate the class of reproduction content in second processor 140-2 Type.For example, data identification model can based on the information and content of content (for example, video/audio data) (for example, video/ Audio data) the related learning data of type estimate the type (for example, video/audio data) of content.

At operation S1355, second processor 140-2 can be exported according to estimated content type will be from server The quantity of the fingerprint of (for example, server 200 of Fig. 1) request.Due to the picture frame being present in server 200 quantity according to The type of content and change, therefore correspond to each frame and the quantity of fingerprint being present in server 200 can also basis The type of content and change.

At operation S1360, second processor 140-2 can be sent the derived fingerprint quantity to be requested at first Manage device 140-1.At operation S1365, first processor 140-1 can based on received fingerprint quantity to be requested to determine The quantity of fingerprint.

According to various embodiments of the present disclosure, first processor 140-1 can receive estimation from second processor 140-2 Content type, and can be executed at operation S1355.

Figure 14 a, Figure 14 b, Figure 15 a and Figure 15 b are shown according to the various embodiments of the disclosure for predictive display device Content method view.

With reference to Figure 14 a, display device 100 can identify the content currently reproduced at operation S1410.

In addition, display device 100 can information and viewing history based on the content about identification at operation S1420 To calculate the probability that the content currently reproduced is changed.For example, viewing history may include channel, the viewing that user has watched Time, the ID of display device, user information, additional application of operation etc..

According to various embodiments of the present disclosure, data identification model can be used (for example, Fig. 3 to Fig. 4 b in display device 100 Described in data identification model) estimate probability that the content currently reproduced is changed.

In addition, display device 100 can change the content recognition period according to calculated probability at operation S1430. For example, being shown in response to determining that user generally prefers that with reference to viewing history come the different content of the content watching from currently identify Device 100 can determine that the content currently reproduced is most likely altered.In this case, display device 100 can be by content The identification period is changed into short.

On the other hand, the content for corresponding to usually viewing history is reproduced in response to determining, display device 100, which can determine, to be worked as A possibility that content of preceding reproduction is changed is very low.In this case, display device 100 can change the content recognition period For length.

Figure 14 b is to show according to an embodiment of the present disclosure be used in the display including first processor and second processor The view of the method for predictive content and change content recognition period in device.

With reference to Figure 14 b, first processor 140-1 can be controlled to run and be mounted in display device 100 at least One application.For example, first processor 140-1 can capture display image over the display and generate fingerprint, and can be with Execute ACR.

Data identification model can be used to estimate probability that content is changed in second processor 140-2.Data identify mould Type can be, for example, being used for type and video/audio based on information, video/audio data with video/audio data The relevant learning datas of viewing history (for example, the history for having been changed to another video/audio data) of data is estimated Video/audio data is changed to one group of algorithm of the probability of another video/audio data during reproduction.

With reference to Figure 14 b, at operation S1440, first processor 140-1 can identify the content currently reproduced.

At operation S1445, first processor 140-1 can send second processor 140- for the result of content recognition 2。

At operation S1450, data identification model is can be used to estimate to reproduce content and be changed in second processor 140-2 Probability.

At operation S1455, second processor 140-2 can identify the period according to estimated probability come export content.

At operation S1460, second processor 140-2 can send first processor for the derived content recognition period 140-1。

At operation S1465, first processor 140-1 can be known based on institute's received content recognition period to change content The other period.

According to various embodiments of the present disclosure, first processor 140-1 can change from second processor 140-2 reception content The estimated probability of change, and can be executed at operation S1455.

With reference to Figure 15 a, display device 100 can identify the content currently reproduced at operation S1510.In addition, operating At S1520, display device 100 can predict down the content of secondary reproduction based on viewing history.For example, frequent in response to having The user of the display device 100 of the viewing history of specific two channels is watched, display device 100 can be predicted will be in two channels The content of the content of middle reproduction secondary reproduction as under.

According to various embodiments of the present disclosure, data identification model can be used (for example, Fig. 3 to Fig. 4 b in display device 100 Described in data identification model) estimate the content to be reproduced after the content currently reproduced.

At operation S1530, display device 100 can request the finger print information of predictive content from server 200.In addition, Display device 100 can receive the information about predictive content from server 200 in advance, and can deposit at operation S1540 Store up the information.

The information about predictive content for being sent to display device 100 from server 200 may include currently filling in display The fingerprint of the information, the content currently reproduced of setting the content reproduced in 100 and be predicted as next time reproduction content and for changing Become at least one of the control signal of content recognition period of display device 100.For example, display device 100 can in advance from Server 200 receives the finger print information for the content to reproduce in above-mentioned two channel, and can be stored.In this way It does, display device 100 can receive the optimization fingerprint that be used for local ACR.

Figure 15 b is to show according to an embodiment of the present disclosure be used for including first processor and second processor 140-2 Display device in predictive content and receive about institute's predictive content information method view.

With reference to Figure 15 b, first processor 140-1 can be controlled to run and be mounted in display device 100 at least One application.For example, first processor 140-1 can capture display image over the display and generate fingerprint, and can be with Execute ACR.

Data identification model can be used to estimate probability that content is changed in second processor 140-2.Data identify mould Type can be, for example, type and video/audio data based on information, video/audio data with video/audio data The related learning data of viewing history (for example, the history for having been changed to another video/audio data), for estimate again The algorithm for the video/audio data to be reproduced after now completing.

With reference to Figure 15 b, at operation S1550, first processor 140-1 can identify the content currently reproduced.

At operation S1555, first processor 140-1 can send second processor 140- for the result of content recognition 2。

At operation S1560, data identification model is can be used to estimate in currently reproducing in second processor 140-2 The content to be reproduced after appearance.

At operation S1565, second processor 140-2 the content of secondary reproduction can be sent to first processor by under 140-1。

At operation S1570, first processor 140-1 can request to close from server (for example, server 200 of Fig. 1) In the information of predictive content.

First processor 140-1 can receive the letter about predictive content from server (for example, server 200 of Fig. 1) Breath, and the information is stored at operation S1575.

According to various embodiments of the present disclosure, second processor 140-2 can be executed at operation S1570.

Figure 16 is to show according to an embodiment of the present disclosure learnt by display device interlocked with one another and server and identification Data view.

With reference to Figure 16, server 200 can learn for identification for estimating the standard of content and/or for estimating content Type standard and/or standard for estimating probability that content is changed, and display device 100 can be set for base The standard of picture frame is distinguished in the learning outcome of server 200, and can be the type of content and that content is changed is general Rate.

In this case, the data unit 240 of server 200 may include data capture unit 240-1, pre- place Manage unit 240-2, learning data selecting unit 240-3, model learning unit 240-4 and model evaluation unit 240-5.Data Unit 240 can execute the function of data unit 141 shown in Fig. 4 a.The data unit of server 200 240 can learn so that data identification model has the standard for analyzing the characteristic of video/audio data.Server 200 can With the characteristic of the frame of each capture of standard analysis according to acquistion, and fingerprint can be generated.

Data unit 240 can determination what learning data will determine the spy of capture content screen (or frame) using Property.In addition, identified learning data can be used to learn the characteristic of the content for extracting capture in data unit 240 Standard.Data unit 240 can obtain the data that be used for learning, and can by the data application that will obtain in The data identification model that will be described below learns the standard for analytical characteristics.

According to various embodiments of the present disclosure, data unit 240 can learn so that data identification model has use In estimated based on learning data relevant to the type of the information of predetermined video/audio data and video/audio data video/ The standard of the type of audio data.

According to various embodiments of the present disclosure, data unit 240 can learn so that data identification model has use In based on predetermined video/audio data the type of information, video/audio data and the viewing of video/audio data go through The related learning data of history (for example, having the history for changing into another video/audio data), estimation regard during reproduction Frequently/audio data is changed to the sound of probability or the estimation secondary reproduction in the case where reproducing after completion of another video/audio data The standard of frequency data video.

In addition, the recognition result of display device 100 unit 142-4 is provided can be by will be by identification data selection unit The data application of 142-3 selection is in the data identification model generated by server 200 come certain situation.In addition, recognition result mentions The data identification model generated by server 200 can be received from server 200 for unit 142-4, and can be used and connect The data identification model of receipts come analyze image or determine content type.In addition, the model modification unit 142- of display device 100 5 can be supplied to analysed image and identified content type the model learning unit 240-4 of server 200, so that It can more new data identification model.

For example, display device 100, which can be used, identifies mould by using the data of the computing capability generation of server 200 Type.In addition, multiple display devices 100 will learn or the data information of identification is sent to server 200, data are updated The identification model of server 200.In addition, each of multiple display devices 100 will learn or the data information of identification is sent To server 200, so that server 200 can be generated as the personalized data identification model of each display device 100.

Figure 17 is the flow chart for showing the content identification method of display system according to an embodiment of the present disclosure.

With reference to Figure 17, display system 1000 may include display device 100 and server 200.Figure 17 shows showing That fingerprint is requested at device 100 pulls method.

Firstly, display device 100 can capture the content screen currently reproduced at operation S1605.In addition, display dress Setting 100 can analyze captured screen and extraction property.At operation S1610, display device 100 can be used extracted Characteristic generates the fingerprint for distinguishing the screen captured and other picture frames.

Operation S1615 at, display device 100 can execute local ACR so that fingerprint generated and storage fingerprint Match.The case where content currently reproduced is identified by local ACR will not described.In response to the content that currently reproduces not by local ACR identification, display device 100 can send server for the inquiry including fingerprint generated at operation S1625.

Server 200 may analyze various contents, and can establish fingerprint database at operation S1620. Server 200 can take the fingerprint from the received inquiry of institute.In addition, server 200 can will be mentioned at operation S1630 The fingerprint taken matches with the multiple fingerprints being stored in fingerprint database.Server 200 can be identified by matching fingerprint What the content that display device 100 is inquired is.At operation S1635, server 200 can be by the letter about the content of identification Fingerprint on breath and the next image frame of the content of identification is sent to display device 100.

For example, display device 100 can send the inquiry for requesting the information of the fingerprint generated at server 200. Server 200 inquiry API can be used generate to received inquiry response, and response can be provided.Inquiring API can To be API, the fingerprint for including in search inquiry in fingerprint database and the relevant information that storage is provided.In response to receiving Inquiry, the inquiry API of server 200, which may search for the fingerprint for including in inquiry, whether there is in fingerprint database.In response to Searched fingerprint, inquiry API can will correspond to the fingerprint searched in response to inquiring in the title of content, entire content Position, recovery time of frame etc. are sent to display device 100.In addition, inquiry API can will correspond in entire content when Between frame fingerprint after the upper frame for being located at display fingerprint be sent to display device 100.

In addition, when the content reproduced at display device 100 can be transmitted as a stream by server 200 (for example, VOD Or broadcast singal), server 200 can send display device 100 for the next frame of fingerprint and the content identified.This In the case of, the respective frame of fingerprint and the content identified can be matched and be sent.For example, can be to be added to one of content The form of file provides fingerprint, and can include in fingerprint by the information for being used to map fingerprint and respective frame.

Display device 100 can be used received content information determine the type of content, importance etc..In addition, grasping Make at S1640, display device 100 can change the content recognition period based on standards such as the type of content, importance.Separately Outside, at operation S1645, the viewing history with content information is can be used to predict down secondary reproduction in display device 100 Content.Under the content of secondary reproduction refer to the content different from the content currently reproduced.

At operation S1650, display device 100 can request the fingerprint about the content predicted from server 200.Separately Outside, display device 100 can request predicted content itself and fingerprint from server 200.In response to this, server 200 Display device 100 can be sent by requested fingerprint at operation S1655.Clothes are stored in response to the content predicted The content be engaged in device 200 or predicted can be streamed to display device 100 by server 200, and server 200 is not Only requested fingerprint can be sent to display device 100, the content with fingerprint pairing can also be sent.

Display device 100 can store the received fingerprint of institute, and can incite somebody to action when next content recognition period arrives It is used for local ACR.

Figure 18 is the flow chart for showing the content identification method of display system according to an embodiment of the present disclosure.

With reference to Figure 18, display system 1000 may include display device 100 and server 200.Figure 18 is shown in which to take Business device 200 preemptively sends the method for pushing of fingerprint.

Firstly, display device 100 can capture the content screen currently reproduced at operation S1805.In addition, display dress Setting 100 can analyze captured screen and extraction property.At operation S1810, display device 100 can be used extracted Characteristic generates the fingerprint for distinguishing the screen of capture and other picture frames.

Operation S1815 at, display device 100 can execute local ACR so that fingerprint generated and storage fingerprint Match.In addition, not identified by local ACR in response to the content currently reproduced, display device 100 can be at operation S1825 Server is sent by the inquiry including fingerprint generated.

Other than fingerprint, inquiry can also include the information of display device 100.For example, the information of display device 100 can To be the IP address of the physics ID of display device 100, display device 100, or the letter for specifying the user of display device 100 Breath, the User ID such as sent by display device 100 to server 200.

The information of display device 100 can be used to manage the viewing history in each display device 100 in server 200. For example, server 200 can be with collecting device ID, and can be used for managing each set in response to client device access The client-side management API for checking historical record of standby ID executes aforesaid operations.

Server 200 may analyze various contents, and can establish fingerprint database at operation S1820. Server 200 can will correspond to the information storage of the content of fingerprint in the database.For example, server 200 can be in data Title of the storage corresponding to the content of fingerprint, the position of the frame corresponding to the fingerprint in entire content, recovery time, content in library ID, content supplier, content series information, type, about content whether be real-time broadcast information, about content whether be pay Take the information etc. of content.

Server 200 can take the fingerprint from the received inquiry of institute.For example, the inquiry API of server 200 can only from Information corresponding with fingerprint is extracted in the character string information of the inquiry received.In addition, at operation S1830, server 200 Extracted fingerprint and the multiple fingerprints being stored in fingerprint database can be matched.

What server 200 can be come the content that identification display device 100 is inquired by matching fingerprint.Server 200 type, the importance etc. that identified content information can be used to determine content.In addition, at operation S1835, service Device 200 can change the content recognition period based on the standard of the type of content, importance etc..In addition, in operation S1840 The viewing history with content information can be used to predict down the content of secondary reproduction in place, server 200.For example, in Figure 18 Embodiment in, server 200 can execute change the content recognition period and prediction under secondary reproduction content operation.

Based on the information grasped in this process, at operation S1845, server 200 can be from display device Display device 100 is sent by finger print information etc. in the case where 100 reception requests.For example, server 200 can be to display device 100 send the information of content currently reproduced at display device 100, the sum of the content currently reproduced is predicted as reproduction next time The fingerprint of content and the control signal of the content recognition period for changing display device 100.In another example, it services Device 200 can send the content itself and fingerprint for being predicted to be reproduction next time.In this case, fingerprint can be with the institute of content There are frame combination or fingerprint that can combine with each interval frame being arranged according to the content recognition period.

In addition, server 200 can be sent in the case where not receiving request from display 100 to display device 100 Advertisement screen, product purchase UI of the product for including in frame about the content to be shown in display device 100 etc..Server 200 can buy product based on the information from the received display device 100 of display device 100 during resting in viewing time History.Server 200 can be used such as product purchasing history or watch the customized information of history to change transmission advertisement screen The frequency etc. of curtain.

Server 200 can be used the resolution information of display device 100, indicate which of frame of content partially corresponds to Information of background etc. has the advertisement screen for being suitable for the size that each screen in display device 100 is shown to generate.Separately Outside, server 200 will can be used to show the control signal of advertisement screen on the appropriate location on the screen of display device 100 Display device 100 is sent collectively to advertisement screen.

In another example, server 200 can request second server 300 to provide advertisement screen.For example, the second clothes Business device 300 can be to provide advertisement and provide the individual server of function.Second server 300 can be received from server 200 and be wrapped Include the information of the product for wanting advertisement, resolution ratio of display device 100 etc..According to the received information of institute, second server 300 can be with Generate advertisement screen of a size suitable.Second server 300 can send server for advertisement screen generated 200, or can display device 100 directly be sent by advertisement screen.Directly advertisement screen is sent in second server 300 Into the embodiment of display device 100, server 200 can be to second server 300 with providing such as IP of display device 100 The communication information of location.

Figure 19 is to show display device according to an embodiment of the present disclosure by interlocking with server to be changed according to content The probability of change is come view the case where changing the content recognition period.

With reference to Figure 19, data identification model is can be used to estimate probability that content is changed in server 200.Data identification Model can be, for example, type and video/audio number based on information, video/audio data with video/audio data According to viewing history (for example, the history for having been changed to another video/audio data) relevant learning data estimate again Current video/audio data is changed to one group of algorithm of the probability of another video/audio data.

In such a case, it is possible to define for transmission/reception data between display device 100 and server 200 Interface.

For example, can define with will be applied to data identification model learning data as factor values (or parameter value or Delivery value) API.The API can be defined as one group of subroutine or function, in an agreement (for example, by display device The agreements of 100 definition) at be called to execute certain processing (for example, the agreement defined by server 200) of another agreement.Example Such as, by API, can provide a kind of agreement can execute the environment of operation of another agreement.

With reference to Figure 19, at operation S1910, display device 100 can identify the content currently reproduced.For example, display dress Setting 100 can be used local ACR or server A CR to identify content.

At operation S1920, display device 100 can send server 200 for the result of content recognition.For example, aobvious Showing device 100 can send server 200 for the result of content recognition, estimate that reproducing content is changed with request server 200 The probability of change.

At operation S1930, server 200 can be used the estimation of data identification model and reproduce the probability that content is changed.

At operation S1940, server 200 can identify the period according to the probability of content changing come export content.For example, In response to determining that user usually likes based on the query history (for example, channel change) requested from display device 100 to server 200 Vigorously with reference to viewing history come the different content of the content watching from currently identify, display device 100, which can determine, currently to be reproduced Content is most likely altered.In this case, display device 100 can change into the content recognition period short.

At operation S1950, server 200 can send display device 100 for the derived content recognition period.It is grasping Make at S1960, display device 100 can change the content recognition period based on institute's received content recognition period.

Figure 20 be display device according to an embodiment of the present disclosure be shown predicted down by being interlocked with server it is secondary again Existing content and receive in advance information about predictive content method view.

With reference to Figure 20, data identification model is can be used to estimate probability that content is changed in server 200.Data identification Model can be, for example, type and video/audio data based on information, video/audio data with video/audio data The related learning data of viewing history (for example, the history for having been changed to another video/audio data), for estimate again The algorithm of the video/audio data of secondary reproduction under after now completing.

At operation S2010, display device 100 can identify the content currently reproduced.

At operation S2020, display device 100 can send server 200 for the result of content recognition.

At operation S2030, data identification model is can be used to estimate after the content currently reproduced in server 200 The content to be reproduced.

At operation S2040, server 200 may search for the information about estimation content.

At operation S2050, server 200 can send display device for the information of the content of reproduction secondary about under 100。

Display device 100 can receive the information about estimation content from server 200, and can be in operation S2060 Place stores the information.

Figure 21 is to show display device according to an embodiment of the present disclosure to predict next time and interlocking with multiple servers The view of the content to be reproduced and the method for receiving information about institute's predictive content in advance.

With reference to Figure 21, data identification model is can be used to estimate probability that content is changed in first server 200.Data Identification model can be, for example, type and video/sound based on information, video/audio data with video/audio data The related learning data of viewing history (for example, the history for having been changed to another video/audio data) of frequency evidence, for estimating Count the algorithm of the video/audio data of secondary reproduction in the case where reproducing after completion.

For example, third server 201 may include the Cloud Server for storing the information about content.

At operation S2110, display device 100 can identify the content currently reproduced.

At operation S2120, display device 100 can send first server 200 for the result of content recognition.

At operation S2130, data identification model is can be used to estimate in the content currently reproduced in first server 200 The content to be reproduced later.

At operation S2140, first server 200 can send second server for the content of estimation to request second Server search information.

In operation S2150, third server 201 may search for the information about content.

At operation S2160, third server 201 can send first for the information of the content of reproduction secondary about under Server 200.In addition, first server 200 can send out the information of the content of reproduction secondary about under at operation S2170 It is sent to display device 100.However, according to various embodiments of the present disclosure, third server 201 can will the secondary reproduction about under The information of content be sent to display device 100.

Display device 100 can receive the information about estimation content from first server 200 or third server 201, And the information can be stored at operation S2180.

The some aspects of the disclosure may be embodied in computer-readable in non-transitory computer readable recording medium Code.Non-transitory computer readable recording medium be can store thereafter can by computer system read data it is any Data storage device.The example of non-transitory computer readable recording medium includes read-only memory (ROM), random access memory Device (RAM), CD-ROM (CD-ROM), tape, floppy disk and optical data storage.The computer-readable record of non-transitory is situated between Matter can also be distributed in the computer system of network coupling, so that storing and executing computer-readable generation in a distributed way Code.In addition, function program used to implement the present disclosure, code and code segment can very be held by the programmer of disclosure fields It changes places explanation.

It should be noted at this time that the various embodiments of the present disclosure as described above are usually directed to the processing and certain journey of input data The generation of output data on degree.The input data handles and exports data generation can be real with hardware or software in conjunction with hardware It is existing.For example, can be realized in mobile device or similar or interlock circuit using specific electronic components and sheet as described above The disclosed associated function of various embodiments.Alternatively, can be real according to the one or more processors of the instruction operation of storage Existing function associated with the various embodiments of the present disclosure as described above.If it is the case, within the scope of this disclosure These instructions can be stored on one or more non-transitory processor readable mediums.The example packet of processor readable medium Include ROM, RAM, CD-ROM, tape, floppy disk and optical data storage.Processor readable medium can also be distributed in network coupling In the computer system of conjunction, to store and execute instruction in a distributed way.In addition, function calculating used to implement the present disclosure Machine program, instruction and instruction segment can be easily explained by the programmer of disclosure fields.

It according to various embodiments of the present disclosure, can be by using the finger including storage in a computer-readable storage medium The S/W program of order realizes the disclosed embodiments.

Computer is such equipment, call stored instruction from storage medium, and can be according to being called Instruction may include the display device according to the disclosed embodiments to execute the operation according to the disclosed embodiments.

Computer readable storage medium can be provided in the form of non-transitory storage medium.Here, " non-transitory " It mean onlys that storage medium not and includes signal and be tangible, and do not consider that data are semi-permanently or provisionally to store In storage medium.

In addition, may include in computer program product and providing according to the control method of disclosed embodiment.It calculates Machine program product can be used as product and trade between the seller and buyer.

The computer program product may include S/W program, and the computer readable storage medium of storage S/W program. For example, computer program product may include by the manufacturer of display device or electronic market (for example, Google Play quotient Shop, application shop) product (for example, Downloadable application) of S/W program form electronically distributed.For electronics point Hair, at least part of S/W program can store in storage medium or can temporarily generate.In this case, it stores Medium can be the intermediate clothes of the storage medium of the server of manufacturer, the server of electronic market or interim storage S/W program Business device.

The computer program product may include the storage medium of server or the system including server and display device In equipment storage medium.Alternatively, when there is the third equipment communicated to connect with server or display device (for example, intelligence Phone) when, which may include the storage medium of third equipment.Alternatively, computer program product can wrap Include the S/W program itself for being sent to display device or 3rd device from server or being sent to display device from 3rd device.

In this case, one in server, display device and third equipment can execute computer program product And execution is according to the method for the disclosed embodiments.Alternatively, two or more in server, display device and third equipment Computer program product can be executed and execute the method according to disclosed embodiment in a distributed way.

For example, server (for example, Cloud Server or AI server) can execute the computer journey of storage in the server Sequence product, and can control the display device connecting with server communication to execute the method according to the disclosed embodiments.

In another example, third equipment can run computer program product, and can control logical with third equipment Believe the display device of connection to execute the method according to the disclosed embodiments.When third equipment runs computer program product When, third equipment can be from server downloading computer program product, and can run downloaded computer program product. Alternatively, third equipment can run the computer program product provided with preload condition, and can execute according to Embodiment method.

Although the disclosure has shown and described in the various embodiments by reference to the disclosure, those skilled in the art will Understand, it, can be in the case where not departing from the spirit and scope of the present disclosure as defined in the following claims and their equivalents It carries out various changes of form and details wherein.

Claims

1. a kind of display device, comprising:

Display；

Memory is configured as storing the fingerprint generated by the characteristic for extracting content and about corresponding to the fingerprint The information of content；

Communication equipment, is configured as and server communication, and

At least one processor, is configured as:

It extracts the characteristic of the screen of the content currently reproduced on the display and generates fingerprint, to search in the memory The fingerprint of rope and fingerprint matching generated in the presence/absence of, and

Result based on search, it is determined whether send the server for the inquiry comprising fingerprint generated with request about The information of the content currently reproduced.

2. display device as described in claim 1, wherein at least one described processor is also configured to

In response to searching the fingerprint with fingerprint matching generated in the memory, based on about correspond to searched Fingerprint content information, identify the content that currently reproduces, and

In response to not searching the fingerprint with fingerprint matching generated in the memory, controlling the communication equipment will be wrapped Inquiry containing the fingerprint is sent to the server to request the information about the content currently reproduced.

3. display device as claimed in claim 2, wherein at least one described processor is also configured in response in institute The fingerprint not searched in memory with fingerprint matching generated is stated, controls the communication equipment from the service in response to inquiry Device receives the information of the fingerprint about the content currently reproduced and the content currently reproduced.

4. display device as claimed in claim 3, wherein at least one described processor is also configured to

The type of the content is determined based on the information about the content currently reproduced, and

Change the content recognition period according to the type of the identified content.

5. display device as claimed in claim 4, wherein at least one described processor is also configured to

It is ad content in response to the content, identifies the content in each first period, and

It is broadcast program contents in response to the content, identifies content when than described first in second period of each of segment length.

6. display device as claimed in claim 3, wherein at least one described processor is also configured to

Change the fingerprint quantity for the content that received will currently reproduce according to the type of the identified content.

7. display device as claimed in claim 3, wherein at least one described processor is also configured to

The probability that the reproduction content is changed is calculated with viewing history based on the information about the content currently reproduced, and

Change the content recognition period according to probability calculated.

8. display device as claimed in claim 2, wherein at least one described processor is also configured to

The content of secondary reproduction is predicted down based on viewing history, and

The information in relation to institute's predictive content is requested from the server.

9. display device as claimed in claim 3, wherein at least one described processor is also configured to

Additional information related with the content currently reproduced is received from the server, and

Control display display received additional information and the content that currently reproduces.

10. a kind of method of the content of identification display device, which comprises

It extracts the characteristic of the screen of the content currently reproduced and generates fingerprint；

Whether the fingerprint of search and fingerprint matching generated is stored in the display device；And

Result based on search determine whether by comprising fingerprint generated inquiry be sent to external server with request about The information of the content currently reproduced.

11. method as claimed in claim 10, wherein determine whether that sending a query to external server includes:

In response to searching the fingerprint with fingerprint matching generated in the display device, based on about with the finger that searches The information of the corresponding content of line identifies the content currently reproduced, and

In response to not searching the fingerprint with fingerprint matching generated in the display device, Xiang Suoshu server sends packet Inquiry containing the fingerprint, to request the information about the content currently reproduced.

12. method as claimed in claim 11, further includes: in response to do not searched in the display device with it is generated The fingerprint of fingerprint matching receives the information and current reproduction about the content currently reproduced from the server in response to inquiry Content fingerprint.

13. method as claimed in claim 12, further includes:

The type of the content is determined based on the information about the content currently reproduced；And

According to the type change content recognition period of the identified content.

14. method as claimed in claim 13, wherein change the content recognition period include:

15. method as claimed in claim 12, further includes: