JP2013219447A - Content display device, content reception device, program, and content reproduction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a content display device etc. which allows for appropriate utilization of a content, by user's predetermined instruction on a group of still images displayed on the basis of the content.SOLUTION: The content display device comprises: still image group display means which generates or extracts from image data included in the content the group of still images corresponding to the display content with the timing of an inputted trigger, and displays the group of still images; display position designation means which designates the display position of the displayed group of still images; and recommendation display means which is capable of displaying a recommendation on the basis of information material recognized from the designated display position.

Description

  The present invention relates to a content display device capable of displaying content including video data.

  The use of various contents has increased the need to use multiple contents and services. Taking various broadcast contents as an example, various efforts for fusion of broadcasting and communication of TV are being sought. That is, user needs are increasing for access to related information triggered by broadcast content.

  In this case, out of the information contained in the broadcast program itself displayed on the TV or the like, it is intuitive to directly work on the object of interest or the screen area that it feels necessary to search / save / use on an external device. There is a need for an automated means.

  For example, when the user designates a display object in the video with a touch panel or the like, the instruction coordinate information and instruction timing information corresponding to the instruction are transmitted to the search server, and the data of the corresponding display object is acquired from the search server. Thus, there is disclosed an invention in which detailed information about a desired display item in a video can be displayed on the screen without increasing the amount of distribution data when the video data is distributed. (For example, refer to Patent Document 1).

International Publication No. 2009/145233

  In this way, the user can utilize various information by directly instructing the displayed broadcast content. However, when the user wants to utilize information included in the broadcast content, at present, the distribution side such as a broadcasting station needs to add metadata such as link information using data broadcasting or the like in advance.

  On the other hand, since user needs are very diversified, there has been a problem that the information providing operation is enormous and a great amount of labor and cost are generated on the distribution side. In addition, these operations essentially have a problem that it is difficult to cope with unpredictable live images.

  In addition, when a user wants to acquire information from content, there is currently no means for easily and appropriately pointing objects such as various characters, objects, and people unless detailed metadata is attached. Since information related to these objects is generally diverse, such as videos, blogs, tweets, etc., even if the target can be pointed appropriately, it can lead the user to information utilization and actions appropriately. There was a problem that it was difficult.

  In Patent Document 1 described above, in order to appropriately display information desired by the user, information called a service identifier is used as a search key in addition to instruction coordinate information and instruction timing. This service identifier needs to be prepared in advance by the video data supply side as target data, and cannot always be used appropriately.

  In addition, since an instruction is input to the displayed video data, it is difficult to instruct a fast moving image or a detailed area. In particular, there has been a problem that scenes are switched before an instruction is input for information desired by the user.

  In view of the above-described problems, an object of the present invention is to provide a content display device that allows a user to appropriately use content by giving a predetermined instruction to a group of still images displayed based on the content. Etc. is to provide.

In order to solve the above-described problem, the content display device of the present invention provides:
A content display device capable of displaying content including video data,
Still image group display means for generating or extracting a still image group corresponding to the display content of the input trigger timing from the video data included in the content and displaying it;
Display position designation means capable of designating the display position of the displayed still image group;
Recommendation display means capable of displaying a recommendation based on the information material recognized from the designated display position;
It is characterized by providing.

The content receiving device of the present invention is
In the content receiving device that is connected to the content display device of the present invention and receives content including video data,
Content transmitting means for transmitting content at a trigger timing input to the content receiving device or the content display device to the content display device;
It is characterized by providing.

The content display device of the present invention is
In a content display device connected to a content receiving device that receives content including video data and capable of displaying content transferred from the content receiving device,
Content acquisition means for acquiring content from the content reception device based on the input trigger timing;
Still image group display means for generating or extracting a still image group corresponding to the display content at the input trigger timing from the video data included in the acquired content, and displaying it.
Display position designation means capable of designating the display position of the displayed still image group;
Recommendation display means capable of displaying a recommendation based on the information material recognized from the designated display position;
It is characterized by providing.

The program of the present invention is
In a computer capable of displaying content including video data,
A still image group display function for generating or extracting a still image group corresponding to the display content of the input trigger timing from video data included in the content, and displaying it;
A display position designation function capable of designating the display position of the displayed still image group,
A recommendation display function capable of displaying a recommendation based on the information material recognized from the designated display position;
It is characterized by realizing.

In addition, the content reproduction apparatus of the present invention is
In a content playback apparatus capable of playing back content including video data,
Still image group generation means for generating or extracting a still image group corresponding to the input trigger timing content from the video data included in the content;
Designated position receiving means for receiving an input of a specified position in the still image group;
Recommendation execution means for determining and executing a process for generating a recommendation to the user based on a designated position in the still image group;
It is characterized by providing.

  According to the content display device of the present invention, the still image group corresponding to the display content at the input trigger timing is generated or extracted from the video data included in the content and displayed, and the displayed still image group is displayed. Specify the position. The recommendation can be displayed based on the information material recognized from the designated display position. Thus, when an area is selected based on a group of still images generated from video data included in the content, the information material is recognized based on the selected area, and processing according to the information material can be appropriately executed. It becomes like this.

It is a figure for demonstrating the whole in 1st Embodiment. It is a figure for demonstrating the function structure of the content display apparatus in 1st Embodiment. It is a figure for demonstrating the capture process in 1st Embodiment. It is a figure for demonstrating the analysis process in 1st Embodiment. It is a figure for demonstrating operation | movement of the touch input in 1st Embodiment. It is a figure for demonstrating the character information processing with respect to the still image group in 1st Embodiment. It is a figure for demonstrating the character information processing with respect to the still image group in 1st Embodiment. It is a figure for demonstrating the character information processing with respect to the still image group in 1st Embodiment. It is a figure for demonstrating the 1st Example in 1st Embodiment. It is a figure for demonstrating the 2nd Example in 1st Embodiment. It is a figure for demonstrating the whole in 2nd Embodiment. It is a figure for demonstrating the function structure in 2nd Embodiment. It is a figure for demonstrating the Example in 2nd Embodiment. It is a figure for demonstrating the Example in 2nd Embodiment. It is a figure for demonstrating the Example in 2nd Embodiment. It is a figure for demonstrating the Example in 2nd Embodiment. It is a figure for demonstrating the Example in 2nd Embodiment. It is a figure for demonstrating 3rd Embodiment.

  Next, an embodiment when the content display device of the present invention is applied will be described with reference to the drawings. The content display apparatus in the present embodiment will be described by taking an example of an apparatus that receives and displays broadcast content (program) distributed from a transmission source such as a broadcast station.

[1. First Embodiment]
[1.1 General description]
First, the overall description will be described with reference to FIG. FIG. 1 is an external view of a content display device 10 according to the first embodiment. The content display device 10 can display various contents, and may be portable or stationary. Although details will be described later, for example, a display unit configured by an LCD (Liquid Crystal Display) and an audio output unit are included, and the user can select content (for example, broadcast content received from a broadcasting station, network The communication content received from the device).

[1.2 Functional configuration]
Next, the functional configuration of the content display device 10 will be described with reference to FIG. The content display device 10 includes a control unit 100, a content reception unit 110, a demultiplexing unit 115, a capture control unit 120, an information analysis unit 130, a storage unit 140, a trigger input unit 150, and a network communication unit. 160, an output control unit 170, an audio output unit 175, a display unit 180, and an operation unit 185.

  The control unit 100 is a functional unit for controlling the entire content display device 10. The control unit 100 realizes various functions by reading and executing various programs stored in the storage unit 140, and is configured by, for example, a CPU (Central Process Unit).

  The content receiving unit 110 is a functional unit for the content display device 10 to receive content. Here, the content acquired in the present embodiment is broadcast content received from an external broadcasting station via, for example, terrestrial digital broadcasting, digital satellite broadcasting, cable television, or the like.

  The content received by the content receiving unit 110 is not limited to broadcast content, for example, storage unit 140, multimedia content recorded on an external recording medium, IP multicast broadcast received via a network, It may be communication content such as WEB.

  The demultiplexing unit 115 is a functional unit for demultiplexing multiplexed data (stream) such as video data, audio data, character data, and the like from content (for example, received broadcast content). Various data (streams) separated by the demultiplexing unit 115 is output to each functional unit.

  The capture control unit 120 is a functional unit used when executing a capture process described later. The capture control unit 120 captures a still image (group) from video data extracted (separated) from the content by executing a capture process.

  The information analysis unit 130 is a functional unit for executing analysis processing described later. For example, a character string is recognized as an information material by analyzing a still image (group) image, or an object is recognized as an information material. Further, it is possible to perform control so that the category of the character string can be determined and appropriate processing can be executed based on the identified character string and category. That is, the recommendation processing corresponding to the information material is determined by analyzing the input from the user, the coordinates inputted by the instruction, and the like. A specific mechanism of the information analysis unit 130 will be described below with an example.

  (1) When it is determined that the image around the coordinates is a character or URL based on the tap input of the operation unit 185, the character or the like is recognized as an information material. Then, after obtaining the title of the web page indicated by the URL through the network communication unit 160, the recommendation process of jumping to the user is shown.

  (2) When the image on the line is determined to be a character string based on the slide input of the operation unit 185, the character string is recognized as an information material. When the information material is an address, a recommendation process such as a jump to travel information, map information or the like, and when the information material is not an address, a recommendation process such as a jump to a search result using the character is shown.

  (3) The process of “save” is shown as the recommendation process, and when the user selects the process, the format of text, image, video or the like is further shown as the recommendation process.

  (4) When an area designation is input in the form of a circle or rectangle for one still image, a recommendation process such as transmitting an image contained therein to a printer belonging to the home network is added.

  (5) When a tap or region designation is performed on one still image and it is determined that no character is included therein, a jump to a similar image search is indicated as a recommendation process. In addition, when performing a search such as a similar image search, image data on the same coordinates of still images that are temporally adjacent are also used together as search data.

  (6) For example, when region designation is performed for a plurality of still images with different times, when the images included in the region are compared and determined to be the same object, Analyzing the “movement” of the object in the still image at the time, for example, adding a recommendation process that utilizes “cat jump” or the like as an information material (search key).

  (7) When an operation such as a long tap, flick, or flip is performed, an analysis is performed to determine whether information such as characters is included in a part of the image on which the operation is performed. When not extracted, it is determined that the entire video data or the entire content has been designated, and the presence / absence of VOD content including communication content related to the entire content is checked and indicated as recommendation processing.

  The storage unit 140 is a functional unit that stores various programs and various data necessary for the operation of the content display device 10. The storage unit 140 includes, for example, a semiconductor memory, an HDD (Hard Disk Drive), or the like.

  In the storage unit 140, a content data storage area 142, a content work area 144, and a still image group storage area 146 are secured as storage areas.

  The content data storage area 142 is an area in which content received by the content receiving unit 110 is stored. The stored content includes data related to the content, for example, video data, audio data, program information, metadata, and the like for broadcast content.

  The content work area 144 is an area in which displayed content or stored content that is captured as necessary is stored for work (temporary storage). For example, the content for 10 seconds is stored from the received content.

  The still image group storage area 146 is an area in which a still image group generated from the content stored in the content work area 144 is stored. Specifically, a still image group is generated from video data included in work content stored in the content work area 144.

  Here, the still image group is composed of a plurality of still images obtained by capturing a plurality of frame images of video data in principle, but may be composed of a single still image. In this case, when the still image group is composed of a plurality of still images, the accuracy of the recognition process is further increased.

  The trigger input unit 150 is a functional unit for detecting a trigger for executing a later-described capture process in accordance with a user operation. For example, the movement of the content display device 10 may be detected as a trigger by using an acceleration sensor or an illuminance sensor built in the content display device 10. Further, the movement of the user may be detected by an optical sensor, a temperature sensor, an infrared sensor, or the like, and a trigger may be input according to the movement of the user.

  The network communication unit 160 is a functional unit for communicating with other devices. For example, it includes an interface for wired connection such as Ethernet (registered trademark) and USB, and a wireless LAN interface such as IEEE802.11a / b / g / n.

  The output control unit 170 is a functional unit for controlling the output of various functional units. For example, the audio output of the audio output unit 175, the display control of the display unit 180, and the operation input of the operation unit 185 are detected.

  The audio output unit 175 is a functional unit for performing audio output. For example, it is configured by a speaker, headphones, or the like. The display unit 180 is a functional unit for performing various types of display, and includes, for example, an LCD, an organic EL panel, or the like.

  The operation unit 185 is a functional unit for receiving an operation input from the user. In the present embodiment, a touch panel configured integrally with the display unit 180 is assumed. Other types such as a mouse, a voice, and a motion sensor may be used.

[1.3 Process flow]
Subsequently, the flow of processing in the present embodiment will be described with reference to the drawings.

[1.3.1 Capture processing]
First, the capture process will be described with reference to FIG. The capture process is a process executed on the content display device 10 while the content is displayed (for example, while the broadcast content is received from the broadcast station via the content receiving unit 110 and displayed). .

  When the trigger input is detected by the trigger input unit 150 (step S102; Yes), the content is acquired (step S104) and temporarily stored in the content work area 144 (step S106). It should be noted that the received content may be continuously displayed on a part of the display unit during each process after the trigger input is detected.

  In this case, for example, the received content can be continuously displayed in an area where the display of the still image group is not hindered as much as possible, such as a corner in the display unit, or the received content display and the still image group display can be switched by a button, a tab, or the like. It can also be made. In addition, when displaying based on a method including them, recording of the received content is started at the timing of trigger input, or the received content is automatically stored only for a period of time not displayed based on the display content of the user. You can also do it.

  Here, as an example of content, in the case of broadcast content, program information is included as information accompanying the broadcast content program. The program information includes various information such as the title of the program, broadcast time, broadcast station, program content, performers, and keywords. Moreover, various character information, metadata, etc. may be included as accompanying information of broadcast content.

  And the recommendation request | requirement which is a request | requirement which performs a recommendation process based on the content currently acquired from the user is detected (step S108). When the recommendation request is not detected for a predetermined time (step S108; No → step S116; Yes), the content temporarily stored in the content work area 144 is deleted (step S118), and the processing is repeatedly executed from step S102.

  In this case, as a method for detecting the recommendation request, for example, a method for explicitly displaying the recommendation request on the screen with a button, or the presence / absence of the acquired content is displayed in a list form in a part of the display unit for the predetermined time. For example, the user's action on the display area can be determined as a recommendation request.

  That is, content is automatically acquired and stored by trigger input, but a trigger that is not intended for the user may be detected and stored. In this case, since the related content request is not input from the user, the convenience is improved by deleting (erasing) the temporarily stored content.

  On the other hand, when there is a recommendation request from the user (step S108; Yes), video data is extracted from the content temporarily stored in the content work area 144 (step S110). Then, a still image group is generated from the extracted video data (step S112). The generated still image group is stored in the still image group storage area 146, and a predetermined still image is displayed on the display unit 180 (step S114). In this case, for example, the first still image in the still image group may be displayed, or any one of the recorded still images such as the center and the last of the plurality of recorded still image groups may be displayed. A plurality of images in the group may be displayed in time order.

  Note that the content automatically acquired in steps S104 and S106 by the recommendation request from the user is stored so that it can be used in the next analysis processing.

  Note that, depending on a trigger input method (for example, an intentional trigger input method in which a user taps the screen), the process may be directly shifted from step S106 to step S110.

[1.3.2 Analysis processing]
Next, the analysis process will be described. The analysis process is a process executed when input detection from the user is performed on the displayed still image (still image group) after the capture process of FIG.

  When the input from the user is detected (step S202; Yes), it is determined whether the input detection is an instruction operation (step S204). That is, the user can instruct the content display apparatus 10 for various operations such as content switching, display method switching, and content output. When it is determined that the input detection from the user is a predetermined instruction operation (step S204; Yes), a corresponding instruction operation process is executed (step S206).

  On the other hand, when the detected input is not an instruction operation (step S204; No), that is, in the case of an instruction for content, the still image group, the action type, and the coordinates determined from the input detection are determined (step) S208).

  Here, the target still image group from the input detection is information for specifying the still image group corresponding to the still image that is currently input-detected (that is, displayed by the user). The action type is information indicating whether the “designation part” or the “designation area” is entered by the user's input method. Further, “coordinates” corresponding to “designating part” or “designated area” are also acquired from the input detection.

  Here, an outline of a method for determining an action type and coordinates from input detection will be described with reference to FIG. FIG. 5 is a diagram for describing input detection when input is made by a user using a touch panel.

  For example, as shown in FIG. 5A, when a predetermined point is tapped, the tapped coordinate is detected as a designation unit. In the case of FIG. 5A, since (x1, y1) is tapped, the (x1, y1) is detected as the designated portion, and the periphery is detected as the designated portion periphery. For example, an area surrounded by (x1-10, y-10) to (x1 + 10, y1 + 10) is recognized as the designated part periphery. Here, the numerical value of each axis of (x, y) may be a unit of distance such as millimeters, or may be the number of pixels or a value corresponding thereto.

  On the other hand, when the input is detected over a predetermined range as shown in FIG. 5B, it is detected that the designated area is input. In FIG. 5B, tap input is performed so as to draw a circle, and an area surrounded by the input areas (x2, y2) to (x3, y3) is recognized as a designated area.

  Furthermore, not only the coordinate range as described above but also more accurate detection may be performed. For example, pixel chromaticity information is utilized. Specifically, an area of a predetermined range (30 × 30 pixels, 50 × 50 pixels, etc.) is determined around the tapped coordinates, the pixel chromaticities within the area are averaged, and the area is expanded. Detect areas where the degree is almost constant. The designated area is determined by a rectangle inscribed in the detected area.

  Further, contour extraction of the entire still image is performed, and when tapped, brightness information (0 to 255) is first calculated from pixel color information (RGB values) for the entire still image. Then, the image data is binarized with a predetermined threshold. As the threshold value, for example, it may be set to 127 as half of the maximum value, or may be set to the center after calculating the average value of the brightness of the entire still image. Thereafter, it is possible to detect an edge portion from the binary image and find an edge surrounding the tapped coordinate. Then, the designated area is determined by a rectangle inscribed in the area surrounded by the found edge. Furthermore, after contour extraction, it is possible to infer a content object (information material) from the obtained shape.

  Further, face recognition technology may be used. In general, face recognition processing such as pattern matching is performed on the entire still image, the face area is detected, and if the tapped coordinate is within or near the face area, a rectangle that inscribes the face area Confirm the specified area with etc. In the case of pattern matching, since recognition processing is performed with facial parts such as both eyes, nose and mouth, the detected face area is generally smaller than the entire head. After determining the circle inscribed in the result, a method of extracting a rectangle inscribed in the circle may be used.

  In the present embodiment, as a device (means) for detecting input, a touch panel is used. However, other methods may be used. For example, it may be a mouse, a remote controller, or the movement of the user may be detected by a sensor built in the apparatus main body.

  Subsequently, it is determined whether or not the action type is “region designation” (step S210). When the action type is “area designation” (step S210; Yes), it is determined whether character information is included in the designated area (step S220). Here, if character information is included in the designated area, character information processing is executed based on the character information (step S216). On the other hand, if character information is not included, image information processing is executed (step S222). Here, when the image information processing is executed but the recognition result cannot be acquired, the content information processing is executed (step S224; No → step S230).

  On the other hand, when the action type is “designation part” in step S210 (step S210; No), it is determined whether character information is included around the designation part (step S212). If character information is included around the designated portion (step S212; Yes), character information processing is executed (step S216).

  If the character information cannot be acquired as the information material recognition result by executing the character information processing (step S218; No), the content information processing is executed (step S230).

  On the other hand, if character information is not included in the vicinity of the designated portion in step S212, it is determined whether or not an object such as a face, an animal, or an article is contained in the vicinity of the designated portion (step S212; No → step S214). .

  If an object is included around the designated part, the process proceeds from step S222 (image information processing) (step S214; Yes → step S222). However, if no object is included around the designated portion (step S214; No), content information processing is executed (step S230).

  When character information processing is executed and there is an information material recognition result (step S218; Yes), image information processing is executed and there is an information material recognition result (step S224; Yes), or content information processing is executed If so (step 230), a recommendation process is executed based on the information material (step S232).

  The recommendation process is a process that is determined based on the information material and seems to be an appropriate operation for the user. Various methods can be considered as recommendation processing. For example, on a search site, a search result based on information material is displayed, a site based on information material is displayed, or a telephone number based on information material is displayed. It is possible to execute various processes such as sending a call. The recommendation processing for the information material is instructed together with the information material from each processing. Each process will be specifically described below.

(1) Character information processing Character information processing is processing for recognizing a character (string) from an image as an information material. For example, this is a process realized by executing so-called OCR, which recognizes characters from an image in a predetermined area by pattern recognition.

  In the case of the present embodiment, when the character string includes, for example, “http: //” or “〒123”, the information analysis unit 130 performs categorization such as “web address name” and “address”. As the recommendation process, “jump to web page”, “map display process”, and the like are determined. At this time, for example, when metadata such as link information, mailto, and phoneto is pre-assigned to the target character string, processing based on the metadata is preferentially determined as recommendation processing. The

  In particular, in the case of a long character string, the network communication unit 160 uses the external network to search for the character string as it is, extract the highlighted word in the search result page, and search key ( Phrase). Then, using the search key as an information material, “search for XXX” or the like can be used as a recommendation process.

  Even if you do not know the category of the character string, you can search for the character string once from the communication department, and determine the words such as “person name” or “facility name” from the result page. For example, if it is “person name”, a search term “profile” or “image” is added to perform recommendation processing.

  Furthermore, in the case of a long sentence, a character string whose category is unknown, or when it is determined that character information is not included in the vicinity of the designated image, program information, metadata, subtitles, etc. may be included in the captured content. If the accompanying information of the content is stored, the character string of the content-derived information is also used as information material to improve the accuracy of the search process, or the search process is performed only with the content-derived information in advance. It is also possible to preferentially check the matching during character recognition.

  Furthermore, when character recognition is performed from an image, recognition accuracy can be improved by performing similar character recognition processing on the image portion of the same designated area for several still image groups before and after the specified still image. it can. Details of the processing for the still image group will be described later.

(2) Image information processing In the image information processing, an image included in an area selected by the user is used as an information material. That is, when the user input is an area specification that is enclosed by a circle or square, the “similar image search process” is set as a recommendation process.

  Further, when a specific object is recognized as an information material in the enclosed image area, for example, when a face is detected by the object, “person search” or the like is detected, and when furniture or clothes are detected, “product search” is detected. When food or cooking is detected, such as “price search” or “price search”, processes such as “recipe search” and “restaurant search” are set as recommendation processes.

  For example, the information analysis unit 130 can recognize an image and determine that it is a face, or recognize its shape, color, and the like. In this case, the information material includes “face” and “XX color”. ”And the like are added, and the search process is a recommendation process (“ similar face search ”,“ similar color image search ”, etc.).

  In addition, as in the character information processing example, image analysis, object recognition, search processing, etc. can be performed by using accompanying information (program information, character strings of broadcast-derived information) included in the content, and several still image groups. Accuracy and efficiency can be improved.

(3) Content information processing The content information processing is a process executed when a specific character string, an image, or the like is not detected around a designated portion or within a designated area. In particular, when special information is not included in image analysis and object recognition of the user instruction area, it is determined that there is a high possibility that the user instruction is for the entire content, and information associated with the content (for example, information such as EPG) ) On the basis of "related video search (genre, cast, program title, etc.)" Is a recommendation process.

  In particular, when the user's area specification that does not include special information exists for multiple still image groups with different times, there is a possibility that it is a user instruction not only for the entire content but also for scenes and detailed scene units The information analysis unit 130 extracts information such as audio, subtitles, metadata, etc. in the time domain from the time information of the image, and performs processing such as extracting a character string. Processes such as “Search videos by genre or performers”, “BGM search (Search music name from a part of music, etc.)”, “Location search (use place names, building names, etc.)” are recommended processing. .

  Here, although the case where there is one recommendation has been described as the recommendation processing, for example, a case where there are a plurality of recommendations may be assumed. For example, when a plurality of objects are recognized in image information processing, a recommendation list may be displayed and the user may be selected.

  In the above-described processing, for convenience of explanation, character information processing, image information processing, and content information processing have been described, but a plurality of processing may be executed. For example, after the text information processing is executed, the image information processing is executed, and a recommendation list based on the result may be displayed.

[1.4 Use of still images]
[1.4.1 Description of processing]
Here, the use of the still image group in the present embodiment will be described. As described above, the still image group is composed of still images captured (acquired) from video data extracted from content. For example, a still image group is obtained by capturing still images from video data in units of frames and collecting a plurality of them. There may be a case where only one still image group is included (in this case, the still image group is a normal still image) or a plurality of still image groups. This may be determined according to the specification, or may be set in advance by the user. Further, it may be configured to vary depending on the content reception status and the scene switching status.

  Specifically, when the reception status is bad, the content may not be acquired correctly, so a large number of still images are captured from the video data to generate a still image group. In addition, in the case of content where scene switching is intense, an image flows (blurs), so a large number of still images are captured from video data to generate a still image group. Conversely, in the case of content such as a landscape video where the scenes do not change so much, a still image group is generated by reducing the number of still images to be captured. Thereby, the storage amount of the still image group can be adjusted.

  The processing based on the still image group will be described by taking, for example, character information processing as an example. FIG. 6 is a diagram for explaining character information processing for a still image group.

  First, the instruction input image is No. A is detected as a coordinate area (designated coordinates) designated by the user (step S302). That is, as the designated coordinates, the coordinates of the designated part peripheral area or the coordinates of the designated area are stored.

  Subsequently, image character conversion processing is performed (step S304). The image character conversion process is a process of recognizing characters from an image by using pattern recognition, for example. Characters (strings) can be recognized by executing processing such as OCR that has been performed conventionally.

  Subsequently, it is determined whether or not the character string is recognized (step S306). Here, if any character string is recognized, the recognized character string is listed in association with the image No (step S308).

  Subsequently, when there are a plurality of listed character strings, the characters of the character strings are compared to determine a common character and a different character. Then, by extracting common characters, a common character string is recognized from the still image group (step S310). For example, when the first sheet “A”, the second sheet “Λ”, and the third sheet “A” are recognized, the character is recognized as “A”.

  Subsequently, it is determined whether or not the recognized character string can be classified by the categorization process (step S312). Here, the categorization process is a process for determining in which category the recognized character string is classified.

  For example, the URL may be classified into categories such as URLs starting from “http: //”, addresses starting from “〒”, “Tokyo”, “XX prefecture”, etc., and telephone numbers starting from “telephone” and “TEL”. it can. Furthermore, the recognized character string is compared with dictionary data (dictionary data may be stored in advance or may be inquired via a network) to compare whether or not it is a general word. If it can be classified into any category, it is determined as a recognized character string (step S312; Yes → step S314), and the process ends.

  In addition, regarding the categorization process, it may be determined whether the recognized character string is correct according to the category. For example, in the case of starting with “http: //”, if the required length as a URL is not recognized, classification is not possible, and in the case of a telephone number category, the number of digits of the telephone number is recognized. If it is not done, it can be classified.

  On the other hand, when the character string is not recognized (step S306; No) or when the character string cannot be classified by the categorization process (step S312; No), a different still image is designated as a target image from the still image group (step S316). Specifically, the current No. Use n cards before and after A. For example, the current instruction input image (or the image being processed) is processed in a loop one before and one after.

  At this time, the image selected from the still image group recognizes a character string in the same area as the designated coordinate area set in the first image (image No. A) (step S316 → step S318; No → Step S320).

  If the character string cannot be determined and all the still images included in the still image group have been processed, it is determined that the character string could not be determined (recognized), and this processing is terminated. (Step S318; Yes).

[1.4.2 Processing example]
A specific example of processing will be described with reference to the drawings. FIG. 7 is a diagram schematically illustrating a still image group including five still images. In this figure, FIG. 7C is an instruction input image. FIGS. 7B and 7D show one front and rear (frame) still image, and FIGS. 7A and 7E show two front and rear still images.

  First, when the image character conversion process (step S304 in FIG. 6) is executed on the still image in FIG. 7C, the character string is not recognized (step S306; No). Therefore, the image character conversion process is similarly performed on the target images for A-1 (FIG. 7B) and A + 1 (FIG. 7D).

  Then, the character string “http: //○Δ×.jp/” is recognized in FIG. 7B (step S306; Yes). The character string is recognized as “URL”, and the recognized character string correctly represents the character string as the URL. Therefore, the character string “http: //○Δ×.jp/” is determined as the information material (step S312; Yes → step S314).

  Next, FIG. 8 will be described. FIG. 8 is also a diagram schematically showing a still image group including five still images as in FIG.

  First, since a character string is not recognized in FIG. 8C, recognition is performed using FIGS. 8B and 8D. From FIG. 8B, “http: // ◯ △” is recognized. Further, “Δ × .jp /” is recognized from FIG. Here, “http: // ○ △” and “Δ × .jp /” are stored in the character extraction result list.

  Similarly, with respect to FIGS. 8A and 8E, when the image character conversion process is executed, “http: // ◯ △” and “Δ × .jp /” are recognized. Here, although stored in the list of character extraction results, when a common character string is extracted from five still images, “http: //◯△×.jp/” is obtained. Therefore, the character string “http: //○Δ×.jp/” is determined as the information material.

  In this way, the character string can be appropriately recognized by using the still image group. In particular, a still image obtained by capturing a moving image such as video content is extremely effective because a character string may not be properly recognized.

[1.5 Examples]
Next, examples will be described with reference to the drawings.

[1.5. 1 First Example]
First, as a first embodiment, an embodiment in which other related contents (for example, WEB communication contents) are displayed by selecting (tapping) a character displayed in the contents will be described.

  FIG. 9A is a diagram for explaining the operation when video data extracted from the currently received content is captured. That is, FIG. 4 is a diagram illustrating a state in which a part of video data is captured as a result of executing the capture process of FIG. 3. For example, in the region R100, a character display for notifying the user that the capture has been performed is performed.

  In FIG. 9B, a still image (still image group) corresponding to the captured video data is displayed on the screen. Here, the user taps a part of the region R102 in order to select a character displayed in the region R102.

  As a result, the action type is “designation part”, and character information processing is executed because character information is included around the designation part. Then, “http: //○Δ×.jp” is recognized as an information material as a recognition result, and “URL” is detected as a category.

  FIG. 9C shows the recommendation process based on the information material and the category. In FIG. 9, since the category is “URL”, the recognition result WEB page is automatically displayed.

  In this way, when the user taps a character (string) displayed on the screen, the tapped character string is recognized as an information material, and appropriate processing (recommendation processing) based on the information material is executed. Will be. Then, other content desired by the user is displayed.

[1.5. 2 Second Example]
Next, the second embodiment will be described. FIG. 10A is a diagram for explaining an operation when video data extracted from currently received content is captured.

  Here, as shown in FIG. 10B, a tap (slide operation) is input by the user so as to draw a circle. As a result, the action type is recognized as “region designation”, and an image or an object included in the region designation is detected and processed as an information material.

  For example, FIG. 10C shows accompanying information (for example, program information) of the content corresponding to the captured still image group in FIG. Referring to the program information, it is understood that the content (broadcast content) of “Store A” has been received and captured.

  Therefore, in the content information processing (or image information processing), “A store” is extracted as an information material, and “A store” is searched as a recommendation process, and the search result is displayed.

[2. Second Embodiment]
Next, the second embodiment will be described. In the second embodiment, a case will be described in which a content display device and a content reception device that receives content are configured as separate devices and are connected.

[2.1 General description]
FIG. 11 is a diagram in which the content display device 20 and the content receiving device 30 are connected. The content is received by the content receiving device 30. The received content is transferred to the content display device 20 connected by wire or wirelessly and displayed.

[2.2 Functional configuration]
Next, functional configurations of the content display device 20 and the content reception device 30 will be described with reference to FIG. In the content display device 10 of FIG. 2 in the first embodiment, the content receiving device 30 includes a functional unit used to receive content, and the other functional units are the same. Here, the same reference numerals are given to the same functional units as those of the content display device 10 and the content display device 20, and detailed description thereof is omitted.

  The content display device 20 includes a control unit 100, a capture control unit 120, an information analysis unit 130, a storage unit 140, a trigger input unit 150, a network communication unit 160, an output control unit 170, and an audio output unit 175. A display unit 180, an operation unit 185, and a device communication unit 190.

  The device communication unit 190 is a functional unit for communicating with the content receiving device 30. It is connected to the device communication unit 390 of the content receiving device 30 and the content is transferred. Here, the device communication unit 190 is connected to the content receiving device 30 by wireless connection or wired connection.

  As the wireless connection method, for example, near field communication such as infrared data method (IrDA: Infrared Data Association), Bluetooth (registered trademark), ZigBee (registered trademark), NFC (Near Field Communication), or UWB (Ultra Wide Band) is used. Communication (wireless USB, wireless IEEE 1394, etc.), wireless HDMI, or the like can be used. As a wired connection method, for example, USB (Universal Serial Bus), IEEE 1394, HDMI (High-Definition Multimedia Interface), or the like can be used.

  Further, by using the network communication unit 160, for example, it may be connected by a wireless LAN or the like. In this case, for example, in the case of an IP network, it is only necessary to know each other's IP address, and the device communication unit 190 may not be provided.

  The content receiving apparatus 30 includes a control unit 300, a content receiving unit 310, a demultiplexing unit 315, an audio output unit 375, a display unit 380, an operation unit 385, and a device communication unit 390. ing. Here, the content receiving unit 310 is the content receiving unit 110 of FIG. 2, the demultiplexing unit 315 is the demultiplexing unit 115 of FIG. 2, the audio output unit 375 is the audio output unit 175 of FIG. 2, and the display unit 380. 2 and the operation unit 385 are substantially the same functional units as the operation unit 185 in FIG. 2, and detailed description thereof is omitted.

  The control unit 300 is a functional unit for controlling the entire content receiving device 30. The device communication unit 390 is a functional unit that communicates with the content display device 20 and is substantially the same as the device communication unit 190. When the content display device 20 communicates with the content reception device 30 via the network communication unit 160, the content reception device 30 needs a network communication unit corresponding to the network communication unit 160 instead of the device communication unit 390. Become.

[2.3 Process flow]
Subsequently, the flow of the second embodiment will be described. Also in the second embodiment, the basic processing is the same as in the first embodiment.

  The second embodiment differs from the first embodiment in the flow of capture processing. That is, the normal content is received by the content receiving device 30 and displayed on the content receiving device 30.

  In this state, when a trigger input is detected in the content display device 20, a request signal is transmitted from the content display device 20 to the content reception device 30 so as to transmit the content.

  The content receiving device 30 that has received the request signal for transmitting the content transmits (transfers) the content to the content display device 20. The content display device 20 then receives the content (including at least video data and, if necessary, accompanying information) (step S106 in FIG. 3). That is, in step S104, the content receiving device 30 is the destination from which the content is acquired.

  As described above, according to the second embodiment, when the capture processing and the analysis processing are executed based on the content received by the content receiving device 30, for example, the content display device 20 for operation at hand is used. Thus, the capture process can be easily executed.

[2.4 Examples]
Next, examples relating to the second embodiment will be described. First, FIG. 13 is a diagram showing a state in which video data extracted from content is displayed in the content receiving device 30 (FIG. 13A) and a trigger is detected in the content display device 20 (FIG. 13B). It is. In this case, the content is displayed on the content receiving device 30, and the content display device 20 displays a screen for notifying that the capture has started.

  FIG. 14 is a diagram illustrating a state where the capture process is completed. In this case, a still image (group) based on the captured content is displayed in the region R200 of the content display device 20 in FIG. When the face portion of a person displayed in the still image (group) is tapped, the person is used as an information material, and recommendation processing is executed corresponding to the information material. Note that information on a person is selected by tapping the person portion. However, for example, an operation method in which an instruction to select the entire still image is given by long-tapping a still image may be incorporated.

  Next, FIG. 15B shows various selection items displayed based on the object detected from the tapped region. In this case, the recommendation list is displayed based on the information of the object included in the peripheral area tapped in FIG. 14B, the content detected from the image recognition, and the content detected from the accompanying information. Here, for example, FIG. 16B shows a case where “about“ person ○ mountain Δ child ”” is selected by the user. Then, in FIG. 16B, the state where “search with ○ mountain Δ child” is tapped (selected) is shown in FIG. 17B.

  In this way, it is possible to appropriately search for a related item from the portion that the user first tapped (selected), determine the related information as a recommendation process (list), and display it. In the case of the present embodiment, the content receiving device 30 continues to display the content as it is (FIGS. 14A, 15A, 16A, and 17A). However, for example, the content may not be displayed while being transmitted to the content display device 20.

[3. Third Embodiment]
Next, a third embodiment will be described. The third embodiment is an embodiment in which an external device is further connected.

  Here, the content display device has substantially the same configuration as the first embodiment or the second embodiment. The difference from the first embodiment or the second embodiment is that the content display device and the external device can be linked.

  The functional configuration of the third embodiment is shown in FIG. FIG. 18 shows a diagram in which an external device 40 is connected to the content display device 20 of the second embodiment as an example. Here, although the content display apparatus 20 is shown in this figure, if the apparatus communication part 190 is provided, the same thing can be implement | achieved also with the content display apparatus 10 of 1st Embodiment. The external device 40 is assumed to be a variety of devices that cooperate with the content display device 20, and will be described below using a printer and a refrigerator as an example.

[3.1 Printer]
For example, as described in the above-described embodiment, when the user instructs the entire captured still image, the device communication unit 190 of the content display device 20 requests a data to the printer, and the printer status in the device information storage unit 420 (Power on / off, paper amount, etc.) data is acquired. When printing is possible, cooperation such as displaying “print this image” as the recommendation processing is possible.

[3.2 For refrigerators]
When the character recognition process is performed on the area where the tap input is detected or the entire still image, if the food name, the number of cups, the weight, and the like are described, the information is recognized as information on the recipe.

  In that case, the device communication unit 190 of the content display device 20 makes a data request to the external device 40 (refrigerator), and obtains data of the refrigerator state (stock information, etc.) in the device information storage unit 420.

  Then, the information analysis unit 130 on the content display device 20 side compares the image content with the refrigerator inventory. When there are insufficient ingredients, “Refrigeration inventory shortage display” is presented as a recommendation process, and when selected, the recipe and the shortage are displayed. Furthermore, regarding a shortage of ingredients, “search for price information of the ingredients” can be used as a recommendation process, and “search for alternative recipes” that can be made only with inventory can be used as a recommendation process.

  In this way, by cooperating with the external device, various processes can be realized with the content displayed in the content as a trigger.

[4. effect]
By using the content display device of each embodiment described above, the following effects can be obtained.

  It is possible to cope with various user needs such as content acquisition and information utilization while reducing the labor and cost of providing detailed metadata etc. to the broadcast content by the broadcasting station side and the distribution side.

  In addition, since the capture process is operated only when the user needs occur, useless processes and a reduction in recording capacity can be achieved.

  Further, since the user inputs an instruction to the still image group, it is possible to specify an appropriate area without hurrying. In addition, by performing region designation or the like for a plurality of still images in a continuous still image group, it is possible to designate not only an object in the content but also its operation and action.

  In addition, when performing object recognition or the like by image analysis in response to user input for a still image, “Nearby Image Area Information of Continuous Still Image Group”, “EPG Information, Metadata, Subtitles, etc. Included in Broadcast Content” It is possible to utilize information such as “statistical data on the cloud via EPG information and hashtags,” etc., improving the accuracy and efficiency of recognition and responding to more precise user needs. is there.

  Since content analysis and result utilization and action guidance are performed on the receiving side, environment information around the user at the time of reception can also be acquired and utilized. For example, if there is a printer nearby, it is possible to guide to print output, or to compare the recipe information in the content with the inventory in the user's refrigerator.

[5. Modified example]
The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and the design and the like within the scope of the present invention are also within the scope of the claims. include.

  Moreover, you may comprise so that the content receiver 30 in 2nd Embodiment can perform a part or all function among the functions of a content display apparatus. For example, the same functional configuration as that of the content display device 10 in the first embodiment may be included in the content reception device 30 in the second embodiment.

  In the above-described embodiment, an example of an apparatus that receives and displays broadcast content has been described as an example of a content display apparatus. However, as a matter of course, the present invention can be applied to any device that can display content other than the device.

  For example, the content may be multimedia content recorded on a medium such as a DVD or Blu-ray (registered trademark) Disc, or communication content that can be received on a network such as IP multicast broadcasting. Also good. In these cases, even if the content receiving unit 110 is not provided, the present invention is applicable to the content data storage area 142 and the content recorded on the medium.

  Furthermore, although the liquid crystal television has been described as an example in the above-described embodiment, it is needless to say that the present invention can be applied to other content display devices such as a computer, an HDD recorder, a car navigation system, a mobile phone, and a smartphone.

  In addition, the program that operates in each device in the embodiment is a program (a program that causes a computer to function) that controls the CPU and the like so as to realize the functions of the above-described embodiments. Information handled by these devices is temporarily stored in a temporary storage device (for example, RAM) at the time of processing, then stored in various ROM or HDD storage devices, and read and corrected by the CPU as necessary. • Writing is performed.

  Here, as a recording medium for storing the program, a semiconductor medium (for example, ROM, a non-volatile memory card, etc.), an optical recording medium / a magneto-optical recording medium (for example, a DVD (Digital Versatile Disc), an MO ((Magnetoto (Optical Disc), MD (Mini Disc), CD (Compact Disc), BD, etc.), magnetic recording medium (eg, magnetic tape, flexible disk, etc.), etc. Also, the loaded program is executed. Thus, not only the functions of the above-described embodiment are realized, but also the functions of the present invention are realized by processing in cooperation with an operating system or other application programs based on the instructions of the programs. There is also.

  In addition, when distributing to the market, the program can be stored and distributed in a portable recording medium, or transferred to a server computer connected via a network such as the Internet. In this case, of course, the storage device of the server computer is also included in the present invention.

  In addition, a part or all of each device in the above-described embodiments may be realized as an LSI (Large Scale Integration) that is typically an integrated circuit. Each functional block of each device may be individually chipped, or a part or all of them may be integrated into a chip. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when integrated circuit technology that replaces LSI appears due to progress in semiconductor technology, it is of course possible to use an integrated circuit based on this technology.

DESCRIPTION OF SYMBOLS 10 Content display apparatus 100 Control part 110 Content receiving part 115 Demultiplexing part 120 Capture control part 130 Information analysis part 140 Storage part 142 Content storage area 144 Content additional information storage area 146 Still image group storage area 150 Trigger input part 160 Network communication Unit 170 output control unit 175 audio output unit 180 display unit 185 operation unit

Claims (9)

In a content display device capable of displaying content including video data,
Still image group display means for generating or extracting a still image group corresponding to the display content of the input trigger timing from the video data included in the content and displaying it;
Display position designation means capable of designating the display position of the displayed still image group;
Recommendation display means capable of displaying a recommendation based on the information material recognized from the designated display position;
A content display device comprising:   The content display apparatus according to claim 1, wherein the recommendation display unit recognizes a character string included in the designated display position as an information material.   The content display device according to claim 1, wherein the recommendation display unit recognizes an image included in the designated display position as an information material. The content further includes accompanying information,
The content display device according to claim 1, wherein the recommendation display unit recognizes an information material using the accompanying information. In the content receiving apparatus which the content display apparatus as described in any one of Claim 1 to 4 is connected, and receives the content containing video data,
A content receiving apparatus comprising content transmitting means for transmitting content at a trigger timing input to the content receiving apparatus or the content display apparatus to the content display apparatus. In a content display device connected to a content receiving device that receives content including video data and capable of displaying content transferred from the content receiving device,
Content acquisition means for acquiring content from the content reception device based on the input trigger timing;
Still image group display means for generating or extracting a still image group corresponding to the display content at the input trigger timing from the video data included in the acquired content, and displaying it.
Display position designation means capable of designating the display position of the displayed still image group;
Recommendation display means capable of displaying a recommendation based on the information material recognized from the designated display position;
A content display device comprising: In a computer capable of displaying content including video data,
A still image group display function for generating or extracting a still image group corresponding to the display content of the input trigger timing from video data included in the content, and displaying it;
A display position designation function capable of designating the display position of the displayed still image group,
A recommendation display function capable of displaying a recommendation based on the information material recognized from the designated display position;
A program to realize In a content playback apparatus capable of playing back content including video data,
Still image group generation means for generating or extracting a still image group corresponding to the input trigger timing content from the video data included in the content;
Designated position receiving means for receiving an input of a specified position in the still image group;
Recommendation execution means for determining and executing a process for generating a recommendation to the user based on a designated position in the still image group;
A content playback apparatus comprising: 9. The content reproduction apparatus according to claim 8, wherein the recommendation executing unit is determined based on information material extracted from the content based on a designated position in the still image group.