JP2014011592A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus and an image processing method that can set a chapter even for a CM having no accompanying information in a content.SOLUTION: There is provided an image processing apparatus including a reception part which receives a content, a CM detection part which detects CMs from the content, a character data recognition part which recognizes character data from a video of the content, and a CM grouping part which groups the CMs according to outputs of the CM detection parts and character data recognition part. Also there is provided an image processing method including a reception process of receiving the content, a CM detecting process of detecting the CMs from the content, a character data recognizing process of recognizing the character data from the video of the content, and a CM grouping process of grouping the CMs according to results of the CM detecting process and character data recognizing process.

Description

  Embodiments described herein relate generally to an image processing apparatus and an image processing method related to an image processing system using video character information.

  When playing video content that is played continuously, CMs (Commercial Messages) that are broadcast in the middle of a TV broadcast or between programs are short and each corner telop is short. Since there is no accompanying information indicating a delimiter, etc., it is difficult to set chapters, or it is necessary to register only CM information in the CM management server in advance and create a database.

  More specifically, when playing video content that is continuously played back, group the content according to the accompanying information indicating the break such as corner telop and set the chapter (“break” for each scene or scene). The method of reproducing the content can be examined based on the chapter-set list. However, CMs (Commercial Messages) that are broadcast between TV broadcasts or between programs are short in each CM, and there is no accompanying information that indicates a break such as a corner telop. Chapter setting is difficult.

  In Patent Document 1, information related to a CM being broadcast on television is stored, and the purchase of an article related to the CM is supported by the information. The CM information is registered in advance in the CM management server, and the CM When the start of broadcasting is detected, the information on the article related to the CM acquired from the CM management server is displayed, and the information related to the CM needs to be registered in the CM management server in advance.

  In other words, there is a demand for a technology that allows a chapter to be set even in the case of a CM that does not have accompanying information indicating a break such as a corner telop, and allows a CM to be searched without creating a database on a CM management server or the like. However, no means for realizing such a request is known.

JP 2011-71581 A

  An object of the embodiment of the present invention is to provide an image processing apparatus and an image processing apparatus method capable of setting a chapter even in the case of a CM having no accompanying information in the content.

  In order to solve the above-described problem, according to the embodiment, an image processing apparatus includes a receiving unit that receives content, a CM detecting unit that detects a CM from the content, and character data that recognizes character data from the video of the content. A recognition unit; and a CM grouping unit that groups CMs based on outputs from the CM detection unit and the character data recognition unit.

The figure for demonstrating the image processing apparatus concerning one Embodiment. The figure for demonstrating the image processing apparatus concerning one Embodiment. The functional block (system) block diagram which shows one Embodiment. The flowchart which shows the process mainly having the control part of the embodiment. The flowchart shown in order to demonstrate a part of process of FIG.

  Hereinafter, an image processing apparatus and a control method for the image processing apparatus according to an embodiment will be described in detail with reference to the drawings.

  FIG. 1 shows an example of a broadcast receiving apparatus 100 as an image processing apparatus according to an embodiment.

  The broadcast receiving apparatus 100 includes a main body provided with a display (display unit 400) for displaying video, and legs that support the main body in a state where it can stand on its own.

  The broadcast receiving apparatus 100 includes a broadcast input terminal 110, a reception unit 111, a restoration unit 112, a communication interface 114, an audio processing unit 121, a video processing unit 131, a display processing unit 133, a control unit 150, an operation input unit 161, a card. A connector 164, a USB connector 166, a disk drive 170, a LAN connector 171, a power supply control unit 180, and a storage 190 are provided. The broadcast receiving apparatus 100 further includes a speaker 300 and a display 400.

  The broadcast input terminal 110 is an input terminal to which a digital broadcast signal received by the antenna 200 is input, for example. The antenna 200 receives, for example, a terrestrial digital broadcast signal, a BS (broadcasting satellite) digital broadcast signal, and / or a 110-degree CS (communication satellite) digital broadcast signal. That is, content such as a program supplied by a broadcast signal is input to the broadcast input terminal 110.

  The broadcast input terminal 110 supplies the received digital broadcast signal to the receiving unit 111. The receiving unit 111 is a receiving unit for digital broadcast signals. The receiving unit 111 tunes (tunes) the digital broadcast signal supplied from the antenna 200. The receiving unit 111 transmits the tuned digital broadcast signal to the restoration unit 112. When the signal supplied from the transmission input terminal 110 or the communication interface 114 is an analog signal, the reception unit 111 converts the signal into a digital signal.

  The restoration unit 112 demodulates the received digital broadcast signal. Further, the restoration unit 112 performs signal processing on the demodulated digital broadcast signal (content). Accordingly, the restoration unit 112 restores the video signal, the audio signal, and other data signals from the digital broadcast signal. For example, the restoration unit 112 restores a transport stream (TS) on which a video signal, an audio signal, and other data signals are superimposed from a digital broadcast signal.

  The restoration unit 112 supplies an audio signal to the audio processing unit 121. Further, the restoration unit 112 supplies a video signal to the video processing unit 131. Further, the restoration unit signal processing unit 113 supplies a data signal to the control unit 150. That is, the antenna 200, the reception unit 111, and the restoration unit 112 function as a reception unit that receives content.

  The communication interface 114 receives content such as a high definition multimedia interface (HDMI) (registered trademark) terminal, an audio input terminal, an S video terminal, a component video terminal, a D video terminal, a D-Sub terminal, and a DVI-I terminal. Provide any or more of the possible interfaces. The communication interface 114 receives content in which a digital video signal, a digital audio signal, and the like are multiplexed from another device. The communication interface 114 supplies a digital signal (content) received from another device to the receiving unit 111. The communication interface 114 supplies content received from another device to the restoration unit 112. That is, the communication interface 114 functions as a receiving unit that receives content.

  The restoration unit 112 performs signal processing on the content supplied from the communication interface 114 via the reception unit 111. For example, the restoration unit 112 separates the digital signal into a digital video signal, a digital audio signal, and a data signal. The restoration unit 112 supplies a digital audio signal to the audio processing unit 121. The restoration unit 112 also supplies a digital video signal to the video processing unit 131. Further, the restoration unit 112 supplies the control unit 150 with other information regarding the content.

  Furthermore, the restoration unit 112 supplies the content to the storage 190 described later based on the control of the control unit 150. The storage 190 stores the supplied content. Thereby, the broadcast receiving apparatus 100 can record content.

  The audio processing unit 121 converts the digital audio signal received from the restoration unit 112 into a signal (audio signal) in a format that can be reproduced by the speaker 300. The audio processing unit 121 supplies an audio signal to the speaker 300. The speaker 300 reproduces sound based on the supplied audio signal.

  The video processing unit 131 converts the digital video signal received from the restoration unit 112 into a video signal having a format that can be reproduced on the display 400. That is, the video processing unit 131 decodes (reproduces) the video signal received from the restoration unit 112 into a video signal having a format that can be reproduced by the display 400. In addition, the video processing unit 131 superimposes an OSD signal supplied from an OSD processing unit (not shown) on the video signal. The video processing unit 131 outputs the video signal to the display processing unit 133.

  The OSD processing unit displays a GUI (graphic user interface) screen, subtitles, time, or other information based on the data signal supplied from the restoration unit 112 and / or the control signal supplied from the control unit 150. An OSD signal to be displayed in a superimposed manner is generated. The OSD processing unit may be provided individually as a module in the broadcast receiving apparatus 100 or may be provided as a function of the control unit 150.

  For example, the display processing unit 133 performs color tone, brightness, sharpness, contrast, or other image quality adjustment processing on the received video signal based on the control from the control unit 150. The display processing unit 133 supplies the video signal subjected to the image quality adjustment to the display 400. The display 400 displays a video based on the supplied video signal.

  The display 400 includes, for example, a liquid crystal display device including a liquid crystal display panel including a plurality of pixels arranged in a matrix and a backlight that illuminates the liquid crystal panel. The display 400 displays a video based on the video signal supplied from the broadcast receiving device 100.

  The broadcast receiving apparatus 100 may be configured to include a video output terminal instead of the display 400. Moreover, the broadcast receiving apparatus 100 may be configured to include an audio output terminal instead of the speaker 300. In this case, the broadcast receiving apparatus 100 outputs a video signal to a display device connected to the video output terminal, and outputs an audio signal to a speaker connected to the audio output terminal. Thereby, the broadcast receiving apparatus 100 can display an image | video on a display apparatus, and can output an audio | voice from a speaker.

  The control unit 150 functions as a control unit that controls the operation of each unit of the broadcast receiving apparatus 100. The control unit 150 includes a CPU 151, a ROM 152, a RAM 153, an EEPROM 154, and the like. The control unit 150 performs various processes based on the operation signal supplied from the operation input unit 161.

  The CPU 151 includes arithmetic elements that execute various arithmetic processes. The CPU 151 implements various functions by executing programs stored in the ROM 152, the EEPROM 154, or the like.

  The ROM 152 stores a program for controlling the broadcast receiving apparatus 100, a program for realizing various functions, and the like. The CPU 151 activates the program stored in the ROM 152 based on the operation signal supplied from the operation input unit 161. Thereby, the control part 150 controls operation | movement of each part.

  The RAM 153 functions as a work memory for the CPU 151. That is, the RAM 153 stores the calculation result of the CPU 151, the data read by the CPU 151, and the like.

  The EEPROM 154 is a non-volatile memory that stores various setting information, programs, and the like.

  The operation input unit 161 includes, for example, an operation key, a keyboard, a mouse, a voice input device, a touch pad, or another input device that can generate an operation signal according to the operation input. Further, the operation input unit 161 may be configured to include, for example, a sensor that receives an operation signal from a remote controller. Further, the operation input unit 162 may be configured to include the above input device and sensor. That is, the operation input unit 161 functions as an operation signal receiving unit that receives an operation signal.

  The operation input unit 161 supplies the input operation signal to the control unit 150. The control unit 150 causes the broadcast receiving device 100 to perform various processes based on the operation signal supplied from the operation input unit 161.

  The touch pad includes an electrostatic sensor, a thermo sensor, or a device that generates position information based on other methods. When the broadcast receiving device 100 includes the display 400, the operation input unit 161 may include a touch panel formed integrally with the display 400.

  The remote controller generates an operation signal based on a user operation input. The remote controller transmits the generated operation signal to the sensor of the operation input unit 161 by infrared communication. The sensor and the remote controller may be configured to transmit and receive operation signals by other wireless communication such as radio waves.

  The card connector 164 is an interface for communicating with, for example, a memory card 165 that stores moving image content. The card connector 164 reads moving image content data from the connected memory card 165 and supplies it to the control unit 150.

  The USB connector 166 is an interface for communicating with the USB device 167. The USB connector 166 supplies a signal supplied from the connected USB device 167 to the control unit 150.

  For example, when the USB device 167 is an operation input device such as a keyboard, the USB connector 166 receives an operation signal from the USB device 167. The USB connector 166 supplies the received operation signal to the control unit 150. In this case, the control unit 150 executes various processes based on the operation signal supplied from the USB connector 166.

  For example, when the USB device 167 is a storage device that stores content data of a moving image, the USB connector 166 can acquire content from the USB device 167. The USB connector 166 supplies the acquired content to the control unit 150.

  The disc drive 170 can be loaded with, for example, a compact disc (CD), a digital versatile disc (DVD), a Blu-ray Disc (Blu-ray Disk (registered trademark)), or another optical disc M capable of recording moving image content data. Have a drive. The disk drive 170 reads content from the optical disk M to be loaded and supplies the read content to the control unit 150.

  The LAN connector 171 is an interface for connecting the broadcast receiving apparatus 100 to a network. When the LAN connector 171 is connected to a public line by a LAN cable or a wireless LAN, the controller 150 can download and upload various data via the network.

  The power control unit 180 controls power supply to each unit of the broadcast receiving apparatus 100. The power supply control unit 180 receives power from the commercial power supply 500 via an AC adapter, for example. The commercial power supply 500 supplies AC power to the power supply control unit 180. The power supply control unit 180 converts the received AC power into DC and supplies it to each unit.

  Moreover, the broadcast receiving apparatus 100 may further include another interface. The interface is, for example, Serial-ATA. The broadcast receiving apparatus 100 can acquire and play back content recorded in a device connected by an interface. Also, the broadcast receiving apparatus 100 can output the reproduced audio signal and video signal to a device connected by an interface.

  When the broadcast receiving apparatus 100 is connected to a network via an interface, the broadcast receiving apparatus 100 can acquire and reproduce content data of moving images on the network.

  The storage 190 is a storage device that stores content and the like. The storage 190 includes a mass storage device such as a hard disk (HDD), a solid state drive (SSD), or a semiconductor memory. The storage 190 may be configured by a storage device connected to the USB connector 166, the LAN connector 171, the communication interface 114, or another interface.

  As described above, when the content is recorded, the control unit 150 causes the content data demodulated by the restoration unit 112 to be input to the storage 190. Furthermore, the control unit 150 instructs the storage 190 to specify an address in the storage 190 that stores the content. The storage 190 stores the content supplied from the restoration unit 112 at the address designated by the control unit 150.

  The storage 190 may be configured to store a TS restored from a digital broadcast signal, or may be configured to store compressed content in which the TS is compressed by AVI, MPEG, or other compression methods. .

  In addition, the control unit 150 can read and play the content stored in the storage 190. For example, the control unit 150 instructs the storage 190 to specify an address in the storage 190. The storage 190 reads content from the address designated by the control unit 150. The storage 190 supplies the read content to the audio processing unit 121, the video processing unit 131, the control unit 150, and the like. Thereby, the broadcast receiving apparatus 100 can reproduce the recorded content.

  The broadcast receiving apparatus 100 includes a plurality of receiving units 111 and restoring units 112. Thereby, the broadcast receiving apparatus 100 can simultaneously receive a plurality of contents and restore the received plurality of contents at the same time. Thereby, the broadcast receiving apparatus 100 can simultaneously acquire a plurality of reproducible content data. That is, the broadcast receiving apparatus 100 can record a plurality of contents at the same time.

  FIG. 2 shows an example of functions that the control unit 150 has.

  As shown in FIG. 2, the control unit 150 can realize various functions by executing a program by the CPU 151. For example, the control unit 150 includes a character region extraction unit 1511, a character data recognition unit 1512, an error correction unit 1513, a keyword analysis unit 1514, and a storage unit 1515. Further, the control unit 150 includes a search unit 1521, a list display unit 1522, a list selection unit 1523, and a playback unit 1524.

  FIG. 3 shows the system function configuration. This system includes a reception unit 31, a character string detection unit 32, a character data recognition unit 33, a grouping unit 34, a CM detection unit 35, a CM grouping unit 36, a chapter setting list generation unit 37, a storage unit 38, and the like.

  The receiving unit 31 mainly corresponds to the receiving unit 111. The character string detection unit 32 is based on the character region extraction unit 1511. The character data recognition unit 33 mainly corresponds to the character data recognition unit 1512. The grouping unit 34, the CM detection unit 35, the CM grouping unit 36, the chapter setting list generation unit 37, and the storage unit 38 function based on the remaining part of the control unit 150.

  The receiving unit 31 receives moving image content and transmits image frames one by one to the subsequent character string detection unit 32. The character string detection unit 32 analyzes an image in one frame and detects position information of the character string. The character data recognition unit 33 analyzes the image of the position based on the position information of the character and recognizes the character data.

  Based on the character position information and the character data information, the grouping unit 34 identifies whether or not it is the same as the past frame. When the same position information and the same character data continue for a predetermined frame, the frame in which the group is started is set as the chapter start frame and the character data is set as the chapter name.

  The CM detection unit 35 detects whether or not the received moving image content is a CM, and when the CM is detected, notifies the CM grouping unit 36 of the start of processing. The CM grouping unit 36 performs genre estimation and semantic class analysis (classification of character data) in units of 15 seconds using the character data recognized by the character data recognition unit 33 as input data, and the same if a CM chapter already exists. It is determined whether they are CMs. If they are the same, it is determined that they are the same CM (same group). If it is determined that the CMs are not identical, the chapter setting list generation unit 37 sets the frame where the group is started as the chapter start frame and character data as the chapter name.

  The storage unit 38 stores content in a mass storage device such as an HDD as a storage device.

FIG. 4 shows a flowchart of processing in the configuration of FIG.

  When the CM is detected by the CM detection unit 35 and the start of CM is notified, the character data converted by the character data detection unit 33 and the character data recognition processing (step S21) by the character data recognition unit 33 are input data for CM grouping processing. (Step S22), the data are collected in units of 15 seconds (step S23). This is because the CM is divided in multiples of 15 seconds. The CM detection is not particularly defined as long as it can detect the start and end of the CM.

  The input data collected in units of 15 seconds is processed by the CM grouping unit 36. If the end of the CM is not detected, the input data is initialized, and the input data for CM grouping processing is grouped again in units of 15 seconds (step). S24). This process is repeated until the end of CM is detected (step S25).

  FIG. 5 shows a flowchart of the CM grouping process by the CM grouping unit 36.

  In this CM grouping process, genre estimation and semantic class analysis (classification of character data) are performed using input data collected in units of 15 seconds (step S31). As long as the CM can be distinguished, the genre to be classified and the number of genres are not specified. If there is a CM chapter that has already been created (Yes in step S32), if the genres match or the genres are similar (Yes in step S33), they are regarded as the same CM and character data information is stored in the existing CM chapter. Is added (step S36). If the genres do not match or are not similar (No in step S33), the input data semantic classes (character data classification, for example, words frequently used in the sports field, etc.) collected in units of 15 seconds match a certain level (threshold). If so (Yes in step S34), it is regarded as the same CM and character data information is added to the existing CM chapter (step S36). If the input data semantic class (classification of character data) does not match a certain threshold (threshold) (No in step S34), the input data summarized in units of 15 seconds exceeds a certain threshold (threshold, for example, length or frequency). If they match (Yes in step S35), it is regarded as the same CM, and character data information is added to the existing CM chapter (step S36).

  By determining whether or not the CMs are the same in a stepwise manner as described above, it is possible to determine a break in the case of a CM without a break such as a corner telop. If it is not determined to be the same CM (No in step S35), a different CM is set, the frame in which the group is started in the chapter setting list generation unit 37 is the start frame of the chapter, and the character data most frequently included in the input data is the chapter name. (Step S37). The chapter name is not limited to the character data contained most in the input data, but may be character data having a large character size. When the chapter is set and the listing is performed, the CM chapter is initialized (step S38), and the CM grouping process is terminated. If there is no CM chapter that has already been created (No in step S32), a new CM chapter is created and the process is terminated (step S39).

  This embodiment allows you to set chapters even in the case of CMs that do not have accompanying information that indicates a break such as a corner telop, allowing you to search for CMs in the same way as video content without creating a database on the CM management server. It becomes.

  In the present embodiment, CMs are also handled in the same way as moving image content, and a chapter setting is listed so that a CM search can be performed. For example, when CM is started, video character information in a frame is collected in units of 15 seconds, genre estimation and semantic class analysis (character data classification) are performed, and genre estimation results and semantic class analysis results of preceding and following character data , Compare the character data step by step to determine if it is the same CM, and set the CM chapter.

In addition, this invention is not limited to the said embodiment, In the range which does not deviate from the summary, it can implement in various modifications.

  Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements according to different embodiments may be appropriately combined.

  DESCRIPTION OF SYMBOLS 100 ... Broadcast receiving apparatus, 110 ... Broadcast input terminal, 111 ... Reception part, 112 ... Restoration part, 114 ... Communication interface, 121 ... Audio processing part, 131 ... Video processing part, 133 ... Display processing part, 150 ... Control part, 151 ... CPU, 152 ... ROM, 153 ... RAM, 154 ... EEPROM, 161 ... operation input unit, 164 ... card connector, 165 ... memory card, 166 ... USB connector, 167 ... USB device, 170 ... disk drive, 171 ... LAN Connector, 180 ... Power supply control unit, 190 ... Storage, 200 ... Antenna, 300 ... Speaker, 400 ... Display, 500 ... Commercial power supply, 1511 ... Character area extraction unit, 1512 ... Character data recognition unit, 1513 ... Error correction unit, 1514 ... Keyword analysis unit, 1514a ... Keyword dictionary, 151 ... storage unit, 1521 ... search unit, 1522 ... list display portion, 1523 ... list selecting unit, 1524 ... reproduction unit.

Claims (5)

A receiving unit for receiving content;
A CM detection unit for detecting a CM from the content;
A character data recognition unit for recognizing character data from the video of the content;
An image processing apparatus comprising: a CM grouping unit that groups CMs based on outputs from the CM detection unit and the character data recognition unit.   The image processing apparatus according to claim 1, further comprising a chapter setting list generation unit that generates a chapter setting list based on an output of the CM grouping unit.   The image processing apparatus according to claim 2, further comprising a grouping unit that groups frames of the content based on an output of a character data recognition unit, wherein the chapter setting list generation unit generates the chapter setting list based on the grouping. .   The image processing apparatus according to claim 1, further comprising a character string detection unit that detects a character string from the content video, wherein the character data recognition unit recognizes the character data based on an output of the character string detection unit. A receiving process for receiving content;
A CM detection step of detecting a CM from the content;
A character data recognition step for recognizing character data from the video of the content;
An image processing method including a CM grouping step of grouping CMs based on the results of the CM detection step and the character data recognition step.

Images