WO2024007861A1 - 接收装置及元数据生成系统 - Google Patents

接收装置及元数据生成系统 Download PDF

Info

Publication number
WO2024007861A1
WO2024007861A1 PCT/CN2023/101699 CN2023101699W WO2024007861A1 WO 2024007861 A1 WO2024007861 A1 WO 2024007861A1 CN 2023101699 W CN2023101699 W CN 2023101699W WO 2024007861 A1 WO2024007861 A1 WO 2024007861A1
Authority
WO
WIPO (PCT)
Prior art keywords
metadata
advertisement
data
broadcast
unit
Prior art date
Application number
PCT/CN2023/101699
Other languages
English (en)
French (fr)
Inventor
柴田诚
Original Assignee
海信视像科技股份有限公司
东芝视频解决方案株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司, 东芝视频解决方案株式会社 filed Critical 海信视像科技股份有限公司
Priority to CN202380013680.8A priority Critical patent/CN118020308A/zh
Publication of WO2024007861A1 publication Critical patent/WO2024007861A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H20/00—Arrangements for broadcast or for distribution combined with broadcast
    • H04H20/28—Arrangements for simultaneous broadcast of plural pieces of information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H40/00—Arrangements specially adapted for receiving broadcast information
    • H04H40/18—Arrangements characterised by circuits or components specially adapted for receiving
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/68—Systems specially adapted for using specific information, e.g. geographical or meteorological information
    • H04H60/73—Systems specially adapted for using specific information, e.g. geographical or meteorological information using meta-information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/437—Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server

Definitions

  • Embodiments of the present application relate to a receiving device and a metadata generating system.
  • Metadata Metadata
  • AI Artificial Intelligence
  • Patent Document 1 Japanese Patent Application Publication No. 2006-108984
  • Patent Document 2 Japanese Patent Application Publication No. 2006-109126
  • Patent Document 3 Japanese Patent Application Publication No. 2011-008676
  • An object to be solved by this application is to provide a receiving device and a metadata generation system that can efficiently process broadcast program information extracted on a per-frame basis using limited system resources.
  • a receiving device is a receiving device that receives a broadcast program and provides it for live viewing.
  • the receiving device includes a data processing unit.
  • the data processing unit A broadcast signal generates conversion data capable of generating metadata representing the contents of the broadcast program; and a first transceiver unit transmits the conversion data to a server device that generates the metadata.
  • FIG. 1 is a diagram showing an example of the structure of a metadata generation system according to the embodiment
  • FIG. 2 is a block diagram showing an example of the hardware configuration of the server device according to the embodiment
  • FIG. 3 is a block diagram showing an example of the functional structure of the server device according to the embodiment.
  • FIG. 4 is a diagram showing an example of the hardware structure of the television device according to the embodiment.
  • FIG. 5 is a block diagram showing an example of the functional structure of the television device according to the embodiment.
  • FIG. 6 is a schematic diagram showing an example of how the metadata generation system according to the embodiment generates metadata
  • FIG. 7 is a schematic diagram illustrating an example in which the metadata generation system according to the embodiment determines an insertion position of an advertisement
  • FIG. 8 is a schematic diagram showing an example of task allocation in the television device according to the embodiment.
  • FIG. 9 is a schematic diagram illustrating an example of allocation of remaining resources to tasks of pre-processing of data conversion in the television device according to the embodiment.
  • FIG. 10 is a schematic diagram illustrating an example of allocation of remaining resources to tasks of pre-processing of data conversion in the television device according to the embodiment
  • FIG. 11 is a flowchart showing an example of the procedure of metadata generation processing in the metadata generation system according to the embodiment.
  • 1... metadata generation system 10... server device, 11... transmission and reception department, 12... integration department, 13... advertisement determination department, 14... metadata generation department, 15... storage department, 20... Television device, 21...transmitting and receiving section, 22...task allocation section, 23...data processing section, 24...broadcast receiving section, 29...storage section.
  • FIG. 1 is a diagram showing an example of the structure of the metadata generation system 1 according to the embodiment.
  • the metadata generation system 1 includes a server device 10 and a plurality of television devices 20 (20a, 20b, 20c...20n: where n is an arbitrary integer).
  • This metadata generation system uses the server device 10 and the television The cooperation of the device 20 can generate metadata representing the content of the broadcast program.
  • the server device 10 and the plurality of television devices 20 are connected to each other wirelessly or wiredly through a network 30 such as the Internet.
  • the network 30 may be, for example, a home network based on DLNA (Digital Living Network Alliance) (registered trademark), a home LAN (Local Area Network), or the like.
  • the television device 20 as a receiving device can receive broadcast signals from broadcast stations and receive various broadcast programs, for example.
  • the television device 20 may provide the user with the received broadcast program through live viewing, or may record and play the recorded broadcast program.
  • the television device 20 can generate conversion data based on the broadcast signal of the broadcast program, and the conversion data can generate metadata including scene information of the broadcast program and the like.
  • the server device 10 is configured as a cloud server placed on a cloud, for example.
  • the server device 10 may also be configured as one or more physical structures including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). computer.
  • a CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • the server device 10 receives the conversion data converted by each television device 20 from these television devices 20 and generates metadata.
  • the server device 10 provides the generated metadata to each television device 20 .
  • FIGS. 2 and 3 a structural example of the server device 10 according to the embodiment will be described using FIGS. 2 and 3 .
  • FIG. 2 is a block diagram showing an example of the hardware configuration of the server device 10 according to the embodiment.
  • the server device 10 includes a CPU 101, a ROM 102, a RAM 103, a communication I/F (interface) 104, an input/output I/F 105, an input device 151, a display device 152, and a storage device 106.
  • the CPU 101 controls the entire server device 10 .
  • the ROM 102 serves as a storage area in the server device 10 Function. Even if the power of the server device 10 is cut off, the information stored in the ROM 102 is maintained.
  • the RAM 103 functions as a disposable storage device and becomes the work area of the CPU 101 .
  • the CPU 101 expands the control program and the like stored in the ROM 102 into the RAM 103 and executes it, thereby obtaining the function of the server device 10 that generates metadata based on conversion data collected from a plurality of television devices 20.
  • control program can be recorded on various computer-readable storage media such as a floppy disk, CD-R, DVD (Digital Versatile Disk), Blu-ray Disc (registered trademark), semiconductor memory, etc. and provided.
  • control program may be stored in a computer connected to a network such as the Internet, and may be provided by downloading from the network.
  • control program may be provided or distributed through a network such as the Internet.
  • the communication I/F 104 can be connected to a network 30 such as the Internet, for example. Through the communication I/F 104, various information can be sent and received between the server device 10 and the plurality of television devices 20.
  • the input/output I/F 105 can also be connected to an input device 151 such as a keyboard and a mouse, and a display device 152 such as a monitor. This allows, for example, the administrator of the server device 10 to perform various operations on the server device 10 .
  • the storage device 106 is an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and functions as an auxiliary storage device for the CPU 101.
  • HDD Hard Disk Drive
  • SSD Solid State Drive
  • FIG. 3 is a block diagram showing an example of the functional configuration of the server device 10 according to the embodiment.
  • the server device 10 includes a transmission and reception unit 11 , an integration unit 12 , an advertisement determination unit 13 , a metadata generation unit 14 , and a storage unit 15 .
  • the above-mentioned functional structure of the server device 10 can be realized, for example, by the CPU 101 that executes the control program, or the hardware structure of each part of the server device 10 that operates under the control of the CPU 101 as shown in FIG. 2 .
  • the transmission and reception unit 11 as the second transmission and reception unit can transmit and receive data between the plurality of television devices 20 and the server device 10 .
  • the transmission and reception unit 11 receives, for example, conversion data generated by the television devices 20 based on broadcast signals of broadcast programs from a plurality of television devices 20 . Furthermore, the transmission and reception unit 11 transmits the metadata generated by the server device 10 to the plurality of television devices 20 .
  • the integration unit 12 integrates the conversion data generated for each frame by the plurality of television devices 20 into time-series data arranged in time series. In this case, the integration unit 12 selects and selects the conversion data collected from the plurality of television devices 20 so as to obtain time-series data for all broadcast programs being broadcast by the plurality of broadcasting stations in a predetermined time period.
  • the integration unit 12 may generate time series data of one broadcast program using only the conversion data collected from the television device 20 .
  • the time of one broadcast program can also be generated using conversion data collected from multiple television devices 20 in this situation. sequence data.
  • the advertisement determination unit 13 determines the insertion position of the advertisement based on the time series data.
  • the conversion data from the television device 20 contains information that the television device 20 infers regarding the insertion location of the advertisement.
  • the advertisement determination unit 13 refers to the estimation information estimated by the television device 20 and determines the insertion position of the advertisement.
  • the metadata generation unit 14 generates metadata indicating the content of the broadcast program, such as scene information, based on conversion data other than the insertion position of the advertisement, that is, conversion data generated from the main part of the broadcast program.
  • the storage unit 15 stores various parameters, control programs, and the like required for the operation of the server device 10 .
  • the storage unit 15 may also store conversion data collected from a plurality of television devices 20 , time series data generated from the conversion data, information on specified insertion positions of advertisements, metadata generated based on the conversion data, and the like. .
  • FIGS. 4 and 5 a structural example of the television device 20 according to the embodiment will be described using FIGS. 4 and 5 .
  • FIG. 4 is a diagram showing an example of the hardware configuration of the television device 20 according to the embodiment.
  • the television device 20 includes an antenna 201, input terminals 202a to 202c, a tuner 203, a demodulator 204, a demultiplexer 205, an A/D (analog/digital) converter 206, a selector 207, Signal processing unit 208, speaker 209, display panel 210, operation unit 211, light receiving unit 212, IP communication unit 213, CPU 214, memory 215, and storage 216.
  • A/D analog/digital converter
  • the antenna 201 receives a broadcast signal of digital broadcast and supplies the received broadcast signal to the tuner 203 via the input terminal 202a.
  • the tuner 203 selects a broadcast signal of a desired channel from the broadcast signals supplied from the antenna 201 and supplies the selected broadcast signal to the demodulator 204 .
  • the demodulator 204 demodulates the broadcast signal supplied from the tuner 203, and supplies the demodulated broadcast signal to the demultiplexer 205.
  • the demultiplexer 205 separates the broadcast signal supplied from the demodulator 204 to generate a video signal and an audio signal, and supplies the generated video signal and audio signal to the selector 207 .
  • the selector 207 selects one of the plurality of signals supplied from the demultiplexer 205, the A/D converter 206, and the input terminal 202c, and supplies the selected signal to the signal processing unit 208.
  • the signal processing unit 208 performs predetermined signal processing on the video signal supplied from the selector 207 and supplies the processed video signal to the display panel 210 . Furthermore, the signal processing unit 208 performs predetermined signal processing on the audio signal supplied from the selector 207 and supplies the processed audio signal to the speaker 209 .
  • the speaker 209 outputs speech or various sounds based on the sound signal supplied from the signal processing unit 208 .
  • the speaker 209 changes the volume of the output voice or various sounds based on the control of the CPU 214.
  • the display panel 210 displays videos such as still images and dynamic images, other images, text information, etc. based on the video signal supplied from the signal processing unit 208 or the control of the CPU 214.
  • the input terminal 202b receives analog signals such as video signals and audio signals input from the outside.
  • the input terminal 202c receives digital signals such as video signals and audio signals input from the outside.
  • the input terminal 202c can input a digital signal from a recorder equipped with a drive device that drives a storage medium for recording and playback such as BD (Blu-ray Disc) (registered trademark) to perform recording and playback.
  • BD Blu-ray Disc
  • the A/D converter 206 supplies a digital signal generated by A/D conversion of the analog signal supplied from the input terminal 202 b to the selector 207 .
  • the operation unit 211 receives the user's operation input.
  • the light receiving unit 212 receives infrared rays from the remote control 219 .
  • the IP communication unit 213 is a communication interface for performing IP (Internet Protocol) communication via the network 30 .
  • the television device 20 may be connected to a network other than the Internet such as LAN, and may be connected to the above-mentioned server device 10 via such a network so that various information can be sent and received.
  • the CPU 214 controls the entire television device 20.
  • the memory 215 is a ROM that stores various computer programs executed by the CPU 214, a RAM that provides a work area for the CPU 214, etc.
  • the ROM stores control programs and application programs for realizing various functions of the television device 20 .
  • the memory 216 is HDD (Hard Disk Drive, hard disk drive) or SSD (Solid State Drive, solid state drive). hard drive) etc.
  • the memory 216 stores the signal selected by the selector 207 as recording data, for example.
  • FIG. 5 is a block diagram showing an example of the functional configuration of the television device 20 according to the embodiment.
  • the television device 20 includes a transmission and reception unit 21, a task allocation unit 22, a data processing unit 23, a broadcast receiving unit 24, an operation receiving unit 25, a live viewing processing unit 26, a recording processing unit 27, a playback processing unit 28, and Storage unit 29.
  • the above-mentioned functional structure of the television device 20 can be realized by, for example, the CPU 214 executing the control program or the hardware structure of each part of the television device 20 shown in FIG. 4 operating under the control of the CPU 214.
  • the broadcast receiving unit 24 receives a broadcast signal of a broadcast program transmitted from a broadcast station.
  • the broadcast signal includes video information and audio information, and program arrangement information (SI: Service Information) indicating the content of the broadcast program is multiplexed in consideration of convenience in program selection.
  • SI Service Information
  • program arrangement information there is information related to an Electronic Program Guide (EPG: Electronic Program Guide) including information on a TV column equivalent to news.
  • the video information, sound information and accompanying transmission control information as described above constitute a transport stream (TS: Transport Stream) that is compressed into MPEG 2 format and multiplexed.
  • TS Transport Stream
  • the broadcast receiving unit 24 can receive multiplexed video information, audio information, program arrangement information, etc. in the broadcast signal.
  • the operation receiving unit 25 receives various operations from the user, such as live viewing operations, recording operations, recording reservation operations, and playback operations.
  • the live viewing processing unit 26 performs live viewing processing of the broadcast program.
  • Live viewing means, for example, real-time display of a broadcast program being broadcast at that time.
  • the recording processing unit 27 executes the process of recording the broadcast program based on the recording operation and recording reservation operation from the user.
  • the recording processing unit 27 stores the recorded program in the storage unit 29 .
  • the playback processing unit 28 reads the recorded program from the storage unit 29 and performs playback processing.
  • the data processing unit 23 converts a broadcast signal of a broadcast program that the television device 20 is currently providing for real-time viewing into a format in which the above-mentioned server device 10 can generate metadata. More specifically, the data processing unit 23 converts the data based on the broadcast signal of the broadcast program being provided for live viewing into a multi-dimensional array for each frame, and further generates an estimation result including the content of the broadcast program estimated from the multi-dimensional array. conversion data within.
  • an estimated start position and an estimated end position of an advertisement for distinguishing the main article from an advertisement there are, for example, an estimated start position and an estimated end position of an advertisement for distinguishing the main article from an advertisement, information identifying a performer in a broadcast program, and the like.
  • the task allocation unit 22 allocates tasks indicating the contents of the processing.
  • tasks required for data conversion may differ depending on the content of the broadcast program or each frame.
  • the task allocation unit 22 refers to, for example, program arrangement information included in the broadcast signal, appropriately determines necessary tasks, and allocates them to the data processing unit 23 .
  • the transmission and reception unit 21 as the first transmission and reception unit can transmit and receive data between the television device 20 and the server device 10 .
  • the transmitting and receiving unit 21 transmits the conversion data generated by the television device 20 based on the broadcast signal of the broadcast program to the server device 10 .
  • the transmission and reception unit 21 receives the metadata generated by the server device 10 .
  • the storage unit 29 stores various parameters, control programs, and the like required for the operation of the television device 20 .
  • the storage unit 29 may store recorded programs, conversion data generated by the data processing unit 23, metadata received from the server device 10, and the like.
  • FIG. 6 is a schematic diagram showing an example of how the metadata generation system 1 according to the embodiment generates metadata. It should be noted that in FIG. 6 , time is set to pass from left to right on the paper.
  • the data processing unit 23 captures the viewing screen of each frame, and obtains a multi-dimensional arrangement from the captured viewing screen IM. More specifically, the data processing section 23 converts the transport stream of the broadcast program into a multi-dimensional arrangement based on floating point values or integer values.
  • a multidimensional arrangement is a multi-column arrangement that uses the concept of a matrix to store multiple variables.
  • the viewing screen IM captured by the data processing unit 23 as an example
  • the task allocation unit 22 selects the start estimation position and the end estimation position of the advertisement as tasks, in the case of terrestrial digital broadcasting, it has (3 ⁇ 1440 ⁇ 1080) pixels. number.
  • the output can be set to (1 ⁇ 576) pixels or (1 ⁇ 5) pixels, as an example.
  • these specific numerical values related to input and output are just an example. In any case, by using multi-dimensional arrangement, the number of arrangement elements of the original viewing picture IM can be greatly reduced.
  • DNN Deep Neural Network
  • the data processing unit 23 generates conversion data including estimation results of differences between the main part of a broadcast program and commercials, performers in the broadcast program, and other scene information using, for example, the DNN technology described above.
  • the transmission and reception section 21 of the television device 20 transmits the generated conversion data of each frame to the server device 10 through the network 30 together with the program arrangement information multiplexed in the received broadcast signal.
  • the conversion data is uploaded from the television device 20 to the server device 10 periodically, for example, every five minutes.
  • the integration unit 12 of the server device 10 selects and selects the conversion data of each frame collected from the plurality of television devices 20, and generates time-series data for all plurality of broadcast programs broadcast simultaneously, for example.
  • the time series data generation process is also performed periodically, for example, every five minutes, in accordance with the timing of data upload from the plurality of television devices 20 .
  • the advertisement determination unit 13 determines the insertion position of the advertisement based on the time series data. Furthermore, the metadata generation unit 14 refers to the program arrangement information added to the converted data, and generates metadata indicating the program content for the main part of the broadcast program.
  • the metadata generated by the metadata generation unit 14 includes, for example, metadata indicating the difference between the main part of a broadcast program and an advertisement.
  • the metadata generation unit 14 generates, in association with the time corresponding to the time series data: metadata representing the advertisement part in association with the time series data corresponding to the insertion position of the advertisement determined by the advertisement determination unit 13; or metadata indicating the advertisement part except the advertisement.
  • the time series data corresponding to the part other than the insertion position associatively represents the metadata of the main part.
  • the metadata generation unit 14 may omit the above-mentioned processing when there is no broadcast program with advertisements like NHK.
  • the metadata generation unit 14 refers to the program arrangement information, and specifies, for example, the respective singing times of singer A, singer B, singer C, etc., together with the time series data. Metadata including these singer names, etc. are generated in association with the corresponding moments.
  • the metadata generation unit 14 refers to the program arrangement information and determines, for example, the performance times of artist A, artist B, artist C, etc., and corresponds to the time series data. Metadata including these artist names, etc. are generated in association with time.
  • the metadata generation unit 14 refers to the program arrangement information, and determines, for example, the time when the opening or ending theme song is broadcast, and associates it with the time corresponding to the time series data. Generate metadata representing parts of the theme song.
  • the metadata generation unit 14 refers to the program arrangement information, and determines, for example, the broadcast time from the studio (studio), the time of input rebroadcast, etc., and the time series data. Metadata indicating the broadcast part from the studio or metadata indicating the relay part is generated in association with the time.
  • the transmission and reception unit 11 of the server device 10 transmits the metadata generated as described above to the plurality of television devices 20 .
  • the metadata may be distributed to all the television devices 20 connected to the server device 10, or may be sent to the television device 20 in which the request exists among the plurality of television devices 20.
  • each television device 20 when playing a recorded program, etc., corresponding metadata is displayed based on the user's operation. Thereby, the user can refer to the metadata and effectively watch the recorded program.
  • the playback of a recorded program includes not only playing the recorded program at any time after the program ends, but also including playing the recorded content retroactively during live viewing without waiting for the end of the recording while the program is being broadcast.
  • This playback method is also called time shift playback, etc.
  • FIG. 7 is a schematic diagram showing an example of how the metadata generation system 1 according to the embodiment determines an insertion position of an advertisement. It should be noted that in FIG. 7 , time is set to pass from left to right on the paper.
  • the data processing unit 23 of the television device 20 adds information on the estimated start position and the estimated end position of the advertisement using, for example, DNN technology, based on a multidimensional array in which data based on the broadcast signal of the broadcast program is converted. .
  • the information on the inference start position and the inference end position may also include the accuracy of these inferences.
  • the advertisement determination unit 13 determines the insertion position of the advertisement in the time series data.
  • information on the estimated start position of the advertisement (accuracy: 80%) is included at the beginning of the time series data ((1) in FIG. 7 ).
  • time series data does not include information on the estimated end position of the advertisement beyond the prescribed period.
  • the advertisement determination unit 13 determines that the estimation result of the estimated start position of the advertisement at the beginning of the time series data is wrong, and the advertisement is not started at that time.
  • the broadcast time of advertisements is often standardized to a maximum of 1 minute in units of 15 seconds. Therefore, the predetermined period used by the advertisement determination unit 13 for determination may be set to, for example, one minute.
  • the subsequent time series data includes information on the estimated start positions of the two advertisements (accuracy: 90%, accuracy: 85%) ((2) and (3) of FIG. 7 ) .
  • the advertisement determination unit 13 adopts a combination whose advertisement insertion time is close to a multiple of the shortest broadcast period of the advertisement among the combinations of each of the two estimated start positions and one estimated end position.
  • the broadcast time of the advertisement is standardized to increase in units of 15 seconds.
  • the insertion time of the advertisement becomes 67 seconds.
  • the insertion time of the advertisement becomes 60 seconds.
  • the advertisement determination unit 13 adopts the slower estimated start position among the two estimated start positions as the start position of the advertisement.
  • the advertisement determination unit 13 may also add the level of accuracy to the determination criterion in addition to determining whether it is close to a multiple of the shortest broadcast period of the advertisement.
  • FIG. 8 is a schematic diagram showing an example of task allocation in the television device 20 according to the embodiment. It should be noted that in FIG. 8 , time is set to pass from the upper part to the lower part of the paper.
  • the tasks required for data conversion may differ for each content or frame of the broadcast program.
  • facial authentication of performing singers and performers may be performed to determine the performance time of each performer.
  • it is a news program or the like, such processing is usually not required.
  • processing may be performed to determine the estimated start position and the estimated end position of the advertisement.
  • processing is not required.
  • the capacity of the captured viewing screen may be adjusted in some cases.
  • tasks 1 to 7 represent tasks that may differ depending on the content of the broadcast program or each frame.
  • the processing of tasks 1 to 3 is a task that can be executed with relatively few resources.
  • the processing of tasks 4 to 7 requires many resources.
  • the task allocation unit 22 refers to the program arrangement information at that time and determines the tasks required for generating conversion data based on the viewing screen IM captured by the data processing unit 23 . More specifically, the task allocation unit 22 selects, for example, task 1 from a plurality of candidates for tasks 1 to 3 and allocates it to the data processing unit 23 .
  • the processing of Task 1 to Task 3 including Task 1 can be executed with relatively few resources.
  • the task allocation unit 22 determines that there are remaining resources in the data processing unit 23 .
  • the task allocation unit 22 refers to the program arrangement information at that time and determines the tasks required for generating conversion data based on the viewing screen IM captured by the data processing unit 23 .
  • the task allocation unit 22 selects, for example, task 1 and task 2 from a plurality of candidates for tasks 1 to 3 and allocates them to the data processing unit 23 .
  • the data processing unit 23 has two processes of Task 1 and Task 2.
  • the task allocation unit 22 determines that the data processing unit 23 has no remaining resources and cannot process other tasks.
  • the data processing unit 23 further captures another viewing screen IM ((3) in FIG. 8) at the time in the lowest stage of FIG. 8.
  • the task allocation unit 22 refers to the program arrangement information at that time, and based on the data processing
  • the viewing image IM captured by the unit 23 is used to determine the tasks required for generating the conversion data.
  • the task allocation unit 22 selects, for example, task 1 from a plurality of candidates for tasks 1 to 3 and allocates it to the data processing unit 23 .
  • the task allocation unit 22 selects, for example, task 5 from a plurality of candidates for tasks 4 to 7 and allocates it to the data processing unit 23 .
  • task 5 is a process of adjusting the viewing screen IM to the viewing screen im. Such processing requires relatively more resources.
  • the task allocation unit 22 also determines that the data processing unit 23 has no remaining resources and cannot process other tasks.
  • the conversion data of each frame including the estimation results through these tasks is uploaded to the server device 10 .
  • FIGS. 9 and 10 are schematic diagrams showing an example of allocation of remaining resources to tasks of pre-processing of data conversion in the television device 20 according to the embodiment.
  • a preprocessing of data conversion a process of extracting feature values of performers of a predetermined TV drama is performed in advance.
  • the data processing unit 23 of the television device 20 may perform processing such as detecting the performance time of a predetermined performer in these broadcast programs. At this time, for example, face authentication using DNN technology is performed to identify each performer and determine the time when these performers appear on the viewing screen.
  • facial authentication of each performer can be quickly performed based on the extracted feature amounts during the data conversion process.
  • the task allocation unit 22 of the television device 20 extracts the facial features of the performer of the predetermined TV drama, for example.
  • a large number of tasks are assigned to the data processing unit 23. Facial photos and the like of each performer used for extracting facial feature amounts may be stored in, for example, the server device 10 .
  • the administrator of the server device 10 or the like may store facial photograph data obtained by the performer of the TV drama in advance in the storage unit 15 of the server device 10 .
  • the television device 20 may autonomously acquire facial photos of performers included in program arrangement information and the like and store them in the storage unit 29 .
  • full permission may be obtained in advance regarding the use of the program arrangement information in the metadata generation system 1 including the television device 20 .
  • the data processing unit 23 of the television device 20 extracts the facial feature amount of each performer from the provided facial photograph of the performer and stores it in, for example, the storage unit 29 . Then, when conversion data is generated for the TV series performed by these performers, each performer can be identified with reference to the feature amounts stored in the storage unit 29 . Its status is shown in Figure 10.
  • the data processing unit 23 converts the information included in the viewing screen captured for each frame into a multi-dimensional array.
  • the data processing unit 23 performs analysis based on the generated multi-dimensional array and extracts the facial feature amount of the performer included in the viewing screen.
  • the data processing unit 23 reads the facial feature amount data of each performer stored in the storage unit 29 and compares it with the facial feature amount of the performer included in the viewing screen. Confirm to watch the screen Performers included.
  • the television device 20 may transmit the generated facial feature amount of the performer to the server device 10 .
  • the server device 10 may distribute the feature amounts generated by a predetermined television device 20 to other television devices 20 so that when each television device 20 performs data conversion, the feature amounts generated by one television device 20 may be shared among these television devices 20 .
  • FIG. 11 is a flowchart showing an example of the procedure of metadata generation processing performed by the metadata generation system 1 according to the embodiment.
  • the task allocation unit 22 of the television device 20 allocates a task to the data processing unit 23 (step S101 ).
  • the task allocation unit 22 determines the tasks required for data conversion according to the content of the broadcast program or each frame while referring to the program arrangement information. In addition, the task allocation unit 22 allocates the determined tasks to the data processing unit 23 .
  • the task allocation unit 22 determines whether there are any remaining resources in the data processing unit 23 based on the specified tasks (step S102).
  • step S102 When it is determined that no remaining resources are generated (step S102: No), the data processing unit 23 executes the processing of steps S103 to S105 based on the assigned task. When it is determined that surplus resources are generated (step S102: YES), the data processing unit 23 further allocates tasks corresponding to the remaining resources to the data processing unit 23 (step S107).
  • the data processing unit 23 captures the viewing screen of the broadcast program that has been started for each frame (step S103). Furthermore, the data processing unit 23 converts the information included in the viewing screen into a multidimensional array (step S104). In addition, the data processing unit 23 performs various estimations based on the multidimensional arrangement and generates estimation results (step S105).
  • the data processing unit 23 simultaneously performs the processing of steps S103 to S105 and executes processing based on the allocated task (step S107).
  • the integration unit 12 selects and selects the conversion data collected from the plurality of television devices 20, integrates the plurality of conversion data related to each broadcast program, and generates time series data corresponding to each of these broadcast programs (step S112).
  • the advertisement determination unit 13 refers to the program arrangement information corresponding to the generated time series data, and determines whether there is insertion of advertisements in the time series data (step S113).
  • the advertisement determination unit 13 determines that the advertisement is not inserted when the broadcast program corresponding to the time series data is an NHK broadcast program, and determines that the advertisement is not inserted when the broadcast program is a private broadcast program. There are advertisements inserted.
  • the advertisement determination unit 13 may perform the above process based on whether the data processing unit 23 of the television device 20 assigns the estimated start position and the estimated end position of the advertisement to the time series data instead of the program arrangement information, or in addition to the program arrangement information. determination.
  • step S113 determines the insertion position of the advertisement based on the estimated start position and the estimated end position of the advertisement given in the time series data (step S114 ).
  • step S114 determines the advertisement determination unit 13 skips the process of step S114.
  • the metadata generation unit 14 generates a metadata representation of the broadcast program based on the time series data and the estimation result of the data processing unit 23 given to the time series data for the main part excluding the insertion position of the advertisement in the time series data. Metadata of the content (step S115).
  • the transmitting and receiving unit 11 transmits the metadata generated as described above to the television device 20 (step S116).
  • the transceiver unit 21 of the television device 20 receives metadata from the server device 10 (step S121), and stores the metadata in the storage unit 29 so that it can be displayed when watching a recorded program, for example (step S122).
  • the metadata generation process performed by the metadata generation system 1 of the embodiment is completed.
  • the server device automatically generates metadata
  • the cost on the server device side becomes huge in order to process a large amount of content in the server device.
  • a logo image (Logo image), a pattern file (Pattern file), etc. are prepared and distributed in the cloud in advance, so that the television device can detect the advertising interval.
  • a logo image (Logo image), a pattern file (Pattern file), etc.
  • the television device is required to detect the commercial interval, there is a concern about processing delays.
  • a terminal device in the medical field generates a feature value using artificial intelligence in real time, and uses the feature value to execute a process using time-series data.
  • conversion data capable of generating metadata representing the content of the broadcast program is generated based on the broadcast signal of the broadcast program.
  • each television device 20 can be responsible for processing a huge content, for example, unlike the case where it is processed together with the server device 10 , thereby reducing the load on the server device 10 .
  • the above-described data conversion processing can be performed using the existing hardware structure of the television device 20, for example. There is no need to add a new structure to the television set 20 or to reconstruct the television set 20 itself, so that component costs and running costs can be suppressed.
  • the converted data is sent to the server device 10 that generates metadata.
  • the server device 10 that generates metadata.
  • data processed by the server device 10 is frequently updated, so that For example, this can also cope with time-shift playback that requires a short processing time.
  • the information of the broadcast program is converted into a multi-dimensional array for each frame to generate conversion data.
  • the information of the broadcast program is converted into a multi-dimensional array for each frame to generate conversion data.
  • huge data can be processed, and for example, by using a multi-dimensional array in the input and output data of a DNN, data analysis using the DNN technology becomes easier.
  • the capacity of processing data in the server device 10 can be reduced, and more information of contents can be collected and processed.
  • tasks required for data conversion are determined based on the program arrangement information, and the determined tasks are assigned to the data processing unit 23 . Accordingly, for example, in data conversion processing that requires different tasks depending on the content of the broadcast program or each frame, an appropriate task can be assigned to the data processing unit 23 and executed. In addition, the limited resources of the data processing section 23 can be effectively utilized.
  • the television device 20 of the embodiment when there are remaining resources in the data processing unit 23 , a task that can be used for pre-processing of data conversion is allocated to the data processing unit 23 . As a result, the resources of the data processing unit 23 can be effectively utilized.
  • metadata is generated based on the conversion data converted by the television device 20 .
  • server device 10 This allows the server device 10 to be responsible for generating metadata that requires processing of time-series data, thereby preventing obstacles in the real-time processing of viewing, recording, and playback of the television device 20 .
  • server device 10 only needs to execute the generation process of metadata that mainly processes time series data, and the load on the server device 10 can also be reduced.
  • the server device 10 of this embodiment when the predetermined period has elapsed from the estimated start position of the advertisement based on the data processing unit 23 and the information of the estimated end position is not included, it is determined that the advertisement is not inserted at the estimated start position. As described above, by performing the above determination in the server device 10 capable of processing time series data, the insertion position of the advertisement can be determined with high accuracy.
  • the advertisement when another inferred start position or other inferred end position is included between the inferred start position and the inferred end position of the advertisement, the advertisement is selected from the combinations of these inferred start positions and inferred end positions.
  • the combination of multiples of the shortest broadcast time closest to the advertisement during the insertion period is used as the start position and end position of the advertisement.
  • the server device 10 of the embodiment among the plurality of television devices 20 , conversion data collected from at least the television device 20 that is providing live viewing of the broadcast program is integrated into time series data. As described above, by dispersing the data conversion processing in each of the plurality of television devices 20 , the burden on the television device 20 can be further reduced.
  • the server device 10 collects and integrates conversion data from a plurality of television devices 20 that provide live viewing, thereby generating time series data.
  • the service provider that generates metadata may install a plurality of television devices 20 in its own company or the like, and upload the conversion data from these television devices 20 to the server device 10 .
  • the receiving device is the television device 20 in the above-described embodiment, the structure of the present embodiment is not limited to this.
  • the receiving device may be another device such as a personal computer, a smartphone, a tablet, a mobile phone, or the like that has a broadcast signal reception function, a broadcast signal projection function, and a voice recognition service function.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请涉及接收装置及元数据生成系统。使用有限的系统资源有效地进行按照每个帧提取出的广播节目的信息的处理。接收装置是接收广播节目并以可直播收看的方式提供的接收装置，其具备：数据处理部，在提供广播节目的直播收看时，数据处理部根据广播节目的广播信号生成转换数据，该转换数据能够生成表示广播节目的内容的元数据；以及第一收发部，其将转换数据发送给生成元数据的服务器装置。

Description

接收装置及元数据生成系统
相关申请的交叉引用
本申请要求在2022年7月8日提交日本专利局、申请号为2022-110677、发明名称为“接收装置及元数据生成系统”的日本专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请的实施方式涉及接收装置及元数据生成系统。
背景技术
为了实现有效地收看各种内容，与以场景信息为首的内容相关的元数据(Meta data)的可用性受到关注。在电视的广播节目中，到目前为止人工生成元数据是主流，但近年来开始尝试使用人工智能(AI：Artificial Intelligence)自动生成元数据。
在先技术文献
专利文献
专利文献1：日本特开2006-108984号公报
专利文献2：日本特开2006-109126号公报
专利文献3：日本特开2011-008676号公报
发明内容
然而，由于元数据的自动生成需要庞大的处理，所以问题在于如何用有限的系统资源对例如按照每帧提取的广播节目的信息进行处理。
本申请所要解决的课题是提供一种接收装置以及元数据生成系统，其能够用有限的系统资源对按照每帧提取的广播节目的信息有效地进行处理。
本申请实施方式的接收装置是接收广播节目并且可直播收看地提供的接收装置，其具备：数据处理部，在提供所述广播节目的直播收看时，所述数据处理部根据所述广播节目的广播信号生成转换数据，该转换数据能够生成表示所述广播节目的内容的元数据；以及第一收发部，其将所述转换数据发送给生成所述元数据的服务器装置。
附图说明
图1是表示实施方式所涉及的元数据生成系统的结构的一例的图；
图2是表示实施方式所涉及的服务器装置的硬件结构的一例的框图；
图3是表示实施方式所涉及的服务器装置的功能结构的一例的框图；
图4是表示实施方式所涉及的电视装置的硬件结构的一例的图；
图5是表示实施方式所涉及的电视装置的功能结构的一例的框图；
图6是表示实施方式所涉及的元数据生成系统生成元数据的情况的一例的示意图；
图7是表示实施方式所涉及的元数据生成系统确定广告的插入位置的情况的一例的示意图；
图8是表示在实施方式所涉及的电视装置中进行任务(task)分配的情况的一例的示意图；
图9是表示在实施方式所涉及的电视装置中剩余资源对数据转换的前处理的任务进行分配的情况的一例的示意图；
图10是表示在实施方式所涉及的电视装置中剩余资源对数据转换的前处理的任务进行分配的情况的一例的示意图；
图11是表示在实施方式所涉及的元数据生成系统中元数据生成处理的顺序的一例的流程图。
附图标记说明
1……元数据生成系统、10……服务器装置、11……收发部、12……整合部、13……广告判定部、14……元数据生成部、15……存储部、20……电视装置、21……收发部、22……任务分配部、23……数据处理部、24……广播接收部、29……存储部。
具体实施方式
(元数据生成系统的结构)
图1是表示实施方式所涉及的元数据生成系统1的结构的一例的图。如图1所示，元数据生成系统1具备服务器装置10和多个电视装置20(20a、20b、20c……20n：其中n是任意的整数)，该元数据生成系统通过服务器装置10和电视装置20的协作而能够生成表示广播节目的内容的元数据。
服务器装置10和多个电视装置20通过例如因特网等网络30而无线或有线地相互连接。网络30例如可以是基于DLNA(Digital Living Network Alliance)(注册商标)的家庭网络、或家庭内LAN(Local Area Network)等。
作为接收装置的电视装置20例如可以从广播电台接收广播信号而接收各种广播节目。此外，电视装置20可以将接收到的广播节目通过直播收看而提供给用户，或者可以进行录像并且将录像的广播节目进行播放。
电视装置20在将接收到的广播节目提供给用户的过程中，能够根据该广播节目的广播信号而生成转换数据，该转换数据能够生成包含广播节目的场景信息等在内的元数据。
服务器装置10例如构成为放置在云上的云服务器等。服务器装置10还可以构成为，具备CPU(Central Processing Unit，中央处理器)、ROM(Read Only Memory，只读存储器)、以及RAM(Random Access Memory，随机存取存储器)等物理结构的一个以上的计算机。
服务器装置10从这些电视装置20接收各个电视装置20转换的转换数据并生成元数据。服务器装置10将生成的元数据提供给各个电视装置20。
(服务器装置的结构例)
接着，使用图2和图3对实施方式的服务器装置10的结构例进行说明。
图2是表示实施方式所涉及的服务器装置10的硬件结构的一例的框图。如图2所示，服务器装置10具备CPU 101、ROM 102、RAM 103、通信I/F(接口)104、输入输出I/F 105、输入装置151、显示装置152、以及存储装置106。
CPU 101控制服务器装置10的整体。ROM 102作为服务器装置10中的保存区域而 发挥功能。即使服务器装置10的电源被切断，存储在ROM 102中的信息也被保持。RAM 103作为一次性存储装置发挥功能，成为CPU 101的作业区域。
例如，CPU 101将存储在ROM 102中的控制程序等展开到RAM 103中并执行，从而获得基于从多个电视装置20收集的转换数据而生成元数据的服务器装置10的功能。
另外，上述控制程序可以被记录于软盘、CD-R、DVD(Digital Versatile Disk，数字光盘)、蓝光光盘(注册商标)、半导体存储器等、计算机可读取的各种存储介质中而被提供。
此外，还可以构成为，将控制程序存储于连接到互联网等网络的计算机上，通过网络下载而被提供。此外，还可以构成为，通过互联网等网络提供或发布控制程序。
通信I/F 104例如能够与互联网等网络30连接。通过通信I/F 104，能够在服务器装置10与多个电视装置20之间收发各种信息。
输入输出I/F 105上还可以连接键盘、鼠标等输入装置151、以及监视器等显示装置152。由此，例如服务器装置10的管理者等能够对服务器装置10进行各种操作。
存储装置106为HDD(Hard Disk Drive，硬盘驱动器)、SSD(Solid State Drive，固态硬盘)等，作为CPU 101的辅助存储装置发挥功能。
图3是表示实施方式所涉及的服务器装置10的功能结构的一例的框图。如图3所示，服务器装置10具备收发部11、整合部12、广告判定部13、元数据生成部14、以及存储部15。
服务器装置10的上述功能结构例如可以通过执行控制程序的CPU 101、或在CPU 101的控制下工作的服务器装置10的图2所示的各部的硬件结构来实现。
作为第二收发部的收发部11能够在多个电视装置20与服务器装置10之间收发数据。收发部11例如从多个电视装置20接收由电视装置20根据广播节目的广播信号而生成的转换数据。此外，收发部11将服务器装置10生成的元数据发送给多个电视装置20。
整合部12将由多个电视装置20按照每个帧生成的转换数据整合为按时间序列排列的时间序列数据。在这种情况下，整合部12取舍选择从多个电视装置20收集到的转换数据，使得对多个广播电台在规定的时间段广播中的全部广播节目而获得时间序列数据。
例如，某一个电视装置20的用户一贯收看一个广播电台的广播节目的情况下，整合部12可以只使用从该一个电视装置20收集到的转换数据来生成一个广播节目的时间序列数据。
或者，当某一个电视装置20的用户反复选台并且收看多个广播节目的情况下，也可以使用从处于这种状况下的多个电视装置20收集到的转换数据来生成一个广播节目的时间序列数据。
广告判定部13根据时间序列数据而判定广告的插入位置。来自电视装置20的转换数据包含电视装置20推断出的关于广告的插入位置的信息。广告判定部13参照由该电视装置20推断出的推断信息，确定广告的插入位置。
元数据生成部14根据除了广告的插入位置以外的转换数据、即从广播节目的正篇部分生成的转换数据，生成场景信息等、表示该广播节目的内容的元数据。
存储部15中存储有服务器装置10的工作中需要的各种参数和控制程序等。此外，存储部15中还可以存储有从多个电视装置20收集的转换数据、从转换数据生成的时间序列数据、确定出的广告的插入位置的信息、以及根据转换数据而生成的元数据等。
(电视装置的结构例)
接着，使用图4和图5对实施方式的电视装置20的结构例进行说明。
图4为表示实施方式所涉及的电视装置20的硬件结构的一例的图。
如图4所示，电视装置20具备天线201、输入端子202a～202c、调谐器203、解调器204、解复用器205、A/D(模拟/数字)转换器206、选择器207、信号处理部208、扬声器209、显示面板210、操作部211、受光部212、IP通信部213、CPU 214、内存(memory)215、以及存储器216。
天线201接收数字广播的广播信号，将接收到的广播信号经由输入端子202a供给到调谐器203。
调谐器203从由天线201供给的广播信号中对期望的频道的广播信号进行选台，将选台的广播信号供给到解调器204。
解调器204解调从调谐器203供给的广播信号，将解调过的广播信号供给到解复用器205。
解复用器205分离从解调器204供给的广播信号并生成视像信号和声音信号，将生成的视像信号和声音信号供给到选择器207。
选择器207从由解复用器205、A/D转换器206、以及输入端子202c供给的多个信号中选择一个，将选择的一个信号供给到信号处理部208。
信号处理部208对从选择器207供给的视像信号实施规定的信号处理，将处理后的视像信号供给到显示面板210。此外，信号处理部208对从选择器207供给的声音信号实施规定的信号处理，将处理后的声音信号供给到扬声器209。
扬声器209基于从信号处理部208供给的声音信号而输出语音或各种声音。此外，扬声器209基于CPU 214的控制，变更输出的语音或各种声音的音量。
显示面板210基于从信号处理部208供给的视像信号或CPU 214的控制，显示静态图像和动态图像等视像、其它图像、以及文字信息等。
输入端子202b接收从外部输入的视像信号和声音信号等模拟信号。此外，输入端子202c接收从外部输入的视像信号和声音信号等数字信号。例如，输入端子202c可以从搭载了驱动装置的录像机(Recorder)等输入数字信号，该驱动装置驱动BD(Blu-ray Disc)(注册商标)等用于录像播放的存储介质而进行录像和播放。
A/D转换器206向选择器207供给数字信号，该数字信号是通过对从输入端子202b供给的模拟信号实施A/D转换而生成的信号。
操作部211接收用户的操作输入。
受光部212从遥控器219接收红外线。
IP通信部213是用于进行经由网络30的IP(互联网协议)通信的通信接口。其中，电视装置20可以连接于不同于LAN等互联网的网络上，还可以以经由这种网络而可收发各种信息地连接于上述的服务器装置10。
CPU 214控制电视装置20整体。
内存215是存储CPU 214执行的各种计算机程序的ROM、以及为CPU 214提供工作分区(area)的RAM等。例如，ROM中存储有用于实现电视装置20的各种功能的控制程序以及应用程序等。
存储器216是HDD(Hard Disk Drive，硬盘驱动器)或SSD(Solid State Drive，固态 硬盘)等。存储器216例如将通过选择器207选择出的信号作为录像数据进行存储。
图5是表示实施方式所涉及的电视装置20的功能结构的一例的框图。如图5所示，电视装置20具备收发部21、任务分配部22、数据处理部23、广播接收部24、操作接收部25、直播收看处理部26、录像处理部27、播放处理部28以及存储部29。
电视装置20的上述功能结构可以通过例如执行控制程序中的CPU 214或CPU 214的控制下工作的电视装置20的图4中表示的各部的硬件结构实现。
广播接收部24接收从广播电台发送的广播节目的广播信号。广播信号包括视像信息和声音信息，并且考虑到节目选择的便利性等，表示该广播节目的内容的节目排列信息(SI：Service Information)被多路复用。作为节目排列信息的一例，例如有包含相当于新闻的电视栏的信息在内的电子节目指南(EPG：Electronic Program Guide)相关的信息。
如上所述的视像信息、声音信息以及其附带的传输控制信息等构成被压缩为MPEG 2形式且多路复用的传输流(TS：Transport Stream)。
广播接收部24可以接收广播信号中多路复用的视像信息、声音信息以及节目排列信息等。
操作接收部25接收来自用户的直播收看操作、录像操作、录像预约操作以及播放操作等各种操作。
直播收看处理部26进行广播节目的直播收看的处理。直播收看是指例如实时显示该时刻正在广播的广播节目。
录像处理部27根据来自用户的录像操作和录像预约操作来执行广播节目的录像的处理。录像处理部27将录像节目存储在存储部29中。
根据来自用户的播放操作，播放处理部28从存储部29读取录像节目并进行播放处理。
数据处理部23例如将电视装置20此时正在提供实时收看的广播节目的广播信号转换为上述的服务器装置10能够生成元数据的格式。更具体而言，数据处理部23将基于正在提供直播收看的广播节目的广播信号的数据按照每个帧转换为多维排列，并进一步生成包含从多维排列中估计出广播节目的内容的估计结果在内的转换数据。
作为节目内容的估计结果，例如存在用于对正篇和广告进行区分的广告的推断开始位置和推断结束位置、确定了广播节目中的表演者的信息等。
当数据处理部23进行数据转换处理时，任务分配部22对表示这些处理内容的任务进行分配。当数据处理部23进行数据转换处理时，数据转换所需的任务有时针对广播节目的内容或每个帧而不同。任务分配部22参照例如广播信号中包含的节目排列信息等，适当地确定需要的任务，分配给数据处理部23。
作为第一收发部的收发部21能够在电视装置20与服务器装置10之间收发数据。收发部21将由电视装置20根据广播节目的广播信号而生成的转换数据发送给服务器装置10。此外，收发部21接收服务器装置10生成的元数据。
存储部29中存储有电视装置20的工作中所需的各种参数和控制程序等。此外，存储部29中可以存储有录像节目、数据处理部23生成的转换数据、以及从服务器装置10接收的元数据等。
另外，关于将电视装置20的资源应用于服务器装置10中的元数据的生成处理中的情况，设置为电视装置20的用户接受许可。
(元数据的生成例)
接着，使用图6对实施方式的服务器装置10和电视装置20的元数据的生成例进行说明。
图6是表示实施方式所涉及的元数据生成系统1生成元数据的情况的一例的示意图。需要说明的是，在图6中设置为时间从纸面左朝向右经过。
如图6所示，对于电视装置20提供直播收看的广播节目，数据处理部23例如捕捉每帧的收看画面，从捕捉到的收看画面IM得到多维排列。更具体而言，数据处理部23将广播节目的传输流转换成基于浮点值或整数值的多维排列。多维排列是使用矩阵的概念存储多个变量的多列排列。
通过使用多维排列可以处理大量的数据。例如，以由数据处理部23捕捉到的收看画面IM为例，下面举出具体例。对于原始的收看画面IM，若假设由任务分配部22选定广告的开始推断位置和结束推断位置作为任务的情况，则在地面数字广播的情况下，具有(3×1440×1080)像素的像素数。通过将其转换为多维排列，作为一例，可以将输出设为(1×576)像素或(1×5)像素。但是，关于这些与输入输出相关的具体数值只不过是一个例子。无论如何，通过使用多维排列，可以大大减少原始的收看画面IM的排列元素的数量。
因此，作为向深度神经网络(DNN：Deep Neural Network)等的输入输出数据而使用多维排列，从而基于DNN的图像识别和声音识别等变得容易。此外，如后文所述，通过转换成多维排列，能够缩小服务器装置10中的处理数据的容量，进而能够收集更多的内容信息并进行处理。
数据处理部23例如使用上述的DNN技术等生成包含估计结果的转换数据，所述估计结果是对广播节目的正篇与广告的区别、广播节目中的表演者、其它场景信息进行估计的结果。
电视装置20的收发部21通过网络30将所生成的每个帧的转换数据和在接收到的广播信号中多路复用的节目排列信息一起发送到服务器装置10。从电视装置20向服务器装置10的转换数据的上传例如按照每5分钟等定期进行。
服务器装置10的整合部12取舍选择从多个电视装置20收集的每个帧的转换数据，例如对同时广播的所有多个广播节目，生成时间序列数据。时间序列数据的生成处理也与来自多个电视装置20的数据上传的时刻匹配地，例如按照每5分钟等定期进行。
如上所述，广告判定部13根据时间序列数据确定广告的插入位置。此外，元数据生成部14参照附加到转换数据的节目排列信息，针对广播节目的正篇部分而生成表示节目内容的元数据。
作为元数据生成部14生成的元数据，例如有表示广播节目的正篇与广告的区别的元数据。
例如，元数据生成部14以与时间序列数据对应的时刻关联地生成：与由广告判定部13确定的广告的插入位置对应的时间序列数据关联地表示广告部分的元数据；或者与除了广告的插入位置以外的部分对应的时间序列数据关联地表示正篇部分的元数据。
或者，元数据生成部14根据节目排列信息，在如NHK那样没有插入广告的广播节目的情况下，可以省略上述处理。
此外，例如当元数据的生成对象的广播节目为音乐节目时，元数据生成部14参照节目排列信息，例如确定歌手A、歌手B、歌手C……的各个演唱时刻等，与时间序列数据 的对应时刻关联地生成包含这些歌手名等在内的元数据。
此外，例如当元数据的生成对象的广播节目为联合演出时，元数据生成部14参照节目排列信息，例如确定艺人A、艺人B、艺人C……的各个演出时刻等，与时间序列数据对应的时刻关联地生成包含这些艺人名等的元数据。
此外，例如当元数据的生成对象的广播节目是动画片或电视剧时，元数据生成部14参照节目排列信息，例如确定开场或结尾的主题曲播出的时刻，与时间序列数据对应的时刻关联地生成表示主题曲部分的元数据。
此外，例如当元数据的生成对象的广播节目为新闻节目时，元数据生成部14参照节目排列信息，例如确定来自演播室(studio)的广播时刻、输入转播的时刻等，与时间序列数据的对应时刻关联地生成表示来自演播室的广播部分的元数据或表示来自中继部分的元数据。
服务器装置10的收发部11将如上所述生成的元数据发送给多个电视装置20。这时，元数据可以分发给与服务器装置10连接的全部电视装置20，或者可以发送给多个电视装置20中存在请求的电视装置20。
在各个电视装置20中，当播放录像节目时等，根据用户的操作，显示对应的元数据。由此，用户可以参照元数据而有效地收看录像节目。
另外，录像节目的播放除了包括在节目结束后的任意时刻播放录像节目的情况，还包括正在广播节目时不等待录像的结束而在直播收看的途中追溯过去而播放录像内容的情况。这种播放方法还称为时移(Time shift)播放等。
(广告插入位置的确定例)
接着，使用图7对元数据生成系统1中的广告的插入位置的确定例进行说明。
图7是表示实施方式所涉及的元数据生成系统1确定广告的插入位置的情况的一个例子的示意图。需要说明的是，在图7中设置为时间从纸面左朝向右经过。
如图7所示，电视装置20的数据处理部23根据例如将基于广播节目的广播信号的数据进行了转换的多维排列，例如使用DNN技术等而附加广告的推断开始位置和推断结束位置的信息。在推断开始位置和推断结束位置的信息中，也可以包含这些推断的准确度。
在接收到包含这些信息在内的转换数据的服务器装置10中，广告判定部13确定时间序列数据中的广告的插入位置。
在图7的例子中，例如在时间序列数据的开头中包含广告的推断开始位置(准确度：80％)的信息(图7的(1))。但是，在时间序列数据中，此后超过规定期间，不包含广告的推断结束位置的信息。在这种情况下，广告判定部13判定为：时间序列数据开头的广告的推断开始位置的估计结果是错误的，在该时刻没有开始广告。
另外，广告的广播时间通常标准化为以15秒为单位且最长1分钟等的情况较多。因此，可以将广告判定部13用于判定的规定期间设定为例如1分钟等。
此外，在图7的例子中，此后的时间序列数据中包含两个广告的推断开始位置的信息(准确度：90％、准确度：85％)(图7的(2)、(3))。与此相对，被认为与该推断开始位置对应的广告的推断结束位置的信息只有一个(准确度：80％)(图7的(4))。在这种情况下，广告判定部13在两个推断开始位置中的每一个与一个推断结束位置的组合中，采用其广告插入时间接近广告的最短广播期间的倍数的组合。
即，如上所述，广告的广播时间被标准化为以15秒为单位而增加。在图7的例子中， 如果组合两个推断开始位置中较早的推断开始位置(图7的(2))和其后的推断结束位置，则广告的插入时间成为67秒。另一方面，如果组合两个推断开始位置中较慢的推断开始位置(图7的(3))和其后的推断结束位置，则广告的插入时间成为60秒。
如上所述，在较慢的推断开始位置和其后的推断结束位置的组合中，广告的广播时间更接近15秒的倍数。因此，广告判定部13采用两个推断开始位置中较慢的推断开始位置作为广告的开始位置。
另外，在本例中最终采用两个推断开始位置中准确度低的位置。但是，广告判定部13也可以在判定基准中，除了判定是否接近广告的最短广播期间的倍数，还可以添加准确度的高低。
(任务的分配例)
接着，使用图8对实施方式的电视装置20中的任务的分配例进行说明。
图8是表示在实施方式所涉及的电视装置20中进行任务的分配的情况的一个例子的示意图。需要说明的是，在图8中设置为时间从纸面上部朝向下部而经过。
如上所述，当数据处理部23对规定的广播节目进行数据转换时，对于广播节目的内容或每个帧，有时数据转换所需的任务不同。
例如，在歌曲节目或电视剧等中，如上所述，有时进行演出歌手和表演者的面部认证等，从而进行确定各个表演者的演出时刻的处理。另一方面，如果是新闻节目等，则通常不需要这样的处理。
此外，例如如果是民办广播节目，则有时进行确定广告的推断开始位置和推断结束位置的处理。另一方面，如果是NHK的广播节目，则不需要这样的处理。
此外，例如如果是高清广播等，则有时也进行调整捕捉到的收看画面的容量的处理等。
在图8的例子中，任务1～任务7表示按照广播节目的内容或每个帧而可能不同的任务。此外，在这些任务1～任务7中，任务1～任务3的处理是能够以比较少的资源来执行的任务。另一方面，在这些任务1～任务7中，任务4～任务7的处理是需要较多资源的任务。
例如，在图8的最上段的时刻中，设为数据处理部23正在捕捉规定的收看画面IM(图8的(1))。任务分配部22参照该时刻中的节目排列信息，并基于由数据处理部23捕捉到的收看画面IM而确定用于生成转换数据所需的任务。更具体而言，任务分配部22从多个任务1～任务3的候补中选择例如任务1，分配给数据处理部23。
如上所述，包含任务1在内的任务1～任务3的处理都可以用比较少的资源来执行。在这种情况下，任务分配部22判定为数据处理部23中存在剩余的资源。
此外，例如在图8的中段的时刻中，设为数据处理部23正在捕捉其它的收看画面IM(图8的(2))。任务分配部22参照该时刻中的节目排列信息，并基于数据处理部23捕捉到的收看画面IM来确定用于生成转换数据所需的任务。在图8的例子中，任务分配部22从多个任务1～任务3的候补中选择例如任务1和任务2，分配给数据处理部23。
这时，虽然一个一个任务1、任务2、任务3的处理可以用比较少的资源来执行，但数据处理部23具有任务1和任务2这两个处理。在这种情况下，任务分配部22判定为数据处理部23不具有剩余的资源，不能再进行其他任务的处理。
此外，例如在图8的最下段的时刻中，设为数据处理部23进一步捕捉其他的收看画面IM(图8的(3))。任务分配部22参照该时刻中的节目排列信息，并基于由数据处理 部23捕捉到的收看画面IM来确定用于生成转换数据所需的任务。
在图8的例子中，任务分配部22在多个任务1～任务3的候补中选择例如任务1，分配给数据处理部23。此外，任务分配部22在多个任务4～任务7的候补中选择例如任务5，分配给数据处理部23。
这时，在分配给数据处理部23的任务1和任务5中，任务5是将收看画面IM调整为收看画面im的处理。在这样的处理中需要比较多的资源。
在这种情况下，任务分配部22也判定为：数据处理部23不具有剩余的资源，不能再进行其他任务的处理。
另外，经过这些任务而包含估计结果在内的每个帧的转换数据被上传到服务器装置10。
(剩余资源的使用例)
接着，使用图9和图10对实施方式的电视装置20中的剩余资源的使用例进行说明。
图9和图10是表示在实施方式所涉及的电视装置20中剩余资源对数据转换的前处理的任务进行分配的情况的一个例子的示意图。在图9和图10的例子中，作为数据转换的前处理，进行预先提取规定的电视剧的表演者的特征量的处理。
如上述图6所示，当电视装置20的数据处理部23例如对综艺或电视剧等广播节目进行解析的情况下，有时进行检测这些广播节目中的规定的表演者的演出时刻等处理。这时，例如进行使用了DNN技术的面部认证等来对各个表演者进行判定，并确定这些表演者出现在收看画面上的时刻。
因此，例如通过预先提取规定的表演者的面部特征量，从而在数据转换处理时能够根据提取到的特征量迅速地进行各个表演者的面部认证。
如图9中的(a)所示，在规定的时刻在数据处理部23中生成了剩余的资源的情况下，电视装置20的任务分配部22例如将提取规定的电视剧的表演者的面部特征量的任务分配给数据处理部23。用于提取面部特征量的各个表演者的面部照片等可以存储在例如服务器装置10中。
在该情况下，服务器装置10的管理者等可以预先将获得了电视剧的表演者的使用许可的面部照片数据存储于服务器装置10的存储部15中。或者，电视装置20也可以自主性地获取包含在节目排列信息等中的表演者的面部照片并存储于存储部29中。在该情况下，关于包含电视装置20的元数据生成系统1中的节目排列信息的使用，也可以预先获得全面的许可。
如图9中的(b)所示，电视装置20的数据处理部23从被提供的表演者的面部照片中提取各个表演者的面部特征量并将其存储于例如存储部29中。然后，当对这些表演者演出的电视剧生成转换数据时，可以参照存储在存储部29中的特征量来确定各个表演者。其状态如图10所示。
如图10中的(a)所示，对于已提取表演者的面部特征量的电视剧，数据处理部23将按照每个帧捕捉到的收看画面中包含的信息转换为多维排列。
如图10中的(b)所示，数据处理部23根据生成的多维排列进行分析，提取收看画面中包含的表演者的面部特征量。
如图10中的(c)所示，数据处理部23读出存储在存储部29中的各个表演者的面部特征量的数据，通过与收看画面中包含的表演者的面部特征量进行对照来确定收看画面中 包含的表演者。
另外，电视装置20也可以将生成的表演者的面部特征量发送给服务器装置10。此外，服务器装置10也可以将由规定的电视装置20生成的特征量分发给其他电视装置20，在各个电视装置20进行数据转换时，在这些电视装置20中共用由一个电视装置20生成的特征量。
(元数据的生成处理例)
接着，使用图11对实施方式的由元数据生成系统1实施的元数据生成处理的例子进行说明。图11是表示实施方式所涉及的元数据生成系统1实施的元数据生成处理的顺序的一例的流程图。
如图11所示，电视装置20的任务分配部22在开始收看规定的广播节目时，向数据处理部23分配任务(步骤S101)。
换句话说，任务分配部22一边参照节目排列信息，一边按照广播节目的内容或每个帧，确定数据转换所需的任务。此外，任务分配部22将确定的任务分配给数据处理部23。
此外，任务分配部22根据确定出的任务，判定数据处理部23中是否产生剩余的资源(步骤S102)。
当判定为没有产生剩余的资源的情况下(步骤S102：否)，数据处理部23执行基于被分配的任务的步骤S103～S105的处理。当判定为产生剩余的资源的情况下(步骤S102：是)，数据处理部23进一步向数据处理部23分配与剩余资源相应的任务(步骤S107)。
具体而言，数据处理部23按照每个帧捕捉已开始收看的广播节目的收看画面(步骤S103)。此外，数据处理部23将收看画面中包含的信息转换为多维排列(步骤S104)。此外，数据处理部23根据多维排列进行各种估计并生成估计结果(步骤S105)。
此外，当进一步分配与剩余资源相应的任务的情况下，数据处理部23同时进行步骤S103～S105的处理，执行基于分配的任务的处理(步骤S107)。
收发部21将如上所述生成的转换数据发送给服务器装置10(步骤S106)。服务器装置10的收发部11从电视装置20接收转换数据(步骤S111)。
整合部12对从多个电视装置20收集的转换数据进行取舍选择，分别整合与各个广播节目相关的多个转换数据，生成与这些广播节目分别对应的时间序列数据(步骤S112)。
广告判定部13例如参照与生成的时间序列数据对应的节目排列信息，对该时间序列数据判定是否存在广告的插入(步骤S113)。
换句话说，广告判定部13根据节目排列信息，在与该时间序列数据对应的广播节目为NHK的广播节目的情况下判定为没有广告的插入，在是民办广播的广播节目的情况下判定为有广告的插入。但是，广告判定部13也可以代替节目排列信息，或者在节目排列信息的基础上，根据电视装置20的数据处理部23是否将广告的推断开始位置和推断结束位置赋予给时间序列数据来进行上述判定。
当判定为时间序列数据中有广告插入的情况下(步骤S113：是)，广告判定部13根据时间序列数据中被赋予的广告的推断开始位置和推断结束位置来确定广告的插入位置(步骤S114)。当判定为时间序列数据中没有广告插入的情况下(步骤S113：否)，广告判定部13跳过步骤S114的处理。
元数据生成部14对除了时间序列数据的广告的插入位置以外的正篇部分，根据时间序列数据以及时间序列数据中被赋予的数据处理部23的估计结果，生成表示广播节目的 内容的元数据(步骤S115)。
收发部11将如上所述生成的元数据发送给电视装置20(步骤S116)。电视装置20的收发部21从服务器装置10接收元数据(步骤S121)，例如在收看录像节目时等可显示地存储于存储部29中(步骤S122)。
如上所述，实施方式的由元数据生成系统1实施的元数据生成处理结束。
(比较例)
随着近几年的内容增加，要求内容的检索功能、以及针对各个内容的推荐功能等，同时提供有效的收看方式也变得重要。为了实现内容的有效收看，例如以场景信息为首的元数据是有用的。
到目前为止，电视广播节目的元数据的主流是人工制作，除了从广播结束到制作需要几个小时以外，元数据的制作成本也变得巨大。因此，开始尝试使用人工智能自动生成元数据。然而，元数据的自动生成例如存在以下课题。
电视装置需要收看、录像和播放的实时处理。因此，难以使电视装置自动生成需要对时间序列数据进行处理的元数据。
此外，在电视装置中系统资源是有限的。因此，难以使电视装置进行使用了人工智能的复杂的处理以及同时并列进行的处理。因此，基于这样的理由，难以使电视装置进行元数据的自动生成。
另一方面，当使服务器装置进行元数据的自动生成的情况下，为了在服务器装置中一并处理庞大的内容，服务器装置侧的成本变得巨大。
例如，在上述专利文献1、2的技术中，通过预先在云端准备并分发标识图像(Logo image)和图案文件(Pattern file)等，使电视装置进行广告区间的检测。然而，必须从广告的提供者处购买标识视像等，为了扩大对象内容需要巨大的成本。此外，由于使电视装置进行广告区间的检测，所以担心处理的延迟。
此外，在上述专利文献3的技术中，使医疗领域的终端装置实时地进行使用人工智能的特征量的生成，使用该特征量而执行使用了时间序列数据的处理。然而，难以将这样的技术应用于电视的广播节目。
在电视的广播节目中，例如要求在直播收看的途中播放录像内容的时移播放功能、以及汇总录像所有频道的全部广播节目的功能等。为了应对这些要求，需要将多个AV(Audio/Visual)解码器和用于实时并行处理多个内容的AI运算器安装于电视装置。由于这样的结构在部件成本和运行成本两方面都是很大的障碍，因此适用于作为消费者(consumer)设备的电视装置是不现实的。
根据实施方式的电视装置20，在提供广播节目的直播收看期间，根据广播节目的广播信号生成转换数据，该转换数据能够生成表示该广播节目的内容的元数据。
如上所述，可以使各个电视装置20承担庞大的内容的处理，例如与服务器装置10一并处理的情况不同，可以减轻服务器装置10的负担。
此外，上述数据转换处理例如可以使用电视装置20的现有硬件结构来执行。不需要向电视装置20添加新的结构或重新构建电视装置20本身，从而可以抑制部件成本和运行成本。
根据实施方式的电视装置20，将转换数据发送给生成元数据的服务器装置10。通过提高向这种服务器装置10的上传频度，由于服务器装置10处理的数据被频繁地更新，因 此例如也能够应对要求短时间的处理的时移播放等。
根据实施方式的电视装置20，将广播节目的信息按照每个帧而转换成多维排列而生成转换数据。如上所述，通过进行向多维排列的数据转换，可以处理庞大的数据，并且例如通过在DNN的输入输出数据中使用多维排列，从而使用了DNN技术的数据解析变得容易。此外，可以缩小服务器装置10中的处理数据的容量，并且可以收集和处理更多的内容的信息。
根据实施方式的电视装置20，基于节目排列信息而确定数据的转换所需的任务，并将确定出的任务分配给数据处理部23。由此，例如在按照广播节目的内容或每个帧而需要不同的任务的数据转换处理中，可以将适当的任务分配给数据处理部23来执行。此外，可以有效地利用数据处理部23的有限资源。
根据实施方式的电视装置20，在数据处理部23中存在剩余的资源的情况下，将可用于数据转换的前处理的任务分配给数据处理部23。由此，可以有效地利用数据处理部23的资源。
根据实施方式的服务器装置10，根据由电视装置20转换的转换数据而生成元数据。
由此，能够使服务器装置10承担需要处理时间序列数据的元数据的生成，能够抑制在电视装置20的收看、录像、以及播放的实时处理中产生障碍。
此外，在服务器装置10中只要执行主要处理了时间序列数据的元数据的生成处理即可，也能够减轻服务器装置10的负担。
根据该实施方式的服务器装置10，在从基于数据处理部23的广告的推断开始位置起超过规定期间而未包含推断结束位置的信息的情况下，判定为在推断开始位置处没有广告的插入。如上所述，通过在能够处理时间序列数据的服务器装置10中进行上述判定，能够高精度地确定广告的插入位置。
根据实施方式的服务器装置10，在广告的推断开始位置和推断结束位置之间包含其他推断开始位置或其他推断结束位置的情况下，从这些推断开始位置和推断结束位置的组合中，选择广告的插入期间最接近广告的最短广播时间的倍数的组合作为广告的开始位置和结束位置。如上所述，通过在能够处理时间序列数据的服务器装置10中进行上述判定，从而能够高精度地确定广告的插入位置。
根据实施方式的服务器装置10，在多个电视装置20中，将至少从正在提供广播节目的直播收看的电视装置20中收集到的转换数据整合为时间序列数据。如上所述，通过将数据转换处理分散在多个电视装置20中的每一个中，可以进一步减轻电视装置20的负担。
另外，在上述的实施方式中，例如服务器装置10从正在提供直播收看的多个电视装置20中收集转换数据并整合，从而生成时间序列数据。但是，进行元数据生成的服务提供者也可以在本公司内等设置多个电视装置20，从这些电视装置20向服务器装置10上传转换数据。
在这种情况下，可以分别向各个电视装置20始终提供特定的广播电台的广播节目的实时收看。由此，可以从一个电视装置20中收集与一个广播电台的广播节目相关的转换数据，从而例如不需要整合多个电视装置20的转换数据的处理。
此外，虽然在上述实施方式中接收装置为电视装置20，但本实施方式的结构并不限定于此。例如，接收装置也可以是具备广播信号的接收功能和放映功能、以及声音识别服务功能的个人计算机、智能手机、平板电脑、便携电话等其他设备。
虽然对本申请的实施方式进行了说明，但该实施方式是作为例子提出的，并不限定发明的范围。该新的实施方式可以以其他的各种方式实施，在不脱离发明的主旨的范围内，可以进行各种省略、替换、变更。这些实施方式及其变形包含在发明的范围和主旨中，并且包含在权利要求书中记载的发明及其等同的范围内。

Claims (12)

  1. 一种接收装置，其接收广播节目并且以可直播收看的方式提供该广播节目，其中，
    所述接收装置具备：
    数据处理部，在提供所述广播节目的直播收看时，所述数据处理部根据所述广播节目的广播信号而生成转换数据，该转换数据能够生成表示所述广播节目的内容的元数据：以及
    第一收发部，其将所述转换数据发送给生成所述元数据的服务器装置。
  2. 根据权利要求1所述的接收装置，其中，
    所述数据处理部将所述广播节目按照每个帧转换为多维排列而生成所述转换数据。
  3. 根据权利要求2所述的接收装置，其中，
    所述数据处理部根据所述多维排列而生成所述转换数据，其中，所述转换数据包含对所述广播节目的内容进行了估计的估计结果。
  4. 根据权利要求1所述的接收装置，其中，
    所述接收装置还具备：
    广播接收部，其接收所述广播节目的广播信号以及所述广播信号中多路复用的节目排列信息；以及
    任务分配部，其根据所述节目排列信息确定所述转换数据的生成中所需的任务，对所述数据处理部分配确定出的所述任务。
  5. 根据权利要求4所述的接收装置，其中，
    所述任务分配部在对所述数据处理部分配了所述任务之后，在所述数据处理部中存在剩余的资源的情况下，将可用于所述转换数据的生成的前处理的任务分配给所述数据处理部。
  6. 一种元数据生成系统，其中，
    所述元数据生成系统具备：
    权利要求1至权利要求5中任一项所述的接收装置；以及
    服务器装置，其以与所述接收装置可通信的方式连接于所述接收装置，
    所述服务器装置具备：
    第二收发部，其从所述接收装置接收由所述接收装置生成的所述转换数据；以及
    元数据生成部，其根据所述转换数据生成所述元数据。
  7. 根据权利要求6所述的元数据生成系统，其中，
    所述服务器装置还具备广告判定部，所述广告判定部确定所述广播节目中的广告的插入位置，
    所述元数据生成部生成所述元数据，所述元数据包含表示所述广告的插入位置的信息。
  8. 根据权利要求7所述的元数据生成系统，其中，
    所述数据处理部按照每个帧生成所述转换数据，所述转换数据包含表示所述广告的推断开始位置和推断结束位置的信息，
    所述广告判定部根据时间序列数据判定所述广告的插入位置，其中，该时间序列数据是按照时间序列而排列了每个帧的所述转换数据的时间序列数据。
  9. 根据权利要求8所述的元数据生成系统，其中，
    所述广告判定部在从所述推断开始位置起超过规定期间而未包含所述推断结束位置的信息的情况下，判定为在所述推断开始位置处没有所述广告的插入。
  10. 根据权利要求8所述的元数据生成系统，其中，
    所述广告判定部在所述推断开始位置和所述推断结束位置之间包含其他推断开始位置或其他推断结束位置的情况下，从这些推断开始位置和推断结束位置的组合中，选择所述广告的插入期间最接近广告的最短广播时间的倍数的组合来作为所述广告的开始位置和结束位置。
  11. 根据权利要求7所述的元数据生成系统，其中，
    所述服务器装置以与包含所述接收装置在内的多个接收装置可通信的方式与所述多个接收装置连接，
    所述服务器装置还具备整合部，所述整合部将从所述多个接收装置中的至少正在提供所述广播节目的直播收看的接收装置中收集到的所述转换数据整合为时间序列数据。
  12. 根据权利要求11所述的元数据生成系统，其中，
    所述服务器装置向所述多个接收装置中的至少任一个接收装置的所述第一收发部发送所述元数据，其中，根据所述转换数据生成所述元数据。
PCT/CN2023/101699 2022-07-08 2023-06-21 接收装置及元数据生成系统 WO2024007861A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202380013680.8A CN118020308A (zh) 2022-07-08 2023-06-21 接收装置及元数据生成系统

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022110677A JP2024008646A (ja) 2022-07-08 2022-07-08 受信装置およびメタデータ生成システム
JP2022-110677 2022-07-08

Publications (1)

Publication Number Publication Date
WO2024007861A1 true WO2024007861A1 (zh) 2024-01-11

Family

ID=89454189

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/101699 WO2024007861A1 (zh) 2022-07-08 2023-06-21 接收装置及元数据生成系统

Country Status (3)

Country Link
JP (1) JP2024008646A (zh)
CN (1) CN118020308A (zh)
WO (1) WO2024007861A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103079092A (zh) * 2013-02-01 2013-05-01 华为技术有限公司 在视频中获取人物信息的方法和装置
US20160371729A1 (en) * 2015-06-16 2016-12-22 Quixey, Inc. Advertisement Selection Using Uncertain User Data
CN108401176A (zh) * 2018-02-06 2018-08-14 北京奇虎科技有限公司 一种实现视频人物标注的方法和装置
CN111669612A (zh) * 2019-03-08 2020-09-15 腾讯科技（深圳）有限公司 基于直播的信息投放方法、装置和计算机可读存储介质
CN112602077A (zh) * 2018-05-29 2021-04-02 索尼互动娱乐有限责任公司 交互式视频内容分发
CN112789686A (zh) * 2018-10-02 2021-05-11 翰林大学产学合作团 利用胃内窥镜图像的深度学习诊断胃病变的装置及方法
CN113010701A (zh) * 2021-02-25 2021-06-22 北京四达时代软件技术股份有限公司 以视频为中心的融媒体内容推荐方法及装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103079092A (zh) * 2013-02-01 2013-05-01 华为技术有限公司 在视频中获取人物信息的方法和装置
US20160371729A1 (en) * 2015-06-16 2016-12-22 Quixey, Inc. Advertisement Selection Using Uncertain User Data
CN108401176A (zh) * 2018-02-06 2018-08-14 北京奇虎科技有限公司 一种实现视频人物标注的方法和装置
CN112602077A (zh) * 2018-05-29 2021-04-02 索尼互动娱乐有限责任公司 交互式视频内容分发
CN112789686A (zh) * 2018-10-02 2021-05-11 翰林大学产学合作团 利用胃内窥镜图像的深度学习诊断胃病变的装置及方法
CN111669612A (zh) * 2019-03-08 2020-09-15 腾讯科技（深圳）有限公司 基于直播的信息投放方法、装置和计算机可读存储介质
CN113010701A (zh) * 2021-02-25 2021-06-22 北京四达时代软件技术股份有限公司 以视频为中心的融媒体内容推荐方法及装置

Also Published As

Publication number Publication date
CN118020308A (zh) 2024-05-10
JP2024008646A (ja) 2024-01-19

Similar Documents

Publication Publication Date Title
US10158892B2 (en) Apparatus, systems and methods for a content commentary community
RU2601446C2 (ru) Оконечное устройство, серверное устройство, способ обработки информации, программа и система подачи сцепленного приложения
US8196168B1 (en) Method and apparatus for exchanging preferences for replaying a program on a personal video recorder
US20050132401A1 (en) Method and apparatus for exchanging preferences for replaying a program on a personal video recorder
US8275245B2 (en) Replace content with like content to enhance program experience
US9204200B2 (en) Electronic programming guide (EPG) affinity clusters
US20080260346A1 (en) Video recording apparatus
US7584483B2 (en) Content-exhibition control apparatus and method
US20090031357A1 (en) Image display apparatus and method for controlling the same
JP5306550B2 (ja) 映像解析情報送信装置、映像解析情報配信システム及び配信方法、映像視聴システム及び映像視聴方法
WO2024007861A1 (zh) 接收装置及元数据生成系统
US11895367B2 (en) Systems and methods for resolving recording conflicts
JP5198643B1 (ja) 映像解析情報アップロード装置及び映像視聴システム及び方法
JP6425423B2 (ja) 記録再生装置および記録再生システム
JP5078417B2 (ja) 信号処理装置及び信号処理方法
JP2002051287A (ja) 番組録画支援システムおよび番組録画支援方法、並びに、番組視聴サービスシステムおよび番組視聴サービス提供方法
WO2022174595A1 (zh) 显示装置以及用于显示装置的方法
KR20130013938A (ko) 코너별 시청이 가능한 디지털 방송 시스템 및 그 서비스 제공 방법
JP2013098742A (ja) コンテンツ出力装置、及びコンテンツ出力方法
JP6896661B2 (ja) サーバの番組情報処理方法及びサーバ
CN114731384A (zh) 显示装置以及用于显示装置的方法
JP2022127309A (ja) 電子機器、システム、およびプログラム
KR20140134097A (ko) 클라우드 환경에서 녹화 서비스 제공 방법 및 장치
JP2015118713A (ja) 管理サーバー、番組情報処理システム
RU2628773C2 (ru) Устройство обработки информации, способ обработки информации, программа и система совместного использования контента

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23834635

Country of ref document: EP

Kind code of ref document: A1