WO2019112398A1

WO2019112398A1 - Method and system for providing sign language broadcast service using companion screen service

Info

Publication number: WO2019112398A1
Application number: PCT/KR2018/015611
Authority: WO
Inventors: 김태우; 홍성욱; 김회웅
Original assignee: 주식회사 에어코드
Priority date: 2017-12-08
Filing date: 2018-12-10
Publication date: 2019-06-13

Abstract

A method and a system for providing a sign language broadcast service are provided. The method is a method for providing a sign language broadcast service by a device operating by at least one processor, and comprises the steps of: pairing with at least one companion device; extracting signaling information of a sign language video service from a broadcast signal received through a broadcasting network; and transmitting the extracted signaling information to the at least one paired companion device, wherein the at least one companion device is used to connect to a server for streaming a sign language video and receiving the sign language video.

Description

Method and system for providing a sign language broadcast service using a companion screen service

The present invention relates to a technology for providing a sign language broadcast service in a digital broadcast system.

The Advanced Television System Committee (ATSC) is developing a broadcasting technology specification under the name of ATSC 3.0, which is a terrestrial UHD (Ultra High Definition) broadcasting transmission standard for IP (Internet Protocol) based next generation terrestrial broadcasting service.

ATSC 3.0 is an IP-based transmission system that provides various services such as service guide, HTML (HyperText Markup Language) 5 app service, on-demand video, mobile interworking service, dynamic advertisement etc. in addition to basic TV service through hybrid broadcasting environment of broadcasting network and communication network. Service can be provided.

On the other hand, many countries have enacted legislation on prohibition of discrimination against persons with disabilities and rights relief, so that persons with disabilities can obtain various information through TV. In particular, the sign language broadcast service is a broadcast service with sign language interpretation for the hearing impaired. Sign language broadcast service is used as one of important means of ensuring broadcasting access right equal to that of non - handicapped persons by providing convenience for broadcasting of closed -

At present, terrestrial broadcasting, satellite broadcasting, cable broadcasting, and so on, a sign language broadcast service for the hearing impaired is being provided mainly on the news channel. This sign language broadcast service is a form in which a sign language video of a sign language interpreter is provided together with a part of the main broadcast screen. Therefore, the size of the sign language image is so small that it is difficult for a person with a hearing impairment to see the expression or hand gesture of the sign language interpreter.

SUMMARY OF THE INVENTION The present invention provides a method and system for providing a sign language broadcast service using a Companion Screen Service.

According to one aspect of the present invention, a method for providing a sign language broadcast service comprises the steps of: a device operated by at least one processor for providing a sign language broadcast service, the method comprising: pairing with at least one companion device; Extracting signaling information of a sign language video service from a signal and transmitting the extracted signaling information to the paired at least one companion device, wherein the extracted signaling information includes at least one companion device Is connected to a server for streaming and is used to receive the sign language video.

The extracting step extracts a MPD (Media Presentation Description) of the sign language video service from the service level signaling information, and the MPD can be transmitted to the at least one companion device.

Extracting a first MPD of a 2D (Dimensional) sign language video service and a second MPD of a virtual reality (VR) sign language video service from service level signaling information, The MPD and the second MPD may be transmitted to different companion devices and used to receive the 2D sign language video and the virtual reality sign language video, respectively.

According to another aspect of the present invention, there is provided a method for providing a sign language broadcast service by a broadcast server operated by at least one processor, the method comprising: transmitting a broadcast signal to which signaling information for a sign language video service is added to a main device Receiving a sign language video request using the signaling information from a companion device paired with the main device, and transmitting the requested sign language video to the companion device, wherein the companion device outputs A sign language video describing the broadcast screen is received from the broadcast server and is output to the screen.

Wherein the signaling information comprises first signaling information for a sign language video service of a first content type and second signaling information for a sign language video service of a second content type that is different from the first content type, Receiving a sign language video request using the first signaling information from a first companion device and receiving a sign language video request using the second signaling information from a second companion device different from the first companion device And the step of transmitting to the companion device may transmit a sign language image using the first signaling information and a sign language image using the second signaling information, respectively.

The step of transmitting to the main device may further include a MPD (Media Presentation Description) of the sign language video service in the service level signaling information of the broadcast signal.

According to another aspect of the present invention, a broadcasting system includes a signaling encoder for encoding first signaling information for a main video and second signaling information for a video signal describing the main video, a main signal, A transmission signal transmission device for transmitting the broadcasting signal multiplexed with the signaling information and the second signaling information to a main device through a broadcasting network and a request signal for a sign language video using the second signaling information from a companion device paired with the main device, And a sign language video transmission device for transmitting the requested sign language video to the companion device.

The sign language video transmission apparatus may be a DASH server that transmits the sign language video using a Moving Picture Expert Group (MPEG) -DASH (Dynamic Adaptive Streaming over HTTP) protocol.

Wherein the signaling encoder further encodes third signaling information for a sign language video stream of a virtual reality (VR) type describing the main video, and the sign language video transmission device further comprises: Receiving a request for a sign language video image using the third signaling information from a device and transmitting the requested sign language video image to the virtual reality device, wherein the main video, the sign language video and the virtual reality type sign language video Can be output simultaneously.

The second signaling information and the third signaling information may be included in service level signaling information of the broadcast signal.

According to the embodiment of the present invention, it is possible to solve the inconvenience that a device in which a main image is outputted and a device in which a mime image is output are separated using a companion screen service, so that a mime image is conventionally small in size.

In addition, a sign language video image is output through a TV, a companion device, a VR (Virtual Reality) device, etc. through a communication network separately from the main video image, thereby allowing the viewer to watch the sign language video image on various devices separate from the main video image output device.

1 shows a network configuration for providing a sign language broadcast service according to an embodiment of the present invention.

2 is a flowchart illustrating a process of providing a sign language broadcast service according to an embodiment of the present invention.

3 illustrates a network configuration for providing a sign language broadcast service according to another embodiment of the present invention.

4 is a flowchart illustrating a process of providing a sign language broadcast service according to another embodiment of the present invention.

5 is a hardware block diagram of a broadcasting system to which an embodiment of the present invention can be applied.

6 is a hardware block diagram of a device to which an embodiment of the present invention may be applied.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. Also, the terms " part, "" ... "," module ", and the like described in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software .

In the following description, signaling refers to transmission and reception of service information (SI) provided in a broadcasting system, an Internet broadcasting system, a broadcasting and Internet convergence system.

The broadcast signal can be used in a bidirectional broadcasting such as Internet broadcasting, broadband broadcasting, communication broadcasting, data broadcasting and / or VOD (Video On Demand) in addition to terrestrial UHD (Ultra High Definition) broadcasting, cable broadcasting, satellite broadcasting and / And a signal and / or data to be provided.

Embodiments of the present invention may be supported by the Advanced Television System Committee Standard (ATSC) document. That is, the steps or portions of the embodiments of the present invention that are not described in order to clearly illustrate the technical idea of the present invention can be supported by the document. In addition, all terms disclosed in this document may be described by the standard document.

Companion Screen (Companion Screen) service is a type of N screen, which provides services related to broadcasting contents by linking TV with various smart devices.

In the embodiment of the present invention, a sign language broadcast service is provided using a companion screen service.

FIG. 1 shows a network configuration for providing a sign language broadcast service according to an embodiment of the present invention, and FIG. 2 is a flowchart illustrating a process for providing a sign language broadcast service according to an embodiment of the present invention.

Referring to FIG. 1, the broadcasting system 100 is a digital broadcasting system such as a terrestrial UHD broadcasting system. The broadcast system 100 includes a signaling encoder 101, an audio and video (AV) encoder 103, a sign language video encoder 105, a broadcast signal transmission device 107 and a sign language video transmission device 109.

The signaling encoder 101 encodes signaling information including signaling information including meta data of a main image and meta data of a sign language image. Here, the main video refers to video data constituting a specific broadcast program.

The AV encoder 103 receives and encodes the main video. The main picture may include video data and audio data (AV).

The sign language video encoder 105 receives and encodes the sign language video. Here, the sign language image is a sign language image that is synchronized with the main image and describes the main image.

The broadcast signal transmission apparatus 107 generates a broadcast signal in which an encoded signaling signal output from the signaling encoder 101 and an encoded main video signal output from the AV encoder 103 are multiplexed. And transmits the broadcast signal to the main device 300 through the broadcasting network 200. [ Here, the broadcasting network 200 provides a path for transmitting a broadcasting signal to the main device 300, for example, a terrestrial broadcasting network, a satellite broadcasting network, and a cable broadcasting network.

The sign language video transmission device 109 transmits the encoded sign language video signal output from the sign language video encoder 105 to the

companion devices

501 and 503 via the communication network 400. The communication network 400 may be a broadband network such as an IP (Internet Protocol) network or a mobile broadcasting network.

The sign language video signal is transmitted through the communication network 400 according to MPEG (Moving Picture Expert Group) -DASH (Dynamic Adaptive Streaming over HTTP) standard. DASH is a streaming standard that divides AV data into files called segments and delivers them to the HTTP (HyperText Transfer Protocol).

At this time, the sign language video transmission apparatus 109 may be a DASH server. The sign language video transmission apparatus 109 converts media files such as an MPEG-2 TS (Transport Stream) file or an ISO (International Organization for Standardization) BMFF (Base Media File Format)

In addition, the

companion devices

501 and 503 may include a DASH client corresponding to the DASH server.

The main device 300 is a device that can receive and reproduce a digital broadcast signal from the broadcast signal transmission device 107 and includes a TV (Television), a settop box, and the like. At least one of the

companion devices

501 and 503 may include a mobile terminal such as a smart phone or a tablet capable of receiving a digital broadcast signal, a desktop personal computer (PC), a notebook PC, .

The main device 300 is connected to the broadcasting network 200 and the at least one

companion device

501 and 503 is connected to the communication network 400.

The main device 300 transmits a Uniform Resource Locator (URL) that can acquire a sign language image to at least one companion device (501, 503) and a media presentation description (MPD), which is signaling information of a sign language video.

Here, MPD describes a DASH media presentation corresponding to a sign language video service. The MPD describes a presentation that is delivered over the network 400. The MPD provides all the URLs for the initialization segment or media segment files needed to initialize the decoder.

At this time, it is assumed that the main device 300 and the at least one

companion device

501, 503 have already performed pairing.

According to one embodiment, the main device 300 and the at least one

companion device

501, 503 may generate a pairing session using a Universal Plug and Play (UPnP) protocol.

According to another embodiment, the main device 300 and the at least one

companion device

501, 503 may be capable of web socket communication.

The main device 300 and the at least one

companion device

501, 503 reproduce a sign language video image received from the communication network 400 based on the MPD.

The

companion devices

501 and 503 request the sign language video image transmission device 109 using the MPD and receive them. Then, the sign language video streamed from the sign language video transmission device 109 is reproduced using the MPD.

Companion devices

501 and 503 may include a DASH client. Here, the DASH client includes an MPD parser for parsing an MPD, a segment parser for parsing a segment, an HTTP client for transmitting an HTTP request message and an HTTP response message, Lt; RTI ID = 0.0 > engine. &Lt; / RTI >

2, the main device 300 searches neighboring

companion devices

501 and 503, and pairs the

companion devices

501 and 503 with the searched companion devices 501 and 503 (S101 and S103).

The main device 300 extracts the MPD for the sign language video service from the signaling information of the broadcast signal broadcasted from the broadcast signal transmission device 107 (S107). When the main device 300 receives the broadcast signal, it checks whether the signaling information includes information related to the sign language video service.

The main device 300 transmits the extracted MPD to the

companion devices

501 and 503 that have been paired (S103) (S109).

The

companion devices

501 and 503 having been paired with each other in step S103 parse and analyze the received MPD in step S111 and request a sign language video image transmission device 109 based on the parsed and interpreted content S113). The sign language video transmission apparatus 109 streams the sign language video that has been requested (S113) to the companion devices 501 and 503 (S115).

Then, the main device 300 outputs the main image of the broadcast signal received in step S105 (S117), and the

companion devices

501 and 503 paired with the main device 300 (S103) (S119). &Lt; / RTI > At this time, steps S117 and S119 are synchronized.

FIG. 3 shows a network configuration for providing a sign language broadcast service according to another embodiment of the present invention, and FIG. 4 is a flowchart illustrating a process for providing a sign language broadcast service according to another embodiment of the present invention.

3 and 4 are similar to those of FIG. 1 and FIG. 2, and a description overlapping with FIG. 1 and FIG. 2 will be omitted.

Referring to FIG. 3, a signaling encoder 101 encodes main video signaling information, sign language video signaling information, and VR sign language video signaling information. The broadcast signal transmission apparatus 107 broadcasts the broadcasting signal multiplexed with the encoded signal output from the signaling encoder 101 and the AV encoder 103 to the broadcasting network 200.

In addition, the sign language video encoder 105 encodes and outputs the sign language video. Then, the VR sign language video encoder 111 encodes and outputs the sign language video of the virtual reality type.

The sign language video transmission device 109 streams the sign language video stream at the request of the companion device 505. In addition, the sign language video transmission device 109 streams the sign language video signal of the virtual reality type at the request of the VR device 507.

Referring to FIG. 4, the main device 300 searches for

nearby companion devices

505 and 507 and pairs them with the searched companion devices 505 and 507 (S201, S203, and S205 S207).

The main device 300 extracts MPD1 for the sign language video service and MPD2 for the VR sign language video service from the signaling information of the broadcasting signal broadcasted from the broadcast signal transmission device 107 in step S211.

The main device 300 transmits the MPD1 extracted (S211) to the companion device 505 that is paired (S203) (S213). Then, the companion device 505 having been paired (S103) parses and analyzes the received MPD (S213) (S215), and requests the sign language video transmission apparatus 109 to receive the sign language video based on the parsed and analyzed content S217). The sign language video transmission device 109 streams the sign language video that has been requested (S217) to the companion device 505 (S219).

Further, the main device 300 transmits the MPD 2 extracted (S211) to the VR device 507 that is paired (S207) (S221). Then, the VR device 505 having been subjected to the pairing (S207) parses and analyzes the received MPD (S213) (S223), and requests the sign language video transmission apparatus 109 for the sign language video based on the parsed and analyzed contents S225). The sign language video transmission device 109 streams the request signal (S225) to the VR device 505 (S227).

Thereafter, the main device 300 outputs the main image of the broadcast signal received in step S209 to the screen (step S229), and the

companion devices

505 and 507 that are paired with the main device 300 (steps S103 and S) And outputs a sign language video image received in step S231 (S233). At this time, steps S229, S231, and S233 are synchronized.

FIG. 5 is a hardware block diagram of a broadcasting system to which an embodiment of the present invention can be applied, and shows a hardware configuration of the broadcasting system 100 described with reference to FIG. 1 to FIG.

5, a broadcast system 600 includes a communication device 601, a memory 603, a storage device 605, and at least one processor 607. [ The communication device 601 is connected to at least one processor 607 to transmit and receive data. The memory 603 stores programs that are associated with at least one processor 607 and include instructions that cause the configuration and / or methods in accordance with the embodiments described in Figs. 1-4 to execute. The program is implemented in combination with hardware such as memory 603, storage device 605 and at least one processor 607 to implement the present invention.

FIG. 6 is a hardware block diagram of a device to which an embodiment of the present invention can be applied. The hardware configuration of the main device 300 and the

companion devices

501, 503, 505, and 507 illustrated in FIG. 1 to FIG.

6, a device 700 is comprised of hardware including a communications device 701, a memory device 703, an input device 705, a display 707, and at least one processor 709, A program executed in combination with hardware at a specified location is stored.

The communication device 701 is connected to at least one processor 709 to receive a broadcasting signal through the broadcasting network 200 and receive the media signal through the communication network 400. [

The memory device 703 is coupled to the processor 709 and stores a program that includes instructions for causing the computer to perform the configuration and / or the method according to the embodiments described in Figs. 1-4. The program is implemented in conjunction with hardware such as memory device 703 and processor 709 to implement the present invention.

The input device 705 is coupled to the processor 709 and is a means for user input operation in accordance with the embodiments described in Figures 1-4.

The display 707 is connected to the processor 709 and outputs data according to the embodiments described in Figs. 1 to 4 to the screen. At this time, the input device 705 and the display 707 may be implemented as a single device.

Processor 709, in combination with hardware, such as memory device 703, implements the present invention.

The embodiments of the present invention described above are not implemented only by the apparatus and method, but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims

A method for providing a sign language broadcast service by a device operating by at least one processor,

Pairing with at least one companion device,

Extracting signaling information of a sign language video service from a broadcast signal received through a broadcast network, and

And transmitting the extracted signaling information to the paired at least one companion device,

The extracted signaling information includes,

Wherein the at least one companion device is connected to a server for streaming a sign language video image and is used to receive the sign language video image.
The method of claim 1,

Wherein the extracting comprises:

Extracting a MPD (Media Presentation Description) of the sign language video service from service level signaling information,

The MPD,

To the at least one companion device.
The method of claim 1,

Wherein the extracting comprises:

Extracting a first MPD (Media Presentation Description) of a 2D sign language video service and a second MPD of a virtual reality (VR) sign language video service from service level signaling information,

Wherein the first MPD and the second MPD comprise:

Wherein the video signal is transmitted to different companion devices, and is used to receive a 2D sign language video and a virtual reality sign language video, respectively.
A method for providing a sign language broadcast service by a broadcast server operating by at least one processor,

Transmitting a broadcasting signal to which signaling information for a sign language video service is added to a main device through a broadcasting network,

Receiving a sign language video request using the signaling information from a companion device paired with the main device, and

And transmitting the requested sign language video to the companion device,

The companion device comprising:

Receiving a sign language video describing a broadcast program outputted to the main device from the broadcast server and outputting the sign language video to a screen;
5. The method of claim 4,

Wherein the signaling information comprises:

The first signaling information for a sign language video service of a first content type and the second signaling information for a sign language video service of a second content type different from the first content type,

Wherein the receiving comprises:

Receiving a sign language video request using the first signaling information from a first companion device, and

Receiving a sign language video request using the second signaling information from a second companion device different from the first companion device,

Wherein the step of transmitting to the companion device comprises:

And a sign language video using the first signaling information and a sign language video using the second signaling information, respectively.
5. The method of claim 4,

Wherein the step of transmitting to the main device comprises:

Wherein the service level signaling information of the broadcast signal further includes a MPD (Media Presentation Description) of the sign language video service.
A signaling encoder for encoding the first signaling information for the main video and the second signaling information for the video signal describing the main video,

A transmission signal transmission device for transmitting a broadcast signal multiplexed with the main video, the encoded first signaling information, and the second signaling information to a main device through a broadcasting network, and

Receiving a sign language video request using the second signaling information from a companion device paired with the main device, and transmitting the requested sign language video to the companion device,

The broadcast system.
8. The method of claim 7,

The sign language video transmission apparatus includes:

Wherein the DASH server transmits the sign language video using a Moving Picture Expert Group (MPEG) -DASH (Dynamic Adaptive Streaming over HTTP) protocol.
8. The method of claim 7,

Wherein the signaling encoder comprises:

Encoding the third signaling information for a sign language video image of a virtual reality (VR) type describing the main video,

The sign language video transmission apparatus includes:

Receiving a sign language video request using the third signaling information from a virtual reality device paired with the main device, transmitting the requested sign language video to the virtual reality device,

Wherein the main image, the sign language video, and the virtual reality type sign language video,

And outputting at the same time to each device.
The method of claim 9,

The second signaling information, and the third signaling information,

Wherein the service level signaling information of the broadcast signal is included in the service level signaling information of the broadcast signal.