KR20080086640A - The apparatus and method of reception a talking with the hands and superimposed dialogue with set-top box - Google Patents

Abstract

An apparatus and a method for receiving a sign language/caption through a digital complex receiver are provided to convert video and voice data transmitted from a broadcasting station into a sign language video, a sign language animation and a caption image, and display an image and a sign language and caption image expressing the image seamlessly on a display. A sign language/caption converter(100) receives digital broadcasting data including video and voice data transmitted from a broadcasting station, and stores the received data after separating the data into video and voice data. The sign language/caption converter outputs the stored video and voice data, and converts the data into caption data through sign language video data and quick writing. A digital complex receiver(200) receives a sign language video, sign language animation and a caption image transmitted from the sign language/caption converter, and outputs PIP(Picture In Picture) screen data where a sign language video and a sign language animation are overlapped with video data or outputs the PIP screen data together with a caption image on a display. The digital complex receiver controls a size and location of a PIP screen outputted on the display and controls a location, font size and font color of the caption data according to setup of a user.

Description

Sign language and subtitle receiving apparatus and method through digital composite receiver {THE APPARATUS AND METHOD OF RECEPTION A TALKING WITH THE HANDS AND SUPERIMPOSED DIALOGUE WITH SET-TOP BOX}

1 is a block diagram of a sign language subtitle receiver through a digital composite receiver according to the present invention;

2 is a block diagram showing the components of a sign language / subtitle converter of a sign language / subtitle receiver through a digital composite receiver according to the present invention;

3 is a block diagram showing the components of the sign language conversion unit among the components of the sign language and subtitle conversion unit according to the present invention;

4 is a block diagram showing the components of the subtitle converter of the sign language / subtitle converter according to the present invention;

FIG. 5 is a block diagram showing the components of a digital composite receiver among a sign language / subtitle receiving apparatus through a digital composite receiver according to the present invention; FIG.

6 is a block diagram showing the components of the PIP output unit of the digital composite receiver according to the present invention;

7 is a block diagram showing the components of the playback speed control unit of the digital composite receiver according to the present invention;

8 is a diagram illustrating a process of configuring shot information by finding peaks of adjacent frame differences in order to detect a video scene change between adjacent frames from a TV video stream according to the present invention.

9 is a cumulative difference (Cumulative) by accumulating (C1, C2, C3, ....., Cm) the image difference between adjacent frames in units of shots (S1, S2, S3, ..., Sm) according to the present invention and a process of calculating the relative velocity of each shot using the cumulative differences of the shot units.

FIG. 10 is a diagram illustrating a process of changing the size of a PIP screen displayed through a sign language / subtitle receiving apparatus through a digital composite receiver according to an enlargement and reduction signal through manipulation of a key input unit of a user according to an embodiment of the present invention. Ye

11 is a flowchart illustrating a sign language / subtitle reception method through a digital composite receiver according to the present invention;

12 is a sign language video and a sign language animation screen output to PIP screen data through a PIP output unit 240 in a method of receiving a sign language and a subtitle through a digital composite receiver according to the present invention, and a TV image is 16: 9 or 4: 3. Flow chart showing the process of forming the size and output on the display.

※ Brief description of reference numerals ※

100: sign language and subtitle converter 110: tuner

120: image / audio separation unit 130: image data generation unit

140: voice data generation unit 150: sign language conversion unit

160: subtitle converter 170: transmission unit

200: digital composite receiver 210: antenna unit

220: memory unit 230: memory unit

240: caption image output unit 250: PIP output unit

260 playback speed control unit

The present invention transmits video data and audio data transmitted from a broadcasting station to a sign language / subtitle converter to convert a sign language video, a sign language animation, and a caption video, and then to a satellite digital composite receiver installed in a home or office. The digital composite receiver outputs PIP screen data superimposed on a sign language video and a sign animation on video data, or outputs a caption shape on one side of the PIP screen data, and outputs the size and position of the PIP screen and the caption data. The present invention relates to a sign language and subtitle receiving apparatus and method through a digital composite receiver configured to control the position, the font size, and the font color according to a user's setting.

Nowadays, digital technology has digitized all information, making the distinction between audio data, video data and data meaningless. At the same time, all the data is integrated into one single network, and the broadcasting network and the communication network are merged into one. In addition, due to the convergence of broadcasting and communication, the amount of multimedia data in the network in which broadcasting and communication are converged has increased enormously.

However, among the users who use the multimedia data in the broadcast communication convergence system, the hearing impaired people have a lot of difficulties in receiving and utilizing the multimedia data accurately compared to the general public.

In order to help the hearing impaired, various attempts have been made in broadcasting stations providing the multimedia data such as subtitle broadcasting to some multimedia data.

However, most deaf people tend to feel and understand sign language much more than text. Therefore, when transmitting the multimedia data such as news, a person directly sign language corresponding to the multimedia data and transmits the same simultaneously with the multimedia data. However, in the case of transmitting multimedia data through the above-described method, each of the multimedia data had to be digitized and transmitted by a person. Although there are limitations due to the above-described transmission method, there are various limitations on the multimedia data transmission method for the hearing impaired.

Therefore, when the multimedia data is transmitted in the broadcast communication convergence network for the hearing impaired, a sign language video and a sign language animation for the deaf and hearing impaired are superimposed on the image data and provided as a PIP screen, or simultaneously with the PIP screen. There is a need for the dissemination of systems that provide video.

In order to solve this problem, Japanese Patent Publication (Heb. 13-186430, filed December 22, 1999) receives digital broadcast data including image data and audio data. Data separating means for separating video data and audio data from the digital broadcasting data, and drawing data for expressing a sign language word in a sign language unit One memory means and speech data are recognized in the sign language word, and the drawing data corresponding to the recognized speech data is sequentially read from the memory means, and a predetermined animation is read. Number of sign language animation data generated by generating sign language animation data A superimposing means for superimposing the sign language animation data on the image data and obtaining superimposed image data; and displaying an image on which the image is displayed based on the superimposed image data. Although a digital broadcast receiver composed of means has been proposed,

This divides the digital broadcast data transmitted from the broadcasting station into video data and audio data in the digital broadcast receiver, converts the separated audio data into one-to-one sign language animation for each word unit, and superimposes and outputs the video data on the display unit. As a result, when a new word or a foreign word is included in the voice data, it is difficult to convert to a sign language animation, and thus, a sign language expression is not smooth and a break occurs frequently.

In addition, since a picture in picture (PIP) screen displaying sign language video data using a sign language avatar or the like on a screen is fixed to a fixed size, a user can enlarge and reduce the size of the PIP screen. There is a problem that the user can not want to watch, the user is inconvenient to watch a broadcast program, and the user's activeness is excluded.

To solve the above problems, the present invention converts video data and audio data transmitted from a broadcasting station into a sign language video, a sign language animation, and a caption video so that the hearing impaired person and the hearing impaired person can see and hear the actual TV, and The purpose is to output a sign language and subtitle image representing an image on a display without interruption.

In addition, it is an object of the present invention to control the position and size of the PIP screen so that the size of the PIP screen displayed at the time of output is changed and displayed according to the enlargement and reduction signal through manipulation of the key input unit of the user.

Sign language and subtitle receiving apparatus through a digital composite receiver according to the present invention for achieving the above object,

In the broadcast communication convergence system having a transceiver for transmitting and receiving multimedia data,

Receives digital broadcasting data including video data and audio data transmitted from broadcasting stations, stores them separately into video data and audio data, and outputs the stored video data and audio data. A sign language / subtitle converter for converting the subtitle data through the sign language video data and the shorthand;

Receives a sign language video, a sign language animation, and a caption video transmitted from the sign language / subtitle converter, and outputs the PIP screen data superimposed on the sign data and the sign language animation, or displays the caption video simultaneously with the PIP screen data. It is achieved by including a digital composite receiver to control the output to the image, and to control the size and position of the PIP screen, the position of the caption data, the font size, the text color to be output according to the user's settings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a sign language and subtitle receiving apparatus through a digital composite receiver according to the present invention, and FIG. 2 is a view illustrating a component of a sign language and subtitle converting unit among a sign language and subtitle receiving apparatus through a digital composite receiver according to the present invention. 3 is a block diagram showing the components of the sign language conversion unit among the components of the sign language and subtitle conversion unit according to the present invention, and FIG. 4 is a view of the components of the sign language and subtitle conversion unit according to the present invention. It relates to a block diagram showing the components of the subtitle conversion unit, Figure 5 relates to a block diagram showing the components of the digital composite receiver of the sign language / subtitle receiving apparatus through the digital composite receiver according to the present invention, Figure 6 The block diagram showing the components of the PIP output unit of the digital composite receiver according to the present invention, Figure 7 is a playback speed of the digital composite receiver according to the present invention It relates to a diagram showing the components of a fisherman block, which is composed of a sign language and caption conversion unit 100, the digital composite receiver 200.

First, as shown in FIG. 2, the sign language / subtitle converter 100 according to the present invention will be described.

The sign language / subtitle converter 100 receives digital broadcast data (video data, audio data) transmitted from a broadcasting station to a home or office in the middle, divides and stores the video data and audio data, and stores the stored video data. And outputting audio data and converting the sign language video data and the sign language animation data or the caption data through the shorthand in real time through a sign language expert and a registrar, which includes a tuner unit 110, a video / audio separation unit 120, The video data generator 130, the audio data generator 140, the sign language converter 150, the subtitle converter 160, and the transmitter 170 are included.

The tuner 110 receives a digital broadcast radio wave through an antenna and performs processing such as frequency conversion and recovery of a digital signal on the received image data and audio data.

The video / audio separation unit 120 separates and stores original image data and original audio data from data output from the tuner unit 110.

The image data generation unit 130 applies the decoding process such as video decoding processing of MPEG2 to the original image data transmitted from the image / audio separation unit 120 to generate the image data, the sign language conversion unit 150, and the subtitle conversion. It serves to deliver to the unit (160).

The audio data generating unit 140 performs decoding processing such as audio decoding processing of MPEG2 on the original audio data transmitted from the video / audio separating unit 120 to generate the audio data, the sign language conversion unit 150, and the subtitle conversion. It serves to deliver to the unit (160).

The sign language converter 150 converts a sign language video and a sign language animation form into one-to-one correspondence through the voice data and the image data generated by the image data generator 130 and the voice data generator 140. As shown in FIG. 3, there is a sign language video converter 151 and a sign language animation converter 152.

Sign language video converter 151 is equipped with a monitor, the sign language experts to watch and listen to the video data and audio data output from the monitor, the sign language, focusing on the sign language operation, and then recorded on the TV screen and recorded with a recording camera Save it.

The sign language animation converter 152 extracts voice data for each word unit and converts the voice data into a sign language animation form through a sign language animation program that has a one-to-one correspondence with the extracted voice data, which is a voice data extraction program unit 152a. And a sign language animation conversion program unit 152b.

The voice data extraction program unit 152a extracts subtitle data stored in the voice data generation unit 140 for each word unit.

Here, extracting by word unit means that all the words ending with '~', '~', '~', and '~' in a sentence are included in the subject part, and '~' is used in the subject part. ',' Includes all the words ending in the first part of the object, and includes all the words ending in '~ da' or the period in the object part, and is a descriptive word. The word order is first listed by breaking the word in order of fishermen, then divided into 9 parts of Korean nouns, nouns, pronouns, investigations, investigations, verbs, adjectives, adjectives, adverbs, and interjections.

The sign language animation conversion program unit 152b converts the extracted voice data into an actual sign language animation. The sign language conversion algorithm stores the sign language animations if they are compared and analyzed in each sign language animation database divided into the primary part, the object word, and the description word of the subject part, the object word, and the description word that are listed first. You will be taken to the database.

That is, if a sign language animation is compared and analyzed in each sign language animation database divided into nouns, pronouns, investigations, investigations, verbs, adjectives, adjectives, adverbs, and interjections constituting the subject, object and descriptive fishermen, the sign language animation is stored.

The caption converting unit 160 is shorthanded in real time while watching and listening to video data and audio data generated by the video data generating unit 130 and the audio data generating unit 140, and inserts the shortened contents into captions. In this case, as shown in Fig. 4, the character input key input unit 161 for inputting audio data driven by the shorthand inputter at the same time as the video data in shorthand and general characters according to the divided time, and the characters Character input display unit 162 for displaying the characters output from the key input unit 161 for input, and character correction display unit for displaying the characters output from the key input unit 161 for character correction so that the operator can correct the characters. 163 and a plurality of characters received from the plurality of character input key input units 161 and output to the character input display unit 162 and the character correction display unit 163. A subtitle conversion control unit 164 for controlling and completing one sentence sequentially by sequentially outputting the characters output from the plurality of character correction key input units 161, and inserting the completed sentence as a subtitle to record a subtitle image. do.

The transmitter 170 transmits a sign language video and a sign language animation completed through the sign language converter 150, and a caption video completed through the subtitle converter 160 to the digital composite receiver 200. do.

Next, as shown in FIG. 5, the digital composite receiver 200 will be described.

Digital composite receiver 200 according to the present invention consists of a set-top box for receiving a cable or satellite broadcasting and connecting to a TV in the home and office, transmitted from the sign language / subtitle converter (150, 160) Receives a sign language video, a sign language animation, and a caption video, and outputs the PIP screen data superimposed on the video data and the sign language animation, or controls the caption video to be output on the display simultaneously with the PIP screen data. The size and position of the PIP screen output on the screen, the position of the caption data, the font size, and the font color are controlled according to the user's setting.

The digital composite receiver 200 includes an antenna unit 210, a memory unit 220, a caption image output unit 230, a PIP output unit 240, and a reproduction speed control unit 250.

The antenna unit 210 receives a sign language video, a sign language animation, and a caption image transmitted from the sign language / subtitle converters 150 and 160.

The memory unit 220 stores a sign language video, a sign language animation, and a caption image transmitted from the antenna unit 210.

The caption image output unit 230 outputs the caption image extracted from the memory unit 220 to one side of the PIP screen data on the display, which receives caption position data, decodes the caption position data, and detects and outputs the caption position data accordingly. The subtitle display program unit 231 is configured.

Here, the caption position data includes position information about a horizontal line start point (horizontal_data_start_point) and an endpoint (horizontal_data_end_point) and a vertical line start point (vertical_data_start_point) and an endpoint (vertical_data_end_point) of the caption output on the display.

The caption position data consisting of the horizontal line star point + end point and the vertical line start point + end point can be pre-positioned to the top, bottom, and side of the PIP screen so that the user can set the caption position through key input.

The PIP output unit 240 superimposes a sign language video and a sign language animation extracted from a memory unit on a TV image, and outputs a sign language video and a sign language animation screen to PIP screen data, and the TV image is 16: 9 or 4: 3 size. Formed to serve to output on the display, as shown in Figure 6, the first tuner 241, the second tuner 242, IF amplifier 243, PIP mode switching unit ( 244, a PIP signal processor 245, a sub-screen size controller 246, a PIP position controller 247, and a microcomputer 248.

The first tuner 241 tunes a channel so that a sign language video and a sign language animation screen are displayed on a main screen among the high frequency signals received through the antenna unit 210, and performs a high frequency amplification process on the composite video signal of the tuned channel. In addition, the channel is tuned with the oscillation frequency oscillated by the channel tuning to amplify the intermediate frequency (IF) which is easy to amplify.

The second tuner 242 converts a channel to be displayed on the sub-screen among the high-frequency signals received through the antenna unit 210 into an intermediate frequency that is easy to tune and amplify.

The IF amplifier 243 amplifies the sign language video, the sign language animation signal, and the TV image signal applied from the first tuner 241 and the second tuner 242 at an intermediate frequency IF.

The PIP mode switching unit 244 switches the sign language moving picture, the sign language animation signal, and the TV video signal applied by being amplified from the IF amplifier 243 to the PIP mode according to the control signal of the microcomputer.

The PIP signal processor 245 receives a sign language video, a sign language animation signal, and a TV video signal from the PIP mode switching unit 244, respectively, and superimposes a sign language video signal on a TV video signal, or superimposes a sign language animation signal on a TV video signal. PIP processing to output.

The sub-screen size control unit 246 adjusts the size of the PIP screen according to the control signal of the microcomputer unit after receiving the TV video signal from the PIP signal processing unit 245.

That is, the user can adjust the size of the TV image consisting of the PIP screen to 30%, 50%, 70%, 100%, 150%.

The PIP position controller 247 receives a TV video signal consisting of a PIP screen from the PIP signal processor 245 and sets the position of the PIP screen according to the control signal of the microcomputer 248.

That is, the user can set the position of the TV image composed of the PIP screen on one side of the upper, lower, left, and right sides of the display.

The microcomputer unit 248 is connected to the first tuner unit 241, the second tuner unit 242, the PIP mode switching unit 244, and the PIP signal processing unit 245, and is generated from a key input unit operated by a user. It serves to apply a signal for adjusting the size of the PIP screen to the sub-screen size control unit 246 according to the signal.

The playback speed control unit 250 controls the playback speed so that the sign language video and the sign language animation can be played in the form of simultaneous viewing with the TV image, so that the sign language video and the sign language animation can be simultaneously displayed with the TV image. The delay time is about 2 to 5 seconds longer than that of the TV video output from the memory unit and the sign language animation output from the memory unit.

At this time, in the present invention, in order to reduce the delay time caused by the conversion of the sign language video and the sign animation animation, one of a key frame distribution, an image scene boundary distribution, or an image difference distribution between adjacent frames for a TV video stream is provided through a play speed control unit. Detects the distribution of video scenes by the video signal and automatically speeds up the playback speed of the sign language video and sign animation according to the detected video scene change distribution. In this case, the video frames can be played faster than the normal speed in the part where the difference between the key frame and the video scene between the frames is large, and slower than the normal speed in the lesser part.

To this end, the playback speed control unit 250 according to the present invention,

As illustrated in FIG. 7, a speed detector 251a which detects a video scene change distribution from a TV stream and outputs relative velocity data according to the detected video scene change distribution, and a relative output from the speed detector 51a. The playback speed calculation unit 251b for playing back a sign language video and a sign language animation simultaneously with the TV image using the speed data, and directly controlling the playback speed calculated by the playback speed calculation unit 251b through a user's key input. It consists of a key input unit 251c.

The speed detector 251a calculates a relative speed according to a key frame distribution for a TV video stream, an image scene boundary point distribution for a video scene change, and an image difference distribution between adjacent frames.

Here, the image difference distribution between adjacent frames uses the distribution of the image differences between the frames to detect a change in the video scene between adjacent frames from the TV video stream to obtain an inter-frame difference as shown in FIG. 8. The peaks of the frame differences are found to form shot (S1, S2, S3, ..., Sm) information.

As shown in FIG. 9, the image difference between adjacent frames is accumulated (C1, C2, C3, ...., Cm) in units of shots S1, S2, S3, ..., Sm. The cumulative difference is calculated and the relative speed of each shot is calculated using the cumulative differences of the shot units.

As described above, the image scene change detected by the speed detector 251a is output as relative speed data according to a distribution of cumulative differences.

The relative speed with respect to the distribution of the video scene changes normalizes the TV video stream, and then the portion where the distribution of the video scene changes is denser than the specified value has a relatively high speed value for the playback speed of the sign language video and the sign animation. The portion where the distribution of the video scene change is not dense has a relatively small speed value for the playback speed of the sign language video and the sign animation.

Then, the playback speed calculator 251b receives relative speed data according to the scene change distribution of the TV video stream output from the speed detector and calculates the playback speed. That is, the playback speed calculator 251a speeds up the playback speed of the sign language video and the sign language animation in a portion where the distribution of the video scene change is dense according to the relative velocity data by the distribution of the video scene change, and in the part where the distribution of the video scene change is not dense. It will slow down the playback speed of sign language videos and sign language animations.

That is, by using the distribution of peak points of video scene changes or parts of large changes in video scene changes, the playback speed of a sign language video and a sign language animation is slowed down for a portion where the video scene change points are concentrated, and a playback speed is performed for a portion that is not dense. Will be taken quickly.

In addition, the playback speed calculator 251b is a function of a relative speed and weight calculated from a TV video stream using a weight inputted through a key input unit so that a user can directly control the playback speed. The playback speed of the video and sign language animation is determined.

The playback speed according to the present invention is determined by the relative speed and the weight. That is, different reproduction speeds may be determined by different weights added to the relative speeds. Therefore, by controlling the reproduction speed by the weight, it is possible to efficiently reduce the delay time caused by the sign language / subtitle converter.

Hereinafter, a sign language / subtitle reception method through a digital composite receiver according to the present invention will be described.

First, the operation process of the sign language / subtitle converter according to the present invention will be described.

As shown in FIG. 11, the tuner unit 110 receives digital broadcast data (video data, audio data) transmitted from a broadcasting station to a home or office through an antenna, and receives the received video data and audio data. Processing such as frequency conversion and digital signal recovery is performed (S100).

Subsequently, the original image data and the original audio data are separated and stored from the data output from the tuner 110 through the image / audio separator 120 (S110).

Subsequently, decoding processing such as video / audio decoding processing of MPEG2 is performed on the original image data and the original audio data transmitted from the video / audio separation unit 120 to generate the image data and the audio data, and the sign language conversion unit 150 and The caption converting unit 160 transmits the caption conversion unit 160 (S120).

Subsequently, the sign language video converter 151 and the sign language animation converter 152 of the sign language conversion unit 150 are driven (S130).

That is, the sign language expert hydrates the video data and the audio data output from the monitor through the sign language video converter 151, focuses on the sign language operation, and then records and stores the recorded video while displaying it on a TV screen. ,

Alternatively, the sign language animation converter 152 is driven to extract caption data stored in the voice data generation unit by word units through the voice data extraction program unit, and converts the voice data extracted from the sign language animation conversion program unit into an actual sign language animation.

Subsequently, the caption converting unit 160 shortens in real time while watching and listening to the video data and the audio data generated by the video data generating unit and the audio data generating unit, and inserts the shortened contents into captions to record as caption images (S140). ).

Subsequently, the transmission unit 170 transmits the sign language video, the sign language animation completed through the sign language conversion unit 150, and the completed caption image through the subtitle conversion unit 160 to the digital composite receiver 200 (S150). .

Subsequently, a sign language video, a sign language animation, and a caption image transmitted from the sign language / subtitle converters 150 and 160 are received through the antenna unit 210 of the digital composite receiver 200 (S160).

Subsequently, a sign language video, a sign language animation, and a caption image transmitted from the antenna unit 210 are stored in the memory unit 220 (S170).

Subsequently, a sign language video and a sign language animation screen output by superimposing a sign language video and a sign language animation extracted from the memory unit 220 on the TV image through the PIP output unit 240 and outputting the PIP screen data, and the TV image is 16: 9 or It is formed in 4: 3 size and output on the display (S180).

Subsequently, the caption image output unit 240 outputs the caption image extracted from the memory unit 220 to one side of the PIP screen data on the display, which receives caption position data from the caption display program unit 241 and decodes it. The caption position data is detected and output accordingly (S190).

Subsequently, the playback speed control unit 260 controls the playback speed such that the sign language video and the sign language animation are reproduced in the form of simultaneous viewing together with the TV image (S200).

Through this process, the present invention converts video data and audio data transmitted from a broadcasting station into a sign language video, a sign language animation, and a caption video so that the hearing impaired person and the hearing impaired person can watch and listen to a real TV. The sign language and subtitle images can be output on the display without interruption.

Hereinafter, a sign language video and a sign language animation screen outputted on PIP screen data by superposing a sign language video and a sign language animation extracted from the memory unit 220 on a TV image through the PIP output unit 240 according to the present invention, and a TV image The process of forming a 16: 9 or 4: 3 size and outputting it on a display will be described in detail.

As shown in FIG. 12, the first tuner 251 tunes a channel so that a sign language video and a sign language animation screen are displayed on a main screen among high frequency signals received through the antenna unit 210. The composite video signal is subjected to high frequency amplification and mixed with the oscillation frequency oscillated by the channel tuning, thereby amplifying the intermediate frequency (IF) which is easy to amplify (S181).

Subsequently, among the high frequency signals received through the antenna unit 210 in the second tuner unit 252, the channel to be displayed on the sub-screen is selected and converted into an intermediate frequency that is easy to amplify (S182).

Subsequently, the IF amplifier 253 amplifies a sign language video signal, a sign language animation signal, and a TV image signal applied from the first tuner 251 and the second tuner 252 at an intermediate frequency IF (S183). .

Subsequently, the PIP mode switching unit 254 switches the sign language video signal, the sign language animation signal, and the TV video signal, which are amplified from the IF amplifier 253, to the PIP mode according to the control signal of the microcomputer 259, and outputs the PIP mode. (S184).

Subsequently, the PIP signal processor 256 receives a sign language video signal, a sign language animation signal, and a TV video signal from the PIP mode switching unit 254, respectively, and superimposes a sign language video signal on the TV video signal, or sign language animation signal on the TV video signal. Are superimposed and output by PIP processing (S185).

Subsequently, after receiving the TV video signal from the PIP signal processor 256 through the sub-screen size control unit 257, the size of the PIP screen is adjusted according to the control signal of the microcomputer 259 (S186).

Subsequently, after receiving the TV video signal consisting of the PIP screen from the PIP signal processor 256 through the PIP position controller 258, the position of the PIP screen is set according to the control signal of the microcomputer 259 (S187).

Through this process, as shown in FIG. 10, the position and size of the PIP screen are controlled so that the size of the PIP screen displayed at the time of output is changed and displayed according to the enlargement and reduction signal through manipulation of the user's key input unit. You can do it.

As described above, the present invention converts the video data and audio data transmitted from a broadcasting station into a sign language video, a sign language animation, and a caption video so that the hearing impaired person and the hearing impaired person can see and hear the actual TV, and the video Sign language and subtitle images to be displayed can be output on the display without interruption,

It is possible to control the position and size of the PIP screen so that the size of the PIP screen displayed at the time of output is variable and displayed according to the enlargement and reduction signal through manipulation of the user's key input unit.

Claims (9)

In the broadcast communication convergence system having a transceiver for transmitting and receiving multimedia data, Receives digital broadcasting data including video data and audio data transmitted from broadcasting stations, stores them separately into video data and audio data, and outputs the stored video data and audio data. A sign language / subtitle converter 100 for converting the subtitle data through the sign language video data and the shorthand, Receives a sign language video, a sign language animation, and a caption video transmitted from the sign language / subtitle converter, and outputs the PIP screen data superimposed on the sign data and the sign language animation, or displays the caption video simultaneously with the PIP screen data. And a digital composite receiver 200 which controls the output to the image, and controls the size and position of the PIP screen, the position of the caption data, the font size, and the font color according to the user's setting. Sign language and subtitle receiving device through a digital multiplex receiver. The method of claim 1, wherein the sign language subtitle converter 100 A tuner unit 110 which receives digital broadcast radio waves through an antenna and performs processing such as frequency conversion, recovery of digital signals, etc. on the received video data and audio data. A video / audio separation unit 120 for separating and storing original image data and original audio data from data output from the tuner unit 110; Video data generated by applying decoding processing such as video decoding processing of MPEG2 to the original video data transmitted from the video / audio separation unit 120, and then transmitted to the sign language conversion unit 150 and the subtitle conversion unit 160. A data generator 130, Decode processing such as audio decoding processing of MPEG2 is performed on the original audio data transmitted from the video / audio separating unit 130 to generate audio data, and transmitted to the sign language conversion unit 150 and the subtitle conversion unit 160. A data generator 140, A sign language converter 150 for converting a sign language video and a sign language animation form into one-to-one correspondence through the voice data and the image data generated by the image data generator 130 and the voice data generator 140; A caption converting unit for shortening in real time while viewing and listening to the video data and the audio data generated by the video data generating unit 130 and the audio data generating unit 140, and inserting the shortened contents into captions to record as caption images ( 160), A digital composite receiver comprising: a sign language video, a sign language animation completed through a sign language conversion unit 150, and a transmission unit 170 which transmits a subtitle image completed through a subtitle converter to a digital complex receiver 200. Sign language subtitle receiver through. The sign language converting unit 150 is provided with a monitor, and a sign language expert performs sign language while watching and listening to video data and audio data output from the monitor, and focuses on the sign language operation. Sign language video converter 151 for recording and recording with a recording camera while shining, Sign language through a digital composite receiver characterized in that it comprises a sign language animation converter 152 that extracts the speech data by word units and converts the speech data into a sign language animation form through one-to-one correspondence with the extracted speech data. Subtitle Receiver. The subtitle converting unit 160 of claim 2, further comprising: a key input unit 161 for inputting audio data driven by the shorthand inputter at the same time as the video data in shorthand and general text according to the divided time; A character input display unit 162 for displaying a character output from the character input key input unit 161, A character correction display unit 163 for displaying a character output from the character input key input unit 161 so that a correction operator corrects the character; Receives characters from the plurality of character input key input units 161 and controls the characters to be output to the character input display unit 162 and the character correction display unit 163, and from the plurality of character correction key input units 161. Sign language and subtitle receiving apparatus through a digital composite receiver, characterized in that it consists of a subtitle conversion control unit 164 for successively completing the output characters to complete one sentence, and inserts the completed sentence into a subtitle to record as a subtitle image . The method of claim 1, wherein the digital composite receiver 200 An antenna unit 210 for receiving a sign language video, a sign language animation, and a caption image transmitted from the sign language / subtitle converters 150 and 160; A memory unit 220 for storing a sign language video, a sign language animation, and a caption image transmitted from the antenna unit 210; A caption image output unit 230 for outputting a caption image extracted from the memory unit 220 to one side of the PIP screen data on a display; A sign language video and sign language animation screen superimposed on a PIP screen data by superimposing a sign language video and a sign language animation extracted from a memory unit on a TV video, and a TV video screen having a 16: 9 or 4: 3 size to be output on a display PIP output unit 240, Sign language and subtitle receiving apparatus through a digital composite receiver, characterized in that the playback speed control unit 250 is configured to control the playback speed so that the sign language video and the sign language animation is reproduced in the form of simultaneous viewing with the TV image. The method of claim 5, wherein the PIP output unit 240 Among the high frequency signals received through the antenna unit 210, a channel is tuned to display a sign language video and a sign language animation screen on a main screen, and a high frequency amplification process is performed for the composite video signal of the tuned channel and the channel is oscillated. A first tuner unit 241 for amplifying an intermediate frequency (IF) which is easily amplified by mixing with the oscillation frequency A second tuner unit 242 for converting a channel to be displayed on the sub-screen among the high-frequency signals received through the antenna unit 210 into an intermediate frequency that is easy to tune and amplify; An IF amplifier 243 for amplifying a sign language video and a sign language animation signal and a TV image signal applied from the first tuner 241 and the second tuner 242 at an intermediate frequency (IF), A PIP mode switching unit 244 for converting a sign language video signal, a sign language animation signal, and a TV video signal that are applied by being amplified by an intermediate frequency from the IF amplifier 243 to a PIP mode according to a control signal of a microcomputer; The PIP mode switching unit 244 receives a sign language video signal, a sign language animation signal, and a TV video signal, respectively, and superimposes a sign language video signal on the TV video signal, or superimposes a sign language animation signal on the TV video signal to output the PIP process. PIP signal processing unit 245, A sub-screen size control unit 246 for adjusting the size of the PIP screen according to the control signal of the microcomputer unit after receiving the TV video signal from the PIP signal processing unit 245; After receiving the TV video signal consisting of the PIP screen from the PIP signal processing unit 245, PIP position control unit 247 for setting the position of the PIP screen according to the control signal of the microcomputer unit 248 Sign language and subtitle receiver using digital composite receiver. The tuner unit 110 receives digital broadcast data (video data, audio data) transmitted from a broadcasting station to a home or office through an antenna, and performs frequency conversion and recovery of the digital signal on the received video data and audio data. Performing a process (S100); Separating and storing the original image data and the original audio data from the data output from the tuner unit 110 through the image / audio separator 120 (S110); Decoding processing such as video / audio decoding processing of MPEG2 is performed on the original video data and the original audio data transmitted from the video / audio separation unit 120 to generate video data and audio data, and the sign language conversion unit 150 and subtitle conversion are performed. Delivering to the unit 160 (S120), The sign language video converter 151 of the sign language converter 150 and the sign language animation converter 152 are driven (S130), The caption conversion unit 160 records and listens to the video data and the audio data generated by the video data generation unit and the audio data generation unit in real time, and inserts the shortened contents as captions to record as caption images (S140). )Wow, In step S150, the transmitting unit 170 transmits the sign language video and the sign language animation completed by the sign language conversion unit 150, and the closed caption image through the subtitle conversion unit 160 to the digital composite receiver 200 (S150). , Receiving a sign language video, a sign language animation, and a caption image transmitted from the sign language / subtitle converters 150 and 160 through the antenna unit 210 of the digital composite receiver 200 (S160); Storing a sign language video, a sign language animation, and a caption image transferred from the antenna unit 210 to the memory unit 220 (S170); A sign language video and a sign language animation screen output to the PIP screen data through the PIP output unit 240 and a TV image having a 16: 9 or 4: 3 size to be output on a display (S180); Outputting the caption image extracted from the memory unit 220 to one side of the PIP screen data through the caption image output unit 240 on the display (S190); Receiving a sign language and subtitle through a digital composite receiver, comprising: controlling a playback speed such that a sign language video and a sign language animation are reproduced in a form capable of simultaneous viewing with a TV image through a playback speed controller 260. Way. The method of claim 7, wherein the sign language video converter 151 of the sign language converter 150 and the sign language animation converter 152 are driven (S130). The sign language video converter 151 hydrates the video data and audio data output from the monitor by a sign language expert while watching and listening to the sign language, focusing on the sign language operation, and then recording and storing it on a TV screen while recording it. The sign language animation converter 152 is driven to extract subtitle data stored in the voice data generation unit by word units through the voice data extraction program unit, and converts the voice data extracted from the sign language animation conversion program unit into an actual sign language animation. Sign language and subtitle reception method using a digital composite receiver. The method of claim 7, further comprising: displaying a sign language video and a sign language animation screen output to PIP screen data through a PIP output unit 240, and a TV image having a size of 16: 9 or 4: 3 on a display (S180). ) The first tuner 251 tunes a channel so that a sign language video and a sign language animation screen are displayed on a main screen among high frequency signals received through the antenna unit 210, and amplifies a composite video signal of the tuned channel. And amplifying the intermediate frequency (IF) which is easy to amplify by mixing with the oscillation frequency oscillated by the channel being tuned (S181), Converting the channel into an intermediate frequency that is easy to tune and amplify so that the TV video screen is displayed on the sub-screen among the high frequency signals received through the antenna unit 210 in the second tuner unit 252 (S182); Amplifying a sign language video signal, a sign language animation signal, and a TV image signal applied from the first tuner unit 251 and the second tuner unit 252 by the IF amplifier 253 at an intermediate frequency (S183); , The PIP mode switching unit 254 switches the sign language video signal, the sign language animation signal, and the TV video signal, which are applied by amplifying the intermediate frequency from the IF amplifier 253 to the PIP mode according to the control signal of the microcomputer 259 and outputs the PIP mode. Step S184, The PIP signal processor 256 receives a sign language video signal, a sign language animation signal, and a TV video signal from the PIP mode switching unit 254, respectively, and superimposes a sign language video signal on a TV video signal, or superimposes a sign language animation signal on a TV video signal. PIP processing to output (S185) and After receiving the TV video signal from the PIP signal processing unit 256 through the sub-screen size control unit 257, adjusting the size of the PIP screen according to the control signal of the microcomputer unit 259 (S186), After receiving the TV video signal consisting of the PIP screen from the PIP signal processing unit 256 through the PIP position control unit 258, and setting the position of the PIP screen according to the control signal of the microcomputer unit (259) (S187) Receiving and subtitle receiving method through a digital composite receiver, characterized in that.

Images