JP2013176141A - Device and method for receiving stereoscopic image data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To maintain compatibility of perspective with each object in an image in display of superposition information.SOLUTION: An image data reception section receives stereoscopic image data including left eye image data and right eye image data. An image data processing section gives parallax to the same superposition information superposed on a left eye image and a right eye image on the basis of parallax information of the left eye image or the right eye image to the other image. The parallax information is obtained by processing the left eye image data and the right eye image data, which are included in the stereoscopic image data received by the image data reception section. The image data processing section obtains the left eye image data on which the superposition information is superposed and the right eye image data on which the superposition information is superposed.

Description

  The present invention relates to a stereoscopic image data receiving apparatus and a stereoscopic image data receiving method, and more particularly to a stereoscopic image data transmission method and the like that can favorably display superimposition information such as graphics information and text information.

  For example, Patent Document 1 proposes a transmission method that uses a television broadcast radio wave of stereoscopic image data. In this case, stereoscopic image data including left-eye image data and right-eye image data is transmitted, and stereoscopic image display using binocular parallax is performed in the television receiver.

  FIG. 42 shows the relationship between the display position of the left and right images of an object (object) on the screen and the playback position of the stereoscopic image in stereoscopic image display using binocular parallax. For example, with respect to the object A in which the left image La is displayed on the right side and the right image Ra is shifted to the left side as shown in the figure on the screen, the right and left line of sight intersects in front of the screen surface. The position is in front of the screen surface.

  Further, for example, with respect to the object B in which the left image Lb and the right image Rb are displayed at the same position as shown in the figure on the screen, the right and left lines of sight intersect on the screen surface. It becomes on the surface. Further, for example, with respect to the object C displayed on the screen as shown in the figure, the left image Lc is shifted to the left side and the right image Rc is shifted to the right side, the right and left lines of sight intersect at the back of the screen surface. The playback position is behind the screen.

Japanese Patent Laid-Open No. 2005-6114

  As described above, in stereoscopic image display, a viewer usually recognizes the perspective of a stereoscopic image using binocular parallax. Superimposition information superimposed on an image, such as graphics information and text information, is expected to be rendered in conjunction with stereoscopic image display not only in a two-dimensional space but also in a three-dimensional sense of depth.

  For example, when subtitles, which are graphics information, are superimposed on an image (overlay display), the viewer will be inconsistent with perspective unless it is displayed in front of the closest object (object) in the perspective. You may feel In addition, when other graphics information or text information is superimposed on an image, it is expected that parallax adjustment is performed according to the perspective of each object in the image to maintain the consistency of perspective.

  An object of the present invention is to maintain perspective consistency with each object in an image when displaying superimposition information such as graphics information and text information.

The concept of this invention is
An image data receiving unit for receiving stereoscopic image data including left eye image data and right eye image data;
Based on parallax information of the other of the left eye image and the right eye image obtained by processing the left eye image data and the right eye image data included in the stereoscopic image data received by the image data receiving unit Then, parallax is given to the same superimposition information superimposed on the left eye image and the right eye image, and data of the left eye image on which the superposition information is superimposed and data on the right eye image on which the superposition information is superimposed The stereoscopic image data receiving device includes an image data processing unit to be obtained.

  In the present invention, the image data receiving unit receives stereoscopic image data including left eye image data and right eye image data. Further, the image data processing unit assigns parallax to the same superimposition information to be superimposed on the left eye image and the right eye image based on the other parallax information with respect to one of the left eye image and the right eye image. This disparity information is obtained by processing the left eye image data and the right eye image data included in the stereoscopic image data received by the image data receiving unit.

  For example, the parallax information is received by the parallax information receiving unit in synchronization with the stereoscopic image data received by the image data receiving unit. In this case, it is not necessary to obtain parallax information based on the left eye image data and right eye image data included in the stereoscopic image data received by the image data receiving unit, and the processing on the receiving side is simplified. For example, the parallax information is obtained by a parallax information acquisition unit. In this disparity information acquisition unit, one of the left eye image and the right eye image is determined at a predetermined position in the image based on the left eye image data and the right eye image data included in the stereoscopic image data received by the image data receiving unit. The other parallax information is obtained. In this case, processing using the parallax information is possible even if parallax information is not sent.

  In addition, the image data processing unit obtains left-eye image data on which superimposition information is superimposed and right-eye image data on which superimposition information is superimposed. For example, the image data transmission unit transmits stereoscopic image data including left eye image data and right eye image data obtained by the image data processing unit to an external device. Further, for example, the image display unit displays an image for stereoscopic image display using the left eye image data and the right eye image data obtained by the image data processing unit.

  Thus, parallax is given to the same superimposition information to be superimposed on the left eye image and the right eye image based on the other parallax information with respect to one of the left eye image and the right eye image. Therefore, as the same superimposition information superimposed on the left eye image and the right eye image, information on which parallax adjustment is performed according to the perspective of each object in the image can be used. It is possible to maintain perspective consistency with each of the objects inside.

  In the present invention, for example, the image data processing unit may add a parallax corresponding to the superimposition position of the superimposition information to the same superimposition information superimposed on the left eye image and the right eye image. In this case, since the parallax corresponding to the superimposition position is given to each superimposition information, for example, it is possible to give the superimposition information a perspective equivalent to the perspective of the object existing at the superposition position.

  In the present invention, for example, a multi-channel speaker and a control unit that controls the multi-channel speaker output based on the other parallax information with respect to one of the left-eye image data and the right-eye image data are further provided. May be. In this case, the stereoscopic effect can be further emphasized.

  According to the present invention, as the same superimposition information superimposed on the left eye image and the right eye image, information on which parallax adjustment is performed according to the perspective of each object in the image can be used. It is possible to maintain perspective consistency in display.

It is a block diagram which shows the structural example of the stereo image display system as embodiment of this invention. It is a block diagram which shows the structural example of the transmission data generation part in a broadcast station. It is a figure which shows the image data of a pixel format of 1920 * 1080p. It is a figure for demonstrating the "Top & Bottom" system, the "Side By Side" system, and the "Frame Sequential" system which are the transmission systems of stereo image data (3D image data). It is a figure for demonstrating the example which detects the parallax vector of the right eye image with respect to a left eye image. It is a figure for demonstrating calculating | requiring a parallax vector by a block matching system. It is a figure which shows an example of the parallax vector VV in the predetermined position in an image detected by the parallax vector detection part. It is a figure which shows the transmission content of a parallax vector. It is a figure which shows the transmission content of the example of a parallax detection block, and the parallax vector in that case. It is a figure for demonstrating the example of the timing which detects and transmits a parallax vector. It is a figure for demonstrating the example of the timing which detects and transmits a parallax vector. It is a figure which shows the example of a stream of each data multiplexed in a transmission data generation part. It is a block diagram which shows the other structural example of the transmission data generation part in a broadcast station. It is a figure for demonstrating the superimposition position etc. of the left eye graphics information and right eye graphics information in case a transmission system is a 1st transmission system ("Top & Bottom" system). It is a figure for demonstrating the production | generation method of left eye graphics information and right eye graphics information in case a transmission system is a 1st transmission system ("Top & Bottom" system). It is a figure for demonstrating the production | generation method of the left eye graphics information and right eye graphics information in case a transmission system is a 2nd transmission system ("Side By Side" system). It is a figure for demonstrating the production | generation method of the left eye graphics information and right eye graphics information in case a transmission system is a 2nd transmission system ("Side By Side" system). It is a block diagram which shows the other structural example of the transmission data generation part in a broadcast station. It is a figure which shows the superimposition position of the left eye graphics information and right eye graphics information in case a transmission system is a 2nd transmission system ("Side By Side" system). It is a figure which shows the state which superimposed the graphics image by the graphics data transmitted by the conventional method extracted from the bit stream data as it is with respect to the left eye image and the right eye image. It is a figure which shows the disparity vector (View Vector) of three object positions in time T0, T1, T2, T3. It is a figure which shows the example of a subtitle (graphics information) display on an image, and the perspective of a background, a foreground object, and a subtitle. It is a figure which shows the example of a display of the caption (graphics information) on an image, and the left eye graphics information LGI and the right eye graphics information RGI for displaying a caption. It is a figure for demonstrating using the thing corresponding to the superimposition position among the parallax vectors detected in the several position in an image as a parallax vector. It is a figure which shows superimposing the text information which shows each object of A, B, and C in the image and the annotation of each object on the vicinity position of each of these objects. It is a block diagram which shows the structural example of a set top box. It is a block diagram which shows the structural example of the bit stream process part which comprises a set top box. It is a figure which shows the speaker output control example in case the parallax vector VV1 is larger in the left video object toward a television display. It is a block diagram which shows the other structural example of the bit stream process part which comprises a set top box. It is a block diagram which shows the other structural example of the bit stream process part which comprises a set top box. It is a figure which shows the structural example of a television receiver. It is a block diagram which shows the structural example of an HDMI transmission part (HDMI source) and an HDMI receiving part (HDMI sink). It is a block diagram which shows the structural example of the HDMI transmitter which comprises an HDMI transmission part, and the HDMI receiver which comprises an HDMI receiving part. It is a figure which shows the example of a structure of TMDS transmission data (when the horizontal * length is 1920 pixels * 1080 line image data is transmitted). It is a figure which shows the pin arrangement (type A) of the HDMI terminal to which the HDMI cable of the source device and the sink device is connected. It is a figure which shows the example of TMDS transmission data of a 1st transmission system ("Top & Bottom" system). It is a figure which shows the example of TMDS transmission data of a 2nd transmission system ("Side By Side" system). It is a figure which shows the example of TMDS transmission data of a 3rd transmission system ("Frame Sequential" system). It is a figure for demonstrating the "Frame Sequential" system in HDMI 1.4 (New HDMI) and the "Frame Sequential" system in HDMI 1.3 (Legacy HDMI). It is a block diagram which shows the other structural example of the bit stream process part which comprises a set top box. It is a figure which shows the other structural example of a stereo image display system. In a stereoscopic image display using binocular parallax, it is a figure which shows the relationship between the display position of the left-right image of the object on a screen, and the reproduction | regeneration position of the stereoscopic image.

Hereinafter, modes for carrying out the invention (hereinafter referred to as “embodiments”) will be described. The description will be given in the following order.
1. 1. First embodiment Modified example

<1. Embodiment>
[Configuration example of stereoscopic image transmission / reception system]
FIG. 1 shows a configuration example of a stereoscopic image transmission / reception system 10 as an embodiment. The stereoscopic image transmission / reception system 10 includes a broadcasting station 100, a set top box (STB) 200, and a television receiver 300.

  The set top box 200 and the television receiver 300 are connected via an HDMI (High Definition Multimedia Interface) cable 400. The set top box 200 is provided with an HDMI terminal 202. The television receiver 300 is provided with an HDMI terminal 302. One end of the HDMI cable 400 is connected to the HDMI terminal 202 of the set top box 200, and the other end of the HDMI cable 400 is connected to the HDMI terminal 302 of the television receiver 300.

[Description of broadcasting station]
The broadcast station 100 transmits bit stream data on a broadcast wave. This bit stream data includes stereoscopic image data including left-eye image data and right-eye image data, audio data, graphics data, text data, and a disparity vector as disparity information.

  FIG. 2 shows a configuration example of the transmission data generation unit 110 that generates the above-described bit stream data in the broadcast station 100. This configuration example is an example in which a disparity vector is transmitted as numerical information. The transmission data generation unit 110 includes cameras 111L and 111R, a video framing unit 112, a video encoder 113, a video encoder 113, a parallax vector detection unit 114, and a parallax vector encoder 115. The transmission data generation unit 110 includes a microphone 116, an audio encoder 117, a graphics generation unit 118, a graphics encoder 119, a text generation unit 120, a text encoder 121, and a multiplexer 122.

  The camera 111L captures a left eye image and obtains left eye image data for stereoscopic image display. The camera 111R captures the right eye image and obtains right eye image data for stereoscopic image display. The video framing unit 112 processes the left eye image data obtained by the camera 111L and the right eye image data obtained by the camera 111R into a state corresponding to the transmission method.

[Example of transmission method of stereoscopic image data]
Here, the following first to third methods are listed as transmission methods for stereoscopic image data (3D image data), but other transmission methods may be used. Here, as shown in FIG. 3, the case where the image data of the left eye (L) and the right eye (R) is image data of a predetermined resolution, for example, a pixel format of 1920 * 1080p will be described as an example. To do.

  The first transmission method is a “Top & Bottom” method, as shown in FIG. 4A, in which the data of each line of left-eye image data is transmitted in the first half in the vertical direction, and the left eye in the second half in the vertical direction. This is a method for transmitting data of each line of image data. In this case, since the lines of the left eye image data and the right eye image data are thinned out to ½, the vertical resolution is halved with respect to the original signal.

  The second transmission method is the “Side By Side” method, and as shown in FIG. 4B, the pixel data of the left eye image data is transmitted in the first half in the horizontal direction, and the right eye image data in the second half in the horizontal direction. This is a method for transmitting pixel data. In this case, in the left eye image data and the right eye image data, the pixel data in the horizontal direction is thinned out to 1/2. For the current signal, the horizontal resolution is halved.

  The third transmission method is a “Frame Sequential” method, in which left-eye image data and right-eye image data are sequentially switched and transmitted for each field as shown in FIG.

  Returning to FIG. 2, the video encoder 113 performs compression encoding such as MPEG4-AVC or MPEG2 on the stereoscopic image data processed by the video framing unit 112 to generate a video elementary stream. Further, the video encoder 113 divides the elementary stream of the video to generate a video PES (Packetized Elementary Stream) packet, and finally generates a video TS (Transport Stream) packet. Alternatively, it is multiplexed in a file container such as MP4, or generated so that a packet for real time is transmitted as a container.

  The disparity vector detection unit 114 detects a disparity vector that is disparity information of the other of the left eye image and the right eye image at a predetermined position in the image based on the left eye image data and the right eye image data. Here, the predetermined position in the image is all pixel positions, a representative position of each area composed of a plurality of pixels, or a representative position of an area where graphic information or text information is superimposed.

[Disparity vector detection]
A detection example of a disparity vector will be described. Here, an example in which the parallax vector of the right eye image with respect to the left eye image is detected will be described. As shown in FIG. 5, the left eye image is a detected image, and the right eye image is a reference image. In this example, the disparity vectors at the positions (xi, yi) and (xj, yj) are detected.

  A case where a disparity vector at the position (xi, yi) is detected will be described as an example. In this case, for example, an 8 * 8 or 16 * 16 pixel block (parallax detection block) Bi is set in the left eye image with the pixel at the position (xi, yi) at the upper left. Then, a pixel block matching the pixel block Bi is searched in the right eye image.

  In this case, a search range centered on the position (xi, yi) is set in the right-eye image, and each pixel in the search range is sequentially set as a pixel of interest, for example, 8 * 8 similar to the pixel block Bi described above. Alternatively, 16 * 16 comparison blocks are sequentially set.

  Between the pixel block Bi and the sequentially set comparison blocks, the sum of absolute difference values for each corresponding pixel is obtained. Here, as shown in FIG. 6, when the pixel value of the pixel block Bi is L (x, y) and the pixel value of the comparison block is R (x, y), the pixel block Bi, a certain comparison block, The sum of absolute differences between the two is represented by Σ | L (x, y) −R (x, y) |.

  When n pixels are included in the search range set in the right eye image, n total sums S1 to Sn are finally obtained, and the minimum sum Smin is selected. Then, the position of the upper left pixel (xi ′, yi ′) is obtained from the comparison block from which the sum Smin is obtained. Thereby, the disparity vector at the position (xi, yi) is detected as (xi′−xi, yi′−yi). Although detailed description is omitted, for the disparity vector at the position (xj, yj), for example, an 8 * 8 or 16 * 16 pixel block in the left-eye image with the pixel at the position (xj, yj) at the upper left Bj is set and detected in the same process.

  FIG. 7A shows an example of the parallax vector VV detected by the parallax vector detection unit 114 at a predetermined position in the image. In this case, as shown in FIG. 7B, if the left eye image (detected image) is shifted by the parallax vector VV at a predetermined position in the image, it means that it overlaps with the right eye image (reference image). .

  Returning to FIG. 2, the disparity vector encoder 115 generates an elementary stream of disparity vectors including the disparity vector detected by the disparity vector detection unit 114. The disparity vector elementary stream constitutes a TS packet together with a video elementary stream and the like.

  Here, the elementary stream of disparity vectors includes the following contents. That is, the ID (ID_Block) of the parallax detection block, the vertical position information (VerticaL_Position) of the parallax detection block, the horizontal position information (Horizontal_Position) of the parallax detection block, and the parallax vector (View_Vector) are set as one set. This set is repeated for the number N of parallax detection blocks. FIG. 8 shows the transmission content of the disparity vector.

  Note that the vertical and horizontal positions of the parallax detection block are offset values in the vertical and horizontal directions from the upper left origin of the image to the upper left pixel of the block. The reason why the ID of the parallax detection block is attached to the transmission of each parallax vector is to allow a link to a superimposition information pattern such as graphics information and text information to be superimposed on the image.

  For example, as shown in FIG. 9A, when there are parallax detection blocks A to F, the transmission contents include the IDs of the parallax detection blocks A to F as shown in FIG. , Vertical and horizontal position information, and disparity vectors. For example, in FIG. 9B, regarding the disparity detection block A, ID2 indicates the ID of the disparity detection block A, (Ha, Va) indicates the vertical and horizontal position information of the disparity detection block A, and the disparity vector a Indicates a disparity vector of the disparity detection block A.

  Here, the timing for detecting and transmitting the parallax vector will be described. Regarding this timing, for example, the following first to fourth examples can be considered.

  In the first example, as shown in FIG. 10A, synchronization with picture coding is performed. In this case, the disparity vector is transmitted in units of pictures. This picture unit is the finest unit for transmitting a disparity vector. In the second example, as shown in FIG. 10B, it is synchronized with a video scene. In this case, the disparity vector is transmitted in scene units.

  In the third example, as shown in FIG. 10 (c), it is synchronized with the I picture (Intra picture) of the encoded video. In the fourth example, as shown in FIG. 11, the display is synchronized with the display start timing of graphics information, text information, and the like superimposed on the image.

  Returning to FIG. 2, the microphone 116 detects sound corresponding to the images photographed by the cameras 111 </ b> L and 111 </ b> R, and obtains sound data. The audio encoder 117 performs compression encoding such as MPEG-2 Audio AAC on the audio data obtained by the microphone 116 to generate an audio elementary stream. Further, the audio encoder 117 divides the audio elementary stream to generate an audio PES packet, and finally generates a TS packet.

  The graphics generation unit 118 generates graphics information data (graphics data) to be superimposed on the image. The graphics information is, for example, captions. This graphics data is bitmap data. The graphics data is added with idling offset information indicating the superimposed position on the image. The idling offset information indicates, for example, offset values in the vertical direction and the horizontal direction from the upper left origin of the image to the upper left pixel of the superimposed position of the graphics information. The standard for transmitting caption data as bitmap data is standardized and operated as DVB_Subtitling in DVB, which is a European digital broadcasting standard.

  The graphics encoder 119 generates an elementary stream of graphics data generated by the graphics generation unit 118. Then, the graphics encoder 119 finally generates the above-described TS packet.

  The text generator 120 generates text information data (text data) to be superimposed on the image. The text information is, for example, an electronic program guide or text broadcast content. As with the above-described graphics data, idling offset information indicating the superimposed position on the image is added to the text data. This idling offset information indicates, for example, offset values in the vertical and horizontal directions from the upper left origin of the image to the upper left pixel of the text information overlapping position. Examples of text data transmission include EPG used as a program reservation and CC_data (Closed Caption) of the American digital terrestrial standard ATSC.

  The text encoder 121 generates an elementary stream of text data generated by the text generator 120.

  The multiplexer 122 multiplexes packetized elementary streams output from the encoders of the video encoder 113, the disparity vector encoder 115, the audio encoder 117, the graphics encoder 119, and the text encoder 121. The multiplexer 122 outputs bit stream data (transport stream) BSD as transmission data.

  The operation of the transmission data generation unit 110 shown in FIG. 2 will be briefly described. The camera 111L captures a left eye image. The left eye image data for stereoscopic image display obtained by the camera 111L is supplied to the video framing unit 112. In addition, the camera 111R captures a right eye image. Right-eye image data for stereoscopic image display obtained by the camera 111R is supplied to the video framing unit 112. In the video framing unit 112, the left-eye image data and the right-eye image data are processed into a state corresponding to the transmission method to obtain stereoscopic image data (see FIGS. 4A to 4C).

  Stereoscopic image data obtained by the video framing unit 112 is supplied to the video encoder 113. In the video encoder 113, compression encoding such as MPEG4-AVC or MPEG2 is performed on the stereoscopic image data, a video elementary stream is generated, and finally the video packet is supplied to the multiplexer 122.

  Further, the left eye image data and the right eye image data obtained by the cameras 111L and 111R are supplied to the parallax vector detection unit 114 through the video framing unit 112. In the disparity vector detection unit 114, a disparity detection block is set at a predetermined position in the image based on the left eye image data and the right eye image data, and is the other disparity information for one of the left eye image and the right eye image. A disparity vector is detected.

  The disparity vector at a predetermined position in the image detected by the disparity vector detection unit 114 is supplied to the disparity vector encoder 115. In this case, the ID of the parallax detection block, the vertical position information of the parallax detection block, the horizontal position information of the parallax detection block, and the parallax vector are passed as one set. In the disparity vector encoder 115, an elementary stream of disparity vectors including disparity vector transmission content (see FIG. 8) is generated and supplied to the multiplexer 122.

  Further, the microphone 116 detects sound corresponding to the images photographed by the cameras 111L and 111R. Audio data obtained by the microphone 116 is supplied to the audio encoder 117. The audio encoder 117 performs compression encoding such as MPEG-2 Audio AAC on the audio data, generates an audio elementary stream, and supplies it to the multiplexer 122.

  Further, the graphics generator 118 generates graphics information data (graphics data) to be superimposed on the image. This graphics data (bitmap data) is supplied to the graphics encoder 119. The graphics data is added with idling offset information indicating the superimposed position on the image. In the graphics encoder 119, predetermined compression coding is performed on the graphics data to generate an elementary stream, which is supplied to the multiplexer 122.

  The text generator 120 generates text information data (text data) to be superimposed on the image. This text data is supplied to the text encoder 121. As with the above-described graphics data, idling offset information indicating the superimposed position on the image is added to the text data. In the text encoder 121, the text data is subjected to predetermined compression encoding to generate an elementary stream, and finally a text TS packet is obtained. The text TS packet is supplied to the multiplexer 122.

  The multiplexer 122 multiplexes the elementary stream packets supplied from the encoders to obtain bit stream data (transport stream) BSD as transmission data.

  FIG. 12 illustrates an example of each data stream multiplexed in the transmission data generation unit 110 illustrated in FIG. This example shows a case where a disparity vector is detected and transmitted in units of video scenes (see FIG. 10B). Each stream packet is given a time stamp for synchronous display, and the reception side can control the superimposition timing of graphics information, text information, and the like on the image.

  2 is configured to transmit disparity vector transmission content (see FIG. 8) to the receiving side as an independent elementary stream. However, it is also conceivable that the transmission content of the disparity vector is embedded in another stream for transmission. For example, the transmission content of the disparity vector is embedded and transmitted as user data in a video stream. Also, for example, the transmission content of the disparity vector is transmitted by being embedded in a graphics or text stream.

  FIG. 13 illustrates a configuration example of the transmission data generation unit 110A. This example is also an example in which a disparity vector is transmitted as numerical information. This transmission data generation unit 110A is configured to transmit the transmission content of the disparity vector by embedding it in the video stream as user data. In FIG. 13, parts corresponding to those in FIG. 2 are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the transmission data generation unit 110A, a stream framing unit 123 is inserted between the video encoder 113 and the multiplexer 122. The disparity vector at a predetermined position in the image detected by the disparity vector detection 114 is supplied to the stream framing unit 123. In this case, the ID of the parallax detection block, the vertical position information of the parallax detection block, the horizontal position information of the parallax detection block, and the parallax vector are passed as one set.

  In the stream framing unit 123, the transmission content of the disparity vector (see FIG. 8) is embedded as user data in the video stream.

  Although the detailed description is omitted, the transmission data generation unit 110A shown in FIG. 13 is otherwise configured in the same manner as the transmission data generation unit 110 shown in FIG.

  Also, the transmission data generation unit 110 shown in FIG. 2 and the transmission data generation unit 110A shown in FIG. 13 transmit the disparity vector as numerical information (see FIG. 8). However, instead of transmitting the parallax vector as numerical information, it is also conceivable to transmit it by including it in data of superimposition information (for example, graphics information, text information, etc.) to be superimposed on the image.

  For example, when transmitting by including the graphics information data, the transmission side generates graphics data corresponding to both the left-eye graphics information to be superimposed on the left-eye image and the right-eye graphics information to be superimposed on the right-eye image. The In this case, the left eye graphics information and the right eye graphics information are the same graphics information. However, the display position in the image is shifted in the horizontal direction by, for example, the horizontal component of the parallax vector corresponding to the display position of the right-eye graphics information with respect to the left-eye graphics information.

  In addition, for example, when transmitting by including in text information data, on the transmitting side, text data corresponding to both left eye text information to be superimposed on the left eye image and right eye text information to be superimposed on the right eye image is provided. Generated. In this case, the left eye text information and the right eye text information are the same text information. However, the superimposed position in the image is shifted in the horizontal direction by the horizontal component of the parallax vector, for example, with respect to the left-eye text information.

  For example, as the parallax vector, a parallax vector detected at a plurality of positions in the image corresponding to the superimposed position is used. Further, for example, as the parallax vector, a parallax vector at a position that is recognized closest in terms of perspective among parallax vectors detected at a plurality of positions in the image is used.

  FIG. 14A shows the superimposed positions of left-eye graphics information and right-eye graphics information when the transmission method is the above-described first transmission method (“Top & Bottom” method). These left-eye graphics information and right-eye graphics information are the same information. However, with respect to the left eye graphics information LGI superimposed on the left eye image IL, the right eye graphics information RGI superimposed on the right eye image IR is shifted in the horizontal direction by the horizontal component VVT of the parallax vector. It is said that.

  As shown in FIG. 14A, graphics data is generated so that the graphics information LGI and RGI are superimposed on the images IL and IR. Accordingly, as shown in FIG. 14B, the viewer can observe the graphics information LGI and RGI together with the images IL and IR with parallax, and can recognize the perspective in the graphics information.

  For example, the graphics data of each graphics information LGI, RGI is generated as single area data as shown in FIG. In this case, the data other than the graphics information LGI and RGI may be generated as transparent data. Further, for example, the graphics data of each graphics information LGI, RGI is generated as data of another area as shown in FIG.

  FIG. 16A shows the superimposed positions of left-eye graphics information and right-eye graphics information when the transmission method is the above-described second transmission method (“Side By Side” method). These left-eye graphics information and right-eye graphics information are the same information. However, with respect to the left eye graphics information LGI superimposed on the left eye image IL, the right eye graphics information RGI superimposed on the right eye image IR is shifted in the horizontal direction by the horizontal component VVT of the parallax vector. It is said that. IT is an idling offset value.

  As shown in FIG. 16A, graphics data is generated so that the graphics information LGI and RGI are superimposed on the images IL and IR. Thus, as shown in FIG. 16B, the viewer can observe the graphics information LGI and RGI together with the images IL and IR with parallax, and can recognize the perspective in the graphics information.

  For example, the graphics data of each graphics information LGI and RGI is generated as single area data as shown in FIG. In this case, the data other than the graphics information LGI and RGI may be generated as transparent data.

  FIG. 18 illustrates a configuration example of the transmission data generation unit 110B. This transmission data generation unit 110B is configured to transmit a disparity vector included in data of graphics information and text information. In FIG. 18, the same reference numerals are given to the portions corresponding to those in FIG.

  In the transmission data generation unit 110B, a graphics processing unit 124 is inserted between the graphics generation unit 118 and the graphics encoder 119. In the transmission data generation unit 110B, a text processing unit 125 is inserted between the text generation unit 120 and the text encoder 121. Then, the parallax vector at a predetermined position in the image detected by the parallax vector detection 114 is supplied to the graphics processing unit 124 and the text processing unit 125.

  In the graphics processing unit 124, based on the graphics data generated by the graphics generating unit 118, the data of the left eye graphics information LGI superimposed on the left eye image IL and the right eye graphics information superimposed on the right eye image IR. RGI data is generated. In this case, the left-eye graphics information and the right-eye graphics information are the same graphics information, but the superimposed position in the image is, for example, the left-eye graphics information, and the right-eye graphics information is the horizontal component of the disparity vector. Only VVT is shifted in the horizontal direction (see FIGS. 14A and 16A).

  Thus, the graphics data generated by the graphics processing unit 124 is supplied to the graphics encoder 119. Note that idling offset information indicating the superimposed position on the image is added to the graphics data. In the graphics encoder 119, an elementary stream of graphics data generated by the graphics processing unit 124 is generated.

  Also, the text processing unit 125, based on the text data generated by the text generation unit 120, the left-eye text information data superimposed on the left-eye image and the right-eye text information superimposed on the right-eye image. Data is generated. In this case, the left-eye text information and the right-eye text information are the same text information, but the superimposed position in the image is, for example, the left-eye text information, and the right-eye text information is the horizontal component of the disparity vector. Only VVT is shifted in the horizontal direction.

  Thus, the text data generated by the text processing unit 125 is supplied to the text encoder 121. Note that idling offset information indicating the superimposed position on the image is added to the text data. The text encoder 121 generates an elementary stream of text data generated by the text processing unit.

  Although detailed description is omitted, the other parts of the transmission data generation unit 110B shown in FIG. 18 are configured in the same manner as the transmission data generation unit 110 shown in FIG.

[Description of Set Top Box]
Returning to FIG. 1, the set-top box 200 receives bit stream data (transport stream) transmitted from the broadcast station 100 on a broadcast wave. This bit stream data includes stereoscopic image data including left-eye image data and right-eye image data, audio data, graphics data, text data, and a disparity vector as disparity information.

  The set top box 200 has a bit stream processing unit 201. The bit stream processing unit 201 extracts stereoscopic image data, audio data, graphics data, text data, disparity vectors, and the like from the bit stream data. In addition, the bit stream processing unit 201 generates left-eye image data and right-eye image data on which superimposition information is superimposed, using stereoscopic image data, graphics data, text data, and the like.

  Here, when the disparity vector is transmitted as numerical information, left-eye graphics information and right-eye graphics information to be superimposed on the left-eye image and the right-eye image are generated based on the disparity vector and the graphics data. In this case, the left eye graphics information and the right eye graphics information are the same graphics information. However, the superposition position in the image is shifted in the horizontal direction by the horizontal component of the parallax vector, for example, with respect to the left-eye graphics information.

  FIG. 19A shows the superimposed positions of left-eye graphics information and right-eye graphics information when the transmission method is the above-described second transmission method (“Side By Side” method).

  In contrast to the left-eye graphics information LGI superimposed on the left-eye image IL, the right-eye graphics information RGI superimposed on the right-eye image IR is set at a position shifted in the horizontal direction by the horizontal component VVT of the parallax vector. ing. IT is an idling offset value.

  As shown in FIG. 19A, graphics data is generated so that the graphics information LGI and RGI are superimposed on the images IL and IR.

  The bit stream processing unit 201 combines the generated left eye graphics data and right eye graphics data with the stereoscopic image data (left eye image data and right eye image data) extracted from the bit stream data, and performs processing Later stereo image data is acquired. According to this stereoscopic image data, as shown in FIG. 19B, the viewer can observe the graphics information LGI and RGI together with the images IL and IR with parallax, and the graphics information has a sense of perspective. It becomes possible to recognize.

  FIG. 20A shows a state in which graphics images based on graphics data extracted from the bit stream data are directly superimposed on the images IL and IR. In this case, as shown in FIG. 20B, the viewer observes the left half of the graphics information together with the left eye image IL and the right half of the graphics information together with the right eye image IR. Therefore, the graphics information cannot be recognized correctly.

  When the disparity vector is transmitted as numerical information, left-eye text information and right-eye text information to be superimposed on the left-eye image and right-eye image are generated based on the disparity vector and text data. In this case, the left eye text information and the right eye text information are the same text information. However, the superimposed position in the image is shifted in the horizontal direction by the horizontal component of the parallax vector, for example, with respect to the left-eye text information.

  The bit stream processing unit 201 generates left eye text information and right eye text information data (bitmap data) for the stereoscopic image data (left eye image data, right eye image data) extracted from the bit stream data. ) To obtain stereoscopic image data after processing. According to this stereoscopic image data, the viewer can observe each text information with parallax together with the left eye image and the right eye image as in the case of the above-described graphics information, and also recognizes perspective in the text information. It becomes possible.

  In this case, the following disparity vectors may be used as disparity vectors that give disparity between left-eye graphics information and right-eye graphics information, or between left-eye text information and right-eye text information.

  For example, as the disparity vector, it is conceivable to use the disparity vector at the closest recognized position in terms of perspective among the disparity vectors detected at a plurality of positions in the image. FIGS. 21A, 21B, 21C, and 21D show disparity vectors (view vectors) at three object positions at times T0, T1, T2, and T3, respectively.

  At time T0, the disparity vector VV0-1 at the position (H0, V0) corresponding to the object 1 is the maximum disparity vector MaxVV (T0). At time T1, the disparity vector VV1-1 at the position (H1, V1) corresponding to the object 1 is the maximum disparity vector MaxVV (T1). At time T2, the parallax vector VV2-2 at the position (H2, V2) corresponding to the object 2 is the maximum parallax vector MaxVV (T2). At time T3, the parallax vector VV3-0 at the position (H3, V3) corresponding to the object 1 is the maximum parallax vector MaxVV (T3).

  Thus, by using the parallax vector at the closest recognized position among the parallax vectors detected at a plurality of positions in the image as the parallax vector, Graphics information and text information can be displayed in front of an object (object) in a close image.

  FIG. 22A shows a display example of captions (graphics information) on an image. In this display example, captions are superimposed on an image composed of a background and a foreground object. FIG. 22B shows the perspective of the background, the foreground object, and the caption, and indicates that the caption is recognized as being closest.

  FIG. 23A shows a display example of captions (graphics information) on an image, which is the same as FIG. FIG. 23B shows left-eye graphics information LGI and right-eye graphics information RGI for displaying a caption. FIG. 23C shows that disparity is given to each piece of graphics information LGI and RGI in order to recognize that the caption is closest.

  Further, as the parallax vector, it is conceivable to use a parallax vector detected at a plurality of positions in the image corresponding to the superimposed position. FIG. 24A shows graphic information based on graphic data extracted from bit stream data and text information based on text data extracted from bit stream data.

  FIG. 24B shows a state in which the left eye graphics information LGI and the left eye text information LTI are superimposed on the left eye image. In this case, in the left eye graphics information LGI, the superimposition position thereof is regulated by the idling offset value (IT-0) in the horizontal direction. Further, the left eye text information LTI has its superposition position regulated in the horizontal direction by an idling offset value (IT-1).

  FIG. 24C shows a state where the right eye graphics information RGI and the right eye text information RTI are superimposed on the right eye image. In this case, in the right eye graphics information RGI, the superimposition position is restricted by the idling offset value (IT-0) in the horizontal direction, and the horizontal component VVT-0 of the parallax vector corresponding to this superposition position is also set to the left eye. It is shifted from the superimposed position of the graphics information LGI. The right-eye text information RTI has its superposition position restricted by an idling offset value (IT-1) in the horizontal direction, and the left-eye text corresponding to the horizontal component VVT-1 of the parallax vector corresponding to this superposition position. It is shifted from the superimposed position of the information LTI.

  In the above description, a case has been described in which graphics information based on graphics data extracted from bit stream data or text information based on text data extracted from bit stream data is superimposed on the left eye image and right eye image. In addition, it is also conceivable that graphics data or text data is generated in the set-top box 200, and information based thereon is superimposed on the left eye image and the right eye image.

  Even in that case, using the disparity vector at a predetermined position in the image extracted from the bit stream data, between the left eye graphics information and the right eye graphics information, or the left eye text information and the right eye text information A parallax can be given between the two. Thereby, in the display of graphics information and text information, it is possible to give an appropriate perspective that maintains the consistency of perspective with the perspective of each object (object) in the image.

  FIG. 25A shows that the objects A, B, and C exist in the image, and, for example, text information indicating the annotation of each object is superimposed on the vicinity of each object.

  FIG. 25B shows the positions of the objects A, B, and C, the disparity vector list indicating the correspondence between the disparity vectors at the positions, the disparity vectors, and the annotations of the objects A, B, and C. It shows that it is used when parallax is given to the text information shown. For example, the text information “Text” is superimposed in the vicinity of the object A, but the disparity vector at the position (Ha, Va) of the object A is between the left-eye text information and the right-eye text information. Parallax corresponding to VV-a is given. The same applies to the text information superimposed in the vicinity of the B and C objects.

  Next, a case where a disparity vector is transmitted by being included in graphics information and text information data will be described. In this case, the graphics data extracted from the bit stream data includes data of left-eye graphics information and right-eye graphics information to which disparity is given by a disparity vector. Similarly, the text data extracted from the bitstream data includes data of left-eye text information and right-eye text information to which disparity is given by a disparity vector.

  Therefore, the bit stream processing unit 201 simply combines the graphics data and text data extracted from the bit stream data with the stereoscopic image data (left eye image data and right eye image data) extracted from the bit stream data. Thus, the processed stereoscopic image data is acquired. Regarding text data, it is necessary to convert text data (code data) into bitmap data.

[Configuration example of set-top box]
A configuration example of the set top box 200 will be described. FIG. 26 shows a configuration example of the set top box 200. The set top box 200 includes a bit stream processing unit 201, an HDMI terminal 202, an antenna terminal 203, a digital tuner 204, a video signal processing circuit 205, an HDMI transmission unit 206, and an audio signal processing circuit 207. ing. The set top box 200 includes a CPU 211, a flash ROM 212, a DRAM 213, an internal bus 214, a remote control receiving unit 215, and a remote control transmitter 216.

  The antenna terminal 203 is a terminal for inputting a television broadcast signal received by a receiving antenna (not shown). The digital tuner 204 processes the television broadcast signal input to the antenna terminal 203 and outputs predetermined bit stream data (transport stream) corresponding to the user's selected channel.

  As described above, the bit stream processing unit 201 extracts stereoscopic image data (left-eye image data, right-eye image data), audio data, graphics data, text data, disparity vectors, and the like from the bit stream data. Then, as described above, the bit stream processing unit 201 synthesizes superimposition information (graphics information, text information) data with the stereoscopic image data, and acquires display stereoscopic image data. The bit stream processing unit 201 outputs audio data. The detailed configuration of the bit stream processing unit 201 will be described later.

  The video signal processing circuit 205 performs image quality adjustment processing on the stereoscopic image data output from the bit stream processing unit 201 as necessary, and supplies the processed stereoscopic image data to the HDMI transmission unit 206. The audio signal processing circuit 207 performs sound quality adjustment processing or the like on the audio data output from the bit stream processing unit 201 as necessary, and supplies the processed audio data to the HDMI transmission unit 206.

  The HDMI transmission unit 206 transmits baseband image (video) and audio data from the HDMI terminal 202 by communication conforming to HDMI. In this case, since transmission is performed using the HDMI TMDS channel, image and audio data are packed and output from the HDMI transmission unit 206 to the HDMI terminal 202. Details of the HDMI transmission unit 206 will be described later.

  The CPU 211 controls the operation of each part of the set top box 200. The flash ROM 212 stores control software and data. The DRAM 213 constitutes a work area for the CPU 211. The CPU 211 develops software and data read from the flash ROM 212 on the DRAM 213 to activate the software, and controls each part of the set top box 200.

  The remote control receiving unit 215 receives a remote control signal (remote control code) transmitted from the remote control transmitter 216 and supplies it to the CPU 211. The CPU 211 controls each part of the set top box 200 based on the remote control code. The CPU 211, flash ROM 212 and DRAM 213 are connected to the internal bus 214.

  The operation of the set top box 200 will be briefly described. A television broadcast signal input to the antenna terminal 203 is supplied to the digital tuner 204. The digital tuner 204 processes the television broadcast signal and outputs predetermined bit stream data (transport stream) corresponding to the user's selected channel.

  The bit stream data output from the digital tuner 204 is supplied to the bit stream processing unit 201. The bit stream processing unit 201 extracts stereoscopic image data (left-eye image data, right-eye image data), audio data, graphics data, text data, disparity vectors, and the like from the bit stream data. Further, in the bit stream processing unit 201, the superimposition information (graphics information and text information) data is synthesized with the stereoscopic image data to generate display stereoscopic image data.

  The display stereoscopic image data generated by the bit stream processing unit 201 is supplied to the HDMI transmission unit 206 after image quality adjustment processing or the like is performed as necessary by the video signal processing circuit 205. Also, the audio data obtained by the bit stream processing unit 201 is supplied to the HDMI transmission unit 206 after the audio signal processing circuit 207 performs sound quality adjustment processing or the like as necessary. The stereoscopic image data and audio data supplied to the HDMI transmission unit 206 are transmitted from the HDMI terminal 202 to the HDMI cable 400 via the HDMI TMDS channel.

[Configuration example of bit stream processing unit]
A configuration example of the bit stream processing unit 201 will be described. FIG. 27 shows a configuration example of the bit stream processing unit 201. The bit stream processing unit 201 has a configuration corresponding to the transmission data generation unit 110 shown in FIG. The bit stream processing unit 201 includes a demultiplexer 220, a video decoder 221, a graphics decoder 222, a text decoder 223, an audio decoder 224, and a disparity vector decoder 225. The bit stream processing unit 201 includes a stereoscopic image graphics generation unit 226, a stereoscopic image text generation unit 227, a video superimposition unit 228, and a multi-channel speaker control unit 229.

  The demultiplexer 220 extracts TS packets of video, audio, disparity vector, graphics, and text from the bit stream data BSD, and sends them to each decoder.

  The video decoder 221 performs processing opposite to that of the video encoder 113 of the transmission data generation unit 110 described above. That is, the video decoder 221 reconstructs a video elementary stream from the video packet extracted by the demultiplexer 220, performs decoding processing, and generates stereoscopic image data including left-eye image data and right-eye image data. Get. The transmission method of the stereoscopic image data is, for example, the first transmission method (“Top & Bottom” method) described above, the second transmission method (“Side By Side” method), and the third transmission method (“Frame”). Sequential ”method) (see FIGS. 4A to 4C).

  The graphics decoder 222 performs processing opposite to that of the graphics encoder 119 of the transmission data generation unit 110 described above. That is, the graphics decoder 222 reconstructs a graphics elementary stream from the graphics packet extracted by the demultiplexer 220, performs decoding processing, and obtains graphics data.

  The text decoder 223 performs processing reverse to that of the text encoder 121 of the transmission data generation unit 110 described above. That is, the text decoder 223 reconstructs a text elementary stream from the text packet extracted by the demultiplexer 220 and performs a decoding process to obtain text data.

  The audio decoder 224 performs processing opposite to that of the audio encoder 117 of the transmission data generation unit 110 described above. That is, the audio decoder 224 reconstructs an audio elementary stream from the audio packet extracted by the demultiplexer 220, performs decoding processing, and obtains audio data.

  The disparity vector decoder 225 performs a process opposite to that of the disparity vector encoder 115 of the transmission data generation unit 110 described above. That is, the disparity vector decoder 225 reconstructs an elementary stream of disparity vectors from the disparity vector packets extracted by the demultiplexer 220, performs decoding processing, and obtains disparity vectors at predetermined positions in the image.

  Based on the graphics data obtained by the decoder 222 and the parallax vector obtained by the decoder 225, the stereoscopic image graphics generation unit 226 superimposes left-eye graphics information and right-eye graphics on the left-eye image and right-eye image, respectively. Generate information. In this case, the left-eye graphics information and the right-eye graphics information are the same graphics information, but the superimposed position in the image is, for example, the left-eye graphics information, and the right-eye graphics information is the horizontal component of the disparity vector. Only to be shifted horizontally. Then, the stereoscopic image graphics generation unit 226 outputs data (bitmap data) of the generated left eye graphics information and right eye graphics information.

  Based on the text data obtained by the decoder 223 and the parallax vector obtained by the decoder 225, the stereoscopic image text generating unit 227 superimposes the left-eye text information and the right-eye image to be superimposed on the left-eye image and the right-eye image, respectively. Generate text information. In this case, the left-eye text information and the right-eye text information are the same text information, but the superimposed position in the image is, for example, the left-eye text information, and the right-eye text information is the horizontal component of the disparity vector. Only to be shifted horizontally. Then, the stereoscopic image text generation unit 227 outputs data (bitmap data) of the generated left eye text information and right eye text information.

  The video superimposing unit 228 generates the data generated by the graphics generating unit 226 and the text generating unit 227 for the stereoscopic image data (left-eye image data and right-eye image data) obtained by the video decoder 221. The data is superimposed to obtain display stereoscopic image data Vout.

  The multi-channel speaker control unit 229 gives the sound data obtained by the audio decoder 224 a process for generating sound data of a multi-channel speaker for realizing 5.1ch surround, for example, and predetermined sound field characteristics. Apply processing. The multi-channel speaker control unit 229 controls the output of the multi-channel speaker based on the disparity vector obtained by the decoder 225.

  The greater the size of the parallax vector, the more effective the stereoscopic effect. By controlling the multi-channel speaker output according to the degree of 3D, it is possible to provide a further 3D experience.

  FIG. 28 shows an example of speaker output control when the parallax vector VV1 is larger for the left video object toward the television display. In this control example, the rear left speaker volume of the multi-channel speaker is increased, the front left speaker volume is medium, and the front right and rear right speaker volumes are decreased. In this way, by applying the disparity vector of the video content (stereoscopic image data) to other media data such as audio data on the receiving side, the viewer can experience a stereoscopic effect comprehensively. .

  The operation of the bit stream processing unit 201 shown in FIG. 27 will be briefly described. The bit stream data BSD output from the digital tuner 204 (see FIG. 26) is supplied to the demultiplexer 220. The demultiplexer 220 extracts TS packets of video, audio, disparity vector, graphics, and text from the bit stream data BSD, and supplies them to each decoder.

  In the video decoder 221, a video elementary stream is reconstructed from the video packet extracted by the demultiplexer 220, and further, decoding processing is performed, so that stereoscopic image data including left eye image data and right eye image data is obtained. can get. The stereoscopic image data is supplied to the video superimposing unit 228. In addition, the disparity vector decoder 225 reconstructs the disparity vector elementary stream from the disparity vector packet extracted by the demultiplexer 220 and further performs decoding processing to obtain a disparity vector at a predetermined position in the image. (See FIG. 8).

  In the graphics decoder 222, a graphics elementary stream is reconstructed from the graphics packet extracted by the demultiplexer 220, and further, decoding processing is performed to obtain graphics data. The graphics data is supplied to the stereoscopic image graphics generation unit 226. The stereoscopic image graphics generation unit 226 is also supplied with the disparity vector obtained by the disparity vector decoder 225.

  In this stereoscopic image graphics generation unit 226, based on the graphics data obtained by the decoder 222 and the parallax vector obtained by the decoder 225, left-eye graphics information and right-eye information to be superimposed on the left-eye image and the right-eye image, respectively. Graphics information is generated. In this case, the left-eye graphics information and the right-eye graphics information are the same graphics information, but the superimposed position in the image is, for example, the left-eye graphics information, and the right-eye graphics information is the horizontal component of the disparity vector. Only to be shifted horizontally. From the stereoscopic image graphics generation unit 226, data (bitmap data) of the generated left-eye graphics information and right-eye graphics information is output.

  Further, the text decoder 223 reconstructs a text elementary stream from the text TS packet extracted by the demultiplexer 220 and further performs a decoding process to obtain text data. This text data is supplied to the stereoscopic image text generator 227. The stereoscopic image text generation unit 227 is also supplied with the disparity vector obtained by the disparity vector decoder 225.

  In this stereoscopic image text generation unit 227, based on the text data obtained by the decoder 223 and the disparity vector obtained by the decoder 225, left-eye text information to be superimposed on the left-eye image and the right-eye image, Eye text information is generated. In this case, the left-eye text information and the right-eye text information are the same text information, but the superimposed position in the image is, for example, the left-eye text information, and the right-eye text information is the horizontal component of the disparity vector Only to be shifted horizontally. From the stereoscopic image text generating unit 227, data (bitmap data) of the generated left eye text information and right eye text information is output.

  In addition to the stereoscopic image data (left-eye image data and right-eye image data) from the video decoder 221, the video superimposing unit 228 is supplied with data output from the graphics generating unit 226 and the text generating unit 227. . In the video superimposing unit 228, the data generated by the graphics generating unit 226 and the text generating unit 227 is superimposed on the stereoscopic image data (left eye image data, right eye image data), and the display stereoscopic image data Vout is obtained. can get. The display stereoscopic image data Vout is supplied as transmission image data to the HDMI transmission unit 206 (see FIG. 26) via the video signal processing circuit 205.

  Also, the audio decoder 224 reconstructs an audio elementary stream from the audio TS packet extracted by the demultiplexer 220 and further performs a decoding process to obtain audio data. This audio data is supplied to the multi-channel speaker control unit 229. In this multi-channel speaker control unit 229, for example, processing for generating multi-channel speaker audio data for realizing 5.1ch surround, processing for giving predetermined sound field characteristics, and the like are performed on the audio data. The

  The multi-channel speaker control unit 229 is also supplied with the disparity vector obtained by the disparity vector decoder 225. The multi-channel speaker control unit 229 controls the output of the multi-channel speaker based on the parallax vector. The multichannel audio data obtained by the multichannel speaker control unit 229 is supplied as transmission audio data to the HDMI transmission unit 206 (see FIG. 26) via the audio signal processing circuit 207.

  Note that the bit stream processing unit 201 illustrated in FIG. 27 is configured to correspond to the transmission data generation unit 110 illustrated in FIG. The bit stream processing unit 201A illustrated in FIG. 29 has a configuration corresponding to the transmission data generation unit 110A illustrated in FIG. In FIG. 29, parts corresponding to those in FIG. 27 are denoted by the same reference numerals, and detailed description thereof is omitted.

  This bit stream processing unit 201A is provided with a disparity vector extracting unit 231 instead of the disparity vector decoder 225 of the bit stream processing unit 201 shown in FIG. The disparity vector extracting unit 231 extracts a disparity vector embedded in the user data from a video stream obtained through the video decoder 221. Then, the disparity vector extracting unit 231 supplies the extracted disparity vector to the stereoscopic image graphics generating unit 206, the stereoscopic image text generating unit 227, and the multi-channel speaker control unit 229.

  Although the detailed description is omitted, the rest of the bit stream processing unit 201A shown in FIG. 29 is configured in the same manner as the bit stream processing unit 201 shown in FIG.

  Also, the bit stream processing unit 201B illustrated in FIG. 30 is configured to correspond to the transmission data generation unit 110B illustrated in FIG. In FIG. 30, portions corresponding to those in FIG. 27 are denoted by the same reference numerals, and detailed description thereof is omitted.

  The bit stream processing unit 201B is obtained by removing the disparity vector decoder 225, the stereoscopic image graphics generating unit 206, and the stereoscopic image text generating unit 207 from the bit stream processing unit 201 shown in FIG. In this case, the disparity vector is transmitted by being included in the graphics information and text information data. Further, as described above, the transmitted graphics data includes left-eye graphics information data superimposed on the left-eye image and right-eye graphics information data superimposed on the right-eye image. Similarly, as described above, the transmitted text data includes left-eye text information data superimposed on the left-eye image and right-eye text information data superimposed on the right-eye image. Therefore, the disparity vector decoder 225, the stereoscopic image graphics generation unit 206, and the stereoscopic image text generation unit 207 are not necessary.

  Since the text data obtained by the text decoder 223 is code data, a process for converting it into bitmap data is necessary. This processing is performed at the final stage of the text decoder 223 or at the input stage of the video superimposing unit 228, for example.

[Description of TV receiver]
Returning to FIG. 1, the television receiver 300 receives stereoscopic image data sent from the set top box 200 via the HDMI cable 400. The television receiver 300 includes a 3D signal processing unit 301. The 3D signal processing unit 301 performs processing (decoding processing) corresponding to the transmission method on the stereoscopic image data to generate left-eye image data and right-eye image data. That is, the 3D signal processing unit 301 performs processing opposite to that of the video framing unit 112 in the transmission data generation units 110, 110A, and 110B illustrated in FIGS. Eye image data and right eye image data are acquired.

[Configuration example of TV receiver]
A configuration example of the television receiver 300 will be described. FIG. 31 illustrates a configuration example of the television receiver 300. The television receiver 300 includes a 3D signal processing unit 301, an HDMI terminal 302, an HDMI receiving unit 303, an antenna terminal 304, a digital tuner 305, and a bit stream processing unit 306. The television receiver 300 also includes a video signal processing circuit 307, a panel driving circuit 308, a display panel 309, an audio signal processing circuit 310, an audio amplification circuit 311, and a speaker 312. In addition, the television receiver 300 includes a CPU 321, a flash ROM 322, a DRAM 323, an internal bus 324, a remote control receiving unit 325, and a remote control transmitter 326.

  The antenna terminal 304 is a terminal for inputting a television broadcast signal received by a receiving antenna (not shown). The digital tuner 305 processes the television broadcast signal input to the antenna terminal 304 and outputs predetermined bit stream data (transport stream) corresponding to the user's selected channel.

  The bit stream processing unit 306 has the same configuration as the bit stream processing unit 201 of the set top box 200 shown in FIG. The bit stream processing unit 306 extracts stereoscopic image data (left eye image data, right eye image data), audio data, graphics data, text data, a disparity vector, and the like from the bit stream data. Then, the bit stream processing unit 306 synthesizes superimposition information (graphics information, text information) data with the stereoscopic image data, and acquires display stereoscopic image data. The bit stream processing unit 306 outputs audio data.

  The HDMI receiving unit 303 receives uncompressed image data (stereoscopic image data) and audio data supplied to the HDMI terminal 302 via the HDMI cable 400 by communication conforming to HDMI. Details of the HDMI receiving unit 303 will be described later. The 3D signal processing unit 301 performs processing (decoding processing) corresponding to the transmission method on the stereoscopic image data received by the HDMI receiving unit 303 or obtained by the bit stream processing unit 306, and thereby the left eye image Data and right eye image data are generated.

  The video signal processing circuit 307 generates image data for displaying a stereoscopic image based on the left eye image data and right eye image data generated by the 3D signal processing unit 301. The video signal processing circuit performs image quality adjustment processing on the image data as necessary. The panel drive circuit 308 drives the display panel 309 based on the image data output from the video signal processing circuit 307. The display panel 309 includes, for example, an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), or the like.

  The audio signal processing circuit 310 performs necessary processing such as D / A conversion on the audio data received by the HDMI receiving unit 303 or obtained by the bit stream processing unit 306. The audio amplification circuit 311 amplifies the audio signal output from the audio signal processing circuit 310 and supplies the amplified audio signal to the speaker 312.

  The CPU 321 controls the operation of each unit of the television receiver 300. The flash ROM 322 stores control software and data. The DRAM 323 constitutes a work area for the CPU 321. The CPU 321 develops software and data read from the flash ROM 322 on the DRAM 323 to activate the software, and controls each unit of the television receiver 300.

  The remote control receiving unit 325 receives the remote control signal (remote control code) transmitted from the remote control transmitter 326 and supplies it to the CPU 321. The CPU 321 controls each part of the television receiver 300 based on the remote control code. The CPU 321, flash ROM 322, and DRAM 323 are connected to the internal bus 324.

  The operation of the television receiver 300 shown in FIG. 31 will be briefly described. The HDMI receiving unit 303 receives stereoscopic image data and audio data transmitted from the set top box 200 connected to the HDMI terminal 302 via the HDMI cable 400. The stereoscopic image data received by the HDMI receiving unit 303 is supplied to the 3D signal processing unit 301. The audio data received by the HDMI receiving unit 303 is supplied to the audio signal processing circuit 310.

  A television broadcast signal input to the antenna terminal 304 is supplied to the digital tuner 305. The digital tuner 305 processes the television broadcast signal and outputs predetermined bit stream data (transport stream) corresponding to the user's selected channel.

  The bit stream data output from the digital tuner 305 is supplied to the bit stream processing unit 306. The bit stream processing unit 306 extracts stereoscopic image data (left-eye image data, right-eye image data), audio data, graphics data, text data, disparity vectors, and the like from the bit stream data. In addition, in the bit stream processing unit 306, superimposition information (graphics information, text information) data is combined with the stereoscopic image data to generate display stereoscopic image data.

  The display stereoscopic image data generated by the bit stream processing unit 306 is supplied to the 3D signal processing unit 301. The audio data obtained by the bit stream processing unit 306 is supplied to the audio signal processing circuit 310.

  In the 3D signal processing unit 301, processing corresponding to the transmission method (decoding processing) is performed on the stereoscopic image data received by the HDMI receiving unit 303 or obtained by the bit stream processing unit 306, and the left eye Image data and right eye image data are generated. The left eye image data and right eye image data are supplied to the video signal processing unit circuit 307. In the video signal processing circuit 307, image data for displaying a stereoscopic image is generated based on the left eye image data and the right eye image data. Therefore, a stereoscopic image is displayed on the display panel 309.

  In the audio signal processing circuit 310, necessary processing such as D / A conversion is performed on the audio data received by the HDMI receiving unit 303 or obtained by the bit stream processing unit 306. The audio data is amplified by the audio amplification circuit 311 and then supplied to the speaker 312. Therefore, sound is output from the speaker 312.

[Configuration Example of HDMI Transmitter and HDMI Receiver]
FIG. 32 shows a configuration example of the HDMI transmission unit (HDMI source) 206 of the set top box 200 and the HDMI reception unit (HDMI sink) 303 of the television receiver 300 in the stereoscopic image display system 10 of FIG.

  The HDMI transmission unit 206 transmits a differential signal corresponding to pixel data of an uncompressed image for one screen in an effective image section (hereinafter, also referred to as an active video section as appropriate) using a plurality of channels. Send in one direction. Here, the effective image section is a section obtained by removing the horizontal blanking section and the vertical blanking section from the section from one vertical synchronization signal to the next vertical synchronization signal. In addition, the HDMI transmission unit 206 receives the differential signals corresponding to at least audio data, control data, other auxiliary data, etc. associated with the image on a plurality of channels in the horizontal blanking interval or the vertical blanking interval. Transmit to the unit 303 in one direction.

  The transmission channels of the HDMI system including the HDMI transmission unit 206 and the HDMI reception unit 303 include the following transmission channels. That is, three TMDS channels # 0 to ## as transmission channels for serially transmitting pixel data and audio data in one direction in synchronization with the pixel clock from the HDMI transmission unit 206 to the HDMI reception unit 303. There are two. There is also a TMDS clock channel as a transmission channel for transmitting a pixel clock.

  The HDMI transmission unit 206 includes an HDMI transmitter 81. The transmitter 81 converts, for example, pixel data of an uncompressed image into a corresponding differential signal, and is connected via the HDMI cable 400 with three TMDS channels # 0, # 1, and # 2 that are a plurality of channels. Serial transmission in one direction to the HDMI receiving unit 303.

  The transmitter 81 converts audio data accompanying uncompressed images, further necessary control data and other auxiliary data, etc. into corresponding differential signals, and converts them into three TMDS channels # 0, # 1, #. 2 serially transmits to the HDMI receiving unit 303 in one direction.

  Further, the transmitter 81 transmits the pixel clock synchronized with the pixel data transmitted through the three TMDS channels # 0, # 1, and # 2 to the HDMI receiving unit 303 connected via the HDMI cable 400 using the TMDS clock channel. Send. Here, in one TMDS channel #i (i = 0, 1, 2), 10-bit pixel data is transmitted during one pixel clock.

  The HDMI receiving unit 303 receives a differential signal corresponding to the pixel data transmitted from the HDMI transmitting unit 206 in one direction on a plurality of channels in the active video section. Further, the HDMI receiving unit 303 receives differential signals corresponding to audio data and control data transmitted in one direction from the HDMI transmitting unit 206 through a plurality of channels in a horizontal blanking interval or a vertical blanking interval. Receive.

  That is, the HDMI receiving unit 303 includes an HDMI receiver 82. This HDMI receiver 82 uses TMDS channels # 0, # 1, and # 2 to transmit a differential signal corresponding to pixel data and a difference corresponding to audio data and control data transmitted from the HDMI transmission unit 206 in one direction. Receive a motion signal. In this case, reception is performed in synchronization with the pixel clock transmitted from the HDMI transmission unit 206 via the TMDS clock channel.

  In addition to the TMDS channels # 0 to # 2 and the TMDS clock channel described above, the transmission channel of the HDMI system including the HDMI transmission unit 206 and the HDMI reception unit 303 is called a DDC (Display Data Channel) 83 or a CEC line 84. There is a transmission channel. The DDC 83 includes two signal lines (not shown) included in the HDMI cable 400, and the HDMI transmission unit 206 receives an E-EDID (Enhanced Extended Display Identification Data) from the HDMI reception unit 303 connected via the HDMI cable 400. Used to read

  That is, the HDMI receiving unit 303 has an EDID ROM (Read Only Memory) 85 that stores E-EDID, which is performance information related to its performance (Configuration / capability), in addition to the HDMI receiver 81. . For example, in response to a request from the CPU 211 (see FIG. 26), the HDMI transmission unit 206 sends the E-EDID of the HDMI reception unit 303 from the HDMI reception unit 303 connected via the HDMI cable 400 to the DDC 83. Read through. The HDMI transmission unit 206 sends the read E-EDID to the CPU 211. The CPU 211 stores this E-EDID in the flash ROM 212 or the DRAM 213.

  The CPU 211 can recognize the performance setting of the HDMI receiving unit 303 based on the E-EDID. For example, the CPU 211 recognizes image data formats (resolution, frame rate, aspect, etc.) that can be supported by the television receiver 300 having the HDMI receiving unit 303.

  The CEC line 84 includes one signal line (not shown) included in the HDMI cable 400, and is used for bidirectional communication of control data between the HDMI transmission unit 206 and the HDMI reception unit 303. The CEC line 84 constitutes a control data line.

  Further, the HDMI cable 400 includes a line (HPD line) 86 connected to a pin called HPD (Hot Plug Detect). The source device can detect the connection of the sink device using the line 86. Also, the HDMI cable 400 includes a line 87 that is used to supply power from the source device to the sink device. Further, the HDMI cable 400 includes a reserved line 88.

  FIG. 33 shows a configuration example of the HDMI transmitter 81 and the HDMI receiver 82 of FIG. The HDMI transmitter 81 has three encoder / serializers 81A, 81B, and 81C corresponding to the three TMDS channels # 0, # 1, and # 2, respectively. Each of the encoders / serializers 81A, 81B, and 81C encodes the image data, auxiliary data, and control data supplied thereto, converts the parallel data into serial data, and transmits the data by a differential signal. Here, when the image data has, for example, three components R, G, and B, the B component is supplied to the encoder / serializer 81A, the G component is supplied to the encoder / serializer 81B, and the R component is supplied to the encoder / serializer 81C. Supplied.

  The auxiliary data includes, for example, audio data and control packets. The control packets are supplied to, for example, the encoder / serializer 81A, and the audio data is supplied to the encoder / serializers 81B and 81C. Further, the control data includes a 1-bit vertical synchronization signal (VSYNC), a 1-bit horizontal synchronization signal (HSYNC), and 1-bit control bits CTL0, CTL1, CTL2, and CTL3. The vertical synchronization signal and the horizontal synchronization signal are supplied to the encoder / serializer 81A. The control bits CTL0 and CTL1 are supplied to the encoder / serializer 81B, and the control bits CTL2 and CTL3 are supplied to the encoder / serializer 81C.

  The encoder / serializer 81A transmits the B component of the image data, the vertical synchronization signal and the horizontal synchronization signal, and auxiliary data supplied thereto in a time division manner. That is, the encoder / serializer 81A converts the B component of the image data supplied thereto into 8-bit parallel data that is a fixed number of bits. Further, the encoder / serializer 81A encodes the parallel data, converts it into serial data, and transmits it through the TMDS channel # 0.

  The encoder / serializer 81A encodes the 2-bit parallel data of the vertical synchronization signal and horizontal synchronization signal supplied thereto, converts the data into serial data, and transmits the serial data through the TMDS channel # 0. Furthermore, the encoder / serializer 81A converts the auxiliary data supplied thereto into parallel data in units of 4 bits. Then, the encoder / serializer 81A encodes the parallel data, converts it into serial data, and transmits it through the TMDS channel # 0.

  The encoder / serializer 81B transmits the G component of the image data, control bits CTL0 and CTL1, and auxiliary data supplied thereto in a time division manner. That is, the encoder / serializer 81B sets the G component of the image data supplied thereto as parallel data in units of 8 bits, which is a fixed number of bits. Further, the encoder / serializer 81B encodes the parallel data, converts it into serial data, and transmits it through the TMDS channel # 1.

  The encoder / serializer 81B encodes the 2-bit parallel data of the control bits CTL0 and CTL1 supplied thereto, converts the data into serial data, and transmits the serial data through the TMDS channel # 1. Furthermore, the encoder / serializer 81B converts the auxiliary data supplied thereto into parallel data in units of 4 bits. Then, the encoder / serializer 81B encodes the parallel data, converts it into serial data, and transmits it through the TMDS channel # 1.

  The encoder / serializer 81C transmits the R component of the image data, control bits CTL2 and CTL3, and auxiliary data supplied thereto in a time division manner. That is, the encoder / serializer 81C sets the R component of the image data supplied thereto as parallel data in units of 8 bits, which is a fixed number of bits. Further, the encoder / serializer 81C encodes the parallel data, converts it into serial data, and transmits it through the TMDS channel # 2.

  The encoder / serializer 81C encodes 2-bit parallel data of the control bits CTL2 and CTL3 supplied thereto, converts the data into serial data, and transmits the serial data through the TMDS channel # 2. Furthermore, the encoder / serializer 81C converts the auxiliary data supplied thereto into parallel data in units of 4 bits. Then, the encoder / serializer 81C encodes the parallel data, converts it into serial data, and transmits it through the TMDS channel # 2.

  The HDMI receiver 82 has three recovery / decoders 82A, 82B, and 82C corresponding to the three TMDS channels # 0, # 1, and # 2, respectively. Then, each of the recovery / decoders 82A, 82B, and 82C receives image data, auxiliary data, and control data transmitted as differential signals through the TMDS channels # 0, # 1, and # 2. Further, each of the recovery / decoders 82A, 82B, and 82C converts image data, auxiliary data, and control data from serial data to parallel data, and further decodes and outputs the converted data.

  That is, the recovery / decoder 82A receives the B component of the image data, the vertical synchronization signal, the horizontal synchronization signal, and the auxiliary data that are transmitted as differential signals through the TMDS channel # 0. Then, the recovery / decoder 82A converts the B component of the image data, the vertical synchronization signal, the horizontal synchronization signal, and the auxiliary data from serial data to parallel data, and decodes and outputs them.

  The recovery / decoder 82B receives the G component of the image data, the control bits CTL0 and CTL1, and the auxiliary data transmitted by the differential signal through the TMDS channel # 1. Then, the recovery / decoder 82B converts the G component of the image data, the control bits CTL0 and CTL1, and the auxiliary data from serial data to parallel data, and decodes and outputs them.

  The recovery / decoder 82C receives the R component of the image data, the control bits CTL2 and CTL3, and the auxiliary data transmitted by the differential signal through the TMDS channel # 2. Then, the recovery / decoder 82C converts the R component of the image data, the control bits CTL2 and CTL3, and the auxiliary data from serial data to parallel data, and decodes and outputs them.

  FIG. 34 shows an example of the structure of TMDS transmission data. FIG. 34 shows sections of various transmission data when image data of horizontal * vertical 1920 pixels * 1080 lines is transmitted in TMDS channels # 0, # 1, and # 2.

  A video field (Video Field) in which transmission data is transmitted through the three TMDS channels # 0, # 1, and # 2 of HDMI has three types of sections according to the type of transmission data. These three types of sections are a video data period, a data island period, and a control period.

  Here, the video field period is a period from the rising edge (active edge) of a certain vertical synchronizing signal to the rising edge of the next vertical synchronizing signal. The video field period is divided into a horizontal blanking period (horizontal blanking), a vertical blanking period (vertical blanking), and an active video period (Active Video). This active video section is a section obtained by removing the horizontal blanking period and the vertical blanking period from the video field section.

  The video data section is assigned to the active video section. In this video data section, 1920 pixel (active pixel) data of 1080 pixels constituting active image data for one screen of uncompressed data is transmitted.

  The data island period and the control period are assigned to the horizontal blanking period and the vertical blanking period. In the data island section and the control section, auxiliary data (Auxiliary data) is transmitted. That is, the data island period is assigned to a part of the horizontal blanking period and the vertical blanking period. In this data island period, for example, audio data packets, which are data not related to control, of auxiliary data are transmitted.

  The control period is allocated to other parts of the horizontal blanking period and the vertical blanking period. In this control period, for example, vertical synchronization signals, horizontal synchronization signals, control packets, and the like, which are data related to control, of auxiliary data are transmitted.

  FIG. 35 shows an example of the pin arrangement of the HDMI terminals 211 and 251. The pin arrangement shown in FIG. 35 is called type A (type-A).

  Two lines, which are differential lines through which TMDS Data # i + and TMDS Data # i−, which are differential signals of TMDS channel #i, are transmitted are pins to which TMDS Data # i + is assigned (the pin number is 1). , 4, 7) and pins assigned with TMDS Data # i− (pin numbers 3, 6, 9).

  A CEC line 84 through which a CEC signal as control data is transmitted is connected to a pin having a pin number of 13, and a pin having a pin number of 14 is a reserved pin. A line through which an SDA (Serial Data) signal such as E-EDID is transmitted is connected to a pin having a pin number of 16 and is an SCL (Serial Clock) which is a clock signal used for synchronization when transmitting and receiving the SDA signal. A line through which a signal is transmitted is connected to a pin having a pin number of 15. The above-described DDC 83 includes a line for transmitting the SDA signal and a line for transmitting the SCL signal.

  Further, as described above, the HPD line 86 for detecting the connection of the sink device by the source device is connected to the pin having the pin number 19. Further, as described above, the line 87 for supplying power is connected to a pin having a pin number of 18.

[Example of TMDS transmission data in each method of stereoscopic image data]
Here, an example of TMDS transmission data in each method of stereoscopic image data will be described. FIG. 36 illustrates an example of TMDS transmission data in the first transmission method (“Top & Bottom” method). In this case, 1920 pixels * 1080 lines of active video section, 1920 pixels (pixels) * 1080 pixel effective pixel (active pixel) data (the synthesis of left eye (L) image data and right eye (R) image data) Data) is arranged. In the case of the first method, as described above, in the left eye image data and the right eye image data, the vertical line is thinned out to 1/2. Here, the left eye image data to be transmitted is either an odd line or an even line, and similarly, the right eye image data to be transmitted is either an odd line or an even line.

  FIG. 37 shows an example of TMDS transmission data of the second transmission method (“Side By Side” method). In this case, 1920 pixels * 1080 lines of active video section, 1920 pixels (pixels) * 1080 pixel effective pixel (active pixel) data (the synthesis of left eye (L) image data and right eye (R) image data) Data) is arranged. In the case of this second transmission method, as described above, the left-eye image data and the right-eye image data are each decimated by half of the pixel data in the horizontal direction.

  FIG. 38 illustrates an example of TMDS transmission data in the third transmission scheme (“Frame Sequential” scheme). In this case, left-eye (L) image data of 1920 pixels * 1080 lines of effective pixels (Active pixels) is arranged in an active video section of 1920 pixels * 1080 lines in an odd field. Also, right-eye (R) image data of effective pixels (active pixels) for 1920 pixels * 1080 lines is arranged in an active video section of 1920 pixels * 1080 lines in the even field.

  Note that the TMDS transmission data example in the “Frame Sequential” method illustrated in FIG. 38 indicates the “Frame Sequential” method in HDMI 1.4 (New HDMI). In this case, as shown in FIG. 39A, in each frame period Vfreq, the left eye image data is arranged in the odd field and the right eye image data is arranged in the even field.

  However, in the case of the “Frame Sequential” method in HDMI 1.3 (Legacy HDMI), as shown in FIG. 39B, left-eye image data and right-eye image data are alternately transmitted every frame period Vfreq. Is done. In this case, the source device needs to send information (L and R signaling information) indicating whether the image data transmitted for each frame is left-eye image data or right-eye image data to the sink device. Become.

  When transmitting 3D image data of the “Top & Bottom”, “Side By Side”, or “Frame Sequential” method to the sink device, specify the method on the source device side, and in the case of the “Frame Sequential” method , L and R are signaled for each frame.

  For example, the following syntax is transmitted by newly defining one of Vendor Specific, AVI InfoFrame, or Reserved defined in Blanking of the Legacy HDMI specification.

In the case of HDMI 1.3, the following information is defined as information sent during the blanking period.
InfoFrame Type # (8bits)
--------------------------
0x01: Vendor Specific
0x02: AVI InfoFrame
0x03: Source Product Description
0x04: Audio InfoFrame
0x05: MPEG Source
0x06-0xFF Reserved

Of these, Vendor Specific, AVI InfoFrame, or one of unused areas is newly defined as follows.

3DVideoFlag 1bit (0: 2D, 1: 3D)
if (3DVideoFlag) {
3DVideoFormat 3bits (0x0: Frame Packing Left View
0x1: Frame Packing Right View
0x2: Side by Side
0x4: Top & Bottom by Frame
0x6: Top & Bottom by Field
(0x3,5,7: Reserved)
Reserved 4bits (0x0)
}
else {
Reserved 7bits (0x0)
}

  The above information includes switching information between 3D image data and 2D image data (1-bit 3D Video Flag information), specification of the format of 3D image data, or switching information between left eye image data and right eye image data ( 3bit 3DVideoFormat information).

This information should be defined as auxiliary information that is sent at a timing corresponding to a picture header in a bitstream in which similar contents are broadcast. In this case, the bit stream alternatively includes three-dimensional image data (stereoscopic image data including left-eye image data and right-eye image data) or two-dimensional image data.
When the receiver (set top box 200) receives the stream, it sends this signaling information to the digital interface at the subsequent stage so that accurate 3D conversion can be performed on the display (television receiver 300).

  In addition, when the switching information (1-bit 3DVideoFlag information) indicates 3D image data, that is, when the data stream includes 3D image data, the receiver broadcasts software for processing the 3D image data. You may make it download and install from external apparatuses, such as.

  For example, in order to transmit the above-mentioned 3D information, it is necessary to additionally support a system that supports HDMI 1.3 or update software of a system that supports HDMI 1.4. Therefore, when updating software, for example, firmware and middleware related software necessary for transmitting the above-mentioned 3D information are to be updated.

  As described above, in the stereoscopic image display system 10 illustrated in FIG. 1, the same superimposition information (superimposed on the left eye image and the right eye image) based on the parallax information on the other of the left eye image and the right eye image ( Disparity is given to graphics information, text information, and the like. Therefore, as the same superimposition information superimposed on the left eye image and the right eye image, information on which parallax adjustment is performed according to the perspective of each object (object) in the image can be used. In this case, it is possible to maintain perspective consistency with each object in the image.

<2. Modification>
In the above-described embodiment, the disparity vector at a predetermined position in the image is transmitted from the broadcast station 100 side to the set top box 200. In this case, the set top box 200 does not need to obtain a disparity vector based on the left eye image data and the right eye image data included in the received stereoscopic image data, and the processing of the set top box 200 is simplified. .

  However, a disparity vector detection unit equivalent to the disparity vector detection unit 114 in the transmission data generation unit 110 in FIG. 2 may be arranged on the stereoscopic image data reception side, in the above-described embodiment, in the set top box 200. In this case, processing using the parallax vector is possible even if the parallax vector is not sent.

  FIG. 40 illustrates a configuration example of a bit stream processing unit 201C provided in the set top box 200, for example. In FIG. 40, portions corresponding to those in FIG. 27 are denoted by the same reference numerals, and detailed description thereof is omitted. In this bit stream processing unit 201C, a disparity vector detection unit 233 is arranged instead of the disparity vector decoder 225 in the bit stream processing unit 201 shown in FIG.

  The disparity vector detection unit 233 detects a disparity vector at a predetermined position in the image based on the left eye image data and the right eye image data constituting the stereoscopic image data obtained by the video decoder 221. The disparity vector detection unit 233 supplies the detected disparity vector to the stereoscopic image graphics generation unit 206, the stereoscopic image text generation unit 227, and the multi-channel speaker output control unit 229.

  Although the detailed description is omitted, the rest of the bit stream processing unit 201C shown in FIG. 40 is configured in the same manner as the bit stream processing unit 201 shown in FIG.

  In the above-described embodiment, the stereoscopic image display system 10 includes the broadcasting station 100, the set top box 200, and the television receiver 300. However, the television receiver 300 includes a bit stream processing unit 201 that functions in the same manner as the bit stream processing unit 201 in the set top box 200, as shown in FIG. Therefore, as shown in FIG. 41, a stereoscopic image display system 10A including a broadcasting station 100 and a television receiver 300 is also conceivable.

  In the above-described embodiment, an example in which a data stream (bit stream data) including stereoscopic image data is broadcast from the broadcast station 100 has been described. However, it goes without saying that the present invention can be similarly applied to a system in which this data stream is distributed to a receiving terminal using a network such as the Internet.

  The present invention can be applied to a stereoscopic image display system that superimposes and displays superimposition information such as graphics information and text information on an image.

  DESCRIPTION OF SYMBOLS 10,10A ... Stereoscopic image display system, 100 ... Broadcast station, 110, 110A, 110B ... Transmission data generation part, 111L, 111R ... Camera, 112 ... Video framing part, 113 ... Video encoder 114... Parallax vector detection unit 115. Parallax vector encoder 116. Microphone 117. Audio encoder 118 118 Graphics generation unit 119 Graphics encoder 120 ... Text generator, 121 ... Text encoder, 122 ... Multiplexer, 123 ... Stream framing unit, 124 ... Graphics processor, 125 ... Text processor, 200 ... Set top Box, 201, 201A, 201B, 201C ... Bi Stream processing unit, 202 ... HDMI terminal, 203 ... antenna terminal, 204 ... digital tuner, 205 ... video signal processing circuit, 206 ... HDMI transmission unit, 207 ... audio signal processing Circuit, 211 ... CPU, 212 ... Flash ROM, 213 ... DRAM, 214 ... Internal bus, 215 ... Remote control receiver, 216 ... Remote control transmitter, 220 ... Demultiplexer 221 ... Video decoder, 222 ... Graphics decoder, 223 ... Text decoder, 224 ... Audio decoder, 225 ... Parallax vector decoder, 226 ... 3D graphics generator, 227 ..Text generation unit for stereoscopic image, 228... Video superimposing unit, 229. Control unit, 231 ... parallax vector extraction unit, 233 ... parallax vector detection unit, 300 ... television receiver, 301 ... 3D signal processing unit, 302 ... HDMI terminal, 303 ... HDMI receiving unit, 304 ... antenna terminal, 305 ... digital tuner, 306 ... bit stream processing unit, 307 ... video signal processing circuit, 308 ... panel driving circuit, 309 ... display panel 310 ... Audio signal processing circuit, 311 ... Audio amplification circuit, 312 ... Speaker, 321 ... CPU, 322 ... Flash ROM, 323 ... DRAM, 324 ... Internal bus, 325: Remote control receiver, 326: Remote control transmitter, 400: HDMI cable

Claims (9)

An image data receiving unit for receiving stereoscopic image data including left eye image data and right eye image data;
Based on parallax information of the other of the left eye image and the right eye image obtained by processing the left eye image data and the right eye image data included in the stereoscopic image data received by the image data receiving unit Then, parallax is given to the same superimposition information superimposed on the left eye image and the right eye image, and data of the left eye image on which the superposition information is superimposed and data on the right eye image on which the superposition information is superimposed A stereoscopic image data receiving device comprising an image data processing unit to obtain. The stereoscopic image data receiving device according to claim 1, wherein the image data processing unit gives a parallax corresponding to a superposition position of the superimposition information to the same superposition information superimposed on the left eye image and the right eye image. The stereoscopic image data receiving device according to claim 1, further comprising: a parallax information receiving unit that receives the parallax information in synchronization with the stereoscopic image data received by the image data receiving unit. Based on the left eye image data and the right eye image data included in the stereoscopic image data received by the image data receiving unit, the other parallax with respect to one of the left eye image and the right eye image at a predetermined position in the image The stereoscopic image data receiving device according to claim 1, further comprising a parallax information acquiring unit that obtains information. The stereoscopic image data receiving device according to claim 1, further comprising: an image data transmitting unit configured to transmit stereoscopic image data including the left eye image data and the right eye image data obtained by the image data processing unit to an external device. The image data transmission unit transmits the left eye image data and the right eye image data to the external device in a frame sequential manner, and whether the image data transmitted in each frame is the left eye image data or the right eye image The stereoscopic image data receiving device according to claim 5, further transmitting a signal for determining whether the image data is image data to the external device. The stereoscopic image data receiving device according to claim 1, further comprising an image display unit that displays an image for stereoscopic image display using the left eye image data and the right eye image data obtained by the image data processing unit. A multi-channel speaker;
The stereoscopic image data receiving device according to claim 1, further comprising a control unit that controls the multi-channel speaker output based on parallax information on the other of the left eye image data and the right eye image data. An image data receiving step for receiving stereoscopic image data including left eye image data and right eye image data;
Based on the parallax information of the other of the left eye image and the right eye image, the left eye image in which parallax is given to the same superimposition information superimposed on the left eye image and the right eye image, and the superimposition information is superimposed A stereo image data receiving method comprising: an image data processing step of obtaining stereo image data including the right eye image data on which the superposition information and the superposition information are superimposed.