JP5047381B1 - VIDEO GENERATION DEVICE, VIDEO DISPLAY DEVICE, TELEVISION RECEIVER, VIDEO GENERATION METHOD, AND COMPUTER PROGRAM - Google Patents

Abstract

A video generation device, a video display device, a television receiver, a video generation method, and a computer program for generating a natural three-dimensional video based on a two-dimensional video are provided.
SOLUTION: In a video generation apparatus that generates a three-dimensional video by calculating a depth value corresponding to each pixel of a two-dimensional video constituted by a plurality of pixels, pixels of each pixel constituting the two-dimensional video A first difference calculation unit that calculates a difference between a value and a pixel value of a pixel adjacent to each pixel; a first extraction unit that extracts a pixel whose difference calculated by the first difference calculation unit is equal to or greater than a predetermined first threshold; A second difference calculation unit that calculates a difference between a pixel value in one frame of each pixel constituting the two-dimensional video and a pixel value in another frame, and the difference calculated by the second difference calculation unit is equal to or greater than a predetermined second threshold value And a depth calculation unit that calculates a depth value corresponding to each pixel of the two-dimensional video based on the extraction results of the second extraction unit, the first extraction unit, and the second extraction unit.
[Selection] Figure 9

Description

  The present invention relates to a video generation device, a video display device, a television receiver, a video generation method, and a computer program that generate a 3D video from a 2D video.

  Although display devices that display 3D video are becoming widespread, the amount of 3D video is still small, and there is also a demand for viewing video recorded in 2D as 3D video.

  Therefore, it is conceivable to generate a pseudo 3D video from the 2D video. For this purpose, appropriate depth information must be given to the 2D video. For example, in Patent Document 1, a ratio for combining three predetermined basic depth models set for the entire screen is calculated from the composition of the entire two-dimensional image, and the three-dimensional image is based on the combined basic depth model. Describes a video generation apparatus that gives a depth value of 2 to a 2D video and generates a 3D video.

JP 2009-44722 A

  However, since the image generation apparatus in Patent Document 1 only applies a basic depth model set in advance to the entire screen to the 2D image, the depth value may be inappropriate, and an unnatural 3D image is generated. There was a thing.

  The present invention has been made under such circumstances, and provides a video generation device, a video display device, a television receiver, a video generation method, and a computer program for generating natural 3D video.

According to another aspect of the present invention, there is provided a video generation device that generates a three-dimensional video by calculating a depth value indicating a depth corresponding to each pixel of a two-dimensional video composed of a plurality of pixels. A first difference calculation unit that calculates a difference between a pixel value of each pixel and a pixel value of a pixel adjacent to each pixel, and a pixel in which the difference calculated by the first difference calculation unit is a predetermined first threshold value or more A first extraction unit for extracting together with the position information of the pixels; a second difference calculation unit for calculating a difference between a pixel value in one frame of each pixel constituting the two-dimensional video and a pixel value in another frame; A second extraction unit that extracts a pixel whose difference calculated by the two difference calculation unit is equal to or greater than a predetermined second threshold together with position information of the pixel, the number of pixels extracted by the first extraction unit and the second extraction unit, or Based on location information, The discriminating unit for discriminating whether the composition type of the three-dimensional video is any of a plurality of predetermined types, the calculation result of the first difference calculating unit or the second difference calculating unit, and the discriminating unit discriminate And a depth calculation unit that calculates a depth value corresponding to each pixel of the two-dimensional video based on the composition type .

  According to the present invention, a part of pixels of a 2D image is extracted, and a depth value is calculated based on the extraction result, so that a natural 3D image can be generated.

  The video generation device according to the present invention extracts a pixel whose difference calculated by the first difference calculation unit is equal to or greater than a predetermined third threshold greater than the first threshold, and the third extraction unit extracts the pixel. A dispersion value calculation unit that calculates a dispersion value of a position indicating a degree of dispersion of a position where each pixel exists, and the depth calculation unit includes the dispersion value of the position calculated by the dispersion value calculation unit, and the first value The depth value is calculated based on the total number of pixels extracted by the extraction unit or the second extraction unit.

  According to the present invention, since the depth value is calculated based on the variance value of the pixel position, a natural 3D image corresponding to the 2D image can be generated.

  The video generation apparatus according to the present invention further includes a second variance value calculation unit that calculates a variance value of the difference calculated by the first difference calculation unit, and the depth calculation unit is calculated by the second variance value calculation unit. The depth value is calculated on the basis of the variance value of the difference of each pixel and the total number of pixels extracted by the first extraction unit or the second extraction unit.

  According to the present invention, since the depth value is calculated based on the variance value of the difference of each pixel, a natural three-dimensional image corresponding to the two-dimensional image can be generated.

  A video display device according to the present invention includes the above-described video generation device and a display unit that displays the 3D video generated by the video generation device.

  According to the present invention, the above-described effects can be realized by the video display device.

  The television receiver according to the present invention includes a tuner unit that receives a television broadcast wave including a two-dimensional image, and generates a three-dimensional image based on the two-dimensional image received by the tuner unit. And a display unit for displaying the three-dimensional video generated by the video generation device.

  According to the present invention, the above-described effects can be realized by a television receiver.

According to another aspect of the present invention, there is provided a video generation method for generating a three-dimensional video by calculating a depth value indicating a depth corresponding to each pixel of a two-dimensional video composed of a plurality of pixels. A first difference calculating step for calculating a difference between a pixel value of each pixel to be performed and a pixel value of a pixel adjacent to each pixel, and a pixel having a difference calculated in the first difference calculating step being a predetermined first threshold value or more. A first extraction step for extracting together with position information of the pixels; a second difference calculating step for calculating a difference between a pixel value in one frame of each pixel constituting the two-dimensional video and a pixel value in another frame; second extraction step in which the difference calculated in 2 difference calculation step extracts pixels is above a predetermined second threshold value with the position information of the pixel, in the first extraction step and second extraction step A determination step of determining whether a composition type of the 2D video is a plurality of predetermined types based on the number of extracted pixels or position information; and the first difference calculation step or And performing a process including a depth calculation step of calculating a depth value corresponding to each pixel of the two-dimensional video based on the calculation result of the second difference calculation step and the composition type determined by the determination step. To do.

  According to the present invention, a part of pixels of a 2D image is extracted, and a depth value is calculated based on the extraction result, so that a natural 3D image can be generated.

A computer program according to the present invention is a computer program for generating a three-dimensional video by calculating a depth value indicating a depth corresponding to each pixel of the two-dimensional video constituted by a plurality of pixels, and forming the two-dimensional video. the first difference calculation step of calculating a difference between pixel values of pixels adjacent to the pixel values and the respective pixels of each pixel, difference calculated by the first difference calculation step the pixels is above a predetermined first threshold value A first extraction step for extracting together with pixel position information; a second difference calculating step for calculating a difference between a pixel value in one frame of each pixel constituting the two-dimensional video and a pixel value in another frame; second extraction step in which the difference calculated in the difference calculation step extracts pixels is above a predetermined second threshold value with the position information of the pixel, the first extraction step And a determination step for determining whether the composition type of the two-dimensional image is one of a plurality of predetermined types based on the number or position information of the pixels extracted in the second extraction step, and Processing including a depth calculation step of calculating a depth value corresponding to each pixel of the two-dimensional video based on the calculation result of the first difference calculation step or the second difference calculation step and the composition type determined by the determination step Is executed by a computer.

  According to the present invention, a part of pixels of a 2D image is extracted, and a depth value is calculated based on the extraction result, so that a natural 3D image can be generated.

  According to the present invention, a pixel whose edge intensity of each pixel of the calculated 2D video is equal to or greater than a predetermined threshold is extracted, and the pixel value in one frame of the calculated 2D video and the pixel value in another frame are Since a pixel whose difference is equal to or greater than a predetermined threshold is extracted and the depth value is calculated based on the extraction result, a natural three-dimensional image can be provided.

It is explanatory drawing for demonstrating the image | video produced | generated by the video production | generation apparatus in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the television receiver in 1st Embodiment. It is a conceptual diagram which shows the two-dimensional image | video which the control part 1 processes. It is explanatory drawing showing a Laplacian filter. It is a schematic diagram which shows edge strength average and strong edge pixel information. It is a schematic diagram which shows the binary image | video based on high difference information. It is a schematic diagram which shows the image | video which divided the image | video of the nth frame for every area | region. It is explanatory drawing which shows a morphology process. It is explanatory drawing which shows the foreground and background in each composition. It is a schematic diagram which shows the example of the predetermined | prescribed system for displaying a three-dimensional image | video on a display part. It is a flowchart which shows the process sequence of the control part in 1st Embodiment. It is a flowchart which shows the process sequence in which a control part calculates strong edge pixel information. It is a flowchart which shows the process sequence in which a control part calculates moving body structure pixel information. It is a flowchart which shows the process sequence in which a control part discriminate | determines the composition of a two-dimensional image. It is a flowchart which shows the process sequence which a control part resets a depth value. It is a flowchart which shows the process sequence in which a control part produces | generates a three-dimensional image | video. It is a flowchart which shows the process sequence of the control part in 2nd Embodiment. It is the flowchart which showed the process sequence in which the control part in 2nd Embodiment sets a depth value. It is a flowchart which shows the process sequence in which the control part in 3rd Embodiment discriminate | determines a composition. It is a block diagram which shows each hardware part of the television receiver in 4th Embodiment.

First Embodiment Hereinafter, a first embodiment will be described with reference to the drawings. FIG. 1 is an explanatory diagram for explaining a video generated by the video generation device according to the first embodiment.

  When generating a 3D image from a 2D image composed of subjects such as birds, trees, the sun, the sky, clouds, and the ground shown in FIG. The foreground and other backgrounds are distinguished based on the edge strength and the magnitude of movement. Here, the foreground is defined as a region having a high feature amount that is easily noticed by a person in the video, and the background is defined as a region other than the foreground in the input video.

  In this embodiment, numerical values indicating the strength of the edge and the magnitude of the movement are used as feature quantities that are easily noticed by a person, and whether the foreground or the background is determined by whether the numerical values are equal to or greater than a certain value. By performing the determination, as shown in FIG. 1B, a bird having a strong edge or a large movement area and a tree having a strong edge area become the foreground and are distinguished from the background.

  Next, the depth value of the 2D image is set. First, the basic depth value calculated by a predetermined calculation is set for all the pixels of the two-dimensional image. Subsequently, the depth value of the foreground is separately calculated based on a predetermined depth pattern set according to the shape, size, movement, etc. of the subject, and the calculated depth value is reset. Based on the set and reset depth values, a left-eye video L and a right-eye video R, which are 3D video with parallax, are generated.

  Also, as shown in FIG. 1C, the foreground representing the bird, which is a region having a large movement, is divided into other regions, and the depth value is reset for the region having a strong movement to generate a three-dimensional image.

  The method of resetting the depth value is determined by the variance value of the position of the pixel having a strong edge strength, the number of pixels constituting the foreground, and the like.

  Configurations of the video generation device, the video display device, and the television receiver in this embodiment will be described. The television receiver and the video display device in this embodiment include a video generation device. FIG. 2 is a block diagram illustrating a configuration example of the television receiver in the first embodiment.

  The control unit 1 is composed of a CPU (Central Processing Unit) or MPU (Micro Processing Unit) and the like, and includes an arithmetic circuit capable of processing video data. The control program stored in advance in the ROM 2 is read out to the RAM 3 as appropriate and executed. To do. In addition, the control unit 1 controls operations of the ROM 2, RAM 3, video storage unit 4, input unit 5, and output unit 6. These are connected to each other via a bus.

  The ROM 2 is composed of a writable and erasable EPROM (Erasable Programmable ROM), a flash memory, or the like, and stores in advance various control programs necessary for the operation of the video generation apparatus.

  The RAM 3 is an SRAM or a flash memory, and temporarily stores various data generated when the control unit 1 executes a control program.

  When a two-dimensional image of a broadcast wave is input from a broadcast wave receiving antenna (not shown) via an input unit 5 connected to a tuner 7 for receiving a broadcast wave, the control unit 1 stores the data of the two-dimensional image in a RAM or The image is stored in the video storage unit 4 composed of a flash memory or the like. The two-dimensional video may be a video input from a playback device that plays back a recording medium, a video of a communication wave input from a communication device, or the like. MPEG-2 (Moving Picture Expert Group phase 4), MPEG-4 (Moving Picture Expert Group phase 4), or H.264. A compressed format such as H.264 may be used, or an uncompressed format may be used.

  The control unit 1 sequentially reads out the 2D video data stored in the video storage unit 4. When the two-dimensional video is a video such as terrestrial digital broadcast or BS digital broadcast, MULTI2 which is a standard encryption of a conditional access system (B-CAS: BS Conditional Access System) is decrypted. If the 2D video is a compressed video, the video is decompressed. If the 2D video is an analog video, the video is converted into, for example, an MPEG-2TS digital video.

  The control unit 1 generates a 3D image by performing processing described later. Also, the 3D video is converted into a predetermined method for displaying on the display unit 10 and is output to the display unit 10 for displaying the video via the output unit 6.

  The display unit 10 is a display of a mobile phone, a PDA, a book reader, a game machine, or a music player in addition to a monitor of a television and an information processing terminal, and displays a three-dimensional image including an image generated by processing to be described later. . In addition to the 3D video, the display unit 10 can also display a 2D video. In this case, the display unit 10 includes a video switching unit (not shown) that switches the video to display the 2D video or the 3D video.

  The control unit 1 will be described. FIG. 3 is a conceptual diagram showing a two-dimensional image that the control unit 1 performs processing, and includes time-series images of the (n−1) th frame, the nth frame, and the (n + 1) th frame.

  The number of pixels in each frame is, for example, 1920 × 1080. A pixel at the position of x columns and y rows is represented by (x, y) with the upper left point of the frame as the origin. The control unit 1 extracts, from each pixel of the two-dimensional image, a strong edge pixel that indicates a pixel in which the difference in shading between adjacent pixels is abrupt.

  Of the RGB components of the pixel value of the pixel (x, y) in the nth frame, the R component is SnR (x, y), the G component is SnG (x, y), and the B component is SnB (x, y). . As shown in Expression (1), the control unit 1 weights SnR (x, y), SnG (x, y), and SnB (x, y) in the same manner as the luminance signal of the ISDB-T system in digital broadcasting. A gray scale value Gn (x, y) is calculated by taking a weighted average.

  The control unit 1 performs processing using a Laplacian filter for Gn (x, y). FIG. 4 is an explanatory diagram showing a Laplacian filter. The Laplacian filter is a process for sharpening the edge of an image by adding a value G ″ n (x, y) obtained by second-order differentiation of Gn (x, y) to a gray scale value Gn (x, y).

  As shown in FIG. 4A, for example, consider a 3 × 3 pixel composed of a pixel of interest and eight neighboring pixels, and the grayscale value Gn (x, y) of the pixel of interest is α0, and the pixel values near eight are α1 to α8. To do. When this pixel is processed by an eight-direction Laplacian filter as shown in FIG. 4B, the second-order differential value G ″ n (x, y) is expressed by Expression (2).

  When processing is performed with the Laplacian filter, the control unit 1 calculates the difference between pixels adjacent in the row direction and the column direction, and calculates the square root of the sum of the squares of the differences. This edge strength En (x, y) is calculated for all pixels.

  When a pixel (x, y) belongs to a 5 × 5 block that is j-th in the column direction and k-th in the row direction, the average value of the edge intensities En (x, y) of 25 pixels constituting the block is calculated as Eave (j , K). Since x belongs to pixels from 5j-4 to 5j columns and y belongs to pixels from 5k-4 to 5k rows, equations (4) and (5) hold for x and j and y and k. . However, j is an integer from 0 to 384, and k is an integer from 0 to 216.

  The control unit 1 divides the entire frame into blocks of 5 × 5 pixels and calculates Eave (j, k). The reason for dividing the block into 5 × 5 pixel blocks is to simplify the process and reduce noise.

  FIG. 5 is a schematic diagram showing edge intensity average Eave (j, k) and strong edge pixel information Hn (x, y). FIG. 5A is a schematic diagram illustrating a two-dimensional image divided into blocks. The block a1 is a block constituting an image representing a bird and has a large difference in pixel value from adjacent pixels, so that the value of the edge intensity average Eave (j, k) is large. On the other hand, the block a2 is a block constituting the sky, and since the difference between the pixel values of adjacent pixels is small, the value of the edge intensity average Eave (j, k) is small.

  As shown in the equations (6) and (7), the control unit 1 sets the strong edge pixel information Hn (x, y) to 1 for pixels whose edge intensity average Eave (j, k) is equal to or greater than a preset threshold Th0. And the pixel having a threshold value less than Th0 has the strong edge pixel information Hn (x, y) set to 0.

  The strong edge pixel information Hn (x, y) is used in the foreground and background classification described later. Therefore, the threshold value Th0 is set to a value suitable for classifying a two-dimensional image into foreground and background from the viewpoint of edge strength.

  Thereby, the strong edge pixel which is a pixel with a strong edge can be extracted from the value of the strong edge pixel information Hn (x, y). In FIG. 5B, a pixel whose strong edge pixel information Hn (x, y) is 1 is shown in white, and a pixel whose 0 is 0 is shown in black.

  The procedure for calculating the strong edge pixel information Hn (x, y) is not limited to the procedure described above. For example, the strong edge pixel information Hn (x, y) may be calculated based on whether or not the number of pixels having an edge intensity equal to or greater than the threshold Th0 among all the pixels in the block is greater than or equal to a predetermined number.

  Subsequently, the control unit 1 extracts a moving object constituent pixel that is a pixel indicating a two-dimensional image with a large movement. A two-dimensional image with a large movement is considered to have a large change in pixel value. Therefore, a pixel having a large change in pixel value compared to the pixel values of the previous and subsequent frames is considered as a pixel having a large movement.

  A procedure for calculating the moving object constituent pixel information Mn (x, y) will be described. First, as shown in the equation (8), the already calculated gray scale value Gn (x, y) and Gn−1 (x, y) obtained by performing gray scale conversion on the pixel value of the pixel in the (n−1) th frame. Absolute difference information Dn (x, y) which is an absolute value of the difference is calculated.

  Subsequently, it is determined whether or not the absolute difference information Dn (x, y) is greater than or equal to a preset threshold value Th1, and high difference information Bn (x, y) indicating whether or not the pixel has a large absolute value of the difference. calculate. As shown in the equations (9) and (10), the pixel having the threshold value Th1 or more sets the high difference information Bn (x, y) to 1, and the pixel having the threshold value Th1 less than 0 sets the high difference information Bn (x, y) to 0. And The high difference information Bn (x, y) is used in the foreground and background classification described later. Therefore, the threshold value Th1 is set to a value suitable for distinguishing between the foreground and the background from the viewpoint of the magnitude of motion in the two-dimensional video.

  FIG. 6 is a schematic diagram showing a binary video based on the high difference information Bn (x, y) and Bn + 1 (x, y). FIG. 6A shows a pixel whose high difference information Bn (x, y) is 1 in white , 0 pixels are shown in black.

  Next, with respect to the (n + 1) th frame, the same calculation as the above-described equations (8), (9) and (10) is performed to calculate the absolute difference information Dn + 1 (x, y). Further, high difference information Bn + 1 (x, y) is calculated based on the absolute difference information Dn + 1 (x, y). FIG. 6B shows pixels whose high difference information Bn + 1 (x, y) is 1 in white and pixels where 0 is 0 in black.

  Finally, as shown in the equation (11), the moving object constituent pixel information Mn (x, y), which is the logical product of the high difference regions Bn (x, y) and Bn + 1 (x, y), is calculated.

  A pixel in which the moving object constituent pixel information Mn (x, y) is 1 has a large change in pixel value, and a pixel in which the moving object constituent pixel information Mn (x, y) is 0 has a small change in pixel value. Therefore, the moving object constituent pixels can be extracted from the value of the moving object constituent pixel information Mn (x, y).

  FIG. 6C shows a region where the moving object constituent pixel information Mn (x, y) is 1 in white and a region where 0 is 0 in black.

  A pixel with a large motion or a strong edge in a two-dimensional image is a pixel that indicates a subject that is viewed by a person viewing the two-dimensional image, and is considered to constitute a foreground. On the other hand, other pixels are considered to constitute the background.

  After obtaining the moving object constituent pixel information Mn (x, y), the control unit 1 performs a labeling process to segment the foreground and the background and extract the foreground. The control unit 1 attaches the same label to a group of pixels in which the strong edge region information Hn (x, y) is 1. The same processing is performed for a group of pixels having the moving object constituent pixel information Mn (x, y) of 1 in series. Thereby, the pixels constituting the foreground can be extracted as a region.

  FIG. 7 is a schematic diagram showing an image obtained by dividing the image of the nth frame for each region. As shown in FIG. 7A, the control unit 1 classifies the strong edge area information Hn (x, y) into areas b1 to b4 where the strong edge area information Hn (x, y) is 1 and the area b5 where the strong edge area information En (x, y) is 0. FIG. 7B is divided into a region b6 where the value of the moving object constituent pixel information Mn (x, y) is 1 and a region b7 where the strong edge region information Hn (x, y) is 0.

  Next, as shown in the equation (12), the foreground information Pn (x, y) is obtained by calculating the logical sum of the strong edge region information Hn (x, y) and the moving object constituent pixel information Mn (x, y). calculate.

  Pixels for which foreground information Pn (x, y) is 1 have pixels in b1 to b4 having strong edge region information Hn (x, y) of 1 or moving body constituent pixel information Mn (x, y) of 1 In FIG. 7C, the area where the foreground information Pn (x, y) is 1 is shown in white, and the area where 0 is shown in black.

  Note that when performing the above processing, not all of the desired foreground may be extracted. For example, when a close-up of a person's face that is in focus is being shot, the area that is easy for people to focus on is the entire person's face that is in focus. It is desirable to extract as However, a contour with a clear edge is easily detected as a foreground, but a portion such as a cheek or a forehead with a relatively weak edge is easily determined as a background. Therefore, there are cases where an area determined to be part of the background exists like a hole even though it is a pixel constituting a “human face” that should be determined to be the foreground.

  In order to cope with such a case, the control unit 1 performs timely morphology processing (Morphological Operations). FIG. 8 is an explanatory diagram showing morphology processing. For the sake of explanation, consider a 7 × 7 pixel represented by a binary value of 0 or 1.

  When there is a pixel with a pixel value of 0 among the pixels with a pixel value of 1 as shown in FIG. 8A, the pixel value of a pixel once in the vicinity of 4 of the pixel with a pixel value of 1 is set to 1 as shown in FIG. Dilate processing to be changed is performed. Then, the pixel value of the pixel existing in the group of pixels having a pixel value of 1 is changed from 0 to 1. Thereafter, as shown in FIG. 8C, a contraction (Erode) process is performed in which the pixel values of the pixels constituting the periphery in the group of pixels having the expanded pixel value of 1 are changed from 1 to 0.

  By this morphology processing, a portion determined as a background such as a hole can be appropriately set as the foreground. In addition, when the location determined as the background which exists like a hole is large, it can be set as a foreground by performing expansion processing and contraction processing a plurality of times.

  Subsequently, the control unit 1 determines the composition of the two-dimensional video. An area composed of pixels with strong edges or pixels with a large movement may be considered as an area in which a subject that is viewed by the viewer of the video is shown. The control unit 1 determines the size, movement, or arrangement of each subject in the two-dimensional image from the number or arrangement of pixels with strong edges or pixels with large movement, and determines whether the composition of the image is of a plurality of types. Determine.

  In the present embodiment, the composition is an “overhead” type in which the entire landscape is captured as an object from an overhead view, an “up” type in which a specific subject is close-up, and an intermediate between the “overhead” and “up”, and the foreground I think that there are three types of "balance" type with a well-balanced background.

  The “balance” type refers to an image type in which a main subject such as a person, an animal, or a building has the same amount of foreground and other background, and the number of pixels in the foreground is relatively small. FIG. 9 is an explanatory diagram showing the foreground and background in each composition. FIG. 9A is an explanatory diagram showing a “balance” type image, in which foreground information Pn (x, y) is 1 in white, and 0 is in black.

  On the other hand, the subject in the “up” type (for example, the close-up of the face of a person or an animal) occupies a very large proportion of the entire image, so the number of foreground pixels is relatively large. FIG. 9B is an explanatory diagram showing an “up” type image, in which foreground information Pn (x, y) is 1 in white, and 0 is in black.

  Further, it is determined that the number of pixels of the foreground (for example, a landscape such as a cityscape) in the “overhead” type is relatively large as in the “up” type. FIG. 9C is an explanatory diagram showing a “bird's-eye view” type image, in which foreground information Pn (x, y) is 1 in white and 0 is in black.

  Then, when the composition of the video is up of the subject, pixels with strong edges are concentrated in a region where the up of the subject is reflected. On the other hand, when the composition of the video is a bird's-eye view, since many subjects are scattered, pixels with strong edges are scattered throughout the frame. Therefore, the positions of pixels with strong edges tend to be more dispersed in the “overhead” type than in the “up” type.

  A procedure for the control unit 1 to determine the composition will be described. The control unit 1 counts the foreground pixels. If the number of foreground pixels is less than a preset threshold value Th2 as a result of the counting, it is determined that the composition is a “balance” type. In the threshold Th2, the total number of pixels constituting the foreground is set to a value suitable for discriminating between the “up” type and the “overhead” type and the “balance” type.

  When the number of foreground pixels is equal to or greater than the threshold Th2, pixels whose edge intensity En (x, y) is equal to or greater than a preset threshold Th3 are extracted. The threshold value Th3 (> Th0) is set to a value suitable for discriminating between pixels constituting a characteristic subject of the video and other pixels.

A variance value σ1 is calculated for the coordinate position of the pixel whose extracted edge strength En (x, y) is equal to or greater than the threshold Th3.
When there are m pixels (x1, y1), (x2, y2),..., (Xm, ym) whose edge intensity En (x, y) is greater than or equal to the threshold Th3, the variance value σ1 is set to the following ( 13) Calculated by the equation.

  The control unit 1 determines whether or not the variance value σ1 is greater than or equal to the threshold value Th4. When the variance value σ1 is equal to or greater than the preset threshold Th4, the control unit 1 determines that the composition is an “up” type. When the variance value σ1 is less than the threshold Th4, it is determined that the composition is the “overhead” type.

  The threshold value Th4 is set to an appropriate value for determining whether the composition of the video is the “up” type or the “overhead” type.

  After determining the composition of the 2D video, the control unit 1 calculates a vanishing point and a vanishing line, and further calculates a basic depth value of the 2D video based on the calculated vanishing point and vanishing line.

  First, the procedure for calculating the vanishing point and the vanishing line will be described. The control unit 1 extracts a pixel having a large edge intensity En (x, y). A pixel having a large edge strength En (x, y) is the same pixel as the pixel having a value of the edge strength En (x, y) equal to or greater than the threshold Th3.

  The control unit 1 selects any one of the pixels having a large edge strength En (x, y), and selects a pixel having a large edge strength En (x, y) in the vicinity of the eight pixels. Subsequently, when there is a pixel having a large edge strength En (x, y) in the vicinity of 8 of the most recently selected pixels, a pixel having a large edge strength En (x, y) is selected. If there are a plurality of pixels having a large edge strength En (x, y) in the vicinity of 8, the procedure of selecting the pixel farthest from the first selected pixel is repeated.

  When the selected pixels are grouped in line segments by such a procedure, the control unit 1 calculates a regression line of the group of pixels. When there are two or more groups of pixels, the regression lines for all of them are calculated.

  Of the calculated regression lines, if there is a combination in which the angle formed by any two regression lines is less than or equal to the predetermined angle, the two regression lines are vanishing lines, and the intersection of the calculated regression lines is vanishing Is a point. When the vanishing point and the vanishing line are calculated, the control unit 1 calculates a basic depth value. The basic depth value is a value from 0 to d, and the depth value of the pixel corresponding to the vanishing point is d.

  The vanishing point is not limited to within the frame, but may exist outside the frame. Further, the depth value of the pixel (x, y) located farthest from the vanishing point is the smallest. The basic depth value calculates the distance from the vanishing point of each pixel (x, y), and the basic depth value is set to be inversely proportional to the distance from the vanishing point.

  In addition, the basic depth value may be calculated based on the composition, the gray scale value Gn (x, y), or the like. For example, if the image shows the sea and the sky with the horizon as the boundary, it is desirable to calculate the depth value by a different setting method for each space that constitutes the sea and the sky. A uniform setting method may be used for the entire video. Therefore, it is desirable to perform different basic depth value setting methods depending on the video.

  In this case, in order to calculate the depth value of the pixel (x, y) other than the vanishing point, the control unit 1 reads a template stored in advance in the ROM 2. In the template, the depth value of the vanishing point is set to d, and the depth value at each pixel (x, y) of the entire two-dimensional video other than the vanishing point is set. The template is associated with a composition, a histogram of gray scale values Gn (x, y) of each pixel of the 2D video, and the like.

  The control unit 1 selects an optimum template from the templates read from the ROM 2 using the composition, the gray scale value Gn (x, y), etc., matches the vanishing point position with the selected template, and other than the vanishing point. The depth value of the pixel (x, y) is calculated.

When the basic depth value is set, the control unit 1 resets the depth value based on the composition. Hereinafter, a procedure for resetting the depth value will be described.
As a result of the determination, if the composition is a “balance” type, the value of y is the largest among the pixels (x, y) constituting each foreground in each foreground b1 to b4 and b6. The foreground depth value is reset according to the basic depth value of one pixel and the pattern of the preset depth value.

  The ROM 2 stores data of a plurality of patterns in which depth values are set in various two-dimensional shapes. The shape of the pattern includes a shape imitating a bird, a tree, a cloud, a person, or the like, a shape obtained by projecting a spherical shape or a rectangular parallelepiped two-dimensionally, and the like, and a depth value corresponding to the shape is set.

  In resetting, the control unit 1 reads a pattern from the ROM 2, performs matching with the pattern on the basis of the shape, size, movement, or the like for each of the foregrounds b1 to b4 and b6, and selects an appropriate pattern.

  Here, since the depth value set in the pattern only indicates the relative depth value in the pattern, in resetting the depth value in the 2D image, the 2D image serving as the reference for each foreground is used. Depth value is required. Therefore, the basic depth value of one pixel having the largest y value among the pixels (x, y) constituting the foreground for each foreground b1 to b4 and b6 is used as a reference value, and each foreground b1 to b4 and b6 is configured. The depth value of the pixel (x, y) is reset using the depth value of the pattern.

  When the composition is “up” type as a result of the determination, the control unit 1 resets the depth value of the pixel (x, y) constituting the foreground to the first depth value DPf, and the pixel ( x, y) resets the depth value to the second depth value DPg.

  An average value of basic depth values of the pixels constituting the foreground is defined as a first depth value DPf, and an average value of basic depth values of the pixels constituting the background is defined as a second depth value DPg.

  Here, the first depth value DPf and the second depth value DPg may be values calculated by a weighted average of the basic depth values weighted based on the number of pixels of each foreground or the variance value σ1. In this case, when the number of pixels constituting the foreground is large, the foreground is entirely on the near side, and the first depth value DPf is set to a small value. Further, when the variance value σ1 is large, it is considered that the subject is photographed from a distance, and the difference between the first depth value DPf and the second depth value DPg is set to be small.

  In “up” type images, the foreground is in focus, and the background is often out of focus. In order to generate a video with such a sense of depth and stereoscopic effect, it is necessary to make a large difference between the depth value of the foreground and the depth value of the background, so the depth value setting method differs from the “balance” type. It is set as follows.

  If the composition is a “bird's-eye view” type as a result of the determination, the basic depth value of, for example, one pixel having the largest y value among the foreground b6 made up of moving object constituent pixels is used as a reference value. The depth value of the foreground is reset according to the pattern of the depth value and the preset depth value.

  In the “overhead” type video, there are many so-called omnifocal images in which the entire frame is in focus. In this case, it is considered that subjects such as trees and buildings have depth values that do not differ greatly from the background even if they constitute the foreground. Therefore, it is only necessary to reset the depth value only for the pixels constituting the moving object in the foreground.

  By resetting the depth value in the foreground, the fast-moving area is considered to be the area in the foreground in the 3D image. The value may be calculated and reset.

  Further, in order to avoid processing complexity for a region where a plurality of foregrounds overlap, the depth value closest to the foreground among the overlapping foreground depth values may be reset.

  The control unit 1 generates a three-dimensional image of the left-eye video L and the right-eye video R having parallax based on the set and reset depth values.

  The control unit 1 calculates a shift amount SFTn (x, y) indicating the magnitude of the parallax based on the depth value calculated for each pixel in the nth frame of the 2D video. When the shift amount is SFTn (x, y) and the depth value is DPn (x, y), the control unit 1 calculates the shift amount by the equation (14) that represents the relationship between the depth value and the shift amount. The shift amount SFTn (x, y) is set to a value from -s to s.

  Here, for the sake of explanation, an equation with a linear change as shown in equation (14) is used for associating the depth value with the shift amount, but there is no problem even if the equation to be used is a nonlinear equation. Needless to say. Alternatively, a lookup table in which the shift amount corresponding to each depth value from 0 to d is created in advance may be used.

  Next, a left-eye video L and a right-eye video R are generated according to the calculated shift amount SFTn (x, y). Regarding the shift, the left-eye video L shifts the pixel in the right direction when the value of SFTn (x, y) is positive, and shifts the pixel in the left direction when the value is negative. On the other hand, for the right-eye video R, when the value of SFTn (x, y) is positive, the pixel is shifted leftward, and when it is negative, the pixel is shifted rightward.

  There may be a case where a pixel in which an effective pixel value cannot be obtained in a part of the image may be generated depending on the image. For this pixel, interpolation is performed from surrounding pixels to set an effective pixel value.

  The control unit 1 converts the three-dimensional image into a predetermined method for displaying on the display unit 10 and outputs the converted 3D image to the display unit 10 via the output unit 6.

  FIG. 10 is a schematic diagram illustrating an example of a predetermined method for displaying a three-dimensional image on the display unit 10. FIG. 10A is a schematic diagram showing a left-eye image L and a right-eye image R. The predetermined method includes a top-and-bottom method in which the left-eye video L and the right-eye video R are halved in the row direction as shown in FIG. 10B, and the column-direction resolution as shown in FIG. 10C. A side-by-side method in which the left-eye video L and the right-eye video R are stored in one frame, and a frame sequential method in which the left-eye video L and the right-eye video R are superimposed in the time axis direction as shown in FIG. 10D. is there.

  A processing procedure in the first embodiment will be described. FIG. 11 is a flowchart illustrating a processing procedure of the control unit 1 according to the first embodiment.

  When a 2D video is input from a broadcast wave receiving antenna (not shown) (step S10), the control unit 1 stores the 2D video data in the video storage unit 4 (step S12). The control unit 1 sequentially reads the stored two-dimensional video data (step S14).

  The control unit 1 extracts strong edge pixels for each pixel of the two-dimensional video (step S16), and then extracts moving object constituent pixels (step S18).

  The control unit 1 performs a labeling process on the foreground of the 2D video, and classifies the foreground and the background (step S20).

  The control unit 1 determines the composition of the two-dimensional video (step S22), calculates and sets the basic depth value of each pixel (x, y) (step S24). The control unit 1 resets the depth value based on the composition (step S26).

  Based on the depth value, the control unit 1 generates a 3D image of the left eye image L and the right eye image R (step S28).

  The control unit 1 converts the generated left-eye video L and right-eye video R into a predetermined format (step S30), and outputs the converted video to the display unit 10 via the output unit 6 (step S32).

  A part of the processing procedures of the control unit 1 will be further described. First, a processing procedure (step S16) in which the control unit 1 extracts strong edge pixels will be described. FIG. 12 is a flowchart illustrating a processing procedure in which the control unit 1 calculates the strong edge pixel information Hn (x, y).

  The control unit 1 assigns constant weights to the R component SnR (x, y), G component SnG (x, y), and B component SnB (x, y) of the pixel (x, y) as shown by the equation (1). A weighted average is taken to calculate a gray scale value Gn (x, y) (step S161).

  Further, processing using a Laplacian filter expressed by equation (2) is performed (step S162), and the edges of the two-dimensional image are emphasized. The control unit 1 calculates the edge intensity En (x, y) expressed by the equation (3) for all the pixels (step S163).

  The control unit 1 divides the entire frame into a predetermined number of blocks (step S164), and calculates Eave (j, k) that is an average value of the edge strength En (x, y) of the block to which the pixel (x, y) belongs. Calculate (step S165).

  The control unit 1 determines whether the edge intensity average Eave (j, k) is equal to or greater than a preset threshold Th0, calculates strong edge pixel information Hn (x, y), and determines strong edge pixel information Hn (x , Y) is a strong edge pixel of 1 (step S166).

  Next, a processing procedure (step S18) in which the control unit 1 calculates the moving object constituent pixel information Mn (x, y) will be described. FIG. 13 is a flowchart illustrating a processing procedure in which the control unit 1 calculates the moving object constituent pixel information Mn (x, y). The control unit 1 calculates the absolute difference information Dn (x, y) by taking the absolute value of the difference between the gray scale values Gn (x, y) and Gn−1 (x, y) as shown in the equation (8). (Step S181).

  Subsequently, it is determined whether or not the absolute difference information Dn (x, y) is greater than or equal to a preset threshold value Th1, and high difference information Bn (x, y) indicating whether or not the pixel has a large absolute value of the difference. Calculate (step S182).

  The control unit 1 performs the same calculation to calculate absolute difference information Dn + 1 (x, y) (step S183), high difference information Bn + 1 (x, y) (step S184), and high difference area Bn. The moving object configuration pixel information Mn (x, y) that is the logical product of (x, y) and Bn + 1 (x, y) is calculated, and the moving object configuration pixel information Mn (x, y) is 1. Pixels are extracted (step S185).

  A processing procedure (step S22) in which the control unit 1 determines the composition of the 2D video will be described. FIG. 14 is a flowchart illustrating a processing procedure in which the control unit 1 determines the composition of the two-dimensional video.

  The control unit 1 counts the pixels constituting the foreground (step S221), and determines whether or not the number of pixels is equal to or greater than a preset threshold value Th2 (step S222). If the number of foreground pixels is less than the threshold Th2 (NO in step S222), it is determined that the composition of the nth frame is the “balance” type (step S223).

  When the number of pixels constituting the foreground is equal to or greater than the threshold Th2 (YES in Step S222), pixels whose edge intensity En (x, y) is equal to or greater than a preset threshold Th3 are extracted (Step S224). The variance value σ1 for the pixel position indicated by the equation (13) is calculated (step S225).

  The control unit 1 determines whether or not the variance value σ1 is greater than or equal to the threshold value Th4 (step S226). If the variance value σ1 is less than the threshold value Th4 (NO in step S226), the composition is “up” type. It is determined that it exists (step S227). If the variance value σ1 is greater than or equal to the threshold Th4 (YES in step S226), it is determined that the composition is the “overhead” type (step S228).

  Then, the process sequence (step S26) in which the control part 1 resets a depth value is demonstrated. FIG. 15 is a flowchart illustrating a processing procedure in which the control unit 1 resets the depth value.

  The control unit 1 determines whether or not the composition determined by the above-described procedure is a “balance” type (step S261). In the case of the balance type (YES in step S261), one pixel having the largest y value is selected for each foreground b1 to b4 and b6 (step S262), and the basic depth value of the selected one pixel is used as a reference. Then, the depth values of the pixels (x, y) constituting the foregrounds b1 to b4 and b6 are reset (step S263).

  If the composition is not “balance” type (NO in step S261), it is determined whether or not the composition is “up” type (step S264).

  When the composition is the “up” type (YES in step S264), the pixels constituting the foreground are reset to the first depth value DPf (step S265), and the pixels constituting the background are set to the second depth value. The depth value DPg is reset (step S266).

  If the composition is not the “up” type (NO in step S264), the “overhead” type is selected. In this case, one pixel having the largest y value of the moving object constituent pixels constituting b6 is selected (step S267). Then, the depth value of the moving object constituent pixel is reset according to the basic depth value of the selected one pixel and the pattern of the preset depth value (step S268).

  A processing procedure (step S28) in which the control unit 1 generates a 3D video will be described. FIG. 16 is a flowchart illustrating a processing procedure in which the control unit 1 generates a 3D video.

  The control unit 1 calculates the shift amount SFTn (x, y) of the two-dimensional image corresponding to the depth value DPn (x, y) as shown in the equation (14) (step S281). Next, the control unit 1 shifts the two-dimensional video according to the calculated shift amount SFTn (x, y) (step S282). Here, a pixel in which an effective pixel value cannot be obtained for a part of the image due to pixel shift is interpolated from surrounding pixels (step S283), and an effective pixel value is set.

  The video generation apparatus according to the present embodiment calculates the depth value in consideration of factors such as the composition of the screen, the magnitude of the motion, and whether the foreground or the background, so that a natural three-dimensional video can be generated.

Second Embodiment A second embodiment will be described. In the second embodiment, when the composition is an “up” type, the first depth value and the second depth value are always set to constant values.

  In the “up” type video, a predetermined value is always used for the depth value in the pixels that make up the foreground and the depth value in the pixels that make up the background, creating a video with a sense of depth and stereoscopic effect. You may be able to. Moreover, the process of the control part 1 can be simplified by always using a fixed value for the depth value.

  Therefore, the control unit 1 in this embodiment does not set the basic depth value when the composition is the “up” type, and sets the depth values of the foreground and the background to constant values.

  The control part 1 in this Embodiment is demonstrated. When a 2D video is input from the broadcast wave receiving antenna, the control unit 1 causes the video storage unit 4 to store 2D video data. The controller 1 sequentially reads the recorded 2D video data.

  The control unit 1 extracts strong edge pixels and moving object constituent pixels, performs a labeling process on the foreground of the two-dimensional image, and classifies the foreground and the background. The control unit 1 sets the depth value based on the foreground information Pn (x, y) and the edge strength En (x, y).

  The control unit 1 counts the foreground pixels and determines whether the number of pixels is equal to or greater than a preset threshold value Th2. When the number of pixels in the foreground is less than the threshold Th2, the composition is a “balance” type, so the control unit 1 generates a basic depth value and resets the foreground depth value.

  When the number of foreground pixels is equal to or greater than the threshold Th2, pixels whose edge intensity En (x, y) is equal to or greater than the threshold Th3 are extracted, and a variance value σ1 for the positions of these pixels is calculated.

  The control unit 1 determines whether or not the variance value σ1 is equal to or greater than the threshold value Th4. If the variance value σ1 is equal to or greater than the threshold value Th4, the composition is “up” type. A first depth value DPf, which is a constant value, is set, and a second depth value DPg, which is a constant value, is set for the pixels constituting the background.

  When the variance value σ1 is less than the threshold value Th4, the composition is “overhead” type, so the basic depth value is calculated and set, and the depth value of the moving object constituent pixel is reset.

  Based on the depth value DPn (x, y), the control unit 1 generates a three-dimensional image of the left-eye video L and the right-eye video R, and uses the generated left-eye video L and right-eye video R in a proper manner. The data is converted and transmitted to the display unit via the output unit 6. The display unit 10 displays a 3D image.

  A processing procedure of the control unit 1 in the present embodiment will be described. FIG. 17 is a flowchart illustrating a processing procedure of the control unit 1 according to the second embodiment. When the 2D video is input from the broadcast wave receiving antenna (step S40), the control unit 1 stores the data of the 2D video in the video storage unit 4 (step S42). The control unit 1 sequentially reads the stored 2D video data (step S44).

  The control unit 1 extracts strong edge pixels for each pixel of each frame of the 2D video (step S46). Further, the moving object constituent pixels are extracted (step S48).

  The control unit 1 performs a labeling process on the foreground of the 2D video. Further, the foreground information Pn (x, y) is calculated by calculating the logical sum of the strong edge region information Hn (x, y) and the moving object constituent pixel information Mn (x, y), and the foreground and the background are distinguished. (Step S50).

  The control unit 1 determines the composition and calculates and sets the depth value based on the determined composition (step S52).

  Based on the depth value, the control unit 1 generates a 3D image of the left eye image L and the right eye image R (step S54). The control unit 1 converts the generated left-eye video L and right-eye video R into a predetermined format (step S56), and outputs the converted video to the display unit 10 via the output unit 6 (step S58).

  The processing procedure (step S52) in which the control unit 1 sets the depth value will be further described. FIG. 18 is a flowchart illustrating a processing procedure in which the control unit 1 according to the second embodiment sets a depth value.

  The control unit 1 counts the pixels constituting the foreground (step S521), and determines whether or not the number of pixels is equal to or greater than a preset threshold value Th2 (step S522).

  If the number of pixels in the foreground is less than the threshold Th2 (NO in step S522), the composition is “balance” type, so the control unit 1 sets a basic depth value (step S523), and the pixels ( Among the x, y), one pixel having the largest y value is selected (step S524), and the depth value of the foreground is reset (step S525).

  If the number of foreground pixels is greater than or equal to the threshold Th2 (YES in step S522), pixels whose edge intensity En (x, y) is greater than or equal to the threshold Th3 are extracted (step S526), and the positions of these pixels are determined. Is calculated (step S527).

  The control unit 1 determines whether or not the variance value σ1 is greater than or equal to the threshold value Th4 (step S528). If the variance value σ1 is greater than or equal to the threshold value Th4 (YES in step S528), the composition is “up” type. Therefore, the depth value DPf is set for the foreground information (step S529), and the depth value DPg is set for the background information (step S530).

  If the variance value σ1 is less than the threshold value Th4 (NO in step S528), the composition is “overhead” type, so a basic depth value is set (step S531). One pixel which is the largest pixel is selected (step S532), and the depth value of the moving object constituent pixels is reset (step S533).

  This embodiment can generate an image with a sense of depth and a stereoscopic effect by using constant values for the depth value of the foreground and the depth value of the background in the “up” type image, The process of the control unit 1 can be simplified.

Third Embodiment A third embodiment will be described. In the third embodiment, the processing procedure for determining the composition performed by the control unit 1 is different from the processing procedure (step S22) in the first embodiment. In the present embodiment, since the variance value of the edge intensity of all pixels is used in determining the composition of the video, the composition of the video can be accurately determined.

  In determining the composition of the two-dimensional video, the control unit 1 in the present embodiment first counts the pixels constituting the foreground. It is determined whether or not the counted number of pixels is equal to or greater than a preset threshold value Th2. When the number of pixels in the foreground is less than the threshold value Th2, the composition is determined to be a “balance” type.

  When the number of foreground pixels is equal to or greater than the threshold Th2, the variance value σ2 of the values of the edge intensity En (x, y) of all the pixels is calculated according to equation (15).

  When the image is up of the subject, pixels with strong edges are concentrated in the area where the subject has moved. On the other hand, when the video is a bird's-eye video, a large number of subjects are scattered, so pixels with strong edges are scattered throughout the frame.

  Therefore, it is considered that the image of the subject up generally has a larger variance value σ2 of the edge intensity En (x, y) than the overhead image.

  It is determined whether or not the variance value σ2 is greater than or equal to the threshold Th5. If the variance value σ2 is greater than or equal to the threshold Th5, it is determined that the composition is the “up” type. If the variance value σ2 is less than the threshold Th5, it is determined that the composition is the “overhead” type.

  Next, a processing procedure in which the control unit 1 determines the composition in the present embodiment will be described. FIG. 19 is a flowchart illustrating a processing procedure in which the control unit 1 according to the third embodiment determines the composition.

  The control unit 1 counts the foreground pixels (step S621), and determines whether the number of pixels is equal to or greater than a preset threshold Th2 (step S622). If the number of foreground pixels is less than the threshold Th2 (NO in step S622), it is determined that the composition of the nth frame is the “balance” type (step S623).

  If the number of foreground pixels is equal to or greater than the threshold Th2 (YES in step S622), the variance value σ2 of the edge intensity En (x, y) of all pixels is calculated (step S624).

  It is determined whether or not the variance value σ2 is greater than or equal to the threshold Th5 (step S625). If the variance value σ2 is less than the threshold Th5 (NO in step S625), it is determined that the composition is the “up” type (step S625). Step S626). If the variance value σ2 is greater than or equal to the threshold Th5 (YES in step S625), it is determined that the composition is the “overhead” type (step S627).

  According to the present embodiment, since the variance value of the edge intensity of all pixels is used in determining the composition of the video, the composition of the video can be accurately determined.

Fourth Embodiment FIG. 20 is a block diagram showing hardware parts of a television receiver according to a fourth embodiment. A program for operating the control unit 1 is stored in the ROM 2 by causing the reading unit 9 such as a disk drive to read the portable recording medium 20A such as a CD-ROM, a DVD disk, or a USB memory.

  Here, the program may include a semiconductor memory 20B such as a flash memory storing the program in the control unit 1, and is connected to the communication unit 8 through a communication network N such as the Internet. You may download from the server computer.

  The control unit 1 shown in FIG. 20 reads a program for executing the above-described various software processes from a portable recording medium or a semiconductor memory, or downloads it from another server computer (not shown) via the communication network N. The program is installed as a control program, loaded into the ROM 2 and executed. Thereby, it functions as the control unit 1 described above.

  The embodiments disclosed herein are illustrative in all respects and should not be considered as restrictive. The scope of the present invention is defined not by the above-mentioned meaning but by the scope of the claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of the claims.

  For example, the composition in the present invention is not limited to the three types described above, and other compositions may be provided based on the size, movement, or arrangement of the subject.

  On the other hand, the control unit 1 does not have to determine the composition. In this case, the control unit 1 extracts the pixels constituting the foreground from the strong edge region information Hn (x, y) and the moving object constituent pixel information Mn (x, y), and calculates the depth value of the extracted pixels by the method described above. It may be reset to generate a 3D video.

  Further, the strong edge region information Hn (x, y) and the moving object constituent pixel information Mn (x, y) include pixel values SnR (x, y) and SnG (x instead of the gray scale value Gn (x, y). , Y), one or more of SnB (x, y), and may be calculated by determining whether or not the pixel value is equal to or greater than a predetermined threshold value. The absolute difference information Dn (x, y) may be calculated based on the gray scale value of the pixel (x, y) in three or more frames, or the pixel values of the R, G, and B components in three or more frames. You may calculate using one or more. Of course, the RGB color space information shown in SnR (x, y), SnG (x, y), SnB (x, y) is converted into another color space represented by HSV, YUV, etc. It goes without saying that is possible.

1 Control unit 2 ROM
3 RAM
4 Video storage unit 5 Input unit 6 Output unit 7 Tuner 8 Communication unit 9 Reading unit 10 Display unit 20A Portable recording medium 20B Semiconductor memory

Claims (7)

In a video generation device that generates a three-dimensional video by calculating a depth value indicating a depth corresponding to each pixel of a two-dimensional video constituted by a plurality of pixels,
A first difference calculation unit that calculates a difference between a pixel value of each pixel constituting the two-dimensional image and a pixel value of a pixel adjacent to each pixel;
A first extraction unit that extracts a pixel whose difference calculated by the first difference calculation unit is equal to or greater than a predetermined first threshold together with position information of the pixel ;
A second difference calculation unit for calculating a difference between a pixel value in one frame of each pixel constituting the two-dimensional video and a pixel value in another frame;
A second extraction unit that extracts a pixel whose difference calculated by the second difference calculation unit is equal to or greater than a predetermined second threshold together with position information of the pixel ;
Based on the number of pixels or the position information extracted by the first extraction unit and the second extraction unit, it is determined whether the composition type of the 2D video is a plurality of predetermined types. Discriminating part to perform and
A depth calculation unit that calculates a depth value corresponding to each pixel of the two-dimensional video based on a calculation result of the first difference calculation unit or the second difference calculation unit and a composition type determined by the determination unit. A video generation apparatus characterized by that. A third extraction unit that extracts pixels whose difference calculated by the first difference calculation unit is greater than or equal to a predetermined third threshold value that is greater than the first threshold value, and dispersion of positions where each pixel extracted by the third extraction unit exists A dispersion value calculation unit for calculating a dispersion value at a position indicating the degree of
The depth calculation unit is configured to calculate the depth value based on the variance value of the position calculated by the variance value calculation unit and the total number of pixels extracted by the first extraction unit or the second extraction unit. The video generation device according to claim 1, wherein: A second variance value calculation unit that calculates a variance value of the difference calculated by the first difference calculation unit;
The depth calculation unit calculates the depth value based on the variance value of the difference of each pixel calculated by the second variance value calculation unit and the total number of pixels extracted by the first extraction unit or the second extraction unit. The image generation device according to claim 1, wherein the image generation device is configured to calculate. A video display device comprising: the video generation device according to any one of 1 to 3; and a display unit that displays the three-dimensional video generated by the video generation device. A tuner unit that receives TV broadcast waves including two-dimensional video,
The video generation device according to any one of claims 1 to 3, which generates a 3D video based on the 2D video received by the tuner unit, and a display for displaying the 3D video generated by the video generation device A television receiver characterized by comprising a unit. In a video generation method for generating a three-dimensional video by calculating a depth value indicating a depth corresponding to each pixel of a two-dimensional video constituted by a plurality of pixels,
A first difference calculating step for calculating a difference between a pixel value of each pixel constituting the two-dimensional image and a pixel value of a pixel adjacent to each pixel;
A first extraction step of extracting a pixel whose difference calculated in the first difference calculation step is equal to or greater than a predetermined first threshold together with position information of the pixel ;
A second difference calculating step of calculating a difference between a pixel value in one frame of each pixel constituting the two-dimensional video and a pixel value in another frame;
A second extraction step of extracting a pixel whose difference calculated in the second difference calculation step is equal to or greater than a predetermined second threshold together with position information of the pixel ;
Based on the number of pixels or the position information extracted in the first extraction step and the second extraction step, it is determined whether the composition type of the 2D video is a plurality of predetermined types. Discriminating step and
Including a calculation result of the first difference calculation step or the second difference calculation step, the depth calculation step of calculating a depth value corresponding to each pixel of the two-dimensional image based on the type of composition in which the determination step determines A video generation method characterized by performing processing . In a computer program for generating a three-dimensional image by calculating a depth value indicating a depth corresponding to each pixel of the two-dimensional image composed of a plurality of pixels,
A first difference calculating step for calculating a difference between a pixel value of each pixel constituting the two-dimensional image and a pixel value of a pixel adjacent to each pixel;
A first extraction step of extracting a pixel whose difference calculated in the first difference calculation step is equal to or greater than a predetermined first threshold together with position information of the pixel ;
A second difference calculating step of calculating a difference between a pixel value in one frame of each pixel constituting the two-dimensional video and a pixel value in another frame;
A second extraction step of extracting a pixel whose difference calculated in the second difference calculation step is equal to or greater than a predetermined second threshold together with position information of the pixel ;
Based on the number of pixels or the position information extracted in the first extraction step and the second extraction step, it is determined whether the composition type of the 2D video is a plurality of predetermined types. Discriminating step and
A depth calculating step of calculating a depth value corresponding to each pixel of the two-dimensional video based on a calculation result of the first difference calculating step or the second difference calculating step and a composition type determined by the determining step. A computer program for causing a computer to execute processing.

Images