JP4236428B2 - Stereoscopic image display method and stereoscopic image display apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to stereoscopic image processing technology, and more particularly, to a method, apparatus, system, and related computer program and data structure for processing or displaying a stereoscopic image.
[0002]
[Prior art]
Over the past few years, the Internet population has increased rapidly and is entering a broadband era, a new stage of Internet usage. In broadband communication, the communication band is dramatically widened, so the distribution of heavy image data, which has been often avoided in the past, is also popular. Concepts such as “multimedia” and “video on demand” have been proposed for a long time. However, in the broadband era, these words can only be experienced by ordinary users.
[0003]
If the distribution of images, especially moving images, spreads, the user naturally wants to enrich the content and improve the image quality. These are largely due to the digitization of existing video software, the development of an authoring tool therefor, and the pursuit of high-efficiency and low-loss image coding technology.
[0004]
[Problems to be solved by the invention]
Under such circumstances, as one form of image distribution service in the near future, distribution of pseudo three-dimensional images (hereinafter also simply referred to as “stereoscopic images”) has attracted technical attention, and it is considered that a considerable market is acquired. Stereoscopic images fulfill the user's desire for more realistic images, and are especially attractive for applications that pursue a sense of reality, such as movies and games. Furthermore, stereoscopic images are also useful for the realistic display of products in product presentations in EC (electronic commerce), which is considered to be one of the standards for commerce in the 21st century.
[0005]
However, it is difficult to say that user-friendly display technology has been presented in promoting the spread of stereoscopic images. The present inventor has made the present invention by paying attention to such a current situation, and an object of the present invention is to provide a stereoscopic image processing technique for assisting a user to correctly display a stereoscopic image.
[0006]
[Means for Solving the Problems]
In order to understand the present invention, the following concepts in this specification are first defined.
“Stereoscopic image”: The term “stereoscopic image” refers to an image that is projected on the user's eyes as a result of being displayed stereoscopically rather than image data itself. Image data that can be displayed as a stereoscopic image is called a “multiplexed image” to be described later. That is, when a multiplex image is displayed, a stereoscopic image can be seen.
[0007]
“Parallax image”: Usually, for stereoscopic viewing with a sense of depth, an image that should be thrown to the right eye (hereinafter simply referred to as a right eye image) and an image that should be cast to the left eye (hereinafter simply referred to as a left eye image) so as to generate parallax. It is necessary to prepare. A pair of images that cause parallax, such as a right-eye image and a left-eye image, may be collectively referred to as a parallax image. In this specification, each image that causes a parallax is referred to as a parallax image. That is, both the right eye image and the left eye image are parallax images. In addition to these, generally, images from the respective viewpoints assumed in the stereoscopic image are respectively parallax images.
[0008]
“Basic image”: An image that is subject to processing necessary for stereoscopic vision or an image that has already been processed in order to display a stereoscopic image. As a specific example, as in a multiplex format, an image (which is also referred to as a “composite image”) that is already formed by combining a plurality of parallax images in some form is included.
[0009]
"Multiplex format": One of the basic image composition modes. Final image data format for displaying stereoscopic images. A basic image in a multiplex format is also simply referred to as a “multiplex image”.
[0010]
“Viewpoint”: A stereoscopic image is assumed to have a viewpoint of viewing it. The number of viewpoints and the number of parallax images are usually equal. When there are two parallax images of a left-eye image and a right-eye image, the number of viewpoints is “2”. However, even if there are two viewpoints, the assumed position of the user's head is one. Similarly, when displaying a stereoscopic image considering the movement of the user in the left-right direction, for example, assuming four viewpoints va, vb, vc, vd in the left-right direction, the parallax images that can be seen from each are Ia, Ib, Ic, Id Then, for example, a stereoscopic image with a sense of depth can be displayed by three sets of parallax images (Ia, Ib), (Ib, Ic), and (Ic, Id). In this state, if four images viewed from the upper direction and four images viewed from the lower direction are used in order to generate a stereoscopic image that wraps in the vertical direction, The number of viewpoints is “8”.
[0025]
Of the present invention is there The aspect relates to a stereoscopic image display method. This method includes a step of displaying an image to be stereoscopically displayed on the screen of the display device and a step of moving the image on the screen, and the unit amount of movement of the image is a different parallax image on which the image is based It is constrained to a value according to the number of.
[0026]
The unit amount may be an integer multiple of a value obtained by multiplying the number of parallax images by a drawing unit of an image, for example, a display unit constituting a pixel, for example, the number of pixels. Assuming that the drawing element is 3 pixels of RGB and the number of viewpoints is 4, an integral multiple of 12 is the unit amount of image movement. When the image is displayed in the window, the unit amount may be referred to at least when scrolling the image in the window.
[0027]
Another embodiment of the present invention relates to a stereoscopic image display apparatus. The apparatus includes a movement instruction acquisition unit that acquires an instruction for moving an image to be stereoscopically viewed on the screen, and a movement processing unit that moves the image according to the instruction. The unit amount of movement is constrained to a value corresponding to the number of different parallax images that are the basis of the image and moved.
[0028]
The image processing apparatus may further include a window display unit that displays a window for displaying an image on the screen, and the movement processing unit may perform restriction based on a unit amount when scrolling the image in the window.
[0029]
It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.
[0030]
DETAILED DESCRIPTION OF THE INVENTION
As a technique for displaying a stereoscopic image, a case is assumed in which a multiplexed image is displayed on the LCD screen and a parallax barrier is attached to the LCD. The positional relationship between the parallax barrier and the multiplexed image needs to be adjusted at the pixel pitch level. The pixel pitch of a standard LCD is about 0.1 mm, and it is very difficult and troublesome to adjust this with a mouse or the like. If the positional relationship between the multiplexed image and the parallax barrier is deviated, the user cannot see the stereoscopic image. Therefore, it is necessary to adjust the positional relationship between them, and the present embodiment relates to a stereoscopic image dedicated viewer that facilitates the adjustment.
[0031]
Embodiment 1:
FIG. 1 is a diagram showing a state in which the viewer 10 according to the present embodiment is displayed on the LCD 8 and the parallax barrier 12 is stuck on the LCD 8. It is assumed that a multiplexed image is displayed on the viewer 10. At this time, since the viewer 10 is located at a position shifted from the parallax barrier 12, the user cannot view a stereoscopic image. As shown in FIG. 2, the viewer 10 is adjusted to an appropriate position with respect to the parallax barrier 12, and window information such as the start position and size of the viewer 10 and the image display area 14 is saved by operating the button 11 after the adjustment is completed. . The storage destination is preferably a place that can be retained even after the software is terminated, such as an initial file of the viewer 10.
[0032]
Once the proper position as shown in FIG. 2 is determined, the position and size of the viewer 10 may be changed again as shown in FIG. 1, and the proper positions of the multiplex image and the parallax barrier 12 may be shifted. When the button 18 is operated at this time, the saved window information is read, and the viewer 10 matches the parallax barrier 12 as shown in FIG. Since the start position of the viewer 10 and the start position of the image display area 14 are reproduced in the same state as in FIG. 2, the first pixel is the correct position with respect to the parallax barrier 12. As a result, the positional relationship of the entire multiplex image with respect to the parallax barrier 12 becomes correct. Even if the state of the viewer 10 once set in this way is changed, it can be easily returned to the original state.
[0033]
The same information can be used for a new viewer 10 that displays another stereoscopic image. In addition, even when the viewer 10 has to be resized by accepting a change in the size of the viewer 10 while keeping the start positions of the viewer 10 and the image display area 14 fixed, a new multiplexed image can be obtained. Is placed in an appropriate position with respect to the parallax barrier 12.
[0034]
As described above, this window information can be used not only for the viewer 10 which is the target when the window information is saved, but also for the viewer 10 opened separately and the initial window when the software is activated. Further, software that is different from the set software may be used. Further, the window information may be stored in a plurality of states as well as in one state. In this case, the window information is selected at the time of reading.
[0035]
In general, there are a plurality of appropriate image start positions. When one appropriate image start position is determined, the other appropriate image start positions are also determined. For example, in the case of a four-eye system with four viewpoints, if it is found that a certain position is an appropriate start position, a pixel that is 12 pixels away from that position is also an appropriate start position. This 12 is a value obtained by multiplying the number of viewpoints 4 by the number of pixels 3 constituting the pixel. Further, all pixels that are a multiple of 12 are appropriate start positions. Therefore, for example, when the window is moved, the optimum start position may be calculated based on the start position recorded in this way, and the image may be displayed with the position as a new start position. Also, when displaying a new image of a different size, for example, if the image is always set to be displayed at the center of the screen, a new appropriate start position is calculated and the image is displayed with that position as the new start position. May be.
[0036]
In addition to the button operation, the save or read command may be selected from the list menu 16 on the viewer 10 functioning by the mouse operation as shown in FIG. 3, or by other keyboard operation or voice. It may be selected.
[0037]
Further, the button may not be on the viewer 10, and may be located anywhere as long as it can be operated, such as on another window.
[0038]
A stereoscopic image display apparatus 102 for realizing the above functions is shown in FIG. The stereoscopic image display apparatus 102 acquires window information such as the display position and size of the viewer 10, stores the window information as an appropriate position in the storage unit 108, receives an instruction from the user, and stores the information in the storage unit 108. A position reading unit 110 for reading the stored appropriate position, a position adjusting unit 112 for moving the viewer 10 to the read appropriate position, a display control unit 114 for displaying the viewer 10 at the appropriate position, and a user instruction A GUI (Graphical User Interface) 99 is provided.
[0039]
This configuration can be realized in terms of hardware by a CPU, memory, or other LSI of an arbitrary computer, and in terms of software, it is realized by a program having an image position adjustment function loaded in the memory. So, functional blocks that are realized by their cooperation are drawn. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.
[0040]
The operation of the configuration of the stereoscopic image display apparatus 102 will be described with reference to the flowcharts of FIGS. FIG. 5 is a flowchart showing a window information saving process. When the user presses the button 11 in FIG. 2, the window information is acquired by the position writing unit 104 (S10). Further, the acquired information is updated in the initial file of the viewer 10 and stored in the storage unit 108 (S12).
[0041]
FIG. 6 is a flowchart showing a window information reading process. The position reading unit 110 reads the stored window information from the initial file (S20), acquires the current window information (S22), and changes the information as necessary (S24). Based on the updated information, the position adjustment unit 112 adjusts the position of the viewer 10, and redraws the viewer 10 together with the multiplexed image (S26).
[0042]
According to the first embodiment described above, even when the proper position of the multiplex image with respect to the parallax barrier is shifted, the user can move the multiplex image to the proper position without performing complicated adjustment work. . Further, when another multiplex image is newly displayed, it becomes easy to display the image at an appropriate position.
[0043]
Embodiment 2:
First, the issues to be overcome are described. Here, it is assumed that there is a window whose position and size can be freely changed, and a multiplex image is displayed therein. The positional relationship between the multiplex image and the parallax barrier is closely related to the observation position of the user, and if the stereoscopic image deviates from the appropriate position, the relationship is broken.
[0044]
7 to 9 show the relationship between the multiplex image 151, the parallax barrier 152, and the observer. The multiplexed image 151 is synthesized from four types of first to fourth parallax images. Each square represents a pixel, and the assigned numbers 1 to 4 correspond to the numbers of the original parallax images. That is, a pixel of the first parallax image is used for a pixel to which “1” is allocated, and a pixel of the second parallax image is used for a pixel to which “2” is allocated. To do. Hereinafter, a pixel in which a pixel of the nth parallax image is used is also simply referred to as a pixel n.
[0045]
Assume that the number of parallax images is 4, the pixel pitch is P, and n is an integer. In FIG. 7, if the shift amount is P × 4n, there is no serious problem that there is a possibility that the end of the image cannot be stereoscopically viewed. However, in other cases, the relationship with the user position is greatly broken. That is, if the set of parallax images viewed by the user is (4, 3) (3, 2) (2, 1), the user can view a stereoscopic image.
[0046]
If the shift amount is P × (4n + 1) or P × (4n + 3), the user can view a stereoscopic image when the front position, that is, the set of parallax images is (2, 1) as shown in FIG. Although it is possible, if it moves to the left or right direction, it will be in the position of reverse view immediately. Further, when the shift amount is P × (4n + 2), the optimum position as shown in FIG. 9 is a reverse view position.
[0047]
As described above, since the pixel pitch P is about 0.1 mm in a standard liquid crystal panel, it is difficult to adjust the window position in units of pixels by operating the mouse. Even if the parallax barrier has a mechanical fine adjustment function, such fine adjustment takes time.
[0048]
In addition, even if such an adjustment is once completed and a stereoscopic image can be viewed, a new multiplexed image and a parallax barrier are adjusted each time to view another stereoscopic image. This is annoying.
[0049]
Also, depending on the multiplex image creation environment, the order in which the N parallax images are arranged is not necessarily the same. Even if a different image is displayed in the same area after a correct adjustment in one image, the adjustment is performed from the beginning. It may be necessary to redo. If the composition order is reversed, stereoscopic display may not be possible.
[0050]
The reverse order of synthesis means that, for example, the arrangement of parallax images is reversed between the method as shown in FIG. 10 where the parallax barrier is arranged on the exit side of the liquid crystal panel and the method as shown in FIG. Therefore, it depends on the creator which image is to be created.
[0051]
Further, when the number N of parallax images is 2, it is up to the creator to decide which image to select for the first pixel column, and it is not determined.
[0052]
The viewer according to the second embodiment overcomes the above problems. This viewer also has a function of appropriately moving the multiplex image inside.
[0053]
FIG. 11 shows a viewer 55 for displaying a multiplexed image on the LCD 54 and a parallax barrier 56 attached to a part of the LCD 54. Three buttons 51, 52 and 53 are arranged in the viewer 55. In the viewer 55, it is assumed that four types of parallax images having parallax in the horizontal direction are rearranged every other pixel row, that is, a multiplex image having four viewpoints is displayed. The viewer 55 is moved from a position 65 away from the parallax barrier 56 as shown in FIG. 12 to the vicinity of a position overlapping the parallax barrier 56 by a mouse operation or the like, and the size of the viewer 55 is adjusted.
[0054]
When the left button 51 is operated in such a state, as shown in FIG. 13A, in the image display area 32a in the viewer 55, four pixels are grouped and the order is one pixel on the right in the group. The pixel that has moved and protruded is arranged at the left end in the group, and the image region 32a is programmed so as to change from the initial state 31 to the adjusted state 311a. As a result, the region where the stereoscopic image can be seen at the observation position moves to the left by one region.
[0055]
In addition, the program is set so that the reverse process is performed by the operation of the right button 52, and the adjusted state 312a is changed from the initial state 31. As a result, the region where the stereoscopic image can be seen at the observation position moves to the right by one region. Thus, every time one pixel is moved, the region where the stereoscopic image can be seen is shifted one by one. Further, by operating the left button 51, the entire image display area 32b is moved in the same direction so as to be in the adjusted states 311b and 312b, not within the group, as shown in FIG. 13B. May be.
[0056]
By the way, a color liquid crystal panel usually forms one pixel by three color pixels of red (R), green (G), and blue (B) separated in the horizontal direction.
[0057]
When pixels are represented for each of the four types of parallax images, a total of 4 × 3 = 12 pixels are required. For normal image display processing, it is sufficient to consider the pixel as a minimum unit, but here, an optimal stereoscopic display is realized by exchanging pixels between pixels. Therefore, as shown in FIG. 14A, the order in the group may be changed with 12 pixels as a group and 1 pixel, here, 3 pixels of RGB as a unit. Further, as shown in FIG. 14B, the entire image display area may be moved in units of three pixels. At this time, the movement of the area where the stereoscopic image is observed by one button operation becomes three areas. That is, for example, a pixel displayed at a certain position changes from pixel 1 to pixel 4.
[0058]
For example, when the number of viewpoints is 8, if attention is paid to the pixel whose first area number is 1, as shown in FIG. 15A, the number is changed every time a button operation is performed (1 → 4 → 7 → 2 → 5 → 8). → 3 → 6) Changes. Each moving pixel number is (0 → 1 → 2 → 3 → 4 → 5 → 6 → 7) with respect to the initial state.
[0059]
As described above, it is possible to move every three areas with respect to one operation. However, there is a case where it is desired to move one area by one operation while moving the image area in units of three pixel columns. . As shown in FIG. 15B, when attention is paid to the leftmost pixel, this is the pixel 1 and has an R element. When moving one area, the leftmost pixel needs to be a pixel 2 having an R element. Therefore, in such a case, if the number of pixels to be moved every time an operation is performed is set to (0 → 3 → 6 → 1 → 4 → 7 → 2 → 5) every three pixels, The edge area number changes as (1 → 2 → 3 → 4 → 5 → 6 → 7 → 8). The same is possible when the number of parallax images is different.
[0060]
Further, as shown in FIG. 16A, 12 pixels may be moved as a group, for example, by a process as shown in FIG.
[0061]
FIG. 17 is a flowchart illustrating a process in which a region where a stereoscopic image is observed moves to the right by one region. Here, g is the number of groups, m is the number of parallax images, c is the number of pixels constituting one pixel, data [n, h] is the luminance information of the hth pixel in group n, and data ′ [n ′, h “] Is luminance information of the h′-th pixel of the new group n ′. Column [n, j, k] is used to temporarily store the luminance information of the image before moving, adding the parallax image number and RGB information, and to generate new image data after moving. The In the case of a binocular system, that is, when the number of parallax images is 2, m = 2, and in the case of a 4-eye system, m = 4. In the LCD, since one pixel is composed of three RGB pixels, c = 3.
[0062]
First, the variable x is initialized to 1 (S51). Subsequently, the variables n and n ′ are initialized to 1 (S52). Similarly, variables h, j, and k are initialized to 1 (S53). The variable t is also initialized to 1 (S54). Next, the value of the luminance information data [n, h] of the image is substituted into Column [n, j, k] (S55). If the variable j is incremented by 1 (S56) and the incremented variable j is larger than m (Y in S57), the variable j becomes 1. If the variable j is less than or equal to m (N in S57), the variable j remains as it is. Subsequently, the variable k is incremented by 1 (S59). If the variable k is larger than c (Y in S60), the variable k is 1 (S61). If the variable k is less than or equal to c (N in S60), the variable k remains as it is.
[0063]
Subsequently, the variable h is incremented by 1 (S62), and the variable t is also incremented by 1 (S63). Next, when the variable t is less than or equal to the value of c × m (N in S64), the process returns to Step S55, and Steps S55 to S63 are repeated. When the variable t exceeds the value of c × m (Y in S64). Then, the process proceeds to the next processing step S65. By repeating steps S55 to S63, the luminance information of one group of the image before movement becomes data having the parallax image number and RGB information. Subsequently, the variable h ′ and the variable k are initialized to 1 and the variable j is initialized to 2 (S65). Next, the variable u is initialized to 1 (S66). The value of Column [n, j, k] is assigned to data ′ [n ′, h ′] (S67). Next, the variable j is incremented by 1 (S68). If the variable j is larger than m (Y in S69), the variable j is 1 (S70). If the variable j is m or less (N in S69), the variable j remains as it is.
[0064]
Subsequently, the variable k is incremented by 1 (S71). If the variable k is larger than c (Y in S72), the variable k becomes 1 (S73). If the variable k is less than or equal to c (N in S72), the variable k remains as it is. Subsequently, the variable h ′ is incremented by 1 (S74). The variable u is incremented by 1 (S75). If the variable u is smaller than 12 (N in S76), the process returns to step S67, and steps S67 to S75 are repeated. If the variable u has a value of c × m (Y in S76), the process proceeds to the next step S77. By repeating steps S67 to S75, luminance information of one group after movement is generated. Next, the variables n and n ′ are incremented by 1 (S77). The variable x is incremented by 1 (S78). If the number of groups g does not exceed 12 (N in S79), the process returns to step S53, and steps S53 to S78 are repeated. When the variable x exceeds the number of groups g (N in S79), the process ends.
[0065]
Further, as shown in FIG. 16B, the entire image display area may be moved in units of one pixel. At this time, stereoscopic image information and window background information may be mixed in pixels in the pixels around the image display area. However, by making the window background colorless, the occurrence of unnatural colors can be prevented. it can. Alternatively, there is a method of not displaying on the screen pixels that may contain such information.
[0066]
Note that the amount of movement of a region where a stereoscopic image is observed by a single command is not limited to one region, but may be moved in units of more regions. In that case, the movement amount of the pixel in the group becomes a value other than 1.
[0067]
When the center button 53 in FIG. 11 is operated, as shown in FIG. 18, the image area 72 on the window is reversed in the order of four adjacent pixel rows as a group, and the initial state 71 is synthesized. The program is set up from the order (4, 3, 2, 1) to the synthesis order (1, 2, 3, 4) of the adjusted state 711. By reversing the pixel rows, the order of the regions where the stereoscopic image can be seen at the observation position is reversed. Here, for example, the composition order (1, 2, 3, 4) indicates that when the multiplexed images are synthesized, the parallax images are arranged pixel by pixel from the first parallax image in order from the left. Yes.
[0068]
Also in this case, when the pixel is composed of a plurality of pixels, all the first pixels RGB of the four types of images are expressed in the first column to the twelfth column. If the conversion is performed by grouping pixels, the order is substantially reversed. FIG. 20 is a flowchart showing this conversion procedure. However, FIG. 20 shows only similarities to FIG.
[0069]
Step 65 of FIG. 17 is changed to step S65a, and step S67 is changed to step 67a. As a result, in the flowchart of FIG. 17, the process in which the region where the stereoscopic image is observed moves to the right by one region is shown, but in the flowchart of FIG. 20, the order in which the parallax images are combined is reversed. The difference that processing is done appears.
[0070]
With the processing shown in FIGS. 17 and 20, the region in which the stereoscopic image can be seen is moved in the horizontal direction and inverted with respect to the horizontal direction. Further, the movement of the region where the stereoscopic image can be seen in the vertical direction and the reversal in the vertical direction are performed in the same manner. Therefore, by repeating these operations, the positional relationship between the stereoscopic image and the parallax barrier becomes optimum, and the region where the stereoscopic image can be seen becomes an appropriate position.
[0071]
The horizontal or vertical movement operation may be performed by moving the window position where the image display area is located by one pixel unit or one pixel unit as shown in FIG. FIG. 22 shows a flowchart of the horizontal movement process. First, the horizontal coordinate X of the start position of the window is acquired (S232). Subsequently, the moving pixel number M is acquired (S234). The one obtained by adding the moving pixel number M to the previously acquired horizontal coordinate X becomes the horizontal coordinate X of the start position of the new window (S236). Finally, a new window is drawn (S238).
[0072]
If the window is moved in units of three pixels and the effective display area is moved in units of one pixel, the two are combined. For example, if the effective display area moves by three pixels, only the frame of the window is displayed without changing the position of the effective display area. May be controlled so that the position of the effective display area in the window is always optimal. Of course, the unit of pixel movement in the image display area or the unit of pixel movement when moving the window can be changed.
[0073]
A stereoscopic image display device 120 for realizing the above functions is shown in FIG. The stereoscopic image display apparatus 120 includes an adjustment instruction acquisition unit 122 that receives an instruction to adjust a multiplex image from a user, an adjustment unit 124 that performs adjustment according to the instruction on the multiplex image, and an adjusted multiplex image. A display control unit 126 for displaying and a GUI 128 for facilitating reception of instructions from the user are provided.
[0074]
The adjustment instruction acquisition unit 122 accepts an instruction for the above-described processing, for example, rearrangement of image data constituting the image and movement processing to an appropriate position for the multiplex image. The adjustment unit receives the instruction and performs processing on the multiplexed image.
[0075]
Embodiment 3:
In the third embodiment, a method for synthesizing a multiplex image from a parallax image will be described. FIG. 24 is a diagram schematically showing this synthesis method. The display software performs the work of combining multiple parallax images into a multiplex image, and a program is created to change the order of combining the multiplex images displayed in the window by button operation on the window. ing. That is, if there are first to fourth parallax images and the composition order at the time of image composition is the composition order (1, 2, 3, 4), the composition order is shifted by one by the button operation and the composition order (2 3, 4, 1). In another button operation, the composition order is the composition order (4, 1, 2, 3). If the button operation is repeated, the composition order can be changed sequentially.
[0076]
The button operation is performed by a mouse operation, and each time the composition order is shifted by 1, the region where the image can be seen at the observation position is shifted one by one. There is another button on the window, and a program is set to reverse the state of the composite image from the composite order (1, 2, 3, 4) to (4, 3, 2, 1). Yes.
[0077]
By combining these two operations, it becomes easy to adjust the positional relationship correctly. The readjustment is also easy when another image with a different synthesis order is displayed.
[0078]
Note that the operation for reversing the composition order is performed with a button for changing the composition order, not another button, and the composition image is composed in the composition order (1, 2, 3, 4), (2, 3, 4, 1). , (3, 4, 1, 2), (4, 1, 2, 3), (4, 3, 2, 1), (3, 2, 1, 4), (2, 1, 4, 3) , (1, 4, 3, 2) may be changed.
[0079]
In addition, when one pixel is formed with three colors of pixels RGB as in the liquid crystal panel, if the first parallax image to the fourth parallax image have the same number of pixels as the synthesized image, respectively, For the pixels in the first column of (1, 2, 3, 4), R (red) in the first column of the first parallax image is selected. If this is expressed as 111R, the pixels are expressed as 111R, 221G, 331B, 442R, 512G, 622B, 733R, 843G, 913B,... Sequentially from the left as shown in FIG. 843G means that “G (green) in the third column of the fourth parallax image is selected for the pixels in the eighth column”. Then, when the composition order is (2, 3, 4, 1) by the button operation, the selected pixel is changed, and 121R, 231G, 341B, 412R, 522G, 632B, 743R, 813G, 923B,. It becomes. As a result, when multiplex images are combined, only 1/4 of each parallax image is used. The same applies to other states.
[0080]
In addition, when images 1 to 4 are pre-compressed in the horizontal direction and the four pixels have the same number of pixels as the composite image, the composite order (1, 2, 3, 4) is from the left. In order, 111R, 221G, 331B, 441R, 511G, 621B, 731R, 841G, 911B,. Then, when the composition order is (2, 3, 4, 1) by the button operation, they are rearranged in order from the left, such as 121R, 231G, 341B, 411R, 521G, 631B, 741R, 811G, 921B. The same applies to other states.
[0081]
In the operation of reversing the order, if images 1 to 4 have the same number of pixels as the composite image, the composite order (4, 3, 2, 1) is 141R, 231G, 321B, 412R, 542G in order from the left. 632B, 723R, 813G, 943B.
[0082]
On the other hand, in the case of an image compressed in the horizontal direction, the composition order (4, 3, 2, 1) is 141R, 231G, 321B, 411R, 541G, 631B, 721R, 811G, 941B,. .
[0083]
In the case of binocular stereoscopic display, the effects of moving the pixel, moving the window, and changing the order of the left and right images are all the same. It only needs to have one function.
[0084]
The above-described operation may be selected from a list menu on a window functioning by mouse operation, by keyboard key operation, by remote control operation, or by voice.
[0085]
Further, the image separation means such as an optical filter may be arranged not on a part of the screen but on the entire screen. In addition, the stereoscopic display may be performed not on the window but on the entire screen.
[0086]
In addition to the parallax barrier, the image separation means may be any device as long as it has the same effect as a lenticular lens.
[0087]
Still further, the image separation means may be a polarized glasses type stereoscopic display device using a micropolarizer as shown in FIG. A micropolarizer is a half-wave version of a half-wavelength plate that is finely processed into stripes. By placing it on the front side of the output-side polarizing plate of a liquid crystal panel, the polarization angle is changed by 90 ° every other row. It is. Then, when wearing polarized glasses whose polarization angles are different from each other by 90 °, the left and right images are separated and a stereoscopic image can be viewed. In this example, since the micropolarizer is formed in the horizontal direction, the parallax image is displayed in units of rows, and the moving direction of the image is the vertical direction.
[0088]
Further, even in a multi-view stereoscopic image having parallax in the vertical direction, it is necessary to move in the vertical direction.
[0089]
Embodiment 4:
Usually, as shown in FIG. 27, the image displayed on the viewer 92 is shifted by the scroll bar 90, and the displayed area moves from the display state of FIG. 27A to the display state of FIG. .
[0090]
In the case of a stereoscopic image, when the shift amount is an integral multiple of the number of pixels constituting the pixel, the observed image pair changes even though the observer does not move the head position. FIG. 28 shows the shift amount when the number of viewpoints is 2. The numbers in the image represent viewpoint numbers, that is, the numbers of the parallax images. For example, the pixel 1R represents the R of the first parallax image. This indicates that a pixel having an element is used. The pixels constituting the pixel are RGB 3 and are shifted by 3 pixels here. If the viewpoint numbers 1 and 2 after the shift are switched, and the observer is not moving, the normal vision and the reverse vision are reversed.
[0091]
A method for overcoming this problem is shown below. The shift amount is an integral multiple of a value obtained by multiplying the number of viewpoints of the image by the number of pixels constituting the pixel. Here, the number of viewpoints is 2, and the number of pixels constituting the pixel is 3. Therefore, the shift amount is an integer multiple of 6. FIG. 29 shows a state where the pixel is shifted by 6 pixels. Viewpoint numbers 1 and 2 are not interchanged before and after the shift. Therefore, this shift does not affect when viewing a stereoscopic image. The same can be said even if the number of viewpoints increases. The same applies to a stereoscopic image having parallax in the vertical direction.
[0092]
FIG. 30 shows a stereoscopic image display device 140 that realizes the above-described method. The stereoscopic image display device 140 displays a movement instruction acquisition unit 142 that receives an instruction from a user for moving a multiplex image, a movement processing unit 144 that moves a multiplex image according to the received instruction, and a multiplex image. A display control unit 146 and a GUI 148 that presents an object such as a scroll bar 90 to the user are provided.
[0093]
The movement instruction acquisition unit 142 receives a shift instruction from the user by the scroll bar 90. If the instruction relates to a shift in the horizontal direction, the movement processing unit 144 shifts the multiplex image in units of an amount obtained by multiplying the number of pixels constituting the pixel and the number of parallax images.
[0094]
The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.
[0095]
Such a modification will be described. In the above-described embodiment, the case where the parallax barrier covers a part of the LCD screen is illustrated, but the same processing is possible even when the parallax barrier covers the entire LCD screen. Not too long. In particular, when the parallax barrier covers the entire screen of the LCD, there are many situations where reverse viewing should be prevented other than when the window is scrolled.
[0096]
For example, when a window is displayed on the screen, there are a multi-window mode in which a part of the screen is displayed together with other windows, and a full-screen mode in which only the window is displayed on the entire screen. The user can display between these modes. Aspect may be changed. Also, when an image is enlarged or reduced, the image may be automatically arranged near the center of the screen. In addition, since the parallax barrier is on the entire screen, the movement range of the window is widened, and the user may move the window to various positions for stereoscopic viewing. When the display mode is changed in this way, it is necessary to redraw the image, and it is necessary to adjust the drawing start position so that normal stereoscopic viewing is possible.
[0097]
As described in the first embodiment, there can be a plurality of candidates for an appropriate drawing start position at the time of redrawing. That is, not only the position stored as the appropriate start position in the first drawing, but also the position that is separated from that position by an integer multiple of the value obtained by multiplying the number of viewpoints by the number of pixels constituting the pixel, Position. The position adjustment unit 112 of the stereoscopic image display apparatus 102 according to Embodiment 1 may adjust the drawing position by selecting one of a plurality of appropriate start positions at the time of redrawing. For example, when switching to the full screen display, a position as close as possible to the edge of the screen is selected as an appropriate start position. When the user drags the window with a mouse or the like and moves the window to an appropriate position, a position closest to the position where the drag is released is selected as an appropriate start position.
[0098]
In the case of full-screen display, since an appropriate start position is generally shifted from the edge of the screen, there is a region where no image is displayed at the edge of the screen. In that sense, it is not strictly a full-screen display. For such an area where pixel information is missing at the edge of the screen, processing such as non-display, background color, or interpolation is performed.
[0099]
FIGS. 31A to 31C are diagrams illustrating an appropriate start position of an image with respect to an optical filter such as a parallax barrier. Here, a case where the number of viewpoints is four will be described. The number of the parallax image to be displayed on each pixel, that is, the viewpoint number is determined from the relationship between the optical filter and the screen. In FIG. 31A, an image 152 drawn at an appropriate start position is displayed with respect to a column 150 of viewpoint numbers to be displayed on each pixel. The viewpoint number of the parallax image synthesized with the image 152 matches the viewpoint number to be displayed on each pixel, and the RGB colors of each pixel also match. An appropriate drawing start position of the image 152 is a position 161, and a position away from this position by an integer multiple of 12 obtained by multiplying the number of viewpoints 4 by the number of pixels 3 constituting the pixel is also an appropriate start position. . In this example, the image 152 is drawn with the position 161 as a start position, and when this is a full screen display, an area in which there are four pixels left from that position is generated.
[0100]
In FIG. 31B, the positional relationship between the optical filter and the screen is different from that in FIG. For the viewpoint number column 154 to be displayed on each pixel determined from the relationship between the optical filter and the screen, the same image 152 as in FIG. In this case, an appropriate drawing start position of the image 152 is a position 163, and a position away from this position by an integer multiple of 12 is also an appropriate start position. Here, the image 152 is drawn with the position 163 as the start position, and in the full-screen display, there is an area where pixels do not exist for three pixels to the left of the position.
[0101]
In FIG. 31 (c), the positional relationship between the optical filter and the screen is the same as in FIG. 31 (a), but the arrangement of the viewpoints of the parallax image combined with the image 156 is different from FIG. 31 (a). An image 156 is drawn at an appropriate starting position as shown in the figure for the viewpoint number column 150 to be displayed on each pixel. An appropriate drawing start position of the image 156 in this case is a position 168, and a position away from this position by an integer multiple of 12 is also an appropriate start position. Here, the image 156 is drawn with the position 168 as the start position, and in the full-screen display, there is a region in which pixels do not exist for five pixels to the left of the position.
[0102]
In the above example, an appropriate start position is set for any pixel of RGB, but in a normal image, RGB is handled as a set, so the appropriate start position is only for the R pixel. Usually, it exists every 12 pixels.
[0103]
Instead of adjusting the drawing start position at the time of redrawing, adjustment may be made so that reverse viewing does not occur by changing the synthesis order of the parallax images constituting the image. If the user needs to redraw the image by moving the window or enlarging or reducing the image, the position of the image is not changed, and the order of synthesis of the parallax images is also changed. The same effect of preventing reverse viewing can be obtained.
[0104]
In FIG. 32A, an image 172 that is freely moved is displayed with respect to a column 170 of viewpoint numbers to be displayed on each pixel determined from the positional relationship between the optical filter and the screen. In this state, the viewpoint number of the parallax image synthesized with the image 172 does not match the viewpoint number to be displayed on each pixel. Therefore, the order of the parallax images combined with the image 172 is changed. FIG. 32B shows an image 174 in which the synthesis order of parallax images is changed. The viewpoint number of the parallax image synthesized with this image 174 matches the viewpoint number to be displayed on each pixel, and the RGB colors of each pixel also match. The method described in Embodiment 3 can be used as processing for changing the synthesis order of parallax images and rearranging pixels.
[0105]
【The invention's effect】
Adjustment is facilitated when displaying a stereoscopic image.
[Brief description of the drawings]
FIG. 1 is a diagram showing a state in which a parallax barrier is pasted on an LCD and displayed on a viewer and an LCD according to an embodiment.
FIG. 2 is a diagram illustrating a state in which a viewer and a parallax barrier according to the embodiment are matched.
FIG. 3 is a diagram showing another form of the viewer according to the embodiment.
FIG. 4 is a configuration diagram of a stereoscopic image display apparatus according to an embodiment.
FIG. 5 is a flowchart showing a procedure for saving window information.
FIG. 6 is a flowchart showing a procedure of window information reading processing;
FIG. 7 is a diagram illustrating a relationship between an observer, a multiplexed image in an appropriate positional relationship, and a parallax barrier.
FIG. 8 is a diagram illustrating a relationship between an observer, a multiplexed image that is not in an appropriate positional relationship, and a parallax barrier.
FIG. 9 is a diagram illustrating a relationship between an observer, a multiplexed image that is not in an appropriate positional relationship, and a parallax barrier.
FIG. 10 is a diagram illustrating a state in which a parallax barrier is disposed on the emission side of the liquid crystal panel.
FIG. 11 is a diagram showing a state in which the viewer and the parallax barrier according to the embodiment are matched.
FIG. 12 is a diagram showing a state in which the viewer and the parallax barrier according to the embodiment are displaced.
FIG. 13 is a diagram illustrating two methods of moving an image display area by one pixel.
FIGS. 14A and 14B are diagrams illustrating two examples of a method of moving an image display region by 3 pixels with 12 pixels as one group. FIGS.
FIG. 15 is a diagram illustrating an example of movement of an image display area when the number of viewpoints is eight.
FIG. 16 is a diagram illustrating an example of movement of an image display area.
FIG. 17 is a flowchart illustrating a processing procedure in which a region where a stereoscopic image is observed moves to the right by one region.
FIG. 18 is a diagram illustrating an example of movement of an image display area.
FIG. 19 is a diagram illustrating an example of movement of an image display area.
FIG. 20 is a flowchart showing a processing procedure when converting pixels by grouping 12 pixels.
FIG. 21 is a diagram showing an example of moving a window position where an image display area exists in units of one pixel or one pixel.
FIG. 22 is a flowchart showing a procedure for horizontal movement processing;
FIG. 23 is a configuration diagram of a stereoscopic image display device according to an embodiment.
FIG. 24 is a diagram simply illustrating a method for synthesizing a multiplexed image having four viewpoints.
FIG. 25 is a diagram illustrating a procedure for synthesizing a multiplexed image having four viewpoints.
FIG. 26 is a diagram showing stereoscopic image display using a micropolarizer and polarized glasses.
FIG. 27 is a diagram illustrating an example in which a displayed area is shifted by a scroll bar.
FIG. 28 is a diagram illustrating an example in which an image with two viewpoints is moved by three pixels.
FIG. 29 is a diagram illustrating an example in which an image with two viewpoints is moved by 6 pixels.
FIG. 30 is a configuration diagram of a stereoscopic image display device according to an embodiment.
FIG. 31 is a diagram illustrating an appropriate start position of an image with respect to a parallax barrier.
[Fig. 32] Fig. 32 is a diagram for describing the order of an image arranged on a parallax barrier and a parallax image synthesized with the image.
[Explanation of symbols]
10 viewers, 11 buttons, 12 parallax barriers, 16 list menus, 55 viewers, 56 parallax barriers, 90 scroll bars, 102 stereoscopic image display devices, 104 position writing units, 108 storage units, 110 position reading units, 112 positions The adjustment unit, 114 display control unit, 120 stereoscopic image display device, 122 adjustment instruction acquisition unit, 124 adjustment unit, 126 display control unit, 140 stereoscopic image display device, 142 movement instruction acquisition unit, 144 movement processing unit.

Claims (3)

A display process for displaying the multiplexed image on the screen of the display device;
Moving the multiplex image on the screen, and
In the moving step, the unit amount of movement of the multiplex image is a stereoscopic image display method that is constrained to an integral multiple of the number of different parallax images constituting the multiplex image ,
The stereoscopic image display method, wherein when the multiplex image is displayed in a window, the unit amount is referred to at least when scrolling the multiplex image in the window. A display process for displaying the multiplexed image on the screen of the display device;
Moving the multiplex image on the screen, and
In the moving step, a stereoscopic image display in which a unit amount of movement of the multiplex image is an integral multiple of a value obtained by multiplying the number of different parallax images constituting the multiplex image by the number of display units of the multiplex image. A method,
The stereoscopic image display method, wherein when the multiplex image is displayed in a window, the unit amount is referred to at least when scrolling the multiplex image in the window. A movement instruction acquisition unit for acquiring an instruction for moving the multiplex image on the screen;
A movement processing unit that moves the multiplex image according to the instruction,
The movement processing unit is a stereoscopic image display device that moves the multiplex image while restricting a unit amount of movement of the multiplex image to an integral multiple of the number of different parallax images constituting the multiplex image.
A window display unit that displays a window for displaying the multiplex image in the screen; and the movement processing unit performs the restriction by the unit amount when scrolling the multiplex image in the window. A stereoscopic image display device characterized by the above.

Images