JP2006262191A - Multiple viewpoint solid image display method, multiple viewpoint solid image display unit and multiple viewpoint solid image display program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To observe individual solid image information in a display mode adaptable for each of a plurality of observers by use of a set of solid image display unit. <P>SOLUTION: In a multiple viewpoint solid image display method, image information b separated by an information separator 11 is decoded in the display mode having a different reproduction permitting degree differed at each observer by a decoder 15, and displayed by using an image display means 16. Further, the solid image information different from each other with respect to each viewpoint of the plurality of observers who observe from a different position is displayed through a local solid image information output means 17. When this takes place, the solid image information adaptable for each of the plurality of observers can be displayed based on private information d and auditory limit information c. Thus, an adult does not take care of the safety of a child's eyes and can enjoy solid image contents together with a child. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a multi-view 3D video display method, a multi-view 3D video display device, and a multi-view 3D video display program, and in particular, using a single 3D video display device, The present invention relates to a multi-view 3D video display method, a multi-view 3D video display device, and a multi-view 3D video display program that suitably realize a stereoscopic video display device that can be observed.

  Various methods have been proposed so far for the practical application of 3D displays (stereoscopic video displays). These methods are (1) a method of sampling parallax information (light intensity and direction information) from a subject (binocular, multi-view, integral photography (IP), etc.), (2) Methods for sampling subject depth information (three-dimensional position information and luminance information of the volume element of the subject) (moving screen method, DFD method, etc.), (3) light wave information from the subject (light intensity and phase information) Are classified into three categories (holographic, CGH, etc.) (see, for example, Non-Patent Document 1).

  The twin-lens system belonging to the above category is a method of observing a stereoscopic image without using special glasses. There are various methods for this, but a stereogram is currently used as a practical method. Are known. The stereogram takes out only two types of light rays that are incident on the pupils of the right eye and left eye of the observer of the subject from among the infinite number of light rays scattered in all directions from the surface of the three-dimensional object (sampled). ) To obtain a parallax image, and then display the image on a plane such as a display.

  The directions of the light beams are controlled so that the light beam groups from the two types of parallax images displayed on the display are incident on the pupils of the right eye and the left eye of the observer observing the display. It is well known that there are a refractive optical element or an aperture optical element as such a light beam controlling means, a lenticular lens system as a system using the former, and a parallax barrier system as a system using the latter.

  Now, the stereo images of the lenticular lens method, parallax barrier method, and other stereograms described above are more visible in the brain than the display surface even though the focus of the eyes is on the display surface, which is the light emission surface of the parallax image. It has the property of being recognized as existing in the front or back. This property is caused by the fact that the point (convergence point) where the vectors of the line-of-sight directions of both eyes facing each other inward in the stereoscopic view do not coincide with the focus by the focus adjustment of the eye lens.

  This discrepancy between vergence and accommodation requires a visual state that is completely different from stereoscopic vision in daily life, and is known to cause biological effects such as eye fatigue and discomfort to the observer. (For example, refer nonpatent literature 2). In this Non-Patent Document 2, it is expected that a natural 3D image display that does not get tired will be put into practical use for several decades or more, and for the time being, a 3D image display in which the eye adjustment function does not work. This is a situation where a stereogram must be used, and a stereogram that requires a special visual form different from the case of viewing a three-dimensional object in the natural world has various biological effects on the human body. Is described.

  In order to alleviate the biological effect due to the contradiction of convergence adjustment in such a conventional stereoscopic video display, an attempt is made to display the distance between the convergence point and the adjustment point so as not to become excessively large (for example, Patent Document 1). reference). That is, Patent Literature 1 discloses a technique for acquiring a biological reaction caused by a visual stimulus by an image and using the trigger as a trigger signal to control a screen or an observation apparatus. By applying this technique, it is possible to control the display contents so that the contradiction between convergence and adjustment does not exceed a certain level while monitoring the eye movement of the observer.

  As another stereoscopic video display method belonging to the category (1), there is IP. This IP is a photographic technique known from about 100 years ago as a method capable of reproducing a three-dimensional image without visual burden, and has been applied as a medical photograph (for example, see Patent Document 2). Application as a natural three-dimensional display is expected by digitization. However, such an electronic IP has not yet reached the stage of displaying a practical three-dimensional image because the resolution of the current display element is about two orders of magnitude lower than that of a photographic film. Moreover, since the high definition of the display element depends on the progress of the semiconductor process technology, it is expected that it will take some time for the two-digit miniaturization. Therefore, the mainstream of current IP research and development is not the research of computerized IP itself, but the research and development of a new method that can be used for practical applications while using display elements of the same resolution as the current resolution. It is shifting.

  As one of such research and development, a stereoscopic image display using the concept of “super multi-view” is known. This is to reduce the interval between viewpoint images in a conventional multi-view video display, and to perform stereoscopic display in a state where a plurality of viewpoint images are incident on a monocular pupil (super multi-view state). . According to this method, the adjustment movement of the crystalline lens can be induced by the monocular parallax effect, so it is an innovative concept that solves the problem of visual burden caused by convergence and adjustment mismatch, which are the main causes of eye fatigue. It has been.

  The details are disclosed in the final result report of the advanced stereoscopic video communication project conducted by the communication / broadcasting organization from 1992 to 1997 (see, for example, Non-Patent Document 3). In addition, as a technique related to sound associated with stereoscopic video, there is known a stereoscopic acoustic technique that can present different sounds depending on the listening position by localizing a sound field in a space using a plurality of speakers (for example, Non-patent document 4).

NHK Broadcasting Technology Laboratory, “Basics of 3D Video”, Ohmsha, 1995 Japan Association for Mechanical Systems Promotion, “Proposed Guidelines for 3D Video”, Japan Electronics and Information Technology Industries Association, July 2002 Communications and Broadcasting Organization, “Advanced 3D Video Communication Project Final Report”, pages 179-192, September 1997 Jens Brauert, “Spatial Acoustics”, Kashima Press, 1985 JP 2003-157136 A Japanese Patent No. 1897532

  However, the stereoscopic image display device disclosed in Patent Document 1 is considered to function effectively for a stereoscopic display that wears spectacles such as a head-mounted display, but for an autostereoscopic display that does not use spectacles. However, there is a problem that it is not easy for an observer to detect a biological reaction without wearing a sensor. Further, even if the biological reaction of the observer is detected by some method, the autostereoscopic display has the following problems due to the possibility of being touched by human eyes other than the observer.

  That is, even if a stereoscopic image is displayed so as to be suitable for a certain observer, it may not be suitable for another observer near the observer. For example, consider a situation in which a parent and a child watch a stereoscopic image together in a living room at home. In such a situation, even if the amount of popping out is appropriate for the parent, the amount of popping out is too large for the child, resulting in significant fatigue, or conversely, even if the amount of popping out is appropriate for the child, It can happen that the amount of popping out is too small to satisfy. In particular, if the child is an elementary school student or less, there is concern about the serious impact on normal eye development, so parents should not be able to see a scene with a large amount of popping out. Must pay attention to.

  On the other hand, IP is an ideal method in which an eye adjustment function works on a stereoscopic image in the future. However, when an electronic device at the present time is used, not only the problem of low resolution but also a serious problem that the eye adjustment function does not always work. As will be described in detail later, according to the super multi-view concept, when the number of rays incident on the eye is small in IP, the adjustment works on the position of the local stereoscopic image 210 or 211 as shown in FIG. Not necessarily.

  Even if the resolution of the electronic device is improved in the future and a super multi-view state can be realized, there are the following problems. Even in such a super multi-view display, the number of light rays incident on the eyes actually differs depending on the viewing distance. Therefore, when a plurality of viewers observe at the same time in the home, it depends on the viewing position. It is quite possible to watch in a multi-view state that is out of the super-multi-view state and does not function in the eyes. Therefore, even with IP, which is considered to be an ideal system, there is a possibility that a child and an adult cannot be enjoyed safely and simultaneously by simply increasing the resolution of the electronic device.

  The present invention has been made in view of the above points. A multi-viewpoint stereoscopic image in which a plurality of observers can individually observe stereoscopic video information in a display mode suitable for each using a single stereoscopic video display device. It is an object to provide a video display method, a multi-view stereoscopic video display device, and a multi-view stereoscopic video display program.

  Another object of the present invention is to provide an excessive amount of protrusion to a child when a plurality of observers observing a conventional autostereoscopic display with eye fatigue are included. To provide a multi-view 3D video display method, a multi-view 3D video display device, and a multi-view 3D video display program for restricting the display of accompanying stereoscopic video content and displaying such unrestricted content for other adults is there.

  Furthermore, another object of the present invention is to maintain a super multi-view state for a child when a plurality of observers observing a natural stereoscopic display such as IP are included. Control the display mode of stereoscopic video content so that it does not fall into the multi-view state, or if it is difficult to maintain the super-multi-view state, display it in the background instead of the image that pops out in front of the display To provide a multi-view stereoscopic video display method, a multi-view stereoscopic video display apparatus, and a multi-view stereoscopic video display program.

  In order to achieve the above object, a first aspect of the present invention is a multiple-view stereoscopic video display method for displaying different stereoscopic video information for each viewpoint of a plurality of observers using a single stereoscopic video information display means. In the first step of inputting the position information and personal information of a plurality of observers, the depth information and / or viewpoint interval information given in advance for each predetermined section of the stereoscopic video information and the personal information, A second step of individually determining the display mode of the local stereoscopic video information for each of the plurality of observers, and a second step for each of the plurality of observers having the input position information And a third step of outputting local stereoscopic video information in the displayed mode.

  In order to achieve the above object, the second invention is a multi-viewpoint stereoscopic image that displays different stereoscopic image information for each viewpoint of a plurality of observers using one stereoscopic image information display means. In the display device, information input means for inputting position information and personal information of a plurality of observers, personal information input by the information input means, depth information given in advance for each predetermined section of stereoscopic video information, and / or Or the display mode determining means for determining the display mode of the local stereoscopic video information individually for each of the plurality of observers by combining the viewpoint interval information, and the plurality of the position information input by the information input means And local stereoscopic video information output means for outputting local stereoscopic video information in the display mode determined by the display mode determining means for each observer.

  In order to achieve the above object, a third aspect of the present invention provides a multi-viewpoint for causing a computer to execute a multi-viewpoint 3D image display apparatus that displays different 3D image information for each view point of a plurality of observers. In the stereoscopic image display program, a first step of linking and storing position information and personal information of a plurality of observers in a computer, and depth information and / or a viewpoint given in advance for each predetermined section of the stereoscopic image information A second step of individually determining the display mode of the local stereoscopic image information for each of the plurality of observers by combining the interval information and the personal information, and each of the plurality of observers having the input position information On the other hand, a third step of outputting local stereoscopic video information in the display mode determined in the second step is executed.

  In order to achieve the above object, according to the present invention, there is provided a multi-view stereoscopic video for displaying different local stereoscopic video information for each of the viewpoints of a plurality of observers using a single stereoscopic video information display means. In the display method, the step of linking the position information of the viewpoint with the personal information of the observer having the viewpoint and inputting the information to the stereoscopic video information display means, and a predetermined section of the stereoscopic video information input to the stereoscopic video information display means A step of individually determining an appropriate display mode for each observer by combining personal information and depth information and / or viewpoint interval information given in advance for each position, and a position by matching the display mode with position information Outputting local stereoscopic video information in a display manner to each observer having information.

  To achieve the above object, the present invention provides a multi-view stereoscopic video display method for displaying different local stereoscopic video information for each of the viewpoints of a plurality of observers using a single video display device. In the above, by combining the depth information of the stereoscopic video content of the local stereoscopic video information corresponding to each observer information of the observer and the viewpoint position of the observer, the local stereoscopic video information suitable for each of the observers A display mode is individually determined, and the same stereoscopic video content is displayed as a local video in an appropriate depth range for each observer in the display mode.

  In order to achieve the above object, the present invention provides a multi-view stereoscopic video display device that displays different stereoscopic video information for each viewpoint of a plurality of observers using a single stereoscopic video information display means. , Information input means for inputting the position information and personal information of the observer, display mode determination means for determining an appropriate display mode based on the control information and personal information given to the stereoscopic video information, and an observer having position information And a local stereoscopic video information output means for locally outputting stereoscopic video information according to a display mode.

  Here, the local stereoscopic video information output means includes an image display means for displaying an image by dividing the area spatially and / or temporally, and an observer for each area of the traveling direction of light rays from the image. And optical deflecting means for optically deflecting based on the positional information.

  In order to achieve the above object, the present invention provides a multi-view stereoscopic video that displays different local stereoscopic video information according to the respective viewpoints of a plurality of observers using one stereoscopic video information display means. In the display device, information input means for inputting viewpoint position information and / or personal information of the observer, information input by the information input means, and depth information and / or viewpoint interval information included in the stereoscopic video content Display mode determining means for individually determining the display mode for each observer based on the above, and local stereoscopic video information output means for locally outputting local stereoscopic video information to each of the observers in the display mode It is characterized by having.

  In order to achieve the above object, the present invention provides a multi-view stereoscopic video display that displays different local stereoscopic video information for each of the viewpoints of a plurality of observers using a single stereoscopic video information display means. In the apparatus, by combining the depth information of the stereoscopic video content of the local stereoscopic video information corresponding to each observer information of the observer and the viewpoint position of the observer, the local stereoscopic video suitable for each of the observers A depth expression mode is individually determined, and stereoscopic video content is displayed for each observer in the determined depth expression mode.

  Furthermore, in order to achieve the above object, the present invention provides a multi-view stereoscopic video display for controlling a stereoscopic video information display device that displays different local stereoscopic video information for each of a plurality of observer viewpoints. In the program, the step of storing the position information of the viewpoint and the personal information of the observer having the viewpoint linked to the computer, and the depth included in each predetermined section of the stereoscopic video information input to the stereoscopic video information display means A step of reading information and / or viewpoint interval information, a step of determining an appropriate depth expression mode for each observer by combining depth information and / or viewpoint interval information and personal information, and a depth expression mode and / or viewpoint By making the interval information correspond to the position information, the local stereoscopic image information is expressed in depth for each observer who has the position information. Characterized in that and a step of outputting the like.

  According to the present invention, when one stereoscopic video information display means is used to display different stereoscopic video information for each viewpoint of a plurality of observers observing from different positions, it is suitable for each of a plurality of observers. 3D video information can be displayed so that adults can enjoy 3D video content with children without worrying about the safety of children's eyes. Gram) has the effect of contributing to the spread of ordinary households.

  Further, according to the present invention, when displaying different stereoscopic image information for each viewpoint of a plurality of observers observing from different positions using one stereoscopic image information display means, a plurality of observers are individually displayed. Appropriate 3D image information can be displayed, so that 3D image information suitable for each observer can be displayed. As a result, 3D images can be displayed without worrying about the difference in age and visual ability between the observers. The content can be viewed together with peace of mind, and the effect of making the stereoscopic viewing style barrier-free is achieved.

  Next, the best mode for carrying out the invention will be described with reference to the drawings. FIG. 1 shows a block diagram of a first embodiment of a multi-viewpoint stereoscopic image display apparatus according to the present invention. In the figure, first, a digital stereoscopic video content a which is the content of video information by an encoded digital signal is input to the information separator 11.

  The digital stereoscopic video content a is configured by multiplexing pure video information b and viewing restriction information c that restricts reading of the video information b. For example, a movie as digital stereoscopic video content a recorded on an optical disc such as a DVD (Digital Versatile Disc) is age-related as viewing restriction information c for selectively reproducing a part of the information together with all video information b. Parental lock information related to regulations and region codes related to regional regulations are recorded. Of course, in the present embodiment, the digital stereoscopic video content a is not limited to content recorded on an information recording medium such as an optical disc, and information on a Web server is communicated to a storage medium of an information terminal. Needless to say, the present invention can be similarly applied to content downloaded via a line or content stored on a Web server and supplied by streaming distribution.

  The information separator 11 has a function of separating multiplexed information included in the above-described digital stereoscopic video content a, and specifically, a demultiplex well known in a digital signal processing circuit. It is. As shown in the figure, the information separator 11 separates the multiplexed digital stereoscopic video content a into video information b and viewing restriction information c, and inputs the separated video information b to the decoder 15. The separated viewing restriction information c is input to the display mode determiner 13.

  On the other hand, as another input to the multi-viewpoint stereoscopic image display device, personal information d of a plurality of observers who observe the images and position information e which is information on the relative positional relationship with the display are observers. The information is input to the information input device 12.

  The personal information d of the observer includes at least the age and / or nationality and / or vision information of the observer. As will be described later, the age of the viewer is input according to the age of the viewer, and to what extent content with mental development problems such as sexual or violent scenes included in the digital stereoscopic video content a This is to determine whether to reproduce. This is similar to the known parental lock technology that comes with a general DVD player and allows a parent to set child viewing restrictions before playing a disc. The present embodiment is characterized in that this conventional technique is developed so that a plurality of viewers of different age groups can view the same content at different playback levels.

  Another reason for inputting the viewer's age is that the depth expression amounts of viewable stereoscopic images are different. That is, as described in Non-Patent Document 2, a visual development stage exists in childhood, and a pseudo three-dimensional system using binocular parallax and motion parallax at such a time It should be noted that watching a stereo stereoscopic image with an excessive amount of depth expression may affect normal visual growth, so that sufficient care must be taken.

  For example, looking at the results of investigations on the cornea, retina, binocular stereopsis, fringe vision, electrical stimulus blink reflex, optic nerve, light stimulus blink reflex, and regulation function from 0 to 10 years old, children are active There are various growth periods (Sekiya: “Infant development, science of vision” (1998), Yamamoto: “Development of visual acuity, neuro-ophthalmology 5” (1998), Sugawara, et al .: “Infant vision Examination of sensitivity period of amblyopia viewed from difficulty of reading, Bulletin of Japanese ophthalmology (1984), Tatsumi: “Functional and morphological development of visual field, new clinical ophthalmology” (1990), Dobson: “Children's visual acuity from infants American Orthopt Journal 35 "(1985), Yamamoto, Sekiya:" 1998 Reporter 1-3 "(1998), Sekiya, et al .:" Photic blink reflex in childhood, Neuro-Ophthalmology 3 "(1986), Okuda:" Optical nerve Journal of Japanese Society for Myelination (1985) ), Yako: “Development of regulation, ophthalmology MOOK38” (1989), etc.). From these data, it can be seen that a playback system that takes into account the biological effects on children with respect to video must be considered.

  Further, the reason why the nationality of the observer is input is to allow the reproduction of scenes that are problematic in the idea or creed included in the digital stereoscopic video content a to be restricted depending on the nationality of the observer, as will be described later. is there. This relates to a known region code technology that is set for each shipping region when an optical disc player is sold. The present embodiment is characterized in that this conventional technique is developed so that a plurality of observers of different nationalities can view the same content at different playback levels.

  Of course, there are many observers with different thoughts and beliefs even though they have the same nationality. Accordingly, in the implementation related to the input of the personal information d, it is conceivable that not only the nationality but also the religion or the like believed by the individual can be directly input. However, since it may cause an unexpected human rights problem when inputting the matter related to the individual's inner mind when viewing the video in this way, an application for implementing this embodiment must be carefully considered.

  Among applications where multinational observers observe images, examples where the present invention can be suitably implemented are installed in movie theaters, international conference halls, international stadiums, etc. used by people from various countries. There is a large screen display. As for movies, content with thought and creed biases throughout the scenario is meaningless even if the present invention is applied, but for movies that have problems only in some scenes, that part Therefore, the present invention can be suitably operated because reproduction in a form in which only viewing is restricted is possible.

  In addition, when applied to a large screen display installed in a stadium, position information e and personal information d may be set in advance for each spectator seat area provided for each country. By doing so, for example, some of the images of the audience that are occasionally projected on the large screen display are highly likely to cause emotional distress, such as clothes that are unacceptable in the culture of the home country. It is possible to restrict playback to a specific spectator seat area without forcing the intention of viewing restriction based on the creed. As described above, the present invention may have a dramatic effect of promoting internationalization of the video viewing style by carefully selecting an application.

  The reason why the visual acuity information of the observer is input is that the maximum range in which the depth can be expressed varies depending on the visual acuity of the left and right, presbyopia, amblyopia and other visual abilities. The present invention makes it possible for people with different visual abilities to enjoy a stereoscopic display at the same time, and also has the potential of promoting the barrier-free 3D video viewing style.

  In order to input the personal information d and the position information e, an observer information input device 12 shown in FIG. 1 is required. As an embodiment of the observer information input device 12, a remote controller (hereinafter abbreviated as a remote controller) is suitable in a home living room. The remote control for TV that is widely used in general households transmits near-infrared light from the light-emitting diode built into the remote control to the light-receiving part of the display, so the approximate position information e of the person who operates it is specified. Can be used to do.

  Further, it is possible to use a general key attached to the remote controller so that personal information d is input on the personal information setting menu screen illustrated in FIG. 9 displayed on the display (image display means 16). Easy. In addition, a portable information terminal such as a mobile phone is suitable as the observer information input device 12 in an application used outdoors such as the large screen display described above.

  However, in an application where the relative positional relationship between the display and the observer is always constant, such as in a movie theater or a stadium seat, it is preset for each seat without having to bother input from the portable information terminal ( It may be preset). In such a case, a general information input device such as a keyboard or a mouse is an embodiment of the observer information input device 12. Of course, in the display in the home living room, if the position where the family sits is roughly determined, the personal information d and the position information e for each viewing position are preset with the remote control, so that the viewing starts. The inconvenience of setting each time before can be avoided.

  Next, the display mode determiner 13 shown in FIG. 1 will be described. The display mode determiner 13 receives the viewing restriction information c output from the information separator 11 and the personal information d and position information e output from the observer information input device 12, and is suitable for each observer. The display mode, that is, the restriction level when the above-described digital stereoscopic video content a is reproduced is determined. As described in Japanese Patent No. 2853727 previously disclosed by the applicant of the present invention, the basic concept of the display mode determination method is a playback device protection level that is a restriction level set on the playback device side, and a medium. The reproduction protection level (display mode) that is the restriction level for actual reproduction is determined by the combination with the medium protection level that is the setting restriction level given to the player.

  In the present embodiment, the concept of the above reproduction protection method is developed to enable reproduction with different protection levels for each observer who observes the same display at the same time. Details of the display mode will be described later.

  In FIG. 1, when the reproduction protection level is individually determined by the display mode determiner 13 based on the individual information d of each of a plurality of observers having different position information e, the display mode control signal generator 14 Corresponding to each reproduction protection level, a display mode control signal f which is a digital signal for controlling the reproduction level of the video information b is generated. This display mode control signal f is input to the decoder (decoder) 15 together with the video information b separated by the information separator 11, and the video information b has a display mode having a different reproduction permission level for each observer. Decrypted.

  When the video information b is compression-encoded by MPEG1, the decoding method is performed by decompression encoding (inverse Fourier transform, inverse quantization, variable length decoding) according to the determined reproduction protection level. . By such decoding, a mosaic-like coarse image can be generated step by step. In addition, as disclosed in Japanese Patent Application Laid-Open No. 2000-41232 by the present applicant, when compression encoding of the video information b is performed by MPEG4, an object having an arbitrary shape that should be restricted for viewing in a scene. Because it is possible to control such as blurring, mosaicing or not displaying the corresponding object only, it is possible to perform limited playback without feeling uncomfortable to the observer compared to mosaic display Is possible.

  The signal thus decoded is amplified by a drive circuit (not shown), and then supplied to the image display means 16 to be displayed individually. Examples of the image display means 16 include a direct-view display such as a liquid crystal display (LCD), a plasma display panel (PDP), a field emission display (FED), and an organic EL display, and a DLP (micro-mirror) manufactured by Texas Instruments, USA. A projection display such as a registered trademark projector or an LCOS (Liquid Crystal on Silicon) projector in which a reflective liquid crystal is formed on silicon can be used. In particular, as a specific example of the LCOS projector, a D-ILA (registered trademark) projector manufactured and sold by the present applicant can be used.

  In general, the display mode of the image display means 16 is a method of dividing the display area of the display into areas corresponding to the number of viewers (space division display), and temporally without performing space division. There is a method of dividing and displaying (also referred to as time division display, time division display, or field sequential display). In the former, the resolution decreases in inverse proportion to the number of viewpoints, but an image without flickering of the screen can be obtained. On the other hand, the latter does not decrease the resolution because the entire display is displayed at each moment, but the refresh rate per viewpoint decreases in inverse proportion to the number of viewpoints, so a sufficiently high drawing speed cannot be secured. There is a problem that flicker occurs as far as possible.

  At the time of this application, the above-described space division display and time division display have a problem that they are insufficient in performance to be applied to a large screen display due to the difficulty in manufacturing high-resolution elements and high-speed drive circuits. . By the way, this is the biggest problem that has hindered the practical application of natural 3D displays (also called 3D displays and 3D displays) that have been researched and developed over many years. It is a well-known fact between.

  In general, a device at present cannot display a stereoscopic video image with a resolution or a refresh rate sufficient to establish a product unless it is a binocular display that displays only two types of viewpoint images. Therefore, also in this embodiment, if only one display with the current resolution is used to display an image with an appropriate display mode for each of a plurality of viewers, the same space as in the case of stereoscopic image display is used. It is easily envisioned that the problem of reduced resolution or reduced temporal resolution will occur.

  However, this problem can be overcome as described below. The reason is that in the present invention, different images are not presented to different observers, and most of the images are the same. In other words, the only difference is the object of the scene whose viewing is restricted, and for other scenes, there is no reduction in the spatial or temporal resolution accompanying the increase in the viewpoint.

  For example, the block of display mode 1, display mode 2, display mode 3,..., Display mode n described in the image display means 16 in FIG. 1 must divide the display spatially or temporally into n. It doesn't mean you have to. Rather, in a normal scene, each viewer can view the highest resolution video from all the pixels of the display.

  Of course, in a view-restricted scene, it is impossible in principle to avoid a reduction in resolution because different images must be displayed for each observer. However, such a scene is highly likely to remain in a spatially or temporally small area in the above-described application assumed by the present invention, so that a reduction in resolution in a viewing-restricted scene is actually a problem. It is thought that it does not become.

  In this way, the present invention makes it possible to observe completely different images from a plurality of viewpoints such as car navigation applications by sharing images other than the spatial and temporal areas that are protected for reproduction to the observers of each viewpoint. It overcomes the problem of resolution reduction that cannot be avoided in the application, and has a feature that the display can be comfortably viewed at the original resolution.

  Now, the light from the image displayed as described above by the image display unit 16 of FIG. 1 is incident by the local stereoscopic image information output unit 17 so as to enter the pupil of each eyeball of a plurality of observers. The traveling direction is controlled. Specific examples of the local stereoscopic image information output means 17 include a refractive optical element (a lenticular lens and a microlens array), a diffractive optical element (a Fresnel lens array and a holographic optical element), or There are aperture elements (slit arrays and pinhole arrays).

  The latter can be configured with an electronic device such as a liquid crystal panel, so that the opening is formed only in the scene where viewing is restricted so that it can be seen only through the opening, and the entire panel is transmitted in other scenes. By doing so, it is possible to display a scene without the resolution deterioration described above. Of course, research on electronic devices such as liquid crystal lenses has also been advanced with respect to refractive optical elements, and it is considered that they can be suitably used in the practice of the present invention in the future.

  Further, the local stereoscopic images g1, g2,..., Gn outputted from the local stereoscopic image information output means 17 shown in FIG. 1 are observed at positions as shown in FIG. That is, the light bundles from the viewpoint images R1 and L1 displayed on the image display means 16 shown in FIG. 29 are deflected in the direction by the local stereoscopic video information output means 17 so that both eyes of the observer at the viewpoint 1 are obtained. Incident EL1 and ER1 shown in FIG. Then, both eyes perform a converging motion that rotates inward so that the viewpoint images L1 and R1 on the retina are fused as one image in the brain. As a result, a stereoscopic video is observed as the local stereoscopic video 210 at a point (convergence point) where the lines of sight in a crossed eye state intersect.

  In such a state, the eyes of both eyes are in alignment with R1 and L1 on the image display means 16. This is because the focus adjustment of the eye performs autofocusing so that the image can be seen more clearly, but the light bundles emitted from R1 and L1 gradually spread, so that the position of the local stereoscopic image 210 is larger than that of the local stereoscopic image 210. This is because a clear image can be observed with a smaller spot at the position of R1 or L1 on the image display means 16. In this way, a mismatch between the convergence point and the adjustment point described above occurs. When observing an object in nature, the convergence point and the adjustment point coincide with each other, except when looking at something very close.

  However, in the stereogram, there is a discrepancy between convergence and adjustment, which causes eye strain and sickness. Therefore, the contradiction amount between the convergence and the adjustment, that is, the distance C1 between the convergence point (the position of the local stereoscopic image 210) and the adjustment point (R1 or L1 on the image display means 16) should not be excessively large. It controls the depth expression amount of the video.

  On the other hand, the observer located at the viewpoint 2 in FIG. 29 observes the viewpoint images R2 and L2 displayed on the image display unit 16 with both eyes, and observes the local stereoscopic video 211. The congestion adjustment contradiction amount for the observer of viewpoint 2 is C2, which is smaller than the congestion adjustment contradiction amount C1 of the observer of viewpoint 1 described above. Therefore, the observer at the viewpoint 2 has a lower degree of fatigue than the observer at the viewpoint 1.

  This is an excellent display mode when, for example, there is an adult observer at viewpoint 1 and a child observer at viewpoint 2. An adult can enjoy a powerful local stereoscopic image 210 having a convergence adjustment contradiction amount C1, and a child can also enjoy a gentle local stereoscopic image 211 having a convergence adjustment contradiction amount C2. The present invention generalizes such a specific embodiment to enable simultaneous viewing by a plurality of observers having different ages and visual states, and promotes a barrier-free stereoscopic image viewing style.

  FIG. 2 is a conceptual diagram showing the relationship between personal information d, position information e, and viewing restriction information c, which are three signals input to the display mode determiner 13 of FIG. As shown in FIG. 2, these three pieces of information c to e are represented by uvw by a variable u containing personal information d, a variable v containing viewing restriction information c, and a variable w containing position information e. Configure the space. And the point in such a three-dimensional space represents the display mode which the display mode determiner 13 determines. For example, the point indicated by (u1, v1, w1) in the figure means that it is determined by the values of the variable u1 of the personal information of a certain observer, the variable v1 of the viewing restriction information, and the variable w1 of the position information. To do.

  Here, when compared with the reproduction protection method described in Japanese Patent No. 2853727 which is the patented invention of the present applicant, the “reproduction machine protection level” described in the above patent publication is the personal information of the present embodiment. Corresponding to the variable u, the “medium protection level” described in the above patent gazette corresponds to the variable v of the viewing restriction information of the present embodiment. Therefore, it may be sufficient to simply determine the number of layers of the position information for the observer having each position information by determining the two-dimensional uv plane layer composed of the variable u of the personal information and the variable v of the viewing restriction information. In such a case, there is no need to bring up the concept of a three-dimensional uvw space.

  However, as will be described later, depending on the application, the variable v for viewing restriction information and the variable w for position information are closely related, or the variable u for personal information and the variable w for position information are closely related. There is a case to do. Therefore, in this embodiment, the concept of a three-dimensional uvw space that can be used for display mode determination is introduced with the variable u for personal information, the variable v for viewing restriction information, and the variable w for position information as three independent variables. .

  3 to 7 are tables showing an example of how to determine the display mode from the personal information variable u and the positional information variable w in the three-dimensional uvw space described above. As shown in the table, depending on the combination of personal information (described as 1 to 5 in the first row of the table) and position information (described as 1 to 5 in the first column of the table), display modes from A to E, that is, The visual grade illustrated in FIG. 13 is determined.

  FIG. 10 shows an example of the variable u of personal information in the tables of FIGS. As shown in FIG. 10, personal information is classified into five levels from 1 to 5 for each of age, sex, nationality, religion, and barrier-free. The observer selects a necessary one from these and inputs it to the observer information input device 12 by the method such as the remote controller described above. For example, when adults and children watch the same movie at home, age information may be selected. By doing so, violent scenes that have mental development problems for children, or content that flashes light at a high speed with photostimulation that is concerned about biological effects such as camera flash Is displayed in a regulated manner, and unregulated content is played back for adults.

  In addition, in a situation where men and women watch the same content at the same time, gender information may be selected when it is desired to restrict display of information that causes discomfort for men or women in general. In addition, when people with different nationalities view images from a single display at international conferences or international stadiums, content can be regulated based on differences in the cultures of each country. The nationality information may be selected as follows. For example, as shown in FIG. 10, it is possible to reproduce content in an appropriate display mode according to various cultural differences such as the United States, Europe, Asia, Middle East, China, and Oceania, Africa, South America (not shown). It becomes.

  Furthermore, although it is related to ethnic and religious issues, even if the nationality is the same, if the thoughts and beliefs are different, the religious information that can regulate the content that touches the faith is entered as personal information. do it. For example, as shown in FIG. 10, if an observer inputs various religion information around the world, such as Christianity, Islam, Hinduism, Buddhism, Confucianism, and not shown, it is violent in stereoscopic video content. It is possible to display the scene and body exposure scene reproduction method in a stepwise manner according to the individual religion.

  Moreover, you may comprise so that barrier-free information may be input in order to make the relationship between a non-disabled person and a disabled person between observers barrier-free. For example, for visually handicapped persons, the brightness and contrast of the video can be optimized and displayed, and for the hearing handicapped persons, sound volume and frequency characteristics can be optimized and reproduced. . Such a barrier-free audiovisual technique is a publicly known technique that is limited to personal viewing, but the present embodiment is characterized in that it is applied to a scene where a plurality of viewers view simultaneously.

  In this way, the personal information d removes the barrier between ages, the barrier between genders, and the barrier between disabled and non-disabled people from the conventional video viewing style where individual videos are viewed individually. It can be said to bring about barrier-free and internationalization that removes the barrier between nations and between different religions.

  FIG. 11 shows an example of position information of the tables of FIGS. As shown in FIG. 11, as the relative position information between the display and the viewer, the angle formed by the line-of-sight vector that is a vector from the viewpoint toward the display (center) and the normal of the display surface is ± 20 degrees or more, Levels are divided into 1 to 5 levels of ± 20 degrees to 10 degrees, ± 10 degrees to 5 degrees, ± 5 degrees to 0 degrees, and 0 degrees or more. In general, since the probability of viewing from the front of the display is the highest, the 0 degree position which is the front of the display is set to “5”, the highest level of regulation.

  Further, as shown in FIG. 11, the positional information includes not only the angle from the display but also the distance from the display, that is, the size of the line-of-sight vector as an important parameter. As shown in the figure, H is the height of the display, and 1.5H or more, 1.2 to 1.5H, 0.8H to 1.2H, 0.4H to 0.8H, 0.4H or less 1 There are 5 levels. Since the visual line vector size 0.4H closest to the display has a great influence on the visual perception, the restriction level is set to “5”.

  FIG. 12 shows an example of the viewing restriction information in the tables of FIGS. As shown in FIG. 12, the viewing restriction information c includes violent scenes (brutality, murder, severe disability, mild disability, all 5 levels of restriction), and exposed scenes (excessive exposure, overall exposure, partial Exposure, exposure clothes, all 5 levels of regulation), religious scenes (extreme expression, gentle expression, all 3 levels of restriction), visually stimulating scenes (high frequency light stimulation, intermediate frequency light) Stimulation, low-frequency light suppression, all four levels of restriction), stereo stereoscopic images (above fusion limit, large popping, medium popping, small popping, all limiting five steps) .

  If necessary, a scene including a high contrast pattern may be added as a visually stimulating scene, or a scene including cultural communication between nations may be added. This is set by the supply side of the stereoscopic video content. Such a technique has already been introduced as a restriction option for general Web browsers, and it is considered that no technically difficult problem occurs.

  FIG. 13 shows one embodiment of such a specific display mode. As shown in the figure, the display mode A is the mode in which the degree of display restriction is the least, and all stereoscopic video content is displayed. The display modes B to E are gradually regulated, and the display mode E has the highest degree of display regulation. Regarding the mode of regulation, there are regulation in the spatial direction, regulation in the time direction, and regulation by the number of data display bits. Since these detailed descriptions are described in Japanese Patent No. 2853727, they are omitted.

  In addition, when the video is a stereo stereoscopic video, the depth reproduction amount of the stereoscopic video is limited according to the display mode in order to alleviate the discrepancy between the vergence movement of the eyes and the adjustment movement (contrast adjustment contradiction). It should be configured. For example, as shown in the figure, in the display mode A, a full-size stereoscopic image without depth compression is reproduced as it is, and in the display modes B to E, the depth amount is reproduced so as to be compressed. In the most restrictive display mode E, all viewpoint interval information is discarded and a two-dimensional image is displayed.

  By the way, as is well known in Patent Document 1 and other documents related to stereoscopic images, the effect of eye fatigue due to contradiction of convergence adjustment is particularly important for children under the age of about 10 years before the lens adjustment function is completed. There are concerns that it will have a very serious impact on the development of the eyes. Such a health problem is that, if the resolution of the display device is improved in the future, a large number of viewpoint images can be displayed. It is thought that it will gradually become a solution with the advancement of device technology because it will cause regulatory stimulation.

  Therefore, until the device advances, it is imperative to take stereo stereoscopic vision at the current resolution or quasi-super multi-view stereoscopic vision with an insufficient number of rays. Then, if you can't show it to your kids, you can't spread it to your home. And if it continues as it is, people's expectation for stereoscopic images will decline, and investment in next-generation stereoscopic image displays will be restrained, which can be said to be a national loss for Japan as a 3D advanced country.

  The present invention not only solves the problems related to parental lock when viewing multiple persons as described above, but also solves the above problems related to stereoscopic image display at a stretch. That is, by performing depth compression for each observer as shown in FIG. 13, almost no depth compression is performed for an adult observer defined by personal information (especially age information 1 to 3) and position information. Display without calling. Then, for the child observer, the viewpoint interval information is compressed so that the congestion adjustment contradiction amount becomes sufficiently small, and an image with almost no pop-out amount is displayed. In this way, adults and children can simultaneously enjoy content in the living room.

  Next, processing performed by the display mode determiner 13 will be described with reference to the flowchart of FIG. First, a signal input waiting state from the information separator 11 is entered (step S100), and the viewing restriction information c included in the multiplexed digital stereoscopic video content a is input from the information separator 11 (step S101). ), And waits for input of a signal from the observer information input device 12 (step S102). This state is continued until personal information d and position information e of all the observers are input in the conditional branch of step S103 (No in step S103).

  The inputted personal information d and position information e are stored in a storage device (RAM) (not shown) included in the display mode determiner 13. When the personal information d and the position information e of all the observers (N persons) are input (Yes in step S103), the variable n is stored in a storage device (RAM) (not shown) included in the display mode determiner 13. Is secured, and the value of the variable n is initialized to 1 (step S104). When the initialization of the variable n is completed, first, the process of determining the display mode of the n = 1st observer is performed (step S105). The display mode determination process is performed by a three-dimensional matrix operation of three variables of position information e, personal information d, and viewing restriction information c. The details of the processing are obtained by extending the two-dimensional matrix operation described in Japanese Patent No. 2853727 to three dimensions, and therefore need not be described separately.

  When the display mode of the nth observer is determined, the value of the variable n is incremented by 1 (step S106). Subsequently, it is determined whether or not the display mode determination process for all the observers has been completed (step S107). If not completed, the process returns to step S105, and the display mode determination process for the next observer is performed. Finally, when the processing of all the observers is completed (Yes in step S107), the display mode determination processing is terminated.

  Thus, for example, for a child aged 4 to 12, the personal information is “4” from FIG. 10, and the viewing restriction information is “5” indicating “all restricted” from FIG. As shown in FIG. 6, irrespective of the position information, the visible grade (display mode) is E, that is, the most restrictive stereoscopic image display is performed as shown in FIG. Further, even if the child is 4 to 12 years old, if the viewing restriction information is set to “4” in FIG. 12 depending on the content, the visible grade is set regardless of the position information as shown in FIG. D, as shown in FIG. 13, stereoscopic image display with relatively strict regulation is performed.

  Next, a second embodiment of the present invention will be described with reference to FIGS. FIG. 14 shows a specific example of the image display means 16 and the local stereoscopic video information output means 17 described in FIG. In the figure, a flat display 161 is an example of the image display means described above, and a refractive optical element such as a lenticular sheet (or microlens array) 171 as an example of the local stereoscopic image information output means 17 is provided on the surface thereof. Has been placed.

  With this configuration, one image on the surface of the flat display 161 is displayed in the first viewing direction, and six images on the surface of the flat display 161 are displayed in the sixth viewing direction. Similarly, in this example, the image on the surface of the flat display 161 is divided into eight and displayed in eight viewing directions. In order to perform this display, a flat display driving circuit (not shown) is required.

  FIG. 15 shows a block diagram of a second embodiment of a multi-viewpoint stereoscopic video display apparatus according to the present invention. In the figure, the same components as those in FIGS. 1 and 14 are denoted by the same reference numerals. In FIG. 15, digital stereoscopic video content information (including audio information) recorded on a recording medium 21 such as an optical disc is reproduced by a reproducing device 22 by a known method, and then input to the information separator 11. Are separated into video information, audio information and viewing restriction information. Multiplexing of the information at the time of recording on the recording medium 21 and separation by the information demultiplexer 11 are realized by using, for example, a multiplexing system system of the MPEG2 standard.

  Multiplexing is performed by packing each elementary element using the program stream method in the MPEG system layer synchronization method, and by multiplexing the presentation time stamp so as to be synchronized at the time of display. As the time stamp, counter information of 27 MHz or 90 kHz is used from a time stamp generator (not shown) and is input to an information multiplexer (not shown). Since this mechanism is possible using the MPEG multiplexing standard, a detailed description is omitted.

  The multiplexed stream data is converted into a package media format such as DVD or D-VHS, and is recorded on the recording medium 21 by a predetermined recorder in a format compliant with the DVD or D-VHS standard described later. ing. The video information separated by the information separator 11 is supplied to the video decoder 23, the separated audio information is supplied to the audio decoder 24, and the separated viewing restriction information is supplied to the display mode determiner 13. .

  The video decoder 23 and the audio decoder 24 decode the video signal and the audio signal and supply them to the video CH (channel) separator 25 and the audio CH separator 26, respectively. The video CH separator 25 and the audio CH separator 26 identify the identified channel ID from the input decoded video signal or decoded audio signal using an MPEG identifier, for example, Stream_id. Each video data is identified.

  In the MPEG2 standard, Stream_id is an 8-bit information body that can describe element compressed data for each channel called a PES (packetized elementary stream) packet in packetized header information. As a result, video and audio can be identified and CH can be identified. The video data separated and output for each CH from the video CH separator 25 is input to the flat display 161, and as described with reference to FIG. 14, the video data is stereoscopically displayed in eight viewing directions through the lenticular sheet (or microlens array) 171. Displayed as an image.

  On the other hand, the audio data separated and output for each CH in the acoustic CH separator 26 is transmitted to the localized sound field generator 27. In the localized sound field generator 27, with respect to audio data to be localized in the n-th line-of-sight direction, from the center of the speaker array 28 to the focal point so as to locally increase the sound pressure near the focal point in space. A display signal provided with a delay amount corresponding to the difference between the path and the path from each speaker to the focal point is transmitted to the speaker array 28. The speaker array 28 is assembled in an array, and each delay circuit is set with a delay value so as to focus on the vicinity of the listening position, and the first line of sight rather than the direct sound from the speaker at the listening position. The sound pressure component generated by the audio display sound localized in the direction is displayed so as to be extremely high.

  For audio data localized in the second line-of-sight direction, a path from the center of the speaker array 28 to the focal point and a path from each speaker to the focal point so as to locally increase the sound pressure near the focal point in space. Similarly, a display signal provided with a delay amount corresponding to the difference is transmitted to the speaker array 28. The sound pressure component generated by the audio display sound localized in the (n + 1) -th line-of-sight direction is displayed so as to be extremely higher than the direct sound from the speaker at the listening position. When transmitting two sound signals in order to generate n localized sound fields in the speaker, each speaker adds n sound signals and performs gain control so as not to exceed the dynamic range. Output.

  Next, a second embodiment of the multi-viewpoint stereoscopic image display method according to the present invention will be described with reference to the flowchart of FIG. This multi-viewpoint stereoscopic video display method is realized by a computer by a computer program. First, MPEG-multiplexed audiovisual data and viewing restriction information are read from a recording medium or a transmission path in a predetermined unit (step S201). Next, the reproduced audio / video data multiplexed with MPEG and the viewing restriction information are separated (step S202). Subsequently, it is determined whether or not the viewer information is input from the viewer (step S203), and the reproduction mode is determined only when the viewer information is input (step S204).

  When there is no input of observer information or when the reproduction mode is determined in step S204, the audiovisual data is decoded (step S205). Subsequently, the decoded video data and audio data are separated into CHs for each line of sight (step S206). Subsequently, video data for each line of sight is output from image display means and local stereoscopic video information output means (such as a lenticular sheet or a microlens array), and acoustic data is generated as a local sound field generating speaker (or superdirective speaker). , Headphones, earphones) (step S207).

  Then, it is determined whether or not all video data and audio data are present (step S208). When there is video data and audio data that should still be reproduced, the process returns to step S201, and the same operation as described above is repeated. And when the reproduction of the sound data is finished, this program is finished.

  In the above embodiment, the information has been described as being reproduced from the recording medium 21. However, the packet information that is a packet specific to communication or broadcasting is received, and the packet release that is a communication specific format is performed. It is also easy to receive and display from broadcasting and communication networks.

  In this embodiment, the lenticular lens method has been described as a method for displaying different images depending on the line-of-sight direction. However, parallax such as a liquid crystal parallax barrier method, a polarization filter method, an integral photography method, and a super multi-view method is used. Any type of 3D video display technique for sampling can be applied, and any system that can perceive different videos may be used.

  Also, the sound image position control method has been described as an array speaker method as a sound image localization control method, but even a method that can realize a virtual sound field space, such as a binaural transoral method, is represented by the Kirchhoff-Helmhotz fractional equation. A method using a sound field control method using wave acoustic theory may be used.

  In addition, data can be transmitted via any transmission medium such as communication and broadcasting without recording data on the recording medium. In that case, the recording apparatus can also be used as a transmission apparatus. The display device can also be used as a receiving device.

  In addition, the definition of the medium includes not only a narrowly-defined medium that can record data, but also an electromagnetic wave and light for transmitting signal data. In addition, the information recorded on the recording medium includes data itself such as an electronic file in a state where the information is not recorded.

  In the above embodiment, the position information, personal information, and viewing restriction information of a plurality of observers are acquired, and the depth expression is compressed and expanded to determine the individual optimum depth expression level. The method and apparatus for individually displaying a local binocular stereoscopic image have been described in the above. Next, the optimal optimal monocular parallax number is determined by compressing / decompressing a multipoint image interval. A method and apparatus for individually displaying IP stereoscopic images in the playback mode will be described. The basic configuration of the multi-view stereoscopic video display device in this case is the same as that shown in FIG.

  First, an embodiment of an optical system for controlling the super multi-view state in the present invention will be described in detail. FIG. 17 is a diagram showing an example of the geometric optical relationship of the optical system in the present invention. The light from the multi-viewpoint image of the interval Pe displayed on the two-dimensional image display surface 200 expressed by a general two-dimensional display such as an LCD (liquid crystal display element) or a projector screen is separated from the display surface 200 by g. The light is deflected by a deflecting optical element 300 expressed by a microlens array or a pinhole array disposed at a different position. Although the interval between the viewpoint images is different from the interval between the microlens arrays, for the sake of simplicity, both are assumed to be the same.

  Light rays from each of these deflecting optical elements 300 intersect in space to form a three-dimensional image 500. FIG. 17 shows only one point constituting the video object, and shows a case where an image is formed in front of the display (real image). As will be described later, there is a case where an image is formed behind the display (virtual image). The light that intersects the real image 500 in the space then travels straight and enters the eyeball 400 of the observer with the binocular distance De and the pupil diameter Dp. As shown in FIG. 17, it is assumed that the eyeball of the observer is separated from the deflection optical element 300 by a distance L and separated from the real image 500 by 1 / D.

  FIG. 17 shows a state in which two light beams having an angle θ emitted from two lenses are simultaneously incident on the pupil. Since the two light beams intersect at a point of the stereoscopic image 500, the observer observes the light as if it is emitted from the stereoscopic image 500, and the eye movement is induced in the stereoscopic image 500. The Thus, there is no contradiction between convergence and adjustment, and a natural stereoscopic image 500 is observed. In such a super multi-view state, it can be easily understood that the following expression is established from the geometric relationship.

ただし、偏向光学素子３００と立体像５００との距離をａとしてａ＝Ｌ−１／Ｄとおいた。同様に、次式が成立することも容易にわかる。

However, the distance between the deflecting optical element 300 and the three-dimensional image 500 is set as a = L−1 / D. Similarly, it can be easily understood that the following equation holds.

超多眼条件、すなわち瞳孔に２.５本以上の視差画像を入射するためには、
φ≧２．５θ （３）
であることが必要であるから、（１）式〜（３）式から次式が得られる。

In order to enter more than 2.5 parallax images into the pupil,
φ ≧ 2.5θ (3)
Therefore, the following equation is obtained from the equations (1) to (3).

つまり、実像の飛び出し量ａが大きい（即ち（４）式の分子が大きい）ほど、又は観察者と実像の距離が近い（即ち（４）式の分母が小さい）ほど、ピッチＰｅに対する要求は緩くなる。即ちＰｅが多少大きくても超多眼条件を満たすことになる。

In other words, the requirement for the pitch Pe becomes less as the real image pop-out amount a is larger (that is, the numerator of the equation (4) is larger) or the distance between the observer and the real image is closer (that is, the denominator of the equation (4) is smaller). Become. That is, even if Pe is somewhat large, the super multi-view condition is satisfied.

  As a specific example, when Dp = 5.5 mm, De = 65 mm, a = 100 mm, g = 10.0 mm, and L = 1500 mm, Pe <0.17 mm is required. For example, when the pitch Pe = 1 mm, the super multi-view condition is satisfied if the observation is performed at a viewing distance L <150 mm. However, if the observation is performed at the viewing distance L = 1500 mm, normal multi-view observation is performed. This will cause stress.

  That is, depending on the position L of the observer's eye 400 with respect to the display, the position a of the stereoscopic image 500, or the like, a super multi-view state in which a plurality of light beams enter the pupil or a multi-view state in which only one light beam enters. You can see that Therefore, when a plurality of observers observe, it is necessary to control the position of the stereoscopic image 500 according to the viewing distance of each observer. In particular, when the observer is a child, it is necessary to avoid a multi-view state that causes a contradiction of convergence adjustment and to enable observation in a super multi-view region as much as possible. The present embodiment is characterized in that a super multi-view state without eye stress is individually set by controlling the stereoscopic image position 500 for each of viewers at different viewing positions. .

  FIG. 18 is a diagram illustrating a state in which a plurality of observers observe different stereoscopic images. Light from the multi-viewpoint image displayed on the image display means 205 (corresponding to the image display means 16 in FIG. 1) such as an LCD is output as local stereoscopic video information output means 206 (local stereoscopic video information output means 17 in FIG. 1). Is incident on the observer at viewpoint 1 or viewpoint 2. A local stereoscopic image 212 or 213 shown in FIG. 18 is an aggregate of the stereoscopic image 500 in FIG. 17, and is configured by a tertiary light source generated by a large number of light beams intersecting in space.

  For example, the viewpoint images R11, R12, L11, and L12 displayed on the image display unit 205 are incident on the left eye (L1) 401 of the observer at the viewpoint 1, but the light ray R11 that is incident on the right eye (R1) 402 of the observer. R12 intersects in space to form one point of the local stereoscopic image 212, and the position of the light source behaves as if it is one point of the local stereoscopic image 212 (not R11, R12). . Similarly, a secondary light source is formed by intersecting the light rays L11 and L12 incident on the left eye (L1) of the observer in space.

  Since the observer at the viewpoint 2 is farther from the display than the observer at the viewpoint 1, it is difficult to observe the super multi-view area. Therefore, it is necessary to control the intervals of the viewpoint images R21, R22, L21, and L22 so as to be narrower than the intervals of R11, R12, L11, and L12 with respect to the observer of the viewpoint 1. The details of the specific method for controlling the interval between the viewpoint images and the method for arranging the viewpoint images will not be described here, but only the basic concept necessary for carrying out the present invention will be described below.

  FIG. 19 is a diagram showing how the real image 500 is observed with the monocular 400. Light from a multi-viewpoint image displayed on the image output unit 205 such as an LCD is converged in space by a light deflecting unit such as a microlens array to form a real image 501. FIG. 19 illustrates a state in which 10 components are emitted from 10 lenses and intersect.

  These light rays converge and then diverge, a part of which is blocked by the iris 411 of the eyeball 405, and about six light rays are refracted by the crystalline lens 410 to form an image on the retina 412. At this time, the lens 410 performs adjustment by changing the lens thickness autonomously, that is, autofocus so that the image on the retina can be seen most clearly. At first glance, the focus of the eye seems to fit on the lens array 206 where the diameter of each light beam is the smallest, but in reality, the entire six that are incident on the pupil are most clearly visible. Since the light beam is at the position 501 where the light beams intersect, the light beam is focused on the position 501. Although this effect depends on the number of rays, it is the content disclosed in Non-Patent Document 3 that at least 2.5 or more are necessary as described above.

  FIG. 20 is an enlarged view of FIG. 19. Light from an image displayed on the image display means 205 such as an LCD is deflected in the traveling direction of light by a local stereoscopic image information output means 206 such as a microlens array. It is a figure which shows a mode. FIG. 20 shows a state in which light from ten pixels arranged at equal intervals is deflected by ten lenses.

  In general, the light from the pixels on the image display means 205 spreads in all directions, but here consider light rays that use individual lenses as exit pupils. In that case, as shown in FIG. 20, it spreads in a fan shape and is refracted by the lens. At this time, if the image display means 205 is located at the focal point of the lens 206, the light beam emitted from the lens becomes almost parallel light, but the off-axis light beam does not become parallel light but generates aberration. At present, it seems that a technique for reducing this aberration is not disclosed, and will be ignored for the time being.

  Now, it can be seen that when the light emitting point is deviated from the optical axis of the lens of the microlens array, the deflection angle, that is, the refraction angle by the lens changes according to the amount of deviation. For example, two of the ten lenses (206e and 206f) near the center are emitted with a small refraction angle because the deviation between the lighted pixels 205e and 206e and the optical axis is small. However, for the peripheral lenses 206a and 206j, the deviation between the lighted pixels 205a and 205j and the optical axis increases, and the refraction angle by the lens increases.

  FIG. 21 is an enlarged view of the four lenses 206d to 206g among the ten lenses in FIG. As shown in FIG. 21, the interval between the pixels 205d to 205g to be lit is wider than the interval between the lenses 206d to 206g. FIG. 21 shows that the pixels to be lit are arranged at intervals of 11 pixels. By controlling this distance, it is possible to control the refraction angle of the lens described in FIG. 20, and by controlling the overlapping of the light beams described in FIG. 19, both the super multi-view state and the multi-view state are controlled. can do. That is, it is specifically possible to display in a different manner for each of a plurality of viewers of the present invention.

  FIG. 22 is a diagram showing the behavior of light rays within one lens constituting the local stereoscopic image output means 206 shown in FIG. 18 and the like, and there are 10 pixels in one lens, all of which This is a drawing of a light beam when is turned on. These ten rays do not intersect in space after being refracted by the lens, but intersect with rays from other lenses to form ten different secondary light sources, respectively. It can be seen that light from 10 viewpoints is emitted from one lens.

  Note that a spherical or aspherical refractive lens 2061 can be formed on the substrate 2062. The molding material may be glass or resin. These lenses may be in close contact with the image display means 205 such as an LCD, but the protective layer 2063 and the glass substrate 2064 may be inserted as a spacer.

  FIG. 23 is a diagram illustrating a state in which a real image is formed by three lenses 207. As described above, the light beam intersects the space in front of the display to form a virtual secondary light source. In order to form a real image, as is apparent from FIG. 23, there is a relationship between the interval Wd between the viewpoint images displayed on the image display unit 205 and the interval Pe of the microlens array 207 which is the local stereoscopic video output unit 206. , Pe <Wd.

  On the other hand, when Pe = Wd, as shown in FIG. 24, the light beams are parallel to each other and never intersect. Therefore, in this case. A three-dimensional image is not formed. Further, when Pe> Wd, as shown in FIG. 25, the light beam emitted from the lens 207 diverges and the actual light beam does not intersect. However, for the observer in front of the display, the diverging light beam is observed as if it intersects at a position 502 behind the image display means 205 that extends in the opposite direction.

  That is, when Pe> Wd, the stereoscopic image is observed as a virtual image in the back of the display which is the image display unit 205. When a stereoscopic image is displayed at the back of the display, it is known that there is less stress on the eyes compared to the case where it is displayed in front of the display, as is known in the binocular and multi-view displays. Yes.

  Therefore, in this embodiment, in the super multi-view display, when it is difficult to create a super multi-view state in which light beams overlap depending on the viewpoint position of the viewer or the type of video content, Alternatively, it may be configured to shift to a virtual image. Since the virtual image is less affected by the beam spread than the real image, the degree of resolution degradation is small due to the depth position, and a safe display mode that is more suitable for observation by children with particularly poor visual function development. It is.

  FIG. 26 is a diagram showing actual light rays when a stereoscopic image is formed as a real image, and FIG. 27 is an enlarged view of FIG. 26 and 27 illustrate 23 rays that contribute to reproducing one point constituting the stereoscopic image 503. The 0 position on the vertical axis indicates the position of the microlens. The distance between these light rays in the image display means is about 105% with respect to the distance between the microlens arrays. In this case, one point forming the stereoscopic image 503 indicated by a broken line in FIG. 26 is formed at a distance of about 170 mm. And within about 600 mm from the display, it is in the super multi-view region where the light beams intersect, and when it is further away, the light cotton is separated into a multi-view region.

  By detecting the position of the observer, if it is in a multi-view area, the viewpoint image interval test is changed to be narrowed so as to be in a super multi-view state. At this time, an area where continuous motion parallax can be obtained (that is, viewing) is also narrowed, but since it can be observed by the adjacent viewing, there is no practical problem. The adjacent viewing occurs when the viewpoint image is emitted through the lens adjacent to the lens from which the viewpoint image is originally emitted, but the degradation of the image has practically no problem. However, since image skipping (flipping) occurs between viewers, when viewing changes by controlling the viewpoint image interval, it is possible to control the viewer so that the viewer does not just come to the viewing boundary. It is important.

  The above embodiment is an example in which stereoscopic video is displayed by a twin-lens system or IP. However, the present invention can further individually link each of the playback modes by linking stereoscopic audio to this stereoscopic video. A method and apparatus for displaying local stereophony can also be constructed.

  FIG. 28 shows a block diagram of a third embodiment of the multi-viewpoint stereoscopic image display apparatus of the present invention in this case. In the figure, the same components as those in FIG. In FIG. 28, digital video and audio content j, which is the content of video and audio information based on the encoded digital signal, is input to the information separator 11.

  The digital audiovisual content j is configured in a form in which audiovisual information k and viewing restriction information c that restricts reading of the audiovisual information k are multiplexed. For example, a movie as digital video and audio content j recorded on an optical disc such as a DVD (Digital Versatile Disc) is used as viewing restriction information c for selectively reproducing a part of the information together with all video and audio information k. Parental lock information related to age regulation and region code related to regional regulation are recorded.

  Of course, in the present embodiment, the digital audiovisual content j is not limited to content recorded on an information recording medium such as an optical disc, and information on a Web server is communicated to a storage medium of an information terminal. Needless to say, the present invention can be similarly applied to content downloaded via a line or content stored on a Web server and supplied by streaming distribution.

  The information separator 11 has a function of separating multiplexed information included in the digital audiovisual content j described above, specifically, a demultiplex well known in a digital signal processing circuit. It is. As shown in the figure, the information separator 11 separates the multiplexed digital video / audio content j into video / audio information k and viewing restriction information c, and the separated video / audio information k is sent to the decoder 18. The separated viewing restriction information c is input to the display mode determiner 13.

  Based on the display mode control signal f from the display mode control signal generator 14, the decoder 18 displays the video and audio information k in a display mode having a different reproduction permission level for each observer. Decrypt. The decoded video information is supplied to the image display means 16, and is further converted into 3D video information by the local 3D video information output means 17.

  On the other hand, the decoded acoustic information is phase-shifted by the phase shifter 19 in accordance with the delay time of the stereoscopic video information, and then supplied to the local acoustic information output means 20. Output as acoustic information h1 to hn for each viewpoint with respect to the observer. Thereby, in this Embodiment, the several observer from which each viewpoint differs can see the stereo image corresponding to each viewpoint, and can listen to the sound corresponding to each viewpoint.

  The present invention includes a computer program that causes a computer to execute the above embodiment. In this case, the computer program may be recorded on a recording medium and taken into a display device, or may be downloaded to a storage unit of the display device via a communication network.

  The present invention includes the following abstract technical ideas that enable the specific embodiments exemplified above. That is, the multi-view stereoscopic video display method of the present invention uses a single video information display unit to simultaneously display different local video information for each of the viewpoints of multiple observers. In the method, the display mode of the local video information suitable for each of the observers is obtained by the stereoscopic information of the local video information corresponding to each observer information of the observer and the viewpoint position of the observer. Individual local video information that is individually determined and optimized for each of the observers in the display mode is displayed.

  Here, the observer information includes personal information such as the height, age, and the like of the observer, and the stereoscopic video content includes viewing restriction information that restricts display of information inappropriate for the age of the observer. You may go out.

  Further, in the multi-view stereoscopic video display method of the present invention, the display mode of the local video information includes a partial display of a scene including the viewing restriction information, a still display of a scene immediately before the scene, or a painful preset. It includes at least one of display or non-display of a safe scene.

  In addition, the multi-view stereoscopic video and audio display method of the present invention uses a single video information display means and two or more audio information display means, and uses different local videos for each of the viewing points of a plurality of observers. In a multi-viewpoint video audio display method for simultaneously displaying information and corresponding local audio information, the stereoscopic information of each of the viewer information and the local video information corresponding to the viewer's viewing point position A display mode of the local video information and local acoustic information suitable for each of the observers is individually determined according to the video content and the acoustic content of the local acoustic information, and the viewer's The local video information and local audio information which are individual to each are displayed.

  In addition, in the multiple viewing point stereoscopic video and audio display method of the present invention, the display mode of the local video information and the local audio information is a partial display of a scene including the display protection information, and the still of the scene immediately before the scene. It includes at least one of display, display of a preset safe scene, or non-display.

  The stereoscopic video information display apparatus of the present invention is a multi-view stereoscopic video display apparatus that simultaneously displays different local video information for each of the viewpoints of a plurality of observers using a single video information display means. , Observer information detecting means for detecting the observer information of the observer together with the position of the viewpoint, stereoscopic video content reading means for reading the stereoscopic video content of the local video information corresponding to the observer, and the observer Video display mode determining means for individually determining a display mode of the local video information suitable for each of the observers by both information and the stereoscopic video content; and And local video information display means for displaying individual local video information for each.

  Here, in the multi-view stereoscopic video display device of the present invention, the observer information may include personal information such as the observer's height, age, or the like, and the stereoscopic video content corresponds to the age of the observer. Viewing restriction information for restricting display of inappropriate information may be included.

  Further, in the multi-view stereoscopic video display device of the present invention, the display mode of the local video information and / or the local acoustic information may be a still display of a scene immediately before the scene including the viewing restriction information, or preset. It includes at least one of display and non-display of a safe scene.

  In addition, the multiple viewing point video / audio display device of the present invention uses a single video information display unit and two or more acoustic information display units to provide different local video for each of the viewing points of a plurality of viewers. In a multi-viewpoint video / audio display device that simultaneously displays information and corresponding local acoustic information, observer information detection means for detecting each observer information of the observer together with the viewer's viewing point position; The content reading means for reading the stereoscopic video content of the local video information and the acoustic content of the local acoustic information corresponding to the observer, and the observer information, the stereoscopic video content, and the acoustic content, Display mode determining means for individually determining the display mode of the local video information and local acoustic information suitable for the display, and the observation in the display mode And having a local video-audio information display means for displaying the discrete local video information and local acoustic information for each.

  Also, the multi-view stereoscopic video display program of the present invention is a multi-view stereoscopic video display device that simultaneously displays different local video information for each of the viewpoints of a plurality of observers using one video information display means. In the multi-viewpoint stereoscopic image display program for controlling the observer, the computer reads the observer information detected together with the position of the viewpoint by the observer information detecting means for detecting the observer information of the observer; Reading the corresponding stereoscopic video content of the local video information, and comparing the observer information and the stereoscopic video content, thereby comparing the observer from the display mode database storing the display mode of the local video information A display mode suitable for each of the viewers, and each of the observers in the extracted display mode And executes a step of controlling to perform the display of the local video information against.

  In the multi-view stereoscopic video display program of the present invention, the observer information may include personal information such as the height, age, and the like of the observer, and the stereoscopic video content is inappropriate for the age of the observer. Display protection information for restricting display of various information may be included.

  The multi-viewpoint stereoscopic video display program according to the present invention may further include a still display of a scene immediately before the scene including the display protection information, display of a preset safe scene, or non-display of the display mode database. It includes at least one of them.

  Further, the multi-view stereoscopic video display program of the present invention uses a single video information display means and two or more acoustic information display means, and uses different local video information for each of the viewing points of a plurality of observers. And a multi-viewpoint video / audio display program for controlling a multi-viewpoint stereoscopic video / audio display device that displays local audio information corresponding thereto and the viewing point by the observer information detecting means for detecting the observer information of the observer Reading the viewer information detected together with the position of the viewer, reading the stereoscopic video content of the local video information corresponding to the viewer and the acoustic content of the local acoustic information, and the read viewer information A table of the local video information and the local acoustic information by comparing the stereoscopic video content and the acoustic content. A step of extracting a display mode suitable for each of the observers from a display mode database in which modes are stored, and display of the local video information and the local acoustic information for each of the viewers in the extracted display mode And controlling to perform.

  Further, in the multi-view stereoscopic video display method of the present invention, the local video information is local stereoscopic video information, and a depth range to be displayed in space is calculated from the three-dimensional information of the objects constituting the scene, It further includes a function of displaying the object in a depth range suitable for each of a plurality of different observers.

  The multi-view stereoscopic video and audio display method of the present invention may further include the three-dimensional image of the object constituting the scene, wherein the local video information and the local audio information are local stereoscopic video information and local stereophonic information, respectively. It has a function of calculating a depth range to be displayed in space from information and displaying the object in a depth range suitable for each of a plurality of different observers.

  In the multi-view stereoscopic video display device of the present invention, the local video information is local stereoscopic video information, and the depth range to be displayed in space is calculated from the three-dimensional information of the objects constituting the scene. And a means for displaying the object in a depth range suitable for each of a plurality of observers having different visual functions.

  Further, in the multi-view stereoscopic video display device according to the present invention, the local video information and the local audio information are local stereoscopic video information and local stereo audio information, respectively, and the three-dimensional information of the objects constituting the scene is used. The method further comprises means for calculating a depth range to be displayed in space and means for displaying the object in a depth range suitable for each of a plurality of observers having different visual functions.

  The multi-view stereoscopic video display method of the present invention further calculates a depth range to be displayed in space from the three-dimensional information of the objects constituting the scene, wherein the local video information is local stereoscopic video information. And a step of displaying the object in a depth range suitable for each of a plurality of different observers.

  Further, the multi-view stereoscopic video and audio display method of the present invention is characterized in that the local video information and the local audio information are local stereoscopic video information and local stereo audio information, respectively, and the three-dimensional information of the objects constituting the scene And a step of calculating a depth range to be displayed in space and a step of displaying the object in a depth range suitable for each of a plurality of different observers.

1 is a block diagram of a first embodiment of a multi-view stereoscopic video display device of the present invention. It is a conceptual diagram which shows the relationship between the personal information input into the display mode determination device of FIG. 1, position information, and viewing restriction information. FIG. 3 is a diagram (part 1) illustrating a table showing an example of how to determine a display mode from the variable u of personal information, the variable w of position information, and viewing restriction information v in FIG. 1; FIG. 8 is a diagram (No. 2) illustrating a table showing an example of how to determine a display mode from the variable u of personal information, the variable w of position information, and viewing restriction information v in FIG. 1; FIG. 9 is a diagram (No. 3) illustrating a table illustrating an example of how to determine a display mode from the variable u of personal information, the variable w of position information, and viewing restriction information v in FIG. 1; FIG. 10 is a diagram (No. 4) illustrating a table showing an example of how to determine a display mode from the variable u of personal information, the variable w of position information, and viewing restriction information v in FIG. 1; FIG. 10 is a diagram (No. 5) illustrating a table showing an example of how to determine a display mode from the variable u of personal information, the variable w of position information, and viewing restriction information v in FIG. 1; It is a flowchart for the process description which the display mode determination device in FIG. 1 performs. It is a figure which shows an example of a personal information setting menu screen. It is a figure which shows the specific example of the personal information of the table of FIGS. It is a figure which shows the specific example of the positional information on the table of FIGS. It is a figure which shows the specific example of viewing restrictions information of the table of FIGS. It is a figure which shows the specific example of the visible grade of the table of FIGS. It is a figure which shows a specific example of the image display means in FIG. 1, and a local stereo image information output means. It is a block diagram of 2nd Embodiment of the multiple viewpoint stereo image display apparatus of this invention. It is a flowchart of 2nd Embodiment of the multi-viewpoint three-dimensional video display method. It is a figure which shows an example of the geometric optical relationship in the super multi-view state of the optical system in this invention. It is a figure which shows a mode that a several observer observes a different three-dimensional image. It is a figure which shows a mode that a real image is observed with a monocular. It is an enlarged view of the principal part of FIG. It is an enlarged view of the part of four lenses of ten lenses in FIG. It is a figure which shows the behavior of the light ray in the inside of one lens which comprises a local stereo image output means. It is a figure which shows a mode that a real image is tied with three lenses. It is a figure which shows a mode that an image is not formed with three lenses. It is a figure which shows a mode that a virtual image is tied with three lenses. It is a figure which shows the actual light ray in case a three-dimensional image is formed as a real image. It is an enlarged view of FIG. It is a block diagram of 3rd Embodiment of the multiple viewpoint stereo image display apparatus of this invention. It is a conceptual diagram which shows that a some observer observes a local three-dimensional image | video. It is a figure which shows a mode that a light ray is deflected by a single lens.

Explanation of symbols

11 Information separator
DESCRIPTION OF SYMBOLS 12 Observer information input device 13 Display mode determination device 14 Display mode control signal generator 15, 18 Decoder 16, 205 Image display means 17, 206 Local stereo image information output means 19 Phase shifter 20 Local acoustic information output Means 21 Recording medium 22 Regenerator 23 Video decoder 24 Audio decoder 25 Video CH separator 26 Acoustic CH separator 27 Local sound field generator 28 Speaker array 200 Two-dimensional image display surface 205a to 205j Pixel 206a in the image display means -206j Lens of microlens array 300 Deflection optical element 400, 405 Eyeball 500 Stereoscopic image

Claims (3)

In a multi-view stereoscopic video display method for displaying different stereoscopic video information for each viewpoint of a plurality of observers using one stereoscopic video information display means,
A first step of inputting position information and personal information of the plurality of observers;
Display mode of local stereoscopic video information individually for each of the plurality of observers by combining depth information and / or viewpoint interval information given in advance for each predetermined section of the stereoscopic video information and the personal information A second step of determining
A third step of outputting the local stereoscopic video information in the display mode determined in the second step to each of the plurality of observers having the input position information. A multi-view stereoscopic video display method. In a multi-view stereoscopic video display device that displays different stereoscopic video information for each viewpoint of a plurality of observers using one stereoscopic video information display means,
Information input means for inputting position information and personal information of the plurality of observers;
A combination of the personal information input by the information input means and depth information and / or viewpoint interval information given in advance for each predetermined section of the stereoscopic video information is individually assigned to each of the plurality of observers. Display mode determining means for determining a display mode of local stereoscopic video information;
Local stereoscopic video information for outputting the local stereoscopic video information in the display mode determined by the display mode determining unit to each of the plurality of observers having the position information input by the information input unit And a multi-viewpoint three-dimensional video display device. In a multi-view stereoscopic video display program for causing a computer to execute a multi-view stereoscopic video display device that displays different stereoscopic video information for each viewpoint of a plurality of observers,
A first step of linking and storing the position information and personal information of the plurality of observers;
Display mode of local stereoscopic video information individually for each of the plurality of observers by combining depth information and / or viewpoint interval information given in advance for each predetermined section of the stereoscopic video information and the personal information A second step of determining
Causing each of the plurality of observers having the input position information to execute the third step of outputting the local stereoscopic video information in the display mode determined in the second step. A featured multi-view 3D image display program.

Images