JP2012003033A - Modulation pattern generation method and holographic stereogram creation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a holographic stereogram creation device that can omit re-acquisition of a multi-viewpoint image used for a holographic stereogram and can omit re-generation of a modulation pattern when a reproduced image is processed or modified.SOLUTION: A modulation pattern generation method comprises: a step for acquiring plural pieces of multi-viewpoint image data for each object by changing relative positions of a subject and a camera and individually imaging each object from plural different viewpoint positions; a step for combining a modulation pattern group that modulates object light to be recorded in a holographic stereogram from the multi-viewpoint image data for each of the plural objects; and a step for re-combining a new modulation pattern from plural modulation patterns corresponding to the plural objects for each element hologram. A holographic stereogram creation device comprises a re-combining modulation pattern generation unit and a spatial light modulator for modulating object light on the basis of a new modulation pattern for modulating the object light.

Description

  The present invention relates to a holographic stereogram creation apparatus (hereinafter also referred to as a holographic printer), and more particularly, to a modulation pattern generation method and a holographic stereo for generating a modulation pattern for modulating object light to be recorded in a holographic stereogram. Relates to Gram equipment.

  In general, a hologram divides laser light (coherent light) into two parts, irradiates one of the objects on the subject and applies the diffuse reflected light (object light) to the photosensitive sheet coated with the hologram recording medium, while simultaneously It is known that light is directly irradiated onto a photosensitive sheet as reference light at a predetermined angle, and interference fringes due to interference between the two lights are recorded on the photosensitive sheet. By illuminating the hologram with reference light at the same angle as at the time of recording, object light having the same intensity and direction as at the time of recording is reproduced, and a three-dimensional image of the subject is obtained.

  Stereograms are known that enable stereoscopic viewing by viewing a set of parallax images obtained by photographing an object corresponding to the viewpoint positions of the left and right eyes of a human. In general, stereoscopic vision means that a human perceives a three-dimensional object by vision, and has five physiological factors, adjustment of eyeball lens, binocular width comparison angle, binocular parallax, motion parallax, and visual field expansion effect. ,by. Among them, the stereogram uses binocular parallax with a high stereoscopic effect. As the two-viewpoint stereogram, there are a naked-eye type using a lenticular and an eyeglass type such as an anaglyph. For example, a two-view stereogram can only be viewed stereoscopically in front of a set of parallax images, but it can reproduce binocular parallax, but ignores other factors necessary for stereoscopic recognition such as convergence and focus adjustment. It can be said that it is insufficient for stereoscopic vision.

  The holographic stereogram records / reproduces a parallax image used for the stereogram by a hologram. Specifically, a plurality of two-dimensional images obtained by photographing the subject from different viewpoints (observation positions) are recorded on a photosensitive sheet so as to be spread as dot-shaped (or strip-shaped) small holograms (element holograms). At this time, if 60 viewpoint holograms are placed, for example, 60 element holograms are spread, 60 parallax images are reproduced, so that motion parallax can also be used, and according to changes in the viewpoint of the observer and a large number of observers. A three-dimensional image can be reproduced.

  Thus, in the holographic stereogram, it is possible to increase the number of parallaxes (number of element holograms, number of images). For example, if the number of element holograms is increased so that a plurality of images are included in one eye pupil, the congestion or Focus adjustment and other functions can also be performed.

  The holographic stereogram and the device thereof have been developed for further natural stereoscopic vision (see, for example, Patent Document 1 and Non-Patent Documents 1 to 4). In the holographic stereogram, there is a case where a camera is moved both horizontally and vertically with respect to a subject in photographing to obtain a two-dimensional image. For example, a holographic stereogram only in the horizontal direction is obtained as an HPO holographic. Some stereograms (Horizontal Parallax Only) are called two-dimensional holographic stereograms as full-parallax holographic stereograms.

Japanese Patent Laid-Open No. 10-143058

“Study on image reproduction characteristics by holographic stereogram”, Masahiro Yamaguchi et al., “Optics”, Vol. 22, No. 11 (November, 1993), pp. 228 714 “Holographic three-dimensional printer: new method”, Masahiro Yamaguchi, APPLIEDOPTICS, vol. 31, no. 2, (1992) “Immediate holographic 3D printing technology”, Akira Shirakura, IEICE Technical Report vol. 22No. 43, pp7-15, (1998) “Creation system for omnidirectional parallax holographic stereogram from live-action”, Kota Kota et al., IEICE Technical Report vol. 31, no. 40, pp9-12, (2007)

  Holographic stereograms, especially full-parallax holographic stereograms, depend on the screen size and resolution of the image being played, depending on the number of parallax images (for example, thousands, tens of thousands, and hundreds of millions) , Also referred to as a multi-viewpoint image).

  Since it is almost impossible to acquire hundreds of millions of parallax images in reality, it has been devised to reduce the number of parallax images acquired using approximation, but even in this case, a relatively natural stereoscopic image is reproduced. In order to do this, it is necessary to acquire at least several thousand parallax images. Acquisition of the parallax image is repeated while moving the camera little by little with respect to the subject. Therefore, in order to acquire thousands of parallax images, considerable shooting time is required. Of course, the subject must remain stationary during that time. In order to create a holographic stereogram, a plurality of acquired original images, that is, multi-view images are acquired and converted into data, and then element holograms are generated by image processing technology based on the converted multi-view image data. It is necessary to generate a modulation pattern group for recording.

  Creating a holographic stereogram of a person against the background of famous scenery in a tourist spot can leave a three-dimensional memorial photo. In this case, the person who is the subject is stationary during the shooting of the multi-viewpoint image, but if the car or other tourist moves as the background, the correct multi-viewpoint image data cannot be obtained. There has been a problem that the multi-viewpoint image data must be acquired again and a modulation pattern group must be generated accordingly.

  In addition, when acquiring multi-viewpoint image data, for example, when date information taken with the camera settings is displayed in a small size in the corner of the screen, the date is also displayed on the stereoscopic image reproduced by the holographic stereogram. If the date information is later deleted or changed, the modulation pattern group is generated again after deleting the date information or changing the date from the multi-viewpoint image. There is a problem that the element hologram has to be printed after correction, which requires a processing time and labor.

  Therefore, the present invention can avoid re-acquisition of the multi-viewpoint image by reusing the multi-viewpoint image data acquired once, or omit regenerating the modulation pattern group when processing and correcting the reproduced image. One example is to provide a modulation pattern generation method and a holographic stereogram device that can be used.

The modulation pattern generation method according to claim 1 is a modulation pattern generation method for creating a holographic stereogram of a subject composed of a plurality of objects,
Acquiring a plurality of multi-viewpoint image data for each object by photographing the object independently from a plurality of different viewpoint positions while changing the relative position of the subject and the camera;
For each of the plurality of objects, synthesizing a modulation pattern group for modulating object light for recording in a holographic stereogram from the multi-viewpoint image data;
Re-synthesizes a new modulation pattern from a plurality of modulation patterns for the plurality of objects for each element hologram.

  The modulation pattern generation method according to claim 2, wherein the step of recombining a new modulation pattern from the plurality of modulation patterns for the plurality of objects compares the context of the plurality of objects for each pixel of the modulation pattern. The pixel value of the modulation pattern for the frontmost object is the pixel value of the new modulation pattern.

  4. The modulation pattern generation method according to claim 3, wherein the step of acquiring each multi-viewpoint image data for the plurality of objects records depth information in addition to a luminance value for each pixel of the multi-viewpoint image data. Including steps.

The holographic stereogram creation device according to claim 4, wherein the holographic stereogram creation device creates a holographic stereogram of a subject composed of a plurality of objects,
A recombination modulation pattern generation unit for recombining a new modulation pattern for modulating object light from the modulation patterns for the plurality of objects by the modulation pattern generation method according to claim 1;
A spatial light modulator for modulating the object light based on a new modulation pattern for modulating the object light;
And an objective lens that condenses the object light and records the element hologram on the hologram recording medium irradiated with the reference light.

It is a schematic diagram which shows an example of a structure of the holographic printer which can be used for embodiment by this invention. It is a flowchart which shows the holographic stereogram preparation method by a holographic printer. It is a typical schematic fragmentary sectional view for demonstrating the recording principle of an initial holographic stereogram. It is a typical schematic fragmentary sectional view for demonstrating the recording principle of an initial holographic stereogram. It is a typical schematic fragmentary sectional view for demonstrating the recording principle of an initial holographic stereogram. FIG. 5 is a schematic schematic partial cross-sectional view for explaining a position where a parallax image is acquired in recording of a holographic stereogram, a relationship between a photosensitive sheet and a subject, and a synthesized SLM image. It is a schematic diagram explaining the example of the method of acquiring the multiview image in the holographic stereogram creation method. It is a schematic diagram explaining the other example of the method of acquiring the multiview image in the holographic stereogram creation method. It is a schematic diagram for demonstrating the method of acquiring a multi-viewpoint image with the imaging | photography method demonstrated using FIG. 7, and synthesize | combining an SLM image based on them. It is a flowchart for demonstrating the method of acquiring a multiview image with the imaging | photography method demonstrated using FIG. 7, and synthesize | combining an SLM image based on them. FIG. 9 is a schematic diagram for explaining a method of acquiring multi-viewpoint images using the photographing method described with reference to FIG. 8 and generating an SLM image based on the multi-viewpoint images. It is a perspective view for demonstrating the virtual three-dimensional photographic subject which records the holographic stereogram in embodiment by this invention. FIG. 5 is a schematic schematic partial cross-sectional view for explaining a relationship between a position where a parallax image is acquired, a photosensitive sheet, and a subject in a holographic stereogram recording according to an embodiment of the present invention and an SLM image. It is a flowchart for demonstrating the SLM image re-synthesis method for producing | generating the multi viewpoint image of embodiment by this invention. It is a flowchart for demonstrating the SLM image resynthesis method for producing | generating the multi-viewpoint image of other embodiment by this invention.

  A holographic stereogram creation system using a holographic stereogram creation apparatus, a so-called holographic printer, will be described below with reference to the drawings.

  The holographic stereogram creation system is largely divided into the following three parts: an imaging section that is an image acquisition unit that captures the subject to be recorded, and a modulation pattern generation that generates a modulation pattern that modulates object light for the holographic stereogram Divided into sections and printed sections that create holographic stereograms.

  First, in a photographing section, a subject to be recorded is photographed sequentially from a number of observation points, whereby a plurality of original images, that is, multi-view images are obtained and converted into data, and multi-view image data is obtained. Next, in the modulation pattern generation section, the computer calculates and synthesizes from the converted multi-viewpoint image data, and generates modulation data corresponding to a modulation pattern (referred to as an SLM image) to be specifically recorded. Then, in the printing section, such a modulation pattern is displayed on, for example, a light-transmitting liquid crystal panel, illuminated with laser light, and the transmitted light modulated by the liquid crystal panel is collected on a photosensitive sheet (hologram surface) by a condenser lens. Illuminate with reference light and record sequentially.

  Here, since the modulation pattern synthesis depends on the method of acquiring the multi-viewpoint image, the printing section will be described first, and the acquisition of the multi-viewpoint image and the modulation pattern synthesis that are the shooting section and the modulation pattern generation section will be described. The details will be described later.

− <Print section: Recording of element hologram> −
FIG. 1 schematically shows an example of the configuration of a holographic printer that can be used in the embodiment. Immediately after the laser light source 21, a shutter 23 that can be controlled to open and close by the control unit 22 is placed so that the laser beam is guided to the rear optical system 24 only when each element hologram is exposed (recorded). The light that has passed through the shutter 23 is converted into parallel light by the collimator lens 241. Note that the cross-sectional area of the laser beam can be enlarged using an expander (not shown) in the optical system. The parallel light from the light source 21 is split into two optical paths by the beam splitter 242, and one light passes through the spatial light modulator 244 (the SLM image is displayed on the panel) via the mirror 243 and then the object. The spherical wave is condensed on the photosensitive sheet 31 by the objective lens 246 through the mirror 245, and an element hologram is recorded. The other parallel light is irradiated obliquely from the back side of the photosensitive sheet 31 through the mirror 247 as reference light. At this time, it is desirable that the optical path lengths of the object light and the reference light are substantially the same.

  The photosensitive sheet 31 of the hologram recording medium is fixed on the stage 35 and held so that the main surface of the photosensitive sheet 31 is perpendicular to the optical axis of the object light. The position of the stage 35 can be freely changed in a plane (XY plane) perpendicular to the optical axis of the object light by, for example, a two-axis stepping motor 41 for X axis and Y axis. The spatial light modulator 244 includes, for example, a transmissive liquid crystal panel and a frame memory that holds modulation data for the modulation pattern, and displays the modulation pattern on the liquid crystal panel via the frame memory, thereby detecting the optical axis of the object light. The light intensity distribution in the vertical plane can be freely modulated. The modulation data of the modulation pattern on the frame memory can be freely rewritten by the control unit 22.

  A holographic stereogram is created according to the procedure shown in the flowchart of FIG. As an example, the photosensitive sheet 31 is exposed with 600 horizontal element holograms, 400 vertical element holograms, and an element hologram interval Δe = 250 μm. In this case, the finally printed holographic stereogram is 15 cm × 10 cm. A modulation pattern (SLM image) displayed on the panel of the spatial light modulator 244 when recording the nxth element hologram in the horizontal direction and the nyth in the vertical direction is denoted as SLM (nx, ny). That is, nx is a horizontal element hologram number (1 to nxmax), and ny is a vertical element hologram number (1 to nymax).

  First, as shown in FIG. 2, when printing is started, it is initialized to nx = 1 and ny = 1 (step S1), and the stepping motor is considered in consideration of the size of the holographic stereogram to be finally printed. 41 is rotated to move the stage 35 to the printing start position (step S2). The position of the stage 35 at this time is the origin, that is, the coordinates (0, 0). At this time, the shutter 23 is closed.

  Next, SLM (1, 1) is sent to the frame memory of the spatial light modulator 244 (step S3). Next, the shutter 23 is opened and the photosensitive sheet is exposed for a predetermined time (step S4). When a certain time has elapsed, the shutter 23 is closed (step S5), and the stepping motor 41 is rotated to set the stage position to the coordinates (0, 250 μm) (step S6). Since there is a possibility that the stage 35 is vibrating even after the rotation of the stepping motor 41 is completed, in this case, the process waits for a certain time (step S7).

  Next, the SLM (1, 2) is sent to the frame memory of the spatial light modulator 244 until the stage 35 is moved after the shutter 23 is closed and the vibration is settled. When the vibration is resolved, the shutter 23 is opened and exposure is started. After exposure for a certain time, the shutter 23 is closed and the stage 35 is moved to the coordinates (0,500 μm). Thereafter, the same processing is repeated until the exposure of SLM (1,400). When the exposure of the SLM (1,400) is completed, the stage 35 is moved in the horizontal direction and the stage position is set to coordinates (250 μm, 10000 μm). Meanwhile, SLM (2,400) is sent to the frame memory of the SLM. When the vibration is resolved, the shutter 23 is opened and exposure is started again. When the exposure is completed, the stepping motor 41 is rotated so that the vertical stage coordinates are decreased, and the stage position is set to the coordinates (250 μm, 99750 μm). When 400 element holograms are exposed in the vertical direction as described above, the next 400 element holograms are exposed in the vertical direction by moving by 250 μm in the horizontal direction. Eventually, the exposure of the SLM (600, 400) is performed with the stage position at coordinates (150,000 μm, 0 μm). That is, the element hologram is recorded by raster scanning for each increment 1 of the element hologram numbers nx and ny in the horizontal and vertical directions by the loop of steps S3 to S14 shown in FIG.

  When nx = nxmax is satisfied in step S11 and all exposures are completed, the photosensitive sheet is taken out from the stage 35 (step S15). Depending on the type of photosensitive sheet, a process for fixing the recording may be required thereafter (step S16). For example, when a photopolymer material is used for the photosensitive sheet, the recording can be fixed by placing it in a high-temperature oven for a certain time after recording. With the above, a full parallax holographic stereogram can be created. In this case, the number of exposures of the element hologram is 600 × 400 = 240,000 times, and 240,000 SLM images for that purpose are required.

  Before describing the photographing section and the modulation pattern generation section, the recording principle of the initial holographic stereogram and the recording principle of the holographic stereogram improved for the hologram printer based on it will be described.

− <Recording principle of early holographic stereogram> −
The reason why we can observe an object (subject) can be interpreted as light rays scattered on the object surface enter the pupil and stimulate the retina. Consider shooting a certain subject with a camera and projecting the image onto a diffusion screen (see FIG. 3). In this case, even if the subject is removed, the light beam projected / diffused on the screen is incident on the camera position (observation point), so that the same image (ray group) as when the subject exists can be observed.

  A photosensitive sheet is placed at the position (camera position) where the image was acquired, and a slit is provided immediately before (screen side) (see FIG. 4). When the same light is irradiated from the back side simultaneously with illuminating the film with coherent light, interference fringes are recorded only on the slits of the photosensitive sheet. That is, a minute hologram is formed by using diffused light from the screen as object light and light irradiated from the back surface as reference light. This micro-hologram is called an element hologram.

  The above operation is performed many times while moving the camera position little by little in the horizontal direction. At that time, the opening position of the slit is always adjusted to the camera position at that time, that is, the position where the projection image is acquired.

  By the way, in general, a hologram has a property of generating object light when illuminated with the same light as the reference light irradiated when forming the hologram. Strictly speaking, the reference light is diffracted as object light by the interference fringes. In other words, the continuous formation of element holograms on the photosensitive sheet can be interpreted as recording / saving the direction and intensity of the light beam observed at each camera position (observation point). Therefore, when the photosensitive sheet is viewed from the reference light irradiation side, an image with parallax can be observed depending on the observation position (see FIG. 5). However, in practice, the direction of the object light generated from each element hologram when illuminated with the reference light is slightly changed due to the influence of diffraction as the distance from the element hologram increases. Therefore, as shown in FIG. 5, if the subject (reproduced image) is located far from the photosensitive sheet 31, the reproduced image is easily blurred.

-<Improved recording principle for holographic printers>
In FIG. 6, the position where the parallax image is acquired (camera) and the position where the element hologram is recorded (photosensitive sheet 31) are separated so that the subject and the photosensitive sheet are close to each other, and the reproduction image is difficult to blur. It is a schematic diagram which shows the recording principle of a holographic stereogram. A holographic printer uses this recording principle to build a system.

  In this case, when recording the element hologram, the captured parallax image is not directly displayed on the spatial light modulator 244, but only necessary pixels are extracted from a plurality of parallax images using a computer, and they are connected. Thus, it is necessary to synthesize one parallax image and display it on the spatial light modulator 244, which will be described later.

  By separating the parallax image acquisition position and the element hologram recording position, there is an advantage that the number of images to be acquired and the number of element holograms to be recorded can be set independently to some extent. As described above, the holographic stereogram becomes more natural as the number of parallaxes during reproduction increases. In a holographic printer, if the number of pixels of the spatial light modulator 244 used is sufficient, the number of parallaxes during reproduction can be increased only by increasing the number of acquired parallax images. Also, using a computer equipped with a 3DCG (3D computer graphics) program, a virtual 3D subject is generated in a virtual space, and a virtual camera is moved little by little, and a virtual image is taken for a very short time. Can be obtained (generated). For this reason, it is possible to increase the number of parallaxes during reproduction only by increasing the number of parallax images without increasing the number of element holograms that directly affects the length of the printing time. Thus, the holographic printer is particularly suitable as a method for creating a holographic stereogram of a virtual subject.

  In addition, the holographic printer uses light condensed by an objective lens as object light when recording an element hologram. For this reason, there exists an advantage that a photopolymer material much lower sensitivity than a silver salt film can be utilized as a photosensitive sheet. Further, since a diffusing screen and a projection optical system are not required, the optical system can be reduced in size.

-<Photographing section and modulation pattern generation section: Acquisition of multi-viewpoint images and synthesis of SLM images>-
In the case of a parallax holographic stereogram, a multi-viewpoint image is required in which the viewpoint (camera position) is changed little by little in two directions, horizontal and vertical.

(Shooting section)
There are several image acquisition methods, that is, photographing methods for moving the viewpoint when acquiring a multi-viewpoint image. Here, the following two multi-viewpoint image acquisition methods will be described. Originally, a case where the viewpoint position is moved two-dimensionally should be explained, but here it will be explained as an HPO holographic stereogram for the sake of simplicity. It goes without saying that the case of a full parallax holographic stereogram can be considered similarly.

  The first photographing technique is a technique (see FIG. 7) of photographing while keeping the direction constant without changing the optical axis direction of the camera and moving the camera parallel to the subject. As shown in FIG. 7, it is obtained by intermittently photographing a large number of subjects from different positions. That is, the camera is moved from a position where the subject enters the camera angle of view range to a position where the subject deviates from the camera angle range, with the camera optical axis directed toward the subject parallel.

  The second shooting method is a method of shooting while rotating the camera so as to surround a fixed subject while keeping the distance between the camera and the subject constant (see FIG. 8). In addition to rotating the camera around the subject, the subject is fixed at the center of rotation of a turntable (not shown), and the distance between the subject and the camera is kept substantially constant while the camera is fixed toward the subject. Thus, even when the turntable is rotated together with the subject, a large number of similar parallax images with different viewpoints can be obtained. That is, a plurality of subject images with different viewpoints can be intermittently shot by moving the camera relative to a fixed subject so that the subject revolves or rotates relative to the fixed camera. The camera photosensitive surface is composed of a photoelectric converter such as a CCD or CMOS sensor, and photoelectrically converts each received image to generate and output multi-viewpoint image data of a plurality of images.

(Modulation pattern generation section)
FIG. 9 is a schematic diagram for explaining a procedure for acquiring multi-viewpoint image data by the imaging method shown in FIG. 7 and synthesizing an SLM image based on the acquired data. The number of pixels of the spatial light modulator 244 is N, the number of element holograms to be recorded is K, the angle of view (full angle) of the objective lens is θo, the angle of view of the camera for acquiring a multi-viewpoint image is θc, and the pixels of the acquired image Let the number be L. That is, the field angle per pixel of the spatial light modulator 244 is θo / N, and the field angle per pixel of the acquired image is θc / L.

As shown in FIG. 9, since parallel light is incident on the spatial light modulator 244, the coordinates of the display surface of the spatial light modulator 244 and the coordinates of the pupil plane of the objective lens 246 correspond to each other. A point on the pupil plane of the objective lens 246 corresponding to each pixel of the spatial light modulator 244 is S _j (j = 1, 2,..., N), and the element hologram coordinates are H _i (i = 1, 2,...). , K), and let C _ij be the point where the light beam connecting S _j and H _i intersects the multi-viewpoint image acquisition plane.

Here will be described a method of synthesizing i th SLM image for the element hologram H _i as an example.
First, a subject is photographed by placing the camera at a point C _i1 where the light beam connecting S ₁ and H _i (the broken line on the left side from S _{1 in} FIG. 9) intersects the multi-viewpoint image acquisition plane. At this time, since the light beam connecting S ₁ and H _i is tilted by + θo / 2 with respect to the optical axis of the camera, the subject (its luminance value) on the line connecting S ₁ and H _i is on the acquired image. Now from the center pixel

Is recorded in pixels that are only a distance away. This pixel is taken out and used as the first pixel of the SLM image (to be synthesized).

Next, the camera is moved to a point C _ij where a light beam connecting S _j (where j = 2) and H _i (an intermediate broken line from S _{j in} FIG. 9) intersects the multi-viewpoint image acquisition plane, and the subject is moved. Take a picture. At this time, since the light beam connecting S _j and H _i is tilted by + θo / 2−θo / N with respect to the optical axis of the camera, the subject on the line connecting S _j and H _i (its luminance value) Is from the center pixel on the acquired image

Is recorded in pixels that are only a distance away. This pixel is extracted and set as the jth (j = 2) pixel of the SLM image (to be synthesized).

  In this way, the multi-viewpoint image is taken while sequentially moving the camera (j = 2, 3,... N).

Finally, the camera is moved to a point C _iN where a light beam connecting S _N and H _i (right dashed line from S _{N in} FIG. 9) intersects the multi-viewpoint image acquisition plane, and the subject is photographed. Since the light beam connecting S _N and H _i is tilted by −θo / 2 with respect to the optical axis of the camera, the subject (its luminance value) on the line connecting S _N and H _i is the center on the acquired image. From pixel

Is recorded in pixels that are only a distance away. This pixel is taken out and set as the Nth pixel of the SLM image (to be synthesized).

This can be synthesized SLM image for elemental hologram H _i.

By performing this series of operations on all the element holograms (H _{i to} H _K ), all necessary SLM images can be synthesized. FIG. 10 shows a flowchart of the SLM image synthesizing method by the photographing technique shown in FIGS. That is, it is possible to acquire a multi-viewpoint image by a procedure such as the flowchart shown in FIG. Here, i is an element hologram number (1 to K), and j is an SLM pixel number (1 to N). When shooting is started, j = 1 and i = 1 are initialized (step Sa1), and the camera is moved to a position _Cij (j = 1, i = 1) by being transferred by a stepping motor or the like (step Sa2). Then, to get the perspective projection image off the camera shutter (step Sa3), and sets the j-th pixel of the coordinates _{H i} for SLM image element holograms (step Sa4). Next, the SLM pixel number j is determined for one of the coordinates H _i of the element hologram (step Sa5), j is incremented by 1 until j = N (step Sa6), and the loop returning to step Sa2 is repeated. When j = N is satisfied, the element hologram number i of the coordinate H _i of the next element hologram is discriminated (step Sa7), i is incremented by 1 until i = K (step Sa8), and the process returns to step Sa2. The loop is repeated, and acquisition of all multi-view images is terminated when j = N and i = K are satisfied.

  FIG. 11 is a schematic diagram for explaining a procedure for acquiring multi-viewpoint images by the second method shown in FIG. 8 and synthesizing an SLM image based on them. Since the number of pixels of the spatial light modulator 244 and the angle of view of the objective lens are the same as those used in the description of FIG. 9, the description of the same elements is omitted.

  In the case of FIG. 11, since the optical axis of the camera is always directed to the element hologram to which attention is paid, the pixels in the vicinity of the center of the two-dimensional image acquired at each viewpoint position are taken out and each of the spatial light modulators 244 is extracted. SLM images can be synthesized by aligning them in the same order as the pixels. The second method has an advantage that a camera with a narrow angle of view can be used because pixels near the optical axis are always used in the acquired image.

  When the SLM image is prepared by the method as described above, printing, that is, recording of the element hologram is started by the holographic printer. By the way, if the number of element holograms to be recorded is K = 600 × 400 and the number of pixels of the spatial light modulator 244 (the number of viewpoints at the time of reproduction) is N = 64 × 64, the acquired two-dimensional image is K × N = 600 × 400 × 64 × 64 = 98.304 million sheets. It is almost impossible to get this number in the real world. Of course, if it is acquired (drawn) in a virtual space using 3DCG (3D computer graphics) technology, it can be processed at a higher speed than the real world, but it is still impractical.

  In view of this, it is conceivable to use a single parallax image as a parallax image for a plurality of element holograms by using approximation. An example of a specific approximation method is described in Non-Patent Document 4 described above. In this document, the number of element holograms is 200 × 120 = 24000 and the number of parallaxes is 100 × 7 = 700. X700 = 16.8 million multi-viewpoint images should be required, but this is 500 x 7 = 3500.

  By applying such ideas, it is possible to greatly reduce the multi-viewpoint images to be acquired, but even in that case, at least several thousand parallax images are acquired in order to reproduce a relatively natural stereoscopic image. There is a need to.

-<Embodiment 1>-
As described above, a multi-viewpoint image for synthesizing SLM images requires an enormous number of images. Therefore, not only a real subject but also a full parallax holographic stereo of a virtual subject using 3DCG (three-dimensional computer graphics) technology. It takes an enormous amount of time to create a gram. Therefore, based on the above configuration, the inventor has intensively studied the processing technology of SLM images, and as a result, when processing and correcting a reproduced image, the subject is separated for each object, and each object exists independently. Embodiment 1 has been devised in which a modulation pattern group (hereinafter also referred to as an SLM image group) is prepared and the modulation pattern group for each object once generated is repeatedly used.

  This eliminates the need to re-acquire a multi-viewpoint image and then generate an SLM image again, thereby shortening the time for creating a modified and modified holographic stereogram.

  In the first embodiment, the subject is divided for each object, and each SLM image group is generated independently. In the printing section, a new SLM image is synthesized from a plurality of SLM images for each element hologram, and the resultant SLM image is spatially generated. By displaying on the light modulator 244, it is possible to observe a stereoscopic image in which all objects are mixed in the finally printed holographic stereogram. The following is an HPO holographic stereogram for ease of explanation, but it goes without saying that a full parallax holographic stereogram can also be realized by a similar method.

  Consider a case where a holographic stereogram of a virtual three-dimensional object and a plurality of objects shown in FIG. 12 is created. Objects A, B, and C are sequentially arranged along the z axis from the back in the xyz coordinate space. It is assumed that the xy plane corresponds to the element hologram recording surface and observation is performed from the positive side of the z axis toward the negative side.

  In general, the objects A, B, and C can be collectively recognized as one subject, a multi-viewpoint image of this virtual subject can be acquired, and an SLM image can be generated based on these images. In this case, if it is desired to create a holographic stereogram in which only the object B is deleted for processing and correction at a later date, a multi-viewpoint image is acquired for a new virtual subject consisting only of the objects A and C. It is necessary to correct and generate an SLM image. In the case of a subject group composed of three objects, there are a total of seven possible combinations (subject A, subject B, subject C, subject group A & B, subject group A & C, subject group B & C, subject group A & B & C). That is, in order to create all combinations of holographic stereograms, it is necessary to acquire multi-viewpoint images seven times.

  In the present embodiment, only a multi-viewpoint image in which the subjects A, B, and C are individually drawn is acquired, that is, three sets of multi-viewpoint images (including luminance value + z coordinate) are acquired, By only generating and storing SLM image data groups (modulation data groups including luminance values + z coordinates), all seven holographic stereograms can be created by recombining them.

  Therefore, as described above, when acquiring a multi-viewpoint image of a single object, not only the luminance value but also the z coordinate (from the element hologram recording surface obtained by converting the distance from the camera photosensitive surface to the subject surface for each pixel) (Distance). FIG. 13 is a schematic diagram for explaining a method of acquiring a multi-viewpoint image by the method of FIG. 5 and generating an SLM image data group based on the multi-viewpoint image when the subject object B exists alone as an example. is there. In the method described with reference to FIG. 7, not only the luminance value of the multi-viewpoint image but also the multi-viewpoint image data holding the distance between the camera photosensitive surface and the subject surface is generated, and the SLM image data group generated in this embodiment has the luminance value. And a set of z coordinates for each pixel. In order to obtain the z coordinate for an object existing in the real world, the distance from the camera to each point on the object is measured by a stereo method or the like, and then the distance to the virtually set element hologram surface is calculated. It will be. On the other hand, when a multi-viewpoint image is acquired by placing a virtual object in the CG space, the z coordinate of the virtual space can be used as it is. The positions of pixels extracted when generating the SLM image data group from the multi-viewpoint image are the same as those described with reference to FIG. Needless to say, the case of acquiring a multi-viewpoint image by moving the camera by the method of FIG.

  In this embodiment, an SLM image data group (modulation data group including a luminance value + z coordinate) generated for each of the three single objects A, B, and C is prepared before recombination. FIG. 14 shows a flowchart of a method for recombining a new SLM image group by using three prepared SLM image data groups of single objects (in the case of an HPO holographic stereogram). Here, obj is an index such as an object number indicating which object's SLM image data, and obj_max is the number of objects to be displayed. For example, when objects A, B, and C are to be displayed, object A, The numbers obj = 1, 2, and 3 are assigned in the order of B and C, and then obj_max = 3. Further, i is an element hologram number (1 to K) of the coordinate Hi of the element hologram, j is an SLM pixel number (1 to N, that is, the number of SLM pixels (number of parallaxes)), z0 is a z coordinate value, and obj_Z (j ) Is the z coordinate value of the jth pixel of the SLM image data of the object number obj, SLM (j) is the luminance value of the jth pixel of the image displayed on the spatial light modulator, and obi_I (j) is the object number of obj The luminance of the jth pixel of the SLM image data is assumed. When resynthesis is started, obj = 1, i = 1, and j = 1 are initialized (step Sb1), and the z coordinate value z0 is initialized to the innermost value (step Sb2). Next, the z coordinate value of the jth pixel of the SLM image data of the object number obj is compared with the z coordinate value z0 of the current value (step Sb3), and if the z coordinate value z0 of the current value is exceeded, the j number of the object number obj The z coordinate value of the pixel is updated (step Sb4). Next, it is determined whether or not the object number obj is the maximum (step Sb5), the object number obj is the maximum, for example, increment 1 until obj = 3 (step Sb6), and the loop returning to step Sb3 is repeated. For example, when obj = 3 is satisfied by scanning all the object pixels of the image, the luminance value of the jth pixel of the SLM image displayed on the spatial light modulator is set to the object number that is the highest z coordinate value. The luminance value of the jth pixel of obj is set (step Sb7). If the previous z coordinate value is not exceeded in step Sb3, step Sb4 is skipped and step Sb5 is executed. Next, the SLM pixel number j is discriminated (step Sb8), and j is incremented by 1 until j = N (step Sb9), and the loop returning to step Sb2 is repeated. When j = N is satisfied, the element hologram The element hologram number i at the coordinate Hi is determined (step Sb10), i is incremented by 1 until i = K (step Sb11), the loop returning to step Sa2 is repeated, and j = N and i = K are satisfied. Sometimes the synthesis of all SLM images is finished. Thereby, a new SLM image data group is recombined.

  The flowchart shown in FIG. 14 shows the case where recombination for all element holograms is continuously processed in the modulation pattern generation section, but the recombination processing is performed independently for each element hologram. Can be performed sequentially in the printing section. Especially when the number of element holograms is very large, if it is performed as a modulation pattern generation section at a time, it will be necessary to wait for a certain amount of time before starting printing, but if it is performed sequentially in the printing section, the stage is moving. Therefore, there is an advantage that printing can be started immediately.

  Assuming that the reconstructed image of a holographic stereogram created once is processed or modified, the SLM image data group generated for each object before recombination can be stored in a database so that it can be reused. desirable. Then, a holographic stereogram of a new subject can be created without re-acquiring a multi-viewpoint image in the shooting section.

-<Embodiment 2>-
For example, consider a case where a holographic stereogram of a person is created against a background famous as a memorial at a tourist spot. In this case, if the object A is the background and the object B is the person, the background is generally the background A is the back and the person B is the front.

  In this way, when the anteroposterior relationship between objects of the subject army (which appears to overlap) is clear in advance, or when the creator of the holographic stereogram intentionally determines the anteroposterior relationship, multi-viewpoint image data or SLM images can be synthesized without adding z-coordinates to the SLM image data. FIG. 15 shows a flowchart of recomposition of a new SLM image data group in that case (in the case of an HPO holographic stereogram). Here, i is the element hologram number (1-K) of the coordinate Hi of the element hologram, j is the SLM pixel number (SLM pixel number (parallax number) 1-N), and obi_A (j) is the SLM image of the object A Is the luminance value of the jth pixel of the SLM image of the object B, and SLM (j) is the luminance value of the jth pixel of the image displayed on the spatial light modulator. . When recombination is started, i = 1 and j = 1 are initialized (step Sc1), and the luminance value of the jth pixel of the SLM image of the object B is zero or not (the part of the object B that does not appear to overlap). (The luminance value of the pixel is zero) is determined (step Sc2), and if it is zero, the jth pixel of the SLM image of the object A is set (step Sc3), and the SLM of the object A displayed on the spatial light modulator The luminance value obi_A (j) of the jth pixel of the image is associated with the z coordinate of the jth pixel (step Sc4). If the luminance value of the jth pixel of the SLM image of the object B is not zero in step Sc2, the jth pixel of the SLM image of the object B is set (step Sc5), and the SLM image of the object B displayed on the spatial light modulator is set. The luminance value obi_B (j) of the j th pixel is associated with the z coordinate of the j th pixel (step Sc4). Next, the SLM pixel number j is determined for one of the coordinates of the element hologram (step Sc6), j is incremented by 1 until j = N (step Sc7), and the loop returning to step Sc2 is repeated, j = When N is satisfied, the element hologram number i of the coordinate Hi of the next element hologram is discriminated (step Sc8), i is incremented by 1 until i = K (step Sc9), and the loop returning to step Sa2 is repeated. , J = N and i = K are satisfied, the synthesis of a new SLM image group is terminated.

  Although only two objects A and B are used in this procedure, the same can be considered when there are three or more objects. That is, the luminance value is examined in order from the object to be displayed in front, and the luminance value of the object having a non-zero value may be used as the luminance value of the image displayed on the spatial light modulator.

  In the flowchart shown in FIG. 15, the recombination for all the element holograms is continuously processed in the modulation pattern generation section. However, as described in the explanation of FIG. There is an advantage that printing can be started immediately if it is performed sequentially.

  As described above, according to any of the embodiments, even when a three-dimensional commemorative photograph such as a sightseeing spot or a holographic stereogram is left, a person as a first subject and a background as a second subject are used as multi-viewpoint image data. Since the modulation pattern groups can be acquired separately and the modulation pattern group can be synthesized from them, it is easy to omit the re-acquisition of the multi-viewpoint image or to regenerate the modulation pattern group when processing and correcting the reproduced image. In addition, even if the captured date information is to be changed later, a new modulation pattern group can be re-synthesized from a plurality of modulation pattern groups, reducing the time and effort required to regenerate a modulation pattern group. This has the effect of reducing time.

  At first glance, the above method seems to be the same as two-dimensional image synthesis such as hidden surface removal in computer graphics processing, but the SLM image itself is generated by image processing based on a multi-viewpoint image, It is a modulation pattern displayed on a spatial light modulator used in a holographic printer, and is not an image normally observed by a human on a display. Further, the context in use is the context in the case of observing the holographic stereogram, not the context in terms of the viewpoint position (element hologram position) of the SLM image.

21 Laser Light 22 Control Unit 23 Shutter 24 Optical System 31 Photosensitive Sheet 35 Stage 41 Biaxial Stepping Motor 241 Collimator Lens 242 Beam Splitter 243, 245, 247 Mirror 244 Spatial Light Modulator 246 Objective Lens

Claims (4)

A modulation pattern generation method for creating a holographic stereogram of a subject composed of a plurality of objects,
Acquiring a plurality of multi-viewpoint image data for each object by photographing the object independently from a plurality of different viewpoint positions while changing the relative position of the subject and the camera;
For each of the plurality of objects, synthesizing a modulation pattern group for modulating object light for recording in a holographic stereogram from the multi-viewpoint image data;
Remodulating a new modulation pattern from a plurality of modulation patterns for the plurality of objects for each element hologram.   The step of recombining a new modulation pattern from a plurality of modulation patterns for the plurality of objects compares the front-rear relationship of the plurality of objects for each pixel of the modulation pattern, and the pixel value of the modulation pattern for the foremost object The modulation pattern generation method according to claim 1, wherein a pixel value of the new modulation pattern is used.   2. The step of acquiring each multi-view image data for the plurality of objects includes a step of recording depth information in addition to a luminance value for each pixel of the multi-view image data. The modulation pattern generation method according to any one of Items 1 to 2. A holographic stereogram creation device for creating a holographic stereogram of a subject composed of a plurality of objects,
A recombination modulation pattern generation unit for recombining a new modulation pattern for modulating object light from the modulation patterns for the plurality of objects by the modulation pattern generation method according to claim 1;
A spatial light modulator for modulating the object light based on a new modulation pattern for modulating the object light;
A holographic stereogram creation apparatus, comprising: an objective lens that condenses object light and records an element hologram on a hologram recording medium irradiated with reference light.

Images