WO2008029529A1

WO2008029529A1 - Stereoscoptic video image synthesizing device, shape data generating method and its program

Info

Publication number: WO2008029529A1
Application number: PCT/JP2007/056012
Authority: WO
Inventors: Shiro Ozawa; Takao Abe; Noriyuki Naruto; Itaru Kamiya
Original assignee: Ntt Comware Corporation
Priority date: 2006-09-08
Filing date: 2007-03-23
Publication date: 2008-03-13
Also published as: JP2008067169A

Abstract

A stereoscopic video image synthesizing device is provided for synthesizing a left image seen from a view point of the left eye and a right image seen from a view point of the right eye to form a stereoscopic video image. The stereoscopic video image synthesizing device is comprised of a shape calculating unit for calculating shape data from the left and right images, and a shape output unit for providing the shape data calculated by the shape calculating unit to a kinesthetic-and-tactile sense indicating unit. Typically, the shape calculating unit extracts an image of a specific object from each of the left and right images and gives coordinates in a vertical direction to the image in accordance with distance from the outer contour of the object image to generate the shape data of the object.

Description

Specification

3D image composition device, shape data generation method and program thereof

TECHNICAL FIELD [0001] The present invention relates to a stereoscopic video composition device, a shape data generation method, and a program therefor, in particular, in addition to the synthesis of a stereoscopic video, and the output of shape data to a haptic haptic presentation device. This application claims priority based on Japanese Patent Application No. 2006-244198 filed on Sep. 8, 2006, the contents of which are incorporated herein by reference.

Background art

[0002] A conventional stereoscopic image display device prepares an image that also has two viewpoint power corresponding to the left and right eyes, and uses a Noria method (for example, see Patent Document 1 and Patent Document 2) or a polarizing glass screen. By displaying on a three-dimensional display with a chatter method, the user can perceive in three dimensions.

In addition, a force feedback device that has a pen-type operation unit that allows you to experience tactile sensation by operating the pen, and to experience the tactile sensation of the hand when worn on the arm. There is also a haptic display device such as a boutique device that can do this.

Patent Document 1: JP-A-8-248355

Patent Document 2: Japanese Translation of Special Publication 2003-521181

Disclosure of the invention

Problems to be solved by the invention

[0003] However, the conventional 3D image display device only presents 3D images, and there is a problem that even if an object appears to be raised, it cannot be touched. There is. In addition, based on shape data such as CAD (Computer Aided Design) data, while displaying CG (Computer Graphics), it was possible to present a haptic and tactile sensation with a haptic haptic device. Shape data that matches the video must be prepared, and even if it can be applied to CG that requires shape data to generate video, it cannot be applied to live-action video shot with a video camera, etc. There was a problem.

[0004] The present invention has been made in view of such circumstances, and an object thereof is a video camera or the like. 3D image data that can be presented by the tactile sense display device is output, while the 3D image display device presents the 3D image of the live-action image taken with the 3D image display device. An object of the present invention is to provide a 3D image synthesizing apparatus capable of performing the above.

Means for solving the problem

[0005] The present invention has been made to solve the above-described problem, and the stereoscopic video composition apparatus according to the present invention includes a left image viewed from the left eye viewpoint and a right image viewed from the right eye viewpoint. In a stereoscopic video composition device that synthesizes a stereoscopic video, a shape calculation unit that calculates shape data of a subject from the left image and the right image, and shape data calculated by the shape calculation unit And a shape output unit for outputting to the tactile sense presentation device.

[0006] As a typical example, the shape calculating unit extracts an image of a specific subject from each of the left image and the right image, and for each pixel of the extracted subject image, an image of the subject is extracted. According to the distance from the outline, a vertical coordinate is given to the image to generate shape data of the subject, and the position of the specific subject in the display space of the stereoscopic video synthesized by the device is extracted. Based on the parallax of the image of the specific subject, the shape data in which the generated shape data is arranged at the calculated position is calculated.

[0007] As another typical example, the shape calculation unit extracts an image of a specific subject from each of the left image and the right image, and performs stereo measurement based on the parallax of the extracted image to thereby determine the shape of the subject. Calculate the data.

[0008] As another typical example, the shape calculation unit calculates shape data of a subject by stereo measurement based on a parallax between the left image and the right image.

[0009] Further, the shape data generation method of the present invention is the shape data generation method in the stereoscopic video composition device that synthesizes a stereoscopic video from a left image viewed from the left eye viewpoint and a right image viewed from the right eye viewpoint. A first process in which the stereoscopic video synthesizer calculates shape data of the object from the left image and the right image, and the shape data calculated in the first process by the stereoscopic video synthesizer, And a second process of outputting to a haptic * tactile presentation device.

[0010] Further, the program of the present invention causes a computer to function as a stereoscopic video composition device that synthesizes a stereoscopic video from a left image viewed from the left eye viewpoint and a right image viewed from the right eye viewpoint. The shape calculation unit for calculating the shape data of the subject from the left image and the right image, and the shape data calculated by the shape calculation unit are output to the haptic sense device. It also functions as a shape output unit.

The invention's effect

[0011] According to the present invention, the stereoscopic video synthesizing device matches the stereoscopic video displayed on the stereoscopic video display device by inputting the real shot video shot by the left and right viewpoint power video cameras and the like to the device. It is possible to generate shape data that can be provided by the tactile sense presentation device.

Brief Description of Drawings

FIG. 1 is a block diagram showing an outline of an embodiment of the present invention.

FIG. 2 is a schematic block diagram showing a configuration of a system using the stereoscopic video image synthesizing apparatus 300 in the same embodiment.

FIG. 3 is a schematic block diagram showing a configuration of a stereoscopic video image synthesizing apparatus 300 in the same embodiment.

FIG. 4 is a flowchart illustrating a method for generating a subject image in a shape calculation unit 33 in the same embodiment.

FIG. 5 is a flowchart for explaining a shape data generation method of a subject based on a left subject image or a right subject image in a shape calculation unit 33 in the same embodiment.

FIG. 6 is a diagram illustrating the contents of state S in the same embodiment.

FIG. 7 is a diagram illustrating a process of calculating the depth of the subject area A1 by the shape calculation unit 33 in the same embodiment.

[Fig. 8] Calculation of initial value D in step S1 in Fig. 5 by shape calculation unit 33 in the same embodiment.

FIG. 3 is a diagram for explaining a method.

Explanation of symbols

[0013] 100 ... Left image photographing device

200 ... Right image shooting device

300 ... stereoscopic image synthesizer

400… 3D image display device 500… Katsu sense 'tactile presentation device

31 ···· Left image data input section

32..Right video data input section

33 ... Shape calculation part

34..Shape data output section

35… 3D image composition part

36 ··· 3D image output

BEST MODE FOR CARRYING OUT THE INVENTION

[0014] An outline of an embodiment of the present invention will be described.

As shown in FIG. 1, the 3D video composition device 300 combines the video from the left eye viewpoint and the video of the right eye viewpoint captured by the left video photography device 100 and the right video photography device 200 respectively. Then, when displaying on the stereoscopic video display device 400, shape data is calculated from the captured video and output to the haptic sense / tactile sense presentation device 500. As a result, the user sees the 3D image displayed on the 3D image display device 400 and is displayed at the same time! The haptic haptic sensation for the shape matching the movement of the 3D image is displayed. Can be obtained.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 2 is a schematic block diagram showing the configuration of the stereoscopic video image synthesizing apparatus 300 according to the embodiment of the present invention.

The left image capturing device 100 is a video camera that captures images of the left eye's viewpoint power. The right image capturing device 200 is a video camera that is installed in parallel to the right side of the left image capturing device 100 and captures images with the right eye viewpoint. The three-dimensional video composition device 300 receives the left-eye and right-eye images from the left video photographing device 100 and the right video photographing device 200, synthesizes and outputs the three-dimensional video to the three-dimensional video display device 400, and Shape data that matches the motion is calculated and output to the haptic * tactile presentation device 500.

FIG. 3 is a schematic block diagram showing a configuration of the stereoscopic video image synthesizing apparatus 300 according to the present embodiment.

31 accepts an image input from the left image capturing device 100 and outputs one frame from the image. This is a left video data input unit for outputting left images extracted one by one.

Reference numeral 32 denotes a right video data input unit that receives a video input from the right video shooting device 200 and outputs a right image extracted from the video frame by frame.

33 generates a left subject image and a right subject image obtained by extracting only the subject from the left image and the right image received from the left video data input unit 31 and the right video data input unit 32. The shape calculation unit calculates shape data synchronized with the stereoscopic video data generated by the stereoscopic video synthesis unit 35 by calculating the shape data of the subject based on the above. Details of the subject image generation method and the subject shape data calculation method in the shape calculation unit 33 will be described later.

Reference numeral 34 denotes a shape output unit that outputs the shape data calculated by the shape calculation unit 33 to the haptic * tactile sense presentation device 500.

The stereoscopic video composition unit 35 synthesizes the left image and the right image received from the left video data input unit 31 and the right video data input unit 32, and generates solid video data in a format suitable for the stereoscopic video display device 400. To do.

A stereoscopic video output unit 36 outputs the stereoscopic video data generated by the stereoscopic video synthesis unit 35 to the stereoscopic video display device 400.

FIG. 4 is a flowchart for explaining a method for generating a subject image in the shape calculation unit 33. The flowchart in FIG. 4 shows the processing when the subject is extracted from the left image to generate the left subject image, but the shape calculation unit 33 extracts the subject for the right image in the same manner as the left image. A right subject image is generated.

The shape calculating unit 33 captures only the background with the left image capturing device 100 and stores the left background image received and extracted by the left image data input unit 31 in advance (Sal).

When the shape calculation unit 33 receives the left image from the left video data input unit 31, the shape calculation unit 33 obtains the left background image stored in step Sal and applies to all the pixels i from i = 0 to Imax constituting the left image. Tsu! /, Repeat steps Sa3 to Sa6 (Sa2).

[0019] First, the shape calculating unit 33 calculates the red, green, and blue component values of the pixel i in the left image and the red, green, and blue component values of the pixel corresponding to the pixel i in the left background image. Acquire (Sa3), and determine whether the power of red, green, and blue components of these two pixels match (Sa4). If it is determined in step Sa4 that they match, the process directly proceeds to step Sa6. If it is determined that they do not match, the shape calculation unit 33 extracts the pixel i in the left image as a pixel of the subject image (the pixel After setting the color (as the subject) (Sa5), the process proceeds to step Sa6.

In step Sa6, the shape calculation unit 33 adds 1 to the value of i, and in step Sa7, determines whether i is less than or equal to Imax. That is, it is determined whether or not the processing of steps Sa3 to Sa5 has been performed for all the pixels i constituting the left image based on whether i is equal to or less than Imax. If i is equal to or less than Imax, the process proceeds to step Sa3. The above process is repeated, and if i is not less than Imax, the process ends.

The image with pixel power extracted by this is the left subject image. That is, in the left image, the color of the pixel in the subject area (that is, the left subject image) is set, but the color is set in the pixel outside the subject area.

FIG. 5 illustrates a method for calculating a direction (depth direction) perpendicular to the subject image, among the shape data generation methods of the subject based on the left subject image or the right subject image in the shape calculation unit 33. It is a flowchart to do.

Fig. 6 is a diagram showing the possible values of state S used in the flowchart. State S has a subject area and a subject area, and the subject area has a depth added and a depth not given. There are each state with grant. Pixels for which a color is set are the subject area (corresponding to the subject image), and pixels that are set and are dark are outside the subject area.

With this method, the position in the direction perpendicular to the image (depth direction) is calculated for each pixel of one of the left subject image and the right subject image (object region). Regarding the position of each pixel in the horizontal and vertical axis directions of the image (position in the stereoscopic image display space), the position of the center of gravity of the subject area is the center of gravity of the subject area in the left subject image and the right subject. It is obtained by moving to the average position of the center of gravity of the subject area in the image.

Before performing this processing in advance, the shape calculation unit 33 calculates the initial value D of the depth to be added using the method described later with reference to FIG. 8 (Sbl).

0

When the process is started, the shape calculation unit 33 sets the depth of each pixel in the subsequent step SblO. Set the initial value D to the variable D to be set (Sb2).

0

Next, the shape calculation unit 33 performs the processing of Steps Sb4 to SblO sandwiched between Step Sb3 and Step Sblla for all the pixels from i = 0 to Imax constituting the subject image (Sb3 ).

In step Sb4, the shape calculation unit 33 acquires the state S of the pixel i in the right or left image including the subject image (Sb4). In acquiring the state S, whether the pixel i is outside the subject area or the subject area (that is, the subject image) can be determined by whether or not the pixel i is set in color.

Whether the depth is given or not given is determined by whether the depth of the pixel i is given! Here, the loop from i = 0 to Imax in steps Sb3 to Sbl la is repeated many times to give different depth values through step Sb14, but between the loops giving the same value of depth. Pixels to which depth has been assigned are processed as if the depth has not been assigned during the execution of the loop (i.e., before the loop is exited and the depth value is changed and the next loop is executed before the loop is executed). Set the pixel with depth added to “Depth added”).

Next, the shape calculation unit 33 determines whether or not the state S of the pixel i acquired in step Sb4 is the subject area (Sb5). The determination in step Sb5 can be made based on whether or not a color is set for the pixel. If the color is set, it is the subject area, and if it is not set, it is outside the subject area.

If it is determined in step Sb5 that the subject area is not !, the process proceeds to step Sbl la, and if it is determined that it is the subject area because the color to be processed for the next pixel is set, the process proceeds to step Sb6. Transition.

[0023] In step Sb6, the shape calculation unit 33 determines whether or not a depth is not applied to the pixel.

(Sb6). If it is determined that the pixel has already been given a depth and is not yet assigned, the process proceeds to step Sbl la, and the process moves to the next pixel. The pixel has not yet been given a depth and has not been assigned. If it is determined that there is, the shape calculation unit 33 acquires the state Sn (n = 1 to 8) of the eight points surrounding the pixel i (Sb7).

The shape calculation unit 33 determines whether there are any pixel states Sn out of the subject area among the eight pixel states Sn. Is determined (Sb8). If the shape calculation unit 33 determines that at least one of the eight pixels is outside the subject area, the shape calculation unit 33 proceeds to step SblO and assigns the value of depth D as the depth value of pixel i (SblO). Then, the process proceeds to step Sb 11a, and the process for the next pixel is performed. If it is determined in step Sb8 that there are no objects outside the subject area in state Sn, the process proceeds to step Sb9, and the shape calculation unit 33 adds depth to state Sn. It is determined whether or not there is (Sb9).

If it is determined that there is no assigned one, the process proceeds to step Sb 11a, and the process for the next pixel is performed.

If it is determined in step Sb9 that depth has been added, the process proceeds to step SblO, and the shape calculation unit 33 assigns the value of depth D as the depth value of pixel i (SblO), Transition to step Sbl la.

In step Sbl la, the shape calculation unit 33 adds 1 to the value of i, and while i is equal to or less than Imax (step Sb l ib), returns to step Sb4, and for all pixels! /, Steps Sb4 to Sb Perform 10 steps.

When i becomes larger than Imax, the process proceeds to step Sbl2, and the shape calculation unit 33 determines whether or not there is a force that has not been given depth to the pixels in the subject area of the subject image (Sb 12). If it is determined that there are no unassigned ones, the shape calculating unit 33 terminates the process, but if it is determined that there are unassigned ones, the process proceeds to step Sbl4 and the preset addition value AD (Sbl3) is used. Add to variable D, and proceed to step Sb3 to repeat the above process. In this way, the shape calculation unit 33 can calculate the position of the depth for all the pixels in the subject area according to the distance of the outer force of the subject area.

Note that the addition value A D added to the variable D in step Sbl3 described above may be a constant or a value that varies depending on the number of additions.

FIG. 7 is a diagram for explaining a process of calculating the depth of the subject area A1 by the method of FIG.

When there is a subject area A1 shown in FIG. 7 (a), the shape calculating unit 33 first assigns a depth D to the outermost area A2 shown in FIG. 7 (b). Next, the shape calculation unit 33 performs No. 2 shown in FIG.

0

Depth D + AD is given to the outer area A3. Next, the shape calculation unit 33 performs the process shown in FIG. Depth D + 2 · AD is given to the third outer area A4 shown. Now all subject areas

0

Since the depth is given to the pixels in the area, the processing in FIG. 5 is finished.

FIG. 8 is a diagram for explaining a method of calculating the initial value D in step S1 of FIG.

0

The coordinate XL is the coordinate in the horizontal axis direction of the center of gravity of the subject area Ml (corresponding to the left subject image) extracted from the left image G1 by the shape calculation unit 33 described in FIG. 4, and the left end of the left image G1 Is the origin.

The position of the center of gravity is obtained by averaging the coordinates of all the pixels in the subject area Ml.

The coordinate XR is the coordinate in the horizontal axis direction of the center of gravity of the subject area M2 (corresponding to the right subject image) extracted from the right image G2 by the shape calculation unit 33 described in FIG. 4, and the left end of the right image G2 is The origin.

In the direction perpendicular to the image, the stereoscopic image display device 400 displays the stereoscopic video displayed on the user's viewpoint as the origin, and the shape calculation unit 33 performs the direction perpendicular to the image. The coordinate Z, that is, the initial value D of the given depth, is calculated using equation (1).

0

D = 1 / (XL-XR)… ')

ο

[0026] For example, the X coordinate XL = 80 and the Y coordinate YL = 42 of the center of gravity of the subject region Ml in the left image calculated by the shape calculation unit 33, and the X coordinate XR = of the center of gravity of the subject region M2 in the right image When 50 and the Y coordinate YR = 40, the shape calculation unit 33 calculates the Z coordinate in the stereoscopic image display space, that is, the initial value D of the assigned depth, using Equation (2).

0

D = 1 / (XL-XR) = 1 / (80-50) = 0.033 (2)

0 Here, the value of the Z coordinate is very small compared to the value of the X coordinate and the Y coordinate, but this is the value of the Z coordinate obtained by equation (2) X coordinate, Y coordinate This is because the scale is different from, and this is adjusted by multiplying the Z coordinate by a predetermined constant C. Further, the size of the predetermined constant C may be adjusted so as to emphasize the position in the Z-axis direction.

[0027] Further, the pixel I is located in the outermost region of the subject region Ml and has an X coordinate X = 78 and a Y coordinate Y = 40. The position of the 3D image in the display space is calculated as follows.

Since the depth is the outermost region, D = 0.033.

0

The average (Xm, Ym) of the center of gravity of subject area Ml and the center of gravity of subject area M2 is

Xm = (XL + XR) / 2 = (80 + 50) / 2 = 65

Ym = (YL + YR) / 2 = (42 + 40) / 2 = 41.

The movement from (XL, YL) to (Xm, Ym) is Xm—XL = 65—80 = —15 in the X axis direction and Ym—YL = 41—42 = —1 in the Y axis direction.

When (X = 78, Y = 40) is translated from (XL, YL) to (Xm, Ym), the X coordinate force S 78 = — 15 = 63, Y coordinate force 40— 1 = 39 It becomes.

Accordingly, the position of the pixel I in the stereoscopic image display space is (63, 39, 0.03).

Accordingly, the stereoscopic video composition apparatus 300 according to the present embodiment calculates and outputs shape data that matches the movement of the live-action stereoscopic video, and displays the real-time stereoscopic video on the stereoscopic video display device 400. At the same time, it is possible to realize the tactile sensation requested by anyone intuitively by using the haptic haptic presentation device 500 that receives shape data.

With the addition of the tactile sensation that was only seen as a three-dimensional object in conventional live-action three-dimensional images, it is possible to grasp the three-dimensional object more reliably and expand the possibilities of new media and interfaces.

In addition, since the system using the 3D image synthesizing apparatus 300 of the present invention has a simple configuration, it can be easily used regardless of the installation conditions, and therefore it can be used particularly effectively in fields such as education and manual presentation. Can do.

In the present embodiment, as a method for generating a subject image in the shape calculation unit 33, pixels prepared in different colors are compared between the left background image and the right background image prepared in advance and the left image and the right image. However, instead of the background image, a chroma process that extracts pixels different from a predetermined color set in advance may be used. As a result, the background image There is no need to shoot or to make the positional relationship between the left image capturing device 100 and the right image capturing device 200 and the background constant, but the background needs to be a predetermined color when shooting.

In the present embodiment, the depth is set according to the distance from the outline of the subject area as a method for calculating the shape data of the subject, but based on the parallax between the left subject image and the right subject image. The shape data may be calculated by stereo measurement for calculating the depth (JP-A-8-254416, JP-A-11-94527, JP-A-2001-241928, etc.).

At this time, as the method for generating the subject image in the shape calculation unit 33, the above-described chroma processing may be used. Thus, although the amount of calculation for calculating the shape data is larger than setting the depth according to the distance from the outline of the subject area, it is possible to calculate shape data close to the actual shape.

In the present embodiment, the shape calculation unit 33 generates a left subject image and a right subject image obtained by extracting a subject from the left image and the right image, and based on the generated left subject image and right subject image, Although the shape data is calculated, the stereo measurement process is performed based on the parallax between the left and right images without extracting the subject from the left and right images. Shape data may be calculated.

This increases the amount of computation, but it is necessary to capture a background image in advance, or to maintain a constant positional relationship between the left image capturing device 100 and the right image capturing device 200, and the background. The shape data close to the actual shape can be calculated.

[0032] Also, it is assumed that an input device, a display device, and the like (not shown) are connected as peripheral devices to the stereoscopic video composition device 300.

Here, the input device refers to an input device such as a keyboard and a mouse. A display device is a CRT (Cathode Ray Tube) or a liquid crystal display device.

[0033] Also, in order to realize the functions of the left video data input unit 31, the right video data input unit 32, the shape calculation unit 33, the shape output unit 34, the stereoscopic video composition unit 35, and the stereoscopic video output unit 36 in FIG. Is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute The processing of the image data input unit 31, the right video data input unit 32, the shape calculation unit 33, the shape output unit 34, the stereoscopic video synthesis unit 35, and the stereoscopic video output unit 36 may be performed.

The “computer system” here includes the OS and hardware such as peripheral devices.

[0034] In addition, the "computer system" includes a home page providing environment (or a display environment) if a WWW system is used.

The “computer-readable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system.

Furthermore, a “computer-readable recording medium” is a program that dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, it is assumed that the server and the client in that case, such as the volatile memory inside the computer system, hold the program for a certain period of time and then include it.

The above program may be for realizing a part of the above-described functions. Furthermore, the above-described function may be realized in combination with a program already recorded in a computer system. good.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope not departing from the gist of the present invention. It is.

Industrial applicability

[0036] The stereoscopic video composition device of the present invention has a simple configuration and does not select installation conditions, and is therefore not limited to this force suitable for use in education or manual presentation.

Claims

The scope of the claims

[1] In a stereoscopic video composition device that synthesizes a stereoscopic video from a left image viewed from the left eye viewpoint and a right image viewed from the right eye viewpoint,

A shape calculation unit that calculates shape data of a subject from the left image and the right image, and a shape output unit that outputs the shape data calculated by the shape calculation unit to a haptic display device.

A stereoscopic image synthesizing apparatus comprising:

[2] The shape calculation unit extracts an image of a specific subject from each of the left image and the right image, and for each pixel of the extracted subject image, according to a distance from the outline of the subject image A vertical coordinate is given to the image to generate the shape data of the subject, and the position of the specific subject in the display space of the stereoscopic video synthesized by the device is determined as the position of the extracted image of the specific subject. The 3D image synthesizing apparatus according to claim 1, wherein the 3D image synthesizing device is calculated based on parallax and calculates shape data in which the generated shape data is arranged at the calculated position.

[3] The shape calculation unit extracts an image of a specific subject from each of the left image and the right image, and calculates shape data of the subject by stereo measurement based on the parallax of the extracted image. The stereoscopic video composition apparatus according to claim 1.

[4] The stereoscopic image synthesizing device according to claim 1, wherein the shape calculating unit calculates shape data of a subject by stereo measurement based on a parallax between the left image and the right image. .

[5] In a shape data generation method in a stereoscopic video composition device that synthesizes a stereoscopic video from a left image viewed from the left eye viewpoint and a right image viewed from the right eye viewpoint,

A first process in which the stereoscopic video composition device calculates shape data of a subject from the left image and the right image;

A second process in which the stereoscopic image synthesizer outputs the shape data calculated in the first process to the haptic sense presentation apparatus;

A shape data generation method comprising:

[6] Stand up the computer from the left image seen from the left eye viewpoint and the right image seen from the right eye viewpoint. In a program for causing a computer to function as a stereoscopic video synthesis device that synthesizes body video,

A shape calculation unit for calculating shape data of a subject from the left image and the right image, and a shape output unit for outputting the shape data calculated by the shape calculation unit to a haptic display device

Program that also functions as