JP5633345B2 - Image processing apparatus, image display system, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image display system, and an image processing program.

  A technique for synthesizing and drawing a CG (Computer Graphics) image of a three-dimensional object with an input still image or moving image is known. For example, an image that captures the surroundings of a vehicle is converted into an overhead image viewed from an arbitrary viewpoint such as the rear of the vehicle, and a CG image of the vehicle viewed from the same viewpoint is synthesized on the overhead image. There is a technology called "system". There is also a technique for performing transparency processing on a CG image when the CG image is combined with the input image.

JP-A-9-50541

Seiya Shimizu, 2 others, “Surrounding Stereo Monitor System”, magazine FUJITSU, Fujitsu Limited, September 2009, VOL. 60, NO. 5, p. 496-501

  By performing the transmission process on the CG image to be synthesized as described above, a background image overlapping with the CG image can be visually recognized together with the CG image. However, since the color value and the gray value of the background image that overlaps with the CG image that has been subjected to the transmission processing change from the original color value and the gray value, the visibility of the background image in the overlapping area with the CG image is It was not necessarily good.

  The present invention has been made in view of such problems, and provides an image processing apparatus, an image display system, and an image processing program capable of improving the visibility of a background image overlapping with an image to be synthesized. With the goal.

  In order to solve the above problems, an image processing apparatus having an image output unit and an image composition unit is provided. The image output unit displays a first image indicating a surface shape of the second three-dimensional object obtained by dividing a part of the first three-dimensional object and dividing the first three-dimensional object into a plurality of parts. Output. The image combining unit combines the first image output from the image output unit with the second image in a semi-transparent state.

In order to solve the above problems, an image display system including the image processing apparatus is provided.
Furthermore, in order to solve the above-described problems, an image processing program that causes a computer to execute the same processing as that of the above-described image processing apparatus is provided.

  According to the image processing apparatus, the image display system, and the image processing program described above, the visibility of the background image overlapping the second image is improved.

It is a figure which shows the structural example of the image processing apparatus which concerns on 1st Embodiment, and its operation | movement. FIG. 2 is a diagram illustrating an example of a three-dimensional object and an image. It is a figure which shows the process in the case of synthesize | combining a 1st three-dimensional object, without dividing | segmenting. It is a figure which shows an example of the process which divides | segments a 1st three-dimensional object. It is a figure explaining the easiness of recognition about the shape of a three-dimensional object. It is a figure which shows the structural example of the image display system which concerns on 2nd Embodiment. It is a figure which shows the example of a division | segmentation parameter. It is a flowchart which shows the process of the image processing apparatus which concerns on 2nd Embodiment. It is a figure which shows the structural example of the image display system which concerns on 3rd Embodiment. It is a figure which shows the structural example of the image display system which concerns on 4th Embodiment. It is a flowchart which shows the process of the image processing apparatus which concerns on 4th Embodiment. It is a figure which shows the structural example of the image display system which concerns on 5th Embodiment. It is a flowchart which shows the process of the image processing apparatus which concerns on 5th Embodiment. It is a figure which shows the hardware structural example of a computer.

Hereinafter, embodiments will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example of an image processing apparatus according to the first embodiment and an operation thereof. An image processing apparatus 1 shown in FIG. 1 includes an image output unit 2 and an image composition unit 3.

  The image output unit 2 outputs an image indicating the surface shape of the second three-dimensional object obtained by deforming the first three-dimensional object to the image composition unit 3. The first three-dimensional object has at least the surface shape defined by the three-dimensional shape data. The second three-dimensional object is obtained by dividing the first three-dimensional object into a plurality of parts by removing a part of the first three-dimensional object.

  Note that the image output unit 2 calculates the shape data of the second three-dimensional object based on the shape data of the first three-dimensional object, for example, thereby displaying the surface shape of the second three-dimensional object. May be output. Alternatively, the image output unit 2 outputs an image indicating the surface shape of the second three-dimensional object based on the shape data of the second three-dimensional object prepared in advance in a storage unit (not shown). May be.

  The image output from the image output unit 2 is an image in which only the surface shape of the second three-dimensional object appears regardless of how the shape data of the first or second three-dimensional object is defined. It is said. For example, even if the first three-dimensional object is a three-dimensional object that is closed to the outside, such as a cube, the output image from the image output unit 2 is an outer surface that can be recognized when the three-dimensional object is viewed from the outside. Only the shape of appears, and the internal state of the three-dimensional object does not appear.

  The image synthesis unit 3 synthesizes the image output from the image output unit 2 with another image in a semi-transparent state, and outputs a synthesized image. The other image is, for example, a background image that is the background of the image output from the image output unit 2.

FIG. 2 is a diagram illustrating an example of a three-dimensional object and an image.
Objects 11 and 12 shown in FIG. 2 are examples of the first three-dimensional object and the second three-dimensional object, respectively. The object 11 has a shape in which two flat plates 11a and 11b having different directions are combined. The object 12 has a shape divided into three parts 12a, 12b, and 12c by removing two portions of the object 11.

  On the other hand, the background image 13 is an example of an image to be combined with an image (hereinafter referred to as “surface image”) indicating the surface shape of the object 12 in the image combining unit 3. In the background image 13, a background object 13a is drawn.

  The synthesized image 14 is an image obtained by synthesizing the surface image of the object 12 with the background image 13 in a translucent state by the image synthesizing unit 3. In the composite image 14, the surface image of the object 12 and the background image 13 are combined in a semi-transparent state, so that the surface of the object 12 is overlapped with the surface image of the parts 12a to 12c of the object 12 and the background object 13a. The shapes of both the parts 12a to 12c and the background object 13a can be visually recognized. In addition to this, in the synthesized image 14, the part 12 a and the part 12 b of the object 12 are separated from each other, and the part 12 b and the part 12 c are separated from each other, and the background image 13 including the background object 13 a is separated in these separated regions. The image is drawn directly without being subjected to a transparent process.

  Note that, when the image composition unit 3 composes the surface image of the object 12 with the background image 13, the image composition unit 3 also performs a transmission process on a region where elements constituting the surface of the object 12 overlap. For example, in a region where the near side surface corresponding to the flat plate 11a and the far side surface corresponding to the flat plate 11b overlap in the surface of the object 12, the near side surface and the far side surface are translucent. Is synthesized. However, such a transmission process may be performed by the image output unit 2 before the surface image of the object 12 is input to the image composition unit 3.

Here, FIG. 3 is a diagram illustrating processing when the first three-dimensional object is synthesized without being divided.
When the object 11 is synthesized in a semi-transparent state with respect to the background image 13 without being divided into a plurality of parts, a synthesized image 15 shown in FIG. 3 is generated. In the composite image 15, for example, in the overlapping region between the object 11 and the background object 13 a on the background image 13, not only the object 11 but also the shape of the background object 13 a can be visually recognized. However, in this overlapping region, the color value and the light and shade value of the background object 13a become different from the original color value and the light and shade value of the background object 13a due to the transmission process. For this reason, depending on the combination of the brightness, color, shape, etc. of the objects to be combined, the background object may be difficult to visually recognize.

  On the other hand, in the synthesized image 14 of FIG. 2, the part 12a and the part 12b of the object 12 and the part 12b and the part 12c are separated from each other. Further, since the object 12 has a shape in which the object 11 is divided into a plurality of parts, a region where the part and the part are separated not only on the front side but also on the back side of the surface of the object 12. There will be more to exist. For this reason, in these separated regions, there is a high possibility that the background image 13 is exposed without overlapping the surface of the object 12.

  If the background image 13 is not overlapped with the surface of the object 12 in the separated area, the background image 13 is drawn as it is without changing the original color value or gray value. Therefore, compared with the composite image 15 of FIG. 3, the composite image 14 of FIG. 2 has higher visibility of the state of the background image 13, and the shape of the object such as the background object 13a included in the background image 13 Colors can be recognized more accurately.

  The number of regions to be removed from the first three-dimensional object, that is, the number of parts of the second three-dimensional object generated by dividing the first three-dimensional object is more than one. However, the space between the parts of the second three-dimensional object becomes wider, and the visibility of the background object is improved. However, it cannot be said that the visibility of the background object is necessarily improved as the number of areas to be removed from the first three-dimensional object is larger or the removal area is larger. The number and size of regions to be removed from the first three-dimensional object depend on the shape of the first three-dimensional object, the size when the second three-dimensional object is synthesized by the image synthesis unit 3, and the like. It is desirable to set appropriately.

  By the way, in the processing in the image processing apparatus 1 described above, the second three-dimensional object obtained by removing a part of the first three-dimensional object is synthesized and drawn to be removed from the first three-dimensional object. Based on the shape of the boundary of the formed region, an effect that the surface shape of the first three-dimensional object can be easily recognized is also produced. Such an effect is achieved by making the cut surface of the region to be removed from the first three-dimensional object along a predetermined plane in the three-dimensional coordinate space in which the shape data of the first three-dimensional object is defined. , Become more prominent. When the first three-dimensional object is cut along a plane and divided, for example, if the surface is flat like the object 11 in FIG. 2, the removed boundary is a straight line, while the surface is a curved surface. If so, the removed boundary is a curve. Therefore, it becomes easy to recognize the shape of the first three-dimensional object.

  Further, as in the case where the object 11 of FIG. 2 is divided like the object 12, the first three-dimensional object is divided along a plurality of planes parallel to each other, so that the shape of the first three-dimensional object is changed. It becomes possible to recognize more accurately.

FIG. 4 is a diagram illustrating an example of a process for dividing the first three-dimensional object.
In the process of dividing the first three-dimensional object along mutually parallel planes, first, an arbitrary reference plane in the three-dimensional coordinate space in which the shape data of the first three-dimensional object is defined is selected. Then, the first three-dimensional object is cut along a plurality of division planes parallel to the reference plane 20, and the stripe-like region sandwiched between the two division planes adjacent to each other is removed.

  FIG. 4 shows an example in which the object 11 shown in FIG. 2 is divided along the dividing surfaces 21 to 24 parallel to the reference plane 20. The dashed-dotted line represented by the object 11 in FIG. 4 indicates a cutting line when being cut by the dividing surfaces 21 to 24. 2 is removed from the object 11 by removing the stripe-shaped region between the dividing surface 21 and the dividing surface 22 and the stripe-shaped region between the dividing surface 23 and the dividing surface 24, respectively. can get.

FIG. 5 is a diagram illustrating the ease of recognizing the shape of a three-dimensional object.
In FIG. 5, the cutting lines cut by the dividing surfaces 21 to 24 illustrated in FIG. 4 are indicated by thick lines on the object 12. By dividing the object 11 along the plane, for example, a cutting line appears on a smooth surface of the surface of the original object 11, and the surface shape of the original object 11 can be easily grasped by the cutting line. Become.

  In particular, when the object 11 is cut along a plurality of planes parallel to each other like the object 12 and the striped region between adjacent planes is removed, the surface of the original object 11 is flat. The cutting lines at both ends of the stripe region are parallel straight lines. Then, the surface shape of the original object 11 can be easily grasped by observing the cutting line with reference to the case where the surface of the original object 11 is a flat surface.

  In addition, the reference plane 20 serving as a reference for the divided surfaces 21 to 24 is desirably a horizontal plane (a plane horizontal to the ground plane) in a virtual three-dimensional coordinate space in which the object 12 exists in the synthesized image 14. The virtual three-dimensional coordinate space referred to here is, for example, a virtual image captured by the virtual camera when the background image 13 that is the synthesis target of the object 12 is considered captured by the virtual camera. Refers to a three-dimensional coordinate space. When the image of the object 12 is combined with the background image 13, the object 12 is projected onto this virtual three-dimensional coordinate space.

  When a human recognizes an object, it is easier to recognize the shape of the object when considering a plane parallel to the ground on which he is standing. Therefore, it becomes easy to recognize the surface shape of the original object 11 by setting the reference plane 20 as a horizontal plane in the virtual three-dimensional coordinate space.

  Further, by removing a plurality of stripe regions from the original object 11, the number of cutting lines appearing on the surface of the object 12 increases, so that the surface shape of the original object 11 is changed from the shape of each cutting line on the object 12. Can be grasped more accurately.

  In the first embodiment described above, the second three-dimensional object is generated by dividing the first three-dimensional object along a division plane parallel to one reference plane. However, as another example, The first three-dimensional object may be divided along dividing planes parallel to the plurality of reference planes. For example, the first three-dimensional object is divided along a plurality of division planes parallel to a certain reference plane, and the divided three-dimensional object is further divided along a plurality of division planes parallel to another reference plane. Thus, a second three-dimensional object may be generated.

[Second Embodiment]
FIG. 6 is a diagram illustrating a configuration example of an image display system according to the second embodiment. The image display system illustrated in FIG. 6 includes an imaging device 110, an image processing device 200, and a display device 120.

  The imaging device 110 captures an image using an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) sensor, and outputs the captured image to the image processing device 200 as an analog signal. The image processing device 200 synthesizes a virtual three-dimensional object image with the image captured by the imaging device 110. The display device 120 displays the image synthesized by the image processing device 200. The display device 120 is, for example, an LCD (Liquid Crystal Display), an organic EL (EelectroLuminescence) display, or the like. Note that the image processing device 200 and the display device 120 may be integrated.

  The image processing apparatus 200 includes an image input unit 210, an image synthesis unit 220, a division processing unit 230, a data conversion unit 240, and an image output unit 250. The image composition unit 220 corresponds to the image composition unit 3 in the first embodiment, and the division processing unit 230 and the data conversion unit 240 correspond to the image output unit 2 in the first embodiment.

  Further, the image processing apparatus 200 includes a storage unit 260. The storage unit 260 is, for example, a nonvolatile storage medium such as a flash memory or an HDD (Hard Disk Drive). The storage unit 260 stores a shape data DB (database) 261, a division parameter 262, a camera parameter 263, an object display parameter 264, and a composition parameter 265.

The image input unit 210 digitizes the analog image signal output from the imaging device 110 and outputs the digitized image signal to the image composition unit 220.
In the synthesis parameter 265, various parameters for synthesizing the image of the three-dimensional object based on the output data from the data conversion unit 240 by the image synthesis unit 220 are set. The synthesis parameter 265 includes, for example, the display color and transmittance of a three-dimensional object. Note that at least some of the parameters included in the synthesis parameter 265 may be arbitrarily set according to a user setting operation or the like.

  The image composition unit 220 applies the display color in the composition parameter 265 to the surface of the three-dimensional object based on the output data from the data conversion unit 240, and outputs the image of the three-dimensional object as an output image from the image input unit 210. On the other hand, the images are synthesized in a semi-transparent state based on the transmittance in the synthesis parameter 265.

  In the shape data DB 261, shape data defining the three-dimensional shape of the surface of the three-dimensional object is registered. The three-dimensional object based on the shape data in the shape data DB 261 corresponds to the first three-dimensional object in the first embodiment. The shape data in the shape data DB 261 is, for example, a set of polygon patches in which the three-dimensional coordinates of each vertex are defined with reference to predetermined reference coordinates. Alternatively, for example, when the three-dimensional object has a simple geometric shape such as a rectangular parallelepiped, the shape data may be the length of each side of the three-dimensional object.

  The division parameter 262 is a parameter that determines how the three-dimensional object is divided in the division processing unit 230. As will be described later, the division parameter 262 is a position parameter with respect to the normal direction of the reference plane with respect to a predetermined reference plane, for example. The division parameter 262 may be arbitrarily set according to a setting operation by the user.

  The division processing unit 230 divides the three-dimensional object based on the shape data in the shape data DB 261 into a plurality of parts based on the division parameter 262 and outputs the divided three-dimensional object shape data to the data conversion unit 240. Note that the three-dimensional object divided by the division processing unit 230 corresponds to the second three-dimensional object in the first embodiment.

  In the present embodiment, the division processing unit 230 removes a plurality of stripe regions along a plurality of parallel planes from the three-dimensional object based on the shape data in the shape data DB 261, thereby obtaining the original three-dimensional Divide the object. This division process corresponds to the process of dividing the object 11 shown in FIG. Hereinafter, the three-dimensional object divided by the division processing unit 230 is referred to as a “striped object”. The division processing unit 230 calculates the shape data of the striped object based on the shape data in the shape data DB 261 and the division parameter 262, and outputs the calculation result of the shape data to the data conversion unit 240.

  The camera parameter 263 is a virtual camera parameter when a striped object is projected onto a virtual three-dimensional coordinate space. The camera parameters 263 include, for example, parameters such as the focal length, viewpoint position, and line-of-sight direction of the virtual camera.

  The object display parameter 264 is a parameter that determines how a striped object is displayed on the composite image. The object display parameter 264 includes, for example, parameters such as the position and rotation direction of the striped object when the striped object is projected onto the virtual three-dimensional coordinate space.

Note that at least some of the parameters included in the camera parameter 263 and the object display parameter 264 may be arbitrarily set according to a user setting operation or the like.
The data conversion unit 240 converts the shape data of the striped object output from the division processing unit 230 based on the camera parameter 263 and the object display parameter 264. By the conversion process in the data conversion unit 240, the three-dimensional coordinates of each vertex of the polygonal patch defining the striped object are converted into coordinates in the two-dimensional coordinate space in which the output image from the image input unit 210 is defined. The

  The image output unit 250 converts the digital data of the composite image output from the image composition unit 220 into an image signal for display and outputs it to the display device 120, and causes the display device 120 to display the composite image.

Next, a processing example of the division processing unit 230 will be described.
The division processing unit 230 divides a three-dimensional object based on shape data in the shape data DB 261 along a plurality of division planes parallel to a reference plane arbitrarily set on a three-dimensional coordinate space in which shape data is defined. . The reference plane is, for example, a plane along the two-axis direction in the three-dimensional coordinate space in which shape data is defined. The division processing unit 230 generates a striped object by removing a striped region sandwiched between adjacent divided surfaces from the three-dimensional object.

If the three-dimensional coordinate of an arbitrary point on the reference plane is PB and the vector indicating the normal direction of the reference plane is n, the position L on the three-dimensional straight line passing through the point of the coordinate PB and along the normal direction is It is expressed by the following equation (1) using the parameter t.
L = n * t + PB (1)
In Expression (1), t is a position parameter indicating an offset amount from the reference plane with respect to the normal direction of the reference plane. Then, the position parameter for each of the divided surfaces is given as the value of t in Equation (1). That is, when the value of t is determined, the position on the three-dimensional coordinate of the target split surface is determined. The dividing plane passes through the three-dimensional coordinate point given by the equation (1) and becomes a plane along the normal direction (the direction of the vector n). Such a dividing plane is given by a plane expression such as the following expression (2). In Expression (2), n ^T represents a transposed matrix of the vector n.
n ^T * (x−L) = 0 (2)
In the storage unit 260, as an example of the division parameter 262, the value of t in Expression (1) for each division plane is set. For example, when a three-dimensional object based on shape data in the shape data DB 261 is divided along m (where m is an even number equal to or greater than 2) division surfaces, the division parameters 262 are t1, t2,. It includes m position parameters tm.

  The division processing unit 230 obtains an equation for each division plane by substituting the position parameter corresponding to each division plane included in the division parameter 262 into the equation (2). The division processing unit 230 removes an area between the odd-numbered divided surface and the next even-numbered divided surface from the three-dimensional object based on the shape data in the shape data DB 261. The shape data is calculated based on the shape data in the shape data DB 261 and the expression of each divided surface.

FIG. 7 is a diagram illustrating an example of the division parameter.
An object plane 301 shown in FIG. 7 is a view of the surface of a three-dimensional object based on the shape data in the shape data DB 261 as seen from a direction parallel to the reference plane. FIG. 7 shows an example in which a three-dimensional object based on shape data in the shape data DB 261 is divided by six dividing surfaces 311 to 316 that are parallel to the reference plane. In this case, position parameters t1 to t6 respectively corresponding to the dividing surfaces 311 to 316 are set in the storage unit 260 as the dividing parameters 262. Then, in the object plane 301, an area between the dividing plane 311 and the dividing plane 312, an area between the dividing plane 313 and the dividing plane 314, and an area between the dividing plane 315 and the dividing plane 316 are removed. Thus, a striped object divided so as to have four striped object surfaces 302a to 302d is obtained.

Next, a processing example of the data conversion unit 240 will be described.
Assume that the camera parameter 263 is set with the focal length fc, the viewpoint position Pc, and the line-of-sight direction Nc, and the object display parameter 264 is set with the object position Pm and the rotation direction Rm for the striped object. If the three-dimensional coordinates of each vertex of the polygon patch constituting the surface of the striped object is q_i (where i is a subscript for identifying the vertex of the polygon patch), each vertex of the polygon patch is expressed by the following formula: Projected into a virtual three-dimensional coordinate space using (3). In Expression (3), r_i indicates the coordinates of the projection position of q_i in the virtual three-dimensional coordinate space.
r_i = Rm * q_i + Pm (3)
The striped object projected based on Expression (3) is projected onto a two-dimensional space in a virtual captured image when observed with a virtual camera having the focal position fc and the center of the imaging position at the viewpoint position Pc. Although the two-dimensional position in the virtual captured image depends on the model of the lens of the virtual camera, here, an example in which a camera with perspective projection corresponding to a normal lens is assumed is shown. The coordinates s_i in the virtual captured image are obtained by the following expressions (4) and (5).
s_i (x) = fc_x * r_i (x) / r_i (z) (4)
s_i (y) = fc_y * r_i (y) / r_i (z) (5)
However, s_i (x) and s_i (y) respectively indicate the x coordinate and the y coordinate of the coordinate s_i, fc_x and fc_y respectively indicate the x component and the y component of the focal length fc, and r_i (x) and r_i ( y) and r_i (z) indicate the x coordinate, y coordinate, and z coordinate of the coordinate r_i, respectively.

  Based on the focal length fc, the viewpoint position Pc, and the line-of-sight direction Nc in the camera parameter 263, and the object position Pm and the rotation direction Rm in the object display parameter 264, the data conversion unit 240 uses the expressions (4) and (5). Then, the coordinates of each vertex of the polygonal patch defining the striped object are converted, and the coordinates of each converted vertex are output to the image composition unit 220. Each vertex of the polygonal patch is converted into a two-dimensional coordinate on the virtual captured image by coordinate conversion using the above equations (4) and (5).

  In the above description of FIGS. 6 and 7, the camera parameter 263 and the object display parameter 264 are set in advance in the storage unit 260, but these parameters are set inside or outside the image processing apparatus 200. It may be changed in conjunction with some processing in. For example, the image input unit 210 is provided with a function of generating an image obtained by capturing a three-dimensional space from a virtual camera installed at an arbitrary viewpoint position based on captured images from one or a plurality of imaging devices. In this case, the viewpoint position and the line-of-sight direction of the virtual camera can be used for the conversion process in the data conversion unit 240 as the viewpoint position Pc and the line-of-sight direction Nc.

  Next, processing in the image processing apparatus 200 will be described using a flowchart. FIG. 8 is a flowchart showing processing of the image processing apparatus according to the second embodiment. The flowchart of FIG. 8 shows processing when a composite image for one frame is output.

  [Step S11] The division processing unit 230 substitutes the position parameter corresponding to each division plane, which is included in the division parameter 262 set in the storage unit 260, into the above equation (2). Thereby, the division | segmentation process part 230 calculates | requires the formula of each division surface.

  [Step S12] The division processing unit 230 converts the odd-numbered divided surface and the next even-numbered divided surface among the divided surfaces obtained in Step S11 from the three-dimensional object based on the shape data in the shape data DB 261. By removing the sandwiched area, the shape data of the striped object is calculated. For example, the division processing unit 230 temporarily records the calculated shape data of the striped object in a RAM (not shown) included in the image processing apparatus 200.

  [Step S13] The data converter 240 uses the shape data of the striped object calculated in step S12 to determine the shape of one polygonal patch that defines the surface shape of the striped object (the coordinates of the vertexes of the polygonal patch). Select.

  [Step S14] Based on the camera parameter 263 and the object display parameter 264 set in the storage unit 260, the data conversion unit 240 converts the three-dimensional coordinates of each vertex of the polygon patch selected in Step S13 to the above equation (4). ) And Equation (5), the image is converted into two-dimensional coordinates in the virtual captured image. The data conversion unit 240 temporarily stores the obtained two-dimensional coordinates of each vertex in a RAM included in the image processing apparatus 200.

  [Step S15] The data conversion unit 240 determines whether data of all polygon patches included in the shape data of the striped object calculated in step S12 has been selected. If there is unselected polygon patch data, the data conversion unit 240 returns to step S13 and selects one polygon patch data from the unselected polygon patch data.

  On the other hand, when the data of all the polygon patches has been selected, the surface image data of the striped object is generated as a set of two-dimensional coordinates of each polygon patch. If all the polygon patch data has been selected, the data conversion unit 240 executes the process of step S16.

  [Step S <b> 16] The image composition unit 220 extracts the display color and transmittance of the striped object from the composition parameter 265 set in the storage unit 260. The image composition unit 220 synthesizes the image of the striped object with the output image from the image input unit 210 in a semi-transparent state based on the transmittance extracted from the composition parameter 265.

  Specifically, the image composition unit 220 uses the display color extracted from the composition parameter 265 as the pixel value of each pixel included in each polygon patch based on the two-dimensional coordinates of each polygon patch converted in step S14. Apply. The image synthesis unit 220 synthesizes the pixel value of each polygon patch and the pixel value of the output image from the image input unit 210 by a predetermined blending method to which the transmittance extracted from the synthesis parameter 265 is applied.

  Note that in a region where polygon patches overlap each other, transparency processing is also performed between overlapping polygon patches. For example, the image synthesizing unit 220 generates image data of a striped object by synthesizing pixel values of each polygon patch in a semi-transparent state in a region where a plurality of polygon patches overlap. Thereafter, the image composition unit 220 synthesizes the generated image data of the striped object and the data of the output image from the image input unit 210 in a translucent state.

  In the processing example of FIG. 8 described above, the color of the surface of the striped object is a single color determined by the display color in the synthesis parameter 265. In this case, it is possible to change the overall color of the striped object by changing the display color set in the synthesis parameter 265. However, as another example, the color of the surface of the striped object may be registered in the shape data DB 261 in advance. In this case, for example, the color information is individually added to the shape data of each of the polygon patches registered in the shape data DB 261, so that the color of the surface of the striped object varies depending on the location. it can.

  In the second embodiment described above, the original three-dimensional object is transformed into a striped object, and the striped object is combined with the output image from the image input unit 210 in a translucent state. This improves the visibility of the state of the background image that overlaps with the image of the striped object, and makes it easier to grasp the surface shape of the original three-dimensional object.

[Third Embodiment]
FIG. 9 is a diagram illustrating a configuration example of an image display system according to the third embodiment. In FIG. 9, components corresponding to those in FIG. 6 are denoted by the same reference numerals.

  The image display system according to the third embodiment is obtained by modifying the image processing apparatus 200 in the second embodiment shown in FIG. 6 into a configuration like the image processing apparatus 200a in FIG. The image processing apparatus 200a does not include the division processing unit 230 illustrated in FIG. Instead, the data conversion unit 240 of the image processing apparatus 200 a reads the shape data of the striped object from the shape data DB 261 a stored in the storage unit 260. In the shape data DB 261a, the same shape data as that obtained by converting the shape data registered in the shape data DB 261 in FIG. 6 by the division processing unit 230 in FIG. 6 is registered.

  In the above third embodiment, in addition to the effects obtained in the second embodiment, the calculation load for calculating the shape data of the striped object is reduced, and the circuit scale of the image processing apparatus 200a is reduced. Moreover, the effect that power consumption can be reduced is also obtained.

[Fourth Embodiment]
FIG. 10 is a diagram illustrating a configuration example of an image display system according to the fourth embodiment. In FIG. 10, components corresponding to those in FIG. 6 are denoted by the same reference numerals.

  The image display system according to the fourth embodiment is obtained by modifying the image processing device 200 in the second embodiment shown in FIG. 6 into a configuration like the image processing device 200b in FIG. The image processing apparatus 200b can change the deformation method when deforming a three-dimensional object based on the shape data in the shape data DB 261 into a striped object according to a parameter that determines how to combine the striped object. It is possible.

  The image processing apparatus 200b includes a parameter setting unit 270 and a divided parameter output unit 280 in addition to the image input unit 210, the image synthesis unit 220, the division processing unit 230, the data conversion unit 240, and the image output unit 250 illustrated in FIG. . In addition to the shape data DB 261 shown in FIG. 6, a division parameter table 266 is stored in advance in the storage unit 260 provided in the image processing apparatus 200b.

  An input device 271 is connected to the parameter setting unit 270. The parameter setting unit 270 outputs a parameter corresponding to a user input operation on the input device 271. The parameters output from the parameter setting unit 270 are parameters included in the camera parameter 263, the object display parameter 264, and the synthesis parameter 265 shown in FIG.

  The parameter setting unit 270 outputs the camera parameter and the object display parameter among the parameters output in response to the input operation to the input device 271 to the division parameter output unit 280 and the data conversion unit 240, and the synthesis parameter to the image synthesis unit 220. Output to. Note that some of these parameters may be output from the parameter setting unit 270 in response to an input operation to the input device 271, and other parameters may be stored in the storage unit 260 in advance.

  The division parameter output unit 280 outputs the division parameter corresponding to the camera parameter and the object display parameter output from the parameter setting unit 270 to the division processing unit 230. As described in the second embodiment, the division parameter is a parameter indicating the position of each of the plurality of division surfaces from the reference plane.

  The parameter setting unit 270 may receive an instruction for a parameter to be output from a processing function provided inside or outside the image processing apparatus 200b instead of the input device 271. For example, the image input unit 210 is provided with a function of generating an image obtained by capturing a three-dimensional space from a virtual camera installed at an arbitrary viewpoint position based on captured images from one or a plurality of imaging devices. In this case, the parameter setting unit 270 may receive the viewpoint position and the line-of-sight direction for the virtual camera from the image input unit 210 and output them to the division parameter output unit 280.

  The division parameter output unit 280 stripes the original three-dimensional object in a stripe shape so that the appearance of the stripe object from the virtual camera becomes appropriate according to the camera parameter and the object display parameter output from the parameter setting unit 270. Adjust how to divide. In the present embodiment, as an example, the division parameter output unit 280 removes the original three-dimensional object in a stripe shape according to the size of the stripe object to be combined with the output image from the image input unit 210. Adjust the number of areas.

  For example, when the focal length of the virtual camera is shortened (that is, when imaging is performed at a wide angle), the striped object appears small on the composite image, so the number of areas to be striped is reduced and divided. Each part should keep a certain size. On the other hand, when the focal length of the virtual camera is increased (that is, when imaging is performed with telephoto), a striped object appears large on the composite image, so the number of regions to be striped is increased.

  In this embodiment, as an example, the division parameter output unit 280 refers to the division parameter table 266 and outputs the division parameters. In the division parameter table 266, the number of divisions for dividing the original three-dimensional object and the division parameters are registered in association with each other.

Hereinafter, an example of processing of the division parameter output unit 280 will be described.
The camera parameters output by the parameter setting unit 270 are the focal length fc, the viewpoint position Pc, and the line-of-sight direction Nc, and the object display parameters output by the parameter setting unit 270 are the object position Pm and the rotation direction Rm. At this time, the length (distance W) when the distance from the virtual camera to the striped object is projected in the line-of-sight direction Nc is expressed by the following equation (6). In Expression (6), Nc ^T represents a transposed matrix in the line-of-sight direction Nc.
W = Nc ^T (Pm−Pc) / | Nc | (6)
The size E in which the striped object appears in the virtual captured image captured by the virtual camera is expressed by the following equation (7) from the geometrical relationship. In Expression (7), a representative magnitude may be set for the constant E0.
E = E0 * fc / W (7)
From equation (7), it can be seen that the magnitude E is proportional to the focal length fc and inversely proportional to the distance W. Therefore, an arithmetic expression using the focal length fc and the distance W will be used below. T_1, T_2,..., T_m (where m is the number of divided surfaces and an even number equal to or greater than 2) when the striped object is projected at a predetermined size as a reference on the captured image, T_k is defined as in the following equation (8).
t_k = T_k * G * W / fc (8)
In Expression (8), G is a constant for making the number of division surfaces at the reference distance and the reference focal length the same as T_k. At this time, the maximum number c of division parameters to be used is determined as in the following equation (9).

  Equation (9) means that the subscript (k) that gives the maximum t_k not exceeding T_m is c. By determining the number of division parameters (that is, the number of division planes) according to Equation (8) and Equation (9), the number of division planes decreases as the apparent size of the striped object in the virtual captured image decreases. The larger the size, the larger the number of dividing surfaces. More specifically, the smaller the focal length fc and the larger the distance W, the smaller the number of divided surfaces.

  FIG. 11 is a flowchart illustrating the processing of the image processing apparatus according to the fourth embodiment. The flowchart of FIG. 11 shows processing when a composite image for one frame is output. In FIG. 11, processing steps in which the same processing as in FIG. 8 is performed are denoted by the same reference numerals.

  [Step S31] Camera parameters, object display parameters, and composite parameters are set by an input operation on the input device 271. The parameter setting unit 270 outputs each parameter according to an input operation on the input device 271. The output parameters are temporarily stored in, for example, a RAM (not shown) provided in the image processing apparatus 200b.

  [Step S <b> 32] The division parameter output unit 280 uses the parameters output from the parameter setting unit 270 to perform the calculation of Expression (6). The division parameter output unit 280 substitutes the calculated distance W into the equation (8), and substitutes the focal length fc output from the parameter setting unit 270 and the constant G into the equation (8). The division parameter output unit 280 uses Equation (8) to calculate the number of division parameters (c in Equation (9)) that matches the condition of Equation (9).

  [Step S33] The division parameter output unit 280 refers to the division parameter table 266 based on the calculated number of division parameters and outputs a division parameter indicating the position of each of the plurality of division planes from the reference plane.

  In the division parameter table 266, the number of division parameters and a division parameter group including division parameters for each of a plurality of division planes are registered in association with each other. The division parameter output unit 280 reads the division parameter group associated with the number of division parameters calculated in step S32 from the division parameter table 266, and outputs it to the data conversion unit 240.

  Note that the division parameter output unit 280 may obtain a division parameter for each of a plurality of division surfaces by calculation based on the number of division parameters calculated in step S32. For example, the division parameter output unit 280 equally divides the reference value of the length of the reference plane with respect to the normal direction by the number of division parameters calculated in step S32, so that the division parameter indicating the position of each division plane is obtained. It may be calculated.

  In any of the above processes, even when the size of the striped object in the composite image becomes small, the interval between the regions to be removed from the original three-dimensional object does not become too small, and the original three-dimensional object in the composite image It becomes easy to grasp the surface shape.

  Thereafter, the processes of steps S11 to S16 shown in FIG. 8 are executed. However, in step S <b> 14, the data conversion unit 240 acquires camera parameters and object display parameters from the parameter setting unit 270. In step S <b> 16, the image composition unit 220 acquires a composition parameter from the parameter setting unit 270.

  In the fourth embodiment described above, the number of regions to be removed from the original three-dimensional object when the stripe object is generated is changed according to the size of the stripe object in the composite image. As a result, even when the striped object shown in the composite image becomes small, the removed region does not become too fine, and the surface shape of the original three-dimensional object can be easily grasped.

[Fifth Embodiment]
FIG. 12 is a diagram illustrating a configuration example of an image display system according to the fifth embodiment. In FIG. 12, the same reference numerals are given to the components corresponding to FIG.

  The image display system according to the fifth embodiment is obtained by modifying the image processing apparatus 200b in the fourth embodiment shown in FIG. 10 into a configuration like the image processing apparatus 200c in FIG. The image processing apparatus 200c includes a shape data selection unit 290 instead of including the division processing unit 230 and the division parameter output unit 280 illustrated in FIG. The storage unit 260 stores a shape data DB 261c. The shape data DB 261c includes shape data for a plurality of striped objects obtained by dividing the original three-dimensional object using different division parameters.

  The shape data selection unit 290 selects and reads the shape data of the striped object corresponding to the parameter from the shape data DB 261c in accordance with the various parameters output from the parameter setting unit 270, and the read shape data is the data conversion unit. Output to 240. In the present embodiment, the shape data selection unit 290 determines the shape data of the striped object divided into fewer parts as the striped object appears smaller in the composite image based on the focal length of the camera parameters. Select from DB 261c. This makes it easy to grasp the surface shape of the original three-dimensional object even when the striped object appears small as in the fourth embodiment.

Here, an example of the shape data stored in the shape data DB 261c will be described in detail.
In the shape data DB 261c, shape data for each of the NB stripe-like objects obtained by dividing the original three-dimensional object using different division parameters is registered. On the other hand, as in the description of the fourth embodiment, the camera parameters output by the parameter setting unit 270 are the focal length fc, the viewpoint position Pc, and the line-of-sight direction Nc, and the object display parameters output by the parameter setting unit 270 are It is assumed that the object position Pm and the rotation direction Rm. At this time, the distance W when the distance from the virtual camera to the striped object is projected in the line-of-sight direction Nc is expressed by the above-described equation (6).

Here, it is assumed that a range that each of the virtual distance W and the focal distance fc to the stripe-shaped object can take is determined in advance. The minimum and maximum values of the distance W are W_min and W_max, respectively, and the minimum and maximum values of the focal length fc are fc_min and fc_max, respectively. A list WL of distance groups in which the distance W is divided into Nw at regular intervals within the above range is defined by the following equation (10), and the focal length fc is divided into Nf at regular intervals within the above range. The group list fL is defined by the following formula (11).
WL = {W_1, W_2,..., W_Nw} (W1 = W_min, W_Nw = W_max) (10)
fL = {fc_1, fc_2,..., fc_Nf} (where fc_1 = fc_min, fc_Nf = fc_max) (11)
All combinations of the elements in the distance group list WL and the elements in the focal distance group list fL are defined by a combination list PL shown in the following equation (12).
PL = {(W_1, fc_1), (W_1, fc_2), ..., (W_1, fc_Nf), (W_2, fc_1), (W_2, fc_2), ..., (W_2, fc_Nf), ... ..., (W_Nw, fc_1), (W_Nw, fc_2), ..., (W_Nw, fc_Nf)} (12)
At this time, the number of elements in the combination list PL is (Nw * Nf), and this number corresponds to the above NB. When integer numbers starting from 1 are assigned in order from the first element of the combination list PL and the i-th element is (W_u, fc_v) (where u and v are element numbers, respectively), i is the following expression (13 ). Moreover, based on Formula (13), u and v are represented by Formula (14) and Formula (15), respectively. In addition, F1 of Formula (14) represents truncation, and F2 of Formula (15) represents a remainder.
i = u * Nw + v (13)
u = F1 (i / Nw) (14)
v = F2 (i / Nw) (15)
The combination list PL is registered in the shape data DB 261c.

  Next, the size E in which the striped object appears in the virtual captured image captured by the virtual camera is proportional to the focal length fc and inversely proportional to the distance W from the above-described equation (7). Therefore, the division parameters when the striped object is projected at a predetermined size as a reference on the captured image are T_1, T_2,..., T_m (where m is the number of division planes and 2 or more. T_k is defined as in the above equation (8).

For the i-th element (W_u, fc_v) of the combination list PL, the division parameter is defined as in the following expression (16) according to the definition of the expression (8).
t_k ⁱ = T_k * G * W_u / fc_v (16)
At this time, the maximum number c_i of the division parameters for the i-th element (W_u, fc_v) of the combination list PL is determined as in the following Expression (17).

Equation (17), means that the c_i subscript (k) to give the maximum of t_k ⁱ does not exceed the T_m.
From equation (16) and formula (17), i th element of the combination list PL (W_u, fc_v) a set tL ⁱ division parameters for, defined as the following equation (18).
tL ⁱ = {t — 1 ⁱ , t — 2 ⁱ ,..., t_c_i ⁱ } (18)
The set tL ⁱ is defined for each i-th element of the combination list PL. Using the set tL ⁱ , the division plane corresponding to each division parameter is obtained from the above equation (2). Then, along the dividing plane which is determined, by dividing the original three-dimensional object in a stripe shape, shape data H ⁱ striped object corresponding to each of the i-th element is required. The shape data ^Hi includes, for example, vertex coordinates of polygonal patches that form the surface shape of the striped object.

A set HL of striped object shape data corresponding to each element of the combination list PL is defined by the following equation (19). In the shape data DB 261a, a set HL of shape data is registered together with the combination list PL.
HL = {H ¹ , H ² ,..., H ^NB } (19)
FIG. 13 is a flowchart illustrating processing of the image processing apparatus according to the fifth embodiment. The flowchart in FIG. 13 shows processing when a composite image for one frame is output. In FIG. 13, processing steps in which the same processing as in FIG. 8 is performed are denoted by the same reference numerals.

  [Step S51] Camera parameters, object display parameters, and composition parameters are set by an input operation on the input device 271. The parameter setting unit 270 outputs each parameter according to an input operation on the input device 271. The output parameters are temporarily stored in, for example, a RAM (not shown) provided in the image processing apparatus 200b.

  [Step S52] The shape data selection unit 290 calculates the above-described equation (6) using the camera parameters and object display parameters output from the parameter setting unit 270, and calculates the distance W.

  [Step S53] The shape data selection unit 290 has values closest to the focal length fc output from the parameter setting unit 270 and the distance W calculated in step S52 from the combination list PL registered in the shape data DB 261c. The set element is searched and the element number i is recognized.

  [Step S54] The shape data selection unit 290 reads the shape data of the striped object corresponding to the element number i recognized in step S53, from the shape data set HL registered in the shape data DB 261c. The shape data selection unit 290 temporarily stores the read shape data of the striped object, for example, in a RAM included in the image processing apparatus 200b.

  Thereafter, the processes of steps S13 to S16 shown in FIG. 8 are executed. However, in step S13, the data conversion unit 240 acquires camera parameters and object display parameters from the parameter setting unit 270, and converts them using the shape data read out from the shape data DB 261c by the shape data selection unit 290 in step S54. Process. In step S <b> 16, the image composition unit 220 acquires a composition parameter from the parameter setting unit 270.

  In the fifth embodiment described above, similarly to the fourth embodiment, even when the stripe-like object shown in the composite image becomes small, the removed region does not become too fine, and the original three-dimensional object It becomes easy to grasp the surface shape. Furthermore, in the fifth embodiment, the shape data of each of the plurality of striped objects deformed from the original three-dimensional object using different division parameters is registered in advance in the shape data DB 261c. For this reason, it is not necessary to calculate the shape data of the striped object for each set camera parameter, and the calculation load on the image processing apparatus is reduced as compared with the fourth embodiment.

  The image processing apparatus or the image display system in each of the above embodiments can be applied to, for example, an omnidirectional stereoscopic monitor system (see Non-Patent Document 1). The omnidirectional stereoscopic monitor system converts an image obtained by photographing the periphery of the vehicle into an overhead image viewed from an arbitrary viewpoint such as the rear of the vehicle and synthesizes a virtual image of the vehicle viewed from the same viewpoint on the overhead image. This assists the driving operation of the driver.

  By applying the process of the image processing device in each of the above embodiments to the process of synthesizing the virtual image of the vehicle on the overhead image, for example, the object existing in the overhead image is hidden by the virtual image of the vehicle. The object can be visually recognized on the composite image, and the driver can easily recognize obstacles around the vehicle. Furthermore, the driver can easily recognize the surface shape of the virtual image of the vehicle, and the driver can more accurately grasp the positional relationship between the obstacles around the vehicle and the vehicle.

Note that the processing of the image processing apparatus in each of the above embodiments can be realized, for example, by a computer having a hardware configuration as shown in FIG.
FIG. 14 is a diagram illustrating a hardware configuration example of a computer.

  The computer 400 shown in FIG. 14 is entirely controlled by a CPU (Central Processing Unit) 401. A RAM 402 and a plurality of peripheral devices are connected to the CPU 401 via a bus 409.

  The RAM 402 is used as a main storage device of the computer 400. The RAM 402 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the CPU 401. The RAM 402 stores various data necessary for processing by the CPU 401.

  Peripheral devices connected to the bus 409 include an HDD 403, a graphic processing device 404, an input I / F (interface) 405, an optical drive device 406, a network I / F 407, and a communication I / F 408.

  The HDD 403 magnetically writes and reads data to and from the built-in magnetic disk. The HDD 403 is used as a secondary storage device of the computer 400. The HDD 403 stores an OS program, application programs, and various data. Note that a semiconductor storage device such as a flash memory can also be used as the secondary storage device.

  A monitor 404 a is connected to the graphic processing device 404. The graphic processing device 404 displays an image on the monitor 404a in accordance with a command from the CPU 401. The monitor 404a is, for example, an LCD. The monitor 404a may be integrated with the main body of the computer 400.

  A keyboard 405a and a mouse 405b are connected to the input I / F 405. The input I / F 405 transmits output signals from the keyboard 405a and the mouse 405b to the CPU 401. Note that the mouse 405b is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball. An input device such as the keyboard 405a may be integrated with the main body of the computer 400.

  The optical drive device 406 reads data recorded on the optical disk 406a using a laser beam or the like. The optical disk 406a is a portable recording medium on which data is recorded so that it can be read by reflection of light. The optical disk 406a includes a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (Rewritable), and the like.

The network I / F 407 is connected to the network 407a. The network I / F 407 transmits / receives data to / from other devices via the network 407a.
An imaging device 408a is connected to the communication I / F 408. The communication I / F 408 outputs the image signal output from the imaging device 408a to the CPU 401. Note that when the output image signal from the imaging device 408a is an analog signal, the communication I / F 408 may have a function of digitizing the output image signal from the imaging device 408a.

  Here, the processing of each functional block in the image processing apparatus shown in FIGS. 1, 6, 9, 10, and 12 is realized by the CPU 401 of the computer 400 executing a predetermined program, for example. Is done. Further, the processing of at least some of the functional blocks in these image processing apparatuses may be realized by a dedicated circuit such as the graphic processing apparatus 404, for example.

  Thus, the processing function of the image processing apparatus in each of the above embodiments can be realized by a computer. In that case, a program describing the processing contents of the functions that each image processing apparatus should have is provided, and the processing functions are realized on the computer by executing the program on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic storage device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Examples of the magnetic storage device include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape. Optical discs include DVD, DVD-RAM, CD-ROM / RW, and the like. Magneto-optical recording media include MO (Magneto-Optical disk).

  When distributing the program, for example, a portable recording medium such as a DVD or a CD-ROM in which the program is recorded is sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. In addition, each time a program is transferred from a server computer connected via a network, the computer can sequentially execute processing according to the received program.

Regarding the above embodiments, the following supplementary notes are further disclosed.
(Supplementary note 1) A first image showing a surface shape of a second three-dimensional object obtained by dividing a part of the first three-dimensional object into a plurality of parts by removing the first three-dimensional object. An image output unit to output;
An image composition unit for compositing the first image output from the image output unit with a second image in a semi-transparent state;
An image processing apparatus comprising:

  (Supplementary Note 2) The cut surface of the region removed from the first three-dimensional object is along a predetermined plane in a three-dimensional coordinate space in which shape data of the first three-dimensional object is defined. The image processing apparatus according to appendix 1.

  (Supplementary Note 3) The surface of the second three-dimensional object is along a plurality of division planes parallel to a predetermined reference plane in a three-dimensional coordinate space in which shape data of the first three-dimensional object is defined. The image processing apparatus according to appendix 2, wherein the surface of the first three-dimensional object is formed in a striped shape.

  (Supplementary note 4) The image processing apparatus according to supplementary note 3, wherein the reference plane coincides with a horizontal plane in a three-dimensional coordinate space onto which the second three-dimensional object is projected during the composition processing in the image composition unit. .

  (Additional remark 5) The said image output part is based on the parameter which determines the appearance of the said 2nd three-dimensional object in the image output from the said image synthetic | combination part, In the said 2nd three-dimensional object, the said 1st The image processing apparatus according to appendix 3 or 4, wherein the number of regions removed from the three-dimensional object is determined.

  (Additional remark 6) The said image output part WHEREIN: Based on the parameter which determines the magnitude | size of the said 1st image which appears in the image output from the said image synthetic | combination part, in said 2nd three-dimensional object, said 1st 3 The image processing apparatus according to appendix 5, wherein the number of regions removed from the three-dimensional object is determined.

  (Supplementary note 7) The supplementary notes 1 to 4, wherein the second three-dimensional object is divided into the plurality of parts by removing a plurality of portions of the first three-dimensional object. The image processing apparatus according to any one of the above.

  (Supplementary Note 8) The image output unit calculates shape data of the second three-dimensional object based on shape data defining a surface shape of the first three-dimensional object, and based on the calculated shape data The image processing apparatus according to any one of appendices 1 to 7, wherein the first image is output.

(Supplementary Note 9) Shape data indicating the surface shape of the second three-dimensional object is stored in advance in a storage device,
The image processing apparatus according to any one of appendices 1 to 7, wherein the image output unit outputs the first image based on shape data read from the storage device.

(Supplementary Note 10) A first image showing a surface shape of a second three-dimensional object obtained by dividing a part of the first three-dimensional object into a plurality of parts by removing the first three-dimensional object. An image output unit to output;
An image composition unit for compositing the first image output from the image output unit with a second image in a semi-transparent state;
An image processing apparatus having
A display device for displaying an image synthesized by the image synthesis unit;
An image display system comprising:

(Additional remark 11) It has an image pick-up device which picturizes an image and inputs a picked-up image into the image processing device,
The second image is an image based on the captured image.
The image display system according to supplementary note 10, wherein

(Supplementary note 12)
A first image showing a surface shape of a second three-dimensional object obtained by dividing a part of the first three-dimensional object into a plurality of parts by removing a part of the first three-dimensional object, Compositing the image in a semi-transparent state,
An image processing program for executing a process.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Image output part 3 Image synthetic | combination part 11,12 Object 11a, 11b Flat plate 12a-12c Part 13 Background image 13a Background object 14 Composite image 20 Reference plane 21-24 Divided surface

Claims (7)

Image output for outputting a first image showing a surface shape of a second three-dimensional object obtained by removing a part of the first three-dimensional object and dividing the first three-dimensional object into a plurality of parts And
An image composition unit for compositing the first image output from the image output unit with a second image in a semi-transparent state;
I have a,
The cut surface of the region removed from the first three-dimensional object is a predetermined reference plane in a three-dimensional coordinate space in which first three-dimensional shape data defining the surface shape of the first three-dimensional object is defined. Is set along
The image output unit outputs second 3D shape data of the second 3D object based on the first 3D shape data and a distance parameter indicating a distance from the reference plane to the cut surface. Calculating and outputting the first image based on the second three-dimensional shape data;
An image processing apparatus. Surface of said second 3-dimensional object, and characterized in that before Kimoto along a plurality of divided surfaces parallel to the quasi-plane, a shape of a surface of said first three-dimensional object is removed in stripes The image processing apparatus according to claim 1 . The image output unit is configured to output from the first three-dimensional object in the second three-dimensional object based on a parameter that determines how the second three-dimensional object appears in the image output from the image synthesis unit. The image processing apparatus according to claim 2, wherein the number of removed regions is determined. The image output unit removes the second three-dimensional object from the first three-dimensional object based on a parameter that determines the size of the first image that appears in the image output from the image synthesis unit. 4. The image processing apparatus according to claim 3 , wherein the number of the determined areas is determined. Image output for outputting a first image showing a surface shape of a second three-dimensional object obtained by removing a part of the first three-dimensional object and dividing the first three-dimensional object into a plurality of parts And
An image composition unit for compositing the first image output from the image output unit with a second image in a semi-transparent state;
An image processing apparatus having
A display device for displaying an image synthesized by the image synthesis unit;
I have a,
The cut surface of the region removed from the first three-dimensional object is a predetermined reference plane in a three-dimensional coordinate space in which first three-dimensional shape data defining the surface shape of the first three-dimensional object is defined. Is set along
The image output unit outputs second 3D shape data of the second 3D object based on the first 3D shape data and a distance parameter indicating a distance from the reference plane to the cut surface. Calculating and outputting the first image based on the second three-dimensional shape data;
An image display system characterized by that. An image pickup device for picking up an image and inputting the picked-up image to the image processing device;
The second image is an image based on the captured image.
The image display system according to claim 5 . On the computer,
Outputs a first image showing a surface shape of the second 3-dimensional object, wherein the first three-dimensional object is divided into a plurality of portions by a portion of the first three-dimensional object is removed,
Combining the first image with the second image in a translucent state;
Let the process run ,
The cut surface of the region removed from the first three-dimensional object is a predetermined reference plane in a three-dimensional coordinate space in which first three-dimensional shape data defining the surface shape of the first three-dimensional object is defined. Is set along
In the output of the first image, the second three-dimensional object of the second three-dimensional object is based on the first three-dimensional shape data and a distance parameter indicating a distance from the reference plane to the cut surface. Calculating shape data and outputting the first image based on the second three-dimensional shape data;
An image processing program characterized by that.

Images