JP2016206766A - Data reducing apparatus and data reducing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the amount of data of a three-dimensional model without impairing an original shape.SOLUTION: A processing unit 11 reads a three-dimensional model stored in a storage unit 13, and acquires color information and depth information, which are set for each of pixels constituting a display unit 14, of the three-dimensional model viewed in one or more directions, in order to display the three-dimensional model viewed in one or more directions on the display unit 14. The processing unit 11 converts the three-dimensional model into display data represented by a point group including color information arranged in a three-dimensional space, by use of the color information and depth information of the pixels acquired in each of states of the three-dimensional model viewed in one or more directions.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a data reduction device and a data reduction method.

  In order to construct a three-dimensional model for representing a three-dimensional object on a display, a surface obtained by connecting a large number of triangles is used. Each triangle that constructs the three-dimensional model has information on the coordinates of three points, the normal of the surface formed by the three points, the color of each point, and the pattern of the surface.

  With the demand for image smoothness in the content industry such as games and movies, the size of the triangle has become small enough to accommodate high-definition and large-screen televisions. For this reason, the data amount of the three-dimensional model is very large.

  In view of this, many 3D model data reduction methods have been proposed, as exemplified by model construction by the silhouette method and 3D model reconstruction from live-action images (see, for example, Patent Document 1). The invention of Patent Document 1 performs data reduction by reducing unnecessary vertices and triangles while maintaining the quality of the model while taking advantage of the features of the original shape.

  While TVs have become high-definition and have a large screen, TVs and PCs have been replaced by smartphones. The memory capacity and processing capacity of smartphones are lower than televisions and personal computers, and the communication speed is also slow. For this reason, in order to display a 3D model on a smartphone, the data of the 3D model must be reduced to a certain amount of data commensurate with the capabilities of the smartphone.

  In order to reduce a large amount of 3D model to a certain amount of data, it is necessary to reduce the original with a high reduction rate. However, when reduction with a high reduction rate is performed using the invention of Patent Document 1, the original shape is destroyed. For this reason, it was difficult to display a three-dimensional model on a smartphone.

JP-A-11-144089

  An object of the present invention is to reduce the amount of data of a three-dimensional model without impairing the original shape.

  In the data reduction device according to the present invention, the computer reads the three-dimensional model stored in the storage unit, and configures the display unit to display the state of the three-dimensional model viewed from one direction on the display unit. The color information and depth information set for each pixel is acquired for a state in which the three-dimensional model is viewed from one or more directions, and each pixel acquired in each state in which the three-dimensional model is viewed from one or more directions. Using the color information and the depth information, the three-dimensional model is converted into display data represented by a point group including color information arranged in a three-dimensional space.

  In the data reduction device according to the present invention, when the color information and the depth information are acquired, the color information and the depth information of the pixels that do not display the three-dimensional model among the pixels constituting the display unit may be deleted. .

  In the data reduction device according to the present invention, the display unit is a display unit provided in the user device, and when the color information and the depth information are acquired, a state viewed from the direction displayed on the user device is displayed on the display unit. In order to do this, color information and depth information set for each pixel constituting the display unit may be acquired.

  In the data reduction method according to the present invention, the computer reads the three-dimensional model stored in the storage unit, and configures the display unit to display the state of the three-dimensional model viewed from one direction on the display unit. The color information and depth information set for each pixel are acquired in an acquisition procedure for acquiring the state when the three-dimensional model is viewed from one or more directions, and in each state when the three-dimensional model is viewed from one or more directions. Using the color information and depth information of each pixel, a display data conversion procedure for converting the three-dimensional model into display data represented by point groups including color information arranged in the three-dimensional space is executed.

  The above inventions can be combined as much as possible.

  According to the present invention, it is possible to reduce the data amount of the three-dimensional model without damaging the original shape.

An example of the data reduction apparatus which concerns on embodiment of this invention is shown. An example of the state which looked at the display object from one direction is shown. The example of a display on the display part of the state which looked at the display object from one direction is shown. Explanatory drawing of the color information acquired by an acquisition procedure and depth information is shown. The schematic diagram of the point group which rearranged the color information and depth information which were acquired in the acquisition procedure as a point in the view volume VV is shown. The example of a display on the display part 24 of display data is shown. The 1st example of the space | interval of the point in the view volume VV in the point group which comprises display data is shown. The 2nd example of the space | interval of the point in the view volume VV in the point group which comprises display data is shown. The 1st example of a display on the display part 24 in the case of the 2nd example of the space | interval of a point is shown. The 2nd example of a display on the display part 24 in the case of the 2nd example of the space | interval of a point is shown. The schematic diagram of the point group which rearranged the color information and depth information which were acquired in the acquisition procedure in Embodiment 2 as a point in the view volume VV is shown. An example of the point group rearranged in the view volume VV using the color information and the depth information acquired when viewed from the left side of the display target 100 is shown. An example of the point group rearranged in the view volume VV using the color information and depth information acquired in a state viewed from the right side surface of the display target 100 is shown. An example of a point group rearranged in the view volume VV using color information and depth information acquired in a state viewed from the bottom surface of the display target 100 is shown. An example of a point group rearranged in the view volume VV using color information and depth information acquired in a state viewed from the upper surface of the display target 100 is shown. The example of a display on the display part 24 of the display data constructed | assembled by the display data construction procedure in Embodiment 2 is shown. An explanatory view of an example when viewing a display object from a plurality of directions is shown.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to embodiment shown below. These embodiments are merely examples, and the present invention can be implemented in various modifications and improvements based on the knowledge of those skilled in the art. In the present specification and drawings, the same reference numerals denote the same components.

(Embodiment 1)
FIG. 1 shows an example of a three-dimensional model display system according to this embodiment. The 3D model display system according to the present embodiment includes a data reduction device 91 and a user device 92. The user device 92 is, for example, a mobile terminal that can perform data communication such as a smartphone and has a display function. The data reduction device 91 acquires 3D model data from a server that provides content, and converts the 3D model data into display data that can be displayed on the user device 92.

  The converted display data is provided to the user device 92. For example, the data reduction device 91 holds display data after conversion, and provides the display data to the user device 92 as a server that provides content or via a server that provides content. The three-dimensional model is not limited to a still image but may be a moving image.

  The data reduction device 91 includes a processing unit 11, a transmission / reception unit 12, a storage unit 13, and a display unit 14. The user device 92 includes a processing unit 21, a transmission / reception unit 22, a storage unit 23, and a display unit 24.

  The data reduction device 91 may be realized by causing a computer to function as the processing unit 11, the communication unit 12, the storage unit 13, and the display unit 14. In this case, each configuration is realized by a CPU (Central Processing Unit) (not shown) included in the data reduction device 91 executing a computer program stored in an information storage unit (not shown). Here, the computer for realizing the data reduction device 91 may further include an arbitrary device controlled by the computer. In addition, a program for realizing the data reduction device 91 can be recorded on a recording medium or provided through a network.

  The display unit 14 includes the same number of pixels as the display unit 24 included in the user device 92. For example, when the user terminal 92 is a smartphone, the display unit 14 has 1280 pixels arranged in the x-axis direction and 720 pixels arranged in the y-axis direction, which is about the same as a smartphone. When the display unit 14 is realized by a computer, the display unit 14 and the pixels of the display unit 14 are virtually realized on a memory of the computer, for example.

The three-dimensional model display method according to the present embodiment includes the data reduction method according to the present embodiment and a display procedure in order. The data reduction method has an acquisition procedure and a display data construction procedure in order.
In the acquisition procedure, the processing unit 11 acquires color information and depth information for displaying the three-dimensional model on the display unit 14.
In the display data construction procedure, the processing unit 11 constructs display data for displaying the three-dimensional model on the display unit 14 using the color information and the depth information acquired in the acquisition procedure. The display data is data obtained by converting a three-dimensional model into a point group including color information arranged in a three-dimensional space.
In the display procedure, the user device 92 displays the display data constructed by the processing unit 11 on the display unit 24.

Hereinafter, the acquisition procedure will be described.
The processing unit 11 reads the three-dimensional model stored in the storage unit 13 and displays a state where the read three-dimensional model is viewed from one direction on the display unit 14. FIG. 2 shows an example of the display target 100 on the display unit 14. The display object 100 shows a state where the three-dimensional model is viewed from one direction.

  FIG. 3 shows a display example on the display unit 14. The display unit 14 includes pixels Px1-1 to Px20-12. The pixel Px1-1 is a pixel of x = 1 and y = 1, the pixel Px20-12 is a pixel of x = 20 and y = 12, and the pixels Px1-1 to Px20-12 are Px1-1 to Px20−. Up to 12 pixels. Thus, the display unit 14 has 20 pixels arranged in the x-axis direction and 12 pixels arranged in the y direction.

  The processing unit 11 acquires color information and depth information set for the pixels Px1-1 to Px20-12 from data for displaying a three-dimensional model. For example, a three-dimensional graphics command is extracted using the OpenGL command capture technique. Depth information cannot be acquired from a photograph of a three-dimensional object, but in this embodiment, since data for displaying a three-dimensional model is used, depth information can be acquired even when there is a pattern or a sphere. it can. The color information is color information of the three-dimensional model itself, and excludes shading.

  The processing unit 11 acquires arbitrary information for displaying the three-dimensional model in addition to the color information and depth information set for each pixel. For example, as shown in FIG. 4, the position of the viewpoint PV, the position of the gazing point PL, the up vector, and the view volume VV are acquired. Although FIG. 4 shows the case of parallel projection, it may be perspective projection.

  The color information and the depth information are color information and depth information of the three-dimensional model of the display target 100 projected on the projection plane PP. The coordinates on the projection plane and the depth information at each coordinate are the three-dimensional coordinate information. When 101 and 102 of the three-dimensional model of the display target 100 are projected at a common position on the projection plane PP, the color information and depth information of 101 close to the projection plane PP are acquired, and 102 far from the projection plane PP. No color information or depth information is acquired. As described above, it is preferable to omit information on a portion of the three-dimensional model that is not displayed on the display unit 14 or an invisible portion inside the three-dimensional model. Specifically, only the smallest value in the z-axis direction among the information in the z-axis direction corresponding to each pixel is extracted from the shaded part of the three-dimensional model and the part that cannot be seen because it is inside. It will be omitted.

  Further, the color information and the depth information of the pixel on which the display target 100 is displayed such as the pixel of x = 8 and y = 2 in FIG. 3 are left, and the pixel that does not display the display target 100, such as the pixel Px1-1. It is preferable to delete the color information and the depth information. Further, when the depth information is infinite, it is preferable to delete the pixel color information and the depth information.

Hereinafter, the display data construction procedure will be described.
Each pixel of the pixels Px1-1 to Px20-1 has information on the x-axis direction and the y-axis direction. For this reason, the processing unit 11 acquires the color information and depth information of each pixel of the display unit 14 so that the x-axis direction, the y-axis direction, and the z-axis direction when the three-dimensional model is displayed on the display unit 14 are displayed. Three-dimensional coordinate information and color information at each coordinate can be acquired. In this procedure, three-dimensional coordinate information and color information at each coordinate are constructed using the color information and depth information set in the pixels Px1-1 to Px20-12 acquired in the acquisition procedure.

  Since the three-dimensional coordinate information is acquired in the acquisition procedure, the points specified by the three-dimensional coordinate information can be rearranged in the view volume VV. In the present embodiment, the number of points equal to the number of pixels of the display unit 14 is rearranged in the view volume VV. Thereby, display data of a three-dimensional model can be constructed. FIG. 5 shows a schematic diagram of a point group rearranged in the view volume VV. In FIG. 5, the display object 100 is represented by points larger and smaller than the pixels of the display unit 14 so as to facilitate understanding.

The display procedure will be described below.
The transmission / reception unit 12 transmits the display data constructed by the display data construction procedure to the user device 92. The transmission / reception unit 22 of the user device 92 receives the display data, and the processing unit 21 displays the display data on the display unit 24 using the display data.

  FIG. 6 shows a schematic diagram of the display object 100 displayed on the display unit 24. In this embodiment, the display data is represented by a point group including color information rearranged in the view volume VV. For this reason, the processing unit 21 can display the state of the display target 100 viewed from the direction in which the viewpoint is moved from the viewpoint PV in the acquisition procedure on the display unit 24. FIG. 6 shows a display example of the display unit 24 when the viewpoint is moved. In FIG. 6, the display object 100 is represented by points larger and smaller than the pixels of the display unit 14 for easy understanding.

  The display data according to the present embodiment can display a three-dimensional model of the display target 100 as shown in FIG. 6 except for a portion of the display target 100 that is not displayed on the display unit 14 in the acquisition procedure. . Since the display data is a point cloud arranged at a density comparable to the resolution of the display unit 24, the three-dimensional model can be displayed on the user device 92 without impairing the shape of the original three-dimensional model.

  However, when no normal line information remains in the display data, the processing units 11 and 21 do not have to perform lighting when displaying the display data on the display unit 24. Further, either of the processing units 11 and 21 may give new display information to the display data and display it on the display unit 24.

  On the other hand, the processing unit 11 may also acquire lighting information when acquiring in the acquisition procedure, and include normal and lighting information together with viewpoint PV information in the display data. Thereby, either of the processing units 11 and 21 can perform lighting using the acquired lighting information.

  When the number of pixels of the display unit 14 is smaller than the number of pixels of the display unit 24, the pixels of the display unit 14 may be assigned to a part of the pixels of the display unit 24 and displayed, or the point group constituting the display data Each point may be processed according to the number of pixels of the display unit 24. In the processing, for example, the diameter of each point in the display data is expanded according to the number of pixels of the display unit 24. In the enlargement, for example, one point assigned to one pixel in the display data is assigned to a plurality of pixels in accordance with the number of pixels of the display unit 24. Either the processing unit 11 or the processing unit 21 may perform this processing.

  When the number of pixels of the display unit 14 is larger than the number of pixels of the display unit 24, a part of the pixels of the display unit 14 may be assigned to the pixels of the display unit 24, and display data may be displayed. You may process according to the number. In the processing, for example, the diameter of each point in the display data is reduced according to the number of pixels of the display unit 24. For the reduction, for example, a plurality of points included in the display data are allocated to one pixel in accordance with the number of pixels of the display unit 24.

  As described above, the invention according to the present embodiment can display a three-dimensional model on the display unit 24 of the user device 92 with a point cloud having the same resolution as the display unit 24. Since the invention according to this embodiment displays a three-dimensional model without using triangles or connections, the amount of data can be reduced. In the invention according to the present embodiment, since a three-dimensional model is constructed using a result (image) displayed by computer graphics, any three-dimensional model is constant regardless of the original data size. The amount of data can be reduced. Further, since the density of the point group is the pixel of the display unit 24 of the user device 92, it is possible to prevent the original three-dimensional model from being damaged.

  Note that the number of pixels of the display unit 14 is preferably at least twice the number of pixels of the display unit 24. As shown in FIG. 7, it is assumed that the interval in the view volume VV of the point group acquired when the plane 201 of the display target 200 is the viewpoint PV1 parallel to the projection plane PP is Dp1. When the plane 201 of the same display object 200 is acquired at the viewpoint PV2 inclined by 45 ° with respect to the projection plane PP, as shown in FIG. 8, the interval Dp2 of the point group in the view volume VV is √2 times Dp1. It becomes. When the point cloud acquired at the interval Dp2 is displayed on the display unit 14, as shown in FIG. 9, when the point cloud acquired at the viewpoint PV2 is further drawn, as shown in FIG. 9, the pixels Pix12, Pix21- A gap occurs in Pix26. When the viewpoint is rotated by 45 °, as shown in FIG. 10, the gap between the point groups spreads to a square of 2Dp on each side, and gaps are generated in the pixels Pix31, Pix32, Pix34, Pix36, Pix43, and Pix45. As described above, when the number of pixels of the display unit 14 and the number of pixels of the display unit 24 are equal, the gap between the point groups is spread to a square of 2Dp per side.

  Therefore, the number of pixels of the display unit 14 is preferably set to be twice or more the number of pixels of the display unit 24. Thereby, the display unit 24 can display the point cloud of the display target 200 without a gap.

(Embodiment 2)
In the present embodiment, the acquisition procedure described in the first embodiment is performed in a state viewed from a plurality of different directions.

  In the acquisition procedure according to the present embodiment, when the processing unit 11 completes the acquisition of the color information and the depth information set in the pixels Px1-1 to Px20-12 in a state where the three-dimensional model is viewed from one direction, The color information and depth information set in the pixels Px1-1 to Px20-12 in a state viewed from a direction different from the direction is acquired, and this is repeated.

  For example, as shown in FIG. 11, a direction obtained by rotating the viewpoint PV of FIG. 5 by 180 ° with respect to the gazing point PL is set as a new viewpoint PV. Then, color information and depth information set for the pixels Px1-1 to Px20-12 are acquired.

  5 is the front of the display object 100, the viewpoint is the second viewpoint from the left side of the display object 100, and the right side of the display object 100 is the third viewpoint. The bottom surface of the target 100 may be the fourth viewpoint, and the top surface of the display target 100 may be the fifth viewpoint.

  When the direction that the user can see can be predicted in advance, the color information and the depth information set in the pixels Px1-1 to Px20-12 are acquired for the state viewed from each direction that the user may see. It is preferable.

  In the display data construction procedure, the processing unit 11 constructs the three-dimensional coordinate information and the color information at each coordinate using the color information and the depth information set in the pixels Px1-1 to Px20-12 obtained in the acquisition procedure. . For example, the points specified by the three-dimensional coordinate information are rearranged in the common view volume VV using the position of the viewpoint PV and the position of the gazing point PL. Thereby, the three-dimensional model displayed by the point cloud as shown in FIG. 6 and FIGS. 12 to 15 can be constructed. 12 to 15 show examples of point groups rearranged in the view volume VV using the color information and depth information acquired when the second viewpoint to the fifth viewpoint are used. In FIG. 12 to FIG. 15, for easy understanding, the point interval is made wider than the point group in units of pixels and one point is made larger.

  In this embodiment, since the point cloud acquired in a state viewed from a plurality of directions is rearranged in the common view volume VV, the point cloud in each direction is integrated. For example, the point groups shown in FIGS. 6 and 12 to 15 can be integrated to construct a three-dimensional model displayed with the point groups as shown in FIG. In FIG. 16, for easy understanding, a point interval is made wider than a point group having pixels as a unit, and one point is made larger.

  Here, the state viewed from a plurality of directions is a state in which the display target 100 is viewed from a plurality of directions corresponding to the six-face views, for example, as illustrated in FIGS. 6 and 12 to 15. Further, the state viewed from a plurality of directions may be a state in which the display target 100 is viewed from three viewpoints Pd1, Pd2, and Pd3 around the display target 100 as shown in FIG. The viewpoints Pd1, Pd2, and Pd3 are located on the circumference at an equal distance DS from the three-dimensional model that is the display object 100, and are equally spaced. In FIG. 17, the display target 100 is rotated by 120 ° because it is viewed from three directions that are equally spaced.

  Here, when viewed from a plurality of directions, the same part may be displayed on the display unit 14. In that case, a plurality of points are arranged at the same coordinates in the view volume VV. For example, when the display from the viewpoints Pd1 and Pd3 includes the grip portion of the teapot that is the display target 100, the color information of the grip portion of the teapot acquired at the viewpoint Pd1, and the color of the grip portion of the teapot acquired at the viewpoint Pd3 The information is all the same color. Thus, it is preferable to integrate a plurality of points representing the same portion of the three-dimensional model into one point.

  Even points that represent the same part have an error that depends on the pixel width of the display unit 14. Therefore, the points located within the error range may be integrated as representing the same location. For example, one point in the point group is arbitrarily selected, and points existing within the radius Rp are deleted from the selected point. If this is repeated, the remaining points exist at least 2 Rp apart. Therefore, the radius Rp is set to ½ of the pixel interval Dp. Thereby, it is possible to prevent a large number of points from remaining in one pixel of the display unit 14 while at least one point exists in one pixel of the display unit 14.

  In the present embodiment, the display data of the display target 100 is constructed by a three-dimensional model of point groups as shown in FIG. For this reason, in the display procedure, the processing unit 21 can display on the display unit 24 a state in which the display target 100 is viewed from an arbitrary direction, using the display data constructed by the data reduction device 91.

  As described above, the display data according to the present embodiment displays the display unit 100 in a state where the display target 100 is viewed from an arbitrary direction without the processing unit 11 acquiring new color information and depth information again. be able to.

(Embodiment 3)
The present invention reduces the data of the three-dimensional model in accordance with the number of pixels of the display unit 24. For this reason, when an attempt is made to enlarge a part of the three-dimensional model, the image quality displayed on the display unit 24 is degraded. Therefore, in the present embodiment, display data is reconstructed depending on the operation of the user device 92.

  For example, when the display unit 24 displays the display target 100 viewed from the viewpoint Pd1 as shown in FIG. 17, the processing unit 24 moves the display target 100 to the viewpoint Pd2 side by 120 ° by user input. Suppose you receive an instruction to rotate and enlarge. The processing unit 24 compares the number of point groups of the enlarged portion in the display data with the number of pixels of the display unit 24. When the number of point groups in the enlarged portion is equal to or less than a certain percentage of the number of pixels in the display unit 24, the transmission / reception unit 22 gives an instruction to rotate the display target 100 to the viewpoint Pd2 side by 120 ° to enlarge the data reduction device 91 Send to.

  When the transmission / reception unit 12 receives an instruction to rotate and enlarge the display object 100 by 120 °, the processing unit 11 changes the viewpoint according to the instruction and changes the size of the display object 100 arranged in the view volume VV, The acquisition procedure described in the first embodiment is executed again. At this time, the processing unit 11 displays on the display unit 14 a state in which the display target 100 is viewed from the viewpoint Pd2 rotated by 120 ° and is enlarged. The processing unit 11 acquires color information and depth information set for the pixels Px1-1 to Px20-12 from data for displaying a three-dimensional model.

  After the acquisition procedure, as in the first embodiment, the processing unit 11 executes the display data construction procedure, and the data reduction device 91 and the user device 92 perform the display procedure. Thereby, the invention concerning this embodiment can display the state where a part of three-dimensional model was expanded on display 24.

  Since the display data has a small amount of data, the user device 92 communicates with the server that provides the content while displaying the three-dimensional model converted into the display data on the display unit 24, or displays the display data every time it is displayed. The user device 92 can be provided.

  The three-dimensional model that is the display target 100 is arbitrary, and examples thereof include machines and furniture. For example, the user device 92 enables browsing of a car from various angles. For this reason, the invention according to the present embodiment can easily collect the operation history of the user device 92, and from which angle the form of the user device 92 is cared about, the headlamp, the side window, etc. It is also possible to collect information from the user viewpoint, such as whether the user is paying attention.

  The present invention can be applied to the information communication industry.

11, 21: processing unit 12, 22: transmission unit 13, 23: storage unit 14, 24: display unit 91: data reduction device 92: user device 100, 200: display target 201: plane PV: viewpoint PL: gazing point PP : Projection plane VV: View volume

Claims (4)

Computer
Color information and depth information set in each pixel constituting the display unit in order to display a state in which the three-dimensional model is viewed from one direction is displayed on the display unit. , Obtaining the state of the three-dimensional model viewed from one or more directions,
Using the color information and depth information of each pixel acquired in each state when the three-dimensional model is viewed from one or more directions, the three-dimensional model is represented by a point group including color information arranged in a three-dimensional space. Converted to display data,
The data reduction device. When obtaining color information and depth information, delete color information and depth information of pixels that do not display the three-dimensional model among the pixels that constitute the display unit,
The data reduction device according to claim 1. The display unit is a display unit provided in a user device,
When acquiring color information and depth information, the color information and depth information set for each pixel constituting the display unit is acquired in order to display the state viewed from the direction displayed on the user device on the display unit. ,
The data reduction device according to claim 1 or 2. Computer
Color information and depth information set in each pixel constituting the display unit in order to display a state in which the three-dimensional model is viewed from one direction is displayed on the display unit. An acquisition procedure for acquiring a state of the three-dimensional model viewed from one or more directions;
Using the color information and depth information of each pixel acquired in each state when the three-dimensional model is viewed from one or more directions, the three-dimensional model is represented by a point group including color information arranged in a three-dimensional space. Display data conversion procedure to convert the display data to
Perform data reduction method.

Images