JP5111739B2 - Texture display device - Google Patents

Description

  The present invention relates to a texture display device that displays an image when an object is viewed from a different viewpoint with the same texture as when an actual object is viewed from that viewpoint. Examples of the object include a painting.

  In order to display the color of the three-dimensional structure and the pattern drawn on it with the same texture as the actual object, the conventional texture display device selects the irradiation angle and observation angle on the three-dimensional structure, and the three-dimensional structure. The selection of the material of the object is accepted. Then, based on the received information, the conventional texture display device specifies the spectral reflectance of each part of the three-dimensional structure represented by the composite information and the pattern from the color sample information stored in the color sample database.

  Further, the conventional texture display device creates texture expression information including the specified color sample information and synthesis information. Then, the conventional texture display device creates display information that matches the characteristics of the colors that can be displayed on the display unit of the terminal based on the texture expression information, and displays the display information on the display unit of the terminal (for example, Patent Literatures). 1). Moreover, the experimental result based on the same idea as the above is also shown (for example, refer nonpatent literature 1).

JP 2004-15019 A (page 4, FIG. 1) Kimura et al., “Oil Painting 3D Appreciation System with Free Perspective Appreciation”, IMPS 2004, I-40, '04 / 11/11

  However, the prior art has the following problems. In conventional texture display devices, it is necessary to perform enormous calculations each time the viewpoint is moved in order to select the irradiation angle, observation angle, and material of the three-dimensional structure each time and synthesize texture expression information accordingly. was there. Moreover, although the viewpoint is moved in real time, a large-scale arithmetic processing unit is required to perform the enormous calculation in real time.

  The enormous amount of processing means that, in the above-mentioned Patent Document 1, it takes 15 to 50 ms to process one image of 454 pixels × 380 lines. It is easy to imagine that real-time processing is difficult when processing an image of about 2000 pixels × pixels.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to obtain a texture display device that does not require large arithmetic processing when performing texture display.

A texture display device according to the present invention is a texture display device that displays an image when an object is viewed from a plurality of different viewpoint positions with the same texture as when an actual object is viewed from each of the plurality of viewpoint positions. The plurality of viewpoint positions are arranged in a predetermined time-series order by time-sequentially in an order defined in a predetermined order, and a first group having compressed image data encoded independently of other viewpoint positions And the second group of viewpoint positions having image compression data in which the difference between the viewpoint position and the viewpoint position one before in time series is encoded, and the viewpoint position of the first group is defined as an access point, The viewpoint position of the first group and the viewpoint position of the second group are mixed and arranged in a predetermined chronological order so that the viewpoint position of the second group continues one or more after a certain access point. A plurality of viewpoint positions viewpoint position and a viewpoint position of the second group and are arranged in a predetermined chronological order formed by mixed in can associate a respective compressed image data corresponding to a plurality of viewpoint positions, when Corresponding to the specified position information, a storage unit that stores as a series of sequenced moving image compressed data, a viewpoint position detection unit that specifies a viewpoint position for viewing an object from a plurality of viewpoint positions, and position information Coordinates for determining the time of a series of compressed video data corresponding to the specified position information by extracting the viewpoint position to be extracted from a plurality of viewpoint positions stored in a predetermined time-series order in the storage unit If the viewpoint position corresponding to the time determined based on the time conversion unit and the series of moving image compressed data stored in the storage unit is the viewpoint position of the first group, Record in memory By decoding the compressed image data, an image corresponding to the time determined by the coordinate / time conversion unit is generated, and the viewpoint position corresponding to the determined time is the viewpoint position of the second group Includes a series of compressed image data stored in the storage unit corresponding to a series of times from the detected access point to the viewpoint position corresponding to the time, by detecting an access point immediately before the viewpoint position corresponding to the time. The image decoding unit generates an image corresponding to the time determined by the coordinate / time conversion unit, and the image decoding unit generates the image corresponding to the time determined by the coordinate / time conversion unit. And a display processing unit for displaying the image.

  According to the present invention, when a texture display is performed by decoding an image from a desired viewpoint position based on moving image compression data corresponding to a plurality of viewpoint positions arranged in a predetermined time series order, A texture display device that does not require arithmetic processing can be obtained.

Hereinafter, preferred embodiments of the texture display device of the present invention will be described with reference to the drawings.
The texture display device according to the present invention arranges a plurality of viewpoint positions in a predetermined time series, stores moving image compressed data of an object in advance, and stores a moving image from a desired viewpoint position based on the stored moving image compressed data. The image is restored and displayed.

Embodiment 1 FIG.
FIG. 1 is a configuration diagram of a texture display device according to Embodiment 1 of the present invention. The texture display device in FIG. 1 includes an encoded data storage unit 10, a viewpoint position detection unit 20, a coordinate / time conversion unit 30, a time information generation unit 40, an image decoding unit 50, a frame buffer 60, and a display processing unit 70. Is done.

  The encoded data storage unit 10 includes a plurality of viewpoint positions arranged in a predetermined time-series order, a series of moving image compressed data of an object captured in advance according to a time-series order of the plurality of viewpoint positions, and an access point. A storage unit that stores the data as encoded data in association with each other.

  The present invention is characterized by storing a series of compressed moving image data for a plurality of viewpoint positions arranged in a predetermined time-series order, rather than storing individual images from a plurality of viewpoint positions. Yes. Further, this series of moving image compression data is based on compression by inter-frame prediction, as represented by MPEG (Motion Picture Expert Group).

  The plurality of viewpoint positions are viewpoint positions having image data obtained by encoding a difference between a viewpoint position having image data encoded independently of other viewpoint positions and a viewpoint position immediately before in time series. The former viewpoint position is referred to as an access point in the following description. That is, this access point indicates a viewpoint position where an image can be restored without inter-frame prediction in time-series image compression data corresponding to a plurality of viewpoint positions.

  Next, the viewpoint position detection unit 20 specifies the viewpoint position at which the object is viewed as position information, and outputs the specified position information as an XYZ coordinate signal. On the other hand, the coordinate / time conversion unit 30 converts the position information specified by the viewpoint position detection unit 20 into time information based on the encoded data stored in the encoded data storage unit 10.

  Specifically, the coordinate / time conversion unit 30 corresponds to position information specified from among a plurality of viewpoint positions stored as encoded data and arranged in a predetermined time series order and associated with time. By extracting the viewpoint position, the position information is converted into time information which is one-dimensional information.

  Next, the time information generation unit 40 is based on the encoded data stored in the encoded data storage unit 10 and the time information converted by the coordinate / time conversion unit 30 from the viewpoint position where the object is viewed. Scene time information necessary for decoding an image is generated.

  Specifically, the time information generation unit 40, first, based on the correspondence between a plurality of viewpoint positions and access points arranged in time series stored as encoded data, a coordinate / time conversion unit The viewpoint position corresponding to the time information converted by 30 is specified. Next, the time corresponding to the viewpoint position that is the access point immediately before the specified viewpoint position is further specified.

  Then, the time information generation unit 40 generates scene time information including time-series time data from the time corresponding to the viewpoint position serving as the access point to the time information converted by the coordinate / time conversion unit 30.

  Next, based on the scene time information generated by the time information generation unit 40 and the encoded data stored in the encoded data storage unit 10, the image decoding unit 50 detects the object from the viewpoint position where the object is viewed. Decode the image.

  Specifically, the image decoding unit 50 converts an encoded stream, which is a series of compressed images corresponding to time-series time data included in the scene time information, into a series of moving image compressed data of the encoded data. To generate an image obtained by decoding the encoded stream.

  That is, the image decoding unit 50 extracts, as an encoded stream, independently encoded image data at a time corresponding to an access point and a series of image data obtained by encoding a difference at a subsequent series of times. By decoding the stream, an image from the viewpoint position where the object is viewed is generated.

  The frame buffer 60 is a storage unit that accumulates decoded images, and the image decoding unit 50 stores the images generated by decoding in the frame buffer 60. Further, the display processing unit 70 extracts an image corresponding to the time determined by the coordinate / time conversion unit 30 from the frame buffer 60 and displays the image.

  As described above, the texture display device of the present invention detects the viewpoint position, calculates time information corresponding to the viewpoint position, and obtains the scene time information from the access point immediately before the calculated time information to the time information. In other words, the encoded stream corresponding to the scene time information is decoded, and an image corresponding to the time information corresponding to the viewpoint position can be displayed from the decoded images.

  According to the first embodiment, an image corresponding to a desired viewpoint position can be displayed based on moving image compression data corresponding to a plurality of viewpoint positions arranged in a predetermined time-series order. Therefore, it is possible to reduce the amount of data to be stored, display an image corresponding to a desired viewpoint position at high speed, and realize a texture display device that secures real-time performance without requiring a large calculation process.

Embodiment 2. FIG.
In the second embodiment, a viewpoint position detection method by the viewpoint position detection unit 20 will be specifically described. FIG. 2 is a diagram illustrating a configuration example of the viewpoint position detection unit 20 according to Embodiment 2 of the present invention. The viewpoint position detection unit 20 in FIG. 2 calculates the three-dimensional position by triangulation by inputting the first camera 21, the second camera 22, and the images of the first camera 21 and the second camera 22 3 A dimension position calculation unit 23 is provided.

  Each of the first camera 21 and the second camera 22 captures the viewpoint 24 and the image is input to the three-dimensional position calculation unit 23 that performs triangulation. Here, the angle determined by the three points of the viewpoint 24-the first camera 21-the second camera 22 is the first angle, and the angle determined by the three points of the viewpoint 24-the second camera 22-the first camera 21 Is the second angle.

  The three-dimensional position calculation unit 23 determines the viewpoint 24 based on the predetermined information on the distance between the first camera 21 and the second camera 22, the first angle, and the second angle. The spatial position of is calculated. Further, the three-dimensional position calculation unit 23 converts the calculated spatial position into XYZ coordinates and outputs position information of the viewpoint position.

  According to the second embodiment, by obtaining the viewpoint position by triangulation, it is possible to display an image according to the viewpoint position, and it is possible to obtain a texture display device that enables viewing of a realistic image.

  In the second embodiment described above, only two cameras are used. However, it is possible to improve the detection accuracy of the viewpoint position by using more cameras.

Embodiment 3 FIG.
In the third embodiment, another viewpoint position detection method by the viewpoint position detection unit 20 will be specifically described. FIG. 3 is a diagram illustrating a configuration example of the viewpoint position detection unit 20 according to Embodiment 3 of the present invention. The viewpoint position detection unit 20 in FIG. 3 includes a pointing device 25 and a position information conversion unit 26.

  The pointing device 25 has a position specifying function that can be controlled by the user, and corresponds to, for example, a mouse, a trackball, a stick pointer, a touch screen, or the like. Then, the position information conversion unit 26 outputs the position information of the viewpoint position by associating the output of the pointing device 25 with the spatial position.

  According to the third embodiment, by obtaining a viewpoint position using a controllable pointing device, a virtual viewpoint position can be designated without actually moving the viewpoint, and images viewed from various directions can be easily viewed. It is possible to obtain a texture display device that can be used.

Embodiment 4 FIG.
Next, in the fourth embodiment, a specific example for defining a plurality of viewpoint positions in a predetermined time series order will be described. A series of moving image compressed data is generated in advance according to a plurality of viewpoint positions defined as a predetermined time-series order.

  FIG. 4 is an explanatory diagram of a configuration example in a predetermined time-series order according to Embodiment 4 of the present invention. Here, for ease of explanation, a case will be described in which a plurality of viewpoint positions defined in a two-dimensional space are converted into a one-dimensional time axis by time series. FIG. 4 shows one-dimensionalization of a two-dimensional space by a Hilbert curve.

  The Hilbert curve can be easily localized by the following means. The plurality of viewpoint positions in FIG. 4 are shown as an image area of 8 pixels in the horizontal direction × 8 lines in the vertical direction, but can be divided into 4 areas of 4 × 4. That is, (1, 1) is the starting point in the upper left area, (1, 5) is the starting point in the lower left area, (8, 4) is the starting point in the upper right area, and (5, 5) is in the lower right area. It can be divided into four areas as starting points.

  Further, a predetermined time-series order is determined by defining the order of 8 × 8 viewpoint positions with the Hilbert curve shown in FIG. 4 using the start points of these four areas as access points. A series of compressed moving image data according to the time series order and the access point is stored in the encoded data storage unit 10 in advance.

  In the present invention, by generating scene time information, an image of an arbitrary viewpoint can be displayed at high speed. As an example, a case will be described in which it is desired to decode an image having a viewpoint position coordinate (1, 6) in a lower left 4 × 4 region. The coordinates (1, 6) are time points in the order of (1, 5) → (2, 5) → (2, 6) → (1, 6), starting from the access point (1, 5). It has become.

  Therefore, the time information generation unit 40 generates time-series information of times composed of these four coordinates as scene time information based on the encoded data stored in the encoded data storage unit 10. Furthermore, the image decoding unit 50 includes the independently encoded image data at the coordinates (1, 5) corresponding to the access point, and the subsequent three coordinates (2, 5), (2, 6), (1, The image at the coordinates (1, 6) is decoded based on the encoded stream of the image data obtained by encoding the difference from the previous viewpoint position as the time series in 6).

  Here, the scanning by the Hilbert curve has the following two merits. First, as a first merit, it is possible to improve the efficiency of encoding the continuous video of the viewpoint position obtained from the spatial coordinates associated with the time information. That is, the two viewpoint positions arranged in the front and rear as the time series defined by the Hilbert curve are in a relationship of being arranged in the vertical and horizontal directions as the spatial coordinates, and the spatial distance between the two viewpoint positions is constant. Therefore, the coding efficiency is good and the data amount of the series of moving image compressed data can be reduced.

  Further, as a second merit, there is a good accessibility that reproduction is possible at any place at high speed. As illustrated in FIG. 4, a plurality of 8 × 8 viewpoint positions can be divided into four 4 × 4 regions that are spatially equal. Furthermore, for each area, for example, the starting point can be defined as an access point.

  In this way, by assigning access points by equally dividing the entire area spatially, it is possible to prevent the number of coordinates included in the scene image information from becoming extremely large for any viewpoint position. The uniform accessibility can be provided in the entire area, and the accessibility can be improved.

  According to the fourth embodiment, by defining a predetermined time-series order corresponding to a plurality of viewpoint positions using a Hilbert curve, it is possible to reduce the volume of compressed moving image data accompanying improvement in encoding efficiency, A texture display device that realizes high-speed generation of a decoded image at the viewpoint position can be obtained.

  In the above description, the form with respect to the two-dimensional viewpoint position is shown, but extension to three dimensions is also possible by the same method.

Embodiment 5 FIG.
Next, in the fifth embodiment, another specific example for defining a plurality of viewpoint positions in a predetermined time series order will be described. A series of moving image compressed data is generated in advance according to a plurality of viewpoint positions defined as a predetermined time-series order.

  FIG. 5 is an explanatory diagram of a configuration example in a predetermined time series order according to the fifth embodiment of the present invention. Here, for ease of explanation, a case where a plurality of viewpoint positions defined in a two-dimensional space are time-sequentially converted to a one-dimensional time axis as in the fourth embodiment described on the left. explain. FIG. 5 shows one-dimensionalization of a two-dimensional space by zigzag scanning.

  As shown in FIG. 5, in this zigzag scan, all viewpoint positions are scanned while reciprocating smoothly, with an 8 × 8 region starting at (1, 1) and ending at (8, 8). Although this zigzag scan is inferior in encoding efficiency and accessibility as compared with the previous scan using the Hilbert curve, there is an advantage that time series of a plurality of viewpoint positions can be easily realized.

  According to the fifth embodiment, by defining a predetermined time-series order corresponding to a plurality of viewpoint positions using a zigzag scan, the scan order can be simplified as compared with the case of using a Hilbert curve, The time-series order of viewpoint positions can be easily specified. Therefore, even when a plurality of viewpoint positions are three-dimensional, it can be easily expanded, and a time series of a plurality of viewpoint positions consisting of three dimensions can be easily realized.

  In this description, a series of decoded images corresponding to the calculated scene time information is temporarily stored in the frame buffer 60. However, the time information of the series of decoded images is displayed without using the frame buffer 60. Only necessary image data may be transmitted to the display processing unit 70 by comparison with power time information. This configuration eliminates the need for the frame buffer 60 and allows a simpler device configuration.

  In the present invention, when the frame buffer 60 is used, not only the spatial position associated with the time information but also other positions can be interpolated and displayed using a plurality of accumulated decoded images. It is. In general, it is possible to obtain an interpolated image by taking an average of a plurality of images, but it is also possible to interpolate an image with higher definition by using a technique such as motion compensation.

  In addition, in the present invention, spatial position information corresponds to time axis information, so that it is possible to apply moving image coding technology such as MPEG that currently exists in the market in large quantities, It is possible to greatly reduce the manufacturing cost by using it.

It is a block diagram of the texture display apparatus in Embodiment 1 of this invention. It is a figure which shows one structural example of the viewpoint position detection part in Embodiment 2 of this invention. It is a figure which shows one structural example of the viewpoint position detection part in Embodiment 3 of this invention. It is explanatory drawing of the example of 1 structure of the predetermined time series order in Embodiment 4 of this invention. It is explanatory drawing of the example of 1 structure of the predetermined time-sequential order in Embodiment 5 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Code | symbol data memory | storage part (memory | storage part), 20 viewpoint position detection part, 21 1st camera, 22 2nd camera, 23 3D position calculation part, 25 pointing device, 26 position information conversion part, 30 coordinate / time Conversion unit, 40 time information generation unit, 50 image decoding unit, 60 frame buffer, 70 display processing unit.

Claims (6)

A texture display device that displays an image when an object is viewed from a plurality of different viewpoint positions with the same texture as when an actual object is viewed from each of the plurality of viewpoint positions,
The plurality of viewpoint positions are arranged in a predetermined time series by time series in an order defined in a predetermined order, and a first group having compressed image data encoded independently of other viewpoint positions And a second group of viewpoint positions having compressed image data obtained by encoding a difference between the viewpoint position of the previous viewpoint and a viewpoint position one before in time series, and defining the viewpoint position of the first group as an access point The viewpoint position of the first group and the viewpoint position of the second group are mixed and arranged in the predetermined time-series order so that one or more viewpoint positions of the second group follow one access point. ,
A plurality of viewpoint positions arranged in the predetermined time series in which the viewpoint position of the first group and the viewpoint position of the second group are mixed, and compressed image data corresponding to the plurality of viewpoint positions; , And a storage unit that stores a series of time-series compressed moving image data,
A viewpoint position detection unit that specifies, as position information, a viewpoint position at which the object is viewed from the plurality of viewpoint positions;
The viewpoint position corresponding to the specified position information is extracted from the plurality of viewpoint positions stored in the storage unit in the predetermined time series order, thereby corresponding to the specified position information. A coordinate / time conversion unit for determining the time of the series of moving image compression data;
When the viewpoint position corresponding to the time determined based on the series of moving image compression data stored in the storage unit is the viewpoint position of the first group, the viewpoint position corresponds to the time. By decoding the compressed image data stored in the storage unit, an image corresponding to the time determined by the coordinate / time conversion unit is generated, and the viewpoint position corresponding to the determined time is the second If it is the viewpoint position of the group, an access point immediately before the viewpoint position corresponding to the time is detected, and the storage is performed corresponding to a series of times from the detected access point to the viewpoint position corresponding to the time. An image decoding unit that generates an image corresponding to the time determined by the coordinate / time conversion unit by extracting and decoding a series of image compression data stored in the unit;
A texture display device comprising: a display processing unit that displays an image generated by the image decoding unit corresponding to the time determined by the coordinate / time conversion unit. The texture display device according to claim 1,
A frame buffer for storing images decoded according to time;
The image decoding unit stores the decoded image in the frame buffer,
The texture display device, wherein the display processing unit extracts and displays an image corresponding to the time determined by the coordinate / time conversion unit from the frame buffer. The texture display device according to claim 1 or 2,
The viewpoint position detection unit is a triangle based on an image obtained by photographing one viewpoint position using two cameras separated by a predetermined distance from one viewpoint position for viewing the object from the plurality of viewpoint positions. A texture display device characterized in that the position information corresponding to the one viewpoint position is determined by surveying. The texture display device according to claim 1 or 2,
The viewpoint-position detecting unit, said that one of the viewpoint position for viewing the object from among the plurality of viewpoint positions, determined as a more specified viewpoint position pointing device, corresponding to the one viewpoint position A texture display device characterized by specifying position information. The texture display device according to any one of claims 1 to 4,
The predetermined time-series order stored in the storage unit is determined by associating discrete viewpoint position information on a two-dimensional plane with one-dimensional time axis information based on scanning defined by a Hilbert curve. A texture display device characterized by being made. The texture display device according to any one of claims 1 to 4,
The predetermined time-series order stored in the storage unit is determined by associating discrete viewpoint position information of a two-dimensional plane with one-dimensional time axis information based on scanning by zigzag scanning. A texture display device characterized by that.

Images