JP2010125040A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique to generate hierarchical data, and to provide a display technique using the hierarchical data. <P>SOLUTION: In an image processor 10, an image data generation section 130 generates image data of each hierarchy. A hierarchical data generation section 140 generates hierarchical data expressing image data of a plurality of hierarchies by a common coordinate system. Then, the hierarchical data generation section 140 adds performance information when image data to be used for displaying to the hierarchical data. The hierarchical data generation section 140 sets performance information to a combination of two image data photographed by a virtual camera having different set camera parameters. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing technique for generating, for example, a game image describing a game scene at a plurality of resolutions and generating hierarchical data in which a plurality of layers of image data are expressed in a common coordinate system, and display using the hierarchical data The present invention relates to an image processing technique for displaying an image on a screen.

Home entertainment systems have been proposed that not only execute game programs, but also play video. In this home entertainment system, a GPU (Graphics Processing Unit) generates a three-dimensional image using polygons.
US Pat. No. 6,563,999

  2. Description of the Related Art Conventionally, there has been proposed a technique for performing display image enlargement / reduction processing and vertical / left / right movement processing using multi-resolution tile images generated from digital images such as high-definition photographs. In this image processing technique, an original image is reduced in a plurality of stages to generate images with different resolutions, and images in each layer are divided into one or a plurality of tile images to represent the original image in a hierarchical structure. Usually, the image with the lowest resolution is composed of one tile image, and the original image with the highest resolution is composed of the largest number of tile images. The image processing apparatus quickly performs enlarged display or reduced display by switching the tile image being used to a tile image of a different hierarchy at the time of enlargement processing or reduction processing of the display image.

  The conventional hierarchical data structure generates images with different resolutions based on the original image, so it satisfies the requirement for simple enlarged or reduced display of the displayed image, but provides a more visual effect. I can't do it. In particular, assuming a usage scene in a game application, it is preferable that display control that gives some visual effect to the user can be performed in addition to simply enlarging and reducing a specific game scene. At this time, by configuring the hierarchical data according to the game application, it is possible to provide the user with a world view of the game. This requirement is common not only to the game field but also to other fields.

  Therefore, an object of the present invention is to provide a technique for generating hierarchical data and a display processing technique using the hierarchical data.

  In order to solve the above problems, an image processing apparatus according to an aspect of the present invention is an image processing apparatus that generates hierarchical data, an image data generation unit that generates image data of each hierarchy, and image data of a plurality of hierarchies. And a hierarchical data generation unit that generates hierarchical data that expresses in a common coordinate system. The hierarchical data generation unit adds effect information when switching image data used for display processing to the hierarchical data.

  Another embodiment of the present invention is also an image processing apparatus. This apparatus is an image processing apparatus that displays an image on a display, and includes a storage device that holds hierarchical data representing image data of a plurality of hierarchies in a common coordinate system, and an instruction to change a display image displayed on the display An effect of determining whether or not effect information is set between the image data before the switching and the image data after the switching when the image data used for the display process is switched according to the reception unit that receives the signal and the change instruction signal An instruction unit and a display image processing unit that generates a display image using the image data are provided. When it is determined that the production information is set by the production instruction unit, the display image processing unit is set for the combination of the image data before switching and the image data after switching when switching the image data. An effect process is performed on the display image using the existing effect information.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to provide a technique for generating hierarchical data and a display processing technique using the hierarchical data.

  FIG. 1 shows an image processing system 1 according to an embodiment of the present invention. The image processing system 1 includes an input device 20, an image processing device 10 that executes various image processing, and a display device 12 that outputs processing results obtained by the image processing device 10. The display device 12 may be a television including a display that outputs an image and a speaker that outputs sound. The display device 12 may be connected to the image processing device 10 by a wired cable, or may be wirelessly connected by a wireless local area network (LAN) or the like. Note that the image processing device 10, the input device 20, and the display device 12 may be integrally formed, for example, may be configured as a mobile terminal device equipped with an image processing function.

  The image processing apparatus 10 of this embodiment has at least two types of image processing functions. One is a function for generating hierarchical data in which a plurality of image data is expressed in a common coordinate system, and the other is a function for performing display image change processing using the generated hierarchical data. The image processing apparatus 10 can execute each image processing function by loading the hierarchical data generation program and the image change processing program. These programs may be stored in advance as a library in the image processing apparatus 10, or may be downloaded from an external server via a network such as the Internet.

  In this embodiment, the image processing apparatus 10 is a game apparatus that executes game software. When a capture instruction for generating hierarchical data of a game image is received from the input device 20 while the game is in progress, a hierarchical data generation program is executed. to start. At this time, the image processing apparatus 10 temporarily stops the game progress, causes the virtual camera to capture a plurality of image data from the game scene at the time of the instruction, and generates hierarchical data composed of a plurality of resolutions. Note that the image processing apparatus 10 does not shoot the image data of all layers with the same camera parameters such as the camera position, the angle of view, the camera viewing direction, and the camera tilt. Is generated by a virtual camera in which camera parameters different from those of other layers are set. Thereby, the continuity of display between hierarchical data can be dared to be interrupted, and a game-specific world view can be provided to the user.

  FIG. 2 shows an external configuration of the input device 20. The input device 20 includes a cross key 21, analog sticks 27a and 27b, and four types of operation buttons 26 as operation means that can be operated by the user. The four types of operation buttons 26 include a circle button 22, a x button 23, a square button 24, and a triangle button 25. When the game program is executed, a function for operating the player character in the three-dimensional virtual space is assigned to the operating means of the input device 20.

  In the image processing system 1, after the image processing device 10 generates the hierarchical data by temporarily interrupting the execution of the game, an instruction to enlarge / reduce the display image and the up / down / left / right directions are given to the operation unit of the input device 20. A function for inputting a scroll instruction (hereinafter collectively referred to as an “image change instruction signal”) is assigned. For example, an input function for a display image enlargement / reduction instruction is assigned to the right analog stick 27b. The user can input an instruction to reduce the display image by pulling the analog stick 27b forward, and can input an instruction to enlarge the display image by pressing the analog stick 27b. The input function of the display image scroll instruction is assigned to the left analog stick 27a. The user can input a scroll instruction in the tilted direction by tilting the analog stick 27a up, down, left and right. As described above, the analog sticks 27a and 27b are used for continuously scrolling the display image and continuously enlarging / reducing the display image.

  The input device 20 has a function of transmitting the input image change instruction signal to the image processing device 10, and is configured to be able to perform wireless communication with the image processing device 10 in this embodiment. The input device 20 and the image processing device 10 may establish a wireless connection using a Bluetooth (registered trademark) protocol, an IEEE802.11 protocol, or the like. The input device 20 may be connected to the image processing apparatus 10 via a cable and transmit an image change instruction signal to the image processing apparatus 10.

  FIG. 3 shows a hierarchical structure of image data used in the image processing system 1. The image data has a hierarchical structure including a 0th hierarchy 30, a first hierarchy 32, a second hierarchy 34, and a third hierarchy 36 in the depth (Z-axis) direction. It is called “data”. The hierarchical data shown in FIG. 3 has a hierarchical structure of a quadtree, and each hierarchy is composed of one or more tile images 38. All the tile images 38 are formed in the same size having the same number of pixels, and have, for example, 256 × 256 pixels. The number of hierarchies constituting the hierarchy data may be two or more.

  The image data of each layer is expressed with different resolutions in a common coordinate system. For example, the resolution of the Nth layer (N is an integer greater than or equal to 0) may be ½ of the resolution of the (N + 1) th layer in both the left and right (X axis) direction and the up and down (Y axis) direction.

  As shown in FIG. 3, the hierarchical structure of the hierarchical data is set with the horizontal direction as the X axis, the vertical direction as the Y axis, and the depth direction as the Z axis, thereby constructing a virtual three-dimensional space. When the image processing apparatus 10 derives the change amount of the display image from the image change instruction signal supplied from the input device 20, information for specifying the hierarchy using the change amount and the texture coordinates (UV coordinates) in the hierarchy. Is derived. A combination of this hierarchy specifying information and texture coordinates is called spatial coordinates. In each layer, the vertex coordinates are the same, and the vertex coordinates of the rectangular image data in each layer are (u0, v0), (u0, v1), (u1, v1), (u1, v0), respectively. . The space coordinates in the virtual space are used for display image generation processing. Note that the image processing apparatus 10 may derive the three-dimensional coordinates of the four corners of the frame in the virtual space using the change amount of the display image. These four corner frame coordinates are also called spatial coordinates.

  The image processing apparatus 10 installs a virtual camera in the three-dimensional game space, acquires a game image projected on the virtual screen from the virtual camera, and generates image data of each layer. In the image processing system 1 of the present embodiment, camera parameters may be changed between different layers, or camera parameters may be changed within the same layer.

  When image data of two layers is generated at different resolutions with the viewpoint position, angle of view, viewing direction, and tilt (hereinafter referred to as camera parameters) of the virtual camera being fixed, the two image data are different game images in the same shooting area. Since it is only expressed by the resolution, the display continuity is not interrupted when the display image is enlarged or reduced and displayed across these layers. On the other hand, if the camera parameters of the virtual camera are changed to generate two layers of image data, the two image data represent the game images in different shooting areas, so the display image is expanded across these layers, In the case of reduced display, the continuity of display is interrupted. Even when two image data are generated by changing the viewpoint position of the camera in the same hierarchy, the two image data represent game images in different shooting areas. The continuity of is interrupted.

  Therefore, when the image processing apparatus 10 switches the image data used for generating the display image to the image data captured by the virtual camera having a different set camera parameter, the image processing apparatus 10 informs the user of the discontinuity in the image switching. A predetermined effect is inserted so that it does not feel. This makes it possible to provide the user with an image that expresses the game's unique view of the world as well as a display that simply enlarges and reduces.

  FIG. 4 is an explanatory diagram for explaining the relationship between hierarchies. In this hierarchical data structure, each hierarchy is expressed as L0 (0th hierarchy 30), L1 (1st hierarchy 32), L2 (2nd hierarchy 34), L3 (3rd hierarchy 36). In the hierarchical data structure shown in FIG. 4, the position in the depth (Z-axis) direction indicates the resolution. The position closer to L0 has a lower resolution, and the position closer to L3 has a higher resolution. Paying attention to the size of the image displayed on the display, the position in the depth direction corresponds to the enlargement ratio. If the enlargement ratio of the display image of L0 is 1, the enlargement ratio in L1 is 4, and the enlargement ratio in L2 is 16, and the enlargement ratio at L3 is 64. Therefore, in the depth direction, when the display image changes from the L0 side to the L3 side, the display image enlarges. When the display image changes from the L3 side to the L0 side, the display image is reduced. Go.

  The first boundary 31, the second boundary 33, and the third boundary 35 are defined as enlargement ratios based on L0, and are used as determination criteria for determining a hierarchy of tile images used for generating a display image. The first boundary 31 is set to an enlargement ratio of 2, and if the requested enlargement ratio of the display image is smaller than 2, the L0 tile image is used. The second boundary 33 is set to an enlargement ratio of 8, and if the requested enlargement ratio of the display image is 2 or more and smaller than 8, the L1 tile image is used. Similarly, the third boundary 35 is set to an enlargement ratio 32, and if the requested enlargement ratio of the display image is 8 or more and smaller than 32, the L2 tile image is used. If the requested enlargement ratio of the display image is 32 or more, the L3 tile image is used. Therefore, when the required enlargement ratio of an image to be displayed is determined, the image processing apparatus 10 reads a tile image having a resolution corresponding to the enlargement ratio from the storage device, and generates a display image adjusted to the enlargement ratio. it can. Note that the enlargement ratio may be determined as a Z value in the depth direction in the three-dimensional space shown in FIG.

  FIG. 5 shows a functional block diagram of the image processing apparatus 10. The image processing apparatus 10 includes a wireless interface 40, a switch 42, a display processing unit 44, a hard disk drive 50, a recording medium mounting unit 52, a disk drive 54, a main memory 60, a buffer memory 70, and a control unit 100. . The display processing unit 44 has a frame memory that buffers data to be displayed on the display of the display device 12.

  The switch 42 is an Ethernet switch (Ethernet is a registered trademark), and is a device that transmits and receives data by connecting to an external device in a wired or wireless manner. The switch 42 may be connected to an external network such as the Internet via a router to download a hierarchical data generation program or an image change processing program. The switch 42 is connected to the wireless interface 40, and the wireless interface 40 is connected to the input device 20 using a predetermined wireless communication protocol. An image change instruction signal input from the user in the input device 20 is supplied to the control unit 100 via the wireless interface 40 and the switch 42.

  The control unit 100 includes a multi-core CPU, and includes one general-purpose processor core and a plurality of simple processor cores in one CPU. The general-purpose processor core is called a PPU (Power Processing Unit), and the remaining processor cores are called a SPU (Synergistic-Processing Unit).

  The control unit 100 includes a memory controller connected to the main memory 60 and the buffer memory 70. The PPU has a register, has a main processor as an operation execution subject, and efficiently assigns a task as a basic processing unit in an application to be executed to each SPU. Note that the PPU itself may execute the task. The SPU has a register, and includes a sub-processor as an operation execution subject and a local memory as a local storage area. The local memory may be used as the buffer memory 70. The main memory 60 and the buffer memory 70 are storage devices and are configured as a RAM (Random Access Memory). The SPU has a dedicated DMA (Direct Memory Access) controller as a control unit, can transfer data between the main memory 60 and the buffer memory 70 at high speed, and the frame memory and the buffer memory 70 in the display processing unit 44 can be transferred. High-speed data transfer can be realized. The control unit 100 according to the present embodiment realizes a high-speed image processing function by operating a plurality of SPUs in parallel. The display processing unit 44 is connected to the display device 12 and outputs an image processing result according to a request from the user.

  The hard disk drive 50 functions as an auxiliary storage device that stores data. When a removable recording medium such as a memory card is mounted, the recording medium mounting unit 52 reads data from the removable recording medium. When a read-only ROM disk is loaded, the disk drive 54 drives and recognizes the ROM disk to read data. The ROM disk may be an optical disk, a magneto-optical disk, or the like, and records game software.

  FIG. 6 shows a flowchart of image processing of the present embodiment. When the image processing apparatus 10 is turned on and an operating system (OS) is started, and the user inserts a ROM disk into the disk drive 54, the control unit 100 stores the game software recorded on the ROM disk on the hard disk drive 50. The game software is executed (S10). At this time, the control unit 100 loads a necessary game program into the main memory 60 and loads drawing data and image data into the buffer memory 70.

  In this embodiment, the image processing apparatus 10 is a game apparatus, and provides a game image in which a player character behaves in a virtual three-dimensional space using three-dimensional (3D) graphics. The control unit 100 performs image processing based on commands and data generated in accordance with the execution of the game program, and generates image data. This image processing includes processing for realizing three-dimensional computer graphics such as texture mapping and light source processing.

  When the control unit 100 receives a capture instruction from the input device 20 during the execution of the game program, the control unit 100 temporarily stops the progress of the game and starts the hierarchical data generation program. At this time, the control unit 100 generates hierarchical data of the game image at the time of receiving the capture instruction (S12) and stores it in the hard disk drive 50. After generating the hierarchical data, when the control unit 100 receives an image change instruction from the input device 20, the control unit 100 starts an image change processing program. Using the hierarchical data stored in the hard disk drive 50, the control unit 100 executes display processing such as enlargement or reduction of the display image according to the image change instruction signal supplied from the input device 20 (S14).

  FIG. 7 shows a configuration of the control unit 100 that generates hierarchical data. The control unit 100 includes an application processing unit 120, an image data generation unit 130, and a hierarchical data generation unit 140. The application processing unit 120 includes an instruction receiving unit 122, a game execution unit 124, a camera control information acquisition unit 126, and an effect information acquisition unit 128.

  The application processing unit 120 reads out and executes the game software stored in the hard disk drive 50 and acquires data necessary for the hierarchical data generation process and the image change process from the hard disk drive 50. Data necessary for the image processing is recorded on the ROM disk together with the game software, and is read in advance on the hard disk drive 50 together with the game software.

  The image data generation unit 130 includes a camera parameter setting unit 132, a drawing condition setting unit 134, a game image generation unit 136, and a tile image generation unit 138. The image data generation unit 130 installs a virtual camera in the virtual three-dimensional game space, and generates a game image projected on the virtual screen from the virtual camera. During the execution of the game, the generated game image is output from the display processing unit 44 to the display device 12 and provided to the user as a moving image. In addition, when the hierarchical data is generated, the generated game image is divided into tile images and passed to the hierarchical data generation unit 140.

  The hierarchical data generation unit 140 receives the tile image generated by the image data generation unit 130 and generates hierarchical data. At this time, the hierarchical data generation unit 140 generates hierarchical data by expressing tile images of a plurality of levels in a common two-dimensional coordinate system, and also performs tile processing to be used when performing display processing using the hierarchical data. Production information at the time of switching is added to the hierarchical data.

  In FIG. 7, each element described as a functional block for performing various processes can be configured by a CPU (Central Processing Unit), a memory, and other LSIs in terms of hardware. This is realized by a program loaded on the computer. As described above, the control unit 100 includes one PPU and a plurality of SPUs, and each functional block can be configured by the PPU and the SPU individually or in cooperation. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one. For example, the image data generation unit 130 may be configured as a GPU (Graphics Processing Unit), and the hierarchical data generation unit 140 may be configured by loading a hierarchical data generation program.

  When the instruction receiving unit 122 receives a game execution instruction from the input device 20, the game executing unit 124 executes game software. The game execution unit 124 supplies a drawing command and data (hereinafter referred to as “drawing instruction information”) to the image data generation unit 130 in accordance with an operation instruction from the input device 20 and the progress of the game. The image data generation unit 130 has a 3DCG (three-dimensional computer graphics) processing function, and generates image data captured by a virtual camera using a known 3DCG technology such as high-speed rendering processing.

  The camera parameter setting unit 132 installs a virtual camera in the virtual three-dimensional space based on the drawing instruction information. The drawing instruction information includes camera parameters for determining the camera angle, such as the viewpoint position of the virtual camera, the angle of view (viewing area), the line-of-sight direction, and the camera tilt. The game execution unit 124 may change the drawing instruction information according to the instruction information from the input device 20 and switch the camera angle. The drawing condition setting unit 134 sets the image size acquired by the virtual camera, and here, the image size is matched with the display size of the display device 12. The game image generation unit 136 acquires an image captured by the virtual camera as a game image for each drawing frame. This drawing frame is updated every 1/60 seconds. The game image generation unit 136 supplies the game image to the display processing unit 44, and the display processing unit 44 supplies the game image developed in the frame memory to the display device 12. Thereby, the user can play the game in real time while watching the moving image.

  FIG. 8 shows an example of a virtual three-dimensional space generated by executing game software. The game execution unit 124 creates a scene layout by arranging the building 204, the tree 206, and the tree 208, which are objects constituting the game space, at predetermined positions, and operates the player character 200 in accordance with an operation instruction from the input device 20. . In this example, a battle scene in which the player character 200 confronts the enemy character 202 is constructed. The image data generation unit 130 moves the player character 200 and the enemy character 202 in accordance with the instruction information supplied from the game execution unit 124 to form a dynamic virtual three-dimensional space. The camera parameter setting unit 132 installs a virtual camera in the virtual three-dimensional space according to the drawing instruction information.

  FIG. 9 shows the virtual camera 210 installed in the virtual three-dimensional space by the camera parameter setting unit 132. At this camera angle, the virtual camera 210 images the player character 200 from behind. FIG. 10 shows a game image taken by the virtual camera 210 shown in FIG. In this game scene, the player character 200 and the enemy character 202 are shown facing each other.

  When the game image shown in FIG. 10 is displayed on the display device 12, the user inputs a game image capture instruction by operating a predetermined operation means of the input device 20. When the instruction reception unit 122 receives a capture instruction, the camera control information acquisition unit 126 acquires camera control information held in the hard disk drive 50. In the image processing apparatus 10, the capture instruction is handled as instruction information for generating hierarchical data of game images. When the instruction receiving unit 122 receives a capture instruction, the game executing unit 124 interrupts the progress of the game and instructs the image data generating unit 130 to generate image data for constituting hierarchical data. At this time, the game image at the time when the capture instruction is input, that is, the game image shown in FIG. 10 is displayed on the display device 12 as a still image.

  The camera control information is drawing instruction information for acquiring image data of a plurality of layers constituting the layer data. The camera control information includes a parameter for specifying the resolution of the image data to be acquired in addition to the camera parameter. The parameter that specifies the resolution may be information that specifies the image size. In the following example, the game image generation unit 136 causes the virtual camera set with the camera parameters to shoot the virtual three-dimensional space according to the resolution information, thereby obtaining the image data of the 0th hierarchy, the 1st hierarchy, and the 2nd hierarchy. Generate. The image data of the first layer and the second layer are generated at the camera angle of the virtual camera at the time when the capture instruction is input. Therefore, the image data of the first hierarchy and the second hierarchy are drawn in the same area, and the image size is set to 4 times (the resolution is 2 times) in the second hierarchy compared to the first hierarchy. .

  The image data of the 0th hierarchy is generated by rearranging the virtual camera at a position behind the player character 200 from the position of the virtual camera when the capture instruction is input. Since the camera angle is changed between the 0th hierarchy and the 1st hierarchy, the display continuity cannot be obtained when the image data is switched between the 0th hierarchy and the 1st hierarchy. Compared with the first layer, the layer does not necessarily have to be set to 1/4 times the image size. The drawing condition setting unit 134 sets the drawing condition (image size) of the game image to be generated based on the resolution information included in the camera control information.

  FIG. 11 shows a virtual camera 210 installed in a virtual three-dimensional space in order to generate the 0th layer image data according to the camera control information. Compared with the installation position of the virtual camera 210 shown in FIG. 9, it can be seen that the virtual camera 210 shown in FIG. 11 is provided at a position pulled behind the player character 200. FIG. 12 shows a game image taken by the virtual camera 210 shown in FIG. By pulling the position of the virtual camera backward, a wide area can be drawn, and an objective image including the surrounding situation can be provided to the user.

  First, the image data generation unit 130 receives an image data generation instruction from the game execution unit 124, and further receives camera control information acquired by the camera control information acquisition unit 126. As described above, the camera control information includes drawing instruction information that determines the camera angle for each layer and drawing condition information that specifies the resolution of the generated image. The information specifying the resolution may be an image size to be acquired.

  FIG. 13 shows a specific example of camera control information set for each layer. The camera control information is recorded on the ROM disk together with the game software, and is read in advance on the hard disk drive 50 together with the game software before the game is executed.

  The resolution information is set as image size information, and is set to 2k (1920 pixels) × 1k (1080 pixels), 4k × 2k, and 8k × 4k in the 0th layer, the 1st layer, and the 2nd layer, respectively. . In this example, the resolution of the first layer is twice that of the zeroth layer, and the resolution of the second layer is twice that of the first layer. As described above, since the image data of the first layer and the second layer are in the same shooting area, the resolution of the second layer is set to be doubled with respect to the first layer. The LOD (Level Of Detail) of the first layer and the second layer is set differently. The LOD expresses the level of detail of the image data. Here, the LOD is changed by increasing the resolution (level of detail) of the game image itself in the second layer rather than the first layer. On the other hand, since the image data of the 0th layer and the 1st layer are different shooting regions, it is not always necessary that the resolution of the 1st layer is doubled with respect to the 0th layer. The camera control information includes camera angle information, and the camera angle of the 0th layer is a position that is drawn backward from the player character 200 by a predetermined distance in the virtual three-dimensional space from the camera position at the time of capture instruction. Is set.

  The camera parameter setting unit 132 and the drawing condition setting unit 134 set the camera angle and resolution of the virtual camera according to the camera control information determined for each layer, and the game image generation unit 136 Generate a game image. Specifically, the game image of the 0th hierarchy is generated in the size of 2k × 1k as the game scene shown in FIG. 12, and the game image of the 1st hierarchy is generated in the size of 4k × 2k as the game scene shown in FIG. Then, the game image of the second hierarchy is generated in the size of 8k × 4k as the game scene shown in FIG.

  The game image generation unit 136 generates a game image by developing an image captured by the virtual camera in the buffer memory 70. The buffer memory 70 has a memory size sufficient for, for example, 8k × 4k image data. It may not be. In such a case, the camera parameter setting unit 132 divides the shooting area into a plurality of areas and appropriately adjusts the camera angle set by the camera control information.

  FIG. 14 shows an example in which an 8 k × 4 k shooting area is divided into four shooting areas. At this time, the camera parameter setting unit 132 adjusts the line-of-sight direction and the angle of view of the virtual camera so as to capture four 4k × 2k shooting areas without changing the installation position of the virtual camera. As a result, a large-sized game image can be taken by the virtual camera.

  The tile image generation unit 138 divides the image data of each layer into tile images of a predetermined size. The generated tile image is supplied to the hierarchical data generation unit 140. Note that the tile image generation unit 138 may compress the generated tile image in a predetermined compression format. The tile image generation unit 138 may compress each tile image, or may compress a plurality of tile images in the same hierarchy.

  The hierarchical data generation unit 140 expresses a plurality of hierarchical image data (tile images) in a common coordinate system, and generates hierarchical data in which these are associated with each other. The hierarchical data generation unit 140 generates hierarchical data as shown in FIG. 3, for example, and the image data of each hierarchy has a rectangular image size with the same aspect ratio. In each hierarchy, the vertex coordinates are the same. For example, the vertex coordinates of four points are (u0, v0), (u0, v1), (u1, v1), and (u1, v0), respectively. By expressing a plurality of image data in a common coordinate system in this way, image data of each layer can be associated with each other, and display image enlargement processing and reduction processing using the layer data can be realized. When generating hierarchical data, the effect information acquisition unit 128 acquires effect data for switching image data used for display processing from the hard disk drive 50 and supplies the acquired effect data to the hierarchical data generation unit 140.

  FIG. 15 shows an example of effect data set at the time of switching between hierarchies. The effect data is recorded on the ROM disk together with the game software, and is read in advance on the hard disk drive 50 together with the game software before the game is executed.

  In FIG. 15, the information between layers specifies the layer before switching and the layer after switching. For example, in “between the 0th layer and the 1st layer”, the image data used for display is changed from the 0th layer to the 1st layer. It means switching, and switching from the first hierarchy to the 0th hierarchy. The production information is information that specifies the production to be executed when the hierarchy is switched. "Crossfade" executes the crossfade when the hierarchy is changed, and "None" does not execute the production when the hierarchy is changed. Means.

  The hierarchy data generation unit 140 uses the effect data supplied from the effect information acquisition unit 128 to add effect information to be executed when switching between hierarchies to the hierarchy data. Thereby, hierarchical data is generated. Specifically, the hierarchical data generation unit 140 sets the effect information for a combination of two image data captured by virtual cameras having different camera parameters that are set according to the effect data. In the embodiment, since the camera parameters for photographing the image data of the 0th layer and the 1st layer are different (see FIG. 13), the effect information is provided for the combination of the 0th layer and the 1st layer image data. Is set.

  The hierarchical data generation unit 140 adds the production information to the hierarchical data based on the production data supplied from the production information acquisition unit 128. For example, from the camera control information shown in FIG. May be determined differently, and the effect information may be added to the hierarchical data. It is assumed that the production information added in this case is determined in advance, such as a cross fade. Thereby, the hierarchy data generation unit 140 can set the effect information for the combination of the image data of the 0th hierarchy and the 1st hierarchy. As a result, when the image data used for the display image transitions between different layers of the shooting region, the user is given a game-specific world view without causing the user to feel uncomfortable due to the discontinuity of the display image. Is possible.

  On the other hand, the hierarchical data generation unit 140 does not set effect information for a combination of two image data shot by a virtual camera having the same camera parameter. Specifically, since the camera parameters for photographing the image data of the first hierarchy and the second hierarchy are the same (see FIG. 13), as shown in the effect data shown in FIG. Production information is not added to the combination of two layers of image data.

  As described above, the hierarchical data generation unit 140 determines from the camera control information shown in FIG. 13 that the camera parameters are set to be the same between the hierarchies so that the production information is not added to the hierarchical data. May be. When the camera parameters are the same, continuous enlargement and reduction processing can be executed when the image data used for the display image transitions between layers, so that the user can smoothly enlarge without adding production information. It is possible to provide an effect of reduced display.

  The hierarchical data generation unit 140 stores the generated hierarchical data in the hard disk drive 50. When the hierarchical data generation unit 140 stores the hierarchical data in the hard disk drive 50, the hierarchical data generation unit 140 notifies the user from the display device 12 that the image change processing is possible.

  FIG. 16 shows a configuration of the control unit 100 that changes the display image. The control unit 100 includes an instruction receiving unit 150 and a display image processing unit 160. The display image processing unit 160 includes an image data acquisition unit 162, a decoding unit 164, a change amount derivation unit 166, a spatial coordinate determination unit 168, an effect instruction unit 170, and a display image generation unit 172. The display image processing unit 160 performs display image change processing using the hierarchical data.

  In FIG. 16, each element described as a functional block for performing various processes can be configured by a CPU (Central Processing Unit), a memory, and other LSIs in terms of hardware. This is realized by a program loaded on the computer. As described above, the control unit 100 includes one PPU and a plurality of SPUs, and each functional block can be configured by the PPU and the SPU individually or in cooperation. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one. For example, the display image processing unit 160 may be configured by loading an image change processing program.

  When the user operates the input device 20 to instruct execution of the image change process, the instruction receiving unit 150 receives the instruction, and the image processing apparatus 10 starts the image change processing program and uses the hierarchical data. Start the change process. At this time, the hierarchical data is read from the hard disk drive 50 and held in the main memory 60.

  When the user operates the analog stick 27 of the input device 20, an image change instruction signal is transmitted to the image processing device 10. The instruction receiving unit 150 receives a change instruction signal for a display image displayed on the display from the input device 20 and supplies it to the display image processing unit 160.

  The image data acquisition unit 162 reads out a tile image used for generating a display image from the main memory 60 based on the change instruction signal. The decoding unit 164 decodes the tile image and stores it in the buffer memory 70. For example, it is preferable that the image data acquisition unit 162 predicts the enlargement ratio of an image that is requested to be displayed by the change instruction signal, prefetches a tile image that will be necessary in the future, and causes the buffer memory 70 to store the tile image.

  The change amount deriving unit 166 derives the required change amount of the display image based on the change instruction signal. The change amount of the display image is a movement amount in the vertical and horizontal directions and a movement amount in the depth direction for each frame. The spatial coordinate determination unit 168 determines the texture coordinates of the current frame that moves according to the amount of change derived from the texture coordinates of the previous frame. The display image generation unit 172 generates a display image using the tile image held in the buffer memory 70 based on the texture coordinates, and supplies the display image to the frame memory 90 of the display processing unit 44.

  In the hierarchical data of this embodiment, as described above, effect information for switching image data between hierarchies is set. The effect instruction unit 170 determines whether or not effect information is set between the image data before the switching and the image data after the switching when the image data used for the display processing is switched according to the change instruction signal. As described with reference to FIG. 15, in this embodiment, the production information is not set when the first layer and the second layer are switched, but the production information (crossfade is used when the layer 0 and the first layer are switched. ) Is set. Therefore, when the tile instruction image 170 switches the tile image from the 0th layer to the 1st layer, or when the tile image is switched from the 1st layer to the 0th layer, the effect instruction unit 170 executes crossfading to the display image generation unit 172. Instruct. As a result, the display image generation unit 172 performs an effect process on the display image. Specifically, at the time of switching the tile image, the original tile image is faded out, and the new tile image is faded in to execute the crossfade. Thus, the discontinuity associated with the transition of the tile image is prevented from being felt by the user.

  The image data of the 0th layer and the 1st layer are images taken with different camera parameters, and when the image data is simply enlarged or reduced, discontinuity of display is given to the user. Therefore, this discontinuity can be alleviated by performing image processing such as crossfading. On the other hand, it is possible not only to simply enlarge or reduce the display image, but also to shift to a new viewpoint with a simple operation, so that it is possible to provide the user with a world view unique to games.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

  In the embodiment described above, the virtual camera parameter information is changed between the 0th hierarchy and the 1st hierarchy to generate image data. However, the virtual camera parameter information may be changed in the same hierarchy. .

  FIG. 17 shows virtual cameras 210a to 210f installed in a virtual three-dimensional space in order to generate image data according to camera control information. The virtual cameras 210a, 210b, and 210c are installed to generate the third layer image data, the virtual cameras 210d and 210e are installed to generate the second layer image data, and the virtual camera 210f It is installed to generate 0th layer image data. In this modification, among the camera parameters of each virtual camera 210, the camera positions are set to be different not only between layers but also within the same layer.

  FIG. 18 shows imaging areas 220a, 220b, and 220c by virtual cameras 210a, 210b, and 210c that take image data of the third hierarchy. In this modification, the image data of the same hierarchy is captured by changing the camera position, so that the continuity between the image data is impaired when switching the image data used for the display image even in the same hierarchy. Therefore, the parameters of the virtual camera are set so that an overlapping area is provided in the shooting area of the adjacent virtual camera, and the display image can be generated from one image data before and after the switching of the image data.

  In this modified example, first, the image data generation unit 130 receives an image data generation instruction from the game execution unit 124, and further receives camera control information acquired by the camera control information acquisition unit 126. The camera control information includes drawing instruction information for determining the camera angle in the virtual three-dimensional space and drawing condition information for specifying the resolution of the generated image. The information specifying the resolution may be an image size to be acquired. In this modification, for example, a plurality of drawing instruction information and drawing condition information are included in the camera control information also in the third hierarchy and the second hierarchy.

  The camera parameter setting unit 132 and the drawing condition setting unit 134 set the camera angle and resolution of the virtual camera according to the camera control information, respectively, and the game image generation unit 136 generates a game image by each virtual camera. The tile image generation unit 138 divides image data captured by each virtual camera into tile images of a predetermined size. The generated tile image is supplied to the hierarchical data generation unit 140. Note that the tile image generation unit 138 may compress the generated tile image in a predetermined compression format.

  The hierarchical data generation unit 140 collects image data captured by the virtual camera for each hierarchy, expresses multiple levels of image data (tile images) in a common coordinate system, and generates hierarchical data in which these are associated with each other. . At this time, the effect information acquisition unit 128 acquires effect data for switching the image data used for the display process from the hard disk drive 50 and supplies it to the hierarchical data generation unit 140.

  The hierarchy data generation unit 140 uses the effect data supplied from the effect information acquisition unit 128 to add effect information to be executed when switching image data to the hierarchy data. Thereby, hierarchical data is generated. In this modification, the display continuity is interrupted when switching the image data used for the display image not only between different layers but also within the same layer, and therefore all adjacent in the XY direction and the Z direction. Production information is set for the combination of image data. In other words, the hierarchical data generation unit 140 not only adds the effect information when switching the image data used for the display process to the image data of a different hierarchy, but also the image data used for the display process from different camera positions. The effect information at the time of switching to the image data of the same layer that was shot is also added to the layer data. The generated hierarchical data is stored in the hard disk drive 50. When the hierarchical data generation unit 140 stores the hierarchical data in the hard disk drive 50, the hierarchical data generation unit 140 notifies the user from the display device 12 that the image change processing is possible. Display image processing unit 160 generates a display image from the hierarchical data according to the effect information.

  In the example shown in FIG. 17, the virtual camera 210 b is arranged inside the tree 208. The virtual camera 210b cannot acquire an appropriate game image even if it is shot at this position. Therefore, the camera parameter setting unit 132 has a contact determination function between the virtual camera 210 and the object in the virtual three-dimensional space, and when it is determined that the viewpoint of the virtual camera 210 is inside the object, The image generation unit 136 changes the captured image to a predetermined color image, for example, a white solid image. Referring to FIG. 18, the captured image in the imaging region 220b is a white solid image. After the hierarchical data is generated using this image data, when the display image is generated using the image data captured by the virtual camera 210b, the display image becomes a white solid image. It becomes possible to easily recognize that it has occurred.

  In this case, instead of using a solid white image, for example, in the virtual three-dimensional space shown in FIG. 3, an area in which image data generated by the virtual camera 210b is used as a display image is set as an invisible area, and the spatial coordinates The determination unit 168 may perform a collision determination between the display area and the invisible area based on the image change instruction signal supplied from the input device 20. When the collision is determined, the change amount deriving unit 166 derives the change amount so that it cannot move to the invisible region. Accordingly, the user can determine that there is an area that cannot be seen due to an obstacle when changing the image, and can input an image change instruction from the input device 20 so as to bypass the invisible area as a result. It becomes.

  In the embodiment, the image data of the 0th layer is shot from a position obtained by subtracting a predetermined distance behind the player character 200 from the camera position at the time of receiving the capture instruction. Thereby, the display image changing process in accordance with the progress of the game is realized. For example, the image data of the 0th hierarchy may be a map image representing the entire map of the game. This map image is included in the game software in advance, and is useful for the user to grasp the entire story. Therefore, in this case, it is not necessary for the image data generation unit 130 to newly generate the image data of the 0th hierarchy, the processing load can be reduced, and the user can view the game map by reducing the display image. Therefore, it is possible to provide the user with an effect unique to the game. Note that the image data of the 0th hierarchy is not limited to the game map, and may be prepared in advance by game software.

  In the embodiment, the input function of the image change instruction signal is assigned to the analog sticks 27a and 27b. However, this input function may be set in accordance with the operation system of the game. For example, when the movement operation of the player character 200 during execution of the game is assigned to the cross key 21, the display image scroll instruction input function may also be assigned to the cross key 21 in the display change process using the hierarchical data. preferable. The game operation system is known by the game execution unit 124. Therefore, the game execution unit 124 notifies the instruction receiving unit 150 of the game operation system, and the instruction receiving unit 150 changes the operation system for the image change instruction from the default setting according to the game operation system, respectively. The operation system can be made common.

  In the embodiment, the image data of each layer is generated by being captured from the virtual three-dimensional space at the time of receiving the capture instruction. However, it is also possible to generate image data of a plurality of layers at different points in time. At this time, the game execution unit 124 causes the display device 12 to display an image at the time when the capture instruction is accepted, but executes a future game progress predicted from that time regardless of the user's operation. For example, with respect to the game scene of FIG. 10, the image data generation unit 130 generates a scene that is not displayed on the display device 12 but has the weapons that the player character 200 and the enemy character 202 are closer to each other and possesses each other's weapons. To do. The image data generation unit 130 generates a game scene (see FIG. 10) at the time when a capture instruction is received as second-layer image data, and further generates a game scene in which the player character 200 and the enemy character 202 approach each other as third-layer image data. Generate as As a result, after the generation of the hierarchical data, if the user expands the image data of the second hierarchy, the user can see the game scene predicted in the future when the transition is made to the image data of the third hierarchy. Can be more interesting.

  In the embodiment, one hierarchical data is generated, but right-eye and left-eye hierarchical data may be generated. The display image generation unit 172 can provide a stereogram to the user by generating a display image by shifting the binocular parallax according to the enlargement ratio and combining the images from the respective hierarchical data. .

  In the embodiment, one hierarchical data having a common line-of-sight direction is generated, but a plurality of hierarchical data may be generated from different line-of-sight directions. At this time, by assigning a line-of-sight direction switching function to the operation means of the input device 20, the user can change the viewpoint and enjoy enlargement and reduction display of the image.

  The display image generation unit 172 can also superimpose and display another image on the display image generated using the hierarchical data. For example, by preparing an image in front of the racing car as hierarchical data and preparing an indicator of the racing car as a separate image, the display image generation unit 172 performs enlargement / reduction processing of the image in front of the racing car. While performing, it is also possible to superimpose a fixed-size indicator image on the display image.

It is a figure which shows the image processing system concerning the Example of this invention. It is a figure which shows the external appearance structure of an input device. It is a figure which shows the hierarchical structure of the image data used in an image processing system. It is a figure which shows the explanatory view for demonstrating the relationship between hierarchy. It is a figure which shows the functional block diagram of an image processing apparatus. It is a figure which shows the flowchart of the image process of a present Example. It is a figure which shows the structure of the control part which produces | generates hierarchy data. It is a figure which shows an example of the virtual three-dimensional space produced | generated by running game software. It is a figure which shows the virtual camera which the camera parameter setting part installed in virtual three-dimensional space. It is a figure which shows the game image image | photographed with the virtual camera shown in FIG. It is a figure which shows the virtual camera installed in the virtual three-dimensional space in order to produce | generate the 0th hierarchy image data according to camera control information. It is a figure which shows the game image image | photographed with the virtual camera shown in FIG. It is a figure which shows the specific example of the camera control information set for every hierarchy. It is a figure which shows the example which divides | segments an imaging area of 8kx4k into four imaging areas. It is a figure which shows an example of the production data set at the time of the switch between hierarchy. It is a figure which shows the structure of the control part which changes a display image. It is a figure which shows the virtual camera installed in the virtual three-dimensional space in order to produce | generate image data according to camera control information. It is a figure which shows the imaging | photography area | region by the virtual camera which image | photographs the image data of 3rd hierarchy.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image processing system, 10 ... Image processing apparatus, 12 ... Display apparatus, 20 ... Input device, 40 ... Wireless interface, 42 ... Switch, 44 ... Display processing part , 50... Hard disk drive, 52... Recording medium mounting section, 54... Disk drive, 60... Main memory, 70. , 120 ... Application processing unit, 122 ... Instruction receiving unit, 124 ... Game execution unit, 126 ... Camera control information acquisition unit, 128 ... Production information acquisition unit, 130 ... Image Data generation unit, 132 ... camera parameter setting unit, 134 ... drawing condition setting unit, 136 ... game image generation unit, 138 ... tile image generation unit, 140 ... hierarchical data Data generation unit, 150 ... instruction reception unit, 160 ... display image processing unit, 162 ... image data acquisition unit, 164 ... decoding unit, 166 ... change amount deriving unit, 168 ... Spatial coordinate determination unit, 170 ... effect instruction unit, 172 ... display image generation unit, 210 ... virtual camera.

Claims (9)

An image processing apparatus for generating hierarchical data,
An image data generation unit for generating image data of each layer;
A hierarchical data generation unit that generates hierarchical data representing image data of multiple hierarchies in a common coordinate system,
The hierarchical data generation unit adds presentation information when switching image data used for display processing to hierarchical data. The image data generation unit generates image data by causing a virtual camera set with camera parameters to shoot a virtual three-dimensional space,
The image processing apparatus according to claim 1, wherein the hierarchical data generation unit sets production information for a combination of two image data captured by virtual cameras having different camera parameters.   The image processing apparatus according to claim 2, wherein the camera parameters include a position of a virtual camera, an angle of view, and a viewing direction of the virtual camera.   The image according to any one of claims 1 to 3, wherein the hierarchical data generation unit adds, to the hierarchical data, effect information when switching image data used for display processing to image data of a different hierarchy. Processing equipment.   The hierarchical data generation unit adds presentation information when switching image data used for display processing to image data of the same hierarchy captured from different camera positions to the hierarchical data. 5. The image processing device according to any one of 4.   The image according to claim 2 or 3, wherein the image data generation unit does not set effect information for a combination of two image data captured by a virtual camera having the same camera parameter. Processing equipment. An image processing apparatus for displaying an image on a display,
A storage device that holds hierarchical data representing image data of a plurality of hierarchical levels in a common coordinate system;
A reception unit for receiving a change instruction signal for a display image displayed on the display;
In response to the change instruction signal, when switching image data used for display processing, an effect instruction unit that determines whether effect information is set between the image data before switching and the image data after switching;
A display image generation unit that generates a display image using the image data,
When it is determined that the production information is set by the production instruction unit, the display image generation unit sets the combination of the image data before switching and the image data after switching when switching the image data. An image processing apparatus characterized in that an effect process is performed on a display image using the rendered effect information. On the computer,
A function to generate image data of each layer;
A function for generating hierarchical data representing image data of multiple hierarchies in a common coordinate system,
The function of generating hierarchical data includes a function of adding effect information to hierarchical data when switching image data used for display processing.   A computer-readable recording medium on which the program according to claim 8 is recorded.

Images