JP2008257127A - Image display device and image display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display device with which a viewer's gazing point can be turned to an intended position. <P>SOLUTION: The image display device includes; a display means of displaying an image; a plurality of shaders varying in definition of shading treatment; an image data memory means in which the image data of the object displayed on the display means is memorized; a scenario data memory means in which the level of importance complying with the relative positional relation between a visual point and an imaginary space is defined for every object; a means for determining the relative positional relation between the viewer's view point and the object to be drawn on the imaginary space; a means for determining the level of importance relating to the respective objects to be drawn by referring to the scenario data memory means; and a means for creating the display screen data to be displayed on the display means by practicing the shading treatment by the shader of the high definition to the image data of the prescribed value in the determined level of importance, and practicing the shading treatment by the shaders of the low definition level to the image data of not prescribed value in the determined level of importance. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image display apparatus and an image display method that can direct a viewer's point of gaze to an intended position.

Conventionally, in the real-time stereoscopic image display method, the entire image displayed on the screen is generated by the same image generation algorithm regardless of where the operator is gazing at the display screen. In order to solve the problem of high image quality, an image display method is known in which the operator's viewpoint is obtained, and the calculation load is reduced by reducing the definition of the image to be displayed in a region other than the periphery of the viewpoint (for example, , See Patent Document 1).
Japanese Patent No. 3478606

By the way, in recent years, images generated in real time have become particularly high-definition as technology advances, and both the amount of model data and the calculation load of shaders (shading processing programs) have been increasing in search of more realistic expressions. ing.
On the other hand, since the large screen has become high-definition, there is a problem that the viewer's gazing point is scattered on the screen, making it difficult to gaze at the position intended by the video producer. Also, since the hardware processing capability is not sufficient, it is difficult to ensure real-time performance. In particular, in order to generate a stereoscopic video, it is necessary to generate at least two images per frame, and therefore a further reduction in calculation load is required compared to a two-dimensional image. In addition, in order to increase the definition of a stereoscopic image, it is necessary to perform a shading process using a shader. However, it is necessary to perform the shading process for each polygon constituting an object (an object in the image) included in the image. Therefore, there is a problem that the calculation load increases according to the number of polygons included in the image. Therefore, since it is difficult to display in real time the number of polygons exceeding the capacity of the display device, the capability of the display device must be improved according to the content of the image to be displayed. There is also a problem that the cost becomes high.

  The present invention has been made in view of such circumstances, and provides an image display device and an image display method that can direct a viewer's gaze point to an intended position and can reduce calculation load. For the purpose.

  The present invention is an image display device that displays an image subjected to shading processing, the display means for displaying an image, a plurality of shaders with different degrees of definition of shading processing performed on an object to be drawn, and the display Image data storage means for storing image data of objects to be displayed on the means, and scenario data storage in which the importance corresponding to the relative positional relationship between the viewpoint and the object to be drawn in the virtual space is defined for each object Means for calculating a relative positional relationship in the virtual space between the viewer's viewpoint and the object to be drawn, and the scenario data storage means. The importance calculation means to be calculated and an object whose calculated importance is a predetermined value are shaded by a high-definition shader. Image generation means for generating display screen data to be displayed on the display means by performing shading processing with a shader having low definition on an object for which the obtained importance is not a predetermined value. It is characterized by.

  The present invention provides an event generating means for generating an event for an object having a predetermined importance value, and the object other than the image data for which the event is generated by the event generating means has a higher importance level. An importance level changing means for changing the importance level stored in the scenario data storage unit is further provided.

  According to the present invention, in order to display an image subjected to shadow processing, display means for displaying an image, a plurality of shaders with different precision of shadow processing performed on an object to be rendered, and display on the display means Image data storage means in which image data of the object to be stored is stored, and scenario data storage means in which the importance corresponding to the relative positional relationship between the viewpoint and the object to be drawn in the virtual space is defined for each object An image display method in the image display device, the viewpoint position calculating step for obtaining a relative positional relationship in the virtual space between the viewer's viewpoint and the object to be drawn, and the drawing by referring to the scenario data storage means Importance level calculating step for calculating the importance level for each target object, and objects for which the calculated importance level is a predetermined value Display screen data to be displayed on the display means by performing a shadow process with a shader having a high definition, and applying a shadow process with a shader with a low definition to an object whose importance is not a predetermined value. And an image generation step for generating.

  According to the present invention, an object having a predetermined importance level is subjected to a shading process by a shader having a high definition, and an object having a determined importance level not having a predetermined value is subjected to a shader having a low definition level. Display screen data that should be displayed with shading processing can be displayed so that only the object that the viewer wants to watch is focused on, so the viewer's point of interest is intended. The effect is that the object can be directed.

  Hereinafter, an image display device according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 denotes a drawing unit that draws an image by comprehensively controlling the processing operation of the image display apparatus. Reference numeral 2 denotes a display unit for displaying an image, which is composed of a liquid crystal display, a projector, or the like. Reference numeral 3 denotes a frame buffer that temporarily stores image data to be displayed on the display unit 2. Reference numeral 4 denotes an image data storage unit in which object data of a stereoscopic image to be displayed on the display unit 2 is stored in advance. Reference numeral 5 denotes a visual acuity data storage unit in which standard human visual acuity data is stored in advance. Reference numeral 6 denotes a shader storage unit in which a plurality of types of shaders (shading process programs) for performing a shadow process on the image data stored in the image data storage unit 4 are stored in advance. Reference numeral 7 denotes a scenario data storage unit in which scenario data for video display by VR (Virtual Reality) using object data stored in the image data storage unit 4 is stored.

  Next, the operation of the image display apparatus shown in FIG. 1 will be described with reference to FIG. First, scenario data generation will be described. The video producer divides the drawing scene displayed by VR into stages for each area (step S1). This is a process for preventing a distant object from being a target to be focused. When this stage division is performed, a name is given to a target object (an object included in the image) as shown in FIG. In the example illustrated in FIG. 4, the names “A” and “B” are given to the two objects existing in the stage 1.

  Next, the video producer sets the importance of the object to which the name is assigned for each stage (step S2). As for the importance of objects, the importance is set to a large value if the scenario is introduced, the importance is medium if there is a topic but not related to the scenario, and the importance is set to a low value if there is no topic. Set as follows. FIG. 5 shows an example in which importance is set for each object. In the example shown in FIG. 5, “5” is set for an object with high importance, “3” is set for an object with medium importance, and “0” is set for an object with low importance. Show. This importance is stored in the scenario data storage unit 7 for each stage in relation to the name of the object.

  Next, the video producer associates an object between stages with a scenario (step S3). For example, when the explanation voice is uttered or the explanation text is displayed on the object A, 3 is added to the importance of the object C, and the object A itself is set to zero. Further, when the explanation is executed for the object B, an association is made such that 3 is added to the importance of the object D, 2 is subtracted from the importance of the object C, and the object B itself is set to 0. Do. By performing the processes of steps S1 to S3, the obtained data is stored in the scenario data storage unit 7 as scenario data.

  Next, upon receiving a drawing start instruction, the drawing unit 1 refers to the scenario data stored in the scenario data storage unit 7 and reads an object to be displayed from the image data storage unit 4. Weighting is performed on the area (step S4). As shown in FIG. 7, the weighting is performed on the importance according to the arrangement on the screen and the distance from the viewpoint (camera), and the higher the value is the closer to the center of the screen, the higher the value is the closer to the viewpoint. Weighting is performed so that Then, the importance of the object to be displayed at the present time is obtained. In FIG. 8, an example in which the importance of object A is “5” and the importance of object B is “30” as a result of weighting object A and object B both having importance “5”. Show. Therefore, in the example shown in FIG. 8, the object B is an object to be focused.

  Next, the drawing unit 1 performs a setting for focusing on an object whose calculated weighted importance exceeds a threshold (see FIG. 9), and executes a predetermined depth of field process (see FIG. 9). Step S5). In other words, only the objects whose calculated weighted importance exceeds the threshold value are focused, and other objects generate images in a blurred state. The display image data of the object that is in focus and the object that is blurred is generated by using a high-definition shader among the shaders stored in the shader storage unit 6 for the object in focus. The depth-of-field process is executed so that the shader of the degree is used for the blurred object. Then, the drawing unit 1 writes the image generated here in the frame buffer 3. As a result, the display unit 2 displays an image in which only the object whose weighted importance exceeds the threshold is focused and the other objects are blurred.

  Next, the drawing unit 1 generates an event such as narration utterance or display of character information for the focused object (here, object B) (step S6). As a result, explanation regarding the object B is given to the viewer.

  Next, when the explanation ends, the drawing unit 1 changes the importance value of the object associated in advance (step S7). For example, when the explanation of the object B is finished, the importance levels of the objects D and E to be displayed in the stage 2 are “based on the related information of the object B (“ D + 3, E + 3, B × 0 ”shown in FIG. 11)”. 6 ”and“ 3 ”(see FIG. 12).

  As described above, since the importance of the objects D and E is changed by completing the explanation of the object B, automatic selection of a focus target is realized while realizing freedom of spatial movement and a selective scenario. Therefore, it is possible to perform video production and video display in consideration of the video creator's intention.

  Next, a case where a relative speed occurs between the object and the viewpoint as in the case where the object is moving or the viewpoint is moving will be described. Here, human visual acuity will be described. The dynamic visual acuity is a visual acuity when viewing an object moving relative to the viewpoint, and is inferior to a static visual acuity. There are two types of human visual acuity: KVA (Kinetic Visual Acuity) dynamic visual acuity and DVA (Dynamic Visual Acuity) dynamic visual acuity. The KVA moving body visual acuity is visual acuity when moving in the front-rear direction with respect to the viewpoint, and is related to a change in the thickness of the lens by the ciliary muscle. On the other hand, DVA moving body visual acuity is visual acuity when moving in the horizontal direction with respect to the viewpoint, and is related to eye movement by the extraocular muscles. FIG. 13 shows the relationship between each moving object visual acuity and the moving speed (cited from Japanese Laid-Open Patent Publication No. 2000-237133). As shown in FIG. 13, both the KVA moving body visual acuity and the DVA moving body visual acuity decrease as the moving body speed increases. In two-dimensional images, the lens thickness does not change because there is no need to move the focal point in the front-rear direction. However, in the three-dimensional image, the focal point movement in the front-rear direction is necessary. To do. Therefore, in the shadow processing of a moving object included in a stereoscopic image, the calculation time can be greatly shortened by selecting an optimum shader according to the moving object visual acuity and performing the shadow processing calculation. The visual acuity data storage unit 5 shown in FIG. 1 stores data in which the relationship between speed (or angular velocity) and visual acuity is defined for each of the two moving object visual acuities shown in FIG. Therefore, when the speed or angular velocity is determined, it is possible to obtain either of the two moving object visual acuities corresponding to these speeds or angular velocities with reference to the visual acuity data storage unit 5.

  Next, the operation of the image display apparatus shown in FIG. 1 will be described with reference to FIG. First, the drawing unit 1 obtains viewpoint position information in a virtual section when referring to an image stored in the image data storage unit 4. The viewpoint position information obtains the positions of the viewer's eyes and the width / height of the drawing surface, and obtains the left and right viewpoint positions in the virtual space based on them (step S11). Next, the drawing unit 1 obtains drawing parameters to be applied to each of the KVA moving object visual acuity and the DVA moving object visual acuity from the angle of view of the image to be generated (step S12). The drawing parameters are obtained as follows. When the angle of view of the generated image is small, the drawing parameter is set to a small value so that the influence of the KVA moving body visual acuity is small, and the drawing parameter is set to a large value so that the influence of the DVA moving body visual acuity is large. On the other hand, when the angle of view of the generated image is large, the drawing parameter is set to a large value so that the influence of the KVA moving body visual acuity becomes large, and the drawing parameter is set to a small value so that the influence of the DVA moving body visual acuity becomes small.

  Next, from the amount of movement of the target polygon to be drawn, the moving direction, and the frame rate, the front-rear speed and the horizontal angular speed are obtained with respect to the viewpoint, and the product of the parameters obtained in step S12 is calculated. The drawing unit 1 refers to the visual acuity data storage unit 5 and the KVA moving body visual acuity corresponding to the obtained two product values (the longitudinal velocity multiplied by the drawing parameter and the horizontal angular velocity). The DVA moving body visual acuity is obtained (step S13). When the moving object is sufficiently far from the viewpoint, even if the moving object moves in the front-rear direction, the human eye cannot recognize the difference in distance. The human visual acuity (detection limit of phase difference between two lines in contact with each other) is about 10 seconds, and anyone can distinguish if the convergence angle is 20 seconds. When the convergence angle is 20 seconds, the distance from the viewpoint to the moving object is about 650 m. Therefore, when the distance from the viewpoint to the target polygon is 650 m or more, KVA visual acuity is not considered.

  Next, the drawing unit 1 obtains a parameter indicating the ease of perception from the surface texture of the target polygon (step S14). Since the ease of perception of a moving object depends on the pattern / pattern on the surface of the object, the calculation load can be further reduced if the surface texture of the target polygon is difficult to perceive. Subsequently, the drawing unit 1 obtains the product of each value of the KVA moving body vision and the DVA moving body vision obtained in step S13 and the perceptibility parameter obtained in step S14, and selects a shader corresponding to the value. (Step S15). The shader storage unit 6 is associated with a shader to be used and a product value of each value of the KVA moving body visual acuity and the DVA moving body visual acuity and a parameter of ease of perception. The drawing unit 1 refers to this relationship and selects an optimum shader.

  Next, the drawing unit 1 performs an operation for shading processing on the polygon to be drawn by the selected shader (step S16), and writes the obtained result as drawing data in the frame buffer 3 (step S16). S17). When the drawing unit 1 executes the processing of steps S12 to S17 for all the polygons to be written to the frame buffer 3, one frame of drawing data is written to the frame buffer 3. As a result, drawing data subjected to different shading processing for each polygon moving speed is written in the frame buffer 3. That is, when the moving speed (relative speed with respect to the viewpoint) is high, low-definition shading processing is performed, and when the moving speed is slow, the drawing data is subjected to high-definition shading processing. The display unit 2 performs image display by sequentially reading the drawing data written in the frame buffer 3.

  Thus, by generating drawing data in consideration of the KVA moving body visual acuity, it is possible to reduce the calculation amount without recognizing a decrease in image quality as compared with the conventional method. In addition, since a shader is selected for each polygon, when a shader with a light calculation load is applied, drawing data can be generated without degradation in image quality uniformly in the object as in the conventional method. It becomes possible.

  In the above-described method, an example in which a shader to be applied is selected for each polygon has been described. However, a shader to be applied may be selected not for each polygon but for each surface texture pixel. By doing in this way, since the texture pixel is generally smaller than the polygon, the movement speed can be calculated more strictly, so that the calculation amount can be further reduced. In addition, the influence of polygon rotation / texture animation on visual acuity, which could not be considered for polygons, can also be considered.

  In FIG. 1, an example in which a plurality of shader programs are stored in the shader storage unit 6 is shown. However, instead of a plurality of shader programs, a plurality of dedicated hardware that executes shading is provided, The shadow processing may be executed by selecting these dedicated hardware.

  As described above, shading processing is performed on an object having a predetermined importance level with a high-definition shader, and an object having a determined importance level is not a predetermined value with a low-definition shader. Display screen data that should be displayed with shading processing can be displayed so that only the object that the viewer wants to watch is focused on, so the viewer's point of interest is intended. Can be directed to an object

  Also, in the shading process where the amount of calculation increases in proportion to the complexity of the image, the moving speed (relative speed with respect to the viewpoint) is obtained for each polygon or pixel to be drawn, and a shader corresponding to the moving speed is selected. Therefore, it is possible to draw a complicated image (an image having a large number of polygons to be drawn) without improving the performance of hardware for drawing. As a result, the amount of calculation can be reduced without the viewer perceiving a decrease in image quality.

  The program for realizing the function of the processing unit in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute image display processing. You may go. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart which shows operation | movement of the drawing part 1 shown in FIG. It is a flowchart which shows operation | movement of the drawing part 1 shown in FIG. It is a figure which shows an example which divided | segmented the scene into the stage for every area | region. It is explanatory drawing which shows an example of the data memorize | stored in the scenario data storage part 7 shown in FIG. It is explanatory drawing which shows an example of the data memorize | stored in the scenario data storage part 7 shown in FIG. It is explanatory drawing which shows the operation | movement which weights an object. It is explanatory drawing which shows the operation | movement which weights an object. It is explanatory drawing which shows a focus process operation. It is explanatory drawing which shows an example of event generation | occurrence | production. It is explanatory drawing which shows an example of the data memorize | stored in the scenario data storage part 7 shown in FIG. It is explanatory drawing which shows an example of the data memorize | stored in the scenario data storage part 7 shown in FIG. It is explanatory drawing which shows an example of the data memorize | stored in the visual acuity data storage part 5 shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Drawing part, 2 ... Display part, 3 ... Frame buffer, 4 ... Image data storage part, 5 ... Vision data storage part, 6 ... Shader storage part, 7 ...・ Scenario data storage

Claims (3)

An image display device that displays an image subjected to shading processing,
Display means for displaying an image;
A plurality of shaders with different degrees of definition of shading applied to an object to be drawn;
Image data storage means storing image data of an object to be displayed on the display means;
Scenario data storage means in which the importance according to the relative positional relationship between the viewpoint and the drawing target object in the virtual space is defined for each object;
Viewpoint position calculation means for obtaining a relative positional relationship in the virtual space between the viewer's viewpoint and the object to be drawn;
With reference to the scenario data storage means, importance calculating means for calculating importance for each of the drawing target objects;
The object whose degree of importance is a predetermined value is subjected to shadow processing by a shader with high definition, and the object whose degree of importance is not predetermined is subjected to shadow processing using a shader with low definition. An image generation device comprising: image generation means for generating display screen data to be displayed on the display means. Event generating means for generating an event for an object whose importance is a predetermined value;
Further comprising importance level changing means for changing the importance level stored in the scenario data storage unit so that the importance level of the object other than the image data that has generated the event by the event generation means increases. The image display device according to claim 1, wherein In order to display an image subjected to shading processing, display means for displaying an image, a plurality of shaders having different definition of shading processing performed on an object to be drawn, and an image of an object to be displayed on the display means Image display device comprising: image data storage means for storing data; and scenario data storage means in which importance corresponding to the relative positional relationship between the viewpoint and the object to be drawn in the virtual space is defined for each object An image display method in
A viewpoint position calculating step for obtaining a relative positional relationship in the virtual space between the viewer's viewpoint and the object to be drawn;
Referring to the scenario data storage means, an importance calculating step for determining the importance for each of the objects to be drawn;
The object whose degree of importance is a predetermined value is subjected to shadow processing by a shader with high definition, and the object whose degree of importance is not predetermined is subjected to shadow processing using a shader with low definition. And an image generation step of generating display screen data to be displayed on the display means.

Images