JP2021111374A - Vehicle design support system - Google Patents

Abstract

To provide a vehicle design support system for supporting design works for vehicles and a computer program therefor.SOLUTION: A vehicle design support system 1 has a computer 50, a VR headset 10, and an operation input unit 120. The computer has an interior data recording unit, an image adjusting unit, and an image rendering unit. The interior data recording unit stores structure data of a vehicle interior and attribute candidate data. An operator 2 selects or adjusts at least a part of attributes of vehicle interiors using the operation input unit 120 with viewing display of a display 102 of the VR headset. The image adjusting unit selects or adjusts the attribute candidate data in response to an operation input to generate attribute data. The image rendering unit generates an interior image from the structure data and the attribute data in response to instruction contents of the operation input. The interior image is output to the display.SELECTED DRAWING: Figure 1

Description

This disclosure relates to a vehicle design support system. More specifically, the present disclosure relates to a vehicle design support system for streamlining vehicle design work using virtual space on a computer and a computer program for that purpose.

In the design of the automobile industry, it is common to perform 3D prototyping with a computer-aided design (CAD) system with the help of computer technology. Computers are also used for driving assistance, autonomous driving, and other purposes. Patent Document 1 (Japanese Unexamined Patent Publication No. 2017-59213) discloses a method and an apparatus for overcoming a technical problem of managing a design change of an object model. In Patent Document 2 (Japanese Unexamined Patent Publication No. 2017-182560), it is possible to obtain a final interior image by sequentially combining selected images of the interior of a vehicle and checking them step by step while displaying a three-dimensional image of the entire interior. Vehicle interior simulation devices and methods that can be used are disclosed. According to Patent Document 3 (Japanese Unexamined Patent Publication No. 2018-129732), a vehicle image display device capable of easily and smoothly switching the image displayed on the head-mounted display device while the head-mounted display device is attached to the head. Is disclosed. Patent Document 4 (Japanese Unexamined Patent Publication No. 2019-43496) discloses an automatic operation adjusting device that enables automatic operation according to a user's taste. Patent Document 5 (Japanese Unexamined Patent Publication No. 2019-148677) discloses a virtual reality training system that trains driving to a destination in virtual reality to reduce driver's anxiety and a vehicle provided with the virtual reality training system.

As shown in these, virtual reality (VR) systems are being used in the automobile industry in step with the progress of computer technology.

JP-A-2017-59213 Japanese Unexamined Patent Publication No. 2017-182560 JP-A-2018-129732 Japanese Unexamined Patent Publication No. 2019-43496 JP-A-2019-148677

A wide variety of factors are considered in order to specifically design a vehicle including an automobile. In particular, the interior design of the vehicle determines the surrounding environment of the driver and passengers (collectively referred to as "passengers" in this application), and also affects the driving operation for the driver. The interior of the vehicle must be evaluated from various points of view, even if only the attributes that can be visually perceived by the passenger are taken up, and not only the aesthetic aspect but also the suitability for driving operations must be examined. For this reason, it is necessary to efficiently carry out a wide range of evaluations and verifications assuming various environments, including the climatic conditions of the destination where the vehicle is shipped over the four seasons and the difference in the light environment 24 hours a day. Has been done.

However, the trial production process is time-consuming, and it is also necessary to make a judgment by integrating a lot of information in various and complicated situations, so that the work of interior design is burdensome. Furthermore, in recent vehicles, various information is provided to passengers through display devices such as meters, and auxiliary lighting in the interior is also being frequently used. Interior design work must overcome this increasing complexity.

The present disclosure aims to solve at least some of the above problems, and contributes to the sophistication of vehicle design by providing an efficient system and computer program that support the vehicle design work.

The present inventor paid attention to each part of the interior, for example, at the stage of designing a vehicle, if the information to be visually acquired by the passengers of the manufactured vehicle can be presented with good reproducibility by the virtual reality technology. It is possible to determine the attributes of the above in the virtual space and improve the efficiency of the design work. The present disclosure provides the operator with visual stimuli visually recognized by the passengers of the vehicle using virtual reality technology based on the vehicle data reproduced in the virtual space, and further, the interior of the vehicle designed by the operator. Allows you to select and adjust attributes. The present disclosure provides such a vehicle design support system that supports the design work of determining the attributes given to each part of the vehicle interior. The present disclosure also provides a vehicle design support system that supports the design work of attributes given to each part of the interior by facilitating the observation of the gaze point of the operator who is presented with the interior reproduced in the virtual space. do.

That is, in some aspects of the present disclosure, an interior that stores structural data for determining the structure of the vehicle interior in a virtual space and attribute candidate data for selecting or adjusting attributes for at least a portion of the vehicle interior. It includes a data recording unit, at least one virtual reality (VR) headset worn by the operator on the head, a posture sensor capable of detecting the posture of the operator's head, and a display capable of displaying a presented image. A VR headset, at least one operation input unit that receives operation input for selecting or adjusting the attribute for at least a part of the vehicle interior from the operator, and the operation input unit received by the operation input unit. An image adjustment unit that selects or adjusts the attribute candidate data for attributes for at least a part of the vehicle interior according to the instruction content to generate attribute data, and the structural data called from the interior data recording unit. From the attribute data from the image adjustment unit, an interior image corresponding to the instruction content of the operation input and corresponding to the direction of the operator's head in the virtual space is generated, and an image signal for the presented image is generated. A vehicle design support system including an image rendering unit that outputs as is provided.

Further, in a certain aspect of the present disclosure, an interior data recording unit that stores structural data for determining the structure of the vehicle interior in a virtual space and attribute data for determining attributes for at least a part of the vehicle interior. And at least one virtual reality (VR) headset worn by the operator on the head, a posture sensor capable of detecting the posture of the operator's head, a display capable of displaying a presented image, and at least one of the operators. A VR headset including an eye tracking detector that outputs an eye tracking signal capable of identifying the line of sight of one eyeball, and a position in the virtual space where the line of sight intersects the structure from the structural data and the eye tracking signal. The gaze point determination unit that determines the gaze point and outputs it as the gaze point position data, the gaze point position recording unit that stores the gaze point position data, and the structural data and the attribute data called from the interior data recording unit. Therefore, a vehicle design support system including an image rendering unit that generates an interior image corresponding to the direction of the operator's head in the virtual space and generates an image signal of the interior image is also provided.

Further, in the present disclosure, a computer program for the vehicle design support system described above is also provided.

In this application, unless otherwise specified, the description is given in technical terms used in the fields of automobiles and computers, which are ordinary vehicles. The virtual space is a three-dimensional space defined on a computer. Vehicle interior is also called interior, and is an object or equipment that exists in or surrounds the space in a space that is distinguished from the outside world, which passengers usually recognize as inside the vehicle. Is shown. The vehicle interior in this application includes windows such as seats, handles, meter cluster panels, center consoles, windshields, moldings, handles, shift levers, and accessories (floor mats, etc.) in automobiles. A VR headset is a device that can present an image at least visually to the operator by fixing it to the operator's head in virtual reality technology. A head-mounted display (HMD) is a typical example of a VR headset. The operator generally refers to the user who uses the vehicle design support system in this application. Design and design work are not necessarily limited to those focusing only on aesthetic viewpoints, but include a wide range of creative work including structural design, interface design such as GUI, experience information design such as UX (user experience), lighting design, etc. .. In the present disclosure, images include still images and moving images. Further, although the video includes a moving image, it may be called a video even if a part or the whole is a still image.

In any aspect of the present disclosure, a vehicle design support system or computer program that streamlines vehicle design work is provided.

FIG. 1 is an explanatory diagram illustrating a hardware configuration of the vehicle design support system according to the embodiment of the present disclosure. FIG. 2 is an image display example showing the display contents presented by the vehicle design support system of the embodiment of the present disclosure. FIG. 3 is a block diagram showing a function of the vehicle design support system according to the embodiment of the present disclosure. FIG. 4 is an explanatory diagram of a vehicle design support system that can be operated simultaneously by a plurality of operators according to the embodiment of the present disclosure. FIG. 5 is an explanatory diagram showing an example configuration in a situation where a plurality of operators according to the embodiment of the present disclosure work together via a network. 6A and 6B are image examples presented to the operator in the vehicle design support system of the embodiment of the present disclosure. FIG. 7 is an explanatory diagram of a vehicle design support system including an eye tracking detection unit adopted for detecting and recording a gazing point in the embodiment of the present disclosure. FIG. 8 is a block diagram of a vehicle design support system that employs a VR headset with an eye tracking detector in the embodiments of the present disclosure. 9A-9C are exemplary heatmap images (FIG. 9A), gaze trace images (FIG. 9B), and gaze cluster images (FIG. 9C) acquired in the vehicle design support system of the embodiments of the present disclosure. .. 10A and 10B are graphs summarizing the cumulative time at which the gazing point acquired in the vehicle design support system of the embodiment of the present disclosure was located, respectively, according to the feature points of FIG. 9B and the area of FIG. 9C, respectively. This is an example of totaling from another. FIG. 11 is a display example in which the transportation function using the controller is operated to support the viewpoint movement in the vehicle design support system of the embodiment of the present disclosure. FIG. 12 is an explanatory diagram showing a state in which a presenter is photographed by a camera and feature points are set by a tracker processing unit in the vehicle design support system of the embodiment of the present disclosure.

An embodiment of the vehicle design support system according to the present disclosure will be described with reference to the drawings below. Unless otherwise specified in the description throughout the figure, common parts or elements are given a common reference code. Further, in the figure, each of the elements of each embodiment is not necessarily shown while maintaining a scale ratio of each other.

FIG. 1 is an explanatory diagram illustrating a hardware configuration of the vehicle design support system 1, and FIG. 2 is an image display example showing display contents presented by the vehicle design support system. The operator 2 wearing the VR headset 10 of FIG. 1 on the head 22 visually recognizes the scenery of the virtual space of FIG. The display 102 of the VR headset 10 has a display area that covers the entire or part of the field of view of the operator 2. The posture sensor mover 104 is used to detect the posture of the VR headset 10, that is, the posture of the operator 2's head 22 with the help of the posture sensor stator 106, if necessary. The controller (operation input unit) 120 is a controller used in VR technology that can be held by the operator 2 for input operation. The biological condition sensor (biological condition detecting device) 140 can acquire a heartbeat, a skin potential, and other biological signals (biological vital signals) of the operator 2. A device (accelerator pedal, brake pedal, handle, shift lever, etc.) that simulates the driving operation of the vehicle can also be adopted as an optional selection. The computer 50 includes an appropriate OS, storage device, arithmetic unit, and graphic device, and is directly or indirectly connected to the VR headset 10, the controller 120, and the biological condition sensor 140. The connections include those of any means of communication, such as wired or wireless.

The presented image 542 of FIG. 2 is displayed by the display 102 toward the operator 2, and includes an interior image 302 and a background image 402. These images generally refer to images that provide visual information by being displayed on the display 102, and include both still images and moving images. The displayed image is a still image unless the operator 2 fixes the head 22 and both the interior image 302 and the background image 402 show a temporal change. When the interior image 302 and the background image 402 show a temporal change or the posture of the head 22 moves, a moving image corresponding to the change is displayed. When the posture of the head 22 changes, the posture sensor mover 104 and the posture sensor stator 106 detect the movement of the posture in the real space, and the direction and inclination of the presented image 542 displayed on the display 102 follow the movement. do. An example of the attitude sensor mover 104 is an angular velocity sensor such as a gyroscope. FIG. 2 illustrates an example of the display range of the presented image 542, which is the shift direction U when the operator 2 raises the jaw, the shift direction D when the operator lowers the jaw, and the shift when the head is turned to the left and right. The directions L and R are also shown. The display range of the presented image 542 is rotated for rotation due to the inclination of the head 22. As a result, the view of the operator 2 becomes as if he / she is placed in the virtual space. For example, when the operator 2 faces greatly to the right from the position where the operator 2 is seated on the driver's seat with the handle facing the front, the interior image 302 includes the passenger seat and the right door interior, and the background image 402 is the right side window (side). It will include a landscape image of the outside of the vehicle through the window). In this way, the operator 2 sees the view as if he / she is seated in the vehicle. If necessary, the display 102 may separately present to both eyes each image shifted in the direction in the virtual space for stereoscopic viewing by the left and right eyes.

The vehicle interior 300 of FIG. 2 is the main object of the design work. Therefore, windows such as seats, handles, meter cluster panels, center consoles, windshields, moldings, handles, shift levers, and accessories (floor mats, etc.) are displayed according to the visibility.

The vehicle interior 300 can be reproduced as a three-dimensional object in a virtual space by combining structural data and attribute data. At that time, the surface color, texture, surface reflection characteristics, etc. specified as attributes are also reproduced. The attribute is a property to be selected or adjusted in the design work. Therefore, when a design change accompanied by a shape change is the purpose of the design work, the structure for specifying the shape can be a typical example of the attribute. Further, the content of the graphic display of the display device area 310, the brightness, the light color and the light intensity in the lighting design of the vehicle interior lighting, and the temporal changes thereof are also typical examples of the attributes in the present application.

The controller (operation input unit) 120 is a device that the operator 2 holds in his / her hand, for example, and can be operated within the reach of the operator 2 even when the display 102 is attached. The signal from the controller 120 is converted into a signal suitable for input to the computer 50 by the signal processing unit 122, if necessary. By operating the controller 120, the operator 2 selects from several candidate options or adjusts parameters that can be continuously changed in order to change at least a part of the attributes of the vehicle interior 300. Can be done. If necessary, the display 102 displays a menu for assisting the selection by the controller (not shown).

The background 400 generally refers to a landscape that the passenger recognizes as the environment in which the vehicle is placed, and the background image 402 is an image of this. The background 400 is, for example, a real space in which a vehicle exists, and the background image 402 for that purpose is typically an image (that is, a still image or a moving image) taken in the real space. In addition to this, the background 400 can be a virtual space in which a three-dimensional object is arranged. The background image 402 can typically correspond to a virtual space with an image previously captured and stored by a camera (not shown) capable of capturing all directions (all celestial spheres) around it. In the example of FIG. 2, a landscape on a highway taken by such a camera is used. Depending on the application, the background image 402 may be a two-dimensional image that does not necessarily have depth information or three-dimensional shape information, and is an inner surface of a sphere having a finite radius in a virtual space (for example, a hemisphere that generally covers a half heaven). ) And the bottom surface representing the ground. Since the image suitable for the background image 402 is an arbitrary landscape image that can be assumed to drive a vehicle, it is acquired and selected according to the purpose of the design work. An artificially prepared composite image or a computer-generated image may be a suitable image.

The background image 402 is combined with the interior image 302, which is an image of the vehicle interior 300, by a method such as Z buffering. By the synthesizing process, the background image 402 is drawn according to the position where the outside of the vehicle can be seen in the vehicle interior 300 and the position where the outside of the vehicle can be observed through. Here, the position where the outside of the vehicle can be seen is typically the position where a light-transmitting substance (hereinafter, simply referred to as "glass") such as a windshield, a side window, a rear window, and a glass sunroof ("window") is placed. .. In addition, the position where the outside of the vehicle is observed through is typically the door opening portion when the door is opened, the circumference of the head in the case of an open car, and the like. In the present embodiment, the window portion may also be treated as the portion of the vehicle interior 300. When observing the background 400 through the glass, the light from a light source (for example, an interior light or a display device) provided so as to be a part of the vehicle interior 300 is visually recognized as an image due to the reflection on the glass surface. Inclusiveness can be a problem. If the window portion is treated as a portion of the vehicle interior 300, such a light source can also be a design target of the vehicle interior in consideration of reflection.

In FIG. 2, the operator 2 has a viewpoint centered on the left seat when seated in the driver's seat, and has a viewpoint centered on the right seat when seated in the passenger seat. As such, the presentation image 542 is displayed. The presented image 542 is displayed so that the operator 2 in a state of being seated in another seat such as the rear seat or taking some posture in the vehicle interior space has those viewpoints. These viewpoints can be switched according to the position of the VR headset 10 by the attitude sensor mover 104 and the attitude sensor stator 106, and in order to intentionally change the viewpoint position in the virtual space, it is a reality. It can be easily changed by means of shifting the corresponding coordinates between the space and the virtual space.

FIG. 3 is a block diagram showing the functions of the vehicle design support system 1. The computer 50 is equipped with a CPU, a graphic board, a main memory (main storage device), a storage (auxiliary storage device), input / output, and other devices, and a VR headset 10 and a controller 120 are connected through an appropriate interface. Has been done. The computer 50 constituting the vehicle design support system 1 includes an interior data recording unit 510, an environmental data recording unit 520, an image adjustment unit 530, and an image rendering unit 540.

The interior data recording unit 510 selects or adjusts the structural data 512 for determining the structure of the vehicle interior 300 which is the basis of the interior image 302 (FIG. 2) and the attributes of at least a part of the vehicle interior 300. Attribute candidate data 514 and. The environmental data recording unit 520 includes background data 522 for the background image 402 (FIG. 2). The background data 522 is arbitrary data that can generate the background image 402, and is typically image data taken in a hemisphere or a whole celestial sphere. The controller 120 receives an operation input from the operator 2 for selecting or adjusting an attribute of interest in the vehicle interior 300.

The image adjustment unit 530 selects or adjusts the attribute candidate data 514 according to the operation input received from the operator 2 through the controller 120 and the signal processing unit 122, and generates the attribute data 516.

The image rendering unit 540 generates an interior image 302 representing the vehicle interior 300 from the structural data 512 and the attribute data 516. Further, the image rendering unit 540 calls the background data 522 from the environment data recording unit 520 to generate a presentation image 542 for synthesizing the background image 402 based on the background data 522 with the interior image 302 and displaying it on the display 102. be able to. The process of the image adjusting unit 530 generating the attribute data 516 from the attribute candidate data 514 and the process of generating the corresponding presented image 542 by the image rendering unit 540 can be performed extremely quickly. Therefore, the operator 2 can check the result of his / her selection and adjustment while instantly switching the display with a high sense of reality. The contrast in such switching takes advantage of the vehicle design support system 1 that can perform a highly realistic visual expression in a virtual space. By using the vehicle design support system 1 in this way, the efficiency of the design work of the present embodiment can be significantly improved.

For this operation, the software for the vehicle design support system 1 utilizes the hardware resources of the computer 50 to include the interior data recording unit 510, the environmental data recording unit 520, the image adjustment unit 530, and the image rendering unit. It functions as 540. In particular, if the image rendering unit 540 prepares in advance a processing module for data format conversion for generating the presented image 542, no particular burden is generated on the operation of the vehicle design support system 1. In such a processing module, for example, by associating the resulting attribute data 516 and the structural data 512 in combination with an interface for assisting the operation of the operator 2 for selecting or adjusting the attribute candidate data 514. Data for the presentation image 542 is generated.

The vehicle design support system 1 of the present embodiment can also be implemented as a vehicle design support system 1M that is assumed to be used by a plurality of operators at the same time. FIG. 4 is an explanatory diagram showing the configuration of the vehicle design support system 1M. The vehicle design support system 1M is provided with a VR headset 10, a display 102, and a computer 50 as a set corresponding to each operator, and the set is connected to the required number of people. Each set can also operate standalone. On the other hand, when supporting collaborative work by a plurality of operators, in the vehicle design support system 1M, for example, the above-mentioned interior data recording unit 510 and environmental data recording unit 520 are stored in a data server 50S capable of communicating with each computer 50. The image adjusting unit 530 and the image rendering unit 540 can be distributed by each computer 50. As a result, even when the heads of each operator are in different postures, it is possible to study the design for a common model while showing each operator an appropriate presentation image. Further, each operator may perform the operation input by the controller 120 by himself / herself, and the presentation image according to the operation input by any operator can be presented to another operator. In particular, when a plurality of operators collaborate with each other, one operator may identify and emphasize or emphasize a part of interest in order to explain to another operator. For such an purpose, it is also preferable to allow the instructing operator to operate the local lighting corresponding to the pointer or the flash light (flashlight) in the lighting in the virtual space. For example, the controller 120 of the operator 2 can be provided with a function of detecting a direction like a pointer. As described above, it is preferable to implement the function of pointing to an arbitrary position and the operation function of local lighting in the vehicle design support systems 1 and 1M. FIG. 4 also shows other data such as the gazing point position recording unit 560, the gazing point position data 562, the subject information recording unit 590, and the biological information data 592. The contents of the embodiments described later can also be adopted for implementing the vehicle design support system 1M which is operated by a plurality of operators at the same time.

The vehicle design support system of the present embodiment can be adopted to support the collaborative work of a plurality of geographically separated operators, which the present inventor calls a remote network function. FIG. 5 is an explanatory diagram showing an example configuration of the vehicle design support system 1N of the present embodiment in a situation where a plurality of operators collaborate via a network, and FIGS. 6A and 6B show a plurality of operators collaborating. This is a display example of a screen presented on the VR headset 10 of each operator in a working situation. In order to implement and function the vehicle design support system 1N as a program of a computer device such as a personal computer (PC), each operator operates an execution program to be realized on his / her own PC. Therefore, each of the PCs 50A to 50C becomes a part of the vehicle design support system 1N through the network 68. Specifically, the operator A operates the server execution unit 60, which may be an execution program in the arithmetic unit, on the PC 50A operated by the operator A. Similarly, the operators B and C operate the client execution unit 66, which may be an execution program in the arithmetic unit, on the PCs 50B and 50C operated by themselves. The PCs 50A to 50C of each operator can communicate with each other by an appropriate protocol (for example, TCP-IP) with each other through a public line such as the Internet or a dedicated line.

The client execution unit 66 operates the PCs 50A to 50C used by the operators A to C as client terminals in a specific design review session (hereinafter referred to as "session") by the design review network community. On the other hand, the server execution unit 60 operates the PC 50A used by the operator A as a server device in the session. The function as a server device is realized by implementing the account management unit 602 and the session management unit 604. The server execution unit 60 is typically implemented as a function of the server execution program, and the account management unit 602 and the session management unit 604 are implemented as the functions. Operator A executes the server execution unit 60 as an administrator to set the functions of the vehicle design support system 1N.

The account management unit 602 implements a function of managing the accounts of users who can access the necessary resources of the vehicle design support system 1N. The account management unit 602, for example, said that the resources of the vehicle design support system 1N are functions and data files typically used for the vehicle design support system 1N, and the function of managing the account is the use of these resources. It may include the setting of authority and access right (permission). The permission table (not shown) records the permission level of each resource for each user's account, and the account management unit 602 refers to the permission table, for example, an operator who logs in accounts for the vehicle design support system 1N. Determine if the user has. Specifically, operators B and C who operate their own PCs 50B and C that function as client terminals log in with their own accounts by means such as password authentication, and are further allowed by preset functions and file permissions. You can access a wide range of resources. The file is stored in, for example, the data server 50S. The account management unit 602 refers to the permission table and executes such login management and permission management for functions and data files. For example, operators A to C are given unique user IDs, and permissions for functions and data files are managed in the permission table for each user ID.

The session management unit 604 controls whether or not a user participates in the session. By referring to the session table (not shown), which is a list of which users participate in each session, the session management unit 604 can make a connection request from the client execution unit 66 and subsequent communication. Determine if the connection is legitimate and control participation in the session. Thus, when operators A to C are legitimate users in a session, they can participate in a session by a pre-determined design review network community.

The identification of the user executed by the account management unit 602 and the session management unit 604 is typically performed by the user ID corresponding to the account, and also includes security means used by those skilled in the art such as a password for security. Will be adopted.

The computers 50A to 50C and the data server 50S can be equipped with a function in which operators A to C manage the functions of the vehicle design support system 1N and the settings for files. Settings for this feature or file include, for example, colors and avatars to distinguish oneself from others in collaborative work. Colors and avatars are used for display in appropriate situations in association with user IDs as needed. As a result, the operators A to C can wear the VR headset 10 and can visually recognize the activity in the virtual space of another person by, for example, the user ID. For example, on the display 102 of each VR headset 10 worn by operators A to C, the display of pointers and flashlights possessed by the operator and other operators and the indication destinations by the display are added to the color distinction, and who displays the display. It is possible to distinguish and visually recognize whether or not the control is performed in an understandable state. 6A and 6B are image examples presented on the displays 102 of the operator A and the operator B, respectively, and the operator A is distinguished from "Server" and the operator B from "Client 01" by the user ID. Both operators illuminate their destinations with different colored laser pointer displays from the graphics-expressed controller. Therefore, the operator A understands that the pointer labeled "Client 01" is due to the operator B. The reverse is also true. The screen viewed by operator A is FIG. 6A, and the screen viewed by operator B is FIG. 6B. Note that Color A and Color B in FIGS. 6A and 6B are display colors that can be distinguished from each other, and it is possible to easily identify the instruction destination of another person by using this as a clue. When adopting an avatar, it is useful to display an avatar display with a graphics mark that can identify the operator near the pointer. Operator A can recognize that a participant other than himself is Operator B by seeing that "Client 01" is attached to the laser pointer of another person in FIG. 6A. Operator B can also recognize by looking at FIG. 6B. Both A and B are screens in which two operators are seated in the driver's seat at the same time, but in a design review using a virtual space, it is possible to set viewpoint settings that cannot be realized at the same time in reality. can.

Computers used by Operator B due to the need for highly confidential management of design process data and the need to distinguish similar systems from those used by different groups for different design review network communities. When the client execution unit 66 of the 50B sends a connection request to the PC 50A (hereinafter, "server device") executing the server execution unit 60, the server execution unit 60 is used for login through the client execution unit 66. It is also useful to have the operator B input the prerequisite connection destination. For example, any information (typically an IP address) for identifying a server device can be entered through a prompt to establish a connection. As another example, the server execution unit 60 sends an invitation mail for the design review network community in response to a connection request, and the server execution unit 60 responds to a click operation by the operator B of the link in the mail text to the computer 50B. The connection with the client execution unit 66 can be established.

The server execution unit 60 and the client execution unit 66 can execute various commands by accepting arbitrary inputs, and the video reflecting the commands is reflected in real time to all the session participants in the community. By implementing the vehicle design support system 1N shown in FIG. 5 with such a function, a bidirectional function is realized through a network, so that design review by connecting a plurality of geographically distant design bases is supported. be able to.

The above description is based on the premise that the operator A among the operators A to C is operating his / her own PC50A as a server terminal as an administrator of the vehicle design support system 1N and a host of the session. At that time, if the operators B and C are legitimate users of the vehicle design support system 1N and are session participants, they should make their PCs 50B and 50C function as client terminals and participate in the session of the design review network community. Can be done. The administrator of the vehicle design support system 1N and the host of the session may or may not be the user. In addition, in the design review network community, there are various system configurations including network configurations such as what kind of computer device on the network the operator uses as a server device and whether the data server is independent in order to establish a session. The mounting form can be adopted.

In the vehicle design support systems 1, 1M, and 1N of the present embodiment, the target object and features that the operator visually confirms consciously or unconsciously can be further investigated by detecting the line of sight. In vehicle design, there is a need to investigate the effect of specific interior design on driving operations. Further, in an embodiment in which a vehicle that requires switching between automatic driving and manual driving is designed in a driving method in which automatic driving is partially adopted and switching is also performed to manual driving, the switching is performed. There is a need to investigate what kind of switching method is preferable by evaluating the display. If the vehicle design support systems 1 and 1M are operated for these purposes and the gaze point is investigated from the operator's line of sight, the interior design of the vehicle can be determined more efficiently.

FIG. 7 is an explanatory view of a VR headset 10A including an eye tracking detection unit 110 adopted for detecting and recording a gazing point, and FIG. 8 is a block of a vehicle design support system 1A adopting the VR headset 10A. It is a diagram. Regarding the VR headset 10A and the vehicle design support system 1A, the elements indicated by the same reference numerals are the same as the description for the VR headset 10 and the vehicle design support system 1. The VR headset 10A includes an eye tracking detection unit 110 for identifying the line of sight of at least one of the eyes of the operator 2. The eye tracking detection unit 110 is realized by, for example, a combination of an infrared camera and an infrared light source, and the line of sight of the operator 2's eyeball, that is, a straight line connecting the gazing point of view and the eyeball, is formed by the corneal reflection method, the bright pupil method, or the dark pupil. Specify by a method such as law. Such an eye tracking detection unit 110 can further detect blinking and opening / closing of the eyelids. From the blinking and opening and closing of the eyelids, it may be possible to determine the state of consciousness such as the driver's arousal status and concentration. The eye tracking signal 112, which is an output from the eye tracking detection unit 110, is received by the computer 50 via an appropriate interface and input to the gazing point determination unit 550. The gazing point determination unit 550 determines the line of sight of the operator 2 in the virtual space by using the eye tracking signal 112 and, if necessary, a signal indicating the posture of the head from an attitude sensor (not shown in FIG. 8). do. The gazing point determination unit 550 uses the line of sight to determine the surface of the three-dimensional object to be visually recognized, which is determined by the structural data 512 and, if necessary, the background data (not shown in FIG. 8) from the environmental data recording unit. Determine the point of view for. This gaze point can be determined by, for example, a so-called collision determination algorithm.

In addition to the gaze point determination method based on collision with an object, one of the other gaze point determination methods that can be adopted for processing in the gaze point determination unit 550 is the line-of-sight intersection method that determines the intersection where the left and right line-of-sight directions converge. be. When the operator 2 gazes, both eyes visually recognize the vehicle interior such as the surface of each interior part having a three-dimensional shape and the meter display in the virtual space, and the background outside the vehicle. At that time, the eyes of both eyes head toward a common gaze point. In response to this, in the line-of-sight crossing method, the eye tracking detection unit 110 separately detects the lines of sight of the left and right eyes and outputs a binocular eye tracking signal. If the convergence position of the line of sight of both eyes can be specified by the principle of triangulation, it will be a clue about the gazing point. The line-of-sight crossing method is particularly useful when it is difficult to make a judgment using only the collision judgment method or when the gazing point is not stable. For example, if the operator 2 sitting on the driver's seat is trying to determine the gazing point only by collision judgment, it is difficult to determine whether the gazing point is on the surface of the handle or on the meter through the handle. Things happen. Since it is difficult to identify from the ex post facto whether the detected gaze point is due to an erroneous judgment or whether the actual gaze point of operator 2 is so and is correctly judged, the gaze point judgment is accurately stabilized. It is extremely useful to do this. In the line-of-sight crossing method, the gazing point is determined based on the direction of the line of sight of both eyes of the actual operator 2, so that the gazing point position can be determined more accurately. In order to implement the line-of-sight crossing method, the gazing point determination unit 550 is made to determine the crossing position of the line of sight of both eyes from the binocular eye tracking signal (not shown), and is used as the line-of-sight crossing position data (not shown). It can be output. The line-of-sight intersection position data itself can be used as gaze point data. Further, it is also practical to implement a program capable of selecting or combining the collision determination method and the line-of-sight intersection method for the gaze point determination unit 550.

The above-mentioned gazing point can typically have information in the depth direction in the presented image 542 by having three-dimensional coordinates in the virtual space. As an example, the gazing point position data 562 is stored in the gazing point position recording unit 560 in the form of temporal data such as being associated with an appropriate time stamp. The gazing point position data 562 is indicated by, for example, three-dimensional coordinates in the virtual space.

It is also useful to extract objective information of the gazing point visually recognized by the operator consciously or unconsciously from the gazing point position data 562 stored in the gazing point position recording unit 560. If the image rendering unit 540 is used, an image that graphically presents the gazing point can be shown superimposed on the presented image 542. For example, the gaze point position data 562 of the gaze point position recording unit 560 can be called to generate a heat map image, a gaze trace image, and a gaze cluster image.

9A-C are exemplary heatmap images, gaze trace images, and gaze cluster images. The heat map image is an image in which the gazing point that has been in the line of sight for a longer period of time is weighted and displayed by coloring or the like, and it is as if the heat distribution has a high temperature portion according to the time in which the line of sight was directed. The distribution image is superimposed on the presentation image 542. In FIG. 9A, the heat display units 702, 704, 706, and 708, which are weighted as the main gazing points, are on the display device area 310 of the center console, on the display device area 310 of the meter cluster panel, and on the left of the background image, respectively. It is located on the vehicle in the lane and on the guide sign of the background image, and is colored or the like so as to be identifiable on the image and displayed on the presented image 542. The actual heat map image can be an image that reflects continuously changing weighting. The gaze trace image is a line-drawn trace image that follows the trajectory that the gazing point draws from moment to moment. In FIG. 9B, a line drawing trace shows that the gazing point changes over the feature points on the image numbered 1 to 5. The line drawing trace is created by connecting the positions of the gazing points at each time in three dimensions in the virtual space. A large number of tiny circles drawn on the line drawing trace are plotted at each position of the gazing point at a fixed time step timing. The gaze cluster image is an image showing the order of accumulation time in which the gazing points existed and the order in which the gazing points transitioned in individual areas divided into areas by a mesh or the like. In the gaze cluster image, as shown in FIG. 9C, by dividing the area by a three-dimensional mesh, it is possible to make a determination corresponding to the three-dimensional surface of the vehicle interior, which is advantageous.

In the present embodiment, the objective information of the gazing point can be displayed superimposed on the gazing point position data 562 according to the display style according to the preference of the analyst. For example, a graphical visual representation such as displaying the time spent by the gazing point for each feature point in FIG. 9B by the size of a circle is useful in that the influence of the feature points on the passenger's vision can be visually expressed.

Further, by using the gaze point totaling unit 580 shown in FIG. 8, the frequency or time of the gaze point indicated by the gaze point position data is totaled for each feature point of the structure of the vehicle interior or for each area divided by the virtual space. It is also useful to obtain statistics on the position of interest.

10A and 10B are graphs summarizing the cumulative time at which the gazing point was located for each feature point in FIG. 9B and for each area in FIG. 9C, respectively. When using the vehicle design support system 1M, it is useful for such a graph to store the gazing point position data 562 of the gazing point position recording unit 560 for each of the VR headset 10 and the operator. ..

As shown in FIGS. 1 and 3, the biological condition sensor 140 can be added to the vehicle design support system 1. The signal from the biological condition sensor 140 reflects the physical condition of the operator 2, and is transmitted to the computer 50 through the signal processing unit 142. The computer 50 stores the biological information data 592 corresponding to the signal from the biological condition sensor 140 in the subject information recording unit 590 in the form of temporal data such as associating with an appropriate time stamp. The biological information data 592 can include an electroencephalogram, an electrocardiogram, a myoelectric potential, and the like. From these signals, it becomes possible to acquire the wakefulness state, the sleep state, and the health state, which can be useful for safety research. In particular, in the awake state, the vehicle design support system 1 of the present embodiment is useful for accumulating knowledge about the method of switching to manual driving in automatic driving and what kind of vehicle design, notification, and stimulus are suitable. Can be done. This is because there is a possibility that the driver will be confirmed even during automatic driving. The awake state is also effective for warning the conscious state that may lead to erroneous operation by the elderly, and for verifying the detection probability of the conscious state that may lead to erroneous operation, by a warning sound for suppressing erroneous start. be. As the biological state sensor 140, a sensor that is not worn on the body, such as a voice sensor such as a microphone, can also be adopted. Further, such voice information can be used not only for checking the state of the operator but also for voice input by the operator.

It is also useful to adopt a tool for supporting the operator's viewpoint movement in the virtual space in the vehicle design support system 1 of the present embodiment. FIG. 11 shows a display example in which the transportation function using the controller 120 is operated in order to support the movement of the viewpoint in the vehicle design support system 1 of the present embodiment. The positions and directions of the head 22 of the operator 2 (FIG. 1) in the space are associated with those in the real space and those in the virtual space. However, while using the vehicle design support system 1, there may be a situation where the correspondence must be adjusted for some reason. A typical example is a scene in which the operator 2 moves a position, that is, a viewpoint or a direction, that is, a line-of-sight direction in a virtual space without moving a position or a direction in a real space. If it takes time to move the viewpoint, the operator 2 often loses sight of the design model he / she wants to see or cannot move to an appropriate standing position, which is important in design evaluation, because it is a virtual space. On the contrary, if the function that can move the viewpoint by intuitive operation is equipped, not only the viewpoint movement becomes easy, but also the time for the operator 2 to distract his / her own consciousness from the evaluation target is shortened. Therefore, the vehicle design support system 1 Practicality is greatly improved. In order to facilitate the movement of the viewpoint, it is more preferable that the movement distance and the adjustment of the line-of-sight direction can be intuitively adjusted with as few operations as possible. The scenes in which the viewpoint needs to be moved include, for example, changing the seating position inside the vehicle, getting on and off the vehicle, and changing the observation position outside the vehicle.

In the vehicle design support system 1 of the present embodiment, a movement tool skillfully using the controller 120 is provided in order to support the movement of the viewpoint in the virtual space. The posture of the controller 120 in the real space is detected by the same sensors (not shown) as the posture sensor mover 104 and the posture sensor stator 106. When an appropriate command instruction is received by an arbitrary method such as menu selection, the operation of the vehicle design support system 1 is switched to the viewpoint movement mode. In the operation by the controller 120 in the viewpoint movement mode, an appropriate index such as the curve symbol 82 (FIG. 11) of the parabolic arch from the controller is graphically displayed in the virtual space. When the parabolic arch is used, the controller 120 facilitates the adjustment of the reference arrival point 84 such as the ground in the virtual space where the parabolic arch reaches by reflecting the detected posture in the direction in the virtual space. be able to. It is also useful to set the parameters given to the parabolic arch. An example of this parameter determines the reach of the parabolic arch, which corresponds to the momentum of the water sprinkled by the hose. The arrival point 84 serves as a guide for the destination of the viewpoint movement. By using such a parabolic arch, it is possible to intuitively operate the destination and the distance to the destination. The operation is to align the horizontal direction with the direction you want to move, point the controller so that it has an elevation angle, change the elevation angle as if sprinkling water with a hose, adjust the distance, and change the above parameters as necessary. Including to do. The landing point 84 is a guideline for the operator 2 to understand that, for example, the movement destination (movement destination position) of the viewpoint movement is a position at a certain height directly above the movement destination (movement destination position). The height here is set to, for example, the same distance as the height of the viewpoint when the operator 2 takes a standing position, or is set to the same distance as the height of the viewpoint when the operator 2 takes a sitting position. Whether it is standing or sitting can be determined by whether the destination is outside the vehicle or on the seat. Further, the height can be set according to whether the operator 2 is standing or sitting in the real space. In the vehicle design support system 1, preferably, the forward direction in the virtual space that the operator sees after moving can be further set. For example, the direction display symbol 86 is graphically displayed at the landing point 84 of the curve symbol 82 of the parabolic arch. The direction indicated by the direction display symbol 86 can be changed, for example, by a button or dial operation (direction input unit) of the controller 120. When the operator 2 determines the landing point 84 by the curve symbol 82 and the direction display symbol 86 by some operation or button operation (direction determination unit), the controller 120 transmits a determination instruction. Then, the viewpoint is determined by the position of the head when standing or sitting according to the position set in the virtual space, and the determined horizontal component is set as the horizontal component that is the front of the line of sight. The correspondence between the space and the virtual space is reset. This completes the viewpoint movement.

The biometric information data 592 stored in the subject information recording unit 590 can include data acquired from a detection device other than the biometric information sensor 140 for acquiring the physical condition. One such data is event data of blinking or opening / closing of the eyelid detected by the eye tracking detection unit 110. This makes it possible to record changes in the state of consciousness that can be obtained from the eyes and eyelids over time. By using the time-dependent change data of the driver's consciousness state obtained from the detection device other than the biological information sensor 140 and the biological information sensor 140, for example, the interior design of the vehicle, the meter display, and the concrete design of the interior lighting can be the driver's consciousness. It is possible to investigate how it relates to the state without going through the actual trial production process.

Further, it is preferable that the vehicle design support system 1 adopts a function of synthesizing an external image of a real space in a virtual space. By displaying a person as a presenter in the virtual space in the design evaluation scene, it is possible to give a presentation in real time and provide the operator with accompanying information such as gestures and explanations by voice. When a person such as a presenter participates in the review, the image of the person in the real space taken by the colored screen for chroma key composition is captured by removing the color signal component that identifies the colored screen and displayed in real time. At that time, it is preferable to mount a tracker on the camera in order to extract the position and direction of the camera to be photographed in the real space and the position of the feature point in the photographed image of the camera. FIG. 12 is an explanatory diagram showing a state in which a presenter (person) is photographed by a camera 90 and a feature point 98 is set in the presenter by a tracker processing unit 94. The presenter 92 against the background of the colored screen 93 for chroma key processing is photographed by the camera 90. Normal video data taken by the camera 90 is output according to existing transmission standards (HDMI (registered trademark) and HD-SDI), and if necessary, can be converted to general-purpose communication standards such as Internet protocol or USB, or input / output bus standards. It is converted and transmitted to a PC (not shown in FIG. 12). At that time, the tracker processing unit 94 determines the position data of the camera 90 and the position data indicating the feature point 98 in the captured image and transmits the position data in the same manner. The tracker processing unit 94 may be an independent device or a functional means by arithmetic processing of the PC that functions by a program running on the PC. The posture of the camera 90 is determined relative to each other by using the camera posture sensor 91 of the tracker as needed. The feature point 98 is a point on the image that continuously extracts the feature points of the presenter 92. The feature point 98 is first specified by xy coordinates with the image plane as the xy plane, and the z coordinates (depth direction) can be included in the coordinates, if necessary. For example, using the focus or zoom encoding data of the camera 90 and the encoding data of the camera attitude sensor 91, the camera platform and the zoom lens (neither shown), the xyz coordinates of the image plane of the feature point 98 can be obtained. Can be decided. In this case, the camera attitude sensor 91, the pan head, and the encoder of the zoom lens (not shown) also form a part of the tracker together with the tracker processing unit 94. The xyz coordinates of the image of the point of interest 98 can be converted into the spatial coordinates of the real space. The position data of each feature point 98 can determine the coordinates in the real space from the data of the posture of the camera 90 from the above two encoders and the data of the position and orientation of the camera calculated from the camera posture sensor 91.

From the video data, the camera position data from the tracker, and the feature point position data, an image to be provided to the VR headset 10 by the image rendering unit 540 in the operator's computer 50 is generated by synthesizing the video. The captured video signal from the camera 90 is only for the presenter 92, which does not include a background due to chroma key processing. In order to synthesize this video signal with an image in the virtual space and use it as a real-time display material, the position data of each feature point 98 in the real space is converted into the position in the virtual space. This conversion is performed by the position conversion means 97 of any computer device. Then, the image is synthesized using the position data of the virtual space of each feature point 98. This video composition is executed by the video composition means 99 of any computer device. The image rendering unit 540 can also function as the video compositing means 99. By specifying the position of the camera 90 and the feature point 98 in the tracker processing unit 94, the camera 90 is moved to a position according to the design review, and even if the presenter 92 moves its own standing position, it is properly synthesized. , Presenter 92 can provide high quality accompanying information on the design. In particular, when the tracker processing unit 94 can specify the position of the camera 90 and the position of the feature point 98, for example, the presenter 92 stands behind an object (for example, an automobile) in the virtual space which is the subject of the review. It is possible to synthesize real-time video and provide accompanying information. In the chroma key composition that does not use the position of the camera 90 and the position of the feature point 98, even if there is a depth such as the interior of the vehicle and the background in the virtual space, the display is limited to the front. It is difficult to display the front and back of an object in the virtual space due to such restrictions. On the other hand, when the tracker processing unit 94 can specify the position of the camera 90 and the position of the feature point 98, even if the position of the presenter 92 changes due to movement or the like, the presenter is in front of the arrangement of the object in the virtual space. Since it can be easily determined by calculation whether the 92 should be displayed or the presenter 92 should be displayed on the background side, a natural display can be realized. When the chroma key composition of the presenter 92 becomes natural, it does not hinder the operator's concentration on the review work. The tracker in the present disclosure can be implemented by any method capable of acquiring feature point position data, and the relationship with the camera 90 described above, selection of the hardware device to be used, features in the device and software, and features in the device and software. The specific process for acquiring the point position data can be modified by a person skilled in the art at the implementation stage.

By using the vehicle design support systems 1, 1M, 1N, and 1A shown above, the design work of the interior of the vehicle is greatly streamlined. For example, by having a designer or design evaluator act as an operator of vehicle design support systems 1, 1M, 1N, and 1A, it is possible to predict and evaluate the finish of the interior of a vehicle with high accuracy without necessarily involving trial production. Can be done. In addition, since the passenger who only drives the vehicle becomes the operator of the vehicle design support systems 1, 1M, 1N, and 1A, the influence of the interior of the vehicle and any interface on the vehicle on the passenger is highly accurate. Can be predicted with.

Next, an exemplary hardware configuration and software for implementing the vehicle design support systems 1, 1M, 1N, and 1A described above will be described. An example of the VR headset 10 and the controller 120 can be, for example, VIVE Pro HMD (HTC). An example of a VR headset 10A with an eye tracking detector 110 can be VIVE Pro EYE (the company). The computer 50 can be any computer device, and for example, a PC equipped with WINDOWS (registered trademark, Microsoft Corporation) and equipped with a graphic card having three-dimensional rendering characteristics is suitable. As the biological condition sensor 140, for example, one of biosignals plus (PLUX-Wireless Biosignals SA) can be adopted. The information that can be acquired by the biological condition sensor 140 is diverse, and for example, the wakefulness state, the sleep state, and the health state can be determined by measuring brain waves, an electrocardiogram, a myoelectric potential, and the like. As the structure data 512 and the attribute data 516, various CAD data and virtual prototyping data can be adopted. Data for 3D visualization software such as VRED (AUTODESK) is typical. In particular, it is also useful that the attribute data 516 is texture data determined by measurement or the like. In addition, VRED can synthesize the operator's two-handed image on the display 102 in combination with VIVE Pro EYE, and can give the operator a realistic feeling to the experience in the virtual space. It is also possible to input by gesture using the two-handed image of the operator. The image rendering unit 540 is realized by various tools that use the graphic card included in the computer 50 for polygon processing and rendering processing, and a game engine such as UNREAL ENGINE (Epic Games) is typically suitable. The background data 522 can be an image of a hemisphere or a spherical image, and in that case, a still image or a moving image can be taken and prepared by using a camera such as INSTA360 Pro (INSTA360). .. In addition to this, it is also useful to mount various digital cameras on a pan head (such as EPIC Pro of GIGA PAN) to shoot higher-definition still images.

The characteristics that can be evaluated by the vehicle design support systems 1, 1M, 1N, and 1A of the present embodiment are arbitrary attributes that can be reproduced by combining the attribute data 516 with the structural data 512 and can stimulate the operator visually. Reproduced to have the specified surface color, texture, and surface reflection characteristics. Note that the structure can also be part of the attributes that can be selected or adjusted.

The structural data is typically arbitrary data that can identify the shape of a polygon, wire frame, or the like. If it is a polygon, it is useful that it is adjusted in advance so that an appropriate surface structure can be reproduced by inversion adjustment of the normal (normal).

The surface color for the attribute data can be adjusted to any attribute that specifies the object color for a reflective object and the light source color for a light emitter. The texture of the surface can be adjusted using texture mapping and normal mapping. Surface data is typically used in the field of CG called textures and materials. Even if the surface data is precisely adjusted in position with the structural data by UV development in advance, there is still room for further adjustment in order to match the designer's intention. A texture refers to an image or data that expresses a pattern, unevenness, etc. in addition to the texture. The material is used to identify the texture of the changed material of the object. The parameters adjusted for the texture are, for example, Diffuse used to set a unique color, Normal used to give unevenness to a smooth expression, Ambient Occlusion to indicate how much indirect lighting is received, and gloss. Special, which expresses the smoothness of the surface, and Rohness, which expresses the smoothness of the fine surface, can be adopted. By combining these, the designer can infer the interior by precisely adjusting the texture such as the so-called textured surface.

In the present embodiment, for example, because of the display device area 310 (FIG. 2) such as the design of the meter display, the image or video that becomes the graphic information of the display device, and the display sign (including the case where the meter display is performed only by the graphic display). The attribute of is also treated as attribute data 516. Meters that show information such as speed to the driver, and navigation devices display graphic information on the display device to present necessary information to the driver and the like. The graphic display is also treated as attribute data and is subject to selection and adjustment because its visibility and design are subject to evaluation.

For the precise design of the vehicle interior, it is important to reproduce the visual representation of the interior as accurately as possible in the virtual space. Therefore, it is necessary to pay attention to the lighting. In the real space, not only the light source but also the reflective object emits the reflected light, which affects the lighting. In particular, when the background 400 or the vehicle interior 300 has image information in a high dynamic range image (HDR), the light source portion and each portion indirectly illuminating by reflection are used by the image-based triting (IBL) method. Since it can be handled in a unified manner, it is possible to improve the reproducibility of lighting. Further, even when the outside light is illumination having a high luminous intensity such as direct sunlight, the effect can be reflected in the presented image in the present embodiment. For example, for sunlight, strong light irradiation from a specific direction can be reproduced by a method such as directional lighting. Due to the pointer function and flash light function described above, it is also useful to allow the operator 2 to operate such a strong light irradiation direction through the controller 120.

The reflection is important in three ways in terms of reproducing the interior of the vehicle. The first is important for verifying that the light source reflected on the surface of the window or meter becomes visual noise as a problem peculiar to the vehicle. Excessive visual noise for the driver when part of the interior is illuminated or when strong illumination light is incident from the outside at night should be suppressed or avoided in the interior design. be. For example, since a window portion such as a window screen exhibits a mirror-like surface reflection characteristic due to the surface reflection of glass, it is preferable to evaluate the window portion with realistic lighting by reflecting the surface reflection as a part of the interior. In particular, when using an illumination light source as a light source or a display device that emits light in a vehicle, the reflection by the light must be carefully examined. In such a case, the numerical value that determines the display itself of the illumination light source or the display device also becomes the attribute data 516, and is an object to be selected or adjusted. The second reason is that directing lighting such as welcome light when riding and lighting to prevent erroneous operation at the feet at night has been widely used in recent years, and it is necessary to design the lighting with more consideration for safety than ever before. This is because the sex is increasing. Illumination sources designed include interior lights and self-luminous meters, and their effects on the driver's vision will be investigated. The attributes that identify these light sources include light intensity, color, and light distribution characteristics such as point light or spot light. The third reason why the reflection is important is that the actual feeling of the interior itself is affected by the reflection such as gloss. For example, if there is a glossy object in a part of the interior that is in the passenger's view (for example, the outer circumference of the meter cluster), the actual feeling of the object is accurate by reproducing the reflection. Can give a good impression. Therefore, the numerical value that determines the gloss becomes the attribute data 516 and is selected or adjusted.

The image rendering unit can execute image rendering by late lacing or ray casting by using a graphic card or the like as a part of its function. In the background image 402, distortion may be felt only by cutting out a region of the hemisphere from the actual omnidirectional image and mapping it on the inner surface of the hemisphere imitating the celestial sphere. In particular, when a building or the like is included in the background, a building in a distant view at a height near the line of sight may appear distorted in a bow shape. In such a case, it is useful to map to the cylinder and adjust the UV development at a certain height from the horizon.

The embodiments of the present disclosure have been specifically described above. The above-described embodiments, modifications and examples are described for explaining the invention disclosed in the present application, and the scope of the invention of the present application should be determined based on the description of the scope of the patent claim. It is a thing. Modifications that exist within the scope of the present disclosure, including other combinations of embodiments, are also within the scope of the claims.

1, 1M, 1N, 1A Vehicle design support system 10, 10A VR headset 102 Display 104 Attitude sensor mover 106 Attitude sensor stator 110 Eye tracking detector 112 Eye tracking signal 120 Controller (operation input unit)
122 Signal processing unit 140 Biological condition sensor (biological condition detection device)
142 Signal processing unit 2 Operator 22 Head 300 Vehicle interior 302 Interior image 310 Display device area 400 Background 402 Background image 50, 50A, 50B, 50C Computer 50S Data server 510 Interior data recording unit 512 Structural data 514 Attribute candidate data 516 Attribute data 520 Environmental data recording unit 522 Background data 530 Image adjustment unit 540 Image rendering unit 542 Presented image 550 Gaze point judgment unit 560 Gaze point position recording unit 562 Gaze point position data 580 Gaze point aggregation unit 590 Subject information recording unit 592 Biological information data 60 Server Execution Department 602 Account Management Department 604 Session Management Department 66 Client Execution Department 68 Network 702, 704, 706, 708 Heat Display Unit 82 Curve Symbol 84 Landing Point 86 Direction Display Symbol 90 Camera 91 Camera Attitude Sensor 92 Presenter 94 Tracker Processing Department 96 Display 97 Position conversion means 98 Attention point 99 Video composition means

Claims (21)

An interior data recording unit that stores structural data for determining the structure of the vehicle interior in the virtual space and attribute candidate data for selecting or adjusting attributes for at least a part of the vehicle interior.
A VR headset that is at least one virtual reality (VR) headset worn by the operator on the head and includes a posture sensor capable of detecting the posture of the operator's head and a display capable of displaying a presented image.
At least one operation input unit that receives an operation input for selecting or adjusting the attribute for at least a part of the vehicle interior from the operator.
An image adjustment unit that generates attribute data by selecting or adjusting the attribute candidate data for attributes of at least a part of the vehicle interior according to the instruction content of the operation input received by the operation input unit.
From the structural data called from the interior data recording unit and the attribute data from the image adjustment unit, an interior image corresponding to the instruction content of the operation input and the direction of the operator's head in the virtual space. A vehicle design support system including an image rendering unit that generates an image and outputs the image as an image signal for the presented image. Further provided with an environment data recording unit for storing background data for the background of the surrounding environment in which the vehicle is arranged in the virtual space.
The image rendering unit calls the background data from the environment data recording unit and generates the interior image by synthesizing the background image in the direction corresponding to the window portion of the vehicle interior in the virtual space. The vehicle design support system according to claim 1. A plurality of the VR headsets, one for each of the plurality of operators,
It is provided with a plurality of image rendering units for each of the plurality of operators, and each image rendering unit generates an interior image corresponding to the direction of the head of each operator in the virtual space.
The vehicle design support system according to claim 1 or 2, wherein an interior image for each operator is displayed as a presentation image on the display of the VR headset worn by each operator. An interior data recording unit that stores structural data for determining the structure of the vehicle interior in the virtual space and attribute data for determining the attributes of at least a part of the vehicle interior.
At least one virtual reality (VR) headset worn by the operator on the head, a posture sensor capable of detecting the posture of the operator's head, a display capable of displaying a presented image, and at least one eyeball of the operator. A VR headset equipped with an eye tracking detector that outputs an eye tracking signal that can identify the line of sight by
From the structure data and the eye tracking signal, a gaze point determination unit that determines a position in the virtual space where the line of sight intersects the structure as a gaze point and outputs it as gaze point position data.
A gazing point position recording unit that stores the gazing point position data and
An image rendering unit that generates an interior image corresponding to the direction of the operator's head in the virtual space from the structural data and the attribute data called from the interior data recording unit, and generates an image signal of the interior image. Vehicle design support system with and. The eye tracking detection unit outputs a binocular eye tracking signal that can separately identify the line of sight of both eyes of the operator.
The gazing point determination unit determines the intersection position of the line of sight of both eyes from the binocular eye tracking signal and outputs it as line-of-sight intersection position data.
The vehicle design support system according to claim 4. The image rendering unit generates at least one selected from a visualization image group consisting of a heat map image, a gaze trace image, and a gaze cluster image from the gaze point position data called from the gaze point position recording unit. The vehicle design support system according to claim 4 or 5, which generates an image signal to be displayed overlaid on the interior image. The gaze point position data is called from the gaze point position recording unit, and the frequency or time of the gaze point indicated by the gaze point position data is totaled for each feature point of the structure of the vehicle interior or for each area delimited by the virtual space. The vehicle design support system according to any one of claims 4 to 6, further comprising a viewpoint aggregation unit. Further provided with an environment data recording unit for storing background data for the background of the surrounding environment in which the vehicle is arranged in the virtual space.
The image rendering unit calls the background data from the environment data recording unit and generates the interior image by synthesizing the background image in the direction corresponding to the window portion of the vehicle interior in the virtual space. And
The vehicle design support system according to claim 4, wherein the gazing point determination unit determines a position in the background image in the virtual space as a gazing point and outputs it as gazing point position data. A plurality of the VR headsets, one for each of the plurality of operators,
It has a plurality of image rendering units for each of the plurality of operators.
Each image rendering unit corresponds to the instruction content of the operation input by at least one of the plurality of operators from the structural data called from the interior data recording unit and the attribute data from the image adjusting unit. It generates an interior image corresponding to the direction of each operator's head in the virtual space.
The interior image generated for each operator is displayed as a presentation image on the display of the VR headset worn by each operator.
Any one of claims 4 to 8, wherein the gaze point position data generated by the gaze point determination unit and stored in the gaze point position recording unit is for either a VR headset or an operator. Vehicle design support system described in. The vehicle interior has a display device area capable of presenting graphic information in at least a part thereof.
The interior data recording unit stores graphic information to be displayed in the display device area.
The vehicle design support system according to claim 1 or 4, wherein the image rendering unit synthesizes the graphic information with at least a part of the presented image to generate the interior image. The attributes include surface reflection properties for at least a portion of the vehicle interior.
The image rendering unit generates the interior image reflecting the surface reflection reflected by at least a part of the vehicle interior in response to the incident light, and outputs it as an image signal for the presented image. 1 or the vehicle design support system according to claim 4. The attributes include the position and emission characteristics of the light source located in the vehicle interior.
The vehicle design support system according to claim 11, wherein the incident light includes light emitted by the light source. A biological condition detection device that acquires the physical condition of any of the operators and outputs a biological information signal,
The vehicle design support system according to any one of claims 1 to 12, further comprising a subject information recording unit that stores the biometric information data of the operator from the biometric information signal. The eye tracking detection unit can acquire an event of blinking or opening / closing of the eyelid of the operator.
The vehicle design support system according to claim 13, wherein the biometric information data includes data of the event. The computer device used by the administrator who is one of the plurality of operators includes an account management unit and a session management unit.
The account management unit includes a permission table that determines what each user can access about the functions and data files of the vehicle design support system.
The session management unit holds a list of users who can participate in the session as a session table, determines whether each of the plurality of operators is a legitimate user by referring to the session table, and determines whether or not each of the plurality of operators is a legitimate user. The vehicle design support system according to claim 3 or 9, which establishes a connection for the session for our legitimate user. The computer device used by the user serves as a client terminal when the computer device used by the one operator is used as a server device, and can communicate through the network.
The vehicle design support system according to claim 15, wherein a user ID for identifying the user is used to establish the session through the network. It also has a sensor that detects the direction of the controller itself.
The image rendering unit displays a curve symbol in the virtual space determined according to the direction in the virtual space while updating it according to the direction of the controller by the output of the sensor.
Claim 1 or claim 4 in which the viewpoint of the operator is reset according to the movement destination position when the determination input of the arrival position of the curve symbol in the virtual space is received from the operator who operates the controller. Vehicle design support system described in. The controller includes a direction input unit and a direction determination unit.
The image rendering unit displays a direction display symbol indicating the direction after movement at the movement destination position according to the direction indicated by the direction input unit.
At least one component of the line-of-sight direction of the operator at the movement destination position is reset according to the movement post-movement direction determined in response to the input to the direction determination unit.
The vehicle design support system according to claim 17. A camera that can output video signals and
A tracker that determines the position of the feature point with respect to the imaging target of the camera,
A position conversion means for converting the feature point position into a position in the virtual space,
An image synthesizing means is further provided, and an image obtained by the image signal from the camera exists at a position in the virtual space converted by the position conversion means based on the feature point position data determined for the imaging target. The vehicle design support system according to claim 3 or 11, which is synthesized by the video synthesizing means as described above. A computer equipped with an arithmetic unit, a recording unit, and a graphic unit,
At least one virtual reality (VR) headset connected to the computer, which is worn by the operator on the head, including a posture sensor capable of detecting the posture of the operator's head and a display capable of displaying a presented image. VR headset and
A computer program that realizes the vehicle design support system according to any one of claims 1 to 19 by operating a computer system that is connected to the computer and includes at least one operation input unit that receives operation input from the operator. And
A computer program that causes the arithmetic unit, the recording device, and the graphic device of the computer to function as the interior data recording unit, the image adjustment unit, and the image rendering unit. A computer equipped with an arithmetic unit, a recording unit, and a graphic unit,
At least one virtual reality (VR) headset connected to the computer, which is worn by the operator on the head, including a posture sensor capable of detecting the posture of the operator's head and a display capable of displaying a presented image. VR headset and
A computer program that realizes the vehicle design support system according to any one of claims 4 to 19 by operating a computer system that is connected to the computer and includes at least one operation input unit that receives operation input from the operator. And
A computer program that causes the arithmetic unit, the recording device, and the graphic device of the computer to function as the interior data recording unit, the gazing point determination unit, the gazing point position recording unit, and the image rendering unit.

Images