JP2022520685A - Methods and devices for adjusting the reaction system based on the sensory input and the vehicle incorporating it - Google Patents

Abstract

Traditional vehicles typically behave like a single rigid body with fixed properties, as defined during the vehicle design phase. The rigid nature of conventional vehicles limits their ability to adapt to different driving conditions, and therefore their usefulness and performance. To overcome these limitations, highly responsive vehicles may be used, including sensors and reaction systems. Sensors may monitor the driver's position and / or orientation, the driving conditions of the vehicle, and / or the environmental conditions around the vehicle. The reaction system may adjust some aspect of the vehicle based on the data acquired by the sensors. For example, the reaction system may include a video-based mirror with a field of view that changes based on the driver's movements. In another example, the reaction system may include joint joints that change the physical configuration of the vehicle based on the driver's movements. [Selection diagram] Fig. 1

Description

Mutual reference to related patent applications This application claims the priority of US Application No. 62 / 664,656, entitled "ARTICULATED VEHICLE" filed April 30, 2018, on April 30, 2019. It is a partial continuation (CIP) application of the international application No. PCT / US2019 / 029793 entitled "ARTICULATED VEHICLES WITH PAYLOAD-POSITIONING SYSTEMS". This application also claims the priority of US application No. 62 / 745,038, entitled "APPARATUS FOR A REACTIVE CAMERA MONITORING SYSTEM AND METHODS FOR THE SAME" filed on October 12, 2018. Each of these applications is incorporated herein by reference in its entirety.

A vehicle driven by a person (eg, a vehicle) is usually controlled by a driver located in the cabin of the vehicle. In order to drive a vehicle safely, the driver should preferably be aware of objects near the vehicle (eg, people, road barriers, other vehicles). However, the visual field (FOV) of the driver's surrounding environment is mainly limited to the area in front of the driver due in part to the limitation of the peripheral visual field of the human eye. Therefore, the driver usually at the expense of moving the driver's FOV from the direction of travel of the vehicle in order to see the surroundings of the vehicle (eg, to see the blind spot when changing lanes). Alternatively, the head should be moved to move the FOV. The driver's FOV is an obstacle in the vehicle cabin, such as a cabin structure (eg, door panel, window size, A, B, or C pillar) or an object in the cabin (eg, another passenger, large cargo). Can be further restricted by.

Conventional vehicles typically include a mirror to extend the driver's FOV. However, the increase in driver's FOV is limited. For example, conventional automotive mirrors typically provide a moderate FOV to reduce distance distortion and focus the driver's attention on a particular area around the vehicle. At normal viewing distances, the horizontal FOVs of mirrors used in automobiles are typically 10-15 ° for driver-side mirrors, 23-28 ° for center (inside) mirrors, and 20 for passenger-side mirrors, respectively. It is in the range of ~ 25 °. Moreover, conventional vehicles are primarily single rigid bodies while driving. Therefore, the FOV of the cabin is determined primarily during the design phase of the vehicle and therefore cannot be easily reconstructed after manufacture without expensive and / or time consuming modifications.

The embodiments described herein are directed to vehicles that include a reaction system that partially responds to changes in the position and / or orientation of the driver (also referred to as the "driver"). (A vehicle with a reaction system may be referred to as a highly responsive vehicle.) For example, when the driver moves his or her head, the reaction system may adjust the driver's FOV. This can be done by physically activating the vehicle's joint joints to change the driver's position with respect to the environment, or by adjusting the video image of the outer region of the vehicle that is displayed to the driver. It can be achieved by several means. In this way, the reaction system expands the driver's FOV, thus allowing the driver to remain aware of the vehicle's direction of travel, while at the same time giving the driver a better situational awareness of the vehicle's surroundings. Can provide awareness. The reaction system may also allow the driver to look around and / or ahead of the object by adjusting the position of the camera on the vehicle or the vehicle itself, which is not possible with conventional vehicles.

In one aspect, the driver's position and / or orientation can be measured by one or more sensors coupled to the vehicle. Sensors may be configured to capture different types of data associated with the driver. For example, the sensor may include a camera for acquiring an RGB (red, green, blue) image of the driver and a depth map sensor for acquiring a depth map of the driver. RGB images and depth maps may be used to determine the coordinates of various facial and / or postural features associated with the driver, such as eye reference points on the driver's head. The coordinates of the various features of the driver are measured as a function of time and can be used as inputs to activate the reaction system.

Using different data types to determine driver characteristics can reduce the occurrence of false positives (ie, detect false features) and enable feature detection under different lighting conditions. .. Detection of these features can be achieved using several methods, such as convolutional neural networks. A motion filtering system (eg, a Kalman filter) is also used to ensure that the driver's measured features change smoothly as a function of time, for example by reducing unwanted jitter in the driver's RGB image. You may. The depth map may also be used with the RGB image by some means. For example, the depth can mask the RGB image so that a smaller portion of the RGB image is used for feature detection, thereby reducing computational costs.

One or more sensors can also measure various environmental conditions such as road surface type, vehicle speed and acceleration, obstacles near the vehicle, and / or the presence or absence of precipitation. The measured environmental conditions may also be used as an input to the reaction system. For example, depending on the environmental conditions, the magnitude of the response of the reaction system may be modified (eg, the height of the vehicle) based on the speed of the vehicle (eg, maneuvering in the city as opposed to maneuvering on the highway). .. In some cases, environmental conditions are also a reaction system to keep the driver and vehicle safe when certain conditions are met (eg, vehicle speed, turning speed, wheel traction). It may be used as a restricted gate that may prohibit activation.

The reaction system may include a video-based mirror assembled using a camera and display. The camera may be coupled to the vehicle and oriented to capture a video image of the area outside the vehicle (eg, the rear of the vehicle). The display can be used to show the driver a video image of that area. As the driver moves, the video image shown on the display may be converted to adjust the FOV of the area captured by the camera. For example, the driver rotates his head and shifts the video image accordingly (eg, by panning the camera or moving a portion of the video image shown on the display). It can emulate a response similar to a conventional mirror. The reaction system may include multiple cameras such that the collective FOV of the cameras substantially covers the perimeter of the vehicle, thus reducing or reducing the driver's blind spot when driving the vehicle. Exclude in some cases. Video images acquired by multiple cameras may be displayed on one or more displays.

The reaction system may include joints for physically changing the configuration of the vehicle. Joint joints adjust the vehicle's tilt / height and / or hinges that change the vehicle's body shape (eg, rotate the vehicle's front relative to the vehicle's tail), such as the vehicle's active suspension. It may include one or more mechanisms. In one example, the joint joint moves the first part of the vehicle along the path with a guide structure that defines the path through which the first part of the vehicle is movable with respect to the second part along the path. It may include a drive actuator to cause the vehicle to hold a first portion of the vehicle at a particular position along the path.

Joint joints may be used to correct the driver's position with respect to the environment. For example, the reaction system may allow the driver to tilt his or her head and tilt the vehicle using joints when looking around an object (eg, another vehicle). In another example, the reaction system may increase the vehicle height as the driver aligns his head upwards to look over an object (eg, a barrier). In such cases, the reaction system may be configured to actuate the joint joints in a manner that does not compromise vehicle stability. For example, the reaction system may reduce the magnitude of the actuation, or in some cases may prevent the joint joints from actuating when the vehicle is traveling at high speed. The reaction system may also activate the joint joint in conjunction with an explicit driver command (eg, a command received from an input device such as a steering wheel, accelerator, brake, etc.).

Another way to drive a (reactive) vehicle is to use the first sensor to receive the first input from the vehicle driver and the second sensor to use the environment outside the vehicle. Includes receiving a second input from. The processor identifies the correlation between the first and second inputs and generates behavior-based commands based on that correlation. When this behavior-based command is applied to the vehicle's actuators, it moves the vehicle in a predefined manner. The processor produces a combined command based on behavior-based commands, explicit commands from the driver via an input device operably coupled to the processor, and a second input. The combined commands are then adjusted and / or filtered to maintain the stability of the vehicle, and then the adjusted and / or filtered combined commands are used to activate the vehicle actuators.

The example reaction system above is described in the context of modifying the FOV of the driver and / or camera, but the reaction system and the various components within it can also be used for other applications. For example, the reaction system may be used as a security system for vehicles. A reaction system may recognize and grant access to an authorized individual's vehicle while blocking access to other individuals (eg, by activating the vehicle to prevent intrusion). In another example, the reaction system causes the vehicle to make a sound through the joints (eg, in a parking lot that accommodates multiple vehicles) so that the driver can easily find the vehicle (eg, in a parking lot that accommodates multiple vehicles). For example, you may honk) and / or turn on / blink the headlights. In another example, the vehicle may have an autonomous driving mode in which the reaction system is configured to instruct the vehicle to follow a driver located outside the vehicle. It can be used, for example, to record a video image of the driver as he travels through the environment. In another example, the reaction system may adjust the driver's position (eg, by a joint) to reduce the glare on the driver's eye area.

All combinations of the above concepts and the additional concepts discussed in more detail below (assuming such concepts are not inconsistent with each other) are part of the subject matter of the invention disclosed herein. It is believed that there is. In particular, all combinations of claims appearing at the end of this disclosure are considered to be part of the subject matter of the invention disclosed herein. Also, of course, the terms explicitly used herein, which may also appear in any disclosure incorporated by reference, should be given the meaning that most closely matches the particular concepts disclosed herein. Is.

Those skilled in the art will appreciate that the drawings are primarily exemplary purposes and are not intended to limit the scope of the subject matter of the invention described herein. The drawings are not necessarily in constant proportions, and in some embodiments, various aspects of the subject matter of the invention disclosed herein are exaggerated or enlarged in the drawings to facilitate understanding of the different features. May be shown. In drawings, similar reference characters generally mean similar features (eg, functionally similar and / or structurally similar elements).
FIG. 1 shows an articulated vehicle articulated so as to shift the driver's field of view in response to the light of a headlight from an oncoming vehicle. FIG. 2 shows a coordinate system in which the origin is centered on the driver's head. FIG. 3 shows a seat with a calibration feature for a highly responsive vehicle system. FIG. 4 shows an exemplary reactive mirror of a vehicle. FIG. 5A shows the various components of the reactive mirror of FIG. 4 placed in and on a conventional vehicle and the field of view (FOV) of each camera. FIG. 5B shows the various components of the reactive mirror of FIG. 4 placed in and on the articulated vehicle and the FOV of each camera. FIG. 6 shows a method of acquiring and transforming a video image acquired by the camera of the reactive mirror of FIG. 4 based on the driver's position and / or orientation. FIG. 7A shows a side sectional view of an exemplary vehicle with articulated joints. FIG. 7B shows a side view of the vehicle of FIG. 7A. 7C shows a top view of the vehicle of FIG. 7B. FIG. 7D shows a side view of the vehicle of FIG. 7B in a low external configuration with the outer shell of the tail removed. FIG. 7E shows a side view of the vehicle of FIG. 7B in a high external configuration with the outer shell of the tail removed. FIG. 8A shows a perspective view of an exemplary joint joint of a vehicle. FIG. 8B shows a side view of the joint joint of FIG. 8A. FIG. 8C shows a side perspective view of the upper surface of the joint joint of FIG. 8A. FIG. 8D shows a side perspective view of the bottom surface of the joint joint of FIG. 8A. FIG. 8E shows a top side perspective view of the lift bracket and track system in the guide structure of FIG. 8A. FIG. 8F shows a side perspective view of the upper surface of the orbital system of FIG. 8E. FIG. 8G shows a cross-sectional view of a rail bearing in the track system of FIG. 8F. FIG. 9 shows a flow chart of a method for operating a vehicle reaction system. FIG. 10A shows various input parameters associated with the driver controlling the vehicle and an exemplary range of input parameters when the vehicle is turning. FIG. 10B shows various input parameters associated with the environment surrounding the vehicle and an exemplary range of input parameters when the vehicle is turning. FIG. 11A shows the displacement of the articulated vehicle along the articulated joint axis as a function of the driver's position where the limit to displacement is adjusted to maintain stability. FIG. 11B shows the displacement of the articulated vehicle along the joint axis as a function of the driver's position, which adjusts the rate of change of displacement to maintain stability. FIG. 12A shows an articulated vehicle equipped with sensors to monitor the position of a second vehicle using video images and depth maps acquired by the sensors. FIG. 12B shows an articulated vehicle of FIG. 12A that is tilted, thereby changing the position of the second vehicle as measured with respect to the sensors on the articulated vehicle. FIG. 13 shows an articulated vehicle whose vehicle height is adjusted to increase the FOV of the driver and / or the sensor. FIG. 14A shows an articulated vehicle whose FOV is limited due to the presence of a second vehicle. FIG. 14B shows the articulated vehicle of FIG. 14A, tilted to look around the second vehicle. FIG. 15A shows a top view of the articulated vehicle and a FOV of the articulated vehicle. FIG. 15B shows a front view of the articulated vehicle and the FOV of the articulated vehicle of FIG. 15A. FIG. 15C shows a side view of the articulated vehicle of FIG. 15A, which climbs a series of stairs in a zigzag manner. FIG. 16 shows an articulated vehicle that identifies a person approaching the vehicle and, if necessary, reacts to prevent the person from accessing the articulated vehicle. FIG. 17 shows an articulated vehicle that acquires a video image of a driver located outside the articulated vehicle.

The following is a more detailed description of reactive vehicle systems, reactive mirror systems, articulated vehicles, and various concepts related to the aforementioned usage, and their implementation. The concepts introduced above and discussed in more detail below can be implemented by multiple means. Examples of specific implementations and applications are provided primarily for illustrative purposes to allow those skilled in the art to practice implementations and alternatives that are apparent to those of skill in the art.

The figures and examples described below do not imply limiting the scope of this implementation to a single embodiment. Other implementations are possible by exchanging some or all of the elements described or illustrated. In addition, if certain elements of the disclosed implementation examples can be partially or completely implemented using known components, in some cases such known components required for understanding this implementation. Only those parts are described and detailed description of other parts of such known components is omitted in order not to obscure the present implementation.

The following discussion describes various examples of vehicles, reaction systems, reactive mirrors, and joint mechanisms. One or more features discussed with respect to a given example may be that the various features disclosed herein are readily combined in a given system according to the present disclosure (provided that the features are consistent with each other). As in), it may be used in other embodiments according to the present disclosure.

Vehicles with Sensors and Reaction Systems FIG. 1 shows a (articulated) vehicle 4000 with a vehicle body 4100. One or more sensors, including the external camera 4202 and the internal camera 4204, include the attitude and / or orientation of the driver (eg, driver 4010), the operating parameters of the vehicle 4000 (eg, speed, acceleration, wheel traction), and. It may be attached to the vehicle body 4100 to measure various inputs associated with the vehicle 4000, including but not limited to environmental conditions (eg, ambient light). The reaction system (shown as the joint joint 4300 in FIG. 1) modifies some aspect of the vehicle 4000 based in part on the inputs measured by the sensors 4202 and 4204 (eg, modifies the FOV of the driver 4010, variable terrain. May be coupled to the vehicle 4000 so as to cross). In FIG. 1, for example, the reaction system 4300 articulates the vehicle so as to move the user's head from the optical path of the headlights of the oncoming vehicle, as detected by the external camera 4204. The vehicle 4000 also manages the transfer of data and / or commands between the sensors 4202 and 4204, as well as the reaction system 4300, as well as the various components of the vehicle 4000 and their respective subsystems of the vehicle 4000. Processor (not shown) may be included.

The reaction system 4300 may include or be coupled to one or more sensors to acquire various types of data associated with the vehicle 4000. For example, the interior camera 4204 may acquire both the vehicle 4000 cabin and / or the driver 4010 depth and red-green-blue (RGB) data. Each pixel in the depth frame can represent the distance between the object to the pixel and the heated source of the depth map sensor. Depth frames can be acquired using structured infrared (IR) projection and two cameras in a stereo configuration (or similar depth capture). The depth frame is used to generate a depth map representation of the driver 4010 and the vehicle cabin. RGB frames may be acquired using a standard visible light camera. Other types of data acquired by the sensor 4200 may include, but are not limited to, the driver's heart rate, the driver 4010's gait recognition, and the face recognition.

External cameras 4202 and / or other sensors, including inertial measurement units or gyroscopes, include vehicle 4000 orientation, vehicle 4000 speed, suspension movement, acceleration speed, road surface topology, precipitation, day / night sensing, road. Various vehicles including, but not limited to, surface types (eg, smooth pavement, rough pavement, gravel, mud), other objects / obstacles near vehicle 4000 (eg, another car, person, barrier). It may be configured to acquire parameters and / or environmental conditions. The operating frequency of these sensors may be at least 60 Hz, and preferably 120 Hz.

Various operating parameters associated with each sensor may be stored, including sensor-related unique parameters (eg, resolution, dimensions), and extrinsic parameters (eg, internal camera 4204 within the vehicle 4000 coordinate space). Position and / or orientation), but not limited to these. The operating parameters of each sensor can be used to convert between the local coordinate system associated with that sensor and the vehicle coordinate system. For reference, the coordinate system used herein may be a right-handed coordinate system based on the International Organization for Standardization (ISO) 16505-2015. In this coordinate system, the positive x-axis is oriented in the direction opposite to the direction of forward movement of the vehicle 4000, the z-axis is pointing upwards orthogonal to the ground plane, and the y-axis is forward. Point to the right when looking in the direction of movement to.

A processor (also referred to herein as a "microcontroller") is used to process the input data captured by the sensor (eg, noise filtering, combining data from different sensors), and compute the transformation. And / or generating commands to modify the reaction system 4300, and communicably coupling various subsystems of the vehicle 4000 (eg, an external camera 4204 to the reaction system 4300). Various functions not limited to these may be performed. For example, the processor can be used to determine the position and / or orientation of the driver 4010 and generate an image transformation that is applied to the video image. Processors can generally constitute one or more processors that are communicably coupled together. In some cases, the processor may be a field programmable gate array (FPGA).

As mentioned above, the internal camera 4202 may detect the position and / or orientation of the driver 4010 (eg, the driver's head or body) in vehicle coordinate space. In the example below, the internal camera 4202 acquires both the depth and RGB data of the driver 4010. Prior to feature detection, the processor uses the pixel coordinates of either RGB or a frame of depth data to access the corresponding color or depth data in the RGB image and depth acquired by the internal camera 4202. The frame may be aligned first. Depth map processing typically uses less computational resources than RGB frame processing. In some cases, depth maps may be used to limit and / or mask areas of RGB frames for processing. For example, a depth map can be used to extract a portion of an RGB frame corresponding to a depth range of about 0.1 m to about 1.5 m for feature detection. By reducing the RGB frames in this way, not only the occurrence of false positives can be reduced, but also the computational power used to process the RGB frames can be significantly reduced.

Feature detection may be achieved by several means. For example, a pre-trained machine learning model (eg, a convolutional neural network) can utilize depth, RGB, and / or combined (RGBD) data to detect features of the driver 4010. The output of the model may include pixel areas corresponding to body, head, and / or facial features. The model may also provide an estimate of the driver's posture. In some cases, when the processor 4400 identifies the driver's head, the processor 4400 subsequently identifies the reference point of the driver's eyes (eg, the midpoint between the driver's eyes, as shown in FIG. 2). ) Can be estimated. The reference point of the eye may then be de-projected and transformed into coordinates within the vehicle reference frame. As described, it may be a software construct that detects features, so the model used for feature detection is at the time of manufacture to incorporate advances in computer vision and / or to improve performance. May be updated after.

The sensor (eg, internal camera 4202) and reaction system 4300 may also be calibrated for the driver 4010. In general, the height and location of the driver (eg, different driving positions) within the cabin of the vehicle 4000 can change over time. If the vehicle 4000 is not specifically calibrated for the driver 4010, changes in the driver's position and orientation will cause the reaction system 4300 to properly adjust the vehicle 4000 to assist the driver 4010. I can't. The driver 4010 includes, but is not limited to, pressing a physical button, selecting a calibration option (eg, an infotainment system) on the control console of the vehicle 4000, and / or using voice commands. The calibration mode may be activated using the various inputs of the vehicle 4000.

In general, calibration may be divided into groups regarding (1) the driver's physical position and movement, and (2) the driver's individual preferences. Calibration related to the driver's physical position and movement includes the driver's initial sitting position and one point of the driver's normal eye while driving the vehicle 4000 within the vehicle coordinates within the vehicle 4000, and the driver. It may include establishment with the driver's operating range, which affects the response range of the response system 4300 to changes in the position of the head of the vehicle. The sensor 4200 may be used to capture the driver's physical position and movement, and the resulting eye reference point is stored for later use when activating the reaction system 4300. be able to.

During calibration, the driver 4010 may be instructed to move his body in a particular manner. For example, audio or visual prompts from the vehicle's speakers and display may prompt the driver 4010 to sit normally, move to the right, or move to the left. The processor records the reference point of the eye at each position to establish the initial position and range of motion. Prompts may be delivered to the driver 4010 by a number of means, including but not limited to, visual cues and / or commands shown in the vehicle infotainment system, as well as voice commands from the vehicle speakers. The eye reference point may be recorded in terms of the vehicle 4000 coordinate system so that the processor can use the eye reference point as an input to the various components of the reaction system 4300.

The internal camera 4202 may also be calibrated for the seat in the vehicle 4000, thereby providing a more standardized standard for finding the internal camera 4202 (and driver 4010) in the vehicle 4000. Can be. FIG. 3 shows a seat 4110 including a calibration pattern 4120 detected by the sensor 4200. The shape and design of the calibration pattern 4120 may be known in advance. These may be printed with visible or invisible inks (eg, inks that are visible only at near-infrared wavelengths). Alternatively or additionally, the seat 4110 may have a characteristic shape or feature (eg, asymmetric feature) that can be used as a reference marker for calibration. By imaging the calibration pattern 4120 (and seat 4110), the relative distance and / or orientation of the sensor 4200 with respect to the seat can be found. In some cases, the calibration pattern 4120 is formed at a visible wavelength (eg, directly observable by the human eye) or an infrared wavelength (eg, invisible to the human eye and detectable using only an infrared imaging sensor). Can be done.

Calibration related to the driver's personal preference may vary based on the type of reaction system 4300 used. For example, the reaction system 4300 may utilize a video-based mirror that allows the driver 4010 to manually adjust the video image shown in a manner similar to adjusting the previous side mirrors. good. In another example, the reaction system 4300 may include a joint joint. The driver 4010 may be able to adjust the magnitude and / or speed of the movement of the joint joint (eg, a gentler movement may result in more comfort, and a faster and more aggressive movement may result in a faster and more aggressive movement. Greater performance may be provided).

Reaction system with video-based mirrors FIG. 4 shows an exemplary reaction system 4300, including a video-based mirror 4320. As shown, the mirror 4320 onto the processor 4400 (also referred to as the microcontroller unit (MCU) 4400) to acquire the source video image 4332 (also referred to as the "source video stream") of the region of the vehicle 4000's outer environment 4500. The combined camera 4330 may be included. The mirror 4320 may also include a display 4340 coupled to the MCU 4400 so that the converted video image 4342 (eg, a portion of the source video image 4332) is shown to the driver 4010. Processor 4400 applies the conversion to the source video image 4332 and adjusts the converted video image 4342 shown to the driver 4010 (eg, FOV and / or) in response to the sensor 4200 detecting the movement of the driver 4010. Angle of view) may be used. In this way, the video-based mirror 4320 can complement or replace conventional mirrors (eg, side mirrors, rearview mirrors) in the vehicle 4000. For example, a video-based mirror 4320 can be used to reduce the aerodynamic resistance normally encountered when using conventional mirrors. In some cases, the mirror 4320 may be classified as a camera surveillance system (CMS) as defined by ISO16505-2015.

The mirror 4320 may acquire a source video image 4332 that covers a sufficient portion around the vehicle to enable safe driving of the vehicle 4000. In addition, the mirror 4320 may reduce or reduce the scale distortion and / or geometric distortion of the converted video image 4342 shown on the display 4340. The mirror 4320 may also be configured to comply with local regulations. Conventional driver side mirrors and center mirrors are generally unable to exhibit these desired characteristics. For example, side mirrors and center mirrors should provide equal magnification in the United States, which means that the height and angle width of the displayed object is the same as the height and angle width of the same object viewed directly at the same distance. Means that it should meet (Federal Motor Vehicle Safety Standards No. 111).

The cameras 4330 may be used individually or as part of an array of cameras 4330, each covering each area of the vehicle 4000's outer environment 4500. The camera 4330 may include a lens (not shown) and a sensor (not shown) for acquiring the source video image 4332 that together define the FOV 4334 of the camera 4330.

5A and 5B show an articulated vehicle 4000 and a conventional vehicle, each including cameras 4330a, 4330b, and 4330c (collectively, camera 4330), which cover the outer left, right, and rear regions of the vehicle 4000, respectively. 4002 is shown. Each vehicle 4000, 4002 also includes corresponding displays 4340a and 4340b showing the converted video image 4342 acquired by cameras 4330a, 4330b, and 4330c. (Conventional vehicles may also include an additional display 4340c instead of a rearview mirror.) As shown, the camera 4330 should have a partially overlapping FOV4334 so that no blind spots are formed between the different cameras 4330. It may be oriented.

The placement of the camera 4330 on the vehicle 4000 can depend on several factors. For example, the camera 4330 may be placed on the vehicle body 4100 to capture the desired FOV4334 of the environment 4500 (as shown in FIGS. 5A and 5B). The camera 4330 may also be positioned to reduce aerodynamic resistance on the vehicle 4000. For example, each camera 4330 may be mounted in a recessed opening above the door and / or side panel of the vehicle body 4100, or a rear-facing portion of the trunk of the vehicle 4000. The placement of the camera 4330 may also be partly dependent on local regulations and / or guidelines (eg, ISO16505) based on where the vehicle 4000 is used.

The FOV4334 of the camera 4330 may be large enough to support one or more desired image transformations applied to the source video image 4332 by the processor 4400. For example, the converted video image 4342 shown on the display 4340 corresponds to a portion of the source video image 4332 acquired by the camera 4330 and may therefore have a FOV 4344 smaller than the FOV 4334. The sensor of camera 4330 may acquire the source video image 4332 at a sufficiently high resolution such that the converted video image 4342 is at least compatible with the minimum resolution of the display 4340 over the range of supported image conversions. good.

The size of the FOV4334 may be based in part on the optics used in the camera 4330. For example, the camera 4330 may use a wide-angle lens to increase the FOV4334 and therefore may cover a larger area of the environment 4500. The FOV4334 of the camera 4330 may also be adjusted by an electric mount that couples the camera 4330 to the vehicle body 4100 of the vehicle 4000. The electric mount may rotate and / or pan the camera 4330 and therefore transfer the FOV 4334 of the camera 4330. This can be used, for example, when the camera 4330 includes a lens with a longer focal length. The electric mount may be configured to operate the camera 4330 at a frequency that allows the desired responsiveness to the video image 4342 shown to the driver 4010. For example, an electric mount can operate the camera 4330 at about 60 Hz. If the electric mount operates the camera 4330 at a lower frequency (eg, 15 Hz), the processor 4400 will generate additional frames (eg, by interpolation) to upsample the video image 4342 shown on the display 4340. ) May.

Each camera 4330 may acquire the source video image 4332 at a variable frame rate, depending on the lighting conditions and the desired exposure settings. For example, camera 4330 may acquire source video image 4332 nominally at a frame rate of at least about 30 frames per second (FPS), and preferably 60 FPS. However, in low light conditions, the camera 4330 may acquire the source video image 4332 at a lower frame rate of at least about 15 FPS.

Each camera 4330 also includes, but is not limited to, the visible, near-infrared (NIR), mid-infrared (MIR), and far-infrared (FIR) regions, and the source video image 4332 in a variety of wavelength ranges. May be configured to obtain. In some applications, the array of cameras 4330 placed on the vehicle 4000 may be used to cover one or more wavelength ranges to allow multiple modalities when operating the mirror 4320 (. For example, one camera 4330 acquires a visible video image and another camera 4330 acquires a NIR video image). For example, if the sensor 4200 detects that the vehicle 4000 is driving under poor visibility conditions (eg, driving at night, fog), the processor 4400 may show only the IR video image on the display 4340.

The reaction system 4300 includes specific parameters (eg, focal length, aspect ratio, sensor size) related to the characteristics of the optics and / or sensor, extrinsic parameters (eg, the position of the camera 4330 in the coordinate space of the vehicle 4000 and / /. Or orientation), as well as various operating parameters associated with each camera 4330, including, but not limited to, distortion coefficients (eg, radial lens distortion, tangential lens distortion) may be stored. The operating parameters of camera 4330 may be used to modify the transformations applied to the source video image 4332.

The display 4340 may be a device configured to show the converted video image 4342 corresponding to the FOV 4344. As shown in FIGS. 5A and 5B, the vehicle 4000 may include one or more displays 4340. Display 4340 may generally show video image 4332 acquired by one or more cameras 4330. For example, the display 4340 may be configured to display the converted video images 4342 of the plurality of cameras 4330 in a split screen arrangement (eg, two converted video images 4342 displayed side by side). .. In another example, the processor 4400 acquired the converted video image 4342 shown on the display 4340 by a plurality of cameras 4330 so as to seamlessly transition from one camera 4330 to another. The source video image 4332 may be converted (eg, the source video image 4332 is seamlessly stitched together). The vehicle may also include a plurality of displays 4340, each corresponding to a camera 4330 on the vehicle 4000.

The placement of the display 4340 can depend on several factors. For example, the position and / or orientation of the display 4340 may be based in part on the nominal position of the driver 4010 or the driver's seat of the vehicle of the vehicle 4000. For example, one display 4340 may be located on the left side of the handle and another display 4340 may be located on the right side of the handle. A pair of displays 4340 can be used to show converted video images 4342 from the respective cameras 4330 located on the left and right sides of the vehicle 4000. The display 4340 may be placed so that the driver 4010 can see the converted video image 4342 along the direction of travel without losing sight of the surroundings of the vehicle. In addition, the location of the display 4340 may also be subject to local regulations and / or guidelines based on where the vehicle 4000 is used, as is the camera 4330.

In some cases, the display 4340 may also be touch-sensitive to provide the driver 4010 with the ability to enter explicit commands to control the video-based mirror 4320. For example, the driver 4010 may touch the display 4340 and apply a swipe action to pan and / or scale a portion of the converted video image 4342 shown on the display 4340. When calibrating the video-based mirror 4320, the shift of the display 4340, which is discussed in more detail below, can be adjusted by the touch interface. In addition, the driver 4010 may use the touch interface to adjust various settings of the display 4340, including but not limited to brightness and contrast.

The reaction system 4300 may include unique characteristics of the display 4340 (eg, display resolution, refresh rate, touch sensitivity, display dimensions), extrinsic characteristics (eg, position and / or orientation of the display 4340 within the coordinate space of the vehicle 4000). And various operating parameters associated with each display 4340 may be stored, including but not limited to distortion factors (eg, curvature of the display 4340). The operating parameters of display 4340 may be used by processor 4400 to perform conversion to video image 4332.

As mentioned above, the processor 4400 can be used to control the reaction system 4300. In the case of a video-based mirror 4320, the processor 4400 is based in part on a particular type of camera 4330 and / or display 4340 used (eg, the bit rate of the camera 4330, the resolution and / or refresh rate of the display 4340). , A high speed communication bus may be used to communicate with the display 4340 as well as the camera 4330. In some cases, the communication bus may also be based in part on the type of processor 4400 used (eg, the clock speed of the central processing unit and / or graphics processing unit). Processor 4400 may also use a shared communication bus, such as a CAN (Control Area Network) bus, to communicate with various components of the video-based mirror 4320 and / or other subsystems of the vehicle 4000.

The video-based mirror 4320 of the reaction system 4300 may acquire the source video image 4332, which is modified based on the movement of the driver 4010 and shown as the converted video image 4342 on the display 4340. These modifications may include applying a conversion to the source video image 4332 that extracts the appropriate portion of the source video image 4332 and prepares a portion of the video image 4332 to be displayed to the driver 4010. In another example, the conversion can be used to modify the FOV4344 of the converted video image 4342 so that the mirror 4320 responds in a manner similar to a conventional mirror. For example, the FOV4344 may widen as the driver 4010 moves closer to the display 4340. In addition, the FOV4344 of the converted video image 4342 may be panned when the driver 4010 repositions left and right.

FIG. 6 shows a method 600 for converting a source video image 4332 acquired by the camera 4330 based in part on changes in the position and / or orientation of the driver 4010's head. Method 600 may begin by sensing the position and / or orientation of the driver's head using the sensor 4200 (step 602). As mentioned above, the sensor 4200 may acquire data on the driver's head (eg, RGB images and / or depth maps). The processor 4400 may then determine the eye reference point of the driver 4010 based on the data acquired by the sensor 4200 (step 604). If the processor 4400 can determine the reference point for the eye (step 606), the transformation is calculated and applied to modify the source video image 4332 (step 610).

The conversion may be calculated using the model of the vehicle 4000 video-based mirror 4320 and sensor 4200. The model includes eye reference points, camera 4330 operating parameters (eg, intrinsic and extrinsic parameters, distortion coefficients), display 4340 operating parameters (eg, intrinsic and extrinsic parameters, distortion coefficients), and manufacturer and user calibration parameters. Various inputs may be received, including but not limited to. Various types of transformations, including but not limited to panning, rotation, and scaling, may be applied to the source video image 4332. The conversion may include applying a series of matrix conversion and signal processing operations to the source video image 4332.

In one example, the transformation applied to the source video image 4332 may be based solely on the eye reference point and user calibration parameters. In particular, the distance (calibrated) between the driver 4010's eye reference point and the initial sitting position is used to pan and / or zoom in on a portion of the source video image 4332 using a simple affine transformation. Can be done. For example, the magnitude of the conversion may be scaled to the calibrated operating range of the driver 4010. In addition, the pan speed and / or zoom factor may be constant so that the converted video image 4342 responds uniformly to movements by the driver's head. In some cases, the uniform response of the mirror 4320 may not depend on the distance between the display 4340 and the eye reference point of the driver 4010.

This conversion is such that the display 4340 is in the vehicle 4000 located in front of the driver 4010 and / or the mirror 4320 responds only to changes in the position of the driver's head (viewing angle of the driver 4010, or display). It may be preferable when configured to (do not respond to changes in other parameters, such as the distance between the 4340 and the driver 4010). Thus, this transformation provides a more standardized response to various arrangements of cameras 4330 and displays 4340 in the vehicle 4000, while being easier to implement and more affordable (it). Therefore, it can be done faster). In addition, this transformation may be applied to the source video image 4332 based on the movement of the driver's head.

を決定してもよい。その後、ベクトルを使用して、変換されたビデオ画像４３４２について、標的ＦＯＶおよびパン位置を決定してもよい。例えば、レイキャスティング法を使用して、運転者４０１０の眼の基準点から、ディスプレイ４３４０のそれぞれの角に、光線が投げられるＦＯＶを画定してもよい。 In another example, the transformation applied to the source video image 4332 is partly in the viewing angle of the driver 4010 with respect to the display 4340 and the distance between the reference point of the driver 4010's eyes and the display 4340. May be based. Transformations that include adjustments based on the position, viewing angle, and distance of the driver 4010 with respect to the display 4340 may better emulate the behavior of a conventional mirror and may feel more natural to the driver 4010. Vector 4400 from the reference point of the driver's eye to the center of display 4340 by processor 4400.

May be determined. Vectors may then be used to determine the target FOV and pan position for the converted video image 4342. For example, the raycasting method may be used to define a FOV to which a light beam is cast from the reference point of the driver 4010's eyes to each corner of the display 4340.

The next step is to extract a portion of the source video image 4332 that corresponds to the target FOV. This may involve determining the location and size of a portion of the source video image 4332 used for the converted video image 4342. The size of a portion of the source video image 4332 may be partially dependent on the angular resolution of the camera 4330 (eg, power per pixel), which is one of the unique parameters of the camera 4330. The angular resolution of camera 4330 may be used to dimension a portion of the extracted video image 4332. For example, the horizontal axis of the target FOV may cover an angular range of 45 degrees. If the angular resolution of the camera 4330 is 0.1 degrees per pixel, then a portion of the video image 4332 should have 450 pixels along the horizontal axis to meet the target FOV.

と、ディスプレイ４３４０の中心と交差し、かつ垂直であるベクトル

との間の角度として定義され得る。したがって、

および

の共線性は、ディスプレイ４３４０の中心に位置合わせされている、運転者４０１０の眼の基準点に対応するであろう。運転者の頭部が動くと、結果として生じる視野角によって、変換されたビデオ画像４３４２の場所を、ソースビデオ画像４３３２内の位置にシフトさせ得る。位置のシフトは、視野角（すなわち、水平視野角および垂直視野角）のそれぞれの構成要素に、カメラ４３３０の角度分解能を乗じることによって決定され得る。このように、トリミングされた部分の中心点（例えば、ＸおよびＹのピクセル位置）は、ソースビデオ画像４３３２に対して見出され得る。 The location of the converted video image 4342 extracted from the source video image 4332 captured by the camera 4330 may depend on the viewing angle of the driver 4010 with respect to the display 4340. The viewing angle is a vector

And a vector that intersects and is perpendicular to the center of the display 4340.

Can be defined as an angle between and. therefore,

and

The co-linearity will correspond to the reference point of the driver's eye, aligned with the center of the display 4340. As the driver's head moves, the resulting viewing angle can shift the location of the converted video image 4342 to a position within the source video image 4332. The position shift can be determined by multiplying each component of the viewing angle (ie, the horizontal viewing angle and the vertical viewing angle) by the angular resolution of the camera 4330. Thus, the center point of the trimmed portion (eg, the pixel positions of X and Y) can be found with respect to the source video image 4332.

If processor 4400 is unable to determine a reference point for the driver 4010's eyes, the default or previous conversion may be applied to the source video image 4332 (step 608 of FIG. 6). For example, the previous conversion corresponding to the previous measurement of the eye reference point may be maintained so that the converted video image 4342 does not change if the eye reference point is not detected. In another example, the transformation may be calculated based on the prediction of the driver's movement. If the eye reference point is measured as a function of time, previous measurements may be extrapolated to predict the location of the eye reference point of the driver 4010. Extrapolation of previous measurements is linear extrapolation (eg, the driver's movements are approximately linear with a sufficiently small time increment), and certain behaviors (eg, when changing lanes) of the driver. It can be achieved by one or more means including, but not limited to, modeling the behavior of the driver as the head moves towards the display 4340 in a substantially repeatable manner. Thus, even if the detection of the reference point of the eye is suddenly interrupted, the converted video image 4342 does not appear to be skipped and / or choppy.

After determining the conversion (eg, new calculated conversion, default / previous conversion), the conversion is then applied to the source video image 4332 to produce the converted video image 4342, which is then on the display 4340. (Step 612 in FIG. 6). The method 600 for converting this source video image 4332 may be performed at an operating frequency of at least about 60 Hz. In addition, the distortion coefficients of the camera 4330 and / or the display 4340 may be used to correct the radial and / or tangential distortion of the source video image 4332. Various techniques are used to calculate the corrected pixel position based on previous calibration, and then the pixel position of the source video image 4332 (ie, source video stream) is converted to the converted video image 4342 (ie). Distortion may be corrected, such as by remapping to the corrected pixel position of the converted video stream.

As mentioned above, the sensor 4200 and / or the reaction system 4300 may be calibrated to the driver 4010. For the video-based mirror 4320, the calibration adjusts the converted video image 4342 shown on the display 4340 to the height of the driver and / or the distance between the driver's head and the display 4340. May include aligning with the driver's head, which may vary based on. In addition, as mentioned above, the driver's operating range and / or initial position (eg, the driver's operating position in vehicle 4000) may be used to adjust the transformation applied to the source video image 4332. .. For example, the driver's range of motion may be used to scale the conversion so that the converted video image 4342 can be panned over a larger source video image 4332 (eg, the converted video image). The FOV4344 of the 4342 may cover the FOV4344 of the source video image 4332).

In another example, the driver's initial position can be used as the "baseline" position. The baseline position may correspond to the driver 4010 having a preferred FOV for each display 4340 (ie, in a vehicle 4000 with two or more displays 4340). For example, the converted video image 4342 shown on each display 4340 may be substantially centered with respect to the source video image 4332 acquired by each corresponding camera 4330. In another example, the preferred FOV may be subject to local regulations or manufacturer specifications for the vehicle 4000. In some cases, the initial position of the driver 4010 is the average position of the driver 4010 (eg, the average position when the driver 4010 is sitting) and / or when the driver 4010 uses the vehicle 4000. Based on the operating range, the mirror 4320 can be determined using a dynamic calibration approach that adapts to different drivers 4010.

Calibration of the mirror 4320 may be done in a semi-automatic manner, instructing the driver 4010 to perform certain actions (eg, move his or her limbs) to measure the range of motion and initial position. As mentioned above, the driver 4010 may use various systems such as the vehicle 4000 infotainment system or vehicle speakers to receive calibration instructions. For the video-based mirror 4320, the display 4340 can also be used to provide visual commands and / or cues to the driver 4010. The instructions and / or cues may include one or more graphics overlaid with another reference object that provides the vehicle 4000, road, and / or driver 4010 with a sense of scaling and orientation. Once these measurements are made, the processor 4400 may attempt to adjust the converted video image 4342 shown on each display 4340 in order to provide a suitable FOV around the vehicle.

The driver 4010 may also be provided with control to directly adjust the mirror 4320. Thus, the driver 4010 may calibrate the mirror 4320 according to his or her personal taste, similar to the way the driver can adjust the side mirrors or rearview mirrors of the vehicle. Various control inputs may be provided to the driver 4010, including but not limited to touch controls (eg, infotainment systems, displays 4340), physical buttons, and joysticks. The control input allows the driver 4010 to manually pan the converted video image 4342 up, down, left and right, and / or adjust the magnification shift to increase or decrease the magnification of the converted video image 4342. It can be possible.

These adjustments are made by modifying the conversion applied to the source video image 4332 (eg, adjusting the size and location of the converted video image 4342 extracted from the source video image 4332) and / or the camera. This can be done by physically rotating and / or panning the 4330. In addition, the extent to which the converted video image 4342 can be panned and / or scaled by the driver 4010 may be partially limited by the source FOV4334 and the resolution of the source video image 4332. In some cases, local regulations may also impose restrictions on the pan and / or scaling adjustments applied to the converted video image 4342. Further, these manual adjustments may be made without positioning the driver 4010 in a particular manner (eg, the driver 4010 does not need to be in the initial position).

After the mirrors 4320 have been calibrated, the driver's initial position, range of motion, and individual deviations for each mirror 4320 of the vehicle 4000 may be stored. Collectively, these parameters may define the "center point" of each display 4340, which represents the FOV of the environment shown to the driver 4010 in the initial position when controlling the vehicle 4000. The center point can be determined using only the initial sitting position and the deviation of each display 4340. In some cases, the center point may correspond to the default setting FOV4344 of the converted video image 4342 when the reference point of the eyes of the driver 4010 is not detected.

The operating range of the driver 4010 can be used to scale the speed at which the converted video image 4342 is panned and / or scaled. In addition, the operating range may be constrained by the cabin of the vehicle 4000 and / or otherwise invisible. Therefore, the magnification adjustment of the converted video image 4342 may be partially dependent on the detectable range of motion of the driver 4010 in the cabin of the vehicle 4000. If the driver 4010 is not found with sufficient certainty within a given time, the mirror 4320 displays, by default, a converted video image 4342 corresponding to the calibrated center point of each display 4340. You may.

Reaction system with joints The reaction system 4300 may also include joints that modify the physical configuration of the vehicle 4000 based in part on the behavior of the driver 4010. For example, the joints may be part of an active suspension system on the vehicle 4000 that adjusts the distance between the wheels of the vehicle 4000 and the chassis. The vehicle 4000 may include multiple independently controlled joint joints for each wheel to change vehicle height and / or tilt the vehicle 4000. In another example, the shape and / or shape of the vehicle body 4100 may be changed by joints. This may include joint joints that actuate the truck bed.

In addition, joints may bend various parts of the vehicle body 4100 and / or otherwise twist (see exemplary vehicle 4000 in FIGS. 7A-7E). For example, one or more joint joints and / or other actuators may actuate the payload support mechanism rather than the vehicle itself. For example, these actuators can adjust the seat position and reclining angle to maximize comfort and / or visibility, especially for individual drivers, without necessarily articulating the vehicle. Seat adjustments can be made immediately after the driver enters the vehicle or in anticipation of the driver entering the vehicle. Subsequent adjustments to the seating area and reclining angle can be made while the vehicle is in motion as the driver calms down over time. In these scenarios, articulating the vehicle may not be inefficient or safe.

Both articulations of the vehicle body 4100, and the actuation of its suspension, may allow for several configurations, each of which provides certain desirable characteristics for the performance and / or operation of the vehicle 4000. The vehicle 4000 may be configured to actively transition between these configurations based on changes in the position and / or orientation of the driver 4010 as measured by the sensor 4200. In some cases, a combination of explicit inputs by the driver 4010 (eg, turn signal activation, window descent) and driver behavior may control the response of the vehicle 4000's joint joints.

For example, the vehicle 4000 may support a low contour configuration in which the height of the vehicle 4000 descends closer to the road (see FIG. 7D). With lower configurations, aerodynamic performance can be improved by reducing the drag coefficient and / or front area of the vehicle 4000. Also, the lower profile configuration may increase the wheelbase of the vehicle 4000 and / or lower the center of gravity, which improves driving performance by providing greater stability and faster cornering speeds. The processor 4400 concentrates the driver 4010 on driving the vehicle 4000 (eg, the reference point of the eye indicates that the driver 4010 is concentrated on the perimeter directly in front of the vehicle 4000) and / or. Determining that the vehicle is driving at high speed (eg, on a highway), the processor 4400 may shift and / or maintain the vehicle 4000 to a lower contour configuration.

In another example, the vehicle 4000 may support a high contour configuration in which the height of the vehicle 4000 is lifted above the road (see FIG. 7E). The high contour configuration can be used to assist the entry and / or exit of the vehicle 4000. When combined with an articulated seat mechanism, the seat (or more broadly, the loading platform) is suitable for the driver 4010 (eg, a worker, a robotic automated device) to access the payload stored in the vehicle 4000. Can be presented at height. Also in the ascending position, the FOV of the driver 4010 may be increased and / or any sensor placed on the vehicle 4000 may be increased to monitor the surrounding environment, thereby increasing situational awareness. When the FOV of the driver 4010 is blocked by an obstacle in the environment (eg, another vehicle, barrier, person) and / or by the processor 4400, the driver 4010 actively around the obstacle. When it is determined that it is trying to see (for example, the reference point of the eye indicates that the driver's head is pointing upwards overlooking the obstacle), the processor 4400 makes the vehicle 4000 a high profile. Can be migrated and / or maintained to the configuration.

The vehicle 4000 may also support an intermediate external configuration, which may be defined as an intermediate state between the low and high external configurations described above. Therefore, an intermediate external configuration can provide a mixture of low external characteristics and high external characteristics. For example, an intermediate contour configuration may provide better visibility to the driver 4010 while maintaining a low center of gravity to improve dynamic performance. This configuration can be used to accept a number of scenarios encountered when driving a vehicle 4000 in an urban environment and / or interacting with other vehicles or devices.

Various use cases for intermediate profile configurations include, but are not limited to, mailboxes, automated teller machines (ATMs), drive-through counters, and other people standing by the side of the road (eg, neighbors or bicycles). Includes adjusting the ride height to facilitate interaction with the rider). When the vehicle 4000 is used to transport (possibly autonomously) cargo, intermediate conditions provide better ergonomic and mechanical interactions with delivery and / or outlets, robots, and humans. It will be possible. These use cases may be accompanied by predictable movement of the driver 4010 (or cargo). For example, the driver 4010 may lower the window and stick out his hand to interact with an object or person in the environment. If the sensor 4200 detects that the window is down and the processor 4400 determines that the driver 4010 is sticking out his hand, the processor 4400 is an object detected near the window on the driver's side. The height of the vehicle 4000 can be adjusted according to the height of the vehicle.

7A-7E show the payload positioning joint 2100 (also referred to as the payload positioning mechanism) for supporting the joint joint 106 (also referred to as the joint mechanism), the deformed portion 123, and the payload 2000 (eg, driver, passenger, cargo). Incorporates a vehicle 4000. In this example, vehicle 4000 is a rear-wheel steered three-wheel electric vehicle. The joint joint 106 allows the vehicle 4000 to be articulated or bent around the intermediate position along the length of the vehicle 4000, thus reconstructing the vehicle 4000.

The joint range of the vehicle 4000 has two distinctive configurations: (1) a low external configuration with an expanded wheelbase and the driver close to the ground, as shown in FIGS. 7A, 7B, 7D, and (2). As shown in FIG. 7E, the driver may be defined by a high contour configuration placed in an elevated position above the ground. The vehicle 4000 can be articulated into any configuration between low and high contour configurations. In some cases, the joint joint 106 may limit the configuration of the vehicle 4000 to the number of jumps. This is a preferred example, where a simpler design and / or a lower power design for the joint 106 is preferred and may be desirable.

The vehicle 4000 may be further subdivided into a vehicle front 102 and a tail 104 that are joined together by a joint joint 106. The front 102 is a vehicle support structure of various types, including, but not limited to, a unibody, a monocoque frame / shell, a space frame, and a frame-type structure (eg, a vehicle body mounted on the chassis). It may include the vehicle body 108. In FIGS. 7A-7E, the vehicle body 108 is shown as a monocoque frame. The vehicle body 108 is arranged on a removable side panel (or wheel fairing) 116, a fixed side window 125, a transparent canopy 110 coupled to the vehicle 4000, and a parallel configuration on the vehicle body 108 below. It may include two front wheels 112 mounted on the. The tail 104 may include a rear shell 121, rear glass 124, and steerable wheels 126. The deformed portion 123 may be coupled between the front portion 102 and the tail portion 104 to maintain a smooth and continuous outdoor surface under the vehicle 4000 in various configurations. In FIGS. 7D and 7E, the rear shell 121 and the rear glass 124 have been removed so that the underlying components associated with at least the joint joint 106 are visible.

The canopy 110 can be coupled to the vehicle body 108 by a hinged arrangement so that the canopy 110 can be opened and closed. If the payload 2000 is the driver, the canopy 110 may be hinged towards the top surface of the vehicle 4000 when in the high external configuration of FIG. 7E, so that the driver has two front wheels 112. In between, you may enter and exit the vehicle 4000 by stepping into and / or out of the vehicle 4000.

The front wheels 112 may be powered by an electric hub motor. The rear wheels 126 may also be powered by an electric hub motor. Some exemplary electric motors were published on June 14, 2014, and are entitled "Rotary Drive with Two Degrees of Movement," US Pat. No. 8,742,633, as well as "Guided Multi-Bar Linkage Electroc." It can be found in US Patent Publication No. 2018/0072125 entitled "System", both of which are incorporated herein by reference in their entirety.

The rear surface of the vehicle front 102 is nested within the rear shell 121 as the tail 104 moves relative to the front 102 via the joint 106, with the rear shell 121 of the tail 104 and the vehicle front. The gap between the 102's posterior surface can be formed to remain small. As shown, the vehicle 4000 may be reconstructed by rotating the tail 104 with respect to the front 102 about the axis of rotation 111 by means of a joint 106. In FIGS. 7B, 7C, and 7E, the axis of rotation 111 is perpendicular to the plane that bisects the vehicle 4000. The planes are (1) the longitudinal axis of the vehicle 4000 (eg, the axis that crosses the front portion of the vehicle body 108 and the rearmost portion of the rear outer shell 121) and (2) the horizontal surface on which the vehicle 4000 rests in this way. It can be defined to accommodate a vertical vertical axis.

The joint joint 106 may include a guide structure 107 (also referred to as a guide mechanism) that determines the joint motion contour of the joint joint 106. In the exemplary vehicle 4000 shown in FIGS. 7A-7E, the guide structure 107 may include a track system coupled to the front 102 and a lift bracket 538 coupled to the tail 104. Alternatively, the track system 536 may be coupled to the tail 104 and the lift bracket 538 may be coupled to the front 102. The lift bracket 538 may travel along a path defined by the track system 536, thereby causing the vehicle 4000 to change configuration. The joint joint 106 may also include a drive actuator 540 (also referred to as a drive mechanism) that moves the lift bracket 538 to the desired configuration along the trajectory system 536. The drive actuator 540 may be electrically controllable. The joint joint 106 may also include a brake 1168 to hold the lift bracket 538 in a particular position along the track system 536, thus allowing the vehicle 4000 to maintain the desired configuration.

The vehicle body 108 may also accommodate the payload positioning joint 2100 therein. The payload positioning joint 2100 may align the payload 2000 in a preferred orientation as a function of the configuration of the vehicle 4000. If the joint joint 106 modifies the configuration of the vehicle 4000, the payload positioning joint 2100 may simultaneously reorient the payload 2000 with respect to the vehicle 4000 (particularly the front 102). For example, the payload positioning joint 2100 provides a preferred driver orientation with respect to the ground so that the driver does not have to reposition his head when the vehicle 4000 transitions from a low profile to a high profile. Can be used to maintain. In another example, the payload positioning joint 2100 is used to maintain the preferred orientation of the parcel to reduce the potential for damage to the objects contained within the parcel when the vehicle 4000 is articulated. obtain.

The vehicle 4000 shown in FIGS. 7A-7E is an exemplary implementation of the joint joint 106, the deformed portion 123, and the payload positioning joint 2100. Various designs of the joint joint 106, the deformed portion 123, and the payload positioning joint 2100 will be discussed with reference to the vehicle 4000, respectively. However, the joint joint 106, the deformed portion 123, and the payload positioning joint 2100 may be mounted on other vehicle structures either separately or in combination.

The articulated vehicle 4000 of FIGS. 7A-7E has a single articulated DOF (ie, axis of rotation 111) such that the tail 104 rotates with respect to the front 102 to modify the configuration of the vehicle 4000. Shows. This topology takes into account intermediate and endpoint interactions, especially with the surrounding environment (eg, compact / self-propelled multi-storey car parks, small space maneuverability, low speed visibility, high speed aerodynamics). It may be preferable for a single commuter or passenger traveling in both urban environments and highways. Various mechanisms that support the above topologies and use cases may be broadly applied to a wider range of vehicles, fleet configurations, and / or other topologies.

For example, the vehicle 4000 may support one or more DOFs, each of which may be articulated. The articulation occurs around the axis and results in a rotational movement, and therefore a rotating DOF, such as the rotating shaft 111 of FIGS. 7A-7E, may be provided. Articulation may also occur along the axis and result in translational motion and hence translational DOF. The various mechanisms described herein (eg, joint joint 106, payload positioning joint 2100) can also be used to constrain movement along one or more DOFs. For example, the path may be demarcated by a joint 106 so that the components of the vehicle 4000 move along that path (eg, the lift bracket 538 moves along the path defined by the track system 536. Constraints on). Further, the range of motion along the path may be defined by the joint joint 106. This may be achieved in part by the joint joint 106, using a combination of high-strength and high-rigidity components that are assembled using tight tolerances and / or pressed into contact through external forces. It provides a smooth motion, induced by a small force input along the desired DOF, while at the same time providing mechanical constraints along the other DOFs.

The mechanisms described herein can define movements with respect to an axis or point (eg, a remote center of motion) that may or may not be physically positioned on the joint joint 106. For example, the joint joint 106 shown in FIGS. 7A-7E causes a rotational motion around a rotating shaft 111 across the interior compartment of the vehicle body 108, located separately from the lift bracket 538 and the track system 536. In another example, the payload positioning joint 2100 may have one or more rails 2112 that define the translational movement of the platform (eg, driver's seat).

In addition, the operation along each DOF may also be independently controllable. For example, each desired DOF of the vehicle 4000 may have a separate corresponding joint joint 106. The drive system of each joint 106 can induce movement along each DOF independently of the other DOFs. For FIGS. 7A-7E, the joint joint 106 that causes rotation around the axis of rotation 111 may be independent of other DOFs supported by the vehicle 4000.

However, in some cases, the articulation along one DOF of the vehicle 4000 may depend on another DOF of the vehicle 4000. For example, one or more components of the vehicle 4000 may move relative to another component depending on the other component that is articulated. This dependence can be achieved by mechanically coupling multiple DOFs (eg, one joint 106 has both joints 106 actuated continuously or simultaneously by a single drive actuator 540). Mechanically coupled to another joint 106 to obtain). Another approach is to electronically couple separate DOFs by connecting separate drive actuators 540 together. For example, when the vehicle 4000 is reconfigured, the payload positioning joint 2100 drives using an in-vehicle motor in response to the joint joint 106 reconstructing the vehicle 4000 so that the driver maintains the preferred orientation. The seat can be activated.

The joint joint 106 may generally include a guide structure 107 that defines the motion contour and thus the joint DOF of the joint joint 106. The guide structure 107 may include two reference points that move relative to each other. The first reference point may be coupled to one component of the vehicle 4000 and the second reference point may be coupled to another component of the vehicle 4000. For example, the anterior portion 102 may be coupled to a first reference point of the guide structure 107, and the tail 104 may be a second reference of the guide structure 107 such that the anterior portion 102 is articulated to the tail 104. It may be connected to a point.

In one aspect, the guide structure 107 may provide a joint joint around an axis and / or a point that is not physically located together with the joint joint 106 itself. For example, the joint joint 106 may be an action at a distance (RCM) mechanism. The RCM mechanism is defined as having no physical rotary joint in the same place as the moving mechanism. Using such an RCM mechanism, a vehicle in which a otherwise inconvenient part of the vehicle 4000, such as the interior cabin of the vehicle body 108, where the payload 2000 is located, or, for example, a steering assembly, a battery pack, or an electronic device is present. It may provide a rotating joint located in a subsystem.

The following describes some examples of the joint joint 106 as an RCM mechanism. However, the joint 106 may not be an RCM mechanism in which the axis or point at which the DOF is defined may be physically located with the components of the joint 106.

In one embodiment, the guide structure 107 may be an chassis track type mechanism. The joint joint 106 shown in FIGS. 7A-7E is an example of this type of mechanism. The guide structure 107 may include an chassis and track system 536, further detailed in FIGS. 8A-8G. As shown in FIG. 8A, the track system 536 can be attached to the front 102. The chassis 538 can be part of the tail 104. As shown in FIGS. 8E and 8F, the chassis 538 may ride along a vertically oriented curved path defined by the track system 536. The drive actuator 540 may be mounted on the chassis 538 to mechanically move the chassis 538 along the track system 536 under electrical control.

The track system 536 may include two curved rails 642 that run parallel to each other and are both connected to the back surface of the front vehicle section 102. The curved rail 642 may be similar in design. The vehicle body 108 can be made from a molded rigid carbon fiber shell having a convexly curved back surface (ie, convex when viewed from the rear when looking at the front vehicle portion 102) forming the back surface to which the rail 642 is attached. .. The back surface area to which the rails 642 are attached and to which they fit represents a segment of the cylindrical surface whose axis corresponds to the axis of rotation 111. In other words, the rail 642 may have a constant radius of curvature through the region in which the chassis 538 moves. The arc on which the rail 642 extends can be from about 90 ° to about 120 °.

Each rail 642 may also include a recess 643 that spans a portion of the length of the rail 642. The recess 643 may include one or more holes Z through which bolts (not shown) can attach the rail 642 to the carbon fiber shell 108. Each rail 642 is substantially isosceles in shape, with the narrow side of the trapezoid on the bottom side of the rail 642 close to the front body shell 108 to which it is mounted, and the wider side of the trapezoid on the upper side of the rail 642. Can have a cross section that is Rail 642 is any suitable material including, but not limited to, aluminum, hard coated aluminum (including, for example, titanium nitride) to reduce oxidation, carbon fiber, fiberglass, hard plastics, and hardened steel. Can be made in.

The chassis 538 shown in FIGS. 8A and 8E supports the tail 104 of the vehicle 4000. The tail 104 may further include a rear shell 121, a steering mechanism 200, and a wheel assembly 201. The chassis 538 may be coupled to the track system 536 using one or more bearings. As shown in FIG. 8G, two bearings 644 are used for each rail 642. Each bearing 644 may include an assembly of three parts of an upper plate 645 and two tapered side walls 646 fixed to the upper plate 645. The assembled bearing 644 defines an opening having a cross section substantially similar to the rail 642 (eg, isosceles trapezoid), which may have dimensions slightly larger than the rail 642 to facilitate movement during use. Can be. As shown, the bearing 644 can therefore be coupled to the rail 642 to form a "curved dovetail" arrangement in which the inner sidewall of the bearing 644 can contact the tapered outer sidewall of the rail 642. The bearing 644 may not be separated from the rail 642 along any other DOF other than the desired DOF defined by rotational motion around the axis of rotation 111. FIG. 8G shows an exaggerated representation of the tolerance between the bearing 644 and the rail 642 for illustration purposes. Tolerances can actually be substantially smaller than shown. The plate 645 and side wall 646 may be curved to fit the curved rail 642.

In one embodiment, the bearing 644 may be a plain bearing in which the inner upper surface and side surfaces of the bearing 644 slide when attached to the upper and side wall surfaces of the rail 642, respectively. Bearings 644 may also include screw holes in the top plate (eg, via bolts) that connect the rest of the chassis 538 to the track system 536.

The length of the bearing 644 (eg, the length defined along the direction parallel to the rail 642) can be greater than the width of the bearing 644. The ratio of length to width can be adjusted to adjust the load distribution on the bearing surface and reduce the possibility of coupling between the bearing 644 and the rail 642. For example, the ratio can be in the range of about 3 to about 1. The bearing 644 may also have a low friction, high force, low wear working surface (eg, particularly a surface in contact with the rail 642). For example, the working surface of the bearing 644 may include, but is not limited to, a Teflon coating, a graphite coating, a lubricant, and a polished bearing 644 and / or rail 642. In addition, the plurality of bearings 644 should have an occupied area with a length-to-width ratio in the range of about 1 to about 1.6 in order to reduce coupling, increase stiffness and increase range of motion. Can be placed in. Generally, a bearing 644 with a longer base may have a narrower operating range, while a bearing 644 with a narrower base may have lower stiffness, therefore the length of the bearing 644 may operate. It may be chosen to balance range and stiffness, which may further depend on other constraints imposed on the bearing 644, such as size and / or placement within the vehicle 4000.

The chassis 538 may further include two frame members 539, each frame member 539 aligning with the corresponding rail 642. Two crossbars 854 and 856 may be used to tightly connect the two frame members 539 to each other on the sides of the chassis 538 close to rail 642. The bearing 644 may be attached to the frame member 539 at four attachment points 848a-d. Two support bars 851 may be used to support the wheel assembly 201 and steering mechanism 200 on the side surface of the chassis 538 farthest from rail 642. The two support bars 851 may be connected together by another crossbar 850.

The chassis 538 and track system 536 described above are just one example of a track type joint joint 106. Other exemplary joint joints 106 may include a single rail or two or more rails. As shown above, the RCM may be located in the cabin of vehicle 4000, where the payload 2000 is located without any components and / or structures that enter its space. However, in other exemplary joint joints 106, the RCM includes, but is not limited to, on the joint joint 106, within the vehicle subsystem (eg, within the front 102, within the tail 104), and outside the vehicle 4000. It may be located elsewhere with respect to the vehicle 4000.

As mentioned above, the articulated joints of the reaction system 4300 may modify the physical configuration of the vehicle 4000 to modify some aspects and / or characteristics of the vehicle 4000, such as the driver's FOV. However, in some cases, the joints may be able to modify the physical configuration of the vehicle 4000 to the extent that the vehicle 4000 becomes mechanically unstable, partially or completely controlling the vehicle 4000. Can be lost. To prevent such loss of stability when operating the vehicle 4000, the reaction system 4300 includes a stability control unit that imposes constraints on the joint joints (eg, limits the range of motion and limits the speed of motion). obtain.

For example, the driver 4010 can tilt to one side of the vehicle 4000 when changing lanes to adjust the viewing angle of the rear view display so that the driver 4010 is approaching the vehicle from behind. You can check if. In response to the driver's movements, the joints of the vehicle 4000 may actively rotate the vehicle 4000 in order to increase the FOV available to the driver 4010. However, the amount of rotation commanded by the processor 4400 to enhance the FOV is limited or, in some examples, to the stability control unit to prevent loss of vehicle stability and / or rollover of the vehicle 4000. Can be prioritized.

The constraints imposed on the joint joint by the stability control unit can vary based on the operating conditions of the vehicle 4000. For example, the stability control unit may be used at high speeds (eg, the gyro stabilizing effect of a rotating wheel) when the vehicle 4000 is traveling at low speeds (eg, when changing lanes). Compared to, more restrictions may be imposed on the amount of rotation allowed. In this way, the stability control unit can preemptively filter actuator commands for joint joints intended to improve driver comfort when vehicle stability is affected.

FIG. 9 shows an exemplary control system 5000 that manages joints in the reaction system 4300. As shown, the control system 5000 includes, in part, a behavior control subsystem 5200 that generates behavior-based commands based on the driver's behavior. The control system 5000 also receives inputs from the driver 4010, environment 4500, and behavior control subsystem 5200 and commands commands based on the inputs used to actuate the various actuators of the vehicle 4000, including articulated joints. It may include a vehicle control subsystem 5100 to generate.

The vehicle control subsystem 5100 may operate in the same manner as the previous vehicle control system. For example, the subsystem 5100 receives commands from the driver 4010 (eg steering input, accelerator input, brake input) and environment 4500 (eg precipitation, temperature) to evaluate and / or execute vehicle stability. Change the command before. Therefore, the vehicle control subsystem 5100 may be considered to be extended by the behavior control subsystem 5200, which provides additional functions such as joint flexion of the vehicle 4000 based on the driver's behavior.

The control system 5000 may receive the driver-generated input 5010 and the environment-generated input 5020. The driver-generated input 5010 is an explicit command, i.e., a command derived from the driver 4010 that physically interfaces with the vehicle 4000 input devices such as the steering wheel, accelerator pedal, brake pedal, and / or turn signal knob. Can include. The driver-generated input 5010 is also an implicit command, such as the driver 4010 tilting its head to check the rear view display and / or the driver 4010 squinting for glare. It may include commands generated based on the movement of 4010. The environment-generated input 5020 is a road obstacle (eg, depression, type of road surface), weather-related effects (eg, rain, snow, fog), road obstacles (eg, other vehicles, pedestrians), and / or. It may include various environmental conditions that affect the operation of the vehicle 4000, such as the driver 4010 when not in the vehicle 4000.

As shown in FIG. 9, the driver-generated input 5010 and the environment-generated input 5020 can be used as inputs to both the vehicle control subsystem 5100 and the behavior control subsystem 5200, respectively. Behavior control subsystem 5200 includes motion monitoring, including various sensors, human interface devices, and camera arrays for measuring driver-generated inputs 5010 (both explicit and implicit commands) and environment-generated inputs 5020. It may include system 5210 and external monitoring system 5220. The behavior control subsystem 5200 may also include a situational awareness engine 5230 that processes and merges driver-generated inputs 5010 and environment-generated inputs 5020. The situational awareness engine 5230 may also filter inputs 5010 and 5020 to reduce the possibility of unwanted joint flexion in the vehicle 4000 (eg, joint joints where the driver is watching passengers or playing music. Should not be activated while moving the head while listening).

The situational awareness engine 5230 sends the input combined with the behavior engine 5240, which attempts to identify a predefined correlation between the combined input associated with a particular vehicle behavior and the calibrated input. obtain. For example, various inputs (eg, steering wheel angle, tilt of the driver's head, gaze direction of the driver 4010, and / or presence of a turn signal) may indicate characteristic values when the vehicle 4000 turns.

10A and 10B show tables of various exemplary driver-generated inputs 5010 and environment-generated inputs 5020, comparing the nominal ranges of the various inputs with the input values associated with the left-turning vehicle 4000. do. If the behavior engine 5240 determines that the combined input has a value that is substantially similar to the characteristic input value associated with the vehicle 4000, the behavior engine concludes that the vehicle 4000 is turning left and is appropriate. Behavior-based commands can be generated. Otherwise, the behavior engine 5240 may not generate behavior-based commands.

The behavior engine 5240 may perform this comparison between the combined and calibrated inputs associated with a particular vehicle behavior in several ways. For example, the combined inputs may be represented as a two-dimensional matrix in which each input corresponds to a parameter value. The behavior engine 5240 may perform cross-correlation between the combined inputs and the previously calibrated set of inputs. If the resulting cross-correlation shows a sufficient number of peaks (a peak indicating one or more of the combined inputs coincides with the calibrated input value), the behavior engine 5240 will be used by the vehicle 4000. It can be concluded that it exhibits the specific behavior associated with the calibrated input.

If the behavior engine 5240 generates a behavior-based command, the command is then transmitted to the vehicle control unit 5110 of the vehicle control subsystem 5100. The vehicle control unit 5110 may combine behavior-based commands with explicit commands by the driver 4010 and other inputs such as the environment-generated input 5020 to generate a set of combined commands. The vehicle control unit 5110 may also include the stability control unit described above. Therefore, the vehicle control unit 5110 may evaluate whether the combined set of commands can be executed without loss of vehicle stability.

If the vehicle control unit 5110 determines that the combined set of commands destabilizes the vehicle 4000, the vehicle control unit 5110 may adjust and / or filter the commands so that vehicle stability is maintained. .. This may include reducing the size of behavior-based commands for other inputs (eg, by applying weighting factors). In addition, specific inputs may be prioritized based on a given set of rules. For example, when the driver 4010 applies pressure to the brake pedal, the vehicle control unit 5110 may ignore behavior-based commands so that the vehicle 4000 can properly brake. More generally, the explicit commands provided by driver 4010 may take precedence over behavior-based commands to ensure the safety of vehicle 4000 and driver 4010. When the command set combined with the vehicle control unit 5110 is verified, the commands are then applied to the appropriate actuator 5120 of the vehicle to perform the desired behavior.

11A and 11B are exemplary of the vehicle roll angle φ _vehicle commanded with respect to the vehicle reference frame, function φ of the tilt angle of the driver 4010 φ inertia reference frame as a _passenger (eg, set by the gravity vector). A calibration map is shown. As shown in FIG. 11A, the φ _vehicle is reduced by a smaller value of the φ _passenger to ensure that the vehicle 4000 does not rotate perceptibly in response to a small change in the tilt angle of the driver 4010. It is possible to prevent unintended operation of the vehicle 4000. As the number of φ _passengers increases, the number of φ _vehicles increases rapidly before saturation. The saturation point may represent the limits imposed by the vehicle control unit 5110 to ensure that stability is maintained.

The constraints imposed by the vehicle control unit 5110 can vary based on the operating conditions of the vehicle 4000. For example, FIG. 11A shows that the upper limit for φ _vehicles can be increased or decreased. The change in the upper limit may be based in part on the speed of the vehicle 4000 and the presence of other stabilizing effects (eg, the gyro stabilizing effect of the rotating wheel). FIG. 11B shows that the speed φ _vehicle can also be adjusted to maintain stability. The speed φ _vehicle changes as a function of φ _passengers and can change based on the vehicle height of the vehicle 4000. If the vehicle 4000 has a low contour configuration, the vehicle 4000 has a smaller moment of inertia and can therefore rotate at a higher speed without losing stability.

As shown, the vehicle 4000 may continue to rotate to the saturation limit when the driver 4010 tilts his head. Further, the vehicle 4000 may stop responding to the driver 4010 if the driver 4010 returns to its original position within the vehicle 4000. The sensor 4200 may continuously calibrate the driver's default position within the vehicle 4000 to provide a continuous update of the original position. In some cases, low-pass filtering with a long time constant may be used to determine a reference position to be treated as the original position of the driver 4010.

In one exemplary use case, the driver 4010 may tilt its head to look around an obstacle located near the vehicle 4000. Here, the driver-generated input 5010 may include the tilt angle of the driver's head (acquired with respect to the reference frame of the vehicle), and the environment-generated input 5020 may be obstacle detection. For example, the environment-generated input 5020 is a visibility map constructed by combining 1D or 2D region data (eg, lidar, ultrasound, radar data, etc.) with a forward-looking RGB camera, as shown in FIGS. 12A and 12B. possible. The visibility map may indicate the presence of an obstacle (eg, another vehicle) if the area data indicates that the distance between the obstacle and the vehicle 4000 is less than a predefined threshold (FIG. 12A and FIG. 12A and). See the black box on the 12B obstacle mask). For example, if the obstacle is 10 meters away from the vehicle 4000, it is unlikely that the driver 4010 is leaning to look around the obstacle. However, for example, if the obstacle is less than 2 meters away from the vehicle 4000, the driver 4010 can be considered tilted to look around the obstacle.

Application of Reaction System As mentioned above, the sensor 4200 and reaction system 4300 may allow for further vehicle modality to improve the performance and / or usefulness of the vehicle 4000. For example, the above examples of video-based mirrors 4320 and joints are primarily intended for driver 4010 FOV modifications. As an exemplary use case, FIG. 13 shows a pedestrian crossing shielded by a parked vehicle near vehicle 4000. If vehicle 4000 includes articulated joints, vehicle height allows the driver 4010 and / or sensors on vehicle 4000 to detect and detect cyclists and miniature duck hoods on recumbent bicycles at pedestrian crossings. Can be increased to.

In another example, the vehicle 4000 is long, allowing the vehicle 4000 to tilt (eg ± 45 degrees) in response to the driver 4010's tilt to modify the vehicle shape and improve vehicle dynamic performance. It may have a traveling suspension element. For example, a narrow vehicle is preferable in that it reduces air resistance, reduces the occupied area in urban areas, and increases operability. However, narrow vehicles may have poor dynamic stability, especially when cornering due to narrow truck widths. When the driver 4010 is cornering at high speed, it may be beneficial for the vehicle 4000 to lean into a corner like a motorcycle.

14A and 14B show another exemplary use case where the vehicle 4000 is located behind another vehicle. The driver 4010 can tilt his head (or body) to look around other vehicles, thereby improving FOV and its situational awareness. The vehicle 4000 may detect that the driver 4010 is tilted in the cabin to look around the other vehicle and may respond by tilting the vehicle 4000 to further improve the FOV of the driver 4010. In some cases, the vehicle 4000 may also increase the vehicle height to further improve the FOV as the vehicle 4000 tilts.

15A-15C show a case where the vehicle 4000 uses an automatic security drone. In this case, the reaction system 4300 may respond entirely from the environment-generated input. The vehicle 4000 may include a camera with a 360 degree FOV of the surrounding environment. The reaction system 4300 may be configured to respond in a manner substantially similar to the exemplary vehicle 4000 of FIGS. 14A and 14B, but in this case the reaction system 4300 is not the movement of the driver 4010, but rather the movement of the driver 4010. Respond to video images captured by the environment's camera. For example, the vehicle 4000 may be configured to detect obstacles in the environment, accordingly the reaction system 4300 may actuate the joints to avoid the camera looking around the obstacles and / or colliding with the obstacles. It may be possible to do.

The camera can also be configured to detect uneven surfaces. To traverse these surfaces, the vehicle 4000 may be configured to use a walking motion. In some cases, the vehicle 4000 may include additional independent actuation of each wheel to extend the static vehicle height of the vehicle 4000. This walking motion can also be used to allow the vehicle 4000 to traverse a series of stairs by a complex motion derived from the joint DOF and / or the long running suspension DOF (see FIG. 15C). This capability can enable safe operation of self-driving vehicles while compromising with uncontrollable environments. If the vehicle 4000 has a cabin for the driver 4010, the cabin may be oriented in the desired orientation (eg, substantially) in order to reduce discomfort to the driver 4010 as the vehicle 4000 travels on uneven surfaces. Can be maintained horizontally).

Joint joints may also provide some dynamic benefit to the operation of the vehicle 4000. For example, vehicle stability can be improved by tilting the vehicle 4000 to a corner using joints, which increases the stability margin, maintains traction, or, in some cases, rolls over. Shift the center of mass in such a way as to avoid or eliminate it. Joint joints may allow for greater traction by allowing active control of the rotation of the vehicle 4000 through dynamic geometric optimization of the joint joints. The cornering performance of the vehicle 4000 can also be improved by tilting the vehicle 4000. In addition, the inverted pendulum principle can be used by articulating the vehicle 4000 into a high profile configuration and increasing the height of the center of mass (COM), especially in low speed vehicles in high density urban environments. Vehicle 4000 can also prevent motion sickness by predicting and / or mitigating dynamic movements that generally induce such discomfort to driver 4010.

The reaction system 4300 may also provide the driver 4010 with the ability to set the vehicle 4000 for himself. For example, the vehicle 4000 greets and / or recognizes the presence of the driver 4010 by activating a joint to cause the vehicle 4000 to sway and / or start so that the vehicle 4000 recognizes the presence of the driver. Can be configured as It can be used to greet the owner and / or customer of the vehicle 4000 (for vehicle allocation or sharing use).

In another example, the vehicle 4000 may also be configured to have a personality. For example, the vehicle 4000 may be configured to respond to the environment 4500 and provide a platform for communicating various goals and / or intents to other individuals or vehicles on the road. For example, the vehicle 4000 can be articulated to a high profile configuration and tilted to one side to indicate that the vehicle 4000 gives way to another vehicle (eg, at an intersection with a stop sign). In another example, vehicle 4000 may travel on a highway. The vehicle 4000 may be configured to gently rock from side to side to indicate to the other vehicle that the vehicle 4000 is about to join the other vehicle on the freeway. In another example, the vehicle 4000 may be configured to behave like an animal (eg, like a dog, like a tiger). In some cases, the type of action performed by vehicle 4000 may be reconfigurable. For example, it may be possible to download, customize, replace, evolve, adapt, and / or otherwise modify the character of the vehicle 4000 to suit the driver's tastes.

In another example, the joints of the vehicle 4000 can also be used to make the vehicle 4000 known to the driver 4010, for example in a parking lot. It is common to forget where you parked your vehicle in a crowded parking lot. In the sea of sport utility vehicles (SUVs) and trucks, it can be difficult to find a very small and lightweight mobility platform. The degree of freedom of joint flexion and long-distance movement (DOF) of the vehicle 4000 significantly reduces the vehicle 4000 by flexing the vehicle 4000 to adjust the height of the vehicle 4000 and / or inducing a sway / turning motion. You can make it stand out. In some cases, the vehicle 4000 may also make a sound (eg, make a sound through a joint to make a horn) and / or and / or blink the light of the vehicle 4000.

Vehicle 4000 also has non-transport features that can take advantage of Reaction System 4300, including but not limited to virtual reality, augmented reality, games, movies, music, tours around the world, sleep / health monitoring, meditation, and exercise. Can be provided. As the vehicle becomes more autonomous, the driver 4010 may have the freedom to use some of these services while moving between locations on the vehicle 4000. In general, the reaction system 4300 may allow the vehicle 4000 to reshape to better fit one of the additional services provided by the vehicle 4000. For example, vehicle 4000 adjusts its height while traveling across a bridge to provide driver 4010 with a desirable view of the landscape for photography (eg, for Instagram influencers). Can be configured in.

Vehicle 4000 can also be articulated to reduce glare. For example, the sensor 4200 may detect glare (eg, from the sun or the headlights of an oncoming vehicle) on the driver's eye area based on the RGB image acquired by the sensor 4200. Accordingly, the vehicle 4000 may adjust its height and / or tilt angle to reposition the driver's eye area to reduce glare.

FIG. 16 shows another exemplary vehicle 4000 that includes a joint joint that is partially used as a security system. In general, the vehicle 4000 may be configured to inform itself when a person attempts to steal the vehicle 4000. For example, the vehicle 4000 may make a sound, blink its light, or flex its joints. If an attempt is made to steal the vehicle 4000, the vehicle 4000 will also use joints to prevent entry into the vehicle 4000 and / or hit the vehicle body of the vehicle 4000 against a thief (eg, backing action). By rotating the vehicle 4000), the thief can be disturbed.

The vehicle 4000 may also include an externally facing camera to enhance situational awareness to prevent potential thieves. The camera can be used to perform face recognition for an individual approaching the vehicle 4000 (eg, from behind the vehicle 4000). The calculated unique aspects of an individual may be cross-referenced with a database of approved drivers. If no match is found, the individual can be cross-referenced with the law enforcement database to determine if the individual is a criminal.

FIG. 17 shows another exemplary use in which the vehicle 4000 is used as a tool. The vehicle 4000 may have a relatively compact footprint, joint flexion range, and spatial awareness, making it a promising tool for tasks beyond transport. For example, vehicle 4000 may include an on-board camera or on-board camera that simultaneously captures, lights, and smoothly tracks news anchors in the field, as shown in FIG. Active suspension can be used to stabilize the shot, while joint flexion can keep the camera at the desired height. In another application, the vehicle 4000 may be used to remotely monitor and / or inspect a site (eg, for spatial mapping) with a dashcam that provides a 360 ° field of view around it.

The position and / or orientation of the driver 4010 measured by the sensor 4200, and camera data may also be used in other subsystems of the vehicle 4000. For example, the desired listening position (“sweet spot”) for a typical multi-speaker configuration is a small fixed area that depends on speaker spacing, frequency response, and other spatial characteristics. Stereo immersion is maximal within the region of the desired listening position and drops rapidly as the listener exits this region. The vehicle 4000 may include a voice subsystem that maps the desired listening position above the driver's head using the location data of the driver 4010 and the acoustic model of the cabin of the vehicle 4000. As the driver 4010 shifts in the cabin, the signal time delay, phase, and amplitude of each speaker are independent to shift the desired listening position in order to maintain the desired listening position above the driver. Can be controlled.

In another example, the depth map and RGB camera data acquired by the sensor 4200 can be used to identify the driver 4010. For example, vehicle 4000 may include an identification subsystem that can identify the driver 4010 based on a pre-trained set of faces (or bodies). For example, vehicle 4000 may acquire an image of driver 4010 when first calibrating the identification subsystem. Identification subsystems can be used to adjust various vehicle settings according to user profiles, including, but not limited to, seating settings, music, and destinations. The identification subsystem can also be used for anti-theft by preventing unauthorized persons from accessing and / or driving the vehicle 4000.

In another example, the depth map and RGB camera data acquired by the sensor 4200 can be used to monitor the attention of the driver 4010. For example, the fatigue of the driver 4010 can be monitored based on the movement and / or position of the driver's eyes and / or head. If it is determined that the driver 4010 is tired, the vehicle 4000 may provide the driver 4010 with a message to park the car on the shoulder and rest.

CONCLUSIONS All parameters, dimensions, materials, and configurations described herein are exemplary, and actual parameters, dimensions, materials, and / or configurations are specific, one or more in which the teachings of the invention are used. Depends on the application. The aforementioned embodiments are presented primarily as examples, and within the scope of the appended claims and their equivalents, embodiments relating to the invention are practiced in forms other than those specifically described and claimed. Please understand what you get. Embodiments relating to the inventions of the present disclosure are directed to the individual features, systems, articles, materials, kits, and / or methods described herein.

In addition, any combination of two or more of these features, systems, articles, materials, kits, and / or methods, if these features, systems, articles, materials, kits, and / or methods are consistent with each other. It is included within the scope of the invention of the present disclosure. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of each element of the exemplary implementation without departing from the scope of the present disclosure. The use of numerical ranges does not preclude that equivalents outside the range that satisfy the same function by the same means produce the same result.

The above embodiment can be implemented by a plurality of means. For example, embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, software code can be run on the appropriate processor or set of processors, whether provided to a single computer or distributed among multiple computers.

In addition, the computer can be embodied in any of a number of forms, including rack-mounted computers, desktop computers, laptop computers, or tablet computers. In addition, a computer is not a device that is generally considered a computer, but a device with appropriate processing power, including a personal digital assistant (PDA), smartphone, or any other suitable portable or fixed electronic device. Can be embedded inside.

Also, a computer may have one or more input and output devices. These devices can be used, among other things, to present a user interface. Examples of output devices that can be used to provide a user interface include display screens for visual representation of printers or outputs, and other audio generator devices for audible representation of speakers or outputs. Examples of input devices that can be used for the user interface include keyboards and pointing devices such as mice, touchpads, and digitizer tablets. As another example, the computer may receive the input information by voice recognition or in other audible formats.

Such computers may be interconnected by one or more networks of appropriate form, including local area networks, or wide area networks such as enterprise networks, intelligent networks (INs), or the Internet. Such networks may be based on appropriate technology, may operate according to appropriate protocols, and may include wireless networks, wired networks, or fiber optic networks.

The various methods or processes outlined herein may be encoded as software that can be run on one or more processors using any one of different operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages and / or programming or scripting tools, and may be run or run on a framework or virtual machine. It may be compiled as possible machine language code or intermediate code. Some implementations may specifically use one or more of a particular operating system or platform, as well as a particular programming language and / or scripting tool, for ease of execution.

Also, the concepts of various inventions may be embodied as one or more methods, and at least one of them has been provided. The actions performed as part of the method may, in some cases, be ordered by different means. Therefore, the practice relating to some inventions may include performing each act of a given method in a different order than specifically exemplified, and may include performing some acts simultaneously (such). Even if the act is shown as a continuous act in the exemplary embodiment).

All publications, patent applications, patents, and other references referred to herein are incorporated by reference in their entirety.

All definitions defined and used herein are to be understood as governing dictionary definitions, definitions of documents incorporated by reference, and / or the usual meaning of defined terms.

As used herein and in the claims, the indefinite articles "a" and "an" should be understood to mean "at least one" unless explicitly stated otherwise.

As used herein and in the claims, the phrase "and / or" means "either or both" of the combined elements, i.e., in some cases connected and in other cases. It should be understood to mean an element that exists detachably in. Multiple elements listed in "and / or" should be construed as "one or more" of the same modalities, ie, coordinated elements. In addition to the element specifically identified by the "and / or" clause, other elements may optionally exist, whether or not they are related to the specifically identified element. Therefore, as a non-limiting example, the reference to "A and / or B" is only A (optional) in one embodiment when used in conjunction with an unrestricted wording such as "contains". Including elements other than B), in another embodiment only B (optionally including elements other than A), and in yet another embodiment both A and B (optionally including other elements) ) Etc. can be pointed out.

As used herein and in the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" is comprehensive, i.e. at least one of a large number of elements or a list of elements, and optionally an additional item not in the list. Includes one, but shall be construed to include more than one. Only terms that are explicitly indicated to be the opposite, such as "only one of" or "exactly one of", or "consisting of" when used in the claims. , Refers to embracing exactly one of many or enumerated elements. In general, the term "or" as used herein is exclusive, such as "any", "one of", "only one of", or "exactly one of". When preceded by a term, it shall only be construed as indicating an exclusive choice (ie, "one or the other, not both"). "Contains basically" shall have the usual meaning used in the field of patent law when used in the claims.

As used herein and in the claims, the phrase "at least one" associated with a list of one or more elements is selected from any one or more of the elements in the list of elements. Means at least one element, but does not necessarily include at least one of all the elements specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. Should be understood. This definition also allows the phrase "at least one" to refer to elements other than those specifically identified in the list of elements, whether or not they are related to the specifically identified element. It is also permissible that it can exist at will. Therefore, as a non-limiting example, "at least one of A and B" (or equivalently "at least one of A or B", or equivalently "of A and / or B". At least one of "), in which B does not exist in one embodiment and optionally contains two or more A's, at least one A (optionally including an element other than B), another embodiment. A does not exist in the embodiment and optionally contains two or more Bs, at least one B (optionally includes elements other than A), and in another embodiment optionally two or more. It is possible to refer to at least one A including A, and at least one B (optionally including other elements) including two or more Bs.

In the claims, as well as in the specification above, all transitional clauses, such as "comprising," "included," "carrying," "having," and "accommodating." "Patenting", "involving", "holding", "composed of", and the like are understood to be unrestricted, i.e., but not limited to. It means that it will not be done. Only the transitional phrases "consisting of" and "consisting essentially of" are closed or semi-closed, respectively, as provided in the U.S. Patent Office's Handbook of Patent Examination Procedures, Section 2111.03, respectively. It shall be a closed transitional phrase.

Claims (49)

It ’s a vehicle,
With the car body
A sensor coupled to the vehicle body for capturing RGB (red, green, blue) images and depth maps of the environment that houses the driver's head.
A reaction system coupled to the vehicle body for adjusting the driver's field of view (FOV) during operation.
To activate the reaction system to determine the reference point of the driver's eye based on the RGB image and the depth frame and to change the FOV of the driver based on the reference point of the eye. A vehicle comprising the sensor and a processor operably coupled to the reaction system. The vehicle according to claim 1, wherein the depth map is used to mask the RGB image and therefore the area of the RGB image is reduced for processing. The vehicle according to claim 1, wherein the depth map is aligned with the RGB image so that the depth of the environment corresponds to the location of the environment captured in the RGB image. The reaction system
Chassis connection components and
A joint joint having a first end coupled to the vehicle body and a second end coupled to a component of the chassis connection and operably coupled to the processor.
An actuator coupled to the joint for moving the second end relative to the first end.
The processor is configured to actuate the actuator to move the second end relative to the first end based on the reference point of the user's eye, thereby the user. The vehicle according to claim 1, wherein the FOV of the above is changed. The first, along the first axis of the vehicle, according to the reference point of the eye, which is moved by the joint joint along a second axis that is substantially parallel to the first axis. The vehicle according to claim 4, wherein the second end is moved with respect to the end. The car body defines the cabin,
The environment is the cabin
The reaction system
A camera mounted on the vehicle body for capturing a video image of the outer region of the vehicle,
A display located in the cabin and operably coupled to the processor and camera.
The processor is configured to modify the video image based on the reference point of the eye of the driver in order to change the FOV of the driver.
The vehicle according to claim 1, wherein the display is configured to show the video image modified by the processor. The processor
Calculating the distance between the reference point of the eye and the center point of the display,
Scaling the magnitude of the conversion based on the range of motion of the driver's head,
6. The vehicle of claim 6, configured to modify the video image by adjusting the video image based on the distance and the magnitude of the conversion. The camera is the first camera
The first video image covers the first FOV and
The region outside the vehicle is the first region outside the vehicle.
The reaction system further comprises a second camera mounted on the vehicle body for capturing a second video image of the second region outside the vehicle with a second FOV.
The processor is configured to combine the first video image and the second video image so that the display transitions seamlessly between the first video image and the second video image. The vehicle according to claim 6. An interior position sensor located in the cabin of the vehicle to detect the position and / or orientation of the driver's head of the vehicle.
A camera mounted on or in the vehicle for capturing video images of the rear region of the vehicle.
A reference point for the driver's eyes is determined based on the position and / or orientation of the driver's head, and the field of view (FOV) or angle of view of the video image is determined based on the reference point for the eyes. With the indoor position sensor and a processor operably coupled to the camera to modify at least one of them,
It comprises a display in the cabin of the vehicle for displaying at least a portion of the video image modified by the processor to the driver and operably coupled to the camera and processor. Reactive mirror system. At least one of a pair of infrared (IR) cameras in a stereo configuration for the indoor position sensor to generate a depth map representing at least the head of the driver, or at least the head of the driver. 9. The reactive mirror system according to claim 9, further comprising a visible light camera for capturing RGB (red, green, blue) images. 9. The reactive mirror system of claim 9, wherein the indoor position sensor is configured to sense the position and / or orientation of the driver's head at a frequency of at least about 60 Hz. The reactive mirror system of claim 9, wherein the camera has a field of view (FOV) that ranges between about 10 degrees and about 175 degrees. The reactive mirror system of claim 9, wherein the camera is configured to capture the video image at a frame rate of at least about 15 frames per second. In the vehicle for adjusting at least one of the brightness of the portion of the video image, the contrast of the portion of the video image, the pan position of the portion of the video image, or the FOV of the camera. The reactive mirror system of claim 9, further comprising a control interface operably coupled to said camera, said display, and said processor. A method of converting the video image displayed to the driver of the vehicle.
Measuring a representation of the cabin of the vehicle, wherein the representation includes at least one of a depth map or an RGB (red, green, blue) image, and the representation is the driver driving the vehicle. To show the head of
Determining the reference point for the driver's eyes based on the above expression,
Acquiring the video image of the area outside the vehicle using a camera mounted on or inside the vehicle.
Applying the transformation to the video image based on the reference point of the eye,
A method comprising displaying the video image to the driver on a display in the cabin of the vehicle. The representation may include the depth map and the RGB image to determine the reference point for the eye.
15. The method of claim 15, comprising masking the RGB image with the depth map to reduce the area of the RGB image for processing. 15. The method of claim 15, further comprising calibrating the driver's initial sitting position. 17. The method of claim 17, further comprising calibrating the driver's operating range. 18. The method of claim 18, further comprising calibrating the misalignment of the display. 19. The method of claim 19, further comprising calculating the center point of the display. The conversion is
Calculating the distance between the reference point of the eye and the center point of the display,
Scaling the magnitude of the transformation based on the driver's operating range.
20. The method of claim 20, comprising adjusting at least one of the camera's field of view or the camera's pan position based on said distance and said magnitude of the transformation. The conversion is
To calculate the target field of view and the target pan position based on the vector from the reference point of the eye to the center point of the display.
To calculate at least one of the translations or magnifications based on at least one of the camera's focal length, camera's aspect ratio, or camera's sensor size.
Adjusting at least one of the visual fields or pan positions of the video image based on the at least one of the translations or magnifications to simulate the target field of view and the target pan position. , The method of claim 20. Before applying the transformation
15. The method of claim 15, further comprising applying corrections to the video image so as to reduce at least one of the radial or tangential distortion of the video image. A method of adjusting at least one camera mounted on or in a vehicle.
Measuring a representation of the cabin of the vehicle, wherein the representation includes at least one of a depth map or an RGB (red, green, blue) image, and the representation is the driver driving the vehicle. Showing the head and
Determining the reference point for the driver's eyes based on the above expression,
Adjusting at least one of the field of view (FOV) or pan position of the at least one camera based on the reference point of the eye.
A method comprising displaying a video image on at least one display in an area outside the vehicle acquired by the at least one camera. 24. The method of claim 24, wherein the at least one camera comprises a first camera for acquiring a first video image and a second camera for acquiring a second video image. 25. Claim 25, further comprising stitching the first video image with the second video image in order to provide an uninterrupted FOV between the first camera and the second camera. the method of. Calibrating the driver's initial sitting position and
To calibrate the misalignment of the video image displayed on at least one of the displays.
24. The method of claim 24, further comprising calculating the center point of the video image displayed on the at least one display using the initial sitting position and the misalignment. Calibrating the driver's range of motion for the initial sitting position and
24. The method of claim 24, further comprising scaling the pan speed of the at least one camera based on the driver's operating range. Before displaying the video image
24. The method of claim 24, further comprising applying corrections to the video image to reduce at least one of radial or tangential distortions using one or more strain coefficients. It ’s a vehicle,
With the car body
Chassis connection components and
A joint joint having a first end coupled to the vehicle body and a second end coupled to a component of the chassis connection.
A guide structure coupled to the first end and the second end to define a path, wherein the second end moves along the path with respect to the first end. Possible, guide structure,
A drive actuator coupled to the guide structure for moving the second end along the path.
A joint joint comprising a brake coupled to the guide structure for holding the second end in a fixed position along the path in response to activation.
One or more sensors coupled to the vehicle body to detect at least one of the driver and the environment surrounding the vehicle.
With the one or more sensors for activating the joint and a processor operably coupled to the joint based on at least one of the environment surrounding the driver or vehicle. , A vehicle. The vehicle according to claim 30, wherein the component of the chassis connection is a rear vehicle body. 30. The vehicle according to claim 30, wherein the component of the chassis connection is a wheel. The vehicle body defines a cabin that accommodates the driver.
30. The vehicle of claim 30, wherein the one or more sensors are configured to generate a representation of the cabin, wherein the representation represents the driver's head. The processor is configured to identify the movement of the reference point of the driver's eye based on the representation of the cabin.
As the processor identifies the movement of the driver's eye reference point along the first axis of the vehicle, the joint joint may cause the driver's eye to the environment. 33. The vehicle of claim 33, configured to move the vehicle body along a second axis that is substantially parallel to the first axis in order to increase the displacement of the reference point. 34. The vehicle of claim 34, wherein the driver's field of view (FOV) is modified by the movement of the vehicle body along the axis. The processor is configured to activate the joint joint in order to detect the bright light perceived by the driver and reduce the bright light perceived by the driver, based on the representation. Item 33. The vehicle according to item 33. 33. The vehicle of claim 33, wherein the representation comprises at least one of a depth map or an RGB (red, green, blue) image. 30. The vehicle of claim 30, wherein the one or more sensors comprises a camera that captures a video image of the region of the environment, wherein the video image indicates the driver's head. The processor is configured to determine the relative position of the driver's head in the video image.
In response to detecting that the driver's head is moving with respect to the one or more sensors, the processor activates the joint joint to activate the driver's head. 38. The vehicle according to claim 38, wherein the vehicle body is configured to move the vehicle body so as to return to the position in the video image. The car body defines the cabin,
A camera mounted on or in the vehicle and operably coupled to the processor for capturing video images of the outer region of the vehicle.
30. The vehicle of claim 30, further comprising a display arranged in the cabin and operably coupled to the camera and the processor for displaying the video image to the driver. The processor determines a reference point for the driver's eye based on the video image and a first field of view (FOV) or angle of view of the video image based on the reference point for the driver's eye. Configured to fix at least one of
40. The vehicle of claim 40, wherein the display is configured to show at least a portion of the video image modified by the processor. In response to the driver's eye reference point moving along the first axis of the vehicle, the processor activates the joint joint to substantially parallel the first axis. 41. The vehicle of claim 41, configured to move the vehicle body along two axes, thereby modifying the driver's FOV. It ’s a way to drive a vehicle.
Using the first sensor to receive the first input from the driver of the vehicle,
Using the second sensor to receive the second input from the environment outside the vehicle,
Using a processor to identify the correlation between the first input and the second input,
The processor is used to generate behavior-based commands based on the correlation, which, when applied to the actuator of the vehicle, causes the vehicle to have predefined behavior. To move with
Generating a combined command based on the behavior-based command, an explicit command from the driver via an input device operably coupled to the processor, and the second input. When,
At least one of adjusting or filtering the combined commands to maintain the stability of the vehicle.
A method comprising activating the actuator of the vehicle using the adjusted and / or filtered, combined command. 43. The method of claim 43, wherein the first input comprises one representing the cabin of the vehicle, wherein the representation indicates the driver's head. The pre-defined behavior is based on the representation and the processor identifies the movement of the driver's head as it moves along a second axis that is substantially parallel to the first axis. 44. The method of claim 44, comprising moving the vehicle along the first axis accordingly. The processor is configured to detect the glare light perceived by the driver based on the representation so that the predefined behavior reduces the glare light perceived by the driver. 44. The method of claim 44, comprising moving the vehicle. 43. The second input comprises at least one of the images of the environment showing the traction of the wheels of the vehicle, the temperature of the environment, or at least one of another vehicle or person. The method described. 43. The method of claim 43, wherein the input device is at least one of a steering wheel, accelerator, or brake. 43. The method of claim 43, wherein the explicit command supersedes the behavior-based command.

Images