Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
  • G06T19/003—Navigation within 3D models or images
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/50—Constructional details
  • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2200/00—Indexing scheme for image data processing or generation, in general
  • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2200/00—Indexing scheme for image data processing or generation, in general
  • G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation

Definitions

• the present invention relates to a method and a vehicle control system for a vehicle for generating continuous images of a stored one
• Environmental models for vehicles such as known from WO 2013/060323 AI provide information about objects in one
• Subregions of the environment model are generated and displayed to the driver on a vehicle display.
• Image information is missing, so that an advertisement is no longer possible and the process is not executable. It is therefore an object of the invention to generate continuous images of an environment model, avoiding the said undesired orientations of the virtual camera. This object is achieved by a method for generating continuous images of a stored
• a vehicle control system of a vehicle for generating continuous images of a stored three-dimensional environmental model of the vehicle
• the invention accordingly provides a method for generating continuous images of a stored three-dimensional environment model of a
• a camera device of the vehicle By means of a camera device of the vehicle, at least one camera image is generated and the generated camera image is stored on a projection surface in the camera
• the virtual camera in this case includes a set of camera parameters whose elements a
• Camera position of the virtual camera and an optical axis of the virtual camera include. Along a first
• Movement path in the environment model is the amount of
• the optical axis of the virtual camera which is an element of the second set of camera parameters, is determined such that it passes through a predetermined axis point which lies on a connecting path between a first reference point and a second reference point.
• the first reference point is a predetermined point of that optical axis of
• the second reference point is on given point of that optical axis of the virtual camera, which is an element of the third set of
• An image of the environment model is an image of a specific generated by the virtual camera
• the projection surface may, for example, be a flat plane, a spherical shell segment, a cylindrical surface or a paraboloidal or hyperboloidal surface.
• optical axes extend from the corresponding camera position in the detection direction of the virtual
• the optical axes of the first and third sets of camera parameters are preferably aligned with the projection surface.
• the invention allows, through the first and third sets of camera parameters, a start and end position of the virtual camera with corresponding orientations
• Camera parameters defined an additional support point, so that the interpolation path is divided into the first and the second movement path.
• the camera parameters predetermined at the interpolation point are selected by the construction according to the invention in such a way that during interpolation it is prevented that the orientation of the virtual camera changes in such a way that it is aligned with a region of the surrounding model which can not be displayed or is not provided with image information.
• the invention Method thus allows the generation of continuous images.
• the inventive method increases the
• Method is the interpolation of the optical axis of the moving virtual camera along the first
• the camera position which is an element of the second set of camera parameters, lies in a middle third of the camera
• Circular section and in particular on a central point of the circular section This is the length of the first one
• Movement path substantially the same size as the length of the second movement path, leaving a natural and
• Camera parameters is substantially the same size as a distance of the second reference point from one
• optical axis at the transition point from the first movement path to the second movement path which is given by the optical axis of the second set of camera parameters, aligned in the correct direction.
• the predetermined connection point lies substantially in a middle of the connecting path between the first reference point and the second reference point.
• the vehicle functions may include driving and activating or deactivating actuators, such as turn signals or side mirrors, or else semi-autonomous or autonomous acceleration, deceleration or steering of the vehicle.
• the output continuous images are displayed on a display device to a driver of the vehicle.
• the invention relates to a
• Vehicle control system includes a camera device of the vehicle for generating at least one camera image and a computing device.
• the computing device is adapted to the generated camera image on a Projection of projected area in the stored three-dimensional environment model of the vehicle, and
• the vehicle control system comprises an output device, which is designed to output the generated continuous images.
• the virtual camera has a set of camera parameters whose elements include a camera position of the virtual camera and an optical axis of the virtual camera.
• the computing device is configured to continuously supply the set of camera parameters along a first movement path in the environment model between a predefined first set of camera parameters and a second set of camera parameters
• the computing device is further configured to determine that optical axis of the virtual camera, which is an element of the second set of camera parameters, such that it passes through a predetermined axis point lying on a connection path between a first reference point and a second reference point ,
• the first reference point here is a predetermined point of the optical axis of the virtual camera, which is an element of the first set of camera parameters.
• the second reference point is a predetermined point of the optical axis of the virtual camera which is an element of the third set of
• the computing means is adapted to determine the axis point, the first reference point and the second reference point.
• Vehicle control system describe the first movement path and the second movement path in the environment model
• Vehicle control system is the computing device to do so
• the vehicle control system comprises a driver assistance system which is designed to control a vehicle function of the vehicle on the basis of the output continuous images.
• the output device comprises a display device, which is designed to be the output continuous
• the present invention relates to a vehicle having a vehicle control system.
• Fig. 1 is a flowchart for explaining a
• FIG. 2 is a plan view of an exemplary vehicle according to an embodiment of the invention.
• Fig. 3 is a schematic oblique view of a
• Fig. 4 is a schematic cross-sectional view of
• Fig. 5 is a schematic cross-sectional view of a
• FIG. 6 is a schematic block diagram of a
• FIG. 1 shows a flowchart for explaining a
• the camera device 22 of the vehicle 21 includes a plurality of vehicle cameras 22a to 22d, which surround the
• Vehicle 21 are arranged around and allow a surround view view.
• a first method step S1 at least one camera image is generated by the camera device 22 of the vehicle 21.
• FIG. 3 Furthermore, a three-dimensional environment model 31 illustrated in FIG. 3 is provided in which a
• the projection surface 32a may be, for example, a flat
• the projection surface 32a corresponds to a surface shown in FIG.
• the generated camera image is projected onto the projection surface 32a. If the at least one camera image forms a surround-view view of the vehicle surroundings, the at least one camera image is projected onto the entire projection surface 32a. However, according to one embodiment, the generated camera image can also be projected or projected only onto a partial area of the projection area 32a.
• step S3 continuous images of the projection surface 32a are formed by means of a
• step S4 the generated continuous images are output.
• the virtual camera 33 has a lot of camera parameters or is characterized by these camera parameters.
• Elements of the set of camera parameters include a
• the virtual camera 33 is characterized in three configurations, which by a first set Ml of camera parameters with a corresponding camera position PI and optical axis AI, a second set M2 of Camera parameters with appropriate
• Camera position P3 and optical axis A3 are marked.
• the first set Ml of camera parameters and the third set M3 of camera parameters are fixed and correspond to an initial configuration or
• the second set M2 corresponds to an auxiliary configuration which provides a correct interpolation between the initial configuration and the
• This surface 32b indicates the possible camera positions of the virtual camera 33, that is, the virtual camera 33 is in principle freely displaceable or movable on the surface 32b.
• Camera position can then be specified, for example, by specifying coordinates x, y, z of the environment model 31.
• the camera position is given by spherical coordinates r, ⁇ , ⁇ , where r is the radius of the sphere 32
• optical axis can according to an embodiment of
• the optical axis is described by a quaternion.
• a movement path W between the initial configuration and the final configuration is divided into a first one
• Movement path Wl between the camera position PI of the first set Ml and the camera position P2 of the second set M2 and a second movement path W2 between the camera position P2 of the second set M2 and the camera position P3 of the third set M3.
• the camera position P2 of the second set M2 is determined by a point of the movement path W and is preferably located in a middle third of the through the
• the camera position P2 of the second set M2 is at the middle point of the
• Figure 4 shows a schematic cross-sectional view of
• a first reference point yl is predetermined or determined, which lies on the optical axis AI of the first set M1.
• the first reference point yl has a distance from the camera position PI of the first set M1, which essentially corresponds to a radius r of the sphere 32.
• a second reference point y3 which is a predetermined point of the optical axis A3 of the third set M3 of camera parameters is determined. A distance of the second
• Reference point y3 from the camera position P3 of the third set M3 preferably corresponds to the distance of the first
• Reference point yl of the camera position PI of the first set Ml and is preferably also substantially the same size as the radius r of the sphere 32nd
• an axis point y2 is determined, which lies on the connecting path S between the first reference point y1 and the second reference point y3.
• the axis point y2 is located substantially at a center of Link S between the first reference point yl and the second reference point y3.
• the camera parameters of the virtual camera 33 are now continuously interpolated between the initial configuration and the final configuration.
• the set of camera parameters along the first movement path Wl in the environment model 31 is continuously interpolated between the first set M1 and the second set M2 of camera parameters, and along the second movement path W2 the set of camera parameters continuously becomes between the second set M2 and the third set M3 interpolated from camera parameters.
• optical axis of the movable virtual camera is preferably along the first and / or second
• Motion path Wl, W2 performed by spherical linear interpolation, SLERP, wherein the optical axis is preferably represented as a quaternion.
• Camera parameters and an opening angle x3 of the third set M3 of camera parameters is interpolated.
• the interpolation is carried out linearly.
• the opening angle of the first amount Ml and the second amount M2 is respectively predetermined and the Opening angle x2 of the second set determined by linear interpolation.
• Image generation by the virtual camera 33 is made difficult or impossible.
• the reason of this configuration is that a shortest connection between the optical axis AI of the first set M1 and the optical axis A3 of the third set M3 corresponds to interpolation which occurs by rotating the optical axis away from the center z.
• the continuous images of the projection surface 32a become from the moving along the movement path W. generated virtual camera 33, which has the camera parameters generated by the interpolation.
• the driver assistance system can have a
• FIG. 6 shows a block diagram of a vehicle control system 61 of a vehicle 21 for generating continuous images displayed on a display device to a driver of the vehicle.
• Figure 6 shows a block diagram of a vehicle control system 61 of a vehicle 21 for generating continuous
• the vehicle control system 61 has a
• the vehicle control system 61 comprises a computing device 62, which is adapted to the generated camera image on a
• Projection screen in the stored three-dimensional environment model 31 of the vehicle to project The
• Environment model 31 may preferably be on a
• Memory device of the vehicle control system 61 are stored or provided to the computing device 62 via a network.
• the environment model 31 can have more
• Lidarsensoren the vehicle the vehicle control system 61 are provided.
• the computing device 62 is further configured to
• the virtual camera 33 moves along a
• Motion path W which has a first movement path Wl and a second movement path W2.
• the virtual camera 33 has a set of camera parameters whose elements are a camera position of the virtual camera 33 and a camera
• optical axis of the virtual camera 33 include.
• the computing device 62 is designed to continuously interpolate the set of camera parameters along the first movement path W1 in the environment model 31 between a predetermined first set M1 of camera parameters and a predetermined second set M2 of camera parameters.
• the computing device 62 is designed to continuously vary the set of camera parameters of the virtual camera 33 along the second movement path W2 in the environment model 31 between the second set M2 of FIG
• Camera parameters and a predetermined third set M3 of camera parameters to interpolate are given and corresponding to a start and end configuration of the
• the camera position P2 of the second set M2 of camera parameters corresponds to a point on the camera
• Movement path W which in particular by connecting the camera position PI of the first set Ml and the camera position P3 of the third set M3 by means of one in Figure 31
• illustrated great circle can be determined by the computing device 62.
• the computing device is designed to set the optical axis A2 of the second set M2 of camera parameters in such a way that it passes through a predetermined axis point y2, which lies on a connecting path S between a first reference point y1 and a second reference point y3.
• the first or second reference point y1, y3 in this case corresponds to a predetermined point of the optical axis AI, A2 of the first quantity M1 or third quantity M3.
• the computing device 62 may be the second set M2 of Set camera parameters according to the method described above.
• the vehicle control system 61 further includes a
• Output device 63 which is designed to output the generated continuous images.
• Output device 63 may have an interface, in particular a cable connection, a USB interface or a wireless interface.
• the images produced are in particular via the output device 63 to more
• the vehicle control system 61 preferably further comprises a driver assistance system 64, which is designed to control a vehicle function of the vehicle 21 on the basis of the output continuous images.
• the output device 63 further comprises a
• Display device such as a vehicle display, which is adapted to the issued continuous
• Figure 7 shows a block diagram of a vehicle 21 with a vehicle control system 61, in particular according to an embodiment of the invention described above.

Landscapes

• Engineering & Computer Science (AREA)
• Software Systems (AREA)
• General Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Remote Sensing (AREA)
• Theoretical Computer Science (AREA)
• Computer Graphics (AREA)
• Computer Hardware Design (AREA)
• General Physics & Mathematics (AREA)
• Radar, Positioning & Navigation (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Closed-Circuit Television Systems (AREA)
• Image Processing (AREA)
• Traffic Control Systems (AREA)
• Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
• Processing Or Creating Images (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=59683361

Family Applications (1)

Country Status (5)

Families Citing this family (6)

Citations (1)

Family Cites Families (5)

• 2016
  • 2016-06-27 DE DE102016211453.0A patent/DE102016211453A1/de not_active Withdrawn
• 2017
  • 2017-06-20 JP JP2018563545A patent/JP6936816B2/ja active Active
  • 2017-06-20 WO PCT/DE2017/200055 patent/WO2018001422A1/de not_active Ceased
  • 2017-06-20 EP EP17755058.9A patent/EP3475922B1/de active Active
  • 2017-06-20 DE DE112017002285.1T patent/DE112017002285A5/de not_active Withdrawn
  • 2017-06-20 US US16/310,907 patent/US10692284B2/en active Active

Patent Citations (1)

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events