FR3069657A1 - OPTICAL DEVICE FOR OBSERVING A VEHICLE CAR - Google Patents

Abstract

The subject of the invention is an optical device for observing a user and a space surrounding the user, comprising: an optical objective (101), defining an optical axis (Ox) and an image plane; an optical sensor array (103) disposed in the image plane of the optical lens (101), characterized in that: ° the optical lens has a spatially variable distortion, ° the center of the optical sensor array (103) is eccentrically disposed with respect to the optical axis of the optical objective (101).

Description

Optical device for observing a vehicle interior

The present invention relates to an optical device for observing a passenger compartment of a vehicle, and more particularly to users present in said passenger compartment.

Motor vehicles, in particular with automated driving, may include optical devices, in particular shooting cameras, aimed at the driver and passengers, in particular at the front of the vehicle, the shots of which make it possible to establish indicators as to the presence and / or current state of a user, driver or passenger.

Such optical devices generally comprise a high resolution camera, with a narrow angle of view (30 to 60 °), directed towards the expected position of the face of the driver of the vehicle (upper part of the driver's seat). The camera records high resolution images of the driver's face, and a control unit extracts from said images estimates, in particular as to a level of vigilance of the driver.

To do this, the control unit can in particular establish a direction of the driver's gaze (towards the front, towards a side window, onto an object inside the passenger compartment, etc.) and / or a number of blinking eyelids per unit of time to assess driver fatigue and level of attention to surrounding traffic.

As an alternative or in addition, the optical devices include cameras with a wide angle of view (greater than 80 °) and at low resolution, for observing the entire cabin or at least its front portion. These wide-angle cameras, in particular with spherical optics, are generally directed towards the center of the passenger compartment, so as to observe both the driver and passenger seats.

A control unit connected to said wide-angle camera then uses the captured images to establish the presence of an adult passenger.

Wide-angle cameras, often arranged at a console

-2 central or a vehicle ceiling light, can also or alternatively be used to perform gesture recognition for a gesture recognition control module.

The two types of camera (reduced angle and high resolution versus wide angle at low resolution) cannot, a priori, be combined in a single optical device.

Approaching solutions can be obtained with dynamic cameras, the objective of which includes lenses movable in translation along the optical axis to allow a dynamic zoom according to the portion of image to be studied. However, these devices do not allow simultaneous analysis of a reduced portion of the field of vision with high resolution and an extended portion of vision with lower resolution, but only sequentially. In addition, dynamic zoom and moving parts require precise and rapid actuation, and are therefore subject to wear and tear over time.

It is also possible to produce non-uniform lenses of variable focal length, but these lenses are complex in their design and manufacture and expensive to produce. In addition, they require very precise positioning and alignment during assembly.

In order to at least partially solve the previously mentioned problem, the invention relates to an optical device for the observation of a user and of a space surrounding the user, comprising:

° an optical objective, defining an optical axis and an image plane, ° an array of optical sensors, arranged in the image plane of the objective, characterized in that:

° the optical objective has a spatially variable distortion, and ° the center of the optical sensor array is arranged eccentrically with respect to the optical axis of the optical objective.

-3 We thus obtain an optical device with fields of vision with a high resolution portion offset eccentrically in a controlled manner as required, and allowing the simultaneous observation of two portions of the observed space with two levels of resolution: a high resolution around of the optical axis and a lower resolution on the edges of the field of vision.

The optical device can then have one or more of the following characteristics taken alone or in combination.

According to one aspect, the offset between the center of the optical sensor array and the optical axis of the optical objective is between 0.1î / and Q, 7d, d being the spatial extension of the optical sensor array in the image plane.

In another aspect, the distortion of the lens is barrel-shaped and increasing with the deviation of the incident rays from the optical axis.

The matrix of optical sensors is for example a rectangular matrix of optical sensors of diagonal d.

According to another aspect, the optical objective is configured so that the image plane comprises a high resolution portion, in particular circular and centered around the optical axis, and a lower resolution annular portion surrounding the high resolution portion.

In yet another aspect, the resolution in the high resolution area is greater than a value between three and ten pixels per degree of angle.

The resolution in the annular zone can be between a low value equal to 0.3-1.0 pixels per degree of angle and a high resolution value at the border with the high resolution portion.

The array of optical sensors is arranged with a vertex of the rectangle placed on the image circle corresponding to the low resolution value.

The invention also relates to a module for monitoring the state of attention of a driver of a vehicle and for detecting the presence of a

-4passenger, characterized in that it comprises an optical device as defined above, and in that the optical axis of the objective is directed towards an expected position of the face or the bust of the driver or a passenger of the vehicle .

Said monitoring module can have one or more of the following characteristics, taken alone or in combination.

It further comprises a control unit configured to isolate in the portion of the matrix of optical sensors contained in the high resolution portion from the characteristics of the driver's face such as:

° the direction of the driver's gaze, ° the open or closed state of the driver's eyes, ° the driver's eyelid blinking frequency.

It also includes a control unit configured to isolate from the images from a portion of the matrix of optical sensors contained in the low resolution annular portion from the characteristics of the passenger or the driver such as:

° the presence or absence of a passenger, ° the size of the passenger, ° the position of the driver's hands, ° the position of the driver's bust.

It also includes a control unit configured to isolate in the images from the matrix of optical sensors specific gestures executed by the driver and / or the passenger for controlling vehicle functions.

Other characteristics and advantages of the invention will appear more clearly on reading the following description, given by way of illustrative and nonlimiting example, and of the appended drawings among which:

Figure 1 schematically shows an optical device arranged in a passenger compartment of a motor vehicle, Figure 2 schematically shows the cabin in top view with a

-5optical device and the associated field of vision, Figure 3 shows in more detail the optical device, Figure 4 is a graph illustrating the angular resolution of a barrel-distortion objective, according to the angle of incidence, Figure 5 is a diagram of the image plane of the objective, with examples of arrangement of an array of optical sensors therein, FIGS. 6a, 6b show two examples of arrangement of array of optical sensors in the image plane of l 'Objective, Figures 7a, 7b schematically show two different arrangements of an optical device in a passenger compartment in side view.

In all the figures, the same references relate to the same elements.

The embodiments described with reference to the figures are examples. Although the description refers to one or more embodiments, this does not necessarily mean that each reference relates to the same embodiment, or that the characteristics apply only to a single embodiment. Simple features of different embodiments can also be combined to provide other embodiments.

The optical elements, in particular the lenses used, have an invariance by rotation around an optical axis Ox, which makes it possible to define a longitudinal direction. The image plane of the optical device makes it possible to define, with local gravity, the transverse directions Oy and vertical Oz.

FIG. 1 schematically shows a passenger compartment 1 of a motor vehicle with an optical device 100.

The optical device 100 is here installed at the center console of the vehicle: the vertical or inclined front wall, located between the driver and the passenger at the front of the vehicle. This location allows the driver, who is here the user U of the optical device 100, to be in the field of vision of the optical device 100 in the normal driving position.

Alternatively, the optical device 100 can be installed at the level of a ceiling module or at the level of or in the mirror.

The optical device 100 is, for example, part of a module for monitoring the vigilance state of the driver and for detecting the presence of a passenger. Such a monitoring module can be used both to determine a level of attention of the driver U and to establish the presence or absence of a passenger, as well as for example the possible size of the passenger for the selective activation of the passenger airbags. .

The optical device 100 can be part of a control module by gesture recognition.

Such a control module allows the control of at least one function of an organ of the motor vehicle by performing specific gestures in the field of vision of the optical device 100, functions such as the control of an air conditioning system, an audio system, a telephone system or a navigation assistant. The control module can also be used for window regulator controls, positioning of exterior mirrors or for moving motorized seats or for controlling interior lights, central locking, a sunroof, hazard lights, lights or the handbrake.

A simplified view of the passenger compartment 1 of the vehicle is shown in plan view in FIG. 2. In FIG. 2 is also shown schematically in more detail the optical device 100.

The passenger compartment 1 comprises a driver's seat 3 and a passenger seat 5, arranged on each side of a central console 7, here in particular comprising a display screen 9.

The optical device 100 comprises an objective 101 with one or more lenses defining an optical axis Ox, and an array of optical sensors 103.

The array of optical sensors 103 is arranged parallel to the focal plane of the optical objective 101 in the image plane of the optical objective 101 and therefore perpendicular to the optical axis Ox.

-7The optical lens 101 presents spatially variable distortion. According to an exemplary embodiment, it is a wide angle lens which, according to one embodiment, has optics or at least one spherical lens.

Such wide-angle objectives have a very short focal length, in particular with a significantly curved and spherical face, so that obtaining a viewing angle is very large, typically of the order of 100 ° and more.

Other spatially variable distortions can be envisaged, for example with an optical objective having lenses with low focal length, to generate an image covering a narrower field of vision, while maintaining two or more levels of resolution.

The optical objective 101 has for example a barrel distortion, that is to say that the incident rays with a high angle relative to the optical axis are concentrated on an area of smaller surface in the objective image plane optical 101. Objects located at the periphery of the field of vision then appear as compressed.

It is observed that the compression is all the more important as one moves away from the center of the image obtained.

The optical device 100 may also include means for positioning and translating the objective 101 along the optical axis Ox, to allow dynamic focusing.

In the present embodiment, the optical axis Ox of the objective 101 is directed towards the expected position of the conductor U, and more particularly of the face and / or the bust thereof, that is to say at the level of the headrest of the driver's seat 3, this, as will be described later to allow observation at high resolution.

The matrix of optical sensors 103 is for example a CCD matrix ("charge coupled device") or a matrix of a CMOS sensor comprising a matrix of miniature photodiodes. The matrix of optical sensors 103 is in particular rectangular.

The array of optical sensors 103 comprises a grid of sensors

-8 individual optics, with each optical sensor being associated with one or more pixels of the image captured by the array of optical sensors 103.

The array of optical sensors 103 is arranged in the image plane of the objective 101, eccentrically, that is to say that the center C of the array of optical sensors 103 is eccentric and therefore offset with respect to the optical axis of the optical lens 101.

As can be seen in FIG. 2, the center C of the optical sensor array 103 is offset from the optical axis Ox in the direction of the driver's seat 3.

As a result, the field of vision of the optical device 100 is then asymmetrical, with a greater angle of observation on the side of the passenger seat 5 which is then also observed by the optical device 100.

With reference to FIGS. 2 and 3, the field of vision of the optical device 100 can then be divided into three conical zones, with apex located at the center O of the lens 101. The three zones comprise two zones of lower resolution LD separated by a zone high resolution HD centered on the optical axis Ox. The lower resolution zone LD on the left in FIG. 2 has an angular opening less than the low resolution zone LD on the right.

In FIG. 3, it can be seen in more detail that the array of optical sensors 103 is arranged eccentrically with respect to the optical axis Ox, with an offset δ in the image plane of the objective 101.

The image captured by the sensor array 103 corresponds to the field of vision of the camera, which is in the form of a cone with vertex O, delimited by the projection of the edges of the array of optical sensors 103. Due to the offset δ (here downwards on the image of FIG. 3) with respect to the optical axis Ox, said cone is asymmetrical with respect to the optical axis Ox.

Due to the distortion described above, the objective 101 has a greater angular resolution around its optical axis Ox, so that by the arrangement of the matrix of optical sensors 103 relative to the optical axis Ox of l objective, the high resolution HD area of the field of vision is

-9excentrée.

In particular, in FIG. 3, the lower resolution zone LD situated below the high resolution zone HD is cone-shaped with less angular opening than the lower resolution zone LD situated above the high resolution zone HD .

FIG. 4 is a graph of the resolution R in pixels by degree of field of vision (px / °), as a function of the angle a of observation relative to the optical axis Ox.

The resolution of a lens 101 with conventional barrel deformation is, at least around the origin, in the form of a convex curve, symmetrical with respect to the ordinate axis.

To allow an evaluation of a state of attention of the conductor U, details such as the pupils of the eyes and their position are to be taken into account, for example to determine the direction of gaze of the conductor U, which implies distinguishing from details on the order of three millimeters. Considering that the conductor U is at a distance of approximately one meter from the optical device 100, a pixel must correspond to an angular opening of approximately 0.15 ° (arctan (0.003)), or a resolution of the order of l 'order of six pixels per degree of field of view.

To assess the presence and size of a passenger, a resolution of the order of a centimeter is required. Considering that the passenger is at a distance of approximately one meter from the optical device 100, a pixel must correspond to an angular opening of approximately 1.5 ° (arctan (0.01)), or a resolution of the order of 0.6 pixels per degree.

There is thus a high resolution limit value Rhd which can be of the order of three to ten pixels per degree of angle, and a low resolution limit value Rld of the order of 0.3-1.0 pixels per degree .

These considerations are reflected in the focal plane or the image plane of the lens 101 as illustrated in FIG. 5. FIG. 5 is a representation of the image plane of the objective 101, with two axes corresponding to the directions y

-10 (ordinates) and z (dimension). In the image plane of the optical objective 101, two concentric circles 11, 13 are represented, corresponding to the points of resolution being worth respectively Rld for the outside circle 11 and Rhd for the inside circle 13. Thus, the resolution inside the circle 13 is high and the resolution between circle 13 and outer circle 11 is lower.

The diameter of the outer circle 11 is noted D in the image plane. The diameter D can be calculated according to the angle a at which the low resolution Rld is reached as being worth approximately D = / * tan (a) with / the focal distance (considering a focusing around the focal plane). The disk contained in this outer circle 11 corresponds to the portion of the image plane that can be used to obtain images of a resolution greater than the low resolution value Rld ·

Inside the inner circle 13, the resolution is greater than Rhd and between the inner circle 131 and the outer circle 11, the resolution is between Rhd and Rld, at the border of the high resolution area HD defined by the circle interior 13, the resolution being worth Rhd ·

Two examples of an array of optical sensors 103 are shown in FIG. 5, referenced 103a and 103b with their respective centers Ca and Cb.

As can be seen in FIG. 5, the extension d of the rectangle occupied by the array of optical sensors 103 is smaller than the diameter of the image circle D corresponding here to the outer circle 11 to allow its decentering in the outer circle 11 .

The first example of an optical sensor array 103a is of reduced diagonal, being 3D / 4. The second example of an optical sensor matrix 103b is diagonal d being equal to DU. The optical sensor arrays 103a, 103b are arranged in the outer circle 11 with their upper right vertex in contact with said outer circle 11.

The diagonal values d for the arrays of optical sensors 103 preferably comprised between Q, 5D and Q, 9D allow an offset δ, notably comprised between 0, ld and 0.7d, with respect to the centered position

-11susceptible making it possible to capture both an image of the driver U in high resolution and of the rest of the passenger compartment 1, in a lower resolution but greater than the low resolution value Rld.

Alternatively, it is possible to shift the matrix of optical sensors 103a even further towards the outside of the image circle. We can thus have a part of the pixels around the corner located outside the outer circle 11 whose angular resolution lower than the low value 7? Ld which may not allow exploitation, but we can however offset the upper portion HD resolution of the image closest to the edge of the image.

Other forms of matrix of optical sensors 103 can also be used, for example a shape that is thinner at the edges, taking into account the barrel distortion of the final image obtained.

Figures 6a and 6b illustrate the offset of the array of optical sensors 103 relative to a centered position. FIGS. 6a, 6b represent the portion of the image plane of the lens 101, with the outer 11 and inner 13 circles corresponding to the areas of lower resolution LD and of high resolution HD.

In FIG. 6a, the offset is only at y (ordinates), in FIG. 6b, the offset δ also includes a component <5 _Z at z (dimensions).

In FIG. 6a, the array of optical sensors 103 is shifted to the left in the image plane relative to the centered position, shown in dotted lines. The offset <5 _y along the abscissa axis y makes it possible to adjust the position of the cone of vision in high resolution HD in the field of vision of the optical device 100 in the horizontal plane xOy (FIG. 2). The optical sensor array 103 and the optical objective 101 are dimensioned and arranged so that the optical sensor array 103 entirely contains the circular portion of the image plane of the lens 101.

In FIG. 6b, the offset δ of the matrix of optical sensors 103 comprises a component <5 _y along the abscissa axis y, and a component <5 _Z along the axis of the dimensions z.

-12 Figures 7a, 7b illustrate the effect of the component <5 _Z along the axis of the z dimensions of the offset δ and its application. Figures 7a, 7b schematically represent a passenger compartment 1 of a motor vehicle, in side view, in which the driver seat 3 is essentially visible, with the driver U seated above and holding the steering wheel of the motor vehicle.

In FIG. 7a, the optical device 100 is arranged at the level of the central console 7 of the vehicle, at the height of the steering wheel of the vehicle. The overall field of vision of said device 100 has an angular opening going from the pelvis of the conductor U to the hands of the conductor U placed on the steering wheel. The high resolution portion of said field of vision, hatched in FIG. 7a, is centered on the driver's face, and is located relatively in the middle of the overall field of vision.

For this embodiment, the vertical component <5 _Z of the offset δ is then small or even zero, which corresponds to the arrangement of FIG. 6a.

In FIG. 7b, the optical device 100 is arranged at the level of a ceiling lamp or a central rear view mirror of the vehicle. The overall field of vision of the vehicle has an angular opening going from the top of the skull of the driver U to the hands of the driver U placed on the steering wheel. The high resolution portion of said field of vision, hatched in FIG. 7b, is centered on the driver's face, and is therefore almost at the highest in the overall field of vision.

In particular, the portion located under the high resolution portion of the field of vision in low resolution, which covers the torso of driver U, his arms and his hands resting on the steering wheel, is greater than the portion of low resolution located above the high-resolution hatched portion, which is almost non-existent.

To obtain this upward decentering of the high resolution portion, an offset δ with a component <5 _Z along the axis of the dimensions z corresponding to FIG. 6b is indicated.

As part of a state of attention monitoring module

-13conductor U, the matrix of optical sensors 103 is connected to a control unit (not shown) comprising in particular an image processing unit.

To do this, the control unit includes calculation means and an electronic memory. The control unit can in particular be dedicated or integrated into a global electronic circuit of the vehicle. The control unit can in particular control one or more functional modules of the vehicle by means of electronic actuators (integrated circuits, transistors, logic switches).

The control unit receives the images recorded by the array of optical sensors 103 and analyzes them to extract data relating to the driver.

From the high resolution portion of the field of vision of the optical device 100, directed towards the face and the upper body of the conductor U, the control unit can extract information relating to one or more characteristics of the conductor U, from its behavior and its physiological state.

For example, the control unit can deduce from the images a direction of gaze of the driver U, in particular if he is looking towards the front of the vehicle, if he is looking at a point located in the passenger compartment, or if he is looking through a side window.

If driver U looks at a point inside the passenger compartment or through a side window for an extended period, the control unit can trigger the emission of an alert, either with indicator lights, the display of a message on a screen or the emission of a sound by the multimedia playback system of the vehicle.

As an alternative or in addition, the control unit can analyze the open or closed state of the eyes of the driver U, and by monitoring over a predetermined period, deduce therefrom a frequency of blinking of the eyelids of the driver U.

If the eyes of the conductor U remain closed for a long time, the

-14control can trigger the emission of a sound, in particular acute or shrill, to wake up the possibly asleep driver U, and / or invite him to pay his attention to traffic.

Similarly, a high or increasing frequency of blinking or yawning of the driver U can trigger the display or audio playback of a message inviting the driver to take a break or change places with the passenger.

To complete the diagnosis of the vigilance state of the driver U, the control unit can use data extracted from the low resolution portion of the field of vision of the optical device 100, such as the position of the bust of the driver U, the position with his hands, especially in relation to the steering wheel.

These data can either be continuously monitored or, within the framework of an autonomous driving vehicle, in anticipation of a possible handing over of vehicle controls to the driver (entry into agglomeration, exit from motorway, approach to a potentially situation dangerous).

From the low resolution field of vision portion, the control unit can extract data relating to the passenger, such as his presence or absence, as well as his size.

Passenger data can be used to selectively activate a portion of the passenger, front and side airbags, depending on the presence and size of the detected passenger.

Finally, the control unit can be configured to isolate specific gestures in the images, in particular in a space delimited from the passenger compartment (for example in front of the display screen 9 or the central console 7), corresponding to the control. various vehicle functions (multimedia device, navigation assistant, interior and exterior lighting).

Other uses are also possible in sensors, detectors and surveillance systems requiring combining an observation of a small space in high resolution HD, and a large space with a resolution

-15Lower LD. For example, when observing users of an automatic teller machine (ATM), where the users' faces can be recorded in high resolution, while allowing a large portion of the surroundings to be monitored at a lower resolution.

The optical device according to the invention requires only a slight oversizing of the objective with respect to the array of optical sensors 103, and can in particular use an optical objective 101 with a wide angle of conventional invoice. The device is therefore invariant by rotation around the optical axis Ox, and therefore easy to set up. In particular, the optical device 100 and the monitoring module obtained by means of the latter are more compact and less expensive than a solution combining two cameras, one at wide angle and one at high resolution, while allowing observation by two simultaneous resolution levels.

Claims (12)

1. Optical device for observing a user (U) and a space surrounding the user, comprising: ° an optical objective (101), defining an optical axis (Ox) and an image plane, ° an array of optical sensors (103), arranged in the image plane of the optical objective (101) characterized in that: ° the optical objective has a spatially variable distortion, and ° the center of the optical sensor array (103) is arranged eccentrically with respect to the optical axis of the optical objective (101). 2. Optical device according to the preceding claim, characterized in that the offset (<5) between the center of the optical objective (101) and the optical axis (Ox) is between 0.1 // and 0.7 //, d being the spatial extension of the array of optical sensors (103) in the image plane 3. Optical device according to any one of the preceding claims, characterized in that the distortion of the optical objective (101) is in a barrel and increasing with the deviation of the incident rays from the optical axis (Ox). 4. Optical device according to any one of the preceding claims, characterized in that the array of optical sensors (103) is a rectangular array of optical sensors (103) of diagonal d. 5. Optical device according to any one of the preceding claims, characterized in that the optical objective (101) comprises a high resolution portion (HD), and an annular portion (LD) of lower resolution surrounding the high resolution portion . 6. Optical device according to claim 5, characterized in that the resolution in the high resolution zone is greater than a high resolution value (Bhd) between three and ten pixels per degree of angle. 7. Optical device according to one of claims 5 or 6, characterized in that the resolution in the annular zone is between a low value (Bld) being between 0.3-1.0 pixels per degree of angle and the high resolution (Bhd) value at the border with the high resolution (HD) portion. 8. Device according to claim 7, characterized in that the optical sensor array (103) is arranged with an apex of the rectangle disposed on the image circle (11) corresponding to the low resolution value (Bld). 9. Module for monitoring the state of attention of the driver of a vehicle and for detecting the presence of a passenger, characterized in that it comprises an optical device (100) according to any one of claims 1 to 8, and in that the optical axis (Ox) of the optical objective (101) is directed towards an expected position of the face or the bust of the driver (t /) or of a passenger of the vehicle. 10. Monitoring module according to the preceding claim, characterized in that it further comprises a control unit configured to isolate in the portion of the matrix of optical sensors (103) contained in the high resolution portion of the characteristics of the face of the conductor (t /) such as: ° the direction of the driver's gaze, ° the open or closed state of the driver's eyes (t /), ° the driver's eyelid blinking frequency. 11. Monitoring module according to claim 9, characterized in that it further comprises a control unit configured to isolate in the images originating from a portion of the matrix of optical sensors (103) contained in the low resolution annular portion passenger or driver characteristics (U) such as: ° the presence or absence of a passenger, ° the size of the passenger, ° the position of the driver's hands, ° the position of the driver's chest. 12. Monitoring module according to claim 9, characterized in that it further comprises a control unit configured to isolate in the images from the array of optical sensors (103) specific gestures performed by the driver (t / ) and / or the passenger for checking the vehicle's functions.