WO2019220256A1 - A method of measuring illumination, corresponding system, computer program product and use - Google Patents

A method of measuring illumination, corresponding system, computer program product and use Download PDF

Info

Publication number
WO2019220256A1
WO2019220256A1 PCT/IB2019/053669 IB2019053669W WO2019220256A1 WO 2019220256 A1 WO2019220256 A1 WO 2019220256A1 IB 2019053669 W IB2019053669 W IB 2019053669W WO 2019220256 A1 WO2019220256 A1 WO 2019220256A1
Authority
WO
WIPO (PCT)
Prior art keywords
values
environment
image
radiosity
method
Prior art date
Application number
PCT/IB2019/053669
Other languages
French (fr)
Inventor
Theodore TSESMELIS
Irtiza HASAN
Fabio Galasso
Herbert Kaestle
Alessio DEL BUE
Marco Cristani
Original Assignee
Osram Gmbh
Fondazione Istituto Italiano Di Tecnologia
Università Degli Studi Di Verona
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to IT201800005381 priority Critical
Priority to IT102018000005381 priority
Application filed by Osram Gmbh, Fondazione Istituto Italiano Di Tecnologia, Università Degli Studi Di Verona filed Critical Osram Gmbh
Publication of WO2019220256A1 publication Critical patent/WO2019220256A1/en

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRA-RED, VISIBLE OR ULTRA-VIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • G01J1/02Details
    • G01J1/0228Control of working procedures; Failure detection; Spectral bandwidth calculation
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRA-RED, VISIBLE OR ULTRA-VIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • G01J1/02Details
    • G01J1/0266Field-of-view determination; Aiming or pointing of a photometer; Adjusting alignment; Encoding angular position; Size of the measurement area; Position tracking; Photodetection involving different fields of view for a single detector
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRA-RED, VISIBLE OR ULTRA-VIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • G01J1/42Photometry, e.g. photographic exposure meter using electric radiation detectors
    • G01J1/4204Photometry, e.g. photographic exposure meter using electric radiation detectors with determination of ambient light
    • H05B47/11
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRA-RED, VISIBLE OR ULTRA-VIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • G01J1/42Photometry, e.g. photographic exposure meter using electric radiation detectors
    • G01J2001/4247Photometry, e.g. photographic exposure meter using electric radiation detectors for testing lamps or other light sources

Abstract

A method of measuring illumination of an environment (A) comprises:- capturing (e.g. via an RGB-D camera arrangement 12) images of an illuminated (L) environment (A), - extracting (10) from the image captured light emittance values M(x,t) (or R(x,t)) as well as albedo values Ρ(x) of surface areas ("patches") across the environment (A),- computing (10) illumination values E(x,t) of surface areas across the environment (A) as a function of the emittance values M(x,t) and the albedo values Ρ(x).

Description

A ME THOD OF MEASURING ILLUMINATION , CORRESPONDING SYSTEM , COMPUTER PROGRAM PRODUCT AND USE

k k k k

Technical field

The description relates to lighting techniques.

One or more embodiments may be applied to "smart" lighting systems for lighting e.g. an indoor environment by measuring the illumination thereof.

Throughout this description, various documents will be referred to by reproducing between square parentheses (e.g. [X]) a number identifying the document in a LIST OF DOCUMENTS CITED which appears at the end of the description.

Technological background

Evaluating and measuring the spatial illumination changes of luminous indoor environments may play a significant role in lighting practice. The spatial illuminance distribution on surfaces of objects and light-emitting surfaces is oftentimes non-uniform. The non-uniformity level may vary dramatically from one surface to another and even from one sub-area to another sub-area of a same surface. A non uniform illumination of surfaces may be the source of discomfort to observers, insofar as this may militate against proper eye adaption.

Therefore, being able to measure and evaluate a non-uniform illuminance distribution and its changes over space and time is a desirable goal to pursue. For lightplanners , this may involve the capability of modeling the illumination distribution of a scene by simulating the luminous flux which derives both from installed light sources and from the cumulative light emittance of surface elements across the scene.

Uncomplicated methods for measuring luminous illuminance across a scene, e.g. in order to verify the results achieved by means of a certain lighting arrangement are thus desirable.

Current methods for measuring and evaluating spatial and temporal illuminance/luminance (light intensity level) over space and time may include, e.g.:

- direct measurement with conventional luxmeters, camera-based measurements determining the luminance and the emittance of a scene,

using certain conventional S/W-based light- planning tools as Dialux, Relux and others, as discussed in the following.

Direct measurement (e.g. point-wise measurement with conventional luxmeters across the scene) may suffer from low spatial resolution in limited areas with direct access.

This is particular inconvenient and expensive in large architectural spaces with varying daylight illumination over time.

The output from a camera (image pixels) may be calibrated by referring to a certain, known light source. Each pixel value in a known range can be mapped onto a corresponding light intensity value which may then be used for deriving a spatial emittance of a Lambertian scene surface.

Based on the knowledge of a corresponding albedo map (reflectivity) , the prevailing input illumination intensity E [lm/m2] - also denoted as the input- illuminance for an elementary surface - can be derived directly from the camera pixel response as:

Figure imgf000004_0001
namely, by considering the lambertian albedo value
Figure imgf000004_0002

where : R = exit intensity (emittance, exitance) of a surface element;

E = input illumination intensity (input illuminance, input flux density) on a surface element in the scene;

Lpjxei = measured visible luminous flux of the observed surface element in the scene (calibration of the camera response is assumed) ;

p = Lambertian albedo coefficient of the surface element in the scene (albedo map = diffuse map) .

Albedo denotes the reflecting power of a body, expressed as a ratio of reflected light to the total amount falling on the surface.

When using conventional S/W-based light-planning tools as Dialux [1], Relux [2] and others, the spatial illumination intensity across the scene surface can be determined by considering the mutual inter-illumination of the surface elements.

Certain parameters, such as the albedo values of the surface of the scene, can be obtained as experimental values (e.g. table values from data libraries) .

The table-based reflectivity values of the scene surface can thus be considered as an ad-hoc assumption for describing the scene.

Also, the geometry and positioning of the light sources as well as the lighting intensity of the scene can be considered to be known.

Object and summary

An object of one or more embodiments is to contribute in providing further improvements in the area of technology discussed in the foregoing by overcoming various drawbacks of the solutions discussed in the foregoing.

According to one or more embodiments, such an object may be achieved by means of a method as set forth in the claims that follow.

One or more embodiments may relate to a corresponding system.

One or more embodiments may relate to a computer program product loadable in the memory of at least one processing module (that is, a computer) and including software code portions for executing the steps of the method when the product is run on at least one processing module.

As used herein, reference to such a computer program product is understood as being equivalent to reference to a computer-readable means containing instructions for controlling the processing system in order to co-ordinate implementation of the method according to one or more embodiments.

Reference to "at least one computer" is intended to highlight the possibility for one or more embodiments to be implemented in modular and/or distributed form.

One or more embodiments may relate to possible use of a method as disclosed herein.

The claims are an integral part of the disclosure provided herein in relation to the one or more embodiments.

One or more embodiments can provide a cheap, convenient and effective (e.g., camera-based) method for determining albedo values (albedo map) across a scene .

For instance, the albedo values p can be retrieved if the luminaire-based illumination is known. Otherwise, if the albedo values p are known, the luminaire-based illumination can be retrieved.

One or more embodiments can provide geometrical form factors which may be used for conventional light simulation tools such as those based, e.g., on ray tracing and/or radiosity.

One or more embodiments facilitate measuring the resulting illumination values across the scene for verification. In one or more embodiments, this may occur through a constrained radiosity model under the assumption that raw pixel values can also be used as a pre-estimation of the illumination values as measured.

For instance, under the assumption of a calibrated camera and under the assumption of a Lambertian scene, the pixel-values from the camera provides in the ideal case the exact illumination values over the scene.

The designation "ray tracing" applies to techniques that can generate near photo-realistic computer images. A variety of software tools (both free and commercial) are currently available for generating corresponding images.

Similarly, "radiosity" denotes the light energy given off by a surface.

The amount of light emitted from a surface can be specified as a parameter in a model, where the reflectivity of the surface is also specified.

For instance, the amount of light emitted from a surface element (patch) depends on its albedo value p and the amount of its incident illumination, which is the sum of the direct illumination from the luminaires and the indirect illumination from the adjacent surface patches of the environmental objects.

The amount of incident light hitting the surface can be calculated by adding the amount of energy other surfaces contribute.

Various rendering methods used in computer graphics may be based on radiosity in order to analyze the light reflected from diffuse surfaces.

Light at a given point can be modelled as comprising diffuse, specular, and ambient components, with the possibility of resorting for simplicity to an ideal model which handles only the diffuse ambient component (Lambertian scene) while retaining the possibility of also considering specular and ambient components .

These components can be combined to determine the illumination or color of a point.

The images resulting from a radiosity renderer may exhibit soft, gradual shadows. Radiosity can be used, e.g. to render (indoor) images of buildings with realistic results for scenes including diffuse reflecting surfaces.

Also, by using a camera, one or more embodiments can cover larger areas with a (much) higher spatial resolution as compared to conventional manual light measurement .

One or more embodiments facilitate real-time measurement of the illumination across a scene even in the absence of previous knowledge of the environment.

One or more embodiments can be automated making it unnecessary to rely on the expertise of dedicated technical personnel.

In one or more embodiments, multiple light sources and/or multiple types of light sources (natural/artificial) can be addressed simultaneously.

In those cases where a complete geometry and reflectance map of the scene is acquired, the possibility exists, at least notionally, of providing information also for those areas of, e.g., a room that the camera sensors do not "see".

One or more embodiments may consider (only) the visible part of the scene, since as the part where form factors and corresponding albedo values can be measured. Brief description of the figures

One or more embodiments will now be described, by way of example only, with reference to the annexed figures, wherein:

- Figure 1 is exemplary of a space (e.g., an indoor environment) where one or more embodiments are applied,

Figure 2 is an exemplary view which can be produced in one or more embodiments,

- Figure 3 is exemplary of certain processing which may be implemented in embodiments, and

- Figure 4 is exemplary of a processing pipeline which can be implemented in one or more embodiments.

Detailed description of exemplary embodiments

In the following, one or more specific details are illustrated, aimed at providing an in-depth understanding of examples of embodiments.

The embodiments may be obtained without one or more of the specific details, or with other methods, components, materials, etc.

In other cases, known structures, materials, or operations are not illustrated or described in detail so that certain aspects of embodiments will not be obscured .

Reference to "an embodiment" or "one embodiment" in the framework of the present description is intended to indicate that a particular configuration, structure, or characteristic described in relation to the embodiment is comprised in at least one embodiment.

Hence, phrases such as "in an embodiment" or "in one embodiment" that may be present in one or more points of the present description do not necessarily refer to one and the same embodiment. Moreover, particular conformations, structures, or characteristics may be combined in any adequate way in one or more embodiments.

The references used herein are provided merely for convenience and hence do not define the extent of protection or the scope of the embodiments.

One or more embodiments may be based on the recognition that conventional illuminance measurements obtained from multiple discrete points may not reflect spatial changes of light levels with a desired degree of resolution.

This may be related to the fact that conventional luxmeter-based illuminance measurement solutions are limited (only) to certain areas of installation (point- wise measurements) .

These conventional meter-based methods for light level measurement inevitably involve limitations in interpreting spatial changes of a luminous environment in large architectural spaces and/or luminous environments exposed to varying daylight.

It is noted that a camera-based measurement of the illuminance of a surface in relation to directly- converted emittance of a surface may represent an alternative worth being investigated.

The pixel intensity from a camera sensor is a relative measure of the light reflected by a surface (luminance) which is the visible brightness of the object surface element.

Such a visible brightness may depend on a variety of components related to the materials of the object (s) and the reflectance properties thereof. Also, such visible brightness may depend on ambient (e.g., room) geometry and on how light propagates in the environment and is finally influenced by the amount of light reaching the given surface (luminous flux) .

The knowledge of these characteristic parameters of the environment facilitates developing a method for computing spatial illuminance ratios of the surface elements involved and providing a reliable estimation of the global illumination uniformity across a scene even when using the restrictive assumption of Lambertian surfaces.

To that effect a number of characteristic parameters of the environment may be collected upfront.

These may include:

- illumination power and position with respect to lighting sources (e.g., luminaires);

- albedo map of the scene surface;

- geometrical layout of the scene surface; this facilitates determining geometrical form factors of the scene for calculating the influence on the illumination level of the surface elements which may derive primarily from direct illumination of the luminaires and may also be due to mutual surface-to-surface inter radiation .

These parameters of the scene may be used both for simulating the illumination with common light planning tools (such as ray tracing or radiosity tools) and for verifying the results by measuring the illumination values across the scene in the aftermath.

As exemplified in Figure 1, one or more embodiments may comprise a processing unit 10 (e.g., an embedded platform) coupled - in any known manner for that purpose - with a set of sensors (e.g. a RGB camera sensor and a 3D-depth-camera sensor) collectively designated 12.

In one or more embodiments, the set of sensors 12 can be installed, e.g. at a "elevated" position in an ambient (e.g. a room A) to be lit, e.g., via luminaires L, which facilitates obtaining a perspective top view of the ambient.

Figure 2 is exemplary of such (e.g. "fish-eye") view .

In one or more embodiments as exemplified in Figures 1 and 2, the possibility thus exists of providing a camera-based long-term observation of the ambient (that is, the "scene") A, e.g., by using an RGB camera sensor 12a in the set 12.

In one or more embodiments, the position and the strength (intensity) of the light sources (e.g. luminaires) L can thus be derived (detected) . This may occur, e.g., as disclosed in [3], or by resorting to other known alternative solutions.

Similarly, by considering the geometrical layout from a 3D-depth-sensor 12b possibly included in the set 12, a corresponding albedo map of the scene A can also be derived. This may again occur, e.g., as disclosed in [3], or by resorting to other known alternative solutions .

By considering the geometrical layout from a 3D- depth-camera the corresponding form factors of the scene can also be derived, thus obtaining a set of valuable features for a radiosity model.

As discussed previously, radiosity is a (global) illumination model commonly used in the field of computer graphics for representing photorealistic rendering techniques.

The input of the classic radiosity method consists of a list of patches (e.g., triangles may be considered as an example for simplicity) having:

- an (average) self-emitted radiosity eps\,

- an (average) reflectivity p,

from which (average) total radiosities r,· can be computed .

That is, the input to a conventional classic radiosity model may comprise three main parameters for a list of patches (triangles, tetrahedral, etc) into which an entire scene can be "scattered".

Also, mutual geometrical interaction/representation can be extracted as a square matrix called form factor matrix, along with their average self-emittance and reflectivity factor (albedo) .

This may occur according to arrangements that are known per se, thus making it unnecessary to provide a more detailed description herein.

A possible mathematical formulation can be as follows :

Figure imgf000013_0001
or, in the M-notation
Figure imgf000013_0002
where

R(x) or M(x) is the radiosity, that is, the (total) power per unit area leaving the patch surface;

Eps(x) is the emitted power per unit area leaving the patch surface due to self-radiation which will be caused due to self-glowing surface elements;

P(x) is the Lambertian albedo reflectivity of the point ;

S indicates that the integration variable x' runs over the entire (visible) surface in the scene;

- qc and qc' are the angles of the normal in x and x' to their joining line (that is a simple geometry for describing the adjacency of patches with respect to inter-illumination .

As known to those of skill in the art R(x), or M(x) are different notations which can be used to represent the same variable (i.e. radiosity) . In the following formulas in both notations will be provided. For instance, a discrete notation at vector component level can be provides as:

Figure imgf000014_0001

or, in the M-notation

Eps(x,t)— M(x,t)— >;(1— dCr px FXp Mp

p

where the summation å extends from j=l to n over the entire number of surface elements.

In the formulas above, Sjj is the Dirac symbol with 1 for i equal j, else 0: this means that a certain patch i cannot be inter-illuminated by itself, so all the terms with i=j are skipped or nulled in the summation, by means of a cancelling term (l -Sjj) , that is:

Figure imgf000014_0002

Also, r,· and h are the radiosities at patches i and j , respectively and the scalar eps refers to the self- emittance of patch i.

For instance:

- if one has to do with a light source, this value can be set either to 1 (where 1 corresponds as the maximum luminous intensity of the light source) or to the actual luminous intensity of the light source if this is known;

- otherwise, if the patch does not belong to a light source, the value can be set to 0.

Additionally, p is the isotropic reflectivity of patch i and fij is representative of the form factors, e.g., between patch i and patch j .

Various numerical approaches are available for computing the form factors fj . Of these, a ray-tracing- based method may represent a good choice, e.g., due to a more general applicability and efficiency [4, 5] .

More in detail, the form factors can be computed by uniformly sampling rays within a unit disc (i.e. the orthogonal projection of a unit sphere), whereby each point on the unit disc defines therefore the direction of a ray in the space. Therefore, the form factors fj can be computed as the ratio:

kj

f‘>=i

where kj stands for the number of rays emitted by patch i that reaches patch j, and /g is the total number of rays emitted by facet i .

Form factors may encode two main aspects:

visibility, that is whether two patches are visible one from the other; e.g. this value is equal to zero if there is no line-of-sight between them;

- distance and orientation, which is indicative ("encodes") how well two patches see each other; e.g. low values correspond to very far patches with an oblique line of sight, while high values refer to close fronto-parallel patches.

Form factors may be further constrained to be strictly non-negative and to satisfy the reciprocity relation aifij = ajfji, where a is the area of each patch (ίm is not a symmetric relation if patches do not have a same size) . Finally, if the scene is closed, the form factors fij between a patch i and all other patches j in the scene add up to one while if the scene is open the sum is less than one.

The radiosity formulation above may give rise to a set of linear equations which can be solved through the radiosity matrix solution as:

Figure imgf000016_0001

There, F is an n*n square matrix describing the geometry of the whole scene A in combination with the material reflectivity properties p, which is a vector of dimension n.

Then the vector r of dimension n contains the associated radiosities at each patch and the self emission eps vector contains non-zero values at patches corresponding to active light sources.

It will be noted that throughout this description vector notation is not expressly used for simplicity.

One or more embodiments may rely on the concept of resorting to a radiosity model in order to facilitate modeling the effectiveness of light sources L in a space by rendering the visual impression at a certain viewpoint .

For instance, one may consider an indoor environment to be lit of the type depicted in Figure 1.

In general, this may be expected to be:

illuminated by different illumination sources both artificial (e.g., luminaires L) and natural (e.g. windows exposed to daylight) ,

"populated" (e.g., furnished) with different materials and objects (with different photometric properties), and

- oriented in a random way.

As discussed previously, up in the ambient A (e.g. at the ceiling thereof, possibly at the center) a sensor system 12 may be installed comprising, e.g.: a simple RGB camera (possibly with fish-eye lenses), designated 12a, and

- a depth measuring sensor, e.g. time of flight (ToF) or Lidar, designated 12b, this being also currently referred to as an RGB-D system.

This environment (ambient A) can thus be subjected to surveillance (monitoring) over time by recording the images thereof provided by a camera system as discussed previously, e.g. from a top view perspective as exemplified in Figure 2.

The output of the camera system 12a, 12b, e.g. a

RGB image (e.g., fish-eye) - from 12a - and depth information as a point cloud and distance map - from 12b - are exemplary of information which may be collected in one or more embodiments.

Just by way of reference (e.g. for visualization easiness) that information can be considered to represent a sort of partial CAD model of the point cloud which can be used for further processing acts as discussed in the following.

In one or more embodiments, the views from the two sensors (e.g. RGB camera 12a and depth measuring sensor 12b) can be pre-aligned and synchronized.

This facilitates mapping the RGB image over the depth information as schematically exemplified in Figure 3.

That figure shows, by way of example, RGB image pixel intensities mapped over a 3D model (depth information) of an ambient A as exemplified in Figures 1 and 2.

One or more embodiments may resort for that purpose to other arrangements such as, e.g., an automatic PnP solution as described in [6] .

Point-to-Point (PnP) alignment is indicative of how a point from one view is matched with the same point in another view.

The related output - that is, images and a (partial) geometric representation - can be processed in the embedded (or server/cloud based) unit 10 coupled to the sensor system 12 based on a radiosity model in order to provide a measurement/evaluation of the spatial and/or temporal illuminance changes in the ambient A as viewed from the sensor system 12 (see e.g. Figure 2 ) .

Such a measurement/evaluation of the spatial and temporal illuminance/luminance changes in an (e.g., indoor) ambient A as detected make it possible to map pixel values onto respective, corresponding lux values.

The procedure can be repeated over time, e.g. in respect of time-synchronized frames taken from the two camera sensors 12a (RGB) and 12b (depth) to obtain a (partial) representation of the ambient (scene) A.

In one or more embodiments, the parameters from the camera-based long-term observation of the scene may provide a (complete) set of information items. This may be used, similarly to what happens in conventional light planning software, to simulate the prevailing illumination intensity across a scene.

One or more embodiments may thus involve the acquisition of physical components of a scene A which may use in providing a radiosity model based

(exclusively) on a set of observations by camera sensors 12a, 12b.

Once the luminous intensity is determined, measurements of the luminous intensity across the scene can be used for applications related to light commissioning and/or configuring and managing smart lighting systems.

For instance, the knowledge of (e.g. high- resolution) luminous intensity (illumination) values across a scene A may be helpful for light designers and light planners. Also, real-time measurement of the luminous intensity (illumination values) in high resolution across a scene may be exploited in deriving a representative monitoring as input for light management regulation loops.

One or more embodiments may rely on a procedure which takes into account the information provided from an RGB-D camera system (e.g. 12a, 12b) and processes it to provide real-time spatial and temporal measurements of the illumination of a scene.

In a first possible embodiment, the measured pixel values of the image taken from the scene by a calibrated camera system 12 as discussed previously can be directly used for computing a prevailing illumination value across the scene as L=L(Xt).

Here, L(Xt) is a 1-dimensional vector (having a size N = m x n, where m, n are the dimensions of the captured image) indicative of the measured luminance values across the scene, thus representing an image of the scene A captured by a calibrated camera system (e.g., 12 ) .

Under the (reasonable) assumption of a Lambertian scene, the measured luminance values can be converted directly to the corresponding emittance values R(x,t) or M(x t) of the surface areas across the scene as:

Figure imgf000019_0001
or, in the M-notation

Figure imgf000019_0002
where M(Xt) or R(Xt) is another 1-dimensional vector having a size n representative of the measured emittance values across the scene A, which represents the emittance of the surface patches across the scene.

The albedo values p(x) of the surfaces across the scene which are acquired from previous long-term observation of the scene may be subjected to an intrinsic decomposition procedure algorithm (as described, e.g., in [7]) or other techniques (as described, e.g. in [8]), so that p(X t) is a 1-dimensional vector of the size n, can be expressed as a vector of inverted components

Pi

Figure imgf000020_0001

where Pmv(x,t) is a 1-dimensional vector (of size n) of the inverted values of the acquired albedo values of the patches across the scene.

Based on the knowledge of the inverted albedo values Pinv(x,t) of the patches across the scene, the prevailing illumination E(Xt) across the scene can be derived as a simple vector dot-product, e.g. as:

Figure imgf000020_0002
where E(Xt) is a 1-dimensional vector of size n, which represents the illumination values of the surfaces across the scene.

For the sake of clarity, it will be noted that E(x>t) is the incident illumnation on the surface patch and is not related to E(x) or Eps(x) as the emitted power per unit area leaving the patch surface as discussed previously.

It will be appreciated that a vector dot product multiplies the mutual components and keeps the vector dimension . The entire formula chain, the concept of the camera based real-time measurement of the illumination across the scene can be summarized as:

Figure imgf000021_0001

This means that by the observation (measurement) of the scene with a calibrated camera (measuring the real J(x,t)) and by knowing the albedo p of the scene's patches, the real (net) incident illumination E(xt) in [lm/m2] can be derived.

In one or more embodiments, a solution as the one described above can be obtained by considering a radiosity model discussed previously.

Essentially, the radiosity model corresponds to the knowledge of the albedo values p and the form factors fij and the real position and luminous intensity of the light sources.

Consequently, by knowing the radiosity model it is also possible to decompose the measured net illuminations E(X t) into their components as the sum of direct illumination from the luminaires Ed(x,t) and the incident coming from the mutual inter-reflection as:

Ed = p * (/ - pF) * L pinv In one or more embodiments, this may occur according to a processing pipeline (which may be implemented in the unit 10) corresponding to the logical diagram shown in Figure 4.

In the diagram of Figure 4 the block 100 indicates RGB-D input data which may be acquired as discussed previously (camera system 12a, 12b) .

In one or more embodiments as exemplified in Figure 4, such data can be supplied to a set of processing blocks/modules 102 as discussed in the following .

In one or more embodiments as exemplified in Figure 4, the results of computing acts performed in the modules 102 are provided, along with RGB-D data 100 to a further block/module 104 configured for solving a (camera-aided) constrained radiosity model.

As a result the block/module 104 can provide a mapping 106 of radiosity values onto illuminance values adapted to be supplied to a "user" entity 108.

A light design/control/planning instrument/tool as currently used by light designers and light planners and/or a controller unit configured to control (possibly in real time) the lighting action of the luminaries L lighting the scene A may be exemplary of such a user entity 108.

It will be appreciated that configuring the block/modules collectively indicated as 102 in order perform the processing acts discussed in the following involves applying, e.g. software tools which are known per se, thus making it unnecessary to provide a more detailed description herein.

For instance, a radiosity model can be considered as a refinement of the solution discussed previously insofar as this may not depend on a pre-calibrated camera system and can also address the case of a noisy input by just considering the raw input from a camera system.

For instance, one or more embodiments may rely on the availability of, e.g. RGB image pixel intensities mapped onto depth information (3D model) as presented, e.g. in Figure 3.

The related information for the solution of the radiosity matrix can be attempted to be extracted as discussed previously. That information may include (based on a radiosity model) :

the reflectivity properties of the material/objects of the scene, p

- the form factors, fij

the position and identification of the light sources (e.g., L) in space, vector e (sometimes referred to as vector e = direct illumination) .

As noted, these elements can be extracted based on camera-aided methods proposed in the literature and are adapted for the real-time computation of the radiosity model .

To that effect, the block/modules collectively indicated as 102 in Figure 4 can be configured to comprise:

- a first module 102a for computing reflectance,

- a second module 102b for computing form factors,

- a third module 102c for computing the luminous intensity and the positions in space of light sources in ambient A.

In one or more embodiments, conventional techniques of intrinsic decomposition as described in [7], [8] and [3] can be applied in the first module

102a in order to determine the reflectivity properties of the material/objects of the scene by camera-based methods .

In one or more embodiments, the extraction of the geometrical form factors of the surface elements in the scene, as performed in the second module 102b may be based on 3D-data from a Time-of-Flight (ToF) detector or a Lidar can be considered.

In one or more embodiments, the third module 102c may extract the light sources position and identification in space from given images of the scene as disclosed, e.g. in [3], where the positions of the light sources were determined by evaluating long-term observations of the scene with the knowledge of the scenes geometry and lambertian reflectivity.

Under the circumstances, one can make the (sensible) assumption that raw format image pixels correspond to a noisy estimation of the actual radiosity values (see, e.g., [9]).

For that reason, one or more embodiments may not use a straightforward calculation of the illuminance by just considering them as the luminance in the

Lambertian assumption formulation, this may lead to calculating the illuminance with a higher error, which may be acceptable for certain applications.

In one or more embodiments, based on such information, the block 104 can solve the radiosity matrix based on a constrained radiosity estimation by defining e.g. a cost function such as:

minimize j!g—

r

Figure imgf000024_0001

subject to F r— e,

where :

- g is a sub-vector of r which contains the raw pixel information for the surfaces that are visible by the camera system, and

- Dp is the reflectivity properties corresponding to the visible surfaces.

In other words, one may calculate an optimized solution for r considering the raw pixel information taken from the images.

The calculated radiosity (radiant exitance) values can then be back-related to illuminance by considering the Lambertian assumption as presented in the equation above, where L(x t) is now replaced by the computed radiosity values R(x,t) or M(X t) .

Therefore the equation as discussed previously is formed as : E(x,t)— TC * R(x,t) ' Pinv(x,t )

or, in the M-notation

Figure imgf000025_0001

which may facilitate a dense measurement of the illumination across the scene.

Consequently, the camera-based observation of the scene facilitates the provision of the parameters involved in computing the illuminance across a scene by applying the state-of-the-art radiosity concept for scene rendering.

Figure 4 is an exemplary representation of a pipeline which facilitates determining (in a high spatial resolution of the scene) illuminance in an ambient A by computing the parameters to be given as input to the radiosity model, namely:

1) the reflectance values of the scene,

2) the geometrical form factors from the 3D input of the scene, and

3) the real position and luminous intensity of the light sources.

These three parameters (together with the actual pixel intensities) can be used in order to solve the radiosity formulation in an efficient way for computing the prevailing illuminance values.

In one or more embodiments the measured pixel values of the image taken (at 12) from the scene A are used by considering the radiosity model as an improved version of the embodiment discussed previously for computing the illumination value across the scene due to the direct illumination from light sources only.

The formula tool chain bases on the radiosity model for describing the global illumination can take the form:

Figure imgf000025_0002
or, in the i?-notation
Figure imgf000026_0001
that is:
Figure imgf000026_0002

or, in the M- notation

Figure imgf000026_0003
which corresponds to the formula seen previously, where :

- M(x t) or R(x,t) is the emittance of the patch at (x, t)

- Eps(x t) is the emittance of the patch at (x, t) , due to self-emittance from direct illumination

P(x) is the Lambertian albedo reflectivity of the patch itself

p (an abbreviated notation for the surface integration variable xp=xpatch ) indicates the integration variable across the entire surface in the scene

- Q and qr are the angles of the normals in x and xP=p to their joining line.

Also, the denoted integral refers to the collection across the entire visible surface of the scene .

In a discrete notation, the term within the integral can be written as a square matrix F(p x t)

Figure imgf000026_0004
or in the R-notation
Figure imgf000027_0001

where :

- drc is the Dirac symbol with value 1 for p different from x else 0, to facilitate avoiding to include the patch at fat), and

- Fpx represents the form factors as a square matrix of size n x n which carries the information about the geometrical alignment of the patches as well as the albedo values if the patches.

The information about the geometrical alignment of the patches is retrieved, e.g. from a point cloud of a depth camera (e.g. time-of-flight as ToF, Lidar, etc) which observes the scene as discussed previously, while the albedo values of the patches are retrieved from the long term scene observation according to the method of image decomposition as already mentioned above.

In one or more embodiments, in order to facilitate solution for self-emittance, namely Eps(xt), the formula can be written

Eps(

Figure imgf000027_0002

or, in the i?-notation

Figure imgf000027_0003
which can be condensed by using the identity matrix I as :
Figure imgf000027_0004

or, in the ?-notation

Figure imgf000027_0005
Changing ths notation fnoni (x,p) to (i>j) leads to a conventional matrix notation as:
Figure imgf000028_0001
or, in the i?-notation
Figure imgf000028_0002

This facilitates referring to an index-free pure matrix representation as a standard matrix-vector- multiplication :

Eps = (/— p F)M

or, in the ?-notation

Eps = (/— p F)R where :

- (I-F) represents a square matrix of size n x n in the form of an optical transfer matrix,

- Eps and M, R each represent a 1-dimension vector of size n.

In one or more embodiments, Eps can be computed based on the radiosity model described previously.

For a Lambertian scene, Eps is found to correspond directly to the albedo value p with its direct illumination as:

Eps = p * Ed

correspondingly

R = p * E

Since M or R correspond directly to the visible emittance, the visible luminance L for a lambertian scene is:

M R

L L

p or p therefore the formula above can be written as

p * Ed = (/— ) * (L * 7i) that is:

Ed = 7G * (/— ) * ( Pinv)

This facilitates extracting the illumination across the room only due to the illumination from the light source w/o considering the mutual inter reflections occurring in the ambient A.

In one or more embodiments, in order to facilitate solution for the radiosity, namely Mor ?, deriving from equation (2) the formula is to be written as:

M = (/— pF)~1Eps

or, in the ?-notation ? = (/— pF ) 1Eps

In one or more embodiments, in order to facilitate obtaining a solution for the reflectivity or albedo, namely p, the formula above can be written as:

M— Eps

P = FM

or in the R-notation

R— Eps

P = FR

Consequently, depending on what entities can be considered known these formulas can be solved in order to find the unknown entities.

For instance, in one or more embodiments one may explicitly calculate a shading map, which is the remainder of the total illumination E(X t) and the direct illumination Ed(x,t) from luminaires, e.g., as:

Figure imgf000030_0001
where

(Lpinv ) is a vector-dot product (not a scalar product )

- F* (Lpinv) is a standard matrix-vector product

- S(x,t) is a one-dimensional vector of size n, so that S explicity represents the measured shading map of the scene.

LIST OF DOCUMENTS CITED

[1] DIALux software - https://www.dial.de/en/dialux/.

Accessed: 2017-11-16

[2] Relux software - https://relux.com/en/. Accessed: 2017-11-16

[3] PCT Patent Application PCT/IB2017/056246

[4] B. Beckers, et al.: "Fast and accurate view factor generation", FICUP, An International Conference on Urban Physics, 09 2016

[5] L. Masset, et al . : "Partition of the circle in cells of equal area and shape", Technical report, Structural Dynamics Research Group, Aerospace and Mechanical Engineering Department, University of Liege, 'Institut de Mecanique et Genie Civil (B52/3), 2011.

[6] M. Chandraker, et al . : "On the duality of forward and inverse light transport". IEEE Trans. Pattern Anal. Mach. Intell . , 33 ( 10 ): 2122-2128 , Oct.2011

[7] Jian Shi, et al . : "Efficient intrinsic image decomposition for RGBD images", in Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology (VRST '15), Stephen N. Spencer (Ed.) . ACM, New York, NY, USA, 17-25. DOI : https : //doi .org/10.1145/2821592.2821601

[8] Samuel Boivin: "Fitting complex materials from a Single Image", ACM SIGGRAPH 2002 Course Notes #39,

"Acquiring Material Models Using Inverse Rendering" Steve Marschner and Ravi Ramamoorthi

[9] S. Boivin, et al.: "Advanced Computer Graphics and Vision Collaboration Techniques for Image-Based Rendering", Imaging and Vision Systems: Assessment and Applications, Jacques Blanc-Talon and Dan Popescu, Section 4.

A method according to one or more embodiments may thus comprise:

- capturing (e.g., at 12) at least one image of an illuminated (e.g., L) environment (e.g., A),

- extracting (e.g., at 10) from said at least one image captured light emittance values, M(xt) or R(x,t), as well as albedo values p(x) of surface areas across the environment,

computing illumination values E(xt) of surface areas across the environment as a function of said emittance values and said albedo values p(x).

One or more embodiments may comprise:

- computing from said at least one image of an illuminated environment measured luminance values L(Xt) of surface areas across the environment,

- converting the measured luminance values L(xt) to corresponding emittance values of surface areas across the environment.

One or more embodiments may comprise:

- arranging said emittance values in a vector,

- computing a vector of the inverted values of the acquired albedo values,

computing said illumination values E(xt) as a vector dot-product of said emittance value vector and said vector of the inverted values of the acquired albedo values.

One or more embodiments, wherein capturing said at least one image of an illuminated environment comprises acquiring image depth information of the illuminated environment, may comprise:

extracting from said at least one image comprising depth information of the illuminated environment :

- a) reflectance information (e.g., 120a) of said environment,

b) form factors (e.g., 120b) of said environment, and

c) lighting intensity and position information (e.g., 120c) of light sources illuminating said environment,

generating, as a function of said reflectance information, said form factors and said lighting intensity and position information, a radiosity model of said environment,

- solving (e.g., at 104) said radiosity model to produce radiosity values of surface areas across said environment, and

- mapping (e.g., 106) said radiosity values to illumination values of surface areas across said environment.

These last-mentioned embodiments may take advantage of techniques as discussed in documents such as :

B. Boom, et al . : "Point Light Source Estimation based on Scenes Recorded by a RGB-D camera", the British Machine Vision Conference (BMVC) , 2013, or

S. Karaoglu, at al . : "Point Light Source Position Estimation From RGB-D Images by Learning Surface Attributes", in IEEE Transactions on Image Processing, vol. 26, no. 11, pp. 5149-5159, Nov. 2017.

One or more embodiments may comprise solving said radiosity model by solving the radiosity matrix based on a constrained radiosity estimation.

In one or more embodiments, converting the measured luminance values L(Xt) to corresponding emittance values may comprise replacing said measured luminance values L(Xt) with said radiosity values.

One or more embodiments may comprise:

extracting from said at least one image comprising depth information of the illuminated environment visibility information indicative of the presence or absence of line-of-sight visibility between pairs of surface areas across said environment; and

including said visibility information in said radiosity model of said environment (A) .

A lighting system according to one or more embodiments may comprise:

- a set of lighting devices for lighting an ambient,

an image capture arrangement configured to capture at least one image of an illuminated environment,

- a signal processing unit coupled to the image capture arrangement and configured to:

- a) compute illumination values E(xt) of surface areas across the environment with the method of one or more embodiments, and

- b) coordinate operation of said set of lighting devices as a function of the illuminated values E(Xt) computed .

In one or more embodiments the image capture arrangement may comprise:

- a first image sensor configured to capture an image, preferably a colour and/or fish-eye image of the illuminated environment, and

an image depth sensor sensitive to depth information of the illuminated environment.

One or more embodiments may comprise a computer program product, loadable in at least one processing module (e.g., 10) and including software code portions for performing the steps of the method of one or more embodiments .

One or more embodiments may involve the use of the method of one or more embodiments in computing illumination values E(x>t) of surface areas across an environment for the application of light commissioning thereto .

One or more embodiments may involve the use of the method of one or more embodiments in computing illumination values E(x>t) of surface areas across an environment for the application of light management and adjustment based on light standards or predefined scenarios.

One or more embodiments may involve the use of the method of one or more embodiments in computing illumination values E(x>t) of surface areas across an environment for the application of smart lighting and quality living and working environments.

Without prejudice to the underlying principles, the details and the embodiments may vary, even significantly, with respect to what has been described just by way of example, without departing from the extent of protection.

The extent of protection is determined by the annexed claims.

In that respect, it will be appreciated that throughout this description both the M-notation, e.g., M(xJ), and the i?-notation, e.g. R(xJ) , have been used for the sake of clarity, being understood that, unless otherwise indicated, these notations are intended to refer to the same physical entities.

LIST OF REFERENCE SIGNS

Lighting devices/sources L

Environment A Processing unit 10

Set of sensors 12

RGB camera sensor 12a

3D-depth-camera sensor 12b

Reflectance information 120a Form factors 120b

Lighting intensity and position information 120c Block/module 104

Mapping 106

Claims

1. A method, comprising:
capturing (12) at least one image of an illuminated (L) environment (A) ,
- extracting (10) from said at least one image captured light emittance values, M(x t) or R(x,t), as well as albedo values p(x) of surface areas across the environment (A) ,
computing (10) illumination values E(x t) of surface areas across the environment (A) as a function of said emittance values and said albedo values p(x).
2 . The method of claim 1, comprising:
- computing (10) from said at least one image of an illuminated (L) environment (A) measured luminance values L(X t) of surface areas across the environment (A) ,
- converting the measured luminance values L(x t) to corresponding emittance values of surface areas across the environment (A) .
3 . The method of claim 2, comprising:
- arranging said emittance values in a vector,
- computing a vector of the inverted values of the acquired albedo values,
computing said illumination values E(x t) as a vector dot-product of said emittance value vector and said vector of the inverted values of the acquired albedo values.
4 . The method of claim 1, wherein capturing (12a) said at least one image of an illuminated (L) environment (A) comprises acquiring image depth information (12b) of the illuminated (L) environment (A), and wherein the method comprises:
extracting from said at least one image comprising depth information (12b) of the illuminated (L) environment (A) :
- a) reflectance information (120a) of said environment (A) ,
- b) form factors (120b) of said environment
(A) , and
c) lighting intensity and position information (120c) of light sources (L) illuminating said environment (A) ,
generating, as a function of said reflectance information (120a), said form factors (120b) and said lighting intensity and position information (120c), a radiosity model of said environment (A) ,
- solving (104) said radiosity model to produce radiosity values of surface areas across said environment (A) , and
- mapping (106) said radiosity values to illumination values of surface areas across said environment (A) .
5 . The method of claim 4, including solving (104) said radiosity model by solving the radiosity matrix based on a constrained radiosity estimation.
6. The method of claim 4 or claim 5, taken in combination with claim 2, wherein converting the measured luminance values L(X t) to corresponding emittance values comprises replacing said measured luminance values L(X t) with said radiosity values.
7 . The method of any of claims 4 to 6, comprising: extracting from said at least one image comprising depth information (12b) of the illuminated (L) environment (A) visibility information indicative of the presence or absence of line-of-sight visibility between pairs of surface areas across said environment (A) ; and
including said visibility information in said radiosity model of said environment (A) .
8. A lighting system comprising: - a set of lighting devices (L) for lighting an ambient (A) ,
- an image capture arrangement (12) configured to capture at least one image of an illuminated (L) environment (A) ,
- a signal processing unit (10) coupled to the image capture arrangement (12) and configured to:
a) compute (10) illumination values E(xt) of surface areas across the environment (A) with the method of any of claims 1 to 7, and
- b) coordinate operation of said set of lighting devices (L) as a function of the illuminated values E(Xt) computed .
9. The lighting system of claim 8, wherein the image capture arrangement (12) comprises:
- a first image sensor (12a) configured to capture an image, preferably a colour and/or fish-eye image of the illuminated (L) environment (A) , and
- an image depth sensor (12b) sensitive to depth information of the illuminated (L) environment (A) .
10. A computer program product, loadable in at least one processing module (10) and including software code portions for performing the steps of the method of any of claims 1 to 7.
11. Use of the method of any of claim 1 to 7 in computing (10) illumination values E(x,t) of surface areas across an environment (A) for the application of light commissioning thereto.
12. Use of the method of any of claim 1 to 7 in computing (10) illumination values E(xt) of surface areas across an environment (A) for the application of light management and adjustment based on light standards or predefined scenarios.
13. Use of the method of any of claim 1 to 7 in computing (10) illumination values E(xt) of surface areas across an environment (A) for the application of smart lighting and quality living and working environments.
PCT/IB2019/053669 2018-05-15 2019-05-06 A method of measuring illumination, corresponding system, computer program product and use WO2019220256A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
IT201800005381 2018-05-15
IT102018000005381 2018-05-15

Publications (1)

Publication Number Publication Date
WO2019220256A1 true WO2019220256A1 (en) 2019-11-21

Family

ID=62875257

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2019/053669 WO2019220256A1 (en) 2018-05-15 2019-05-06 A method of measuring illumination, corresponding system, computer program product and use

Country Status (1)

Country Link
WO (1) WO2019220256A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016112430A1 (en) * 2015-01-14 2016-07-21 The University Of Sydney Hyperspectral imager method and apparatus
WO2018069827A1 (en) * 2016-10-13 2018-04-19 Osram Gmbh A method of identifying light sources, corresponding system and computer program product

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016112430A1 (en) * 2015-01-14 2016-07-21 The University Of Sydney Hyperspectral imager method and apparatus
WO2018069827A1 (en) * 2016-10-13 2018-04-19 Osram Gmbh A method of identifying light sources, corresponding system and computer program product

Non-Patent Citations (9)

* Cited by examiner, † Cited by third party
Title
B. BECKERS ET AL.: "Fast and accurate view factor generation", FICUP, AN INTERNATIONAL CONFERENCE ON URBAN PHYSICS, vol. 09, 2016
B. BOOM ET AL.: "Point Light Source Estimation based on Scenes Recorded by a RGB-D camera", THE BRITISH MACHINE VISION CONFERENCE (BMVC, 2013
DANIEL MAURER ET AL: "Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth, Illumination and Albedo", INTERNATIONAL JOURNAL OF COMPUTER VISION., vol. 126, no. 12, 27 March 2018 (2018-03-27), US, pages 1342 - 1366, XP055552217, ISSN: 0920-5691, DOI: 10.1007/s11263-018-1079-1 *
JIAN SHI ET AL.: "Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology (VRST '15", ACM, article "Efficient intrinsic image decomposition for RGBD images", pages: 17 - 25
L. MASSET ET AL.: "Technical report, Structural Dynamics Research Group, Aerospace and Mechanical Engineering Department", 2011, UNIVERSITY OF LIEGE, 'INSTITUT DE MECANIQUE ET GENIE CIVIL (B52/3, article "Partition of the circle in cells of equal area and shape"
M. CHANDRAKER ET AL.: "On the duality of forward and inverse light transport", IEEE TRANS. PATTERN ANAL. MACH. INTELL., vol. 33, no. 10, October 2011 (2011-10-01), pages 2122 - 2128, XP011409104, DOI: doi:10.1109/TPAMI.2011.124
S. BOIVIN ET AL.: "Advanced Computer Graphics and Vision Collaboration Techniques for Image-Based Rendering", IMAGING AND VISION SYSTEMS: ASSESSMENT AND APPLICATIONS
S. KARAOGLU: "Point Light Source Position Estimation From RGB-D Images by Learning Surface Attributes", IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 26, no. 11, November 2017 (2017-11-01), pages 5149 - 5159
SAMUEL BOIVIN: "ACM SIGGRAPH", 2002, article "Fitting complex materials from a Single Image"

Similar Documents

Publication Publication Date Title
Miller et al. Illumination and reflection maps
US8648872B2 (en) Systems and methods for rapid three-dimensional modeling with real facade texture
Beraldin et al. Virtualizing a Byzantine crypt by combining high-resolution textures with laser scanner 3D data
Yu et al. Inverse global illumination: Recovering reflectance models of real scenes from photographs
Dorsey et al. Design and simulation of opera lighting and projection effects
JP4220470B2 (en) A reality-based lighting environment for digital imaging in cinema
Shirley et al. Monte Carlo techniques for direct lighting calculations
Yu et al. Recovering photometric properties of architectural scenes from photographs
JP4344606B2 (en) Drawing using the Russian roulette method to evaluate global illumination
EP3242275A1 (en) Using photo collections for three dimensional modeling
Debevec Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography
Jarosz et al. The beam radiance estimate for volumetric photon mapping
US9591236B2 (en) System and method for image processing and generating a body model
US20050013477A1 (en) Three dimensional tangible interface for interacting with spatial-temporal data using infrared light sources and infrared detectors
Hall et al. A testbed for realistic image synthesis
Sato et al. Illumination from shadows
Ikeuchi Modeling from reality
Nishita et al. Continuous tone representation of three-dimensional objects illuminated by sky light
Catita et al. Extending solar potential analysis in buildings to vertical facades
Wang et al. Estimation of multiple directional light sources for synthesis of augmented reality images
CN106133796B (en) For indicating the method and system of virtual objects in the view of true environment
Mardaljevic Daylight simulation: validation, sky models and daylight coefficients
JP3962588B2 (en) 3D image processing method, 3D image processing apparatus, 3D image processing system, and 3D image processing program
Patow et al. A survey of inverse rendering problems
US9420253B2 (en) Presenting realistic designs of spaces and objects

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19730215

Country of ref document: EP

Kind code of ref document: A1