Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/64—Three-dimensional [3D] objects
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/24—Classification techniques
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/0464—Convolutional networks [CNN, ConvNet]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T12/00—Tomographic reconstruction from projections
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—Three-dimensional [3D] image rendering
  • G06T15/10—Geometric effects
  • G06T15/20—Perspective computation
  • G06T15/205—Image-based rendering
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/20—Image preprocessing
  • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2210/00—Indexing scheme for image generation or computer graphics
  • G06T2210/12—Bounding box
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
  • G06V2201/07—Target detection

Definitions

• the invention relates to the detection, location and automatic identification of objects in a 3D volume.
• the aim of the present invention is to improve the situation. [0006] To this end, the present invention provides a method for detecting, locating and identifying objects contained in a complex scene.
• the method comprises the following steps:
• the invention makes it possible to significantly improve the quality of the detection, localization, and identification of objects in a complex scene thanks, from the 3D volume of the scene to be processed, to obtaining k 2D sections in each from which we proceed to a detection, localization and identification of the objects to be processed by artificial intelligence and semantic segmentation and the concatenation of the results in 3D.
• the invention comprises one or more of the following characteristics which can be used separately or in partial combination with one another or in total combination with one another: - the complex scene is previously transformed by 3D imaging in volume the method further comprises a step of indexing the labels for all the objects of interest in the complex scene;
• the resolution of the process depends on the size of the 2D bounding boxes
• the resolution of the process depends on the size of the 3D bounding boxes.
• the method comprises, in order to concatenate in 3D the results of all the k 2D output sections, for an object of interest among said objects of interest, the following steps: define, for each output 2D section, a reference mark local three-dimensional, one of the dimensions of which is perpendicular to the plane defined by the 2D section, and associate said reference with said 2D section; identify, in the output 2D sections, subsets or slices of the object of interest; transform each identified subset or slice of the object of interest by changing the coordinate system, from the local three-dimensional frame of the 2D section to which it belongs to a predetermined absolute Cartesian frame of reference; concatenate the transformed subsets or slices into a 3D icon.
• the invention also relates to a system for implementing the method defined above.
• the invention further relates to a computer program comprising program instructions for the execution of a method as defined above, when said program is executed on a computer.
• Figure 1 schematically illustrates the main steps of the process according to the invention
• Figure 1A is an index table of classes of objects of interest
• FIG. 2 schematically represents the steps for obtaining 2D sections
• FIG. 3 diagrammatically represents the steps of detection, location, and automatic identification of objects of interest in each 2D section by specialized artificial intelligence
• Figure 4 shows a bounding box of the object detected according to the method according to the invention.
• Figure 5 shows a segmented icon in accordance with the invention
• Figure 6 shows a 3D volume reconstructed in voxels in accordance with the invention
• Figure 7 shows principal sections in a 3D volume reconstructed by receptive tomography in accordance with the invention
• FIG. 8 represents an example of an OCT (Optical Coherence Tomography) section
• FIG. 9 represents an example of a complex 3D scene containing a camouflaged object reconstructed by reflective tomography from the 2D images;
• Figure 10 shows an example of a 2D section in the 3D scene
• Figure 11 shows the automatic detection and generation of the bounding box of the object camouflaged in the 2D section of the 3D scene
• Figure 12 shows the identification of the camouflaged object in the 2D section of the 3D scene.
• Figure 13 shows the generation of the bounding box and the identification of the object in the 3D scene.
• the invention relates to the detection, location and automatic identification of objects in 3D three-dimensional imagery forming a 3D three-dimensional volume in voxels (pixel volumetry).
• 3D imagery corresponds to a complex scene in which objects can hide from each other as illustrated in figure 9.
• the three-dimensional volume can be obtained by means of a reconstruction process by transmission or by fluorescence (Optical Projection Tomography, nuclear imaging or X-Ray Computed Tomography) or by reflection (reflection of a laser wave or by solar reflection in the case of the visible band (between 0.4 pm and 0.7 pm) or near infrared (between 0.7 pm and 1 pm or SWIR (Small Wave InfraRed between 1 pm and 3 pm) or taking into account the thermal emission of the object (thermal imaging between 3 pm and 5 pm and between 8 pm and 12 pm), this process three-dimensional reconstruction is described in the patent “Optronic system and method for producing three-dimensional images dedicated to identification” (US8836762B2, EP2333481 B1).
• the index (n) is at the value "n"
• the index (background) is at the value "0”.
• the detection, location and identification method according to the invention comprises the following general steps described with reference to Figure 1.
• k 2D sections are made in the reconstructed 3D volume.
• 3D volume (Section (k) ⁇ , k [1, 2, .., K ⁇ , K being the number of 2D sections made.
• step 20 for each input 2D section thus obtained, one proceeds to an automatic detection, location, and identification of the objects of interest by a specialized AI artificial intelligence method.
• Cut (k) ⁇ Object (k, m), Label (k, m), Boundingbox2D (k, m), lcone2D (k, m) ⁇ .
• the AI Artificial Intelligence method is based on deep learning, also called Deep Learning, of the "Faster R-CNN (Regions with Convolutional Neural Network features) object classification" type.
• the method applies a semantic segmentation of each lcone2D defined by a bounding box Boundingbox2D.
• the semantic segmentation is performed by Deep Learning, for example an R-CNN Mask (Regions with Convolutional Neural Network) designated for the semantic segmentation of the images.
• R-CNN Mask Regions with Convolutional Neural Network
• step 30 we finally proceed to the 3D concatenation of the results of all the 2D sections.
• the 3D concatenation of the results of the 2D sections is carried out by the following steps: Definition of a local three-dimensional coordinate system associated with each 2D section, including one of dimension is perpendicular to the plane defined by the 2D section, o Identification of 2D sections in which subsets or sections of the object of interest have been identified, o Mathematical transformation (translation and / or rotation) of all the local three-dimensional reference frames of the subsets or slices retained in a determined absolute Cartesian three-dimensional reference frame.
• the first precision also called resolution relates to the number and angle of 2D sections, for example 2D sections belong to the group formed by main sections, horizontal sections, vertical sections, oblique sections.
• 2D sections at different angles can provide better detection results and will be used in the 3D concatenation of results which will be described in more detail below.
• Boundingbox2D [(x1, x2), (y1, y2)].
• Boundingbox3D [(x1, x2), (y1, y2), (z1, z2)].
• the cutting module 12 From the 3D volume 1 1 (reconstructed in voxels), the cutting module 12 generates 2D sections 15 (in pixels) in response to the command from the choice module 13.
• the 2D sections 15 (in pixels) are managed and indexed by the management module 14 in accordance with the indexing table TAB (FIG. 1A).
• the output 2D section 23 generated by the IA method 22 comprises a 2D bounding box 24 surrounding an object of interest 25.
• the size of the 2D bounding box 24 surrounding the object 25 is defined by its coordinates on the abscissa X (x1 and x2) and on the ordinate Y (y1 and y2).
• FIG. 5 there is shown a 2D icon 50 semantically segmented in accordance with the invention.
• object 25 is indexed with the value "1" while the background is indexed with the value "0".
• FIG. 6 there is shown a 3D volume reconstructed in voxels according to the invention in which the object of interest 25 has the index "1" while another object of interest has the 'index "2"; in an index background volume "0".
• FIG. 8 an example of an OCT (Optical Coherence Tomography) section has been shown; in which a gap area is to be identified.
• OCT Optical Coherence Tomography
• the complex scene includes a vehicle camouflaged in the bushes.
• the 2D shooting is of the air ground type with 2D images of 415x693 pixels.
• Figure 10 there is shown an example of a 2D section (YZ section) in the 3D scene of Figure 9.
• FIG 1 there is shown the automatic detection and generation of the bounding box of the object camouflaged in the 2D section of the 3D scene illustrated in Figures 9 and 10.
• the fields of application of the invention are wide, covering the detection, classification, recognition and identification of objects of interest.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Evolutionary Computation (AREA)
• Computing Systems (AREA)
• Artificial Intelligence (AREA)
• Health & Medical Sciences (AREA)
• Software Systems (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Health & Medical Sciences (AREA)
• Databases & Information Systems (AREA)
• Medical Informatics (AREA)
• Data Mining & Analysis (AREA)
• Computer Graphics (AREA)
• Geometry (AREA)
• General Engineering & Computer Science (AREA)
• Life Sciences & Earth Sciences (AREA)
• Biomedical Technology (AREA)
• Evolutionary Biology (AREA)
• Bioinformatics & Computational Biology (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Biophysics (AREA)
• Computational Linguistics (AREA)
• Molecular Biology (AREA)
• Mathematical Physics (AREA)
• Image Analysis (AREA)
• Apparatus For Radiation Diagnosis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=69190842

Family Applications (1)

Country Status (6)

Families Citing this family (1)

Family Cites Families (14)

• 2019
  • 2019-07-18 FR FR1908109A patent/FR3098963B1/fr active Active
• 2020
  • 2020-07-07 JP JP2022500701A patent/JP2022540582A/ja active Pending
  • 2020-07-07 KR KR1020227001559A patent/KR20220032562A/ko not_active Ceased
  • 2020-07-07 US US17/621,433 patent/US20220358714A1/en not_active Abandoned
  • 2020-07-07 WO PCT/EP2020/069056 patent/WO2021008928A1/fr not_active Ceased
  • 2020-07-07 EP EP20747343.0A patent/EP4000004A1/de not_active Withdrawn

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events