WO2023243430A1 - 情報処理方法、情報処理装置、およびプログラム - Google Patents

情報処理方法、情報処理装置、およびプログラム Download PDF

Info

Publication number
WO2023243430A1
WO2023243430A1 PCT/JP2023/020592 JP2023020592W WO2023243430A1 WO 2023243430 A1 WO2023243430 A1 WO 2023243430A1 JP 2023020592 W JP2023020592 W JP 2023020592W WO 2023243430 A1 WO2023243430 A1 WO 2023243430A1
Authority
WO
WIPO (PCT)
Prior art keywords
illumination
target object
imaging
information processing
light
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/020592
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
皓一 吉田
崇正 佐藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to JP2024528700A priority Critical patent/JPWO2023243430A1/ja
Publication of WO2023243430A1 publication Critical patent/WO2023243430A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/145—Illumination specially adapted for pattern recognition, e.g. using gratings

Definitions

  • the present disclosure relates to an information processing method, an information processing device, and a program, and particularly to an information processing method, an information processing device, and a program that can restore both the shape and color of a target object with high precision in photogrammetry. and regarding programs.
  • photogrammetry has been used to analyze a group of images of a target object from various directions and create a three-dimensional computer graphic model (hereinafter referred to as a 3D model) of the target object.
  • a 3D model three-dimensional computer graphic model
  • photogrammetry can create a 3D model in which the shape and color of a target object are restored.
  • Patent Document 1 proposes a three-dimensional shape measuring device that measures the three-dimensional shape of a predetermined part of a measurement target based on a plurality of images taken in different photographing directions of a photographing means or in different directions of illumination light. .
  • the present disclosure has been made in view of this situation, and is intended to enable highly accurate restoration of both the shape and color of a target object in photogrammetry.
  • An information processing method includes performing first imaging using the target object as a subject, which is irradiated with illumination light using a first illumination method suitable for restoring the shape of the target object; performing second imaging using the target object as a subject irradiated with illumination light using a second illumination method suitable for restoring the color of the image; and a first image obtained by a plurality of times of the first imaging. and performing 3D restoration processing for restoring the shape and color of the target object as a 3D model using the second image group obtained by the second image capturing a plurality of times.
  • An information processing device includes a first illumination unit that irradiates illumination light using a first illumination method suitable for restoring the shape of a target object, and a second illumination unit that irradiates illumination light using a first illumination method suitable for restoring the color of the target object.
  • a second illumination section that emits illumination light using an illumination method
  • a second illumination unit that performs first imaging using the target object irradiated with illumination light using the first illumination method as a subject
  • an imaging unit that performs second imaging using the target object irradiated with light as a subject
  • a first image group obtained by a plurality of times of the first imaging
  • a first image group obtained by the plurality of times of the second imaging
  • a 3D restoration processing unit that performs 3D restoration processing to restore the shape and color of the target object as a 3D model using the obtained second image group.
  • a program causes a computer of an information processing device to perform first imaging using the target object as a subject, which has been irradiated with illumination light using a first illumination method suitable for restoring the shape of the target object. and performing second imaging with the target object irradiated with illumination light using a second illumination method suitable for restoring the color of the target object as a subject; performing 3D restoration processing for restoring the shape and color of the target object using a 3D model using the first image group obtained by the first image capturing and the second image group obtained by the second image capturing a plurality of times; Execute information processing including
  • first imaging is performed using as a subject a target object irradiated with illumination light using a first illumination method suitable for restoring the shape of the target object;
  • Second imaging is performed using the target object irradiated with illumination light using the second illumination method, and the first image group obtained by the plurality of first imaging and the plurality of second imaging are captured.
  • a 3D restoration process is performed to restore the shape and color of the target object using a 3D model using the second image group obtained by imaging.
  • FIG. 1 is a block diagram showing a configuration example of an embodiment of a photogrammetry device to which the present technology is applied.
  • FIG. It is a figure explaining the line pattern of the 1st illumination method. It is a figure showing an example of a dot pattern and a cross pattern.
  • It is a flowchart explaining the 1st processing example of photogrammetry processing. It is a flowchart explaining 3D restoration processing. It is a flowchart explaining the 2nd processing example of photogrammetry processing.
  • FIG. 3 is a block diagram showing a first modification of the photogrammetry device.
  • FIG. 3 is a block diagram showing a second modification of the photogrammetry device. It is a block diagram which shows the 3rd modification of a photogrammetry apparatus.
  • It is a block diagram showing a fourth modification of the photogrammetry device.
  • 1 is a block diagram showing a configuration example of an embodiment of a computer to which the present technology is applied.
  • FIG. 1 is a block diagram showing a configuration example of an embodiment of a photogrammetry apparatus to which the present technology is applied.
  • the photogrammetry device 11 includes a first illumination section 21, a second illumination section 22, an imaging section 23, a first illumination drive section 24, a second illumination drive section 25, an imaging drive 26, a 3D restoration processing section 27, and a control section 28.
  • the photogrammetry device 11 then performs photogrammetry processing on the target object for which a 3D model is to be created, and can restore the shape and color of the target object using computer graphics.
  • the first illumination unit 21 irradiates the target object with illumination light using a first illumination method suitable for restoring the shape of the target object.
  • illumination light in a pattern as described later with reference to FIGS. 2 and 3 is irradiated.
  • by irradiating illumination light from a plurality of directions using the first illumination method it is possible to grasp the shape of the target object that cannot be covered by irradiation of illumination light from one direction.
  • the first illumination method it is possible to irradiate the target object with illumination light in a plurality of different patterns or directions, and it is possible to irradiate the target object with illumination light in a plurality of different patterns or directions. After generating Sparse point clouds, you can merge them to generate Dense point clouds.
  • the second illumination unit 22 irradiates the target object with illumination light using a second illumination method suitable for restoring the color of the target object.
  • illumination light is irradiated onto the target object from the same direction as the imaging direction, or illumination light is irradiated onto the target object from a direction in which specularly reflected light from the target object does not enter the imaging unit 23.
  • Ru For example, when the target object is irradiated with illumination light from the same direction as the imaging direction, it is possible to avoid creating a shadow on the surface of the target object within the range that the light source can reach. However, in this case, since the color is affected by the specularly reflected light, the specularly reflected light is The influence on color can be eliminated.
  • illumination light is irradiated by light sources with different luminous intensities and wavelengths.
  • the reflection characteristics on the surface of the target object can be estimated.
  • the second illumination method it is possible to irradiate the target object with illumination light in multiple different directions, luminous intensities, or wavelengths, and by applying illumination light suitable for each area to be colored, a certain illumination Areas that cannot be covered by light can be supplemented with other illumination light.
  • the imaging unit 23 performs multiple images of the target object irradiated with illumination light using the first illumination method and multiple times of the target object irradiated with the illumination light using the second illumination method. A group of images acquired by the imaging is supplied to the 3D restoration processing section 27.
  • the first illumination drive unit 24 drives the first illumination unit 21 to emit illumination light under the control of the control unit 28.
  • the second illumination drive unit 25 drives the second illumination unit 22 to emit illumination light under the control of the control unit 28 via the imaging drive unit 26.
  • the imaging drive unit 26 drives the imaging unit 23 to image the target object. Further, the second illumination drive section 25 is controlled by the control section 28 via the imaging drive section 26 .
  • the 3D restoration processing unit 27 receives a group of images supplied from the imaging unit 23, that is, a group of images of the target object irradiated with illumination light using the first illumination method, and a group of images of the target object irradiated with illumination light using the second illumination method. Using a group of images of the target object, 3D restoration processing is performed to create a colored 3D model of the target object. That is, the 3D restoration processing unit 27 creates a 3D model in which the shape of the target object is restored using a group of images of the target object irradiated with illumination light using the first illumination method, and performs processing on the 3D model. , coloring can be performed using a group of images of the target object irradiated with illumination light using the second illumination method.
  • the 3D restoration processing unit 27 can provide feedback to complement the shape in the missing region.
  • the 3D restoration processing unit 27 performs meshing after restoring the shape based on the point cloud, and the 3D restoration processing unit 27 performs meshing to restore the shape of the point group, and detects a point where the length of any one side of a face adjacent to a certain vertex is equal to or greater than a certain value, or a point that touches a certain vertex. It is possible to extract areas where it is impossible to create a closed surface and estimate that there is a missing area beyond that area.
  • the 3D restoration processing unit 27 feeds back the imaging position and imaging direction that cover the surface formed by connecting the missing regions. At this time, if the estimated missing area overlaps with the imaged area, the 3D restoration processing unit 27 feeds back the first illumination method that is more suitable for the nature of the missing area.
  • the 3D restoration processing unit 27 performs a process to complement the color in the shadow area. Can give feedback.
  • the 3D restoration processing unit 27 can detect shadow areas using existing technology, and performs ray tracing using camera parameters and the position of light sources such as sunlight and lighting on the 3D model resulting from the restoration. The shadow area can be estimated by Then, the 3D restoration processing unit 27 determines whether the color of the detected shadow area can be restored using another image, and if the color cannot be restored, the 3D restoration processing unit 27 determines the imaging position for complementing the color of the shadow area. and estimates and feeds back the imaging direction.
  • the control unit 28 controls the first illumination unit 21 , the second illumination unit 22 , the imaging unit 23 , and the 3D restoration processing unit 27 to execute photogrammetry processing in the photogrammetry device 11 . Further, when the photogrammetry device 11 is mounted on a mobile device that can move autonomously (for example, a so-called drone or a robot), the control unit 28 can control the movement mechanism of the mobile device. can.
  • control unit 28 determines a list in which a plurality of imaging positions are selected for imaging the target object from various directions, and controls the movement mechanism of the movement device in which the photogrammetry device 11 is mounted. , moves the photogrammetry device 11 to the imaging position according to the list. Then, the control unit 28 sequentially controls the first illumination unit 21 to take an image of the target object irradiated with the illumination light in the first illumination method, and the second illumination unit 22 to take a second image at each imaging position. Control is performed such that imaging is performed while switching between imaging using the target object irradiated with illumination light using the illumination method as the subject. According to these controls, when a group of images acquired by the imaging unit 23 performing imaging at all imaging positions is supplied to the 3D restoration processing unit 27, the control unit 28 controls the 3D restoration processing to perform 3D restoration processing. Controls the restoration processing unit 27.
  • the control unit 28 controls the movement mechanism of the mobile device in which the photogrammetry device 11 is mounted so that imaging is performed at the imaging position and imaging direction according to the feedback from the 3D restoration processing unit 27. Similar imaging can be performed repeatedly. Thereby, both the shape and color can be restored satisfactorily so that no defects occur in the 3D model of the target object.
  • the photogrammetry device 11 configured as described above has the first illumination section 21, the second illumination section 22, and the imaging section 23 integrated, and the first illumination section 21 and the second illumination section 23 are integrated.
  • the section 22 and the imaging section 23 can be driven independently.
  • the first illumination section 21 emits illumination light from a direction different from the imaging direction of the imaging section 23, and the second illumination section 22 emits illumination light from the same direction as the imaging direction of the imaging section 23.
  • Illumination light can be emitted from any direction.
  • a 3D model in which the shape of the target object has been restored is created using a group of images of the target object illuminated with illumination light using the first illumination method, and the 3D model is then created using the second illumination method.
  • the 3D model shape and color of the target object can be restored with high precision.
  • the photogrammetry device 11 separates the image captured under the first illumination method and the image captured under the second illumination method, respectively. Since they correspond at the pixel level, colors can be directly superimposed on a 3D model in which the shape of the target object has been restored. Therefore, the photogrammetry device 11 can restore the shape and color all at once using one set of images, and can easily create a colored 3D model without expanding the photogrammetry process. can.
  • the photogrammetry device 11 when the target object is moving, the photogrammetry device 11 generates an image captured under the second illumination method so as to correspond to each pixel of the image captured under the first illumination method.
  • a first illumination method suitable for restoring the shape of a target object will be described with reference to FIG. 2.
  • FIG. 2A shows an example of a line pattern of irradiation light that is irradiated onto a target object using the first illumination method.
  • FIG. 2B shows an example of a state in which a rectangular parallelepiped target object is irradiated with a line pattern of irradiation light.
  • line-shaped lights are arranged at equal intervals.
  • the line pattern is deformed in accordance with the shape of the target object, so that the shape of the target object can be easily grasped.
  • irradiation light other than the line pattern shown in A of FIG. 2 can be used.
  • a dot-pattern irradiation light in which multiple dot-shaped lights are arranged at equal intervals as shown in A in FIG. Patterned irradiation light or the like may also be used.
  • a pattern of random dots can be used as the first illumination method suitable for restoring the shape of the target object.
  • each light point For example, the distance of each light point (depth ), only the shape of the target object can be restored.
  • depth the distance of each light point (depth )
  • each light spot of the random dot pattern light has a different pattern or wavelength, it is possible to identify each light spot, and it is possible to identify each light spot based on the light spot in an image taken from multiple directions. It becomes possible to perform position alignment.
  • the feature amounts extracted by the first illumination method will be explained.
  • a feature value tends to appear in a captured image at a location where the color change is large compared to neighboring pixels.
  • the outer periphery of the shadow that appears on the surface of the target object due to the light source tends to appear as a new feature, and the shadow may appear in imaging using other lighting methods or without lighting. It is possible to illuminate the hidden parts and make the features appear.
  • patterned light is irradiated, features tend to appear evenly on the surface of the target object within the range where the patterned light is irradiated, and gentle unevenness also tends to be reflected in the distribution of the features.
  • FIG. 4 is a flowchart illustrating a first processing example of photogrammetry processing.
  • step S11 the control unit 28 determines a list in which a plurality of imaging positions are selected for imaging the target object from various directions.
  • step S12 the control unit 28 controls a moving mechanism (not shown) to move the photogrammetry device 11 to the imaging position according to the list determined in step S11.
  • a moving mechanism not shown
  • step S13 the control unit 28 controls the first lighting drive unit 24 to turn on the first lighting unit 21.
  • the first illumination drive unit 24 drives the first illumination unit 21, and irradiation of the target object with illumination light using the first illumination method is started.
  • step S14 the control unit 28 controls the imaging drive unit 26 so that the imaging unit 23 performs imaging.
  • the imaging drive unit 26 drives the imaging unit 23, and the target object irradiated with illumination light is imaged using the first illumination method suitable for restoring the shape of the target object.
  • step S15 the control unit 28 controls the first illumination drive unit 24 to turn off the first illumination unit 21, and irradiation of the target object with illumination light by the first illumination method is stopped.
  • step S16 the control unit 28 controls the second lighting drive unit 25 to turn on the second lighting unit 22.
  • the second illumination driving section 25 drives the second illumination section 22, and irradiation of the target object with illumination light by the second illumination method is started.
  • step S17 the control unit 28 controls the imaging drive unit 26 so that the imaging unit 23 performs imaging.
  • the imaging drive unit 26 drives the imaging unit 23, and the target object irradiated with illumination light is imaged using the second illumination method suitable for restoring the color of the target object.
  • step S18 the control unit 28 controls the second illumination drive unit 25 to turn off the second illumination unit 22, and irradiation of the target object with illumination light by the second illumination method is stopped.
  • step S19 the control unit 28 determines whether imaging has been performed at all imaging positions in the list determined in step S11.
  • step S19 if the control unit 28 determines that imaging is not being performed at all imaging positions, the process proceeds to step S20.
  • step S20 the control unit 28 controls a moving mechanism (not shown) to move the photogrammetry device 11 to an imaging position where imaging has not yet been performed, according to the list determined in step S11.
  • a moving mechanism not shown
  • step S19 determines in step S19 that imaging has been performed at all imaging positions.
  • step S21 the image group obtained by the imaging in step S14 at all imaging positions and the image group obtained in the imaging in step S17 at all imaging positions are supplied to the 3D restoration processing unit 27 and subjected to 3D restoration processing. (Fig. 5) is performed. Then, the 3D restoration processing unit 27 outputs the 3D model of the target object created as a result of the 3D restoration processing.
  • step S31 the 3D restoration processing unit 27 determines, based on the group of images taken with the target object as the subject under the first illumination method in step S14 of FIG. Extract features.
  • step S32 the 3D restoration processing unit 27 performs matching between the respective images regarding the feature amount of the target object extracted in step S31.
  • step S33 the 3D restoration processing unit 27 performs restoration of a sparse point group that creates the three-dimensional shape of the target object using a small number of points.
  • step S34 the 3D restoration processing unit 27 corrects the distortion of the image based on the sparse point group of the target object restored in step S33.
  • step S35 the 3D restoration processing unit 27 applies the image distortion in step S34 and performs dense point cloud restoration that creates the three-dimensional shape of the target object using a large number of point groups. Create a 3D model that represents the shape.
  • step S36 the 3D restoration processing unit 27 uses the point group for the 3D model created in step S35 based on the image group captured with the target object as the subject under the second illumination method in step S17 of FIG. Color.
  • the photogrammetry device 11 can restore both the shape and color of the 3D model of the target object with high precision.
  • FIG. 6 is a flowchart illustrating a second processing example of photogrammetry processing.
  • steps S41 to S51 the same processes as steps S11 to S21 in FIG. Determine.
  • step S52 if the 3D restoration processing unit 27 determines that there is a defect in the 3D model, the process proceeds to step S53.
  • step S53 the 3D restoration processing unit 27 determines the imaging position and imaging direction that cover the surface formed by connecting the missing regions, and the imaging position and imaging direction that complements the color of the shadow region, as described above. Feedback on direction etc. After that, the process returns to step S42, and the same process is repeated in accordance with the feedback from step S53.
  • step S52 determines in step S52 that there is no defect in the 3D model
  • FIG. 7 is a block diagram showing a first modification of the photogrammetry device 11.
  • the photogrammetry device 11A includes a first illumination section 21, a second illumination section 22, an imaging section 23, a first illumination drive section 24, a second illumination drive section 25, an imaging drive It is configured to include a section 26A, a 3D restoration processing section 27, and a control section 28.
  • the photogrammetry apparatus 11 in FIG. 1 is configured such that the control section 28 controls the second illumination drive section 25 via the imaging drive section 26.
  • the control unit 28 directly controls the second illumination drive unit 25, and the imaging drive unit 26A is configured to only drive the imaging unit 23.
  • the photogrammetry apparatus 11A has a configuration in which the first illumination drive section 24, the second illumination drive section 25, and the imaging drive section 26A are independent.
  • the photogrammetry device 11A includes the first illumination section 21, the second illumination section 22, and the imaging section 23, which are integrated. Each of the sections 23 can be driven completely independently. For example, in the photogrammetry device 11, both the first illumination section 21 and the second illumination section 22 can emit illumination light from a direction different from the imaging direction of the imaging section 23, and the second illumination section 22 can emit illumination light from a direction in which specularly reflected light from the target object does not enter the imaging unit 23 .
  • FIG. 8 is a block diagram showing a second modification of the photogrammetry device 11.
  • the photogrammetry device 11B includes a first illumination section 21, a second illumination section 22, an imaging section 23, a first illumination drive section 24, a second illumination drive section 25, an imaging drive It is configured to include a section 26, a control section 28, and a communication section 29.
  • the communication unit 29 can communicate via the network 12 and communicates with the 3D restoration processing unit 27 provided on the cloud of the network 12.
  • the photogrammetry device 11B captures a group of images captured with the target object as the subject under the first illumination method, and a group of images captured with the target object as the subject under the second illumination method. It can be transmitted to the 3D restoration processing section 27 by the communication section 29 . Then, when the 3D restoration processing unit 27 creates a 3D model of the target object, the photogrammetry device 11B can receive the 3D model of the target object through the communication unit 29.
  • the photogrammetry device 11B can realize photogrammetry processing in which a 3D model of the target object is created on the cloud using a system including the 3D restoration processing unit 27 connected to the network 12.
  • FIG. 9 is a block diagram showing a third modification of the photogrammetry device 11.
  • the photogrammetry device 11C includes a second illumination section 22, an imaging section 23, a second illumination drive section 25, an imaging drive section 26, a 3D restoration processing section 27, a control section 28, and a communication section 23. 29.
  • the illumination device 13 connected via the communication section 29 of the photogrammetry device 11C includes a first illumination section 21, a first illumination drive section 24, a communication section 31, and a control section 32. Ru.
  • the communication unit 31 communicates with the communication unit 29 of the photogrammetry device 11C, and the control unit 32 controls the first illumination drive unit 24 under the control unit 28 of the photogrammetry device 11C.
  • the first lighting section 21 is driven.
  • the photogrammetry device 11C can realize photogrammetry processing by arranging the illumination device 13 at a position independent of the photogrammetry device 11C by using a system including the illumination device 13 connected by communication. can.
  • FIG. 10 is a block diagram showing a fourth modification of the photogrammetry device 11.
  • the photogrammetry device 11D includes an imaging section 23, an imaging drive section 26, a 3D restoration processing section 27, a control section 28, and a communication section 29.
  • the lighting device 13 is configured to include a first lighting section 21, a first lighting drive section 24, a communication section 31, and a control section 32, as in FIG.
  • the illumination device 14 connected via the communication section 29 of the photogrammetry device 11D includes a second illumination section 22, a second illumination drive section 25, a communication section 41, and a control section 42. Ru.
  • the communication unit 41 communicates with the communication unit 29 of the photogrammetry device 11D, and the control unit 42 controls the second illumination drive unit 25 under the control unit 28 of the photogrammetry device 11C.
  • the second lighting section 22 is driven.
  • the photogrammetry device 11D is equipped with a system including the lighting device 13 and the second lighting section 22 connected by communication, so that the lighting device 13 and the second lighting section 22 are located at a position independent of the photogrammetry device 11D. can be arranged to realize photogrammetry processing.
  • the photogrammetry device 11 may have a configuration in which the first illumination section 21 and the second illumination section 22 are integrated, as long as one illumination section can irradiate illumination light using different illumination methods. .
  • FIG. 11 is a block diagram showing a configuration example of an embodiment of a computer in which a program that executes the series of processes described above is installed.
  • the program can be recorded in advance on the hard disk 105 or ROM 103 as a recording medium built into the computer.
  • the program can be stored (recorded) in a removable recording medium 111 driven by the drive 109.
  • a removable recording medium 111 can be provided as so-called package software.
  • examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.
  • the program can also be downloaded to the computer via a communication network or broadcasting network and installed on the built-in hard disk 105.
  • a program can be transferred wirelessly from a download site to a computer via an artificial satellite for digital satellite broadcasting, or transferred to a computer by wire via a network such as a LAN (Local Area Network) or the Internet. be able to.
  • the computer has a built-in CPU (Central Processing Unit) 102, and an input/output interface 110 is connected to the CPU 102 via a bus 101.
  • CPU Central Processing Unit
  • the CPU 102 executes a program stored in a ROM (Read Only Memory) 103 in accordance with the command. .
  • the CPU 102 loads the program stored in the hard disk 105 into the RAM (Random Access Memory) 104 and executes the program.
  • the CPU 102 performs processing according to the above-described flowchart or processing performed according to the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result from the output unit 106 or transmits it from the communication unit 108 via the input/output interface 110, or records it on the hard disk 105, as necessary.
  • the input unit 107 includes a keyboard, a mouse, a microphone, and the like.
  • the output unit 106 includes an LCD (Liquid Crystal Display), a speaker, and the like.
  • the processing that a computer performs according to a program does not necessarily have to be performed chronologically in the order described as a flowchart. That is, the processing that a computer performs according to a program includes processing that is performed in parallel or individually (for example, parallel processing or processing using objects).
  • program may be processed by one computer (processor) or may be processed in a distributed manner by multiple computers. Furthermore, the program may be transferred to a remote computer and executed.
  • a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are located in the same casing. Therefore, multiple devices housed in separate casings and connected via a network, and a single device with multiple modules housed in one casing are both systems. .
  • the configuration described as one device (or processing section) may be divided and configured as a plurality of devices (or processing sections).
  • the configurations described above as a plurality of devices (or processing units) may be configured as one device (or processing unit).
  • part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit) as long as the configuration and operation of the entire system are substantially the same. .
  • the present technology can take a cloud computing configuration in which one function is shared and jointly processed by multiple devices via a network.
  • the above-mentioned program can be executed on any device. In that case, it is only necessary that the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.
  • each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices.
  • the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.
  • multiple processes included in one step can be executed as multiple steps.
  • processes described as multiple steps can also be executed together as one step.
  • the processing of the steps described in the program may be executed in chronological order according to the order described in this specification, in parallel, or in a manner in which calls are made. It may also be configured to be executed individually at necessary timings such as at certain times. In other words, the processing of each step may be executed in a different order from the order described above, unless a contradiction occurs. Furthermore, the processing of the step of writing this program may be executed in parallel with the processing of other programs, or may be executed in combination with the processing of other programs.
  • the present technology can also have the following configuration. (1) Performing first imaging using as a subject the target object irradiated with illumination light using a first illumination method suitable for restoring the shape of the target object; performing second imaging with the target object irradiated with illumination light using a second illumination method suitable for restoring the color of the target object as a subject; The shape and color of the target object are determined in 3D using a first image group obtained by the first image capturing a plurality of times and a second image group obtained by the second image capturing a plurality of times.
  • An information processing method comprising: performing 3D restoration processing for restoration using a model.
  • the first illumination method uses patterned light in which predetermined patterns are arranged at equal intervals.
  • the pattern light and normal illumination light are used in combination.
  • the first illumination method uses patterned light of random dots.
  • the target object is irradiated with illumination light from the same direction as the imaging direction of the imaging unit that performs the second imaging. Information processing method.
  • the target object is irradiated with illumination light from a direction in which specularly reflected light from the target object does not enter an imaging unit that performs the second imaging.
  • the information processing method described in any of the above. (7) The information processing method according to any one of (1) to (6) above, wherein in the second illumination method, the target object is irradiated with illumination light by light sources with different luminous intensities and wavelengths. (8) In the 3D restoration process, coloring of the target object is performed using the second image group on the 3D model made of the shape of the target object created using the first image group. The information processing method according to any one of (1) to (7). (9) If it is determined that there is a defect in the 3D model created in the 3D restoration process, the imaging position and direction that cover the surface formed by connecting the missing areas or the color of the shadow area are complemented.
  • the information processing method wherein the first imaging position and the imaging direction are fed back and the first imaging and the second imaging are repeatedly performed.
  • a first illumination unit that irradiates illumination light using a first illumination method suitable for restoring the shape of the target object; a second illumination unit that emits illumination light using a second illumination method suitable for restoring the color of the target object; an imaging unit that performs second imaging using the target object irradiated with illumination light by the second illumination method as a subject;
  • the shape and color of the target object are determined in 3D using a first image group obtained by the first image capturing a plurality of times and a second image group obtained by the second image capturing a plurality of times.
  • An information processing device comprising: a 3D restoration processing unit that performs 3D restoration processing using a model.
  • the shape and color of the target object are determined in 3D using a first image group obtained by the first image capturing a plurality of times and a second image group obtained by the second image capturing a plurality of times.
  • a program for executing information processing including performing 3D restoration processing for restoration using a model.
  • Photogrammetry device 12 Network, 13 and 14 Illumination device, 21 First illumination section, 22 Second illumination section, 23 Imaging section, 24 First illumination drive section, 25 Second illumination drive section, 26 Imaging drive unit, 27 3D restoration processing unit, 28 control unit, 29 communication unit, 31 communication unit, 32 control unit, 41 communication unit, 42 control unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Optics & Photonics (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2023/020592 2022-06-14 2023-06-02 情報処理方法、情報処理装置、およびプログラム Ceased WO2023243430A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024528700A JPWO2023243430A1 (https=) 2022-06-14 2023-06-02

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-095854 2022-06-14
JP2022095854 2022-06-14

Publications (1)

Publication Number Publication Date
WO2023243430A1 true WO2023243430A1 (ja) 2023-12-21

Family

ID=89190968

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/020592 Ceased WO2023243430A1 (ja) 2022-06-14 2023-06-02 情報処理方法、情報処理装置、およびプログラム

Country Status (2)

Country Link
JP (1) JPWO2023243430A1 (https=)
WO (1) WO2023243430A1 (https=)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005351871A (ja) * 2004-06-14 2005-12-22 Canon Inc 物体情報入力装置および物体生成装置
JP2015059849A (ja) * 2013-09-19 2015-03-30 凸版印刷株式会社 色と三次元形状の計測方法及び装置
JP2017161370A (ja) * 2016-03-09 2017-09-14 株式会社ニコン 検出装置、検出システム、検出方法、及び検出プログラム
JP2018194544A (ja) * 2017-05-16 2018-12-06 キヤノン株式会社 情報処理装置、情報処理方法、コンピュータプログラム
JP2021157237A (ja) * 2020-03-25 2021-10-07 Kddi株式会社 自由視点映像生成方法、装置およびプログラム

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005351871A (ja) * 2004-06-14 2005-12-22 Canon Inc 物体情報入力装置および物体生成装置
JP2015059849A (ja) * 2013-09-19 2015-03-30 凸版印刷株式会社 色と三次元形状の計測方法及び装置
JP2017161370A (ja) * 2016-03-09 2017-09-14 株式会社ニコン 検出装置、検出システム、検出方法、及び検出プログラム
JP2018194544A (ja) * 2017-05-16 2018-12-06 キヤノン株式会社 情報処理装置、情報処理方法、コンピュータプログラム
JP2021157237A (ja) * 2020-03-25 2021-10-07 Kddi株式会社 自由視点映像生成方法、装置およびプログラム

Also Published As

Publication number Publication date
JPWO2023243430A1 (https=) 2023-12-21

Similar Documents

Publication Publication Date Title
KR102473547B1 (ko) 부품의 실장 상태를 검사하기 위한 방법, 인쇄 회로 기판 검사 장치 및 컴퓨터 판독 가능한 기록매체
US10252468B2 (en) Stereolithography printer
US10780641B2 (en) Enhanced three dimensional printing of vertical edges
US11580634B2 (en) System and method for automated surface assessment
CN102998908B (zh) 描画装置以及描画方法
US20150116460A1 (en) Method and apparatus for generating depth map of a scene
EP1814083A1 (en) Photometric measurements using at least three light sources
US20240160002A1 (en) Method and System for Mapping Objects on Unknown Specimens
EP3571667A1 (en) Real-time generation of synthetic data from multi-shot structured light sensors for three-dimensional object pose estimation
CN115937059A (zh) 具有生成式训练模型的零件检查系统
JP2020004085A (ja) 画像処理装置、画像処理方法及びプログラム
TWI647739B (zh) 圖案形成裝置及圖案形成方法
KR20150107426A (ko) 계층적 호겔 기반의 디지털 홀로그래픽 영상 기록 방법 및 장치
JP2012013696A (ja) 撮影装置とその撮影方法
WO2023243430A1 (ja) 情報処理方法、情報処理装置、およびプログラム
JP2022537196A (ja) 表面認識
JP5441752B2 (ja) 環境内の３ｄ物体の３ｄ姿勢を推定する方法及び装置
JP2018077168A (ja) シミュレーション装置、シミュレーション方法およびシミュレーションプログラム
JP2013148380A (ja) 外観検査装置
JP2017016169A (ja) 検査方法、検査装置、画像処理装置、プログラム及び記録媒体
JP7831215B2 (ja) 位置計測システム及び照明条件好適化システム
Denkena et al. Three-dimensional optical Measurement with locally Adapted Projection
JP2025030324A (ja) 検査システム及び検査方法
WO2025143023A1 (ja) モデリング装置、モデリング方法およびモデリングプログラム
US20210051303A1 (en) Information processing apparatus, information processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23823723

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024528700

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23823723

Country of ref document: EP

Kind code of ref document: A1