JP6821326B2 - Information processing equipment, measurement systems, information processing methods and programs - Google Patents

Description

The present invention relates to an information processing device, a measurement system, an information processing method and a program for measuring an object object.

When measuring an object object with a sensor from various viewpoints, there are methods such as determining the position and orientation of the viewpoint where the sensor is placed in advance, and a method in which a person moves the sensor to an appropriate viewpoint while checking the measurement results at any time. is there.
On the other hand, a technique aimed at automatically generating a viewpoint to be measured next from measurement results and the like is known as Next Best View (NBV). In this technology, for example, in order to capture a three-dimensional shape of the surface of an object object using a robot arm equipped with a distance sensor and automatically measure the object without omission, the viewpoint of the distance sensor from where the object object is measured is obtained. It is applied in cases.
In Patent Document 1, in order to measure a measurement target having a complicated shape with high accuracy, the position information of the measurement area is calculated based on the image data obtained from the optical probe that performs optical cutting, and the position information of this measurement area is also included. We also disclose a technique for determining the moving (scanning) direction of an optical probe.

Japanese Unexamined Patent Publication No. 2014-145735

However, the method of Patent Document 1 can be applied only under the limited condition that shape measurement is performed with high accuracy. Therefore, for example, when it is desired to perform shape measurement at high speed or when it is desired to inspect a defect on the surface of an object object, it is not assumed that the viewpoint to be measured next is appropriately determined. Therefore, it has been difficult for a measuring device such as a sensor to appropriately determine the viewpoint (measurement position / posture) at which the target object should be measured next in each case according to various measurement conditions.
The present invention has been made in view of the above problems, and an object of the present invention is to appropriately determine the viewpoint of a measuring device for measuring an object object next according to various measurement conditions.

In order to solve the above problems, according to an aspect of the information processing apparatus according to the present invention, a first acquisition means for acquiring measurement data obtained by measuring a target object by a measurement means for measuring the target object from a plurality of viewpoints, and a first acquisition means. A second acquisition means for acquiring at least one measurement condition among a plurality of measurement conditions when the measurement means images an image of the target object, the measurement data acquired by the first acquisition means, and the first. Based on the measurement conditions acquired by the second acquisition means, a plurality of candidate viewpoints of the measurement means to be measured next are generated, and the plurality of candidate viewpoints are acquired by the second acquisition means. An information processing apparatus is provided that includes a determination means for determining one or more viewpoints from a plurality of candidate viewpoints by applying an evaluation function corresponding to the measurement conditions .

According to the present invention, it is possible to appropriately determine the viewpoint of the measuring device to measure the target object next according to various measurement conditions.

The block diagram of the measurement system which concerns on Embodiment 1. Schematic diagram of the measurement system. A block diagram showing the configuration of a computer. A flowchart showing the measurement and viewpoint determination process. The figure which shows the example of polygon data. The figure which shows the example of how to obtain the candidate viewpoint. The figure which shows the generation example of a candidate viewpoint group. The figure which shows the example of the candidate viewpoint. The figure which shows the change of the state of a voxel. The block diagram of the measurement system which concerns on Embodiment 2. The block diagram of the measurement system which concerns on Embodiment 3. A conceptual diagram showing an example of a process for updating a candidate viewpoint. The block diagram of the measurement system which concerns on Embodiment 4. The block diagram of the measurement system which concerns on the modification. The block diagram of the measurement system which concerns on the modification.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example as a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is described below. It is not limited to the embodiment.
<Embodiment 1>
The measurement system according to the first embodiment appropriately determines the viewpoint of the measuring device (imaging unit) for measuring the measurement target object next, depending on the application for measuring the measurement target object among the measurement conditions. By measuring (imaging) the object to be measured while sequentially moving the imaging unit to the determined viewpoint, the measurement is performed according to the application. In the measurement system according to the first embodiment, it is possible to perform measurement in response to a user's request by allowing the user to select an application for measuring an object to be measured from a plurality of applications. The measurement of the measurement target object includes, for example, imaging of the measurement target object, but also includes measurement of the coordinates of a point cloud on the measurement target object.
In the first embodiment, as an example of a plurality of uses, a case where four uses of the three-dimensional shape measurement of the object to be measured, defect inspection, specific object recognition, and person monitoring can be selected will be described, but the first embodiment can be applied. The application of various measurements is not limited to these.

[Structure of Embodiment 1]
FIG. 1 is a block diagram showing a configuration example of the measurement system according to the first embodiment. This measurement system includes an imaging unit 2 that captures an object 5 to be measured and outputs image data and related information, a viewpoint setting unit 3 that sets a viewpoint at which the imaging unit 2 images the object 5 to be measured, and an imaging unit 2. It is provided with an information processing device 1 that controls the viewpoint setting unit 3 according to the information from. The information processing device 1 is connected to the image pickup unit 2 and the viewpoint setting unit 3, respectively.
As shown in the schematic view of FIG. 2, the imaging unit 2 is composed of, for example, a camera, and is attached to the viewpoint setting unit 3 in order to measure the object 5 to be measured from various viewpoints. The measurement data output by the image pickup unit 2 is, for example, a color image and a distance image obtained by capturing the measurement target object 5. However, the measurement data output by the imaging unit 2 is not limited to these, and may be either a color image or a distance image, or the coordinates of a monochrome image, a distance point cloud, or a point cloud on the object to be measured. Etc. may be used. That is, the measurement data is not limited to the image, and the imaging unit 2 may output the measurement data obtained by measuring the object to be measured from a plurality of viewpoints.

The imaging unit 2 supplies the captured color image and distance image to the data acquisition unit 11 and the viewpoint determination unit 12 of the information processing device 1.
The viewpoint setting unit 3 includes, for example, a robot arm 3a shown in FIG. 2 and an arm control unit (not shown) that controls the robot arm 3a. An imaging unit 2 is attached to the tip of the robot arm 3a. The arm control unit receives from the information processing device 1 the viewpoint of the imaging unit 2 to image the object 5 to be measured next, and controls the robot arm 3a so that the imaging unit 2 can image from the viewpoint to be imaged next. The imaging unit 2 is moved. That is, the viewpoint setting unit 3 sets the position and orientation of the imaging unit 2 based on the received viewpoint. Further, the viewpoint setting unit 3 supplies the position / orientation information (hereinafter, referred to as “viewpoint position / orientation information”) that identifies the viewpoint of the moved imaging unit 2 to the data acquisition unit 11 of the information processing device 1.

Returning to FIG. 1, next, the internal configuration of the information processing apparatus 1 will be described.
The information processing device 1 includes a data acquisition unit 11, a viewpoint determination unit 12, and a condition acquisition unit 13 for acquiring measurement conditions including an application for measuring a measurement target object 5.
The data acquisition unit 11 performs data processing according to the application by using the color image, the distance image, and the viewpoint position / orientation information supplied from the viewpoint setting unit 3 supplied from the imaging unit 2. The viewpoint determination unit 12 determines the viewpoint to be imaged next and supplies the viewpoint to the viewpoint setting unit 3.
The viewpoint determination unit 12 determines the viewpoint of the imaging unit 2 from the data supplied from the data acquisition unit 11. The viewpoint determination unit 12 includes a candidate viewpoint generation unit 121 that generates candidate viewpoints, and a candidate viewpoint evaluation unit 122 that evaluates the candidate viewpoints generated by the candidate viewpoint generation unit 121 according to the intended use.
The condition acquisition unit 13 acquires at least one measurement condition out of a plurality of measurement conditions when measuring the measurement target object 5. In the first embodiment, the condition acquisition unit 13 includes an application acquisition unit 131 that acquires information on the application for measuring the measurement target object 5.

In the first embodiment, since the operation inside the information processing apparatus 1 is changed according to the information of the application acquired by the application acquisition unit 131 of the condition acquisition unit 13, only the outline of the operation will be described first, and then for each application. The details of the operation will be described for each case.
The data acquisition unit 11 receives a color image and a distance image from the imaging unit 2, executes data processing such as data integration processing according to the application, and generates an output result. Then, the data acquisition unit 11 supplies the output result to the viewpoint determination unit 12.
The viewpoint determining unit 12 determines the viewpoint of the imaging unit 2 to measure the object 5 to be measured next according to the output result supplied from the data acquisition unit 11 and the application supplied from the condition acquisition unit 13. .. The viewpoint determination unit 12 supplies the determined viewpoint to the viewpoint setting unit 3.
Specifically, the candidate viewpoint generation unit 121 generates at least one candidate viewpoint according to the application based on sensor information such as the resolution, angle of view, and depth of field of the imaging unit 2. The candidate viewpoint evaluation unit 122 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 121 according to the intended use, and selects a determination viewpoint. The candidate viewpoint evaluation unit 122 supplies the selected determined viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be imaged next.

The condition acquisition unit 13 acquires at least one measurement condition out of the plurality of measurement conditions and supplies the information to the viewpoint determination unit 12. In the first embodiment, the application acquisition unit 131 acquires information on the application for measuring the object to be measured, and supplies the information on the application to the viewpoint determination unit 12. The usage information includes, for example, one or more of three-dimensional shape measurement, defect inspection, specific object recognition, and person monitoring.
In the first embodiment, the method of acquiring the application by the application acquisition unit 131 may be a method of displaying an image corresponding to each application on a touch display (not shown) and allowing the user to select the application. However, the method of acquiring the usage is not limited to this example, and the usage acquisition unit 131 may acquire the usage by voice input from the user. Alternatively, when connected to another device such as a 3D printer, the application acquisition unit 131 may automatically determine and select the application according to the connected device. As a result, for example, when a 3D printer or the like is connected, the application acquisition unit 131 automatically determines the application and selects the three-dimensional shape measurement as the application.

Each functional block shown in FIG. 1 is stored as a program in a storage unit such as ROM 22 described later, and is executed by the CPU 21. At least a part of the functional blocks shown in FIG. 1 may be realized by hardware. When it is realized by hardware, for example, by using a predetermined compiler, a dedicated circuit may be automatically generated on the FPGA from the program for realizing each step. FPGA is an abbreviation for Field Programmable Gate Array. Further, a Gate Array circuit may be formed in the same manner as the FPGA and realized as hardware. Further, it may be realized by ASIC (Application Specific Integrated Circuit).

FIG. 3 shows an example of the hardware configuration of the information processing device 1. The information processing device 1 includes a CPU 21, a ROM 22, a RAM 23, an external memory 24, an input unit 25, and an output unit 26. The CPU 21 performs various calculations and controls each part constituting the information processing apparatus 1 according to the input signal or program. Specifically, the CPU 21 executes measurement and viewpoint determination processing according to information from the imaging unit 2, and then determines a viewpoint to be imaged. The functional block of FIG. 1 described above illustrates a function executed by the CPU 21. The ROM 22 stores a program for executing each functional unit shown in FIG. 1 and various setting information. The RAM 23 stores temporary data and is used for the work of the CPU 21. The external memory 24 is, for example, a removable memory card, and can be attached to a PC (personal computer) or the like to read data.
The input unit 25 stores the information input from the image pickup unit 2 in a predetermined area of the RAM 23 or the external memory 24. The output unit 26 supplies the viewpoint setting unit 3 with the viewpoint to be imaged next, which is determined by the CPU 21.

[Measurement and viewpoint determination processing]
FIG. 4 is a flowchart showing details of the measurement and viewpoint determination processing executed by the measurement system of FIG. The flowchart of FIG. 4 is processed by the CPU 21 included in the information processing device 1 executing a program stored in the ROM 22 or the like. This process is a process that is executed while the execution of the measurement process of the measurement target object 5 is instructed, and is, for example, a process that is started when the user presses a measurement start button (not shown).
Hereinafter, the measurement and the viewpoint determination process by the measurement system of the first embodiment will be described with reference to the flowchart of FIG. In the first embodiment, the entire processing flow will be described with reference to the same flowchart of FIG. 4, regardless of the use described later in each case.

In S1, the initialization process is executed in the measurement system. In this initialization process, the imaging unit 2 is activated, the viewpoint setting unit 3 is activated, the usage acquisition unit 131 of the condition acquisition unit 13 is acquired, the background model is read, and the imaging unit 2 is moved to the initial viewpoint. Etc. are included. The initial viewpoint may be freely determined within the range in which the measurement target object 5 can be observed.
In S2, the imaging unit 2 images the measurement target object 5 from the viewpoint at which the current position is located, and acquires a color image and a distance image. Then, the imaging unit 2 supplies the acquired color image and distance image to the data acquisition unit 11. Further, the viewpoint setting unit 3 supplies the viewpoint position / orientation information to the data acquisition unit 11.

In S3, the data acquisition unit 11 receives data according to the application acquired by the application acquisition unit 131 with respect to the color image and the distance image supplied from the imaging unit 2 and the viewpoint position / orientation information supplied from the viewpoint setting unit 3. Execute the process. The data acquisition unit 11 supplies the data after the data processing to the viewpoint determination unit 12.
In the first embodiment, "data processing according to the application" is data processing according to each application of three-dimensional shape measurement, defect inspection, specific object recognition, and person monitoring, as described later, and the details will be described later. The function for executing this data processing may be provided as a functional unit separate from the data acquisition unit 11, or may be provided outside the information processing device 1. Alternatively, a function for executing data processing may be provided as a function of the imaging unit 2, and the result of the data processing may be supplied to the data acquisition unit 111 as measurement data.

In S4, the data acquisition unit 11 determines whether or not the user has pressed a measurement end button (not shown), and ends the measurement if the measurement end button is pressed. On the other hand, if the measurement end button has not been pressed yet, the process proceeds to S5.
In S5, the viewpoint determination unit 12 determines the viewpoint of the image pickup unit 2 to be imaged next based on the data after data processing supplied from the data acquisition unit 11 and the application acquired by the application acquisition unit 131. More specifically, in the viewpoint determination unit 12, the candidate viewpoint generation unit 121 generates one or a plurality of candidate viewpoints. In addition, the candidate viewpoint evaluation unit 122 evaluates the generated candidate viewpoint and selects the viewpoint with the highest evaluation among the conditions as the determination viewpoint. The viewpoint determination unit 12 supplies the selected determination viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be imaged next.

In S6, the viewpoint setting unit 3 moves the image pickup unit 2 based on the viewpoint to be imaged next received from the viewpoint determination unit 12. After that, the process returns to S2 again, and each part repeats the processes from S2 to S6.
In the above, in S4, an example in which the end of measurement is determined based on whether or not the measurement end button is pressed has been described, but the present invention is not limited to this. For example, an end determination unit (not shown) may determine whether or not to end based on the data obtained from the data acquisition unit 11 and the intended use. That is, the end determination unit determines the end condition for determining the viewpoint based on the measurement data.

Next, regarding the internal operation of the information processing device 1, the operation when the information of each application of three-dimensional shape measurement, defect inspection, specific object recognition, and person monitoring is acquired, and the operation when a plurality of applications are combined. The specific processing contents will be described in order for the operation.
[Use 1: 3D shape measurement]
When the usage acquisition unit 131 of the condition acquisition unit 13 acquires the three-dimensional shape measurement as the measurement purpose, the data acquisition unit 11 generates a three-dimensional model from the entire circumference of the measurement target object 5. In this measurement, the criterion for selecting the viewpoint to be measured (imaging) next is to eliminate the part where the shape is interrupted or the hole in the three-dimensional model.
Specifically, in this three-dimensional shape measurement, the data acquisition unit 11 measures using the color image and the distance image received from the imaging unit 2 and the viewpoint position / orientation information obtained from the viewpoint setting unit 3. The position of the measurement target object 5 of the three-dimensional point cloud in the coordinate system is obtained.

More specifically, the data acquisition unit 11 is supplied with the color image, the distance image, and the viewpoint position / orientation information captured while changing the viewpoint one after another in S3 of FIG. 4 until the measurement is completed. The data acquisition unit 11 uses the newly obtained distance image and the viewpoint position / orientation information to obtain the coordinates of the three-dimensional point cloud corresponding to each pixel in the distance image. Further, the data acquisition unit 11 converts the obtained coordinates of the three-dimensional point cloud into the coordinates of the three-dimensional point cloud in the coordinate system of the object 5 to be measured, and integrates the point clouds. Here, "integration" means an operation of merging the point cloud data obtained from the previously supplied distance image and the point cloud data obtained from the newly supplied distance image.

However, the three-dimensional point cloud acquired from the distance image includes not only the point cloud belonging to the measurement target object 5 but also the point cloud belonging to the background object. Therefore, the data acquisition unit 11 removes the point cloud belonging to the background object by using the background model acquired in advance, and integrates only the point cloud belonging to the measurement target object 5.
Further, depending on the control of the position / orientation of the robot arm 3a of the viewpoint setting unit 3, the viewpoint position / orientation information includes an error. Therefore, when the obtained three-dimensional point cloud is deviated, the data acquisition unit 11 aligns the point cloud by using an alignment method such as ICP (Iterative Closet Point), and then points. Integrate the group.

Further, as described above, in the three-dimensional model, whether or not there is a break or a hole in the obtained shape becomes a problem, so the data acquisition unit 11 generates a mesh from the integrated point cloud data. And use it as polygon data. In this polygon data, when the point clouds are too far apart, the data acquisition unit 11 identifies the portion as if there is a break or a hole. In addition, color image information may be added to the polygon data to obtain textured polygon data, or colored point cloud polygon data may be used.
Further, the end of the measurement is determined by the end determination unit (not shown) receiving the polygon data from the data acquisition unit 11 and confirming whether or not there is a break or a hole in the polygon data. You may. In this case, the end determination unit instructs the viewpoint determination unit 12 to determine the next viewpoint when there is a break or a hole, and when there is no break or a hole, the end determination unit measures. You may end it.

Further, in S5, polygon data is supplied from the data acquisition unit 11 to the viewpoint determination unit 12, and information indicating that the three-dimensional shape measurement is used is supplied from the condition acquisition unit 13. The viewpoint determination unit 12 determines the viewpoint of the image pickup unit 2 to be measured (imaging) next from the polygon data and the usage information.
Here, only the point cloud on the surface of the foremost object can be obtained from the distance image, and the point cloud of the shadowed portion cannot be obtained. For this reason, the polygon data of the object 5 to be measured may have a break or a hole only with a limited number of distance images from the viewpoint. Therefore, the candidate viewpoint generation unit 121 identifies a place where there is a break or a hole in the polygon data.

Hereinafter, a method of detecting a break or a hole in the polygon data will be described with reference to FIG. FIG. 5A shows an example of a mesh definition in polygon data. In the polygon data, three-dimensional coordinates are associated with each vertex (5A, 5B ...). Further, in the polygon data, the vertices 5A and the vertices 5B at both ends of the edge 5C are associated with each other. Further, in the polygon data, the corresponding vertex groups (vertices 5A, 5B ...) or edge groups (5C ...) are associated with the surfaces 5D and 5E.
In such polygon data, for example, as shown by a thick line in FIG. 5B, in a place where the mesh is interrupted 5G or a hole 5F, one side of the edge connecting the vertices of the surface is on one side. Only the face exists. Therefore, the viewpoint determining unit 12 detects a broken portion or a hole by finding an edge (unconnected edge) whose surface exists only on one side of the edge in the polygon data.

When the interrupted portion 5G or the hole 5F is detected, the viewpoint determining unit 12 considers the angle of view of the imaging unit 2 and can image the surface including the interrupted portion 5G or the hole 5F, as shown in FIG. Find the direction of. Further, the viewpoint determination unit 12 causes the candidate viewpoint generation unit 121 to generate a candidate viewpoint 201 at a position separated from the measurement target object 5 by the focal length l of the imaging unit 2 in the direction of the obtained viewpoint.
The shape of the object to be measured 5 in the region of the interrupted portion 5G or the hole 5F is unknown. Therefore, as shown in FIG. 7, a plurality of candidate viewpoints 202a, 202b, 202c. In which the candidate viewpoint generation unit 121 changes the direction of observing the interrupted portion 5G or the region of the hole 5F and the distance to the viewpoint. .. .. May be generated. That is, a plurality of candidate viewpoints of the imaging unit 2 for which the measurement target object should be measured next may be generated based on the measurement data and the measurement conditions.

However, the method of obtaining the candidate viewpoint is not limited to the above. For example, as shown in FIG. 8, assuming a regular icosahedron 203 so as to surround the measurement target object 5, the candidate viewpoint generation unit 121 moves the center of the regular icosahedron 203 from each vertex of the regular icosahedron 203. The observation direction may be generated as a candidate viewpoint. Alternatively, the candidate viewpoint generation unit 121 may randomly generate a large number of candidate viewpoints.
The candidate viewpoint evaluation unit 122 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 121, and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. That is, the candidate viewpoint evaluation unit 122 applies an evaluation function corresponding to the measurement condition to the plurality of candidate viewpoints, and selects one or more viewpoints from the plurality of candidate viewpoints.

Here, as an example of the evaluation value calculation method, a method using voxels will be described. First, the candidate viewpoint evaluation unit 122 arranges the voxels in a range sufficient to surround the measurement target object 5. The size of each voxel is about the same as or slightly larger than the interval of the three-dimensional point cloud obtained based on the resolution and depth of field of the imaging unit 2, that is, one voxel contains one to a dozen point clouds. It should be large enough to be used.
Each voxel has three states, "occupied", "empty", and "unknown". Initially, as shown in FIG. 9 (A), all are "unknown"("not" in FIG. 9 (A)). Is abbreviated.). Note that FIG. 9A shows a horizontal plane for one voxel for the sake of explanation, and the voxels are actually defined in a state where such horizontal planes overlap in the height direction.

Next, for example, as shown in FIG. 9B, when a distance image is captured by the imaging unit 2 from a certain viewpoint 205, the data acquisition unit 11 obtains a distance point cloud from the distance image and the candidate viewpoint evaluation unit 122. Supply to.
When the distance point cloud is supplied, the candidate viewpoint evaluation unit 122 changes the state of the voxels (210a, 210b) in which the distance point cloud is observed in the voxel space from "unknown" to "unknown" as shown in FIG. 9C. Change to "occupation" (abbreviated as "occupation" in FIG. 9C). Further, the candidate viewpoint evaluation unit 122 changes the state of the voxel 210d in the direction in which the observed value was not obtained and the state of the voxel 210c between the imaging unit 2 and the voxel in which the distance point cloud was observed from "unknown". Change to "empty".

When evaluating the candidate viewpoint, the candidate viewpoint evaluation unit 122 obtains the number V (p) of “unknown” voxels observable from the candidate viewpoint p. Further, the candidate viewpoint evaluation unit 122 obtains the number E (p) of unconnected edges that can be observed from the candidate viewpoint from the polygon data, and the number V (p) of voxels and the number E of unconnected edges by the equation (1). The evaluation value Eval (p) is calculated as the weighted linear sum of (p).
Eval (p) = w _V V (p) + w _E E (p) (1)
By evaluating the candidate viewpoint using this equation (1), the candidate viewpoint evaluation unit 122 can observe a voxel that has not been observed yet, a place where polygon data is interrupted, or a region with a hole. The evaluation value of the viewpoint becomes high. This makes it easier for such a candidate viewpoint to be selected as the viewpoint to be imaged next.

Next, the candidate viewpoint evaluation unit 122 can move from the position / posture of the current viewpoint to the position / posture of the candidate viewpoint p in order from the candidate viewpoint having the highest evaluation value Eval (p) among the candidate viewpoints. , Confirm by making a judgment of path planning by simulation.
Specifically, in the simulation in the determination of path planning, the candidate viewpoint evaluation unit 122 moves from the current viewpoint to the candidate viewpoint p based on the movable range of the viewpoint setting unit 3, the interference with the measurement target object 5 and the background, and the like. Determine if it is possible to move.

If it is impossible to move, the candidate viewpoint evaluation unit 122 repeats the path planning determination in the same manner for the candidate viewpoint p having the next highest evaluation value. The candidate viewpoint evaluation unit 122 determines the path planning from the candidate viewpoint having a high evaluation value, and when a movable candidate viewpoint p is found, selects the candidate viewpoint p as the viewpoint to be measured next. It is supplied to the viewpoint setting unit 3.
However, here, for example, each weight value w _V = w _E = 1, but the weight value is not limited to this. As the weight values w _V and w _E , fixed values based on experience may be used, may be changed according to the number of measurements, or the values may be changed according to other conditions.

[Use 2: Defect inspection]
When the use acquisition unit 131 acquires a defect inspection for the purpose of measurement, the data acquisition unit 11 observes the measurement target object 5 from the entire circumference and determines whether or not there is a scratch on the surface of the measurement target object 5. To do. In this measurement, the criteria for selecting the next viewpoint to be measured (imaging) is not only that there are no breaks or holes as in the 3D shape measurement, but also that it is determined whether it is a scratch or not, multiple times. Try to see if it is a scratch from a different perspective.
Specifically, the data acquisition unit 11 generates polygon data in the same manner as in the case of three-dimensional shape measurement, and further performs a scratch discrimination process on the color image received from the imaging unit 2 to determine that the color image is a scratch. The area is mapped on the polygon data of the measurement target object 5. The scratch discrimination process is performed by the data acquisition unit 11 by, for example, template matching with a scratch template image prepared in advance. The scratch discrimination process is not limited to this, and the scratch may be discriminated by using edge detection or the like.

More specifically, the data acquisition unit 11 is supplied with the color image, the distance image, and the viewpoint position / orientation information captured while changing the viewpoint one after another in S3 of FIG. 4 until the measurement for defect inspection is completed. To. The data acquisition unit 11 performs scratch discrimination processing on the newly obtained color image, and maps the scratched area on the polygon data. The data acquisition unit 11 repeats this scratch determination process until it can be determined from the color images from a predetermined number of viewpoints whether or not the scratches are present. Since the ease of detection of scratches differs depending on the viewpoint of observing the scratches, the data acquisition unit 11 integrates the information measured from the plurality of viewpoints and finally determines whether or not the scratches are caused.

The data acquisition unit 11 calculates the degree of scratch R (t) for each minimum unit area t for mapping scratches in order to determine whether or not the scratches are present.
The scratch degree R (t) is an initial value of 1.0 in all regions before the measurement is started, and the scratch degree R (t) in the region determined not to be scratched by the scratch discrimination is multiplied by A. On the other hand, the degree of scratch R (t) in the region determined to be scratch is multiplied by 1 / A.
The data acquisition unit 11 determines, for example, that the region where the degree of scratches exceeds the threshold value _Tu = A ^α is a scratch, and that the region where the degree of scratches is below the threshold value T _b = 1 / A ^α is not a scratch. To do. Note that A is an arbitrary number larger than 1.0, and here, for example, A = 2.0. Further, α is a natural number, and here, for example, α = 2. The data acquisition unit 11 sends polygon data in which the degree of scratches is mapped to the viewpoint determination unit 12.

Here, the following method may be used as a method for ending the measurement. The end determination unit (not shown) receives polygon data with the degree of scratches mapped from the data acquisition unit 11, and first confirms that there are no breaks or holes, as in the case of three-dimensional shape measurement, and ends. judge. The end determination unit instructs the viewpoint determination unit 12 to determine the next viewpoint when there is a break or a hole.
On the other hand, if there is no break or hole, the end determination unit confirms whether or not there is a scratch on the object surface of the object 5 to be measured, and if there is a region, it can be determined. If there is no area, the viewpoint determination unit 12 is instructed to determine the next viewpoint. On the contrary, when there is no area for which it cannot be determined whether or not the scratch is made, the end determination unit ends the measurement.

In S5, the viewpoint determination unit 12 is supplied with polygon data to which the degree of scratches is mapped is supplied from the data acquisition unit 11, and information indicating that the defect inspection is used is supplied from the condition acquisition unit 13.
The viewpoint determination unit 12 determines the viewpoint to be imaged next from the polygon data and the usage information in a two-step process. First, the candidate viewpoint generation unit 121 generates a candidate viewpoint so as to eliminate a broken portion or a hole region, as in the case of three-dimensional shape measurement. Next, the candidate viewpoint generation unit 121 generates a candidate viewpoint so as to eliminate a region for which it cannot be determined whether or not it is a scratch. Candidate viewpoint generating unit 121, the polygon data scratch degree is mapped, the value of the flaw degree R (t) for each minimum unit region t for performing mapping, satisfies the condition _{T b <R (t) <} T u Specify the area t.

Since it is not determined whether or not there is a scratch in the region t that satisfies this condition, the candidate viewpoint generation unit 121 generates a viewpoint at which the region t can be observed as a candidate viewpoint. If there are many regions t for which the presence or absence of scratches cannot be determined, a large number of candidate viewpoints will be generated. Therefore, in order to improve calculation efficiency, the candidate viewpoint generation unit 121 will generate candidate viewpoints at appropriate intervals. May be sampled.
The candidate viewpoint evaluation unit 122 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 121, and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. Here, as a method of calculating the evaluation value, a method using voxels is used as in the case of three-dimensional shape measurement. Further, in the area where polygon data is generated, evaluation is performed based on the degree of scratches mapped. First, the candidate viewpoint evaluation unit 122 calculates the judgment index D (p) of the degree of scratches by the equation (2) for the set Θ (p) of the region t observable from the candidate viewpoint p.

However, g (t) is expressed by the equation (3).

Next, the candidate viewpoint evaluation unit 122 evaluates the evaluation value Eval (p) as a weighted linear sum of the number V (p) of voxels, the number E (p) of unconnected edges, and the judgment index D (p) according to the equation (4). ) Is calculated.
Eval (p) = w _V V (p) + w _E E (p) + w _D D (p) (4)
When the candidate viewpoint evaluation unit 122 evaluates the candidate viewpoint using the equation (4), whether or not there are voxels that have not been observed, areas where polygon data is interrupted, areas with holes, or scratches. The evaluation value of the candidate viewpoint that can observe the area where the determination is not made becomes high. This makes it easier for such a candidate viewpoint to be selected as the viewpoint to be imaged next.

Further, as in the case of the three-dimensional shape measurement, the candidate viewpoint evaluation unit 122 makes a path planning determination in order from the viewpoint having the highest evaluation value Eval (p) among the candidate viewpoints, and selects the viewpoint to be imaged next. , Supply to the viewpoint setting unit 3. Here, for example, the weight value w _V = w _E = w _D = 1, but the weight value is not limited to this, and the weight values w _V , w _E , and w _D are fixed values based on experience. May be used, the value may be changed according to the number of measurements, or the value may be changed according to other conditions.

[Use 3: Specific object recognition]
When the specific object recognition is acquired by the usage acquisition unit 131, the data acquisition unit 11 observes the measurement target object 5 from a certain viewpoint and measures the measurement target in a plurality of predetermined object models. Classify which model the object 5 corresponds to. That is, the data acquisition unit 11 classifies which model object the observed object 5 is the same as the object to be measured. The criterion for selecting the next viewpoint to be measured in this measurement is to observe the characteristic geometric shapes and textures peculiar to the measurement target object 5 from various viewpoints.

Specifically, the data acquisition unit 11 performs feature point detection (feature point extraction) on the color image received from the imaging unit 2, and refers to the viewpoint position / orientation information and the distance image to obtain three-dimensional position information at the feature points. And save in association with. At this time, the data acquisition unit 11 deletes the feature points belonging to the background by using the background model, and saves only the feature points belonging to the measurement target object 5 in association with the three-dimensional position information.
Then, the data acquisition unit 11 matches the detected feature points with the model data of the object recognition target, and selects a model having a high degree of similarity. If only one model with high similarity can be selected, that model will be the recognition result. On the other hand, when there is no model having a high degree of similarity or when the degree of similarity is high in a plurality of models, the data acquisition unit 11 determines the recognition result by using the result observed from the next viewpoint. For example, SURF (Speed-Up Robust Features) feature amount is used for feature point detection, but the feature amount is not limited to this, and other feature amount may be used.

Until the measurement is completed, the data acquisition unit 11 is supplied with the color image, the distance image, and the viewpoint position / orientation information captured while changing the viewpoint one after another in S3 of FIG. The data acquisition unit 11 detects feature points on the newly obtained color image and matches it with the model data of the object recognition target. Here, a recognition result already obtained is a model A and a probability P _A, the probability that model B is P _B, the probability is a model C and a P _C, · ·. Newly obtained recognition result, the probability that the model A is _P ^{A new new,} model B is a probability _P ^{B new new,} probability _P ^{C new new} is a model C, if it is .. a model j recognition The probability P _j ′ is calculated by the equation (5).

In this way, the data acquisition unit 11 performs measurement while updating the recognition probability of each model, and sends it to the end determination unit (not shown). Further, the data acquisition unit 11 supplies the viewpoint determination unit 12 with a feature point cloud associated with the viewpoint position / orientation information and the three-dimensional position information.
The end of the measurement may be determined by the end determination unit (not shown) confirming the recognition probability in each model received from the data acquisition unit 11. End determination unit, when the probability of the most recognized probability model exceeds a threshold value T _M, and ends the measurement the model as a recognition result. On the other hand, which model even when the recognition probability does not exceed the threshold value T _M is end determining unit instructs the viewpoint determining unit 12 to determine the next viewpoint.

Further, in S5, the viewpoint determination unit 12 is supplied with a feature point group in which the viewpoint position / orientation information and the three-dimensional position information are associated with each other from the data acquisition unit 11, and the condition acquisition unit 13 is used for specific object recognition. Information indicating that there is is provided. The viewpoint determination unit 12 determines the viewpoint to be imaged next from the feature point cloud and the information of the application. First, the candidate viewpoint generation unit 121 confirms the spatial distribution of the feature point groups, and sets the feature point groups close to each other as feature point clusters. Then, the candidate viewpoint generation unit 121 generates a viewpoint that is easy to observe many feature points inside the cluster as a candidate viewpoint for each feature point cluster. As for the viewpoint where many feature points are easily observed, viewpoints from various directions can be considered, but the candidate viewpoint generation unit 121 may randomly sample and obtain an appropriate direction.

The candidate viewpoint evaluation unit 122 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 121, and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. The evaluation value Eval (p) is calculated by the candidate viewpoint evaluation unit 122 by the equation (6) using the number F (p) of feature points observable from the candidate viewpoint p and the set Φ of the viewpoints used so far. To do.

When the candidate viewpoint evaluation unit 122 evaluates the candidate viewpoint using the equation (6), the number of observable feature points is large, and the evaluation value of the candidate viewpoint having a different position and orientation from the viewpoints observed so far is obtained. It becomes higher and easier to be selected as the next viewpoint to be imaged.
As in the case of the three-dimensional shape measurement, the candidate viewpoint evaluation unit 122 makes a path planning determination in order from the candidate viewpoint having the highest evaluation value Eval (p), and selects the viewpoint to be imaged next. , Supply to the viewpoint setting unit 3. Here, although the weight value w _{F =} w d _{= 1} is not the weight value is not limited thereto, the weight value w _F, may be used a fixed value based on experience as w _d, The value may be changed according to the number of measurements, or the value may be changed according to other conditions.

[Use 4: Person monitoring]
When the person monitoring is acquired by the usage acquisition unit 131, the data acquisition unit 11 observes the person to be measured so as to track it within the designated area.
In this measurement, the criterion for selecting the viewpoint to be measured next is to predict the position to reach next from the direction in which the person to be measured is moving and observe the position.
Specifically, the data acquisition unit 11 cuts out a person area from the color image and the distance image from the imaging unit 2, classifies the behavior of the person, and determines whether the person is normal or abnormal. The method of classifying a person's behavior may be, for example, applied to a joint model of the human body and associated with the behavior, or estimated from the degree of compatibility with many cases using a classifier such as deep learning. You may.

Until the measurement is completed, the data acquisition unit 11 is supplied with the color image, the distance image, and the viewpoint position / orientation information captured while changing the viewpoint one after another in S3 of FIG.
In the case of person monitoring, it is not necessary to integrate time-series data, and it is sufficient to judge only the classification of each data and normal / abnormal. The data acquisition unit 11 supplies the joint model of the human body and the classification result to the viewpoint determination unit 12.
The end of the measurement may be determined by the end determination unit (not shown) confirming the classification result received from the data acquisition unit 11. The end determination unit instructs the end of the measurement when the person is not detected for a certain period of time. Alternatively, the measurement may be terminated after monitoring is performed for a period separately specified by the user. If the end determination unit does not end, the end determination unit instructs the viewpoint determination unit 12 to determine the next viewpoint.

In S5, the viewpoint determination unit 12 is supplied with the joint model of the human body and the classification result from the data acquisition unit 11, and the condition acquisition unit 13 is supplied with information indicating that the person monitoring is used. The viewpoint determination unit 12 determines the next viewpoint to be measured from the joint model of the human body, the classification result, and the information on the usage. First, the candidate viewpoint generation unit 121 predicts the movement of the joint model from the transition and behavior classification results of the human body joint model and the newly received human body joint model in the past several frames in the vicinity. Then, the candidate viewpoint generation unit 121 generates a viewpoint as a candidate viewpoint so that the human body can be observed at the center of the imaging unit 2 as much as possible.

The candidate viewpoint evaluation unit 122 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 121, and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. Here, in the calculation of the evaluation value, the set Ω of the movement destinations y that the human body model is predicted to move at the next measurement, the probability ρ (y) of moving to each movement destination y, and a certain point x from the candidate viewpoint p are used. The distance Dist (p, x) from the center of the image to the imaging position of x in the case of observation is used. Specifically, the candidate viewpoint evaluation unit 122 calculates the evaluation value Eval (p) using the equation (7).

By evaluating the candidate viewpoint using the equation (7), the candidate viewpoint evaluation unit 122 can cover many predicted movement destinations in the image while capturing the position with a high probability of movement in the center of the image. The evaluation value of a good candidate viewpoint becomes high, and it becomes easy to be selected as the next viewpoint.
As in the case of the three-dimensional shape measurement, the candidate viewpoint evaluation unit 122 makes a path planning determination in order from the candidate viewpoint having the highest evaluation value Eval (p), and selects the viewpoint to be imaged next. , Supply to the viewpoint setting unit 3. Although the weight value w _k = 1 is set here, the weight value is not limited to this, and the weight value w _k may be a fixed value based on experience, depending on the number of measurements. The value may be changed depending on other conditions.

[Operation according to multiple uses]
It is also possible to determine the next viewpoint to be imaged for multiple applications. For example, when it is desired to realize two uses of three-dimensional shape measurement and specific object recognition, the candidate viewpoint generation unit 121 generates a candidate viewpoint group based on the processing in each use described above. Then, the candidate viewpoint evaluation unit 122 obtains the evaluation value Eval (p) by the formula (8) which is a combination of the formula (1) and the formula (6), evaluates each candidate viewpoint, and next the candidate viewpoint having a high evaluation value. The viewpoint to be imaged. This makes it possible to appropriately determine the viewpoint to be imaged next according to a plurality of applications.

As described above, in the first embodiment, the viewpoint of the measuring device (imaging unit 2) to measure the measurement target object next can be appropriately determined according to the application for measuring the measurement target object. Further, in the first embodiment, the imaging unit 2 is moved to the viewpoint determined in this way, and the measurement is repeated. That is, the acquisition of the measurement data by the data acquisition unit 11, the determination of the viewpoint of the imaging unit 2 by the viewpoint determination unit 12, and the output of the viewpoint of the imaging unit 2 are repeated. Since the viewpoint of the imaging unit 2 to measure the object to be measured next is appropriately set according to the application, it is possible to efficiently perform the measurement according to the application by performing the measurement according to this viewpoint. .. In addition, since the user selects the purpose for measuring the object to be measured, it is possible to efficiently perform the measurement according to the user's selection.

<Embodiment 2>
The measurement system according to the second embodiment appropriately determines the viewpoint of the imaging unit 2 to measure the object to be measured next according to the specification request regarding the measurement application. By measuring the object to be measured while sequentially moving the imaging unit to the determined viewpoint, it is possible to perform appropriate measurement according to the specification requirements. Here, the "spec requirement" is an item or factor of specifications that should be prioritized in the measurement of the object to be measured according to the application.
In the first embodiment, the appropriate viewpoint to be measured next is determined according to a plurality of uses, but in the second embodiment, the viewpoint to be measured next is determined according to a plurality of specification requirements such as accuracy and speed. To do. Therefore, even if the purpose of measurement is decided, it is possible to perform measurement that meets the specification requirements related to the use. In the second embodiment, the use of three-dimensional shape measurement of the object to be measured is taken as an example, and a case where three spec requirements of accuracy, speed, and complexity can be selected as an example of a plurality of spec requirements will be described. , The spec requirements to which the second embodiment can be applied are not limited to these.

[Device configuration]
FIG. 10 is a block diagram showing a configuration example of the measurement system according to the second embodiment. This measurement system includes an imaging unit 2, a viewpoint setting unit 3, and an information processing device 30 that controls the viewpoint setting unit 3 in response to information from the imaging unit 2. The measurement system in the second embodiment is a partially modified version of the configuration of FIG. 1 in the first embodiment, and the imaging unit 2 and the viewpoint setting unit 3 in FIG. 10 are the same as those in the first embodiment and have the same operation. I do.
The information processing apparatus 30 according to the second embodiment includes a data acquisition unit 31, a viewpoint determination unit 32, and a condition acquisition unit 33 for acquiring measurement conditions. The data acquisition unit 31 performs data processing for three-dimensional shape measurement. The viewpoint determination unit 32 determines the viewpoint of the imaging unit 2 to be imaged next from the data after data processing supplied from the data acquisition unit 31 and the measurement conditions supplied from the condition acquisition unit 33, and the viewpoint setting unit 32 determines the viewpoint of the imaging unit 2 to be imaged next. Supply to 3.

The viewpoint determination unit 32 includes a candidate viewpoint generation unit 321 that generates a candidate viewpoint and a candidate viewpoint evaluation unit 322 that evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 321. In the second embodiment, the condition acquisition unit 33 includes a spec request acquisition unit 332 that acquires information on the spec request related to measurement.
In the second embodiment, the operation inside the information processing apparatus 30 is changed according to the information of the spec request acquired by the spec request acquisition unit 332 of the condition acquisition unit 33. Therefore, first, only the outline of the operation will be described, and then the details of the operation will be described for each specification requirement.

The data acquisition unit 31 receives a color image and a distance image from the image pickup unit 2 and executes data processing such as three-dimensional shape measurement. Then, the data acquisition unit 31 supplies the data after the data processing to the viewpoint determination unit 32.
The viewpoint determination unit 32 next measures the measurement target object 5 according to the data after data processing supplied from the data acquisition unit 31 and the specification request (priority specification) supplied from the condition acquisition unit 33. The viewpoint of the imaging unit 2 to be used is determined.
Specifically, the candidate viewpoint generation unit 321 generates at least one candidate viewpoint according to the specification request. The candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 321 according to the spec request, and selects a determination viewpoint. The candidate viewpoint evaluation unit 322 supplies the selected determined viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be imaged next.

The condition acquisition unit 33 acquires the conditions in the measurement and supplies the information to the viewpoint determination unit 32. In the second embodiment, the spec request acquisition unit 332 acquires the spec request information and supplies the acquired spec request information to the viewpoint determination unit 32. The spec requirement (information on the priority specification) includes, for example, any one or more of accuracy, speed, and complexity in the measurement of the object 5 to be measured.
In the second embodiment, the method in which the spec request acquisition unit 332 acquires the spec request is a method in which an image corresponding to each spec request is displayed on a touch display (not shown) and the user is made to select the spec request to be prioritized. However, the method of acquiring the spec request is not limited to this, and the spec request acquisition unit 332 may acquire the spec request by voice input from the user, and the user can obtain, for example, accuracy, speed, and complexity. You may get what you input the concrete target value of.

[Measurement and viewpoint determination processing]
Hereinafter, measurement and viewpoint determination processing by the measurement system of the second embodiment will be described. In the second embodiment, the processing flow can be described by using the same flowchart regardless of the spec requirements which will be described later in each case. Further, since the overall flow of the processing in the second embodiment is the same as that of the flowchart of FIG. 4, the description will be given with reference to FIG. Further, the processing in S2, S3, S4, and S6 in the second embodiment is the same as that in the first embodiment, and since the same processing is performed, only the other S1 and S5 will be described.

In S1, the initialization process is executed in the measurement system. In the initialization process, the imaging unit 2 is activated, the viewpoint setting unit 3 is activated, the spec request acquisition unit 332 of the condition acquisition unit 33 acquires the spec request, the background model is read, and the imaging unit 2 is moved to the initial viewpoint. Processing etc. are included. The initial viewpoint may be freely determined within the range in which the measurement target object 5 can be observed.
In S5, the viewpoint determination unit 32 determines the viewpoint of the imaging unit 2 to be imaged next based on the data after data processing supplied from the data acquisition unit 31 and the spec request acquired by the spec request acquisition unit 332. decide. More specifically, the candidate viewpoint generation unit 321 generates a candidate viewpoint, the generated candidate viewpoint is evaluated by the candidate viewpoint evaluation unit 322, and the candidate viewpoint with the highest evaluation among the conditions is selected as the determination viewpoint. .. The candidate viewpoint evaluation unit 322 supplies the selected determined viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be measured next.

Next, with respect to the internal operation of the information processing apparatus 30, specific processing contents will be described in order regarding the operation when the information of each specification requirement of accuracy, speed, and complexity is acquired.
[Specification 1: When priority is given to accuracy]
When the spec request acquisition unit 332 of the condition acquisition unit 33 acquires accuracy as a priority spec request, the viewpoint determination unit 32 generates a three-dimensional model from the entire circumference of the measurement target object 5 with high accuracy. The viewpoint of the imaging unit 2 to be measured next is determined so as to be performed.
The criterion for selecting the next viewpoint to be imaged in this measurement is to observe an edge region having a length exceeding the maximum value of the edge length set in the polygon data. The maximum value may be specified by the user. By selecting the viewpoint in this way, it is possible to measure a region where only a sparse three-dimensional point cloud with long edges is obtained, and to increase the density of the three-dimensional point cloud.

In S3, the data acquisition unit 31 generates polygon data from the distance image obtained from the imaging unit 2 and the viewpoint position / orientation information obtained from the viewpoint setting unit 3 as in the case of the three-dimensional shape measurement of the first embodiment. To do. Since the distance images captured by changing the viewpoint are sequentially supplied during the measurement, the data acquisition unit 31 updates the polygon data based on the supplied distance images.
When accuracy is prioritized, it is desired that the accuracy of the final 3D model result is high even if the measurement time is long. Therefore, the imaging unit 2 repeats imaging from the same viewpoint a plurality of times, and supplies a plurality of color images and a plurality of distance images as a set to the data acquisition unit 31. The data acquisition unit 31 can suppress the influence of the accidental error caused by the imaging unit 2 by updating the polygon data using the color image and the distance image obtained by averaging the plurality of color images and the distance images.
The end of the measurement may be determined by the end determination unit (not shown) receiving polygon data from the data acquisition unit 31 and confirming whether there are any interruptions, holes, or points with a low density of point clouds. .. In this case, the end determination unit ends the measurement when there is no point cloud with a low density.

Further, in S5, polygon data is supplied from the data acquisition unit 31 to the viewpoint determination unit 32, and information indicating that the spec request to be prioritized is accurate is supplied from the condition acquisition unit 33. The viewpoint determining unit 32 determines the viewpoint of the imaging unit 2 to be measured next from the polygon data and the information indicating the spec request to be prioritized.
The process of generating the candidate viewpoint by the candidate viewpoint generation unit 321 can be executed in the same manner as in the case of the three-dimensional shape measurement of the first embodiment. Further, in order to improve the accuracy, the candidate viewpoint generation unit 321 adds a viewpoint moved in the line-of-sight direction so that the distance to the target object is closer to the obtained candidate viewpoint to the candidate viewpoint. By adding such a candidate viewpoint, it becomes easy to image a more detailed part of the measurement target object 5.

The candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 321 and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. That is, the candidate viewpoint evaluation unit 322 applies an evaluation function corresponding to the measurement condition to the plurality of candidate viewpoints, and selects one or more viewpoints from the plurality of candidate viewpoints. In the calculation of the evaluation value, the candidate viewpoint evaluation unit 322 first determines the number of “unknown” voxels V (p) that can be observed from the candidate viewpoint p using the voxels as in the case of the three-dimensional shape measurement of the first embodiment. Ask for. Next, the candidate viewpoint evaluation unit 322 obtains the number E (p) of unconnected edges observable from the candidate viewpoint p and the number L (p) of observable edges longer than a predetermined length in the polygon data. .. Further, the candidate viewpoint evaluation unit 322 calculates the evaluation value Eval (p) by the equation (9).
Eval (p) = w _V V (p) + w _E E (p) + w _L L (p) (9)

When the candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint using the equation (9), there are voxels that have not been observed yet, and places or holes where the polygon data is interrupted, as in the equation (1). The evaluation value of the candidate viewpoint that can observe the area becomes high.
Further, in the second embodiment, the evaluation value of the candidate viewpoint that can observe the region of the three-dimensional model having a low density due to the long edge becomes high, and it becomes easy to select the candidate viewpoint that can be measured with higher accuracy.
Next, the candidate viewpoint evaluation unit 322 determines the path planning in order from the candidate viewpoint having the highest evaluation value, as in the case of the three-dimensional shape measurement of the first embodiment, and sets the movable candidate viewpoint as the next viewpoint to be imaged. decide. Note that fixed values based on experience may be used as the respective weight values w _V , w _E , and w _L in the equation (9), may be changed according to the number of measurements, or may be changed according to other conditions. May be changed.

[Specification 2: When speed is prioritized]
When the spec request acquisition unit 332 acquires the speed as a priority spec request, the viewpoint determination unit 32 then generates a three-dimensional model from the entire circumference of the measurement target object 5 in a short time. The viewpoint of the imaging unit 2 to be measured is determined.
In this measurement, the criterion for selecting the viewpoint to be measured next is that the object 5 to be measured may have a coarse point cloud density, so priority is given to imaging from the entire circumference, and the overlap with the already acquired three-dimensional shape data It is to take an image from a small number of viewpoints.

In S3, the data acquisition unit 31 generates polygon data and updates it with the sequentially acquired data, as in the case of the three-dimensional shape measurement of the first embodiment. When speed is prioritized, it is desirable to complete the measurement within a certain period of time even if the accuracy of the final 3D model is insufficient. Therefore, even if the point clouds are separated from each other in the polygon data, the data acquisition unit 31 connects the point clouds as much as possible and generates them as a mesh so that the results can be output at any time.
Then, the data acquisition unit 31 preferentially generates the entire rough shape, and if time is left, acquires a more detailed shape and updates it to the detailed polygon data. The end of the measurement is determined by the data acquisition unit 31 or the end determination unit (not shown) based on whether or not the measurement processing time exceeds a certain time, and if the time exceeds a certain time, the end of the measurement is instructed. It may be.

In S5, the viewpoint determination unit 32 is supplied with polygon data from the data acquisition unit 31, and the condition acquisition unit 33 is supplied with information indicating that the priority spec request is speed. The viewpoint determining unit 32 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the information of the spec request. The process of generating the candidate viewpoint by the candidate viewpoint generation unit 321 can be executed in the same manner as in the case of the three-dimensional shape measurement of the first embodiment. Further, in order to improve the speed, the candidate viewpoint generation unit 321 adds a viewpoint moved in the line-of-sight direction to the candidate viewpoint so that the distance from the measurement target object 5 is increased with respect to the obtained candidate viewpoint. By adding such a candidate viewpoint, it becomes easier to capture the entire image of the measurement target object 5 with one imaging.

The candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 321 and selects the candidate viewpoint having the maximum evaluation value as the determination viewpoint. In the calculation of the evaluation value, the candidate viewpoint evaluation unit 322 first determines the number of “unknown” voxels V (p) that can be observed from the candidate viewpoint p using the voxels as in the case of the three-dimensional shape measurement of the first embodiment. Ask for. Next, the candidate viewpoint evaluation unit 322 includes the number E (p) of unconnected edges observable from the candidate viewpoint p in the polygon data, the ratio L (p) of the observable edges longer than the predetermined length, and The area F (p) of the observable surface is obtained. Further, the candidate viewpoint evaluation unit 322 calculates the evaluation value Eval (p) by the equation (10).
Eval (p) = w _V V (p) + w _E E (p) + w _L (V _all (p) -V (p)) L (p) -w _F V (p) F (p) (10)
However, _Val (p) is the total number of voxels that can be observed from the candidate viewpoint p.

When the candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint using the equation (10), there are voxels that have not been observed yet, and places or holes where the polygon data is interrupted, as in the equation (1). The evaluation value of the candidate viewpoint that can observe the area becomes high. Furthermore, when the number of unknown voxels that have just started measurement is large, the number of polygon faces that have already been observed is as small as possible. The evaluation value of the candidate viewpoint is set to be high, and the same area is overlapped. The evaluation value of the candidate viewpoint, which is difficult to measure, becomes high.
In addition, as the measurement progresses and the number of unknown voxels decreases, the evaluation value of the viewpoint where edges longer than the predetermined length are observed becomes higher, and it becomes easier to select a candidate viewpoint that can measure coarse polygon data in more detail. ..
Next, the candidate viewpoint evaluation unit 322 determines the path planning in order from the viewpoint having the highest evaluation value, and then determines the viewpoint to be imaged, as in the case of the three-dimensional shape measurement of the first embodiment. In addition, fixed values based on experience may be used as the respective weight values w _V , w _E , w _L , and w _F in the equation (10), may be changed according to the number of measurements, or may be changed. The value may be changed depending on the conditions.

[Specification 3: When complexity is prioritized]
When the spec request acquisition unit 332 acquires the complexity as a priority spec request, the viewpoint determination unit 32 next so that the complex portion as the shape of the measurement target object 5 is preferentially imaged. The viewpoint of the imaging unit 2 to be measured is determined. The criterion for selecting the next viewpoint to be measured in this measurement is to observe a complicated shape part from multiple viewpoints.
In S3, the data acquisition unit 31 generates polygon data in the same manner as in the first embodiment, and updates the sequentially acquired data. When the priority spec requirement is complexity, it is desirable to be able to accurately model areas of complex shape, even if areas of flat or gentle surfaces are not accurately measured. Therefore, the data acquisition unit 31 has a sparse point cloud density when generating polygon data for a region determined to be a flat surface or a gentle surface from both a color image and a distance image. Also connect as an edge and set a low complexity for that area.

On the contrary, the data acquisition unit 31 sets a high degree of complexity in the region considered to have a complicated shape from the color image and / or the distance image. Further, the data acquisition unit 31 calculates the degree of achievement for the set complexity according to the density of the point cloud in the vicinity of each surface. The higher the complexity of the region, the higher the density of the point cloud needs to be, and the data acquisition unit 31 determines the rate at which the density standard according to the complexity is achieved as the achievement level.
The end of the measurement may be determined by the end determination unit (not shown) receiving polygon data with complexity and achievement from the data acquisition unit 31 and evaluating the achievement. When there is an area where the achievement level is less than the reference value, the end determination unit instructs the viewpoint determination unit 32 to determine the next viewpoint, and there is no area where the achievement level is less than the reference value and all areas. If the achievement level is satisfied with, the measurement is terminated.

In S5, the viewpoint determination unit 32 is supplied with polygon data with complexity and achievement from the data acquisition unit 31, and the condition acquisition unit 33 provides information indicating that the priority spec request is complexity. It is being supplied. The viewpoint determining unit 32 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the information of the spec request to be prioritized. In order to generate a candidate viewpoint by the candidate viewpoint generation unit 321, the interrupted portion 5G or the hole 5F in FIG. 7 may be set as a region with a low achievement level, and the candidate viewpoint for observing the region from various directions may be generated. By adding such a candidate viewpoint, it becomes easier to grasp a complicated area and an area where measurement is insufficient.

The candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 321 and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. In the calculation of the evaluation value, the candidate viewpoint evaluation unit 322 first obtains the number V (p) of “unknown” voxels observable from the candidate viewpoint p using voxels as in the first embodiment. Further, the candidate viewpoint evaluation unit 322 obtains the evaluation value Eval (p) by the equation (11).

When the candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint using the equation (11), there are voxels that have not been observed yet, and places or holes where the polygon data is interrupted, as in the equation (1). The evaluation value of the candidate viewpoint that can observe the area becomes high. Furthermore, it becomes easier to select a candidate viewpoint that can observe a region with a low degree of achievement, and it becomes easier to select a candidate viewpoint that images from a direction different from that observed in the past. The respective weight values w _V , w _E , w _H , and w _d may be fixed values based on experience, may be changed according to the number of measurements, or may be changed according to other conditions. You may.

[Operation according to multiple specifications]
Even when a plurality of spec requirements are acquired, it is possible to appropriately determine the viewpoint of the imaging unit 2 to image the measurement target object 5 next. For example, when it is desired to realize both speed priority and complexity priority, the candidate viewpoint generation unit 321 generates a candidate viewpoint group based on each of the cases already described. Then, the candidate viewpoint evaluation unit 322 evaluates the candidate viewpoint by the formula (12) which is a combination of the formula (10) and the formula (11), and the candidate viewpoint having a high evaluation value is the viewpoint of the imaging unit 2 to be imaged next. To do. As a result, it is possible to appropriately determine the viewpoint of the imaging unit 2 to be imaged next in response to a plurality of spec requirements.

As described above, in the second embodiment, the viewpoint of the measuring device (imaging unit 2) to measure the measurement target object next is appropriately determined according to the specification request selected by the user for the measurement target object. be able to. That is, it is possible to appropriately determine the viewpoint of the measuring means to measure the measurement target object next according to the information of the specifications that should be prioritized in the measurement of the target object. By moving the imaging unit 2 to the viewpoint determined in this way and repeating imaging, it is possible to perform appropriate measurement according to the specification requirements for the application.

<Embodiment 3>
The measurement system according to the third embodiment appropriately determines the viewpoint of the imaging unit 2 to measure the object to be measured next according to the characteristics of the moving means such as the movable range and the moving speed. By measuring the object to be measured while sequentially moving the imaging unit to the determined viewpoint, appropriate measurement is performed according to the characteristics of the moving means. In the first and second embodiments, the case where an appropriate viewpoint is determined according to the application and the specification requirements, respectively, has been described. However, in the third embodiment, the moving means for moving the imaging unit 2 to the viewpoint for measuring the measurement target object 5 An appropriate viewpoint is determined according to the characteristic information of the viewpoint setting unit 3). Therefore, even if the user freely replaces the moving means, appropriate measurement can be performed according to the moving means. In the third embodiment, the use of three-dimensional shape measurement of the target object is taken as an example, and three examples of the characteristics of the moving means are the movable range, the moving speed, and the moving accuracy. However, the third embodiment is applied. The characteristics of possible means of transportation are not limited to these.

[Device configuration]
FIG. 11 is a block diagram showing a configuration example of the measurement system according to the third embodiment. This measurement system includes an imaging unit 2, a viewpoint setting unit 3, and an information processing device 40 that controls the viewpoint setting unit 3 in response to information from the imaging unit 2. The measurement system in the third embodiment is a partially modified version of the configuration of FIG. 10 in the second embodiment, and the imaging unit 2 and the viewpoint setting unit 3 in FIG. 11 are the same as those in the first embodiment and have the same operation. I do.
The information processing apparatus 40 according to the third embodiment includes a data acquisition unit 41, a viewpoint determination unit 42, and a condition acquisition unit 43 for acquiring measurement conditions. The data acquisition unit 41 executes data processing such as three-dimensional shape measurement. The viewpoint determination unit 42 determines the viewpoint of the imaging unit 2 to be imaged next from the data after data processing supplied from the data acquisition unit 41 and the measurement conditions supplied from the condition acquisition unit 43, and the viewpoint setting unit 42 determines the viewpoint of the imaging unit 2 to be imaged next. Supply to 3.

The viewpoint determination unit 42 includes a candidate viewpoint generation unit 421 that generates a candidate viewpoint and a candidate viewpoint evaluation unit 422 that evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 421. Further, the condition acquisition unit 43 includes a movement means information acquisition unit 433 that acquires information on the characteristics of the viewpoint setting unit 3 which is a movement means.
In the third embodiment, the operation inside the information processing apparatus 40 is changed according to the information on the characteristics of the moving means acquired by the moving means information acquisition unit 433 in the condition acquisition unit 43. Therefore, first, only the outline of the operation inside the information processing apparatus 40 will be described, and then the details of the operation will be described for each characteristic of the moving means.

The data acquisition unit 41 executes data processing for three-dimensional shape measurement from the image pickup unit 2 on the color image and the distance image. Then, the data acquisition unit 41 supplies the data after the data processing to the viewpoint determination unit 42.
The viewpoint determination unit 42 is an imaging unit that should next image the measurement target object 5 according to the data after data processing supplied from the data acquisition unit 41 and the characteristics of the moving means supplied from the condition acquisition unit 43. Determine the viewpoint of 2. Specifically, first, the candidate viewpoint generation unit 421 generates at least one candidate viewpoint according to the characteristics of the moving means. The candidate viewpoint evaluation unit 422 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 421 according to the characteristics of the moving means, and selects the determination viewpoint. The candidate viewpoint evaluation unit 422 supplies the selected determined viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be imaged next.

The condition acquisition unit 43 acquires the conditions in the measurement and supplies the information to the viewpoint determination unit 42. In the third embodiment, the moving means information acquisition unit 433 acquires the information on the characteristics of the moving means, and supplies the acquired information on the characteristics of the moving means to the viewpoint determination unit 42. Information on the characteristics of the moving means acquired by the moving means information acquisition unit 433 can be obtained from, for example, a setting file of URDF (Unified Robot Description Form) or the like. In the third embodiment, the information on the characteristics of the moving means includes, for example, any one or more of the movable range, the moving accuracy, and the moving speed of the viewpoint setting unit 3.

[Measurement and viewpoint determination processing]
Hereinafter, measurement and viewpoint determination processing by the measurement system of the third embodiment will be described. In the third embodiment, the flow of processing can be described by using the same flowchart regardless of the characteristics of the moving means, which will be described later in each case. Further, since the overall flow of the process in the third embodiment is the same as the flow chart of FIG. 4, the description will be given with reference to FIG. Further, the processing in S2, S3, S4, and S6 in the third embodiment is the same as that in the first embodiment, and since the same processing is performed, only the other S1 and S5 will be described.

In S1, the initialization process is executed in the measurement system. In the initialization process, the imaging unit 2 is activated, the viewpoint setting unit 3 is activated, the characteristics of the moving means in the moving means information acquisition unit 433 of the condition acquisition unit 43 are acquired, the background model is read, and the imaging unit 2 is used as the initial viewpoint. The process of moving to is included. The initial viewpoint may be freely determined within the range in which the measurement target object 5 can be observed.
In S5, the viewpoint determining unit 42 should image the next image capturing unit 2 based on the data after data processing supplied from the data acquisition unit 41 and the characteristics of the moving means acquired by the moving means information acquisition unit 433. Determine the viewpoint of. More specifically, the candidate viewpoint generation unit 421 generates a candidate viewpoint, the generated candidate viewpoint is evaluated by the candidate viewpoint evaluation unit 422, and the candidate viewpoint with the highest evaluation among the conditions is selected as the determination viewpoint. .. The candidate viewpoint evaluation unit 422 supplies the selected determined viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be imaged next.

Next, with respect to the internal operation of the information processing apparatus 40, specific processing contents will be described in order regarding the operation when information on the characteristics of the moving means such as the movable range, the moving accuracy, and the moving speed is acquired.
[Characteristic 1: Movable range]
When the movable range of the viewpoint setting unit 3 is acquired by the moving means information acquisition unit 433 of the condition acquisition unit 43, it can be selected as the viewpoint when generating a three-dimensional model from the entire circumference of the measurement target object 5. The range is limited. The criterion for selecting the viewpoint to be imaged next in this measurement is to set the generated candidate viewpoint within the range in which the viewpoint setting unit 3 can move.
In S3, the data acquisition unit 41 generates polygon data as in the case of the three-dimensional shape measurement in the first embodiment. The data acquisition unit 41 updates the polygon data based on the distance images whose viewpoints are sequentially supplied.

In S5, polygon data is supplied from the data acquisition unit 41 to the viewpoint determination unit 42, and information indicating the movable range is supplied from the condition acquisition unit 43. The viewpoint determining unit 42 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the information indicating the movable range. The process of generating the candidate viewpoint by the candidate viewpoint generation unit 421 can be executed in the same manner as in the first embodiment. Further, a method of limiting the obtained candidate viewpoints within the movable range will be described with reference to FIG. As shown in FIG. 12, the candidate viewpoints 301a to 301c indicate the viewpoint position (position of the apex of the triangle opposite to the arrow) and the line-of-sight direction (direction of the arrow) of the candidate viewpoint generated by the candidate viewpoint generation unit 421. ing. The candidate viewpoint generation unit 421 uses the candidate viewpoint generated within the movable range 302 as it is as the candidate viewpoint, such as the candidate viewpoint 301a. For a candidate viewpoint outside the movable range 302 like the imaging unit 301b, the candidate viewpoint generation unit 421 can move like the candidate viewpoint 301b'when the candidate viewpoint is moved along the line-of-sight direction. If it falls within the range 302, the moved candidate viewpoint becomes a new candidate viewpoint. On the other hand, when the viewpoint is moved along the line-of-sight direction but does not fall within the movable range 302 as in the candidate viewpoint 301c, the candidate viewpoint generation unit 421 excludes the candidate viewpoint from the candidate viewpoint.

The candidate viewpoint evaluation unit 422 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 421 and selects the candidate viewpoint having the maximum evaluation value as the determination viewpoint. That is, the candidate viewpoint evaluation unit 422 applies an evaluation function corresponding to the measurement condition to the plurality of candidate viewpoints, and selects one or more viewpoints from the plurality of candidate viewpoints. Here, the evaluation value is calculated by the equation (1) as in the first embodiment. In this case, since the candidate viewpoint generated by the candidate viewpoint generation unit 421 is limited to the movable range of the viewpoint setting unit 3, processing such as determination of path planning is performed as compared with the case of the first embodiment. It is possible to quickly determine the viewpoint of the imaging unit 2 to be imaged next.

[Characteristic 2: Movement accuracy]
When the moving means information acquisition unit 433 acquires the accuracy (movement accuracy) of the reproducibility of the viewpoint position / posture when the viewpoint setting unit 3 is moved, the viewpoint position / posture assumed by the measurement system and the viewpoint position / posture are determined. It becomes known how much the difference from the actual viewpoint position and posture is. In general, the movement accuracy tends to deteriorate as the movement related to position and rotation increases. The criterion for selecting the viewpoint to be imaged next in this measurement is to prevent the movement accuracy of the viewpoint setting unit 3 from being deteriorated as much as possible when the movement is repeated.

In S3, the data acquisition unit 41 generates polygon data as in the case of the three-dimensional shape measurement in the first embodiment. The data acquisition unit 41 updates the polygon data based on the distance images whose viewpoints are sequentially supplied. Since an error occurs in the position and orientation of the viewpoint depending on the movement accuracy, the data acquisition unit 41 takes the movement accuracy into consideration when aligning the point cloud using an alignment method such as ICP. .. Specifically, the data acquisition unit 41 obtains the maximum deviation amount T _g of the obtained three-dimensional point cloud according to the movement accuracy, and sets T _g as the difference in distance when it is regarded as the corresponding point in ICP. Allow and align. By aligning according to the movement accuracy in this way, when the movement accuracy is high, it is possible to save the time for calculating unnecessary correspondence, and when the movement accuracy is low, the correspondence cannot be found and the position cannot be found. It is less likely that the alignment will fail.

In S5, polygon data is supplied from the data acquisition unit 41 to the viewpoint determination unit 42, and information indicating the movement accuracy is supplied from the condition acquisition unit 43. The viewpoint determining unit 42 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the information indicating the position accuracy. The process of generating the candidate viewpoint by the candidate viewpoint generation unit 421 can be executed in the same manner as in the first embodiment. Further, the candidate viewpoint generation unit 421 extrapolates the obtained candidate viewpoint when the movement accuracy is higher than the reference, and generates a candidate viewpoint when the movement is further performed. When the movement accuracy is lower than the reference, the candidate viewpoint generation unit 421 generates a candidate viewpoint when the movement displacement is reduced by interpolating the obtained candidate viewpoint. By adding such candidate viewpoints, it becomes easier to appropriately increase the number of solution candidates and seek a better viewpoint.

The candidate viewpoint evaluation unit 422 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 421 and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. Here, the evaluation value is calculated by the equation (1) as in the first embodiment. In this case, since the candidate viewpoint generated by the candidate viewpoint generation unit 321 is a high-quality viewpoint in consideration of the movement accuracy, the imaging unit 2 to be imaged next more efficiently than in the case of the first embodiment. You can ask for the viewpoint of.

[Characteristic 3: Movement speed]
The operation when the moving speed when moving the viewpoint setting unit 3 is acquired by the moving means information acquisition unit 433 will be described. When the moving speed is high, there is no particular problem even when moving a long distance, but when the moving speed is slow, if the moving speed is long, a lot of time is devoted to the moving time. Therefore, the criterion for selecting the viewpoint to be imaged next in this measurement is that the movement time does not exceed the reference value based on the movement speed of the viewpoint setting unit 3.
In S3, the data acquisition unit 41 generates polygon data in the same manner as in the case of the three-dimensional shape measurement in the first embodiment. The data acquisition unit 41 updates the polygon data based on the distance images whose viewpoints are sequentially supplied.

In S5, polygon data is supplied to the viewpoint determination unit 42 from the data acquisition unit 41, and the moving speed is supplied from the condition acquisition unit 43. The viewpoint determining unit 42 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the information indicating the moving speed. The process of generating the candidate viewpoint by the candidate viewpoint generation unit 421 can be executed in the same manner as in the first embodiment. Further, when the moving speed is faster than the reference, the candidate viewpoint generation unit 421 extrapolates the obtained candidate viewpoint and generates a candidate viewpoint when further moving. When the moving speed is slower than the reference, the candidate viewpoint generation unit 421 generates a candidate viewpoint when the movement displacement is reduced by interpolating the obtained candidate viewpoint. By adding such candidate viewpoints, the number of solution candidates can be increased appropriately, and it becomes easier to obtain a better viewpoint.

The candidate viewpoint evaluation unit 422 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 421 and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. Here, the candidate viewpoint evaluation unit 422 can first observe the number V (p) of "unknown" voxels that can be observed from the candidate viewpoint p using the voxels and the polygon data from the candidate viewpoint p as in the first embodiment. Find the number of unconnected edges E (p). Further, the candidate viewpoint evaluation unit 422 obtains the moving distance M (q, p) from the current viewpoint q to the candidate viewpoint p, and calculates the evaluation value Eval (p) by the equation (13).

However, v is the moving speed of the viewpoint setting unit 3, and _Tt is a threshold value representing an allowable range of moving time.
By evaluating the candidate viewpoint using the equation (13), the candidate viewpoint evaluation unit 422 has a voxel that has not been observed yet, a place where the polygon data is interrupted, or a hole as in the equation (1). The evaluation value of the candidate viewpoint that can observe the area becomes high. Further, when the movement time exceeds the threshold value, the evaluation value is greatly deducted, so that an appropriate viewpoint can be easily selected within the range where the movement time does not exceed the threshold value according to the movement speed. Similar to the case of the three-dimensional shape measurement of the first embodiment, the candidate viewpoint evaluation unit 422 determines the path planning in order from the viewpoint having the highest evaluation value, and the viewpoint of the imaging unit 2 in which the measurement target object 5 should be photographed next. To determine. The respective weight values w _V , w _E , and w _M may be fixed values based on experience, may be changed according to the number of measurements, or may be changed according to other conditions. ..

[Operation according to multiple characteristics]
Even when a plurality of characteristics of the moving means are acquired, it is possible to appropriately determine the viewpoint of the imaging unit 2 to image the measurement target object 5 next. For example, when two of the movable range and the moving speed are given, the candidate viewpoint generation unit 421 generates a candidate viewpoint group based on each of the cases already described, and further, the candidate viewpoint is based on the movable range. To limit. On this basis, the candidate viewpoint evaluation unit 422 evaluates the candidate viewpoint according to the equation (13), and sets the viewpoint of the imaging unit 2 to be imaged next. As a result, the viewpoint of the imaging unit 2 to be imaged next can be appropriately determined according to the plurality of characteristics of the moving means.
As described above, in the third embodiment, the viewpoint of the measuring device (imaging unit 2) to measure the object to be measured next can be appropriately determined based on the acquired information on the characteristics of the moving means. By moving the imaging unit 2 to the viewpoint determined in this way and repeating the imaging, appropriate measurement can be performed even if the moving means is replaced.

<Embodiment 4>
The measurement system of the fourth embodiment determines an appropriate viewpoint of the imaging unit 2 to measure the object to be measured next according to the information of the measurement instruction from the user. By measuring the object to be measured while sequentially moving the imaging unit to the determined viewpoint, the measurement is performed while reflecting the intention of the user. In the first to third embodiments, the viewpoint to be measured next is determined based on the application, the specification requirement, and the characteristics of the transportation means, respectively, but in the fourth embodiment, the measurement instruction of the area, direction, etc. directly specified by the user is determined. Based on the information in, determine the viewpoint at which the object to be measured should be measured next. Therefore, it is possible to more directly convey the user's intention and obtain a viewpoint. In the fourth embodiment, the use of three-dimensional shape measurement of the object to be measured is taken as an example, and as an example of the measurement instruction from the user, a region of interest on the object to be measured, a region not to be measured (a region that can be ignored), Although the three directions of attention will be described, the measurement instructions from the user to which the fourth embodiment can be applied are not limited to these.

[Device configuration]
FIG. 13 is a block diagram showing a configuration example of the measurement system according to the fourth embodiment. This measurement system includes an imaging unit 2, a viewpoint setting unit 3, and an information processing device 50 that controls the viewpoint setting unit 3 in response to information from the imaging unit 2. The measurement system in the fourth embodiment is a partially modified version of the configuration of FIG. 10 in the second embodiment, and the imaging unit 2 and the viewpoint setting unit 3 in FIG. 13 are the same as those in the second embodiment and have the same operation. I do.
The information processing device 50 according to the fourth embodiment includes a data acquisition unit 51 that acquires information from the image pickup unit 2, a viewpoint determination unit 52 that determines the viewpoint of the image pickup unit 2, and a condition acquisition unit 53 that acquires measurement conditions. ing. The data acquisition unit 51 executes data processing such as three-dimensional shape measurement. The viewpoint determination unit 52 determines the viewpoint of the imaging unit 2 to be imaged next from the data after data processing supplied from the data acquisition unit 51 and the measurement conditions supplied from the condition acquisition unit 53, and determines the viewpoint of the image pickup unit 2 to be imaged next. Supply to 3.
The viewpoint determination unit 52 includes a candidate viewpoint generation unit 521 that generates a candidate viewpoint and a candidate viewpoint evaluation unit 522 that evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 521. Further, the condition acquisition unit 53 includes a measurement instruction information acquisition unit 534 that acquires information on the measurement instruction from the user.

In the fourth embodiment, the operation inside the information processing apparatus 50 is changed according to the information of the measurement instruction from the user acquired by the measurement instruction information acquisition unit 534 of the condition acquisition unit 53. Therefore, first, only the outline of the operation inside the information processing apparatus 50 will be described, and then the details of the operation will be described for each measurement instruction from the user.
The data acquisition unit 51 receives a color image and a distance image from the image pickup unit 2 and executes data processing for three-dimensional shape measurement. Then, the data acquisition unit 51 supplies the data after the data processing to the viewpoint determination unit 52.

The viewpoint determination unit 52 next captures the measurement target object 5 according to the data after data processing supplied from the data acquisition unit 51 and the information of the measurement instruction from the user supplied from the condition acquisition unit 53. The viewpoint of the power imaging unit 2 is determined. Specifically, first, the candidate viewpoint generation unit 521 generates at least one candidate viewpoint according to the information of the measurement instruction from the user. The candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint generated by the supplementary viewpoint generation unit 521 according to the measurement instruction of the user, and selects the determination viewpoint. The candidate viewpoint evaluation unit 522 supplies the selected determined viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be imaged next.

The condition acquisition unit 53 acquires the measurement instruction information and sends the acquired measurement instruction information to the viewpoint determination unit 52. In the fourth embodiment, the measurement instruction information acquisition unit 534 acquires the measurement instruction from the user and supplies the acquired condition instruction to the viewpoint determination unit 52. The measurement instruction information acquired by the measurement instruction information acquisition unit 534 can be acquired, for example, by displaying a captured image of the measurement scene on the touch panel and inputting an area or direction instruction from the user. However, in the case of, for example, three-dimensional shape measurement, it is difficult to obtain a three-dimensional model of the object to be measured 5 or an image from the entire circumference before the measurement is completed. Therefore, when the user inputs a measurement instruction, the measurement target object 5 is represented not by the actual measurement target object 5 but by a simple model such as a rectangular parallelepiped or a cylinder, so that the user can easily select the area of interest or the like. It may be possible to specify. In the fourth embodiment, the information of the measurement instruction from the user includes any one or more of the attention region, the non-measurement target region, and the attention direction on the measurement target object.

[Measurement and viewpoint determination processing]
Hereinafter, measurement and viewpoint determination processing by the measurement system of the fourth embodiment will be described. In the fourth embodiment, the flow of processing can be described by using the same flowchart regardless of the measurement instruction from the user, which will be described later in each case. Further, since the flow of the entire process in the fourth embodiment is the same as the flow chart of FIG. 4, a description will be given with reference to FIG. The processing in S2, S3, S4, and S6 in the fourth embodiment is the same as that in the first embodiment, and since the same processing is performed, only the other S1 and S5 will be described.
In S1, the initialization process is executed in the measurement system. In the initialization process, the imaging unit 2 is activated, the viewpoint setting unit 3 is activated, the measurement instruction information acquisition unit 534 of the condition acquisition unit 53 acquires the measurement instruction from the user, the background model is read, and the imaging unit 2 is initially set. It includes processing to move to the viewpoint. The initial viewpoint may be freely determined within the range in which the measurement target object 5 can be observed.

In S5, the viewpoint determination unit 52 should next take an image based on the data after data processing supplied from the data acquisition unit 51 and the measurement instruction from the user obtained by the measurement instruction information acquisition unit 534. Determine the viewpoint of 2. More specifically, the candidate viewpoint generation unit 521 generates a candidate viewpoint, the generated candidate viewpoint is evaluated by the candidate viewpoint evaluation unit 522, and the viewpoint with the highest evaluation among the conditions is selected as the determination viewpoint. The candidate viewpoint evaluation unit 522 supplies the selected viewpoint to the viewpoint setting unit 3 as the viewpoint of the imaging unit 2 to be imaged next.
Although the explanation here is given as if the measurement instruction from the user is given only in S1, the present invention is not limited to this. It is assumed that the measurement instruction from the user can be specified at any timing in any step, and the candidate viewpoint is generated by reflecting the measurement instruction from the user in the next designated candidate viewpoint generation process.

Next, regarding the internal operation of the information processing device 50, the specific processing content of the operation when the information of the measurement instruction of each of the attention area, the non-measurement target area, and the attention direction on the measurement target object is obtained is described. It will be explained in order.
[Measurement instruction 1: Area of interest]
The operation when the measurement instruction information acquisition unit 534 of the condition acquisition unit 53 acquires the region of interest on the measurement target object 5 as a measurement instruction from the user will be described. Here, the area of interest is designated as a front and back surface. The criterion for selecting the next viewpoint to be imaged in this measurement is to observe the surface of the region of interest specified by the user as much as possible.
In S3, the data acquisition unit 51 generates polygon data in the same manner as in the case of the three-dimensional shape measurement in the first embodiment. The data acquisition unit 51 updates the polygon data based on the distance images whose viewpoints are sequentially supplied.

The end of the measurement may be determined by the end determination unit (not shown) receiving polygon data from the data acquisition unit 51 and confirming whether or not there is an interrupted portion or hole or a user-specified region of interest. The end determination unit ends the measurement when there is no break, a hole, or a user-specified region of interest.
In S5, polygon data is supplied to the viewpoint determination unit 52 from the data acquisition unit 51, and the region of interest is supplied from the condition acquisition unit 53. The viewpoint determining unit 52 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the region of interest. The process of generating the candidate viewpoint by the candidate viewpoint generation unit 521 can be executed in the same manner as in the first embodiment. Further, the candidate viewpoint generation unit 521 generates a candidate viewpoint capable of measuring a region of interest designated by the user.

The candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 521, and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. That is, the candidate viewpoint evaluation unit 522 applies an evaluation function corresponding to the measurement condition to the plurality of candidate viewpoints, and selects one or more viewpoints from the plurality of candidate viewpoints. Here, the candidate viewpoint evaluation unit 522 can first observe the number V (p) of “unknown” voxels that can be observed from the candidate viewpoint p using the voxels and the polygon data from the candidate viewpoint p as in the first embodiment. Find the number E (p) of unconnected edges. Further, the candidate viewpoint evaluation unit 522 obtains the area S (p) of the user-specified region of interest that can be observed from the candidate viewpoint p, and calculates the evaluation value by the equation (14).
Eval (p) = w _V V (p) + w _E E (p) + w _S S (p) (14)

When the candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint using the equation (14), there are voxels that have not been observed yet, and places or holes where the polygon data is interrupted, as in the equation (1). The evaluation value of the candidate viewpoint that can observe the area becomes high. Further, when a user-specified region of interest exists, it becomes easy to select a viewpoint capable of observing the surface of the region of interest. Similar to the case of the three-dimensional shape measurement of the first embodiment, the candidate viewpoint evaluation unit 522 determines the path planning in order from the viewpoint having the highest evaluation value, and the viewpoint of the imaging unit 2 to image the measurement target object 5 next. To determine. The respective weight values w _V , w _E , and w _S may be fixed values based on experience, may be changed according to the number of measurements, or may be changed according to other conditions.

[Measurement instruction 2: Area not subject to measurement]
The operation when the measurement instruction information acquisition unit 534 of the condition acquisition unit 53 acquires the non-measurement target area as a measurement instruction from the user will be described. Here, it is assumed that the non-measurement target area is designated as a front and back surface. The criterion for selecting the next viewpoint to be imaged in this measurement is to exclude the surface of the non-measurement target region specified by the user from the evaluation value calculation.
In S3, the data acquisition unit 51 generates polygon data in the same manner as in the case of the three-dimensional shape measurement in the first embodiment. The data acquisition unit 51 updates the polygon data based on the distance images whose viewpoints are sequentially supplied.

The end of the measurement may be determined by the end determination unit (not shown) receiving polygon data from the data acquisition unit 51 and confirming the interrupted portion or hole and the non-measurement target area designated by the user. If there is no break or hole outside the measurement target area, the end determination unit ends the measurement.
In S5, polygon data is supplied from the data acquisition unit 51 to the viewpoint determination unit 52, and a non-measurement target area is supplied from the condition acquisition unit 53. The viewpoint determining unit 52 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the non-measurement target area. The process of generating the candidate viewpoint by the candidate viewpoint generation unit 521 can be executed in the same manner as in the first embodiment.

The candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 521, and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. Here, as in the first embodiment, the candidate viewpoint evaluation unit 522 obtains the number V (p) of “unknown” voxels observable from the candidate viewpoint p using voxels, and puts them in the non-measurement target region. Find the number V'(p) that is not included. Similarly for polygon data, the candidate viewpoint evaluation unit 522 obtains the number E (p) of unconnected edges that can be observed from the candidate viewpoint p, and among them, the number E'(p) that is not included in the non-measurement target region. Ask for. The candidate viewpoint evaluation unit 522 calculates the evaluation value by the equation (15) using the obtained number V'(p) of voxels and the number E'(p) of unconnected edges.
Eval (p) = w _V V'(p) + w _E E'(p) (15)

When the candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint using the equation (15), there are voxels that have not been observed yet, and places or holes where the polygon data is interrupted, as in the equation (1). The evaluation value of the candidate viewpoint that can observe the area becomes high. Furthermore, if there is a non-measurement target area specified by the user, the part that is interrupted or has a hole in the non-measurement target area is excluded from the evaluation, and the area other than the non-measurement target area is included. It becomes easier to select the viewpoint determined based on. Similar to the case of the three-dimensional shape measurement of the first embodiment, the candidate viewpoint evaluation unit 522 determines the path planning in order from the viewpoint having the highest evaluation value, and the viewpoint of the imaging unit 2 to image the measurement target object 5 next. To determine. The respective weight values w _V and w _E may be fixed values based on experience, may be changed according to the number of measurements, or may be changed according to other conditions.

[Measurement instruction 3: Direction of attention]
The operation when the measurement instruction information acquisition unit 534 of the condition acquisition unit 53 acquires the direction in which the measurement target object 5 should be noted as a measurement instruction from the user will be described. Here, the direction of interest is specified as a space vector. The criterion for selecting the next viewpoint to be imaged in this measurement is that the viewpoint is in the same direction as the attention direction specified by the user.
In S3, the data acquisition unit 51 generates polygon data in the same manner as in the case of the three-dimensional shape measurement in the first embodiment. The data acquisition unit 51 updates the polygon data based on the distance images whose viewpoints are sequentially supplied.

The end of the measurement may be determined by the end determination unit (not shown) receiving polygon data from the data acquisition unit 51 and confirming whether there is a break, a hole, or a user-specified attention direction. The end determination unit ends the measurement when there is no interruption, a hole, or a user-specified direction of attention.
In S5, polygon data is supplied to the viewpoint determination unit 52 from the data acquisition unit 51, and the attention direction is supplied from the condition acquisition unit 53. The viewpoint determining unit 52 determines the viewpoint of the imaging unit 2 to be imaged next from the polygon data and the direction of attention. The process of generating the candidate viewpoint by the candidate viewpoint generation unit 521 can be executed in the same manner as in the first embodiment. Further, the candidate viewpoint generation unit 521 generates a candidate viewpoint having the same line-of-sight direction as the attention direction specified by the user.
The candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint generated by the candidate viewpoint generation unit 521, and selects a candidate viewpoint having a high evaluation value and satisfying the condition as a determination viewpoint. Here, the candidate viewpoint evaluation unit 522 first observes the number V (p) of “unknown” voxels observable from the candidate viewpoint p using the voxels and the polygon data from the candidate viewpoint p, as in the first embodiment. Find the number of possible unconnected edges E (p). Further, the candidate viewpoint evaluation unit 522 calculates the evaluation value using the equation (16).

When the candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint using the equation (16), there are voxels that have not been observed yet, and places or holes where the polygon data is interrupted, as in the equation (1). The evaluation value of the candidate viewpoint that can observe the area becomes high. Further, when a user-specified attention direction exists, it becomes easy to select a viewpoint in the same line-of-sight direction as the attention direction. Similar to the case of the three-dimensional shape measurement of the first embodiment, the candidate viewpoint evaluation unit 522 determines the path planning in order from the viewpoint having the highest evaluation value, and the viewpoint of the imaging unit 2 to image the measurement target object 5 next. To determine. The respective weight values w _V , w _E , and w _d may be fixed values based on experience, may be changed according to the number of measurements, or may be changed according to other conditions.

[Operation according to multiple conditional instructions]
Even when there are a plurality of measurement instructions from the user, it is possible to appropriately determine the viewpoint of the imaging unit 2 to image the measurement target object 5 next. For example, when two regions of interest and a region not to be measured are given, the candidate viewpoint generation unit 521 generates a candidate viewpoint group based on each of the cases already described. Then, the candidate viewpoint evaluation unit 522 evaluates the candidate viewpoint by the formula (17) which is a combination of the formula (14) and the formula (15), and sets the candidate viewpoint having a high evaluation value as the viewpoint to be imaged next. As a result, the viewpoint of the imaging unit 2 to be imaged next can be appropriately determined in response to the plurality of measurement instructions.
Eval (p) = w _V V'(p) + w _E E'(p) + w _S S'(p) (17)
However, S'(p) represents an observable area on the surface of a region of interest specified by the user from the candidate viewpoint p that is not included in the region not included in the measurement target.

In the fourth embodiment, it is possible to appropriately determine the viewpoint of the measuring device (imaging unit 2) that should next image the object to be measured according to the information of the measurement instruction from the user. By moving the imaging unit 2 to the viewpoint obtained in this way and repeating imaging, it is possible to perform measurement reflecting the intention of the user.

<Modification example>
[When the condition acquisition unit has multiple acquisition units]
In each of the first to fourth embodiments, the case where the condition acquisition unit includes the acquisition unit for acquiring one measurement condition has been described. However, the number of acquisition units provided in the condition acquisition unit is not limited to one, and includes a plurality of the application acquisition unit, the spec request acquisition unit, the transportation means information acquisition unit, and the measurement instruction information acquisition unit. May be good. In that case as well, the process can be performed according to the same flowchart as in FIG. 4 in the first embodiment. As for the processing inside the information processing apparatus, the processing described in the first to fourth embodiments may be combined.

For example, as shown in FIG. 14, a configuration will be described in which the condition acquisition unit 63 includes four units: an application acquisition unit 631, a spec request acquisition unit 632, a moving means information acquisition unit 633, and a measurement instruction information acquisition unit 634. As measurement conditions, for example, it is assumed that three-dimensional shape measurement is given as an application, accuracy is given as a priority spec requirement, movable range and moving speed are given as characteristics of a moving means, and a direction of attention is given as a measurement instruction from a user. In this case, the candidate viewpoint generation unit 621 generates a candidate viewpoint according to each measurement condition item, and the candidate viewpoint evaluation unit 622 selects only the candidate viewpoints that satisfy the conditions according to the movable range, and among the selected candidate viewpoints. , The candidate viewpoint is evaluated by the formula (18) that combines the evaluation values.

In this way, by combining the acquisition units that acquire a plurality of measurement conditions, it is possible to appropriately determine the viewpoint of the measuring device to measure the object to be measured next according to various measurement conditions. As a result, it is possible to perform measurement in response to a complicated request from the user regarding measurement.

[When changing the conditions of the condition acquisition section]
In each of the first to fourth embodiments, the operation of determining the viewpoint by using the same evaluation formula until the measurement is completed according to the measurement condition first acquired by the condition acquisition unit has been described. However, the condition acquisition unit may acquire the measurement conditions in the middle of the measurement and perform the measurement while changing as appropriate. For example, in the use of three-dimensional measurement, measurement is first performed with speed-priority spec requirements as measurement conditions, and when a certain period of time elapses or the measurement reaches a certain stage, complexity-priority spec requirements and condition instructions from the user The measurement conditions may be changed based on the region of interest. By performing the measurement while changing the conditions on the way in this way, it is possible to perform the measurement while more flexibly responding to the user's request.

[When moving the object to be measured]
In each of the first to fourth embodiments, an example in which the image pickup unit 2 is attached to the viewpoint setting unit 3 and the viewpoint of the image pickup unit 2 is changed by the viewpoint setting unit 3 has been described. However, the movement of the viewpoint setting unit is not limited to the imaging unit, and the viewpoint setting unit may move the object to be measured. When moving the measurement target object, for example, using a robot hand as a viewpoint setting unit, the robot hand grasps the measurement target object and changes its posture in front of the imaging unit to display the measurement target object at a plurality of viewpoints. It can be measured from. By fixing the imaging unit and moving the object to be measured using the viewpoint setting unit, it is possible to measure from multiple viewpoints even if the imaging unit is huge or heavy. In addition, if both the imaging unit and the object to be measured are moved by the viewpoint setting unit consisting of two different robot hands, the target viewpoint can be moved to the target viewpoint with half the moving distance as compared with the case where only one is moved. The time to complete can be shortened.

[When the viewpoint setting unit guides people]
In each of the first to fourth embodiments, the case where the viewpoint setting unit is a movement-controllable machine such as a robot arm has been described. However, the viewpoint setting unit is not limited to a machine capable of controlling movement, and may be used as an alternative means by guiding a person. When the viewpoint setting unit guides a person, the viewpoint setting unit may guide the movement of the person by displaying the direction and distance of movement on a display or the like. The method of inducing the movement of a person is not limited to the display. For example, the correct movement may be indicated by the pitch, loudness, rhythm, etc. of the sound, or it may be guided by vibration or stimulation.

[Parameter sharing using the communication unit]
In each of the first to fourth embodiments, an example in which the image pickup unit 2 and the viewpoint setting unit 3 are connected to the information processing device has been described, but as shown in FIG. 15, the information processing device 70 is further connected to the communication unit 4. May be connected. The communication unit 4 has a function of sharing with the outside the weight parameters used in the calculation formula of the evaluation value when the measurement is performed for various measurement conditions and the object to be measured, and the data regarding the quality of the measurement result. By providing a large number of this measurement system and collecting the weight parameters and measurement results when measuring various measurement conditions and objects to be measured on a server connected via a network, the relationship between the parameters and the measurement results can be obtained. .. By using this relationship, the user can adjust the weight parameter by acquiring the most effective weight parameter from the server or the like under the measurement conditions selected by the user for the object to be measured for the first time by the user. Effective measurement can be performed without explicitly performing.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

1 ... Information processing device, 2 ... Imaging unit, 3 ... Viewpoint setting unit, 5 ... Measurement target object, 11 ... Data acquisition unit, 12 ... Viewpoint determination unit, 13 ... Condition acquisition unit

Claims (14)

A first acquisition means for acquiring measurement data obtained by measuring the target object by a measurement means for measuring the target object from a plurality of viewpoints,
A second acquisition means for acquiring at least one measurement condition among a plurality of measurement conditions when the measurement means measures the target object, and
Based on the measurement data acquired by the first acquisition means and the measurement conditions acquired by the second acquisition means, a plurality of candidate viewpoints of the measurement means to be measured next are generated. A determination means for determining one or more viewpoints from a plurality of candidate viewpoints by applying an evaluation function corresponding to the measurement condition acquired by the second acquisition means to the plurality of candidate viewpoints.
An information processing device characterized by being equipped with. The second acquisition means acquires the information of the use for measuring the target object by the measurement means, and the determination means obtains the information of the use acquired by the second acquisition means, and then the next The information processing apparatus according to claim 1, wherein the viewpoint of the measuring means to be measured is determined. The information of the application acquired by the second acquisition means includes any one or more of three-dimensional shape measurement, defect inspection, specific object recognition, and person monitoring for the target object. Item 2. The information processing apparatus according to item 2. The second acquisition means acquires information on specifications that should be prioritized in the measurement of the target object by the measurement means.
Any one of claims 1 to 3, wherein the determination means determines the viewpoint of the measurement means to be measured next based on the information of the specifications acquired by the second acquisition means. The information processing device described in the section. The information processing apparatus according to claim 4, wherein the information of the specifications includes any one or more of accuracy, speed, and complexity in the measurement of the target object. The second acquisition means acquires information on the characteristics of the moving means that moves the measuring means to the viewpoint for measuring the target object.
The determination means according to claims 1 to 5, wherein the determination means determines the viewpoint of the measurement means to be measured next based on the information on the characteristics of the moving means acquired by the second acquisition means. The information processing apparatus according to any one item. The information processing apparatus according to claim 6, wherein the information on the characteristics of the moving means includes any one or more of the movable range, the moving accuracy, and the moving speed of the measuring means. The second acquisition means acquires information on measurement instructions from the user and obtains information.
Any of claims 1 to 7, wherein the determination means determines the viewpoint of the measurement means to be measured next based on the information of the measurement instruction acquired by the second acquisition means. The information processing apparatus according to item 1. The information processing apparatus according to claim 8, wherein the information of the measurement instruction includes any one or more of a region of interest, a region not to be measured, and a direction of interest on the target object. The setting means for setting the position and orientation of the measuring means based on the viewpoint of the measuring means is further provided with an output means for outputting the viewpoint of the measuring means to be measured next, which is determined by the determining means.
The acquisition of the measurement data by the first acquisition means, the determination of the viewpoint of the measurement means to be measured next by the determination means, and the output of the viewpoint of the measurement means to be measured next by the output means. The information processing apparatus according to any one of claims 1 to 9, wherein the information processing apparatus is repeated. The information processing apparatus according to claim 10, further comprising a determination means for determining an end condition for determining the viewpoint of the measurement means to be measured next by the determination means based on the measurement data. A measuring means that measures a target object from multiple viewpoints and outputs measurement data,
The information processing apparatus according to any one of claims 1 to 11, which processes the measurement data output from the measurement means.
A setting means for setting the position and orientation of the measuring means based on one or more viewpoints of the measuring means to be measured next, which is determined by the determining means of the information processing device.
A measurement system characterized by being equipped with. A step of acquiring measurement data obtained by measuring the target object by a measuring means for measuring the target object from a plurality of viewpoints, and
A step of acquiring at least one measurement condition among a plurality of measurement conditions when the measurement means measures the target object, and
Based on the acquired measurement data and the acquired measurement conditions , a plurality of candidate viewpoints of the measurement means to be measured next are generated, and an evaluation function corresponding to the measurement conditions is applied to the plurality of candidate viewpoints. Then, the step of determining one or more viewpoints from multiple candidate viewpoints,
An information processing method characterized by being provided with. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 12.

Images