JP7236705B2 - Method of generating teacher data used for learning transport equipment - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a conveying apparatus for conveying pallets.

A transportation device such as a forklift transports a pallet having a predetermined shape. The pallet is provided with holes (fork pockets), and the transport device is provided with forks that can be inserted into the fork pockets.

An operator of the carrier device must adjust the height of the forks, operate the carrier device while performing a steering operation, and insert the forks into the fork pockets.

However, it is not easy for an inexperienced operator to successfully insert the fork into the fork pocket. If the forks collide with anything other than the fork pockets, the load on the pallet may collapse. Also, even if the forks are inserted into the fork pockets, if the forks collide with the inner walls of the fork pockets, the pallet may shift or unfavorable stress may be applied to the forks.

JP 2017-151650 A

The present inventors considered mounting a detection device capable of detecting a specific portion of a pallet on a forklift. The detection device includes a sensor that acquires an image, and a discriminator (also referred to as a classifier) configured to detect a specific location based on the output image of the sensor. The discriminator is constructed by machine learning based on teacher data or learning data obtained in advance. Teacher data is typically image data including a specific part. In order for the discriminator to acquire sufficient discriminating power, a huge amount of training data taken in different shooting directions and shooting distances is required.

If there are many types of pallets to be transported by the forklift, it is necessary to prepare training data for each pallet, so the cost required for learning jumps up. In particular, for various types of pallets, if a serviceman or a user (hereinafter referred to as a human) takes many images of pallets in a forklift environment to generate training data, it requires a lot of human labor, This means that the cost of learning will be very high.

The present invention has been made in view of such problems, and one exemplary purpose of certain aspects thereof is to provide a technique capable of reducing the effort and time required for learning.

One aspect of the present invention relates to a method of generating teacher data used for learning a conveying device that conveys pallets. The transport device can acquire a range image sensor, (i) a range image in which pixel values represent distance based on the output of the range image sensor, and (ii) based on the range image, a pallet or a specific part of the pallet. and a computing unit having a discriminator configured by machine learning using teacher data so that a certain detection target can be specified. The method of generating teacher data includes the following processes.
Step A) Generate a three-dimensional model of the pallet.
Step B) A 3D model of the pallet is placed in a 3D virtual space, and the pallet is measured by the range image sensor while changing the relative positions of the 3D model of the pallet and the model of the range image sensor. generates a virtual range image that would be
Step C) Generate training data based on the virtual distance image obtained in step B).

This transportation device maps 3D point cloud data obtained from distance image sensors such as TOF (Time Of Flight) sensors, LiDAR (Light Detection and Ranging), stereo cameras and structured light cameras into 2D distance images. By constructing a discriminator that takes a distance image as an input, it is possible to improve the accuracy of specifying the pallet or its part.

Here, since pallets come in a wide variety of shapes and materials, it is not easy to prepare training data to be used for classifier learning by photographing the actual pallet with a range image sensor, assuming all kinds of situations for all kinds of pallets. do not have. According to this aspect, by utilizing the three-dimensional model, teacher data can be efficiently generated, and learning can be made more efficient.

Step C may include superimposing the virtual range image on the background range image. By superimposing the virtual range image of the palette and the background range image to generate training data, it is possible to improve the recognition rate in an environment where the palette is actually used.

The background distance image may be generated by actual measurement of a distance image sensor. Sensing by the distance image sensor is greatly affected by ambient light, but for the background, by using an actually captured image, it is possible to generate teacher data that takes into account the effect of ambient light.

A three-dimensional model of a background object may be placed in a three-dimensional virtual space to generate a virtual range image of the background, and the virtual range image of the palette and the virtual range image of the background may be superimposed as training data.

Alternatively, a 3D model of the background object and a 3D model of the pallet may be placed in a 3D virtual space to generate a virtual range image including them, which may be used as teacher data.

An error model that describes the influence of at least one of the distance to the object to be measured, the direction of the surface to be measured, and the material of the object to be measured on the output of the range image sensor is generated, and in step B, the model of the range image sensor and the error model A virtual range image may be generated based on the combination of
The material of the object to be measured, the direction of the surface to be measured, and the distance to the object to be measured affect the distance image. Therefore, by generating a measurement model that introduces an error model, it is possible to generate teacher data that is closer to a range image that would be captured in an actual usage environment.

An environment model describing the influence of at least one of ambient light and ambient temperature on the output of the range image sensor is generated, and in step B, a virtual range image is generated based on the combination of the range image sensor model and the environment model. may be generated.
Range image sensor sensing can be greatly affected by the environment, including response speed, ambient light, and ambient temperature. Therefore, by generating a measurement model incorporating an environment model, it is possible to generate training data that is closer to a distance image that would be captured in an actual usage environment.

The object to be detected may include pallet holes and girders separating the holes. Pallets come in a variety of shapes and structures, and they all have holes and girders separating the holes. Therefore, it is possible to detect the positions of holes in various pallets by installing a discriminator that detects girders.

The calculation unit may specify the positions of the holes based on the geometric information of the pallet held in advance.

After specifying the pallet or the specific part of the pallet to be detected, the result may be used to specify the hole in the pallet based on the distance image. Constructing the classifier to identify pallet holes directly from the original range image may result in reduced accuracy. Therefore, by configuring a discriminator that identifies a pallet or a specific portion of the pallet, and further configuring a discriminator that identifies a hole based on the result, it is possible to improve the hole identification accuracy.

It should be noted that arbitrary combinations of the above-described constituent elements and mutually replacing the constituent elements and expressions of the present invention in methods, devices, systems, etc. are also effective as aspects of the present invention.

According to the present invention, the effort and time required for learning can be reduced.

1 is a perspective view showing an external view of a forklift that is one aspect of a conveying device; FIG. It is a figure which shows an example of the cockpit of a forklift. FIGS. 3A to 3H are diagrams showing pallets to be transported by a forklift. 1 is a functional block diagram of a forklift according to an embodiment; FIG. 5(a) and 5(b) are diagrams for explaining the installation locations of the range image sensor. It is a figure which shows an example of a distance image. FIGS. 7A and 7B are diagrams schematically showing specific parts to be detected. It is a figure which shows the distance image of the specific site|part of Fig.7 (a). 4 is a flow chart of generation of teacher data used for learning of a discriminator. FIG. 3 is a diagram showing a three-dimensional virtual space in which a model of a palette and a range image sensor are arranged; FIGS. 11A to 11C are diagrams for explaining the generation of teacher data based on the background distance image.

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described below based on preferred embodiments with reference to the drawings. The same or equivalent constituent elements, members, and processes shown in each drawing are denoted by the same reference numerals, and duplication of description will be omitted as appropriate. Moreover, the embodiments are illustrative rather than limiting the invention, and not all features and combinations thereof described in the embodiments are necessarily essential to the invention.

FIG. 1 is a perspective view showing an external view of a forklift, which is one mode of a transport device. The forklift 600 includes a vehicle body (chassis) 602 , a fork 604 , a lifting body (lift) 606 , a mast 608 and wheels 610 and 612 . A mast 608 is provided in front of the vehicle body 602 . Elevating body 606 is driven by a power source such as a hydraulic actuator (not shown in FIG. 1) to ascend and descend along mast 608 . A fork 604 is attached to the lifting body 606 for supporting a load. The forklift 600 carries a pallet with two forks 604 inserted into holes (fork pockets) of the pallet (not shown).

FIG. 2 is a diagram showing an example of a cockpit 700 of a forklift. The cockpit 700 includes an ignition switch 702 , a steering wheel 704 , a lift lever 706 , an accelerator pedal 708 , a brake pedal 710 , a dashboard 714 and a forward/reverse lever 712 .

The ignition switch 702 is a switch for starting the forklift 600 . A steering wheel 704 is an operation means for steering the forklift 600 . A lift lever 706 is an operating means for moving the lifting body 606 up and down. The accelerator pedal 708 is an operating means for controlling the rotation of wheels for traveling, and the travel of the forklift 600 is controlled by the operator adjusting the amount of depression. When the operator depresses the brake pedal 710, the brake is applied. The forward/reverse lever 712 is a lever for switching the traveling direction of the forklift 600 between forward and reverse. In addition, an inching pedal (not shown) may be provided.

The operator needs to operate the steering wheel 704 , the accelerator pedal 708 and the brake pedal 710 to control the position of the vehicle body, and operate the lift lever 706 to control the position of the fork 604 .

3A to 3H are diagrams showing pallets to be transported by the forklift 600. FIG. Pallet 800 has a top plate 806 , holes 802 and girders 804 . The shape of the pallet 800 is defined by JIS standards, and there are various shapes depending on the purpose and application. 3(a) to (h) are represented by JIS standards as S2, SU2, DP4, R4, RW2, D2, DU2 and D4, respectively. As shown in FIGS. 3(a) to 3(h), pallets generally come in various shapes and structures, and for unfamiliar operators, a plurality of operation input means (704, 706, 708, 710, 706) to accurately insert the fork into the fork pocket.

A configuration capable of assisting the scooping operation of a pallet will be described below. FIG. 4 is a functional block diagram of the forklift 300 according to the embodiment. FIG. 4 shows only the blocks related to assisting the scooping operation, and omits the other blocks.

The forklift 300 includes a distance image sensor 302 , a camera 303 , notification means 304 and a controller 310 . A distance image sensor 302 is attached in front of the forklift 300 to acquire distance information in front of the vehicle. As the range image sensor 302, although not limited thereto, a ToF camera, LiDAR, a sensor using a structured light method, a stereo camera, or the like can be used. The distance image sensor 302 generates distance measurement data D1 including three-dimensional point cloud data of structures, articles, etc. located in front of the vehicle.

5(a) and 5(b) are diagrams for explaining the installation location of the distance image sensor 302. FIG. FIG. 5(a) is a top view. Since it is necessary to detect the position of the hole into which the fork should be inserted, the distance image sensor 302 is preferably placed in the center of the two forks in the horizontal direction. As a result, when the fork is inserted into the pallet, the distance image sensor 302 can photograph the pallet from the front. In particular, when a specific portion including a girder is to be detected as described later, the distance image of the specific portion can be successfully captured by arranging the distance image sensor 302 in the center of the fork.

FIG. 5B is a side view. With respect to the vertical direction, it may be at the same height as the fork 604 as indicated by 302c, but it is preferable to provide it at a slightly upper position 302a (or a lower position 302b). This is because the upper surface S1 of the upper plate 806 of the pallet 800 (or the lower surface S2 of the upper plate 806) can thereby be reliably captured. This position also has the advantage of being less obscured by the fork or other vehicle structure during the final adjustment phase of the fork height.

Return to FIG. A camera 303 photographs the front of the forklift 300 . Image data D3 captured by camera 303 is displayed on display 306 . Although the camera 303 and the range image sensor 302 are shown as separate functional blocks in FIG. 4, this does not necessarily mean that they are independent of the range image sensor 302 as hardware. 303 may also serve. For example, if the range image sensor 302 is a TOF camera or a stereo camera, a monocular image (infrared intensity distribution, visible light monochrome image, visible light color image) primarily obtained therefrom can be used as the image data D3.

Controller 310 comprehensively controls forklift 300 . The controller 310 includes a distance image acquisition section 330 and a discriminator 340 . The distance image acquisition unit 330 generates a distance image D2 based on the distance measurement data D1 including point cloud data. The distance image D2 is image data having a plurality of pixels vertically and horizontally, and is data in which pixel values represent distances. FIG. 6 is a diagram showing an example of a distance image. Note that when the output of the distance image sensor 302 is distance data, the processing of the distance image acquisition unit 330 is simplified or omitted.

Return to FIG. The classifier 340 is configured by machine learning so as to identify the pallet or a specific portion of the pallet based on the distance image D2.

The processing unit 350 identifies the positions of the holes in the pallet based on the detection result of the discriminator 340 . Based on the detected result, the notification means 304 is controlled to notify the operator. Notification means 304 may include display 306 and speaker 308 . A display 306 is used to visually notify the operator of pallet hole locations and other information. A speaker 308 can be used to audibly notify the operator of pallet locations and other information. Notification by notification means 304 will be described later.

The discriminator 340 and the processing unit 350 can be configured by a microcomputer 320 having an arithmetic processing unit (CPU) and memory, and a software program executed by the microcomputer 320 . Furthermore, the distance image acquisition section 330 can also be implemented as a function of the microcomputer 320 . Therefore, the distance image acquisition unit 330, the discriminator 340, and the processing unit 350 are not necessarily recognized as separate hardware, but represent functions realized by a combination of an arithmetic processing unit and software programs.

Alternatively, the distance image acquisition unit 330 and the discriminator 340 may be configured with separate hardware. For example, as an interface between the distance image sensor 302 and the microcomputer 320, hardware (chips) such as FPGA (Field Programmable Gate Array) and ASIC (Application Specific Integrated Circuit) may be installed. In this case, the processing of the distance image acquisition unit 330 may be executed by this hardware.

The above is the configuration of the forklift 300 . According to this forklift 300, the accuracy of specifying the pallet or its part can be improved by configuring the discriminator that receives the distance image obtained by the distance image sensor by machine learning. The operator's operation can be assisted by reflecting the obtained detection result in the operation of the conveying apparatus or notifying the operator.

As another method for detecting a pallet and its specific parts, a method of detecting characteristic parts by pattern matching using three-dimensional point group data is also conceivable. However, this method has the problem that the object must always be measured from the same direction, and it is difficult to measure three-dimensional point cloud data of characteristic parts with high density and accuracy.

Next, a specific example of the discriminator 340 and its learning will be described. In one embodiment, the discriminator 340 is composed of a combination of linear SVM (Support Vector Machine) and HOG (Histograms of Oriented Gradients) features. Linear SVMs are widely used as classifiers that can determine whether an input belongs to one of two classes. In the field of general image processing, linear SVM inputs are image data whose pixel values are brightness, but in the present embodiment, the input is a range image.

A linear SVM can also be trained to detect pairs of two holes directly. However, as shown in FIGS. 3(a) to 3(h), the shape and size of the holes vary depending on the type of pallet. Specifically, the width of the hole differs depending on the pallet. Also, some pallets have bottom plates, some do not, so the holes may be open or closed at the bottom. For this reason, a classifier that learns directly from holes as objects to be identified may not obtain a sufficiently high classification rate.

Therefore, in order to further improve the identification rate, the present inventors focused on the holes 802 of the pallet 800 and the girders 804 that separate the holes 802, and configured the discriminator 340 with a part (specific part) including the girders 804 as the detection target. It was decided to. The girder 804 is an essential component of the pallet regardless of the pallet 800 standard, is suitable as a detection target, and can be expected to have a high identification rate.

FIGS. 7A and 7B are diagrams schematically showing a specific site 801 to be detected. A specific portion 801 includes a portion of the upper plate 806 , a portion of the lower plate 808 , a girder 804 and two holes 802 . This specific portion 801 can be grasped as including two facing U-shapes. FIG. 8 is a diagram showing a distance image of the specific portion 801 in FIG. 7(a).

As shown in FIG. 3(b), etc., some pallets do not have a lower plate. Therefore, as shown in FIG. 7(b), the classifier 340 may be learned by using the T-shape excluding the lower plate 808 as the specific portion 801. FIG.

The processing unit 350 holds geometric information of the pallet. The processing unit 350 can identify the positions of the two holes into which the forks should be inserted by arithmetic processing based on the position of the specific portion 801 identified by the discriminator 340 and the geometric information. For example, the geometric information may include the relative positional relationship between the position (coordinates) of the center of the pallet and the positions of the holes. The processing unit 350 can detect the center position of the pallet 800 based on the output of the discriminator 340, and set a position offset to the left and right by a predetermined amount from the center position as the position of the hole into which the fork should be inserted.

FIG. 7(c) is a diagram showing a portion of another palette. A pallet may have three (or more) holes, in which case one pallet contains two (or more) girders and two girders per pallet. It is also possible that the specific parts 801 are detected adjacent to each other. In this case, the center of the two specific parts 801 (that is, the center of the two girders 804) may be determined as the center position of the entire pallet 800. FIG.

(Learning of discriminator)
Next, learning of the discriminator 340 will be described.

FIG. 9 is a flow chart for generating teacher data used for learning of the discriminator 340 . First, a three-dimensional model Mp of the palette 800 is generated (S100). The three-dimensional model Mp of the pallet corresponds to three-dimensional CAD data of the pallet.

A model Ms of the distance image sensor 302 is also generated (S102). The model of the distance image sensor 302 describes the angle of view and input/output characteristics of the distance image sensor 302, and can be generated in consideration of the angle of view, resolution, measurement accuracy, temperature, frame rate, response characteristics, and the like. can. Response characteristics may include wavelength sensitivity characteristics and transient response characteristics.

The three-dimensional model Mp of the pallet 800 and the model Ms of the range image sensor 302 are arranged in the three-dimensional virtual space of the computer simulator (S104). In this state, the output of the model Ms of the distance image sensor 302 is obtained as the virtual distance image I1 (S106), and the training data I2 is generated based on the virtual distance image I1 (S108).

The processes S106 to S108 are repeated while changing the relative positional relationship between the pallet and the range image sensor models Mp and Ms (S110).

FIG. 10 is a diagram showing a three-dimensional virtual space 900 in which the palette and the models Mp and Ms of the range image sensor are arranged. Here, it is assumed that the model Ms of the distance image sensor 302 is arranged at the origin. The dashed line represents the field of view of range image sensor 302 . At step S110 in FIG. 8, the position (x, y, z) of the model Mp of the palette 800 is changed. At step S110, the orientation (θp, θy, θr) may be changed in addition to the position (x, y, z).

The above is the method for generating teacher data according to the embodiment.

Since pallets come in a wide variety of shapes and materials, it is not easy to prepare training data for classifier learning by photographing the actual pallet with a distance image sensor, assuming all kinds of situations for all kinds of pallets. According to this aspect, by utilizing the three-dimensional model, teacher data can be efficiently generated, and learning can be made more efficient.

(adding background information)
In the process S106 of FIG. 8, the background distance image I3 can be superimposed on the virtual distance image I1 to generate the teacher data I2.

FIGS. 11A to 11C are diagrams for explaining the generation of teacher data based on the background distance image. FIG. 11(a) is a virtual distance image I1 of the palette 800 obtained in FIG. FIG. 11(b) represents the background distance image I3. As the background distance image I3, it is preferable to use an actual background distance image captured by the actual distance image sensor 302 in the environment where the palette is actually used.

Combining the palette virtual distance image I1 of FIG. 11(a) and the background distance image I3 of FIG. 11(b) yields the synthetic distance image of FIG. 11(c). Part or all of this synthesized range image can be used as training data.

By superimposing the virtual distance image I1 of the palette and the background distance image I3 to generate the training data, the recognition rate in the environment where the palette is actually used can be improved.

Note that the background distance image I3 may be generated by actual measurement of the distance image sensor 302 . Sensing by the distance image sensor 302 is greatly affected by environmental light, etc., but by using an actual photographed image for the background, it is possible to generate teacher data that takes into account the influence of environmental light, which has the effect of improving the recognition rate. There is expected.

Alternatively, a three-dimensional model of an object that serves as a background is placed in a three-dimensional virtual space, and a virtual range image of the background is generated using the model Ms of the range image sensor 302. A virtual distance image may be superimposed as training data.

Alternatively, a 3D model of the background object and a 3D model of the pallet may be arranged in a 3D virtual space to generate a virtual range image including them, and part or all of it may be used as teacher data. .

(error model)
In the description so far, the measurement model considering only the model Ms of the range image sensor 302 has been described, but in order to generate more accurate teacher data, it is preferable to introduce the error model M _ERR .

The error model M _ERR describes the influence of at least one of (i) the distance to the measurement object, (ii) the direction of the measurement plane, and (iii) the material of the measurement object on the output of the range image sensor. Since the distance to the object to be measured, the direction of the surface to be measured, and the material of the object to be measured have the greatest influence on the output of the range image sensor, the error model M _ERR should be created with at least the distance to the object to be measured taken into account. , and more preferably, both the distance to the measurement object and the direction of the measurement surface should be taken into consideration when making the calculation. If it is desired to further improve the accuracy, all three should be taken into consideration when making the calculation.

(i) Distance to measurement object Consider the input/output characteristics of the range image sensor. In an ideal distance image sensor, the pixel value pv of the output distance image changes linearly with respect to the input distance d.
pv=a ₁ ×d (1)

However, in an actual distance image sensor, the inclination _a1 may vary depending on the sensor system and product type. Alternatively, as shown in equation (2), it may include a nonlinear term or an offset term _a0 .
pv= _a0 + _a1 *d+ _a2 * ^d2 + (2)
Therefore, the error model M _ERR describes the relationship between the distance d between the distance image sensor and the measurement target and the pixel value pv in the distance image.

(ii) Orientation of Measurement Plane The input/output characteristics represented by Equation (2) may change according to the orientation of the measurement plane. For example, for an object in front of the range image sensor 302, an input-output characteristic having a set of coefficients S1[a ₀ , a ₁ , . . . ] is obtained. In this case, if the input-output characteristics are not necessarily described by the same set S1 for different directions, and input-output characteristics with different sets of coefficients S2[a ₀ , a ₁ , . . . ] are obtained There is Therefore, the error model M _ERR should be created in consideration of the direction of the measurement surface.

(iii) Materials Depending on the sensing method of the range image sensor 302, the wavelength used, etc., even if the same range image sensor 302 measures an object at the same distance, if the material of the object is different, the output of the range image sensor 302 may vary. In some cases, the pixel values of the distance image based on the distance image are not the same.

Here, the material of the pallet 800 and the material of the surface are diverse, such as resin, wood, metal, or those coated with them. Based on this recognition, the model Ms of the distance image sensor 302 is generated in consideration of the influence of the difference in the material of the measurement target on the output of the distance image sensor 302 . That is, the input/output characteristics of the distance image sensor 302 are measured in advance for each material, and the error model M _ERR is prepared in consideration of the characteristics. The model Mp of the pallet 800 can specify the material for each part of the pallet 800 or the entire material. This makes it possible to generate more accurate training data.

(environmental model)
In addition to or instead of the error model M _ERR , the environment model M _ENV can be introduced into the measurement model. The environment model M _ENV describes the influence of at least one of ambient light and ambient temperature on the output of the range image sensor.

Many distance image sensors 302 are active sensors that sense the distance to an object by irradiating the object with probe light and measuring the reflected light. Here, if ambient light with the same wavelength as the probe light exists in the environment where the distance image sensor 302 is used, noise will be superimposed on the distance image based on the output of the distance image sensor 302 . When the environment model M _ENV is introduced, it is possible to superimpose noise dependent on environmental light on the virtual range image I1, and to bring the teacher data closer to the range image obtained in the real environment.

Moreover, the measurement error changes due to the influence of the ambient temperature. Therefore, it is better to describe the influence of the ambient temperature in the environment model.

The present invention has been described above based on the examples. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and that various design changes and modifications are possible, and that such modifications are within the scope of the present invention. By the way. Such modifications will be described below.

In the embodiments, a forklift was described as an example of a transport device, but the application of the present invention is not limited to that. It can be applied to various conveying devices that handle pallets.

600 forklift 602 vehicle body 604 fork 606 lifting body 608 mast 610 front wheel 612 rear wheel 700 cockpit 702 ignition switch 704 steering wheel 706 lift lever 708 accelerator pedal 710 brake pedal 712 forward/reverse lever 714 dashboard 300 forklift 302 distance image sensor 303 camera 303 Notification means 306 Display 308 Speaker 310 Controller 320 Microcomputer 330 Range image acquisition unit 340 Discriminator 350 Processing unit 360 Pattern matching unit 362 Template image generation unit D1 Range data D2 Range image D3 Image data D4 Partial range image D5 Position information D6 Template image 800 pallet 801 specific part 802 hole 804 column

Claims (5)

A method for generating teacher data used for learning a transport device that transports pallets, comprising:
The conveying device is
a range image sensor;
(i) Based on the output of the range image sensor, it is possible to obtain a range image in which pixel values represent distances, and (ii) Based on the range image, a detection target, which is a pallet or a specific part of the pallet, can be specified. A computing unit having a discriminator configured by machine learning using the teacher data,
and
The generating method is
step A of generating a three-dimensional model of said pallet;
When the 3D model of the pallet is placed in a 3D virtual space and the pallet is measured by the range image sensor while changing the relative positions of the 3D model of the pallet and the model of the range image sensor. a step B of generating the virtual range image that would be obtained;
a step C of generating the training data based on the virtual range image;
A generation method, comprising: 2. The generation method according to claim 1, wherein said step C includes a step of superimposing said virtual range image on a background range image. 3. The generating method according to claim 2, wherein the background distance image is generated by actual measurement of the distance image sensor. In step B, the virtual range image is generated based on an error model that describes the influence of at least one of the distance to the measurement target, the direction of the measurement surface, and the material of the measurement target on the output of the range image sensor. 4. The generation method according to any one of claims 1 to 3, characterized in that: 5. The virtual range image is generated in step B based on an environment model describing the influence of at least one of ambient light and ambient temperature on the output of the range image sensor. 3. The generation method according to any one of

Images