JP2017026533A - Parameter adjustment method of range image sensor, parameter adjusting device, and elevator system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To adjust a plurality of parameters of a range image sensor even when there are little landmark for adjusting the parameters.SOLUTION: A range image sensor 5 provided in an elevator car 4 is connected to a camera parameter adjusting device 1. A range image acquisition part 11 acquires a range image in the car 4, from the range image sensor. A type selection unit 12 acquires three-dimensional shape information of the car and acquires specifications information related to installation and imaging of the range image sensor. An application unit 13 calculates a real-scale shape inside the car 4 from the three-dimensional information, generates a three-dimensional point group from the range image, and calculates an error when the three-dimensional point group is applied to the real-scale shape.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a parameter adjustment method, a parameter adjustment device, and an elevator system for a distance image sensor.

  In the elevator system, measures are taken against overloading of passengers and overloading. For example, in order to prevent driving due to excessive passengers, the load in the car is detected by a load sensor. When the load inside the car exceeds the rated load, the alarm in the car continues to sound and the door closing operation is not performed.

  On the other hand, in recent years, a technique has been studied in which a higher level sensor than the conventional load sensor is provided in the elevator car and the elevator is highly controlled using the recognition result of the passenger recognized by the sensor. For example, in patent document 1, the occupation area which a passenger occupies in an elevator is measured using the distance image sensor which can calculate the distance value of each pixel in an image. In Patent Document 1, the degree of congestion is measured from the area occupied by passengers, and the operation of the elevator is controlled.

  Thus, in Patent Document 1, for example, a passenger who rides in a wheelchair or a passenger who pushes a shopping cart has a small weight difference with a normal passenger, but the congestion when a passenger who occupies a larger area than a normal passenger gets on. It can be measured appropriately. Furthermore, in Patent Document 1, when it is determined that the degree of congestion in the car is high and a new passenger cannot get on even if the door is opened, it is possible to perform operation control that skips the door opening on the middle floor to the destination floor. .

  In Patent Document 1, a group of distance values measured by the distance image sensor is converted into a three-dimensional point group from the imaging system model of the distance image sensor, the installation angle of the distance image sensor, and information on the installation position. And in patent document 1, among the three-dimensional point group, the set of three-dimensional point groups higher than the floor of the car is recognized as a passenger.

  In Patent Document 1, when converting a group of distance values into a three-dimensional point group, it is necessary to calibrate and optimize the parameters in the computer for external parameters such as the installation angle and installation position of the distance image sensor. In this calibration (calibration), the parameters in the computer are optimized so as to reduce the deviation between the real space and the computer.

  Since there are various dimensions in the elevator system, it is necessary to calibrate the external parameters according to the size of the car at each site where the range image sensor is installed. In addition, in order to more accurately measure the conversion of the distance value group to the three-dimensional point cloud, the internal parameters such as the focal length included in the imaging system model of the distance image sensor are used as a target in real space and in the computer. It is desirable to calibrate so that the deviation is small.

  Patent Document 2 discloses a technique for guiding a worker so that a camera is installed at an appropriate azimuth and depression angle using a user interface on an information terminal used by the worker with respect to parameter calibration. In Patent Document 2, when a worker specifies a car model via a user interface, a camera mounting method is displayed on the user interface.

  In Patent Document 2, guidance is provided so that the installation angles of the pan and tilt of the camera are approximately appropriate using the superimposed display in the camera image with the corner of the floor in the car as a mark. Furthermore, in Patent Document 2, the corner of the floor is image-recognized after guidance, and the pan and tilt of the camera between the real space and the computer are calculated from the amount of deviation in the camera image between the guidance position of the floor corner and the actual position. Calibrate to minimize installation angle.

  An ICP (Iterative Closest Point) algorithm is known as an algorithm for aligning two point groups (Non-Patent Document 1).

JP, 2014-021816, A JP 2012-205299 A

Besl, P. and McKay, N. “A Method for Registration of 3-D Shapes”, Trans. PAMI, Vol. 14, No. 2, 1992.

  In Patent Document 2, only one mark is used for calibration, and only two constraint equations can be determined from the image coordinates of the mark. For this reason, in Patent Document 2, the number of parameters that can be calibrated is limited to two or less. Therefore, in Patent Document 2, when the roll angle is calibrated in addition to the azimuth angle and the depression angle of the installation angle of the distance image sensor, the constraint formula is insufficient. In Patent Document 2, the roll angle cannot be calibrated. Furthermore, in Patent Document 2, in addition to the azimuth angle and the depression angle, the constraint formula is insufficient when the internal parameters of the distance image sensor such as the focal length are calibrated. In Patent Document 2, the internal parameters of the distance image sensor cannot be calibrated.

  In order to increase the constraint formula, it is necessary to increase the number of calibration marks. However, depending on the specifications of the car, there may be a lack of calibration marks. In particular, when the distance image sensor is installed near the entrance of the car and the back side of the car is monitored, the mark is scarce. This is because, on the far side of the car, unlike the entrance side of the car, there are no indicators such as an indicator indicating the destination floor, an open / close button, and a door.

  Although it may be possible to intentionally add calibration marks in the car, in this case, it is necessary to prepare a database of three-dimensional coordinates in the car of those marks, and it takes time to prepare for calibration. .

  The present invention has been made in view of the above-described problems, and the object thereof is a method for adjusting a parameter of a distance image sensor that can adjust a plurality of parameters even when the mark for adjusting the parameters is scarce. It is to provide a parameter adjusting device and an elevator system.

  In order to solve the above problems, a parameter adjustment method for a distance image sensor according to the present invention is a method for adjusting a plurality of parameters of a distance image sensor for obtaining a distance image including an image of a target region and a distance to the target region. The following steps: A distance image for a predetermined target region is acquired from the distance image sensor, and the three-dimensional shape information of the predetermined target region is acquired from the pre-registered three-dimensional shape information. In the specified coordinate system set based on the predetermined target area from the acquired three-dimensional shape information of the predetermined target area. The actual size of the target area is calculated, and the distance value of each pixel in the distance image is determined from the acquired distance image sensor specification information and the distance image for the predetermined target area. Generating a three-dimensional point cloud that associates the coordinate system, calculates the error in fitting a three-dimensional point cloud to scale shape, including.

  According to the present invention, it is possible to calculate an error when fitting the three-dimensional point group obtained from the distance value detected by the distance image sensor and the actual size shape of the predetermined target region. Multiple parameters of the range image sensor can be adjusted.

Explanatory drawing which shows the outline | summary of an elevator system provided with a parameter adjustment apparatus. Explanatory drawing which shows the apparatus structure in a cage | basket | car. Explanatory drawing which shows the coordinate system on the basis of a distance image sensor. Explanatory drawing which shows the coordinate system on the basis of the inside of a cage | basket | car. An example of the screen configuration of the model selection unit is shown. The other example of a screen structure of a model selection part is shown. The flowchart which shows the process which applies the three-dimensional point group produced | generated from a distance image to the actual size shape of a cage | basket | car. The schematic diagram which shows an example of the exact size shape of a cage | basket | car. The example of the three-dimensional point group produced | generated from the distance image which image | photographed the inside of a cage | basket is shown. A state in which an actual size shape of a car and a three-dimensional point group are superimposed and displayed is shown. The flowchart which shows the whole process which calibrates several parameters. The functional block diagram of a parameter adjustment apparatus concerning 2nd Example. The example of a screen which shows a mode that the marker for inputting the corresponding location of an exact size shape is set in a distance image. The example of a screen which shows the corresponding position in the actual size shape of the marker set to the distance image. The example of a screen which shows a mode that it replaces with a dotted marker and sets a corresponding location in a distance image with a line segment. The example of a screen which shows the corresponding position in the actual size shape of the line segment set to the distance image. The flowchart which shows the whole process which calibrates several parameters. The function block diagram of a parameter adjustment apparatus concerning 3rd Example. The flowchart which shows the process which applies the three-dimensional point group produced | generated from a distance image to the actual size shape of a cage | basket | car. The flowchart which shows the whole process which calibrates several parameters. The flowchart which shows the process which concerns on 4th Example and obtains a distance image using a stereo camera.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the present embodiment, as will be described in detail below, a distance image sensor for monitoring a predetermined target region that is a monitoring target is automatically or manually set to at least one of the external parameter and / or the internal parameter. It can be adjusted. In this embodiment, not only a method or apparatus for adjusting (calibrating) the parameters of the distance image sensor but also a method or apparatus for supporting the installation of the distance image sensor is disclosed. Here, the predetermined target area is, for example, the interior of an elevator system car.

  Since elevator systems differ in various ways according to customer requirements such as the maximum number of passengers and travel speed, there are a plurality of types of cars. Further, there are a plurality of types of distance image sensors to be installed in the car. In addition, since the elevator system is used in various countries around the world, local workers also perform work for installing the distance image sensor inside the car and maintenance and inspection work.

  In this way, the elevator system uses a plurality of types of cars properly, and has a property that local workers in various countries install and maintain a distance image sensor in the car. In the present embodiment, based on such properties, distance image sensors having different parameters for each model can be appropriately installed in a cage having different dimensions for each model, and a plurality of parameters can be adjusted (or calibrated). I can do it.

  In the present embodiment, the distance image acquisition unit acquires a distance image from the distance image sensor, and the model selection unit as the information acquisition unit reads the model of the car and the model of the distance image sensor. The fitting unit applies a three-dimensional point group generated from the distance image to the actual size calculated from the size determined according to the car model. The fitting unit can calculate a three-dimensional point group from the initial values of internal parameters and external parameters determined from the model of the distance image sensor and the distance value of the distance image sensor. The fitting unit optimizes at least a part (preferably three or more) of the external parameters and the internal parameters of the distance image sensor so that the three-dimensional point group is best fitted to the actual size shape. . The fitting unit can automatically perform parameter optimization, and provides information useful for parameter optimization (actual size shape and 3D point cloud display, error values) from the user interface to the operator. You can also The operator can adjust the desired parameters using information provided from the user interface.

  In the present embodiment configured as described above, in a car having various dimensions, even in a situation where there are few marks for calibration, a predetermined one of the external parameter and / or the internal parameter of the distance image sensor is determined. The parameter of the distance image sensor can be calibrated with respect to the parameter. In this embodiment, the distance image in the car acquired by the distance image sensor can be recognized using the parameters of the distance image sensor after calibration. And based on the result of image recognition, the apparatus in a cage | basket | car or operation of a cage | basket | car can be controlled.

  A first embodiment will be described with reference to FIGS. FIG. 1 shows an overall outline of an elevator system including a camera parameter adjusting device 1. The elevator system includes, for example, a camera parameter adjustment device 1, an image processing unit 2, an elevator control unit 3, a car 4, a distance image sensor 5, and a drive mechanism 6.

  The car 4 is moved up and down by a drive mechanism 6 in a hoistway (not shown) provided in the building. The operation of the car 4 is controlled by the elevator control unit 3. The distance image sensor 5 installed inside the car 4 is connected to the camera parameter adjusting device 1, and takes a distance image in the car 4 and transmits it to the camera parameter adjusting device 1.

  Here, the distance image sensor 5 is a sensor that detects a distance image. The distance image is not only a two-dimensional plane image but also a three-dimensional image having depth information (distance value). As the distance image sensor 5, for example, a sensor that calculates a distance value from the time when reflected light such as infrared rays irradiated toward an object returns can be used. Further, in the embodiments described later, a case where a stereo camera is used as the distance image sensor will be described. Hereinafter, the distance image sensor 5 may be referred to as a camera.

  The camera parameter adjustment device 1 is an example of a “distance image sensor parameter adjustment device” and includes, for example, a distance image acquisition unit 11, a model selection unit 12, and a fitting unit 13. A specific configuration example of the camera parameter adjustment device 1 will be described later. For example, the camera parameter adjustment device 1 is configured using a computer including a microprocessor, a memory, a communication interface, and the like.

  The distance image acquisition unit 11 acquires a distance image obtained by measuring a distance value in an image captured by the distance image sensor 5 in units of pixels. The model selection unit 12 is a user interface (hereinafter sometimes abbreviated as UI) for selecting the model of the car 4.

  The fitting unit 13 applies a three-dimensional point group generated from the distance image to the actual size shape of the car 4. The fitting portion 13 can also be referred to as an actual car shape fitting portion 13. The fitting unit 13 obtains a three-dimensional point group from the distance value of the distance image acquisition unit 11 and the imaging system model of the distance image sensor 5. The actual size of the car 4 is determined by the car type selected by the model selection unit 12. The fitting unit 13 calibrates a predetermined parameter specified in one or both of the external parameter and the internal parameter of the distance image sensor 5 so as to best fit the actual size shape of the car 4. .

  The image processing unit 2 performs image processing on the distance image acquired by the distance image acquisition unit 11 using the predetermined parameter for which calibration has been completed, and recognizes an object (such as an elevator passenger or luggage) existing there. The result of image recognition by the image processing unit 2 is sent to the elevator control unit 3. The elevator control unit 3 controls, for example, the operation of the car 4 based on the result of the image recognition, instructs the passengers in the car 4, and provides information to the passengers.

  FIG. 2 is an example of a device configuration focusing on the car 4. The distance image sensor 5 is installed, for example, above the door 41 of the car 4 and facing the back of the car 4. When there are no passengers in the car 4, the distance image sensor 5 shoots, for example, the wall portion and the floor surface 42 on the side facing the door 41.

  The signal processing device 43 is installed, for example, on the ceiling of the car 4. The signal processing unit 43 may be provided inside the car 4 or the like. It doesn't matter where it is installed. A coordinate system 40 shown in FIG. 2 is an example of “a predetermined coordinate system set based on a predetermined target region”.

  The signal processing device 43 implements the camera parameter adjustment device 1 and the image processing unit 2. That is, the signal processing device 43 realizes the functions of the distance image acquisition unit 11, the model selection unit 12, the fitting unit 13, and the image processing unit 2 described in FIG. In this embodiment, the information terminal 70 is used as a user interface for exchanging information with an operator. Therefore, in this embodiment, the signal processing unit 34 realizes the distance image acquisition unit 11, the model selection unit 12, the fitting unit 13, and the image processing unit 2, and the information terminal 70 is in charge of the user interface function.

  The signal processing device 43 can be configured as a computer having a housing independent of the distance image sensor 5 or can be provided inside the distance image sensor 5. When the distance image sensor 5 and the signal processing device 43 are configured separately, the signal processing device 43 is connected to the distance image sensor 5 by wire or wirelessly. When the signal processing device 43 is provided in the distance image sensor 5, the signal processing device 43 is realized using, for example, a microprocessor and a memory in the distance image sensor 5, a computer program in the memory, and the like. In the following, a case where the signal processing device 43 and the distance image sensor 5 are separately formed as shown in FIG. 2 will be described as an example. However, as described above, the signal processing device 43 is provided in the distance image sensor 5. May be.

  The signal processing device 43 acquires a distance image from the distance image sensor 5, performs image processing, and recognizes the image. The signal processing device 43 is connected to the elevator control unit 3, and the elevator control unit 3 controls each device such as an indicator in the car 4 and controls the operation of the car 4 according to the result of image recognition. You can do it.

  The above-described configurations of the distance image sensor 5, the signal processing device 43, and the information terminal 70 are examples, and are not limited to the above examples. If the distance image in the car 4 can be acquired, a three-dimensional point cloud can be generated from the distance image, the actual size shape of the car 4 can be generated, and the three-dimensional point group can be applied to the actual size shape of the car 4, Other configurations than the above configuration example can be adopted.

  A coordinate system 40 based on the inside of the car 4 is defined with an origin O and three coordinate axes (X, Y, Z). The origin O of the coordinate system 40 is set to any one of the four corners of the floor surface 42 in the car 4. The Y axis of the coordinate system 40 is set in the vertical direction. The X axis of the coordinate system 40 is set in the left direction when the car 4 is viewed from the door 41. The Z axis of the coordinate system 40 is set in the depth direction when the car 4 is viewed from the door 41.

  The distance image sensor 5 is attached to the upper portion of the door 41 so as to face the inside of the car 4 with installation angles of depression angle θ, azimuth angle φ, and roll angle ρ. The depression angle θ and the azimuth angle φ are both 0 ° when the distance image sensor 5 looks in the Z-axis direction. At this time, the rotation axes of the depression angle θ, the azimuth angle φ, and the roll angle ρ are the X axis, the Y axis, and the Z axis, respectively. Coincides with the axis.

  An operator who performs maintenance management of the elevator system can receive guidance on the installation method of the distance image sensor 5 or check the installation angle of the distance image sensor 5 using the information terminal 70.

  The information terminal 70 can be realized by, for example, a notebook personal computer, a tablet personal computer, a PDA (personal digital assistant), a mobile phone, or the like. Alternatively, for example, the information terminal 70 may be realized by connecting a personal computer (not shown) provided outside the elevator system and the signal processing device 43 via a communication network such as the Internet or a LAN (Local Area Network). Good. The information terminal 70 may communicate directly with the distance image sensor 5 or may communicate via the signal processing device 43.

  The information terminal 70 includes a user interface unit 71. The user interface unit 71 includes an information input device and an information output device. The information input device is a device for an operator to input information to the distance image sensor 5 or the signal processing device 43, and can be configured by, for example, a keyboard, a mouse, a touch panel, a voice input device, or the like. The information output device is a device that provides information to a worker, and can be configured by, for example, a display, a voice synthesizer, or the like.

  The operator can confirm the correct installation method by looking at the guidance displayed on the user interface unit 71 prior to the installation of the distance image sensor 5. Further, the operator manually adjusts the installation angle of the distance image sensor 5 based on the image in the car 4 displayed on the user interface unit 71 or adjusts the internal parameters of the distance image sensor 5. You can enter instructions.

  The distance image sensor 5 has an imaging surface similar to that of the surveillance camera, and is a sensor that measures the distance to an object in the space corresponding to each pixel in each pixel on the imaging surface. An image obtained by measuring the distance value of each pixel in the image is called a distance image. The distance image sensor 5 transmits the acquired distance image to the distance image acquisition unit 11.

  As a distance image measurement method, for example, Time Of Flight can be used. The distance image sensor according to this method has a light emitter that irradiates near-infrared light inside, and the time from when the irradiated near-infrared light is reflected by an object within the viewing angle to return. Measure the distance to the object by measuring. In an example described later, a case where a stereo camera is used as a distance image measurement method will be described.

  It will be described with reference to FIG. 3 that the coordinate system 40 can calculate the three-dimensional coordinates of the corresponding points in the space of each pixel in the distance image. First, it is shown that the three-dimensional coordinates in the space of each pixel in the distance image can be calculated by the coordinate system 50 based on the distance image sensor 5.

In FIG. 3, the distance image 500 includes a plurality of pixels 501. A pixel 501 corresponds to a point 401 in the space and has a coordinate i (u, v) in the distance image 500. A point 401 in the space has coordinates I _S (X _S , Y _S , Z _S ) in the coordinate system 50 with the distance image sensor 5 as a reference.

The origin OS of the coordinate system 50 is the projection center of the distance image sensor 5. The coordinate axes X _S , Y _S , and Z _S of the coordinate system 50 correspond to the left direction, the upper direction, and the rear direction as viewed from the distance image sensor 5. Element Z _S of coordinates I _S is equal to the distance value of the pixel 501. Here, if the imaging system model of the distance image sensor 5 is a pinhole model and the focal length of the distance image sensor 5 is f, the element X _S of the coordinate I _S can be obtained from the following equation 1, and the element Y _S is It can be obtained from the following formula 2.

It will be described with reference to FIG. 4 that the coordinates defined in the coordinate system 40 in the car 4 can be calculated from the coordinates of the coordinate system 50 with the distance image sensor 5 as a reference. In FIG. 4, I (X, Y, Z) is the three-dimensional coordinate in the coordinate system 50 of the point 401 in space. This I (X, Y, Z) can be calculated using the following equation (3).

In Equation 3, the position (X _C , Y _C , Z _C ) is the installation position of the distance image sensor 5 in the coordinate system 40. In Equation 3, angles (θ, φ, ρ) are installation angles of the distance image sensor 5 in the coordinate system 40 as shown in FIG.

  In the pixel in which the distance value is obtained in the distance image 500, the corresponding point in the space is obtained by the same procedure as that for obtaining the three-dimensional coordinate I (X, Y, Z) in the coordinate system 40 from the corresponding point 401 of the pixel 501. The coordinate system 40 can be obtained. What calculated | required the three-dimensional coordinate of each pixel contained in the distance image 500 is called a three-dimensional point group. Here, it is not necessary to obtain the three-dimensional coordinates for all the pixels of the distance image 500, and the three-dimensional coordinates may be calculated within a necessary range. That is, a three-dimensional point group can be partially calculated from the distance image 500.

Here, the external parameters of the distance image sensor 5 are the installation position (X _C , Y _C , Z _C ) and the installation angle (θ, φ, ρ). The internal parameters of the distance image sensor 5 are parameters related to the values of the coordinates X _S and Y _S of the point 401 as shown in the equations 1 and 2. As one example, the focal length f is an internal parameter.

However, in the formulas 1 and 2, the calculation can be performed in consideration of the parameters reflecting the imaging system model of the distance image sensor 5 such as the aspect ratio of the pixel 501 in addition to the focal length f. In that case, the aspect ratio and the like are added to the internal parameters of the distance image sensor. Furthermore, when the value of the coordinate Z _S of the point 401 changes according to the parameter in the imaging system model of the distance image sensor 5, the parameter in the imaging system model of the distance image sensor 5 is the internal parameter of the distance image sensor 5. It corresponds to.

  The distance image sensor 5 can be provided with an adjustment function for adjusting one or more of the installation angles (θ, φ, ρ). For example, the distance image sensor 5 may be attached to the car 4 using a fixture having an axis that can be adjusted for each of the installation angles (θ, φ, ρ). Furthermore, an adjustment reference for the installation angle (θ, φ, ρ) may be described in a work manual or the like so that an operator can adjust the installation angle while viewing the adjustment reference. The work manual indicating the installation standard can be created as electronic guidance information, for example, and can be provided to the worker through the user interface unit 71.

  As an example of the adjustment standard of the installation angle, the majority of the floor surface 42 of the car 4 is reflected as the distance image 500. Note that the installation angle of the distance image sensor 5 may be guided in advance to the worker using the floor surface 42 of the car 4 as a mark.

  FIG. 5 shows a screen example of the model selection unit 12. The model selection screen G10 is provided to the operator via the user interface unit 71. On the model selection screen G10, both the model of the car 4 and the model of the distance image sensor 5 can be individually selected.

  When the operator operates the information terminal 70 to select the pull-down menu GP11 for car model selection, the pull-down menu GP11 outputs a list display GP12 of the car 4 types. The operator selects one model corresponding to the car 4 (the car in which the sensor 5 to be adjusted is installed) from the list display GP12.

  The screen G10 is also provided with a sensor type selection pull-down menu GP13 for selecting the type of the distance image sensor 5. When the operator operates the information terminal 70 to select the pull-down menu GP13, the pull-down menu GP13 outputs a list display 114 of the type of the distance image sensor 5. The operator selects a model corresponding to the local distance image sensor 5 from the list display 114.

  The car model in the list display GP 12 defines the shape of the car 4. In this embodiment, since the shape of the car 4 is approximated by a rectangular parallelepiped, the height, frontage, and depth of the car 4 are uniquely determined. The height of the car 4 is the length in the Y-axis direction in FIG. The frontage of the car 4 is the length in the X-axis direction of FIG. The depth of the car 4 is the length in the Z-axis direction of FIG.

  The range image sensor type in the list display GP14 uniquely determines the type of the range image sensor 5 device. The uniquely determined model of the distance image sensor 5 uniquely determines the internal parameters of the distance image sensor 5. An example of the internal parameter is a focal length as described above, but is not limited thereto.

  Further, the model of the distance image sensor 5 in the list display GP 14 uniquely determines the installation position of the distance image sensor 5 in the car 4. The correspondence between the distance image sensor type in the list display GP14 and the installation position of the distance image sensor 5 can be realized by providing an installation reference for the distance image sensor 5.

  FIG. 6 shows another screen example GP10A of the model selection unit 12. When the installation position of the distance image sensor 5 is limited to one or a few places with respect to the car 4 and the type of the distance image sensor 5 attached to the car 4 is limited to one or a small number, the car model and the distance image sensor You may combine it with a model.

  A model selection screen G10A shown in FIG. 6 is a screen example of the model selection unit 12 when a cage model and a distance image sensor model are combined, and includes a pull-down menu GP15 and a list display GP16. The pull-down menu GP15 is a menu for selecting a set of a car model and a distance image sensor model. The list display GP16 is output when the operator selects the pull-down menu GP15.

  The operator selects one set of a car model and a distance image sensor model corresponding to the car 4 and the distance image sensor 5 in the list display GP16. When the model of the distance image sensor is uniquely determined with respect to the car model, a set of the car model and the distance image sensor model may be represented by the car model.

  Limiting the installation position of the distance image sensor 5 with respect to the car 4 to one place or a few places can be realized, for example, by performing pre-construction work for the installation of the distance image sensor 5 on the car 4 before the work at the site. .

  Examples of the pre-construction work include a construction in which a mounting hole for fixing the jig of the distance image sensor 5 is processed into the housing of the car 4 at the installation position of the distance image sensor 5. Further, as the preliminary work, a work for providing a hole for passing each wiring (power cable, video signal transmission cable) connected to the distance image sensor 5 in the car 4 can also be mentioned.

  Limiting the model of the distance image sensor 5 to the car 4 to one or a decimal means that the model and the installation position of the distance image sensor 5 to be applied are made common to a predetermined range group of the model of the car 4. Can be realized.

  In the screen G10 shown in FIG. 5 and the screen G10A shown in FIG. 6, the pull-down menus GP11, GP13, and GP15 are examples of UIs, and can take other display forms. For example, the pull-down menus GP11, GP13, and GP15 may be replaced by a graphical user interface (hereinafter, GUI) such as a text box or a check box.

  The fitting part 13 is demonstrated using the flowchart of FIG. The fitting unit 13 calibrates the predetermined parameter by executing the process shown in FIG. The predetermined parameter is a parameter specified from at least one or both of an external parameter and an internal parameter of the distance image sensor 5. In FIG. 7, when the external parameter is not included in the predetermined parameter, the step of S4 is removed, and when the internal parameter is not included, the step of S5 is removed.

  An outline of FIG. 7 will be described. First, the fitting unit 13 obtains the actual size shape 400 in the coordinate system 40 of the car 4 shown in FIG. 8 according to the selection result of the model selection unit 12 (S1).

  The fitting unit 13 sets initial values of internal parameters and external parameters of the distance image sensor 5 (S2). The fitting unit 13 calculates a three-dimensional point group 502 in the coordinate system 40 as shown in FIG. 9 from the distance value of the distance image sensor 5 using the internal parameters and the external parameters of the distance image sensor 5 (S3).

  The fitting unit 13 updates the external parameter designated as the calibration target so that the actual size shape 400 and the three-dimensional point group 502 fit best (S4). Furthermore, the fitting unit 13 updates the internal parameter designated as the calibration target so that the actual size shape 400 and the three-dimensional point group 502 fit best (S5).

  The fitting unit 13 determines whether or not a predetermined end condition is satisfied (S6). If it is determined that the predetermined end condition is satisfied (S6: YES), the process is completed. If it determines with the fitting part 13 not satisfy | filling completion | finish conditions (S6: NO), it will return to step S3.

  For the internal parameters and external parameters of the distance image sensor 5 in step S3, the initial values set in step S2 are used at the first time in the repetition from step S3 to S6, and the second and subsequent times are updated in steps S4 and S5. Use the value obtained.

  If there is no internal parameter to be calibrated, step S5 is omitted. Even if there is no external parameter to be calibrated, the processing result of step S4 is not omitted because it is referred to in step S5.

  Details of each step in FIG. 7 will be described below. The actual size shape 400 obtained in step S1 is a rectangular parallelepiped having a frontage, a depth, and a height, which is determined according to the car model selected by the model selection unit 12. The data format of the actual size 400 includes, for example, a three-dimensional point group obtained by dividing the surface of a rectangular parallelepiped at a predetermined interval.

  In step S <b> 2, a value preset according to the model of the distance image sensor 5 selected by the model selection unit 12 is used as an initial value for the installation position in the internal parameter and the external parameter of the distance image sensor 5.

  Here, as an example, the initial values of all internal parameters and the initial values of the installation positions in the external parameters of the distance image sensor 5 according to the selection by the model selection unit 12 are registered in advance in a database (not shown). Has been. The fitting unit 13 reads and uses a value corresponding to the model of the distance image sensor 5 from this database.

  If some of the parameters corresponding to the model of the distance image sensor 5 at the site are not registered in the database, a value such as zero or a random number may be used as the initial value of the unregistered parameter.

  With respect to the installation angle included in the external parameters of the distance image sensor 5, for example, the value of the installation angle of the distance image sensor 5 at the time of providing guidance to the operator can be used as an initial value. This is because the worker who has received the guidance can expect to attach the distance image sensor 5 at an angle as instructed in the guidance in a normal case. When performing guidance, the initial value of the installation angle of the distance image sensor 5 in step S2 can be made closer to the installation angle in the real space.

  In step S2, designation of external parameters and internal parameters of the distance image sensor 5 to be calibrated, which is referred to in steps S4 and S5, is also read. The designation follows the setting data set in advance in the fitting unit 13. The setting data may be determined according to the car model or the distance image sensor model selected by the model selection unit 12. Alternatively, as will be described later, the fitting unit 13 receives an instruction such as a reference point or a reference line used when fitting the three-dimensional point group 502 to the actual size shape 400 from the operator via the UI, and uses the instruction. Thus, the calibration target parameter (predetermined parameter) may be adjusted.

  A method for calculating the three-dimensional point group 502 in step S3 will be described. The fitting unit 13 uses the internal and external parameters of the distance image sensor 5 and Equations 1, 2, and 3, and for each pixel 501 in the distance image 500, the coordinate system 40 of the point 401 corresponding to each pixel 501. Find the three-dimensional coordinates I (X, Y, Z) at. Thereby, the fitting unit 13 can acquire the three-dimensional point group 502. Of the pixels 501, a pixel whose distance value cannot be obtained is not a calculation target of the three-dimensional point group.

  As described in the description of the distance image acquisition unit 11, the three-dimensional coordinates of each point in the three-dimensional point group 502 described above depend on the internal parameters and the external parameters of the distance image sensor 5 used in step S3. Change. Then, the internal parameters and external parameters of the distance image sensor 5 used in step S3 are changed by repeating steps S3 to S6.

  In step S4, the fitting unit 13 calculates an external parameter of the distance image sensor 5 that best fits the three-dimensional point group 502 and the actual size shape 400 using an ICP algorithm.

In the ICP algorithm, a corresponding point y _i of the actual size shape corresponding to each point p _i in the three-dimensional point group is searched according to the condition of the following formula 4 in which the distance between the two is the smallest. The corresponding point y _i obtained at this stage is a temporary corresponding point and does not necessarily have to be an exact corresponding point for each point p _i .

In Equation 4, x is a point in the actual size shape, and X is a set of x. In Equation 4, three-dimensional coordinates are defined according to the coordinate system 40 in the car 4. The coordinates of the corresponding point y _i are also defined in the coordinate system 40.

In the ICP algorithm, as shown in the following equation 5, the calibration object is reduced so that the fitting error e obtained from the set of each point p _i in the three-dimensional point group and the corresponding point y _i in the actual size shape becomes small. Optimize the external parameters specified as.

Specifically, coordinates I _S in the coordinate system 50 of p _i point determined from the internal parameters of the distance image sensor 5 in step _{S3 (X S, Y S,} Z S) and the coordinate system 40 definitive coordinates I _S of the point p _i With respect to (X _S , Y _S , Z _S ), the parameter to be calibrated is optimized so as to reduce the fitting error e using the relationship of Equation 3 between these two coordinates. The calibration target parameter is a parameter designated as a calibration target among the installation position (X _C , Y _C , Z _C ) and the installation angle (θ, φ, ρ), which are external parameters of the distance image sensor 5. It is.

In Equation 5, N is a natural number indicating the number of points in the three-dimensional point group. In the ICP algorithm, the search for corresponding points shown in Equation 4 and the optimization of the external parameter to be calibrated using Equation 5 are repeated until the fitting error e is minimized. In the ICP algorithm, if the number N of points in the three-dimensional point group is sufficiently large, it is possible to obtain a correct corresponding point y _i with respect to the point p _i by repeating a sufficient number of times, and the distance image sensor 5 It is known that all external parameters can be obtained with high accuracy.

  In step S5, the internal parameters designated as calibration targets in step S2 are optimized so that the fitting error e shown in Equation 5 is minimized.

Here, a case where the focal length f is a calibration target among the internal parameters of the distance image sensor 5 will be described as an example. The coordinates of the point p _i in Equation 5 in the coordinate system 40 are I (X, Y, Z), and the coordinates in the coordinate system 50 are I _S (X _S , Y _S , Z _S ). At this time, when the focal length Z is changed, the coordinates X _S and Y _{S in} Equations 1 and 2 change according to the change in the focal length f. As a result, I (X, Y, Z) also changes in Equation 3.

In step S5, the focal length f is changed using the above property so that the fitting error e is minimized. The same applies when the internal parameters other than the focal length f are used for calibration. If an internal parameter other than the focal length f is adjusted, the coordinate I (X, Y, Z _x ) of the coordinate system 40 changes according to the coordinate I _S (X _S , Y _S , Z _S ) in the coordinate system 50. By using this property, internal parameters other than the focal length f are optimized so that the fitting error e is minimized.

When two or more M internal parameters are to be calibrated, a constraint equation is added in which the Euclidean distance between each point p _{i in} Equation 5 and its corresponding point y _i is zero. By making the number N of each point p _i greater than M, it is possible to obtain a necessary and sufficient number of constraint equations for obtaining M internal parameters.

  In step S6, when the fitting error e becomes equal to or less than a predetermined threshold value, it is determined that the process is finished. Alternatively, when the fitting error e has converged, or when the repetition from step S3 to step S6 has been repeated a predetermined number of times or more, this processing can be determined to be finished.

  As described above, the fitting unit 13 optimizes the calibration target parameter specified in step S2. Thereby, the fitting part 13 acquires the three-dimensional point group 502A fitted to the actual size shape 400 as shown in FIG. Furthermore, the fitting unit 13 can output the optimized calibration target parameter (predetermined parameter) as a calibration result. As described above, a plurality of calibration target parameters can be designated from either one or both of external parameters and internal parameters.

  The image processing unit 2 can perform image recognition on the distance image 500 using the external parameters and internal parameters of the distance image sensor 5 calibrated by the fitting unit 13. The image processing unit 2 notifies the elevator control unit 3 of the result of image recognition. The elevator control unit 3 can control the operation of the device in the car 4 and the operation of the car 4 according to the result of image recognition.

  As the control, for example, operation control of full vehicle passing according to the degree of congestion can be mentioned. That is, for example, even if the total load of the car 4 is less than the upper limit value, if the vacant area of the floor surface 42 of the car 4 becomes small, no landing call is assigned to the car 4 or assigned. It is possible to perform control such as canceling the call at the platform. In addition to this, the situation in the car 4 can be grasped by image recognition, and appropriate control can be performed according to the grasped result.

  Note that the image processing unit 2 can also perform predetermined control directly by the image processing unit 2 without using the elevator control unit 3. For example, when passengers are gathering on the door 41 side in the car 4 and the back side of the car 4 is vacant, the image processing unit 2 can send a message such as “please pack in the back” into the car 4. it can.

  The entire calibration method in the present embodiment will be described with reference to the flowchart of FIG. In FIG. 11, the steps performed by the operator are indicated by dotted lines. The flowchart shown in FIG. 11 implements the functions of the camera parameter adjustment device 10 (also called the calibration device 10) shown in FIG.

  First, the operator operates the type selection unit 12 through the user interface unit 71 to select the type of the car 4 and the type of the distance image sensor 5 (S11).

  The fitting unit 13 reads the data of the car 4 as “three-dimensional shape information” from a database (not shown) according to the car model selected in step S11 (S12). The data of the car 4 to be read is data necessary for generating the actual size shape 400, and specifically, a height dimension, a width dimension, and a depth dimension. If the shape of the car is cylindrical, the radius and height are the data to be read.

  The fitting unit 13 reads the data of the distance image sensor 5 from a database (not shown) according to the sensor type selected in step S11 (S13). As a result, the fitting unit 13 sets the initial values of the internal parameters and the external parameters of the distance image sensor 5 as described in step S2 of FIG.

  The fitting unit 13 generates the actual size shape 400 of the car 4 based on the data read in step S12 (S14). Further, the fitting unit 13 reads the distance image 500 captured by the distance image sensor 5 from the distance image acquisition unit 11, and calculates the initial value of the three-dimensional point group 502 from the distance image 500 (S15). The fitting unit 13 calculates initial values of the three-dimensional point group 502 from the distance image 500 using the initial values of the external parameters and the internal parameters of the distance image sensor 5 set in step S2.

  The fitting unit 13 is provided on the condition that the fitting error e between the three-dimensional point group 502 and the actual size shape 400 is minimized, and includes at least one of the predetermined parameters (external parameters or internal parameters of the distance image sensor 5) specified in step S2. The calibration target parameters including any one of them are optimized (S16).

  According to the present embodiment configured as described above, the three-dimensional point group generated from the distance image is compared with the actual size shape of the car 4, and the external parameter group possessed by the distance image sensor 5 is the best fit. A predetermined parameter selected from the internal parameter group can be adjusted.

  Therefore, according to the present embodiment, a plurality of (three or more) predetermined parameters of the distance image sensor 5 can be calibrated without providing a plurality of marks on the floor surface 42 of the car 4. The efficiency of installation work and adjustment work can be increased.

  According to the present embodiment, the situation in the car 4 can be image-recognized using the distance image sensor 5 in which a plurality of predetermined parameters are appropriately calibrated, and the car 4 is determined according to the result of the image recognition. It is possible to control the inside device and to control the operation of the car 4.

  A second embodiment will be described with reference to FIGS. Each of the following embodiments including the present embodiment corresponds to a modification of the first embodiment, and therefore, description will be made focusing on differences from the first embodiment.

  FIG. 12 is a functional block diagram of the camera parameter adjustment device 1A. The camera parameter adjustment device 1A of the present embodiment includes a distance image acquisition unit 11, a model selection unit 12, a fitting unit 13A, and a corresponding part input unit 14. The fitting unit 13 </ b> A is connected to the image processing unit 2. The fitting part 13A of the present embodiment is different in the processing contents from the fitting part 13 of the first embodiment, and therefore the reference numerals are changed.

  The corresponding part input unit 14 is a UI for an operator to input information of a corresponding part in the distance image 500 corresponding to a predetermined part of the actual size shape 400. The corresponding location input unit 14 is realized by using a user interface unit 71 on the information terminal 70. The fitting unit 13A reflects the input from the corresponding part input unit 14 on the initial value set in step S2 of FIG.

  FIG. 13 shows an example of the screen G20 of the corresponding location input unit 14. The corresponding location input screen G20 is a UI screen for specifying the corresponding location between the actual size shape 400 and the distance image 500 with the marker GP22.

  The corresponding part input screen G20 includes a distance image display unit GP21 for displaying the distance image 500. The operator sets a marker GP22 at a location corresponding to the actual size shape 400 in the distance image 500 displayed on the distance image display unit GP21. In the present embodiment, of the four corners of the floor surface 42 of the car 4, the corner facing the distance image sensor 5 is used as a reference point for alignment. The operator finds an alignment reference point designated in advance by guidance or the like from the distance image 500, and sets the marker GP22 at the reference point.

  The operator can place the marker GP22 at a desired position by operating a mouse or a touch panel. The position of the marker GP22 can be changed.

  FIG. 14 shows a corresponding portion of the marker GP22 shown in FIG. The marker GP22 is set to correspond to a position 402 facing the distance image sensor 5 among the four corners corresponding to the floor surface 42 of the actual size shape 400. Therefore, when the operator places the marker GP22 on the distance image 500, the fitting unit 13 recognizes that the place where the marker GP22 is placed is a place corresponding to the position 402 in the actual size shape 400.

  The fitting unit 13 </ b> A adjusts the predetermined parameter specified in step S <b> 2 in the parameter group possessed by the distance image sensor 5, similarly to the process shown in FIG. 7. However, in this embodiment, initial values of predetermined parameters are set so that the marker GP22 and the point 402 overlap on the coordinate system 40 in step S2 of FIG.

The three-dimensional coordinates of the marker GP22 are obtained by the following procedure. The fitting unit 13 calculates the coordinate I _S (in the coordinate system 50 of the marker GP22 from the distance value Z _S of the point 150 in the distance image 500 having the same coordinates as the coordinates of the marker GP22 in the distance image display GP21 and the equations 1 and 2. X _S , Y _S , Z _S ) is obtained.

The fitting unit 13 obtains the coordinates I (X, Y, Z) in the coordinate system 40 from the coordinates I _S (X _S , Y _S , Z _S ) and Equation 3. The fitting unit 13 has an installation position (X _C , Y _C , Z _C ) and an installation angle (θ, Y) so that the coordinate I (X, Y, Z) is close to zero in the coordinate system 40. Of the external parameters of the distance image sensor 5 consisting of φ, ρ), the value of the external parameter designated in step S2 is calculated.

  Here, if the number of external parameters designated as calibration targets in step S2 is large and the value cannot be uniquely determined only by the constraint that the distance from the point 402 approaches zero in the coordinate system 40, the constraint is satisfied. Arbitrary within range. In the above description, the method for obtaining the value of the external parameter of the distance image sensor 5 from the restriction has been described. Instead, the value of the internal parameter of the distance image sensor 5 may be obtained on condition that the restriction is satisfied. .

  In this embodiment, since the initial value obtained from the point 402 and the marker GP22 is used in step S2, the accuracy of parameter optimization in steps S4 and S5 can be improved. In the ICP algorithm, as described above, as the initial value in step S2 is more appropriate, the search for corresponding points and the optimization of parameters can be performed with higher accuracy.

  FIG. 15 shows another screen example G30 of the corresponding part input unit 14. The corresponding location input screen G30 is a UI screen for specifying the corresponding location between the actual size shape 400 and the distance image 500 with the linear markers GP32A, GP32B, GP32C. The corresponding part input screen G30 includes a distance image display unit GP31 for displaying the distance image 500.

  In FIGS. 13 and 14, the point 402 is used as the alignment reference, but instead, the line segments 403A, 403B, and 403C may be used as the alignment reference (see FIG. 16). The operator sets linear markers GP32A, GP32B, and GP32C in line segments 403A, 403B, and 403C corresponding to the actual size shape 400 in the distance image 500 displayed on the distance image display unit GP31.

  The linear markers GP32A, GP32B, and GP32C sequentially correspond to the sides 403A, 403B, and 403C in the actual size 400 as shown in FIG. At this time, the fitting unit 13A obtains an initial value so that the linear markers GP32A, GP32B, GP32C and the sides 403A, 403B, 403C of the actual size 400 overlap as much as possible.

  A method for calculating the initial value will be described. The fitting unit 13A obtains, for example, division points obtained by dividing the linear markers GP32A, GP32B, GP32C and the sides 403A, 403B, 403C at predetermined intervals. The fitting unit 13A calculates the initial value in step S2 so that the divided points approach each other, as described in the calculation of the marker GP22 and the point 402. The fitting unit 13A can obtain the initial value by calculating the initial value of step S2 at all the dividing points and calculating the average value thereof. In FIG. 15, the number of alignment reference linear markers GP32A and the like is not limited to three as shown in FIG. One or more arbitrary numbers of linear markers can be provided.

  FIG. 17 is a flowchart showing the overall processing of the calibration method according to this embodiment. Steps performed by the operator are indicated by dotted lines. The processing in FIG. 17 is realized using the functions 11, 12, 13A, and 14 included in the camera parameter adjustment device 1A in FIG. In FIG. 17, steps S11 to S16 other than step S21 and step S22 are common to the first embodiment.

  First, in step S <b> 21, the operator inputs the linear markers GP <b> 32 </ b> A to GP <b> 32 </ b> C corresponding to the corresponding locations 403 </ b> A to 403 </ b> C of the actual size 400 in the distance image 500 by operating the corresponding location input unit 14.

  In step S22, the initial value of the calibration target parameter is set in step S2 of FIG. 7 by reflecting the input in step S21. In the process of FIG. 17, since the initial values of a plurality of predetermined parameters can be set with high accuracy using the plurality of linear markers GP32A to GP32C, the predetermined parameters can be optimized with higher accuracy in step S16.

  Configuring this embodiment like this also achieves the same operational effects as the first embodiment. In this embodiment, since the alignment between the actual size shape 400 and the distance image 500 is partially performed using a marker, the initial values of a plurality of predetermined parameters to be calibrated can be obtained more accurately. Can be optimized.

  A third embodiment will be described with reference to FIGS. The camera parameter adjustment device 1B according to the present embodiment includes a distance image acquisition unit 11, a model selection unit 12, a fitting unit 13B, and a corresponding part input unit 14. The fitting unit 13B is connected to the image processing unit 2.

  The fitting unit 13B is a UI for the operator to adjust the predetermined parameter specified in step S2 so that the three-dimensional point group 502 fits the actual size shape 400. The fitting unit 13B is realized by using the user interface unit 71 on the information terminal 70.

  The process of the fitting part 13B is demonstrated using FIG. Steps S1, S2, and S3 are common to the first embodiment. The fitting unit 13B of the present embodiment outputs a superimposed display of the three-dimensional point group 502 and the actual size shape 400 (S7). In step S7, if the internal parameters and the external parameters of the distance image sensor 5 are appropriate, a superimposed display in which the positional relationship between them is appropriate, such as the three-dimensional point group 502A and the actual size shape 400 in FIG. The On the other hand, if either the internal parameter or the external parameter is inappropriate, the positional relationship between the three-dimensional point group 502A and the actual size shape 400 is also inappropriate. As a result, in the superimposed display, the positions of the two do not overlap or the angles of the two shift.

  The fitting unit 13B waits for reception of an end instruction from the worker (S8). That is, the operator inputs an end instruction via the information terminal 70 when the manual adjustment of the predetermined parameter is completed. For example, the operator can instruct the end by operating an end button (OK button) displayed on the UI screen. As a criterion for issuing the end instruction, for example, a case where the positional relationship between the three-dimensional point group 502A being superimposed and displayed in step S7 and the actual size shape 400 becomes appropriate can be cited. This judgment criterion may be communicated to the worker in advance through a work manual or guidance.

  When the termination instruction is received from the worker (S8: YES), the fitting unit 13B terminates the process. Until the end instruction is received from the worker (S8: NO), the fitting unit 13B receives the parameter adjustment instruction from the worker (S9), and corrects the three-dimensional point group 502 according to the adjustment instruction (S3).

  In other words, when the fitting unit 13B receives the calibration target parameter adjustment instruction from the work, the fitting unit 13B returns to step S3 and recalculates the three-dimensional coordinates of each point of the three-dimensional point group 502 reflecting the adjustment instruction of step S9. . Note that the operator can give an instruction for adjusting the parameter to the fitting unit 13B by, for example, inputting a numerical value in a text box or operating a slide bar.

  FIG. 20 is a flowchart showing the overall processing of the calibration method according to this embodiment. Steps performed by the operator are indicated by dotted lines. The processing in FIG. 20 is realized using the functions 11, 12, 13B, and 14 included in the camera parameter adjustment device 1B shown in FIG.

  In FIG. 20, steps S11 to S15 are common to the first embodiment. In step S31, the operator operates the fitting unit 13B to display a superimposed display in which the positional relationship between them is appropriate, such as the three-dimensional point group 502A and the actual size shape 400 in FIG. A plurality of predetermined parameters of the image sensor 5 can be adjusted.

  Configuring this embodiment like this also achieves the same operational effects as the first embodiment. Furthermore, in this embodiment, the operator can manually adjust the predetermined parameters of the distance image sensor 5 while visually checking the superimposed display of the three-dimensional point group 502 and the actual size shape 400.

  By combining the first embodiment and the present embodiment, the predetermined parameter is automatically adjusted (first stage adjustment) by applying the three-dimensional point group 502 to the actual size shape 400 with the first accuracy. The operator can manually adjust while watching the superimposed display (second stage adjustment), and finally calibrate the predetermined parameter with the second accuracy higher than the first accuracy.

  Furthermore, the accuracy of the first stage adjustment can be further improved by combining this embodiment with the first and second embodiments.

  Furthermore, in this embodiment, since a superimposed display of the three-dimensional point group 502 and the actual size shape 400 can be output, if the superimposed display screen is recorded electronically or as a printed matter, it can be used as a history of parameter adjustment work. it can. Using this adjustment history, it is possible to confirm that the maintenance inspection has been properly performed, and to evaluate the skill of the worker.

  A fourth embodiment will be described with reference to FIG. Instead of the Time Of Flight distance image sensor 5, an arbitrary distance image sensor that can convert the distance value of each pixel in the image into three-dimensional data can be applied. Stereo cameras and laser radars are examples, but not limited to these.

  When the distance image sensor 5 is a stereo camera, it is constructed from two or more cameras. Hereinafter, a case where a stereo camera is constructed from two cameras will be described. When the stereo camera includes three or more cameras, it can be considered as a set of combinations of stereo cameras composed of two cameras.

In the stereo camera including two cameras, the distance value Z _S of the pixel 501 in the distance image 500 can be calculated from the parallax between the cameras installed on the left and right by the processing shown in FIG. When the cameras are installed up and down, rotating the image 90 degrees is equivalent to installing the cameras on the left and right.

The distance image acquisition unit 11 first acquires images of the left and right cameras (S41). The distance image acquisition unit 11 corrects lens distortion in each of the left and right cameras (S42). In step S42, using a lens distortion coefficient k _L of the right and left _cameras, a k _R as a parameter. The lens distortion coefficients k _{L and} k _R are one-dimensional or more real vectors.

The distance image acquisition unit 11 allows the left and right cameras so that corresponding points in the left and right cameras (points where the same object in the space appears in the left and right cameras) appear at the same v coordinate (the vertical coordinate of the image). Each of them is subjected to parallel coordinate transformation to be parallelized (S43). Here, the matrices M _L and M _R of the homogeneous coordinate transformation are the focal points f _L and f _R of the left and right cameras, the relative angles (θ _d , φ _d , ρ _d ) between the left and right cameras, and the relative positions ( X _d , Y _d , Z _d ) and the image size of the distance image 500 are uniquely determined.

The distance image acquisition unit 11 sets the corresponding point i _L (u _L , v _R ) in the left camera of the pixel i _R (u _R , v _R ) in the right camera as a condition that minimizes the brightness difference in the neighboring pixels. And the parallax Δ = u _R −u _L is calculated. This is performed at all points in the image of the right camera (S44).

The distance image acquisition unit 11 calculates the distance value Z _S from the parallax Δ of each pixel of the right camera after the parallelization by the following formula 6 (S45). In Equation 6, f is the focal length of the right camera after parallelization, and is uniquely determined by each parameter in S43.

When the distance image sensor 5 is a stereo camera, the lens distortion coefficients k _{L and} k _R of the left and right cameras in S43 and S44 described above, the focal lengths k _{L and} k _R, and the relative angle (θ _d , φ) between the left and right cameras. _d, and [rho _d), the relative position _{(X d, Y d, Z} d) and, but the internal parameters of the stereo camera. When these internal parameters are changed, the distance value Z _S calculated by Equation 6 changes.

For example, when the X-axis coordinate X _d changes, the distance value Z _S varies in inverse proportion to the coordinates X _d, matrix M _L for homogeneous coordinate transformation, the M _R changes, the parallax Δu changes accordingly. Further, when the distance value Z _S changes, the Xs coordinate and the Z _S coordinate of the point 401 calculated by Equations 1 and 2 also change. Therefore, when the stereo camera is the distance image sensor 5, many internal parameters affect the coordinates I _S (X _S , Y _S , Z _S ) of the point 401. In the first to third embodiments described above, when the stereo camera is the distance image sensor 5, one or more of the internal parameters of the stereo camera described above can be set as calibration targets.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this, The following changes and function additions can be performed in the range which does not deviate from the seed of this invention.

  The fitting unit may display a superimposed display of the three-dimensional point group after calibration and the actual size shape on the information terminal. As a result, the operator can visually confirm whether or not the calibration is properly performed.

  The fitting unit can also notify the information terminal of the fitting error. Further, when the fitting error exceeds a predetermined threshold, an alert indicating that the calibration has failed may be issued. Thereby, the operator can know how accurately the calibration was performed.

  The calibration of the predetermined parameter can be executed at a predetermined opportunity. The predetermined opportunity is, for example, when the elevator system is delivered, when the elevator system is checked, when the distance image sensor is energized, or the like. By repeatedly executing the predetermined parameters of the distance image sensor with a predetermined trigger (for example, at a predetermined cycle), it is possible to cope with aging degradation of the distance image sensor. In addition, by notifying the information terminal of the fitting error at this time, the operator can know the degree of deterioration over time. In addition, the operator can know a decrease in calibration accuracy due to aged deterioration. In addition, if an alert when the fitting error exceeds a predetermined threshold is notified to the information terminal, the operator can know that the deterioration over time has become excessive, and consider replacing the distance image sensor. You can also.

  Here, the aging deterioration is not limited to natural deterioration due to long-term use, for example, when the installation angle of the distance image sensor is shifted due to vibration caused by the operation of the car, the passenger or the passenger's baggage contacts the distance image sensor. In some cases, the installation angle may deviate.

  The fitting portion is not limited to the configuration of each of the embodiments. Another method that can achieve an error in fitting the three-dimensional point group and the actual size shape may be used. For example, external parameter optimization and internal parameter optimization may be performed simultaneously.

  The actual size is not limited to a three-dimensional wire frame model. Any other data structure such as a mesh may be used as long as a fitting error with a three-dimensional point group can be defined. For example, the actual size shape may be defined as a plurality of surfaces (three surfaces). Furthermore, it is not necessary that the points be arranged on the entire surface, and it is sufficient if the points are included with a density that allows the fitting error to be calculated.

  A partial error may be cut out from the distance image so that the accuracy of the distance value is high or the fit with the actual size shape can be easily determined, and the fit error may be calculated. For example, a user interface such as a corresponding part input unit may be used to specify a region to be cut out.

  The actual shape of the car is not limited to a rectangular parallelepiped. It may be cylindrical. Further, for example, when a chair is provided in a car, the shape of the chair can be included in the actual size in addition to the shape of the rectangular parallelepiped.

  In the flowchart described in each embodiment, the order of steps is changed, a plurality of steps are combined into one step, the contents of one step are divided into a plurality of steps, or the contents of some steps are changed. can do. Those modifications are also included in the scope of the present invention.

  In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  1, 1A, 1B: Camera parameter adjustment device, 2: Image processing unit, 3: Elevator control unit, 4: Car, 5: Distance image sensor, 11: Distance image acquisition unit, 12: Model selection unit, 13, 13A, 13B: fitting part, 14: corresponding part input part, 400: actual size shape, 502, 502A: three-dimensional point group

Claims (16)

A method for adjusting a plurality of parameters of a distance image sensor for obtaining a distance image including an image of a target region and a distance to the target region,
The following steps:
Obtaining a distance image for a predetermined target region from the distance image sensor;
Acquire the three-dimensional shape information of the predetermined target region from the three-dimensional shape information registered in advance,
Acquire specification information regarding installation and shooting of the distance image sensor from pre-registered specification information,
From the acquired three-dimensional shape information of the predetermined target region, calculate the actual size shape of the predetermined target region in a predetermined coordinate system set based on the predetermined target region,
From the acquired specification information of the distance image sensor and the distance image for the predetermined target region, a three-dimensional point group in which the distance value of each pixel of the distance image is associated with the predetermined coordinate system is generated. ,
Calculating an error in fitting the three-dimensional point group to the actual size shape;
Adjustment method of distance image sensor including Adjusting a value of a predetermined parameter among a plurality of parameters of the distance image sensor so as to reduce the error;
The parameter adjustment method of the distance image sensor according to claim 1. When outputting the error, display and output from the user interface so that the error can be visually confirmed,
Receiving an instruction to adjust the value of the predetermined parameter;
The parameter adjustment method of the distance image sensor according to claim 2. The distance image according to claim 3, wherein the user interface outputs the error by superimposing the actual size shape and the three-dimensional point cloud, and / or outputs the error value. Sensor parameter adjustment method. When calculating an error when fitting the three-dimensional point group to the actual size shape, the target value of the installation angle included in the guidance information for workers provided from the user interface for installing the distance image sensor is set. use,
The parameter adjustment method of the distance image sensor according to claim 2. When applying the three-dimensional point group to the actual size shape, an input indicating the corresponding position between the distance image and the actual size shape is received, and the received three-dimensional point group is applied to the actual size shape using the received input. ,
The method for adjusting parameters of a distance image sensor according to claim 1. Each of the above steps is performed at a predetermined opportunity set in advance.
The parameter adjustment method of the distance image sensor according to claim 6. The distance image sensor is a stereo camera.
The parameter adjustment method of the distance image sensor according to claim 6. When an external parameter related to the installation of the distance image sensor is selected as the predetermined parameter, the value of the external parameter is adjusted using an ICP (Iterative Closest Point) algorithm.
The parameter adjustment method of the distance image sensor according to claim 2. The predetermined target area is an area inside the elevator car;
The parameter adjustment method of the distance image sensor according to claim 6. An apparatus for adjusting a plurality of parameters of a distance image sensor that obtains a distance image including an image of a target region and a distance to the target region,
A distance image acquisition unit;
An information acquisition unit;
With a fitting part,
The distance image acquisition unit acquires a distance image for a predetermined target region from the distance image sensor,
The information acquisition unit acquires three-dimensional shape information of the predetermined target region from pre-registered three-dimensional shape information, and further installs the distance image sensor from pre-registered specification information And get spec information about shooting,
The fitting unit calculates an actual size shape of the predetermined target region in a predetermined coordinate system set based on the predetermined target region from the three-dimensional shape information of the predetermined target region, and the distance image From the specification information of the sensor and the distance image for the predetermined target region, a three-dimensional point group is generated by associating the distance value of each pixel of the distance image with the predetermined coordinate system, and the three-dimensional point A parameter adjustment device for a distance image sensor that calculates an error when fitting a group to the actual size shape. The fitting unit adjusts a value of a predetermined parameter among a plurality of parameters of the distance image sensor so that the error is reduced;
The parameter adjustment device of the distance image sensor according to claim 11. The fitting unit displays and outputs from the user interface so that the error can be visually confirmed, and receives an instruction to adjust the value of the predetermined parameter;
The parameter adjustment apparatus of the distance image sensor according to claim 12. The fitting unit receives an input indicating a corresponding portion of the distance image and the actual size shape when the three-dimensional point group is applied to the actual size shape, and uses the received input to input the 3D point group to the actual size shape. Apply to actual size shape,
The parameter adjustment apparatus of the range image sensor in any one of Claims 11-13. The apparatus for adjusting a parameter of the distance image sensor according to claim 11,
Elevator system.   The image processing apparatus comprising the parameter adjustment device for the distance image sensor according to claim 11, wherein the distance image of the distance image sensor is recognized.

Images