JP2024025074A - Marker position registration device, marker position registration program, and marker position registration method - Google Patents

Abstract

The present invention provides a marker position registration device that can reduce the effort required to register the positions of a plurality of markers. [Solution] An image acquisition unit that acquires positional data of a subject reflected in an image including a work area of a target device and a plurality of markers as data on a three-dimensional camera coordinate system based on a photographing unit that photographs the image. 113, registered markers for which position information on the target device coordinate system for each specific position of the marker is registered, and position information for the target device coordinate system for each specific position of the marker included in the image are not registered. A detection unit 114 detects an unregistered marker, and when the detection unit 114 detects an unregistered marker, the position information on the target device coordinate system corresponding to the unregistered marker is transferred to the target device coordinate system corresponding to the registered marker. It includes a registration unit 112 that performs registration based on position information on the system. [Selection diagram] Figure 3

Description

The present invention relates to a marker position registration device, a marker position registration program, and a marker position registration method.

When using a camera to teach a robot's movements, it is necessary to adjust the positional relationship between the robot coordinate system and the camera coordinate system so that they match. In Patent Document 1 listed below, the position and orientation of a virtual robot are manually moved, and the robot's movements are taught using position and orientation information when the worker judges that the real robot and the virtual robot match. ing.

Japanese Patent Application Publication No. 2020-55075

In the method of Patent Document 1, it takes time and effort to operate the virtual robot, and the operator judges whether or not they match, so it cannot be said that the accuracy of adjustment is high. To solve these problems, for example, there is a method of using markers placed in the robot's work area. In this method, for example, the positional relationship between the marker and the robot may be registered, and the positional relationship in the coordinate system may be adjusted using a coordinate transformation formula calculated based on the positional relationship.

However, in this method, when a plurality of markers are used, it is necessary to register the positional relationship with the robot for each marker, which is time-consuming.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a marker position registration device, a marker position registration program, and a marker position registration method that can reduce the effort required to register the positions of a plurality of markers.

A marker position registration device according to one aspect of the present invention acquires an image including a work area of a target device and a plurality of markers as data on a three-dimensional camera coordinate system based on a photographing unit that photographs the image. The image acquisition unit selects registered markers that have registered position information on the target device coordinate system for each specific position of the marker, and registers position information on the target device coordinate system from among the markers included in the image. a detection unit that detects an unregistered marker that has not been registered; and a detection unit that detects an unregistered marker that has not been registered; A registration unit that performs registration based on position information on the system.

According to this aspect, an image including the work area of the target device and a plurality of markers is acquired as data on the camera coordinate system, and the target device coordinate system for each specific position of the marker is selected from among the markers included in the image. Detect registered markers for which the above position information has already been registered, and unregistered markers for which position information on the target device coordinate system for each specific position of the marker has not yet been registered, and Based on the position information on the target device coordinate system corresponding to the position information on the target device coordinate system corresponding to the specific position of the unregistered marker, it is possible to register the position information on the target device coordinate system corresponding to the specific position of the unregistered marker. Therefore, by placing a registered marker and an unregistered marker in the work area of the target device and capturing them in the same image, the position information on the target device coordinate system corresponding to the specific position of the unregistered marker can be obtained. , registration can be performed based on position information on the target device coordinate system corresponding to a specific position of a registered marker.

In the above aspect, the coordinate system setting section sets a three-dimensional marker coordinate system based on the registered marker detected by the detection section, and the camera coordinate system and marker coordinate system of the registered marker detected by the detection section, respectively. The registration unit further includes a calculation unit that calculates a conversion formula for converting the coordinates between the camera coordinate system and the marker coordinate system based on a specific position in the calculation unit, and the registration unit calculates the conversion formula calculated by the calculation unit, and Position information on the target device coordinate system corresponding to an unregistered marker may be registered using position information on the target device coordinate system corresponding to the registered marker.

According to this aspect, a three-dimensional marker coordinate system is set based on the registered marker, and the camera coordinate system and the marker coordinate system are set based on specific positions of the registered marker in the camera coordinate system and the marker coordinate system. By further using the above conversion formula in addition to the position information on the target device coordinate system corresponding to the specific position of the registered marker, the coordinates of the unregistered marker can be changed to It becomes possible to register the above location information.

In the above aspect, when the total number of registered markers and unregistered markers detected by the detection unit exceeds a predetermined maximum number of markers, an output that outputs a message to the effect that markers exceeding the upper limit number have been detected The device may further include a section.

According to this aspect, it is possible to remind the worker that a marker that does not need to be placed is placed.

The above aspect may further include an output section that outputs a message to the effect that an unregistered marker has been detected when the detection section detects an unregistered marker.

According to this aspect, it is possible to remind the worker that an unregistered marker is placed.

In the above aspect, the determination unit determines whether the shape of the registered marker or the shape of the unregistered marker detected by the detection unit matches a predetermined shape; The image forming apparatus may further include an output unit that outputs a message to the effect that a marker that does not match the predetermined shape has been detected when it is determined that the shape of the marker does not match the predetermined shape.

According to this aspect, it is possible to remind the worker that an incorrect marker or an object other than the marker has been placed.

A marker position registration program according to another aspect of the present invention allows a computer to transfer position data of a subject to an image including a work area of a target device and a plurality of markers in a three-dimensional manner based on a photographing unit that photographs the image. An image acquisition unit that acquires data as data on the camera coordinate system of the image, registered markers that have registered position information on the target device coordinate system for each specific position of the marker from among the markers included in the image, and the target device. A detection unit that detects an unregistered marker for which position information on the coordinate system is not registered, and when the detection unit detects an unregistered marker, detects the position information on the target device coordinate system corresponding to the unregistered marker. It functions as a registration unit that performs registration based on position information on the target device coordinate system corresponding to the registration marker.

According to this aspect, the position data of the subject reflected in the image including the work area of the target device and a plurality of markers is acquired as data on the camera coordinate system, and the specific position of the marker is determined from among the markers included in the image. Detects registered markers for which position information on the target device coordinate system has already been registered for each marker, and unregistered markers for which position information on the target device coordinate system for each specific position of the marker has not yet been registered. Based on the position information on the target device coordinate system corresponding to the specific position of the registered marker, the position information on the target device coordinate system corresponding to the specific position of the unregistered marker can be registered. Therefore, by placing a registered marker and an unregistered marker in the work area of the target device and capturing them in the same image, the position information on the target device coordinate system corresponding to the specific position of the unregistered marker can be obtained. , registration can be performed based on position information on the target device coordinate system corresponding to a specific position of a registered marker.

According to the present invention, it is possible to provide a marker position registration device, a marker position registration program, and a marker position registration method that can reduce the effort required to register the positions of a plurality of markers.

1 is a diagram illustrating the configuration of a welding robot system including an imaging device according to an embodiment. It is a figure showing an example of a marker. 1 is a diagram illustrating a functional configuration of an imaging device according to an embodiment. It is a figure showing an example of an input form. FIG. 3 is a schematic diagram for explaining a specific example of inputting data into an input form. 3 is a flowchart for explaining the operation of the photographing device according to the embodiment. It is a figure which shows an example of the marker based on a modification.

Preferred embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in each figure, those with the same reference numerals have the same or similar configurations. Furthermore, since the drawings are schematic, the dimensions and proportions of each component may differ from the actual ones.

FIG. 1 is a diagram illustrating the configuration of a welding robot system 100 including an imaging device (marker position registration device) 1 according to an embodiment. Welding robot system 100 includes, for example, an imaging device 1, a robot control device 2, and a manipulator 3. The imaging device 1 and the robot control device 2 are connected, for example, via a network Na, and the robot control device 2 and the manipulator 3 are connected, for example, via a communication cable Nb. The network Na may be wireless or wired (including communication cables). Note that the welding robot system 100 may include a teach pendant. The teach pendant is an operating device used by a worker to teach the operation of the manipulator 3, and can be used by being connected to the robot control device 2.

The manipulator 3 is a welding robot that performs arc welding according to construction conditions set in the robot control device 2. The manipulator 3 includes, for example, a multi-joint arm 31 provided on a base member fixed to a factory floor or the like, and a welding torch 32 connected to the tip of the multi-joint arm 31. A tool or the like can be connected to the tip of the multi-joint arm 31 instead of the welding torch 32.

The robot control device 2 is a control unit that controls the operation of the manipulator 3, and includes, for example, a control section 21, a storage section 22, a communication section 23, and a welding power supply section 24.

The control unit 21 is, for example, a processor, and controls the manipulator 3 and the welding power supply unit 24 by executing a work program stored in the storage unit 22 .

The communication unit 23 controls communication with the imaging device 1 connected via the network Na, and controls communication with the manipulator 3 connected via the communication cable Nb.

The welding power supply unit 24 supplies welding current, welding voltage, etc. to the manipulator 3 according to predetermined welding conditions, for example, in order to generate an arc between the tip of the welding wire and the workpiece. The welding execution conditions include, for example, data items such as welding conditions, welding start position, welding end position, welding distance, and attitude of the welding torch. The welding conditions include, for example, data items such as welding current, welding voltage, welding speed, wire feeding speed, and workpiece thickness. The welding power supply section 24 may be provided separately from the robot control device 2.

The photographing device 1 can be, for example, a tablet terminal equipped with a digital camera, but is not limited thereto, and may be a portable terminal equipped with a digital camera. The portable terminal can include, for example, an AR device, a smartphone, a personal digital assistant (PDA), a notebook PC (personal computer), and the like.

The photographing device 1 includes, for example, a control section 11, a storage section 12, a communication section 13, a photographing section 14, a coordinate acquisition section 15, and a display section 16.

The control unit 11 is, for example, a processor, and controls each unit of the photographing device 1 by executing a program such as a marker position registration program stored in the storage unit 12 .

The storage unit 12 stores various programs and data. The various programs include, for example, the marker position registration program according to the embodiment. The various data include, for example, marker information regarding markers, marker placement information regarding marker placement, and the like. Note that the marker information and marker arrangement information may be stored in the storage unit 22 of the robot control device 2. The marker information and marker placement information will be explained below.

The marker information includes, for example, a marker ID and identification image for identifying the marker, size information regarding the size of the marker (for example, the length of each side of a rectangle and the angle of each vertex), and shape information regarding the shape of the marker (for example, a square or rectangle). Details of the marker will be described later.

For example, the marker placement information can include, for each marker ID, position information on the robot coordinate system for each specific position (for example, four corner positions of the marker) provided on the marker of that ID. Note that the marker placement information may be managed by being included in the marker information.

The communication unit 13 controls communication with the robot control device 2 connected via the network Na.

The photographing unit 14 is, for example, a digital camera including a lens and an image sensor (imaging device), and converts light from a subject received by the lens into an electrical signal (image data). A digital camera may be either a two-dimensional camera or a three-dimensional camera. The photographing unit 14 according to the present embodiment exemplarily photographs an image including the work area of the manipulator 3 and the marker.

FIG. 2 shows an example of a marker. Near the center of the top of the marker M shown in the figure, "1" is assigned as a marker ID (marker identifier) for identifying the marker, and near the four corners of the marker M, an ID for identifying the corner position is assigned. Numbers from "0" to "3" are assigned as corner numbers (corner identifiers). Moreover, an identification image I having a different pattern and shape for each marker is provided inside the peripheral edge of the marker M.

Note that the marker is not limited to the marker shown in FIG. 2, and may be any marker that can be recognized by the imaging unit 14. For example, it is preferable to use an AR marker as the marker. By using an AR marker, when an AR marker is recognized, it becomes possible to easily display a camera coordinate system based on the AR marker superimposed on the actual video. Further, the marker identifier is not limited to a marker ID, and may be, for example, a number, an image, a shape, or a combination thereof. Furthermore, the corner identifier is not limited to the corner number, and may be, for example, a symbol, color, shape, or a combination thereof.

The coordinate acquisition unit 15 shown in FIG. 1 is, for example, a distance measurement sensor, and acquires three-dimensional coordinate data (hereinafter also referred to as "point group data") corresponding to a target object. As the distance measurement sensor, for example, a LiDAR (Light Detection and Ranging) sensor, a millimeter wave sensor, an ultrasonic sensor, etc. can be used. Note that the three-dimensional coordinate data corresponding to the object may be calculated and acquired based on a plurality of images of the object taken from a plurality of different positions.

The display unit 16 is, for example, a display having a touch panel, and displays an image of a subject captured by the imaging unit 14, and also receives input such as operation instructions from a worker. The display unit 16 may be provided separately from the photographing device 1, for example as a display device having a touch panel.

FIG. 3 is a diagram illustrating the functional configuration of the imaging device 1 according to the embodiment. The photographing device 1 has a functional configuration including, for example, a display control section 111, a registration section 112, an image acquisition section 113, a detection section 114, a marker coordinate system setting section 115, a calculation section 116, and a determination section. 117 and an output section 118. Each function will be sequentially explained below.

The display control unit 111 controls the content displayed on the display unit 16. The content displayed on the display unit 16 includes, for example, an input form. The input form is a form for registering marker placement information of markers placed in the work area of the manipulator 3.

FIG. 4 shows an example of an input form. Input form F in the same figure includes, for example, marker ID, corner number, X coordinate, Y coordinate, and Z coordinate as input items to be input by the worker. In the marker ID of the input form F, the ID given to the marker M is input. In the corner number of the input form F, the number given to the corner position of the marker M pointed to by the tip (specific location) of the tool connected to the tip of the multi-joint arm 31 of the manipulator 3 is input. In the X coordinate, Y coordinate, and Z coordinate of the input form F, each coordinate of the robot coordinate system corresponding to the tip of the tool is input. Note that the coordinates input in the X, Y, and Z coordinates of the input form F are not limited to coordinates in the robot coordinate system, and may be coordinates that are different from the coordinate system of the coordinate data acquired by the coordinate acquisition unit 15. It is fine as long as it is a system.

A detailed explanation will be given with reference to FIG. 5 in addition to FIG. 4. The marker M shown in FIG. 5 is the same as the marker M shown in FIG. In this case, the ID "1" given to the marker M is inputted into the marker ID in the first line of the input form F in FIG. 4, for example. Subsequently, the tip of the tool 32t in FIG. 5 points to the corner position corresponding to the corner number "0" of the marker M. Therefore, the corner number "0" given to the marker is input to the corner number in the first row of the input form F in FIG. Subsequently, "50, 50, 50" is displayed on the teach pendant TP in FIG. 5 as the coordinates of the robot coordinate system corresponding to the tip of the tool 32t. Therefore, "50" is input to each of the X coordinate, Y coordinate, and Z coordinate of the first line of the input form F in FIG.

The registration unit 112 shown in FIG. 3 generates position information on the robot coordinate system for each corner position of the marker based on the content input in the input form F, and registers it as being included in the marker placement information described above. This makes it possible to associate the position of the marker M in the camera coordinate system with the position of the marker M in the robot coordinate system.

Here, it is preferable that information regarding three or more corner positions out of the four corner positions of the marker M be input into the input form F and registered in the marker arrangement information, but the present invention is not limited thereto. For example, information about one corner position of a marker M is registered, and based on the registered information about the one corner position and size information and shape information included in the marker information corresponding to that marker M, other Three corner positions may be calculated and registered in the marker placement information. Further, the object to be registered in the marker placement information is not limited to the corner position of the marker, but may be any specific position of the marker, and three or more such specific positions may be registered.

The image acquisition section 113 acquires position data of a subject included in the image photographed by the photographing section 14 . The acquired position data is data on a three-dimensional camera coordinate system based on the photographing unit 14 that photographs the image (for example, the center of the lens).

The detection unit 114 detects registered markers registered in the marker arrangement information (hereinafter referred to as “registered markers”) and unregistered markers not registered in the marker arrangement information from among the images captured by the imaging unit 14. Each registered marker (hereinafter referred to as "unregistered marker") is detected. For example, if an image is analyzed based on the shape, size, pattern, color, etc. of the marker included in the marker information, and a marker identified based on the analyzed content is registered in the marker placement information, the marker Detected as a registered marker. On the other hand, if the marker specified based on the analyzed content is not registered in the marker placement information, the marker is detected as an unregistered marker.

The marker coordinate system setting unit 115 sets a three-dimensional marker coordinate system based on the registered marker detected by the detection unit 114. For example, a three-dimensional marker coordinate system can be set based on a specific point (reference point) of a registered marker. If there are multiple detected registered markers, for example, the worker may arbitrarily select the reference marker and set the marker coordinate system, or the marker coordinate system may be set by arbitrarily selecting the marker that is the reference marker, or the marker located closest to the center of the photographed image may be set. A marker coordinate system may be set based on the marker. The specific point of the marker may be, for example, either the center position or corner position of the marker, or a position selected by the operator on the touch panel.

The calculation unit 116 calculates a conversion formula for converting the coordinates between the camera coordinate system and the marker coordinate system based on the corner positions in the camera coordinate system and the marker coordinate system of the registered marker detected by the detection unit 114. do. This will be explained in detail below.

First, the camera coordinate system A and the marker coordinate system B are defined as follows (1) based on the coordinates of the corner numbers "0", "1", and "2" of the registered markers. Here, the coordinates of corner No. "0" in the camera coordinate system are (x ₀ , y ₀ , z ₀ ), and the coordinates of the marker coordinate system are (x' ₀ , y' ₀ , z' ₀ ). The coordinates of corner No. "1" in the camera coordinate system are (x ₁ , y ₁ , z ₁ ), and the coordinates of the same marker coordinate system are (x' ₁ , y' ₁ , z' ₁ ). The coordinates of corner No. "2" in the camera coordinate system are (x ₂ , y ₂ , z ₂ ), and the coordinates of the same marker coordinate system are (x' ₂ , y' ₂ , z' ₂ ).

A transformation matrix (A0, B0, Mt) between the camera coordinate system and the marker coordinate system is calculated using the camera coordinate system A and the marker coordinate system B in (1) above. The coordinate A0, the coordinate B0, and the rotation matrix Mt that constitute this transformation matrix are each defined as shown in (2) below.

VA _in (2) above is calculated as in (3) below.

vecA ₀ , vecA ₁ , and vecA ₂ in (3) above are each calculated as shown in (4) below.

V _B in (2) above is calculated as in (5) below.

vecB ₀ , vecB ₁ and vecB ₂ in (5) above are calculated as shown in (6) below.

Based on (1) to (6) above, the conversion formula for converting the point p _A of the camera coordinate system acquired by the coordinate acquisition unit 15 to the point p _B of the marker coordinate system is as shown in (7) below. It can be expressed as

Similarly, a conversion formula for converting point v _A of the position vector of the camera coordinate system to point v _B of the position vector of the marker coordinate system can be expressed as shown in (8) below.

Similarly, a conversion formula for converting point p _B in the marker coordinate system to point p _A in the camera coordinate system can be expressed as shown in (9) below.

Similarly, a conversion formula for converting point v _B of the position vector of the marker coordinate system to point v _A of the position vector of the camera coordinate system can be expressed as shown in (10) below.

The conversion equations (7) to (10) above allow the camera coordinate system and the marker coordinate system to be associated with each other, and the marker placement information allows the marker coordinate system and the robot coordinate system to be associated with each other. This also makes it possible to associate the camera coordinate system with the robot coordinate system.

Using the conversion formulas (7) to (10) above, it is possible, for example, to display a model of the manipulator 3 on the display unit 16 in AR or to simulate the operation of the manipulator 3 on the display unit 16. can.

When the detection unit 114 detects an unregistered marker, the registration unit 112 generates position information on the robot coordinate system corresponding to the unregistered marker based on position information on the robot coordinate system corresponding to the registered marker. register. As a result, the marker placement information corresponding to the unregistered marker is registered, and henceforth it can be handled as a registered marker. Position information on the robot coordinate system corresponding to an unregistered marker can be calculated as follows, for example.

An unregistered marker and a registered marker included in the same captured image are located on the same camera coordinate system. The camera coordinate system and the marker coordinate system based on the registered marker can be associated with each other using the conversion formulas (7) to (10) calculated by the calculation unit 116. Therefore, by using the conversion formulas (7) to (10) above for registered markers, the positions of the unregistered marker and the registered marker located on the same camera coordinate system on the camera coordinate system can be changed to Each can be converted to a position on the marker coordinate system based on the marker.

Here, the coordinate transformation formula between the marker coordinate system and the robot coordinate system is the transformation formula (7) to (10) above using marker placement information including position information on the robot coordinate system for each corner position of the marker. It can be calculated in the same way as the calculation. Therefore, the positional relationship between the marker coordinate system of the registered marker and the robot coordinate system can be determined by using the coordinate transformation formula between the marker coordinate system and the robot coordinate system, which is calculated based on the marker placement information of the registered marker. Can be done. Therefore, by using this coordinate transformation formula, the positions of the unregistered marker and the registered marker located on the same marker coordinate system on the marker coordinate system are respectively converted to the positions on the same robot coordinate system. be able to.

The determining unit 117 determines whether the shape of the registered marker or the shape of the unregistered marker detected by the detecting unit 114 matches a predetermined shape. As the predetermined shape, the shape of one or more markers used for teaching the operation of the manipulator 3 can be arbitrarily registered.

The output unit 118 outputs various messages. Outputting a message includes, for example, displaying characters and outputting audio. Messages output by the output unit 118 are illustrated in (1) to (3) below.

(1) When it is determined that the shape of the registered marker or the shape of the unregistered marker does not match the predetermined shape, the output unit 118 outputs a message indicating that a marker that does not match the predetermined shape has been detected. .

(2) When the total number of registered markers and unregistered markers detected by the detection unit 114 exceeds a predetermined maximum number of markers, the output unit 118 outputs an information indicating that markers exceeding the upper limit number have been detected. Output a message.

(3) When the detection unit 114 detects an unregistered marker, the output unit 118 outputs a message to the effect that an unregistered marker has been detected.

An example of the operation of the photographing device 1 according to the embodiment will be described with reference to FIG. 6.

First, the control unit 11 of the photographing device 1 displays, for example, the input form F shown in FIG. 4 on the display unit 16 (step S101). The trigger for displaying the input form F may be, for example, an operation instruction from a worker, or a display instruction from the robot control device 2 to which the teach pendant TP is connected.

Next, the control unit 11 of the photographing device 1 registers position information on the robot coordinate system for each corner position of the marker based on the contents input in the input form F (step S102).

Subsequently, the photographing unit 14 of the photographing device 1 photographs an image including the work area of the manipulator 3 and a plurality of markers (step S103).

Subsequently, the control unit 11 of the photographing device 1 detects a marker from the image photographed in step S103 (step S104).

Subsequently, the control unit 11 of the photographing device 1 determines whether an unregistered marker is included in the plurality of markers detected in step S104 (step S105). If this determination is NO (step S105; NO), the control unit 11 of the photographing device 1 moves the process to step S107, which will be described later.

If it is determined in step S105 that an unregistered marker is included (step S105; YES), the control unit 11 of the imaging device 1, based on the marker arrangement information corresponding to the registered marker detected in step S104, Position information on the robot coordinate system for each corner position of the unregistered marker is registered (step S106).

Next, the control unit 11 of the photographing device 1 adjusts the camera coordinate system and the marker based on the corner position in the camera coordinate system and the marker coordinate system of the reference marker among the plurality of markers detected in step S104. A conversion formula for converting coordinates with the coordinate system is calculated (step S107).

Next, the control unit 11 of the photographing device 1 displays the model of the manipulator 3 on the display unit 16 in AR using the conversion formula calculated in step S107, and simulates the operation of the manipulator 3 using the model. (Step S108). Then, this operation ends.

As described above, according to the imaging device 1 according to the embodiment, an image including the work area of the manipulator 3 and a plurality of markers is acquired as data on the camera coordinate system, and existing markers are selected from among the markers included in the image. Detect registered markers and unregistered markers, set a marker coordinate system based on the detected registered marker, and set the camera coordinate system and marker coordinate system based on the corner position of the registered marker in the camera coordinate system and marker coordinate system, respectively. Calculate the conversion formula with the coordinate system, and use the conversion formula and position information on the robot coordinate system corresponding to the corner position of the registered marker to register the position information on the robot coordinate system of the unregistered marker. Can be done.

Therefore, by placing a registered marker and an unregistered marker in the work area of the manipulator 3 and capturing them in the same image, the position information on the robot coordinate system corresponding to the corner position of the unregistered marker can be acquired. It becomes possible to register based on the position information on the robot coordinate system corresponding to the corner position of the registration marker.

Therefore, according to the imaging device 1 according to the embodiment, it is possible to reduce the effort required to register the positions of a plurality of markers.

[Modified example]
Note that the present invention is not limited to the embodiments described above, and can be implemented in various other forms without departing from the gist of the present invention. Therefore, the above embodiments are merely illustrative in all respects, and should not be construed as limiting.

Furthermore, in the embodiment described above, the photographing device 1 has been described as including an image sensor (photographing section 14), a distance measurement sensor (coordinate acquisition section 15), and each component shown in FIG. 3, but the present invention is not limited to this. . For example, each component may be distributed to a plurality of devices and realized as a system including the plurality of devices. Illustratively, each component of the photographing device 1 shown in FIG. Based on this information, the position information of unregistered markers may be registered, and images, messages, etc. may be sent to the photographing device 1 via the network.

Further, in the embodiment described above, a corner identifier is given to the corner position of the marker, and the coordinates in the robot coordinate system corresponding to the tip of the tool 32t pointing to the corner position are input into the input form F, but the invention is not limited to this. . It is sufficient if a position identifier for specifying the position is given to a specific position of the marker, and coordinates in the robot coordinate system corresponding to the tip of the tool 32t pointing to the specific position can be input into the input form F.

This will be explained in detail with reference to FIG. The identification image Ia shown in the figure is given inside the periphery of the marker Ma, and "10" is given as a marker identifier for specifying the marker, and three specific positions of the marker Ma ( Numbers from "1" to "3" are assigned near the location as a location identifier for identifying a specific location.

Further, in the above-described embodiment, a welding robot is used, but the present invention is not limited to this. For example, the present invention can be applied to industrial robots including handling robots that perform picking, etc., and to general equipment that needs to move and align on a work area. When applied to devices in general, the welding robot and robot coordinate system used in the embodiments described above can be replaced with the target device and target device coordinate system, respectively.

DESCRIPTION OF SYMBOLS 1... Imaging device, 2... Robot control device, 3... Manipulator, 11... Control unit, 12... Storage unit, 13... Communication unit, 14... Imaging unit, 15... Coordinate acquisition unit, 16... Display unit, 21... Control unit , 22...Storage unit, 23...Communication unit, 24...Welding power supply unit, 31...Multi-joint arm, 32...Welding torch, 32t...Tool, 100...Welding robot system, 111...Display control unit, 112...Registration unit, 113 ...Image acquisition section, 114...Detection section, 115...Marker coordinate system setting section, 116...Calculation section, 117...Judgment section, 118...Output section

Claims (6)

an image acquisition unit that acquires position data of a subject reflected in an image including a work area of the target device and a plurality of markers as data on a three-dimensional camera coordinate system based on a photography unit that photographs the image;
Among the markers included in the image, there are already registered markers for which position information on the target device coordinate system is registered for each specific position of the marker, and for which position information on the target device coordinate system is not registered. a detection unit that detects an unregistered marker;
When the unregistered marker is detected by the detection unit, position information on the target device coordinate system corresponding to the unregistered marker is changed to position information on the target device coordinate system corresponding to the registered marker. a registration department to register based on;
A marker position registration device comprising: a coordinate system setting unit that sets a three-dimensional marker coordinate system based on the registered marker detected by the detection unit;
a conversion formula for converting the coordinates of the camera coordinate system and the marker coordinate system based on the specific position of the registered marker detected by the detection unit in each of the camera coordinate system and the marker coordinate system; a calculation section that calculates
Furthermore,
The registration unit uses the conversion formula calculated by the calculation unit and the position information on the target device coordinate system corresponding to the registered marker to determine the location on the target device coordinate system corresponding to the unregistered marker. register the location information of
The marker position registration device according to claim 1. an output unit that outputs a message to the effect that markers exceeding the upper limit number have been detected when the total number of the registered markers and the unregistered markers detected by the detection unit exceeds a predetermined maximum number of markers; further comprising;
The marker position registration device according to claim 1 or 2. further comprising an output unit that outputs a message indicating that the unregistered marker has been detected when the unregistered marker is detected by the detection unit;
The marker position registration device according to claim 1 or 2. a determination unit that determines whether the shape of the registered marker or the shape of the unregistered marker detected by the detection unit matches a predetermined shape;
If the determination unit determines that the shape of the registered marker or the shape of the unregistered marker does not match the predetermined shape, output a message to the effect that a marker that does not match the predetermined shape has been detected. an output section;
The marker position registration device according to claim 1 or 2, further comprising: computer,
an image acquisition unit that acquires position data of a subject reflected in an image including a work area of the target device and a plurality of markers as data on a three-dimensional camera coordinate system based on a photography unit that photographs the image;
Among the markers included in the image, there are already registered markers for which position information on the target device coordinate system is registered for each specific position of the marker, and for which position information on the target device coordinate system is not registered. a detection unit that detects unregistered markers;
When the unregistered marker is detected by the detection unit, position information on the target device coordinate system corresponding to the unregistered marker is changed to position information on the target device coordinate system corresponding to the registered marker. Registration Department to register based on;
A marker position registration program that functions as a marker location registration program.

Images