JP7325594B2 - CODE DISPLAY MEDIUM, INFORMATION PROCESSING DEVICE, METHOD AND PROGRAM - Google Patents

Description

The present disclosure relates to techniques for inspecting fasteners.

Conventionally, various techniques for inspecting fastening portions have been proposed (see Patent Documents 1 to 3). Also, various techniques related to deformed code images have been proposed (see Patent Documents 4 and 5).

JP-A-2002-239939 JP 2014-170446 A JP 2020-186925 A JP-A-8-77289 JP 2013-138757 A

In recent years, digital transformation (DX) related to inspection of joints that constitute structures such as steel towers has been promoted, and various technologies have been proposed that utilize codes that identify joints for inspection of joints. However, in the conventionally proposed technology, the code is hidden when viewed from an angle by the protruding fastening material (bolts, nuts, etc.), or there is little free space for displaying the code around the fastening part. For other reasons, there was a limit to where the code can be displayed.

In view of the above-described problems, an object of the present disclosure is to display a code for identifying a fastening portion and to identify the fastening portion using the code without being restricted by the display location.

In one example of the present disclosure, a plurality of code images each including identification information for identifying at least one of a fastening material for screw fastening and an object to be fastened by the screw fastening are displayed around the screw fastening axis. A code image display unit that is arranged in a circumferential direction and displayed in an optically readable manner, and the code image display unit are fixed at a position where an image can be taken simultaneously with the fastening material in a screwed state among the fastening objects. and a code display medium.

An example of the present disclosure includes a code display medium fixed to a fastening target, a fastening material fastened to the fastening target, and a captured image acquiring means for acquiring a captured image simultaneously captured, and the captured image. code image detection means for detecting the code image from the code image; identification information acquisition means for acquiring identification information for identifying at least one of the fastening material and the fastening target by decoding the code image; and a construction state determination unit that determines the construction state of the fastening portion based on the images of the fastening material and the fastening target included in the captured image.

The present disclosure can be understood as an information processing device, a system, a method executed by a computer, or a program executed by a computer. The present disclosure can also be understood as recording such a program in a recording medium readable by a computer, other device, machine, or the like. Here, a computer-readable recording medium is a recording medium that stores information such as data and programs by electrical, magnetic, optical, mechanical or chemical action and can be read by a computer. say.

According to the present disclosure, it is possible to display a code for identifying a fastening portion and to identify the fastening portion using the code without being restricted by the display location.

It is a figure which shows the structure of the code|cord|chord display medium which concerns on embodiment. FIG. 10 is a plan view showing the fastening portion in a state where the code display medium according to the embodiment is fixed to an object to be fastened and construction is correctly completed. FIG. 10 is a perspective view showing the fastening portion in a state where the code display medium according to the embodiment is fixed to an object to be fastened and construction is correctly completed; FIG. 4 is a diagram showing the fastening portion in a state where the code display medium according to the embodiment is fixed to an object to be fastened and a portion of the code display medium is cut off according to the shape of the periphery of the fastening portion. FIG. 4 is a diagram showing a plurality of code images in Cartesian coordinates before being converted to polar coordinates, according to an embodiment; 4 is a flow chart showing a procedure for constructing a fastening portion using the code display medium according to the embodiment; 1 is a schematic diagram showing the configuration of a system according to an embodiment; FIG. It is a figure which shows the outline of the functional structure of the information processing apparatus which concerns on embodiment. 6 is a flowchart showing the flow of inspection processing according to the embodiment; It is a figure which shows the structure of the code|cord|chord display medium which concerns on a 1st variation. It is a figure which shows the structure of the code|cord|chord display medium which concerns on a 2nd variation.

Hereinafter, embodiments of a code display medium, an information processing device, a method, and a program according to the present disclosure will be described based on the drawings. However, the embodiments described below are examples of embodiments, and the code display medium, information processing device, method, and program according to the present disclosure are not limited to the specific configurations described below. . For implementation, a specific configuration may be appropriately adopted according to the mode of implementation, and various improvements and modifications may be made.

In the present embodiment, an embodiment will be described in which the technology according to the present disclosure is implemented for construction, inspection, and management of a fastening portion that constitutes a structure such as a steel tower. However, the technology according to the present disclosure can be widely used for construction, inspection, and management of fastening portions, and the application target of the present disclosure is not limited to the examples shown in the embodiments. Also, in the present embodiment, an example using bolts, nuts, and washers as fastening materials for screw fastening will be described, but fastening materials to which the technology according to the present disclosure can be applied are not limited to the examples in the present disclosure. For example, screws that do not use nuts or washers can also be used as fasteners to which the technology according to the present disclosure can be applied.

BACKGROUND ART In recent years, digital transformation (DX) related to inspection of fastening parts constituting structures such as steel towers has been promoted. The DX proposes, as an example, an inspection technique that makes full use of image processing and image analysis of the fastening portion. This inspection technology is expected to have the effect of reducing the labor and burden of manual work for inspecting the joints, and it is also expected to facilitate the identification and abnormality detection of each joint. However, with the conventionally proposed technology, the cord is hidden when viewed from an angle by the protruding fasteners (bolts, nuts, etc.), or there is little free space around the fasteners to fix the cord. For some reason, there are various problems such as restrictions on the installation location of the code display medium, an increase in the number of parts, and need for position data for management.

According to the code display medium, information processing device, method, and program according to the present embodiment, in view of such circumstances, there are few restrictions on the installation location, it can be realized with a small number of parts, and a large number of parts are required for inspection. It is possible to provide DX for inspection of fastening parts that does not require data.

<Configuration of code display medium>
FIG. 1 is a diagram showing the configuration of a code display medium 5 according to this embodiment. The code display medium 5 according to this embodiment includes a code image display portion 51 and a fixing portion 52 provided on the back surface of the code image display portion 51 .

The code image display unit 51 arranges a plurality of code images in the circumferential direction around the axis of screw fastening and displays them in an optically readable manner. These multiple code images are code images each including identification information for identifying at least one of the fastening material 7 for screw fastening and the fastening object 6 for screw fastening. According to the code display medium 5 according to the present disclosure, by arranging a plurality of code images each including identification information side by side, even if some of the codes become unreadable due to stains or the like, any Identification information can be obtained from the code image. In this embodiment, the code image is generated by encoding data including identification information into a so-called two-dimensional barcode. However, the format of the code is not limited in implementing the technology according to the present disclosure, and a so-called one-dimensional barcode, circular barcode, character string, or the like may be adopted as the code.

In this embodiment, an example in which the code image display unit 51 displays a plurality of identical code images will be described. They can be different from each other. For example, the code images may differ from each other because they contain information other than the identification information, or they may differ from each other because the code images are encoded in different ways.

The fixing part 52 fixes the code image display part 51 to a position (in other words, near the fastening material 7 ) of the fastening object 6 where it can be imaged simultaneously with the fastening material 7 in a screwed state. In this embodiment, by adopting an adhesive surface as the fixing portion 52, the code display medium 5 as a whole is configured as a sticker, and the work of fixing the code display medium 5 during construction is simplified. However, the specific fixing means that can be employed are not limited to the examples in this embodiment. The code image display portion 51 is directly engraved (laser engraving, etc.) or printed on the object to be fastened, or the code image display portion 51 is fitted into the portion to be fixed provided on the object to be fastened, or the code image is displayed. Various fixing means can be employed, such as mechanically connecting the portion 51 to the object to be fastened.

2 and 3 are a plan view and a perspective view showing the fastening portion in a state where the code display medium 5 according to this embodiment is fixed to the fastening object 6 and construction is correctly completed. In the present embodiment, the code image display portion 51 has an arc shape surrounding a portion of the fastening material 7 or a donut shape surrounding the entire fastening material 7 . More specifically, in this embodiment, the code image display portion 51 includes a first circular arc having a size capable of surrounding part or all of the fastening material 7 and a circular arc having a diameter larger than that of the first circular arc. It has a shape consisting of at least a part of the region sandwiched between the second circular arc and the circular arc. In the example shown in FIG. 2, the code image display portion 51 surrounds most of the fastening material 7, but the code image display portion 51 has a shape that surrounds the fastening material 7 entirely or partially. It is good if there is According to FIG. 3, by adopting the code display medium 5 according to the present disclosure and arranging a plurality of code images each including identification information side by side in the circumferential direction, the fastening material 7 can be aligned when viewed obliquely. It can be seen that even if some code is hidden, the identification information can be obtained from any code image that is not hidden.

FIG. 4 shows that the code display medium 5 according to the present embodiment is fixed to the fastening object 6, and a part of the code display medium 5 is cut out according to the shape of the fastening part periphery (the wall 61 in the example shown in FIG. 4). FIG. 10 is a diagram showing the fastening portion in a folded state; That is, the code display medium 5 according to this embodiment can be cut according to the shape around the fastening portion before and after being fixed to the fastening object 6 . Even if a part of the code image is cut off, according to the code display medium 5 according to the present embodiment, since a plurality of code images including identification information are displayed on the code image display unit 51, the inspection process described later can be performed. It does not interfere with code reading and can be used in various construction locations.

FIG. 5 is a diagram showing a plurality of code images in rectangular coordinates before polar coordinate transformation according to the present embodiment. In the present embodiment, the code image display section 51 is fixed at a predetermined position by the fixing section 52 at an arbitrary point on the axis (in this embodiment, the code image display surface of the code image display section 51 and the axis). intersection) as the origin, and a code image converted to polar coordinates based on the origin is displayed (see FIG. 1). By converting the code image into polar coordinates according to the shape of the code display portion, the code image is displayed on the entire code display medium 5 as shown in FIG. 1 (in other words, the display area of the code image is expanded. ), facilitating recognition of the code image from the captured image, which will be described later.

<Installation procedure>
FIG. 6 is a flow chart showing the construction procedure of the fastening portion using the code display medium 5 according to this embodiment. In this embodiment, the construction procedure for fastening the fastening object 6 using the bolt 71, the nut 72, and the washer 73 as the fastening material 7 will be described. , are substantially the same except that part of the procedure may be changed as appropriate according to the type of fastener.

The worker installs the fastening material 7 on the fastening object 6 and temporarily tightens it (step S1), and performs primary fastening with a primary tightening torque determined according to the standard of the fastening material 7 (step S2). Then, the operator fixes the code display medium 5 to the fastening object 6 so as to surround the fastening material 7 (step S3). At this time, the center of gravity of the code display medium 5 (the center of the arc if the shape of the code display medium 5 is an arc) and the axis of the fastening material 7 (the axis of the bolt 71) should be aligned. is preferred. Moreover, as described above, the code display medium 5 may be cut according to the shape of the periphery of the fastening portion before and after being fixed to the fastening target 6 .

Then, the operator marks the bolt 71, nut 72, washer 73, and fastening target 6 in a state of being temporarily tightened (step S4). The reference display is a display for visualizing the relative angles of rotation of the bolt 71, the nut 72, the washer 73, and the fastening object 6. In general, the screw thread of the bolt 71 is displayed after the primary tightening is completed. A single straight line (hereinafter referred to as a "marker line") 4 passing through all of the bolt 71, nut 72, washer 73 and fastening object 6 is drawn using a pen or the like from the center outward. At this time, the marker line 4 for the fastening object 6 may be drawn across (hide) the code image on the code display section. According to the code display medium 5 according to the present disclosure, since a plurality of codes are displayed on the code display portion, even if one of the codes becomes illegible due to the marker line 4, identification of the fastening portion is hindered. do not have.

When fixing of the code display medium 5 and entry of the marker line 4 are completed, the operator performs secondary tightening (final tightening) (step S5). Specifically, the operator adjusts the nut 72 so that the angle of the nut 72 with respect to the bolt 71, the washer 73, and the fastening object 6 in the primary tightening state is within a predetermined range in the tightening rotation direction of the nut 72. and perform secondary tightening.

When the secondary tightening is completed, the construction state is inspected based on the captured image obtained by imaging the construction portion (step S6). The angle formed by the marker line 4 attached to the bolt 71, washer 73 and fastening object 6 and the marker line 4 attached to the nut 72 (hereinafter referred to as the "marker angle ”) is within a range of plus a predetermined angle in the tightening rotation direction of the nut 72 (see FIG. 2). Therefore, in the inspection of the construction state, the marker angle is calculated based on the captured image obtained by imaging the construction part, and the calculated marker angle is within a predetermined range in the tightening rotation direction of the nut 72. A determination is made as to whether or not Details of the inspection process will be described later using a flowchart.

<System configuration>
FIG. 7 is a schematic diagram showing the configuration of the system according to this embodiment. The system according to this embodiment includes an information processing device 1, a drone 8, and a user terminal 9, which are communicable with each other by being connected to a network.

The information processing device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an EEPROM (Electrically Erasable and Programmable Read Only Memory), an HD Storage such as D (Hard Disk Drive) It is a computer including a device 14, a communication unit 15 such as a NIC (Network Interface Card), and the like. However, the specific hardware configuration of the information processing apparatus 1 can be appropriately omitted, replaced, or added according to the mode of implementation. Further, the information processing device 1 is not limited to a device consisting of a single housing. The information processing device 1 may be realized by a plurality of devices using so-called cloud or distributed computing technology.

The drone 8 is a small unmanned aerial vehicle whose flight is controlled according to an input signal from the outside and / or a program recorded in the device, and includes a propeller, a motor, a CPU, a ROM, a RAM, a storage device, a communication unit, an input A device, an output device, etc. (not shown) are provided. However, the specific hardware configuration of the drone 8 can be appropriately omitted, replaced, or added according to the mode of implementation. In addition, the drone 8 according to the present embodiment includes an imaging device 81, and when flying around the target structure, the fastening portion is adjusted according to an input signal from the outside and/or a program recorded in the device. Take an image.

A user terminal 9 is a terminal device used by a user. The user terminal 9 is a computer including a CPU, ROM, RAM, storage device, communication unit, input device, output device, etc. (not shown). However, the specific hardware configuration of the user terminal 9 can be appropriately omitted, replaced, or added according to the mode of implementation. Also, the user terminal 9 is not limited to a device consisting of a single housing. The user terminal 9 may be realized by a plurality of devices using so-called cloud or distributed computing technology. Via these user terminals 9, users use various services provided by the system according to the present embodiment.

FIG. 8 is a diagram showing an outline of the functional configuration of the information processing apparatus according to this embodiment. In the information processing apparatus 1, a program recorded in the storage device 14 is read out to the RAM 13, executed by the CPU 11, and each hardware provided in the information processing apparatus 1 is controlled to acquire a captured image. It functions as an information processing device including a section 21 , a code image detection section 22 , an inverse conversion section 23 , an identification information acquisition section 24 , a construction state determination section 25 and a construction state management section 26 . Note that in this embodiment and other embodiments described later, each function of the information processing apparatus 1 is executed by the CPU 11, which is a general-purpose processor. It may be executed by a processor.

The captured image acquisition unit 21 acquires a captured image in which the code display medium 5 fixed to the fastening target 6, the fastening material 7 fastened to the fastening target 6, and the fastening target 6 are simultaneously captured. Although the acquisition method of the captured image is not limited, in the present embodiment, an example of acquiring the captured image captured using the drone 8 equipped with the imaging device 81 via the user terminal 9 will be described.

A code image detection unit 22 detects a code image from the captured image. The code image detection unit 22 can detect the code image in the captured image by detecting a predetermined feature displayed together with the code in the code image display unit 51 . As the predetermined feature, it is possible to use various features other than the so-called finder pattern used in the two-dimensional barcode. In addition, the display color of the whole or part of the code image (for example, the finder pattern) is a color different in tone from both the fastening object 6 and the fastening material 7 (for example, if the fastening object 6 and the fastening material 7 are gray, the display color is pink). or red), the code image may be easily detected. In this case, the code image detection section 22 can detect the code image by detecting a predetermined color from the captured image.

The inverse transformation unit 23 inversely transforms the code image that has undergone polar coordinate transformation and is projected at the time of imaging, which is included in the captured image. ). In the present embodiment, the inverse transformation unit 23 detects a predetermined reference image included in the captured image, and converts the captured image so that the reference image has a predetermined shape (shape before polar coordinate transformation and projection). The conversion inversely converts the code image included in the captured image. That is, the predetermined reference image is an image having characteristics that can be detected from the captured image, and the code image is obtained by transforming the captured image including the reference image so that the reference image has a predetermined shape. is an image having a shape that is inversely transformed. For the reference image, for example, a finder pattern included in the two-dimensional barcode may be used, or a predetermined pattern prepared separately from the code image and displayed on the code image display unit 51 as in variations described later. A reference image (see FIG. 10) may be used.

For the inverse transformation processing by the inverse transformation unit 23, the inverse transformation processing for obtaining the code image before the polar coordinate transformation by one transformation processing from the code image that has been subjected to the polar coordinate transformation and is further projected at the time of imaging may be employed. Alternatively, the code image that has been polar coordinate transformed and projected at the time of imaging is first subjected to projective transformation to a planar image to cancel the projection, and then orthogonal coordinate transformation is performed to cancel the polar coordinate transformation. An inverse transformation process may be employed to obtain a code image before being polar coordinate transformed in the process. A specific inverse transform process may be appropriately selected according to circumstances such as the amount of calculation. Note that when the code image included in the captured image has little distortion, part or all of the processing by the inverse transforming unit 23 may be omitted, and may be omitted as appropriate according to the embodiment.

The identification information acquisition unit 24 acquires identification information for identifying at least one of the fastening material 7 and the fastening target 6 by decoding the code image. A specific method for acquiring identification information from a code image differs depending on the code format, and can be appropriately adopted according to the adopted code format. Note that when a character string is adopted as the code format, the identification information may be acquired by optical character recognition (OCR).

The construction state determination unit 25 determines the construction state of the fastening portion based on the images of the fastening material 7 and the fastening target 6 included in the captured image. In this embodiment, the construction state determination unit 25 determines the construction state based on the reference marks attached to the fastening material 7 and the fastening target 6 . In this embodiment, a marker line 4 is used as a reference display. Specifically, the construction state determination unit 25 detects the marker lines 4 from the captured image, and the marker lines 4 attached to the bolt 71, the washer 73, and the fastening target 6, and the marker line 4 attached to the nut 72. Then, it is determined whether or not the calculated marker angle is within a range of plus a predetermined angle in the tightening rotation direction of the nut 72 . As a result of the determination, if the marker angle is within a range of plus a predetermined angle in the tightening rotation direction of the nut 72, the construction state determination unit 25 determines that the construction is correctly performed. On the other hand, if the marker angle is outside the range of the plus predetermined angle in the tightening rotation direction of the nut 72 as a result of the determination, the construction state determination unit 25 determines that the construction is not properly performed (due to improper construction, looseness due to aging, etc.). is determined).

Here, in order to correctly calculate the marker angle, it is preferable that the projections of the surfaces of the fastening material 7 and the fastening target 6 to which the reference indication is attached are correctly canceled. Since the marker lines 4 drawn on the bolt 71, the nut 72, the washer 73, and the fastening object 6 are on different planes, the inverse transformation for correctly deprojecting the code image is applied to the entire captured image. applied and the marker angle calculated based on this, an inaccurate marker angle may be calculated. Therefore, when there is a possibility that the projections of the marker lines 4 written on the fastening material 7 and the fastening target 6 are significantly different from each other (for example, when using an image captured from a short distance), each marker It is preferable to extract the plane associated with line 4 and unproject correctly. However, when the projection difference of the marker line 4 drawn on each of the fastening material 7 and the fastening object 6 is relatively small (for example, when using an image captured from a long distance), the marker angle is set to a predetermined value. If it is allowed within the range, the strict canceling of projection may be omitted by setting an allowable range of angles on the premise of errors due to projection.

As a method for detecting the plane of the nut 72 marked with the marker line, three or more vertices of a polygon (a hexagon in the case of a hexagonal nut) forming the upper surface of the nut 72 are detected by image recognition. A method of defining a plane of the nut 72 marked with a marker line can be employed based on the vertices. Here, if it is difficult to directly detect the vertices by image recognition, it is also possible to detect each side of the polygon forming the upper surface of the nut 72 and treat the intersection of each detected side as the vertex. is.

The construction state management unit 26 associates and accumulates the identification information acquired by the identification information acquisition unit 24 and the construction state determined by the construction state determination unit 25, thereby accumulating one or more fasteners included in the structure. Manage the construction status of the department. In addition, the construction state management unit 26 may further accumulate the marker angle calculated by the construction state determination unit 25 as the construction state together with information indicating whether or not the construction state is correct.

<Process flow>
Next, the flow of processing executed by the information processing apparatus according to this embodiment will be described. Note that the specific content and processing order of the processing described below are examples for carrying out the present disclosure. Specific processing contents and processing order may be appropriately selected according to the embodiment of the present disclosure.

FIG. 9 is a flowchart showing the flow of inspection processing according to this embodiment. The processing shown in this flow chart is performed when a captured image is input in an inspection step (step S6) in the construction procedure or an inspection performed periodically (for example, every year) after construction or by an inspection process performed by an operator. is executed when an execution instruction is input.

In step S101, a captured image is acquired. The operator takes an image of the fastening portion using the imaging device 81 and inputs the image data of the obtained captured image to the information processing device 1 . At this time, the operator may take an image so that a plurality of fastening portions are included in one captured image. When a plurality of fastening portions are included in one captured image, inspection processing is performed for each fastening portion included in the captured image. The imaging method and the input method of image data to the information processing device 1 are not limited, but in the present embodiment, a drone 8 equipped with an imaging device 81 is used to image a plurality of fastening parts constructed on a structure, The image data transferred from the imaging device 81 to the user terminal 9 via communication or a recording medium is further transferred to the information processing device 1 via the network, whereby the image data of the captured image is input to the information processing device 1. be. After the captured image is acquired by the captured image acquisition unit 21, the process proceeds to step S102.

In steps S102 to S104, identification information is acquired from the code image included in the captured image. The code image detection unit 22 detects a code image from the captured image (step S102), and the inverse transformation unit 23 inversely transforms the captured image that has undergone polar coordinate transformation and is projected at the time of imaging. A code image before being read is obtained (step S103). Then, the identification information obtaining unit 24 obtains identification information for identifying at least one of the fastening material 7 and the fastening target 6 by decoding the code image (step S104). As described above, when the code image included in the captured image has little distortion, part or all of the processing in step S102 may be omitted, and may be omitted as appropriate according to the embodiment. After the identification information is obtained, the process proceeds to step S105.

In steps S105 and S106, the construction state is determined. The construction state determination unit 25 determines the construction state of the fastening portion based on the images of the fastening material 7 and the fastening target 6 included in the captured image (step S105). Then, the construction state management unit 26 associates and accumulates the identification information acquired in step S104 with the marker angle and the construction state obtained in step S105, so that one or more fastening portions included in the structure is managed (step S106). Also, the determination result in step S105 may be notified to the user terminal 9 . After that, the processing shown in this flowchart ends.

<effect>
According to the embodiment described above, it is possible to display the code for identifying the fastening portion and to identify the fastening portion using the code without being restricted by the display location.

<Variation>
FIG. 10 is a diagram showing the configuration of the code display medium 5b according to the first variation. In the above-described embodiments, an example has been described in which the finder pattern included in the two-dimensional barcode is used as the predetermined feature for detecting the code image and/or the reference image for the inverse transform. And/or other images may be used as the reference image. In the example shown in FIG. 10, an image 54 resembling a finder pattern of a two-dimensional bar code is arranged at both ends of a code image display portion 51 having an arc shape, thereby allowing the code image detection portion 22 to detect the code image. and a reference image 54 for the inverse transform processing by the inverse transform unit 23 . More specifically, the inverse transformation unit 23 extracts a plurality of reference points 54a-54l from the reference image 54 and performs inverse transformation processing based on the extracted reference points.

FIG. 11 is a diagram showing the configuration of the code display medium 5c according to the second variation. In the embodiment described above, the angle formed by the marker line 4 attached to the bolt 71, washer 73, and fastening object and the marker line 4 attached to the nut 72 is used as the marker angle for determining the construction state. Although examples have been described, the marker angles may be calculated using other methods. In the example shown in FIG. 11, the position of the code display medium 5c relative to the reference mark attached to the fastening material 7 (for example, the marker line 4 drawn by the operator on the bolt 71, nut 72, washer 73, and fastening object). It further comprises a marker section 53 that enables determination of the construction state based on the relationship.

In this case, when fixing the code display medium 5c to the object to be fastened, the operator aligns the reference indication (marker line 4 or the like) attached to the fastening material 7 with the marker portion 53. FIG. Then, in the inspection process, the construction state determination unit 25 determines the construction state based on the positional relationship between the reference indication attached to the fastening material 7 and the marker portion 53 provided on the code display medium 5c. Specifically, the construction state determination unit 25 detects the reference display (such as the marker line 4) and the marker portion 53 from the captured image, and calculates the marker angle formed by the detected reference display and the marker portion 53. It is determined whether or not the marker angle obtained is within the range of plus a predetermined angle in the tightening rotation direction of the nut 72 . In other words, in this variation, the marker portion 53 replaces the reference indication written on the object to be fastened in the conventional construction.

Various configurations that can be employed for the code display media described above with reference to the code display media 5, 5b, and 5c may be appropriately selected and combined according to the embodiment. For example, the reference image 54 described with reference to FIG. 10 and the marker portion 53 described with reference to FIG. 11 may be used in combination. Also, an image that serves as both a reference image and a marker portion may be used.

Further, in the above-described embodiment, the method using the marker line 4 drawn by the operator as the reference indication was explained, but other indications may be used as the reference indication. For example, projections, cutouts, prints, etc. previously attached to the fastening material 7 such as the bolt 71, the nut 72, and the washer 73 may be used as reference indications. Conventionally, as a fastening material 7 having such a reference display, there is known a fastening material 7 which has an asymmetrical appearance by providing a projection or the like on a part thereof, and whose rotation state can be determined from an image (for example, a so-called smart bolt). In addition, when using the fastening material 7 partially provided with projections, the code image display part 51 may be hidden by the projections in the installed state, but according to the code display medium according to the present disclosure Since a plurality of codes are displayed on the code display portion, even if one of the codes is hidden by the protrusion and becomes illegible, the identification of the fastening portion is not hindered. Further, the code display medium may be fixed so as to cover the protrusion of the fastening material 7, or may be fixed at a position away from the protrusion so as not to be hidden by the protrusion.

In the above-described embodiment, the method of using the reference image on the code display medium in the inverse transformation of the captured image has been described, but the inverse transformation of the captured image may be processed using other methods. For example, all or part of an arc that is a perfect circle in plan view but is an ellipse in the captured image due to projection is extracted from the captured image, and the captured image is inversely transformed by ellipse fitting that transforms this into a perfect circle. methods may be employed. Here, examples of an arc that is a perfect circle in a plan view but an ellipse in a captured image due to projection include the edge that is the boundary line between the code display medium and the fastening object (base plate, etc.), and the edge of the code display medium. An edge (periphery or inner periphery), an arc printed on a code display medium, an edge of the fastening material 7 (washer 73 or the like), and the like are included.

In the inverse transformation unit 23, the center of gravity of the code display medium assuming a 360-degree circumference substantially coincides with the center of gravity of the fastening portion on the plane to which the code display medium is fixed. Ellipse fitting is performed on the assumption that the center of gravity is known in advance. Since the ellipse estimated from the photographed edge (arc) changes greatly depending on the center of gravity (center), it is desirable that the center of gravity of the code display medium can be picked up from the center of gravity of the fastening portion.

The center of gravity of the fastening portion on the plane to which the code display medium is fixed (hereinafter referred to as the “target plane”) is from the center of the fastening material 7, such as the center of the screw thread of the bolt 71 or the center of the nut 72 specified from the captured image. It is possible to guess. However, since the center of the fastening material 7 is not on the plane of interest, it should be noted that different projections are applied. For example, the inverse transformation unit 23 estimates the center of the fastening material 7 by performing elliptical fitting based on the shape of the fastening material 7 such as the thread of the bolt 71 and the nut 72, and determines the estimated center of the fastening material 7. Based on the distance to the object plane, the center of gravity of the joint on the object plane is inferred. At this time, since the distance between the center of the fastening material 7 and the target plane becomes substantially the same distance according to the standard of the fastening material 7 and the shape of the fastening target, it is possible to hold it in advance.

In addition, since the code image included in the captured image is usually rotated as well as projected, the image cannot be erected simply by canceling the projection and canceling the polar coordinate transformation. Therefore, if the identification information acquisition unit 24 does not support decoding of a code image that is not upright, the inverse transformation unit 23 rotates the code image in addition to canceling the projection and the polar coordinate transformation. It is also possible to perform a conversion that is established by

Further, in the above-described embodiment, an example of displaying a plurality of code images that have undergone polar coordinate conversion on the code image display section 51 has been explained. is not limited to For example, a plurality of code images (in rectangular coordinates) that have not undergone polar coordinate conversion may be displayed on the code image display section 51 . Even in this example, since a plurality of code images are arranged side by side in the circumferential direction, some of the codes may be hidden by the fastening materials 7 such as the bolts 71 and nuts 72 when viewed obliquely. Also, even if some of the codes become unreadable due to staining or the like, the identification information can be obtained from any of the code images.

1 information processing device 5 code display medium

Claims (7)

A plurality of code images each including identification information for identifying at least one of a fastening material for screw fastening and an object to be fastened by the screw fastening are arranged in a circumferential direction around the screw fastening axis, a code image display unit that displays an optically readable code image display unit; , a captured image acquiring means for acquiring a captured image in which the code display medium fixed to the fastening object , the fastening material fastened to the fastening object, and the fastening object are simultaneously imaged;
code image detection means for detecting the code image from the captured image;
identification information acquisition means for acquiring identification information for identifying at least one of the fastening material and the fastening object by decoding the code image ;
Information processing device. further comprising inverse transformation means for obtaining the code image before the polar coordinate transformation by inversely transforming the code image included in the captured image;
the identification information obtaining means obtains the identification information by decoding the inversely transformed code image;
The information processing device according to claim 1 . The inverse transforming means inversely transforms the code image included in the captured image based on a predetermined reference image included in the captured image.
The information processing apparatus according to claim 2 . The inverse transforming means inversely transforms the code image included in the captured image based on an arc in the captured image that is a perfect circle in plan view but an ellipse by projection.
The information processing apparatus according to claim 2 . The inverse transformation means projectively transforms the code image included in the captured image into a plane image, cancels the polar coordinate transformation of the code image included in the plane image, and transforms the code image before the polar coordinate transformation into a plane image. to get
The information processing apparatus according to any one of claims 2 to 4 . the computer
A plurality of code images each including identification information for identifying at least one of a fastening material for screw fastening and an object to be fastened by the screw fastening are arranged in a circumferential direction around the screw fastening axis, a code image display unit that displays an optically readable code image display unit; , a captured image acquiring step of acquiring captured images of the code display medium fixed to the fastening target, the fastening material fastened to the fastening target , and the fastening target simultaneously;
a code image detection step of detecting the code image from the captured image;
an identification information obtaining step of obtaining identification information for identifying at least one of the fastening material and the fastening target by decoding the code image ;
Information processing method that performs the computer,
A plurality of code images each including identification information for identifying at least one of a fastening material for screw fastening and an object to be fastened by the screw fastening are arranged in a circumferential direction around the screw fastening axis, a code image display unit that displays an optically readable code image display unit; , a captured image acquiring means for acquiring a captured image in which the code display medium fixed to the fastening object , the fastening material fastened to the fastening object, and the fastening object are simultaneously imaged;
code image detection means for detecting the code image from the captured image;
identification information acquiring means for acquiring identification information for identifying at least one of the fastening material and the fastening object by decoding the code image ;
Information processing program that functions as

Images