JP2021009560A - Equipment detecting device and equipment detecting method - Google Patents

Abstract

To provide an equipment detecting device capable of efficiently detecting objects installed in a wide range.SOLUTION: According to one embodiment, an equipment detecting device comprises an imaging unit which acquires an image captured by an imaging device moving in a building, and a detecting unit which detects an equipment identification graphic, which is provided for identifying equipment in the building, from the captured image. The equipment detecting device further includes: an extraction unit which compares first identification information obtained from the equipment identification graphic with second identification information stored in a storage unit as identification information of the equipment, and extracts the second identification information corresponding to the first identification information; a determination unit which determines position information indicating a position of the equipment in the building on the basis of information output from the imaging device or a device moving with the imaging device when the second identification information corresponding to the first identification information is output; and a registration unit which registers the identification information of the equipment in association with the determined position information.SELECTED DRAWING: Figure 3

Description

Embodiments of the present invention relate to a device detection device and a device detection method.

A technique for detecting an object registered in a database by detecting a two-dimensional code existing in an image is known.

Japanese Patent No. 4724800 Japanese Unexamined Patent Publication No. 2018-16489

However, in the above technique, there is a problem that the camera needs to be installed at a distance close to the object. In this case, it is necessary to install many cameras in order to detect an object installed in a wide range.

Therefore, it is an object of the present embodiment of the present invention to provide a device detection device and a device detection method capable of efficiently detecting an object installed in a wide range.

According to one embodiment, the device detection device includes an imaging unit that acquires an image captured by an imaging device moving in a building. The device detection device further includes a detection unit that detects a device identification figure, which is a figure provided in the vicinity of the device or the device for identifying the device in the building, from the captured image. The device detection device further collates the first identification information obtained from the device identification figure with the second identification information stored in the storage unit as the device identification information, and the first identification device. An extraction unit that extracts the second identification information corresponding to the identification information from the storage unit is provided. The device detection device is further output from the information output from the image pickup device or from a device moving with the image pickup device when the second identification information corresponding to the first identification information is extracted. A determination unit for determining position information indicating the position of the device in the building is provided based on the information. The device detection device further includes a registration unit that registers the identification information of the device and the position information determined by the determination unit in association with each other.

It is a block diagram which shows the structure of the device detection system of 1st Embodiment. It is a block diagram which shows the structure of the device detection apparatus of 1st Embodiment. It is a figure which shows the structure of the device detection system of 1st Embodiment. It is a figure which shows the example of the captured image output from the image pickup apparatus of 1st Embodiment, and the example of the device marker of 1st Embodiment. It is a figure which shows the example of the captured image output from the imaging apparatus of 1st Embodiment. It is a figure which shows the example of the stored data in the database of 1st Embodiment. It is a flowchart for demonstrating the apparatus detection method of 1st Embodiment. It is a flowchart for demonstrating the detail of the apparatus detection method of 1st Embodiment. It is a figure which shows the structure of the device detection system of the 1st modification of 1st Embodiment. It is a figure which shows the example of the captured image output from the imaging apparatus of the 1st modification of 1st Embodiment. It is a figure which shows the example of the position coordinate table of the 1st modification of 1st Embodiment. It is a flowchart for demonstrating the apparatus detection method of the 1st modification of 1st Embodiment. It is a block diagram which shows the structure of the device detection apparatus of 2nd Embodiment. It is a flowchart for demonstrating the apparatus detection method of 2nd Embodiment. It is a block diagram which shows the structure of the device detection apparatus of 3rd Embodiment. It is a figure which shows the example of the connected image of 3rd Embodiment. It is a flowchart for demonstrating the apparatus detection method of 3rd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In FIGS. 1 to 17, the same or similar configurations are designated by the same reference numerals, and redundant description will be omitted.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration of the device detection system of the first embodiment.

As shown in FIG. 1, the device detection system of the present embodiment includes a device detection device 1, a device information management device 2, and one or more image pickup devices 3. FIG. 1 shows three image pickup devices 3a to 3c as an example of one or more image pickup devices 3. FIG. 1 further shows a device marker 4 and a position marker 5 to be imaged by the image pickup device 3, and a crane 6 for moving the device detection device 1 and the image pickup device 3. The device marker 4, the position marker 5, and the crane 6 are examples of a device identification figure, a position identification figure, and a moving body, respectively. Details of the equipment marker 4, the position marker 5, and the crane 6 will be described later.

As will be described later, the device detection system of the present embodiment is provided in a building in which a plurality of devices are arranged, and operates so as to detect these devices. An example of a building is a building built on a manufacturing site that manufactures industrial products, and an example of equipment is a control panel for manufacturing industrial products. The device detection device 1 acquires a captured image captured in the building by each imaging device 3, detects the device based on the device marker 4 in the captured image, and bases the device on the position marker 5 in the captured image. Detect the position. The device information management device 2 manages information necessary for the operation of the device detection device 1 and information output from the device detection device 1.

The device detection device 1 is, for example, a computer such as a PC (Personal Computer) or a server device. The device detection device 1 includes an input / output device 11 such as a keyboard, a mouse, a display, and a communication device, a computing device 12 such as a processor and a ROM (Read Only Memory), and a storage device such as an HDD (Hard Disc Drive) and a semiconductor memory. A 13 and a program 14 in the storage device 13 are provided. The device detection device 1 can communicate with each image pickup device 3 by wire communication or wireless communication.

The device information management device 2 is, for example, a computer such as a PC or a server device. The device information management device 2 includes an arithmetic unit 21 such as a processor and a ROM, a storage device 22 such as an HDD and a semiconductor memory, and a database 23. The database 23 may be included in the storage device 22 or may be provided separately from the storage device 22. Further, the database 23 may be composed of one device or a plurality of devices. Database 23 is an example of a storage unit. The device information management device 2 can communicate with the device detection device 1 by wired communication or wireless communication. The device detection device 1 and the device information management device 2 of the present embodiment are connected to each other via a network.

FIG. 2 is a block diagram showing the configuration of the device detection device 1 of the first embodiment.

As shown in FIG. 2, the device detection device 1 of the present embodiment includes an imaging unit 31, a marker detection unit 32, an identification unit 33, a collation unit 34, a position estimation unit 35, a registration unit 36, and a storage unit. It is provided with a unit 37. These blocks are realized, for example, by executing the above-mentioned program 14 in the device detection device 1. Details of these blocks will be described later. The identification unit 33 and the collation unit 34 are examples of the extraction unit, and the position estimation unit 35 is an example of the determination unit.

FIG. 3 is a diagram showing the configuration of the device detection system of the first embodiment.

FIG. 3 is a perspective view showing the overall configuration of the device detection system of the present embodiment, the floor 41 of the building described above, the passage 42 provided in the center of the floor 41, and a plurality of passages arranged on the floor 41. The device 43 and the vehicle 44 that carries the device 43 are shown. An example of the device 43 is a control panel, and an example of the vehicle 44 is a truck. FIG. 3 further shows the x and y directions parallel to the floor 41 and perpendicular to each other, and the z direction perpendicular to the floor 41.

In the present embodiment, as the manufacturing process of a certain product progresses, the installation location of the device 43 used for manufacturing the product is moved. In the present embodiment, the installation location of each device 43 is managed by the device detection device 1 or the like. As a result, it is possible to reduce the labor required for managing the device 43 and to prevent the manufacturing work from being delayed due to the delay in the management of the device 43.

FIG. 3 further shows the device detection device 1, the device information management device 2, the image pickup device 3 (imaging devices 3a to 3c), the plurality of device markers 4, the plurality of position markers 5, and the crane 6. There is.

The device detection device 1 and the image pickup device 3 are attached to a crane 6 that can move in the building. FIG. 3 shows a crane 6 trying to move in the y direction above the floor 41. On the other hand, the device information management device 2 is installed in a stationary state in the building. The image pickup device 3 may move in the building by means other than the crane 6.

The device information management device 2 may be installed in a building different from the building shown in FIG. Such a configuration can be adopted, for example, when the information of the devices 43 in a plurality of buildings is managed by the one device information management device 2. In this case, it is conceivable to arrange the device detection device 1 in each building and configure the device detection system with a plurality of device detection devices 1 and one device information management device 2.

The device marker 4 is a mark used to identify the device 43, and is provided on each device 43. In the present embodiment, one device marker 4 printed on paper is attached to the upper part of each device 43. Device markers 4 holding different identifiers are affixed to the upper part of one device 43 and the upper part of another device 43 in the building. The device marker 4 may be any graphic as long as it can hold such an identifier, and may be, for example, a one-dimensional barcode, a two-dimensional barcode, or a QR code. Further, the figure referred to here may be one or more letters, numbers, symbols, or a combination thereof. The device marker 4 of the present embodiment is a color marker in which any of a plurality of types of colors is arranged in each of the cells divided into m × n.

As described above, the device marker 4 of the present embodiment is provided on the upper part of the device 43 to be managed. However, if the device 43 to be managed does not have a ceiling plate, the device marker 4 of the device 43 may be provided on a desk near the device 43. The orientation of the device marker 4 may be set in any way. For example, the long side of the device marker 4 may be parallel to the x direction, or the long side of the device marker 4 may be parallel to the y direction. Further, since the device marker 4 is an image pickup target of the image pickup device 3, the device marker 4 is arranged between the device marker 4 and the image pickup device 3 so as not to obstruct the image pickup. The surface of the device marker 4 may be processed to suppress light reflection. Further details of the device marker 4 will be described later.

The position marker 5 is a mark used for identifying a position in the building, and is provided at a plurality of places in the building. The position marker 5 of the present embodiment is printed on paper and affixed on the passage 42 of the building. In the present embodiment, by combining the detection result of the device marker 4 and the detection result of the position marker 5, it is possible to determine (estimate) at which position in the building each device 43 is installed. Position markers 5 holding different identifiers are affixed to one place and another place in the building. The position marker 5 may be any graphic as long as it can hold such an identifier, and may be, for example, a one-dimensional barcode, a two-dimensional barcode, a QR code, or an AR code. Further, the figure referred to here may be one or more letters, numbers, symbols, or a combination thereof. The position marker 5 of the present embodiment is a color marker like the device marker 4.

The orientation of the position marker 5 may be set in any way. Further, since the position marker 5 is an image pickup target of the image pickup device 3, the position marker 5 is arranged so as not to cause a shield that hinders the image pickup between the position marker 5 and the image pickup device 3. The surface of the position marker 5 may be processed to suppress light reflection. The position markers 5 may be arranged at equal intervals or may be arranged at non-equal intervals. Each position marker 5 may be imaged by any of the image pickup device 3a, the image pickup device 3b, and the image pickup device 3c. Further details of the position marker 5 will be described later.

The image pickup device 3 is attached to the crane 6 at an angle at which the device marker 4 and the position marker 5 can be imaged. The image pickup apparatus 3 of the present embodiment is attached to the crane 6 at an angle facing the floor 41. The imaging devices 3a to 3c may image regions that overlap each other. The image pickup device 3 is, for example, a CCTV (Closed-Circuit Television) camera, but other image pickup devices may also be used. The image pickup device 3 may be a device for capturing a moving image or a device for capturing a still image.

FIG. 3 shows three image pickup devices 3a to 3c as an example of the image pickup device 3. The device detection system of the present embodiment includes a plurality of image pickup devices 3 in order to cover the entire range in which the device marker 4 and the position marker 5 may be installed as an image pickup range. The number of image pickup devices 3 of the device detection system may be set to an arbitrary number according to the resolution of the image pickup device 3 and the installation range of the device marker 4 and the position marker 5. Therefore, the number of image pickup devices 3 in the device detection system may be other than three.

Here, it should be noted that the imaging devices 3a to 3c of the present embodiment move in the building by the crane 6. These image pickup devices 3a to 3c cannot capture the entire installation range of the device marker 4 and the position marker 5 while stationary at one place. However, when these imaging devices 3a to 3c move in the building, it is possible to image the entire installation range of the device marker 4 and the position marker 5.

The device detection device 1 of the present embodiment acquires the captured image captured by the image pickup device 3, detects the device marker 4 and the position marker 5 from the captured image, and detects the detection result of the device marker 4 and the position marker 5. By combining the results, the installation position of each device 43 is determined (estimated). Prior to this determination, the identifier obtained from the device marker 4 and the identifier in the database 23 are collated, and the identifier obtained from the position marker 5 and the identifier in the database 23 are collated. An example of an identifier obtained from the device marker 4 is a registration number of the device 43, and an example of an identifier obtained from the position marker 5 is information about a section in a building. The result of the above determination is registered in the database 23, and the information in the database 23 is updated. In this way, the device detection system of the present embodiment can automatically manage the position of each device 43.

Examples of the captured image captured by the imaging device 3 include the captured image 7a of FIG. 4A, the captured image 7b of FIG. 5A, the captured image 7c of FIG. 5B, and FIG. 10 as described later. The captured image 7d of (a), the captured image 7e of FIG. 10B, and the like. These figures show the X direction (X coordinate) corresponding to the x direction in the building and the Y direction (Y coordinate) corresponding to the y direction in the building. The device detection system of the present embodiment manages the positions of the device marker 4 and the position marker 5 by the X coordinate and the Y coordinate. Hereinafter, when an arbitrary captured image using these captured images 7a to 7e as an example is described, this will be referred to as “captured image 7”.

Hereinafter, the details of the imaging unit 31, the marker detection unit 32, the identification unit 33, the collation unit 34, the position estimation unit 35, the registration unit 36, and the storage unit 37 of the device detection device 1 will be described with reference to FIG. In this description, FIGS. 4 to 6 will also be referred to as appropriate.

The imaging unit 31 acquires the captured image 7 captured by the imaging device 3. The imaging device 3 of the present embodiment moves to a plurality of locations in the building together with the crane 6, and outputs the captured images 7 captured at these locations to the imaging unit 31. The imaging unit 31 may store the captured image 7 in the semiconductor memory or may store it in the HDD.

The marker detection unit 32 detects the region where the device marker 4 exists and the region where the position marker 5 exists from the captured image 7. The device marker 4 of the present embodiment is composed of m × n cells and an outer frame surrounding these cells. Each cell is filled with one of six colors (red, blue, green, yellow, cyan, magenta). The outer frame is filled with a color that is separable from the background color. The position marker 5 of the present embodiment also has the same configuration as the device marker 4. The marker detection unit 32 detects the device marker 4 and the position marker 5 from the captured image 7 by, for example, an image processing method such as color extraction.

The device marker 4 holds the identifier of the device 43 (registration number of the device 43) according to the color of the cells constituting the device marker 4. The identifier of the device 43 is also stored in the database 23 and is used to collate with the identifier obtained from the device marker 4. The identifier held by the device marker 4 is an example of the first identification information, and the identifier stored in the database 23 for collation with this is an example of the second identification information.

Similarly, the position marker 5 holds an identifier of a position in the building (information about a section in the building) by the color of the cells constituting the position marker 5. The identifier of the position in the building is also stored in the database 23 and is used for collating with the identifier obtained from the position marker 5. The identifier held by the position marker 4 is an example of the third identification information, and the identifier stored in the database 23 for collation with this is an example of the fourth identification information.

FIG. 4 is a diagram showing an example of an image captured image 7 output from the image capturing apparatus 3 of the first embodiment and an example of the device marker 4 of the first embodiment.

FIG. 4A shows the captured image 7a output from any of the imaging devices 3 as an example of the captured image 7. The captured image 7a includes two devices 43 (devices 43a and 43b) arranged on the floor 41 and two device markers 4 (device markers 4a and 4b) attached to the upper part of these devices 43. Includes. FIG. 4B is an enlarged view showing the device marker 4a.

The marker detection unit 32 extracts an image range that matches the color to be extracted from the captured image 7a. The marker detection unit 32 may use an RGB color space or an HSV color space as a method for identifying colors used for extracting an image range. The marker detection unit 32 generates an image in which the image range other than the color to be extracted is converted to black from the captured image 7a. The marker detection unit 32 of the present embodiment performs image processing such as binarization of the converted image and noise removal in order to specify the image range of the device markers 4a and 4b with higher accuracy. The marker detection unit 32 of the present embodiment further calculates the area of the above image range (rectangular area) using the image after image processing, and identifies the device markers 4a and 4b that fall within the threshold range.

The marker detection unit 32 detects the device marker 4 by extracting color information from the captured image 7a. The marker detection unit 32 recognizes that the color that matches the color of the extraction target stored in the storage unit 37 is the color of the outer frame of the device marker 4. Further, a threshold value for determining the range of the area of the rectangular region is also stored in the storage unit 37.

When the marker detection unit 32 detects a region of a certain device marker 4 from the captured image 7a, the marker detection unit 32 sends out the area information of the device marker 4 to the identification unit 33. Examples of area information are the X-coordinate and Y-coordinate of a predetermined corner portion of the device marker 4, and the width in the X direction and the width in the Y direction of the device marker 4. The above processing is similarly performed for the position marker 5. The captured image 7a in FIG. 4A includes two device markers 4a and 4b, but the number of device markers 4 included in each captured image 7 may be other than two.

Hereinafter, the description of each block of the device detection device 1 will be continued with reference to FIG.

The identification unit 33 converts the information of the device marker 4 detected by the marker detection unit 32 into the registration number of the device 43. In this way, the identification unit 33 can identify the registration number indicated by the device marker 4. The identification unit 33 further converts the information of the position marker 5 detected by the marker detection unit 32 into information regarding a section in the building (hereinafter referred to as “section information”). In this way, the identification unit 33 can identify the division information indicated by the position marker 5. Examples of parcel information are numbers or character strings representing each parcel in a building, or numbers or character strings representing boundaries between parcels in a building.

The identification unit 33 acquires the area information of the device marker 4 from the marker detection unit 32. Examples of the area information are the X and Y coordinates of the corners of the device marker 4, and the width of the device marker 4 in the X direction and the width in the Y direction. The identification unit 33 extracts the internal region of the device marker 4 (the area within the outer frame of the device marker 4) from the captured image 7 based on the area information of the device marker 4. Specifically, the identification unit 33 calculates the X and Y coordinates of the corners of the internal region of the device marker 4, and the width of the internal region of the device marker 4 in the X direction and the width in the Y direction. These values are represented, for example, by the number of pixels. The ratio of excluding the outer frame from the device marker 4 is stored in advance in the storage unit 37 as an outer frame exclusion parameter.

The identification unit 33 converts the information of the device marker 4 into the registration number of the device 43 based on the information of the internal area of the device marker 4. This conversion is performed using the color conversion table stored in the storage unit 37.

The identification unit 33 may perform image processing for making the luminance distribution in the captured image 7 uniform or image processing for reducing the number of colors in the captured image 7 in order to suppress erroneous discrimination. For example, a histogram averaging process or a histogram stretching process for making the luminance distribution in the captured image 7 uniform may be performed. Further, in order to suppress an erroneous color arrangement in the cell, the K-means method may be performed. The above processing is similarly performed for the position marker 5.

The collation unit 34 collates the registration number of the device marker 4 identified by the identification unit 33 with the registration number in the database 23, and extracts the registration number corresponding to the registration number of the device marker 4 from the database 23. Specifically, when the same registration number as the registration number of the device marker 4 exists in the database 23, the collation unit 34 sets the registration number of the device marker 4 (that is, the registration number extracted from the database 23). It is sent to the position estimation unit 35. On the other hand, if the same registration number as the registration number of the device marker 4 does not exist in the database 23, the collation unit 34 determines that the detection of the device marker 4 is incorrect, and excludes this detection result from the processing target. To do. The registration numbers of the individual devices 43 are registered in advance in the database 23. In the present embodiment, when the device marker 4 having a certain registration number is created, the registration number may be registered in the database 23.

The collation unit 34 further collates the partition information of the position marker 5 identified by the identification unit 33 with the partition information in the database 23, and extracts the partition information corresponding to the partition information of the position marker 5 from the database 23. Specifically, when the same partition information as the partition information of the position marker 5 exists in the database 23, the collation unit 34 uses the partition information of the position marker 5 (that is, the partition information extracted from the database 23). It is sent to the position estimation unit 35. On the other hand, when the same partition information as the partition information of the position marker 5 does not exist in the database 23, the collation unit 34 determines that the detection of the position marker 5 is erroneous, and excludes this detection result from the processing target. To do. In the database 23, division information of a plurality of positions in the building is registered in advance. In the present embodiment, when the position marker 5 having a certain section information is created, the section information may be registered in the database 23.

As described above, the position marker 5 is a mark used for identifying a position in the building, and is provided at a plurality of places in the building. The position marker 5 of the present embodiment holds the section information as information uniquely indicating the position in the building. In the present embodiment, position markers 5 are installed at locations indicating boundaries between sections, and a plurality of position markers 5 are installed at various locations in the building so that each captured image 7 includes at least one position marker 5. Has been done. Further, the position marker 5 of the present embodiment holds the division information in the form of a number, and this number is hereinafter referred to as a division number.

The position marker 5 may be created with the same specifications as the device marker 4 or with different specifications, but the position marker 5 of the present embodiment is a color marker created with the same specifications as the device marker 4. There is. The position marker 5 of the present embodiment is composed of a plurality of cells and an outer frame surrounding these cells, and the outer frame is filled with a color separable from the color of the floor 41. The position marker 5 holds the section information that uniquely indicates the position in the building by the color of the cells constituting the position marker 5.

The position estimation unit 35 determines (estimates) the installation position of each device 43 by combining the detection result of the device marker 4 and the detection result of the position marker 5. Specifically, when one device marker 4 and one position marker 5 are detected from one same captured image 7, the device 43 is specified based on the registration number indicated by the device marker 4, and the position marker 5 is specified. The position of the device 43 is specified based on the partition information indicated by. The position estimation unit 35 of the present embodiment acquires the Y coordinate of the upper left corner of the device marker 4 and the Y coordinate of the upper left corner of the position marker 5 included in the captured image 7, and compares the values of these Y coordinates. Then, based on the comparison result, the position information indicating the position of the device 43 corresponding to the device marker 4 is determined (set). The position information of the present embodiment is determined based on the section information indicated by the position marker 5.

FIG. 5 is a diagram showing an example of an captured image 7 output from the imaging device 3 of the first embodiment.

FIG. 5A shows an captured image 7b output from any of the imaging devices 3 as an example of the captured image 7. The captured image 7b is provided on the floor 41, one device 43 (device 43c) arranged on the floor 41, one device marker 4 (device marker 4c) attached to the upper part of the device 43, and the floor 41. It includes one position marker 5 (position marker 5a) affixed on the passage 42. FIG. 5A further shows the Y coordinate Y1 at the upper left corner of the device marker 4c and the Y coordinate Y2 at the upper left corner of the position marker 5a.

FIG. 5B shows an captured image 7c output from any of the imaging devices 3 as an example of the captured image 7. The captured image 7c is provided on the floor 41, one device 43 (device 43d) arranged on the floor 41, one device marker 4 (device marker 4d) attached to the upper part of the device 43, and the floor 41. It includes two position markers 5 (position markers 5b, 5c) attached on the passage 42. FIG. 5B further shows the Y coordinate Y3 at the upper left corner of the device marker 4d, the Y coordinate Y4 at the upper left corner of the position marker 5b, and the Y coordinate Y5 at the upper left corner of the position marker 5c. ..

In the case of FIG. 5A, the marker detection unit 32, the identification unit 33, and the collation unit 44 operate as follows. The marker detection unit 32 calculates the X and Y coordinates of the upper left corner of the position marker 5a and the width of the position marker 5a in the X direction and the width of the Y direction from the captured image 7b of FIG. 5A. The identification unit 33 calculates the section number of the position marker 5a detected by the marker detection unit 32. The collation unit 34 collates the division number of the position marker 5a calculated by the identification unit 33 with the division number in the database 23, and extracts the division number corresponding to the division number of the position marker 5a from the database 23. The above is the same in the case of FIG. 5 (b).

The position estimation unit 35 changes the process of determining the position information according to the number of the position markers 5 detected from one captured image 7 by the marker detection unit 32. For example, the captured image 7b of FIG. 5A includes one position marker 5a, and the position information is determined based on the position marker 5a. On the other hand, the captured image 7b of FIG. 5B includes two position markers 5b and 5c, and the position information is determined based on these position markers 5b and 5c. However, when counting the number of position markers 5, only the position marker 5 whose partition information was extracted from the database 23 by the collation unit 34 was counted, and the partition information was not extracted from the database 23 by the collation unit 34. The position marker 5 is excluded from the counting target. The position information is determined by using the position marker 5 from which the section information is extracted.

As shown in FIG. 5A, when the number of position markers 5 detected by the marker detection unit 32 is one (position marker 5a), the Y coordinate Y1 at the upper left corner of the device marker 4c and the position marker Compare with the Y coordinate Y2 at the upper left corner of 5a. Then, when Y1 is larger than Y2, the partition information indicating the partition P1 is determined as the position information indicating the position of the device marker 4c (device 43c) (Y1> Y2). On the other hand, when Y1 is smaller than Y2 or equal to Y2, the partition information indicating the partition P2 is determined as the position information indicating the position of the device marker 4c (Y1 ≦ Y2).

The section P1 is a section on the + Y direction side of the position marker 5a, and the section P2 is a section on the −Y direction side of the position marker 5a. Since Y1 is smaller than Y2 in FIG. 5A, the partition information indicating the partition P2 is determined as the position information indicating the position of the device marker 4c. The determined partition information is sent to the registration unit 36 together with the registration number of the device marker 4c.

As shown in FIG. 5B, when the number of position markers 5 detected by the marker detection unit 32 is two (position markers 5b and 5c), the Y coordinate Y3 at the upper left corner of the device marker 4d and The Y coordinate Y4 at the upper left corner of the position marker 5b is compared with the Y coordinate Y5 at the upper left corner of the position marker 5c. Then, when Y3 is larger than Y4, the partition information indicating the partition P3 is determined as the position information indicating the position of the device marker 4d (device 43d) (Y3> Y4). When Y3 is smaller than Y4, equal to Y4, and larger than Y5, the partition information indicating the partition P4 is determined as the position information indicating the position of the device marker 4d (Y5 <Y3 ≦ Y4). On the other hand, when Y3 is smaller than Y5 or equal to Y2, the partition information indicating the partition P5 is determined as the position information indicating the position of the device marker 4d (Y3 ≦ Y5).

Section P3 is a section on the + Y direction side of the position marker 5b, section P4 is a section on the −Y direction side of the position marker 5b and a section on the + Y direction side of the position marker 5c, and section P5 is a section on the + Y direction side of the position marker 5c. It is a section on the Y direction side. In FIG. 5B, since Y3 is smaller than Y4 and larger than Y5, the partition information indicating the partition P4 is determined as the position information indicating the position of the device marker 4d. The determined partition information is sent to the registration unit 36 together with the registration number of the device marker 4d.

When the number of position markers 5 detected by the marker detection unit 32 is 3 or more, the same processing as in the case of 2 is performed.

Hereinafter, the description of each block of the device detection device 1 will be continued with reference to FIG.

The registration unit 36 registers the registration number and the division information of each device marker 4 sent from the position estimation unit 35 in the database 23. For example, when the registration number "N" and the section information "I" of a certain device marker 4 are registered, the device 43 having the registration number "N" is located in the section indicated by the section information "I". means. The registration number "N" of the device marker 4 and the section information "I" are registered in the database 23 in association with each other as information of the same device 43. The registration unit 36 may further register the registration number of each device marker 4 and the time when the division information is acquired from the position estimation unit 35 in the database 23. Further, the registration unit 36 may register such information in a place other than the database 23.

The storage unit 37 possesses various information necessary for the operation of the device detection device 1, and possesses, for example, a position coordinate table indicating the installation position (coordinates) of each position marker 5. The position coordinate table of the present embodiment is created in advance in consideration of the structure of the building and the like, and is stored in the storage unit 37.

The information acquired from the storage unit 37 in the present embodiment may be acquired from a storage location other than the storage unit 37 instead. Further, the information stored in the storage unit 37 in the present embodiment may be stored in a storage location other than the storage unit 37 instead. Examples of such storage locations are the device information management device 2 and a storage device outside the device detection system.

FIG. 6 is a diagram showing an example of stored data in the database 23 of the first embodiment.

FIG. 6 shows the registration number (ID) of the device marker 4, the partition information indicating the position of the device marker 4, and the timing (update date) when the registration number and the partition information are registered in the database 23 as the items of the stored data. , Indicates the customer of the manufactured product. As an example, FIG. 6 shows the IDs, division information, update dates, and customer destinations of the four devices 43.

In the present embodiment, the status of the waiting state for shipment may be registered in the database 23. Further, when the update date of the division information of a certain device 43 is earlier than a predetermined date, the progress delay status of the device 43 may be registered in the database 23. Further, the registration number of the device marker 4 registered in the database 23 by the registration unit 36 may be the registration number identified by the identification unit 33 or the registration number extracted from the database 23 by the collation unit 34. If the device marker 4 can be identified, any of these registration numbers may be converted into another number.

FIG. 7 is a flowchart for explaining the device detection method of the first embodiment. The device detection method shown in FIG. 7 is performed by the device detection device 1 of the present embodiment.

First, the imaging unit 31 acquires the captured image 7 from any of the imaging devices 3 in the building (step S1). The imaging device 3 may continue imaging all day long, or may start imaging at regular time intervals and end imaging after a certain period of time has elapsed. In the latter case, the storage unit 37 stores a list of shooting start time and shooting end time.

Next, the marker detection unit 32 extracts color information from the captured image 7 in order to detect the device marker 4 and the position marker 5 from the captured image 7 (step S2). When the region of the device marker 4 is detected from the captured image 7 (YES in step S3), the process proceeds to step S4. If the region of the device marker 4 is not detected from the captured image 7 (step S3 NO), the process returns to step S1.

Next, the identification unit 33 converts the color information detected by the marker detection unit 32 into the registration number of the device 43 (device marker 4) (step S4). The identification unit 33 further converts the color information detected by the marker detection unit 32 into the section number of the position marker 5.

Next, the collation unit 34 collates the registration number obtained by the identification unit 33 with the registration number stored (registered) for collation in the database 23. If the registration number obtained by the identification unit 33 is registered in the database 23 (YES in step S5), the process proceeds to step S6. If the registration number obtained by the identification unit 33 is not registered in the database 23 (step S5 NO), the process returns to step S1. The collation unit 34 performs the same processing on the section number obtained by the identification unit 33.

Next, the position estimation unit 35 estimates the position information of the device marker 4 based on the registration number and the section number determined to be registered in the database 23 (step S6). Specifically, the position estimation unit 35 determines the section where the device marker 4 is located based on the section number of the position marker 5, and uses the position information of the device marker 4 as the section indicating the section where the device marker 4 is located. Determine the information. For example, the division information is determined by the methods shown in FIGS. 5 (a) and 5 (b).

Next, the registration unit 36 transmits the division information of the device marker 4 determined by the position estimation unit 35 to the device information management device 2 together with the registration number of the device marker 4 (step S7). As a result, the registration number and the division information of the device marker 4 are registered in the database 23.

FIG. 8 is a flowchart for explaining the details of the device detection method of the first embodiment. FIG. 8 shows the processing performed on the position marker 5 by the device detection device 1 of the present embodiment. In this description, reference is made to FIGS. 5 (a) and 5 (b).

First, the marker detection unit 32 detects the device marker 4 and the position marker 5 from the captured image 7 (step S11). Next, the identification unit 33 identifies the registration number indicated by the device marker 4 and the division number indicated by the position marker 5 (step S12). Next, after collating these registration numbers and partition numbers with the registration numbers and partition numbers in the database 23, the process proceeds to step S13.

When the number of position markers 5 detected by the marker detection unit 32 is one (position marker 5a) as in the case where the captured image 7b of FIG. 5A is captured (step S13 YES), the position is estimated. The unit 35 compares the Y coordinate Y1 at the upper left corner of the device marker 4c with the Y coordinate Y2 at the upper left corner of the position marker 5a. Then, when Y1 is larger than Y2 (YES in step S14), the section information indicating the section P1 is determined as the position information indicating the position of the device marker 4c (device 43c) and transmitted to the device information management device 2. (Step S21). On the other hand, when Y1 is smaller than Y2 or equal to Y2 (step S14 NO), the partition information indicating the partition P2 is determined and transmitted as the position information indicating the position of the device marker 4c (step S22).

On the other hand, when the number of position markers 5 detected by the marker detection unit 32 is two or more (position markers 5b, 5c, here two) as in the case where the captured image 7c of FIG. 5B is captured. (Step S13 NO), the position estimation unit 35 has the Y coordinate Y3 of the upper left corner of the device marker 4d, the Y coordinate Y4 of the upper left corner of the position marker 5b, and the Y coordinate of the upper left corner of the position marker 5c. Compare with Y5. Then, when Y3 is larger than Y4 (YES in step S15), the section information indicating the section P3 is determined as the position information indicating the position of the device marker 4d (device 43d) and transmitted to the device information management device 2. (Step S23). When Y3 is smaller than Y4, equal to Y4, and larger than Y5 (step S15 NO, step S16 YES), the partition information indicating the partition P4 is determined and transmitted as the position information indicating the position of the device marker 4d. (Step S24). On the other hand, when Y3 is smaller than Y5 or equal to Y2 (step S16 NO), the partition information indicating the partition P5 is determined and transmitted as the position information indicating the position of the device marker 4d (step S25).

As described above, the device detection system of the present embodiment detects the device marker 4 installed in each device 43 and estimates the position of the device marker 4 in the building to estimate the device in the entire building. The installation position of 43 can be automatically detected. Further, the device detection system of the present embodiment can automatically manage the position of the device 43 by registering the installed position of the detected device 43 on the device management system.

(1) First Modified Example FIG. 9 is a diagram showing a configuration of an equipment detection system of the first modified example of the first embodiment.

FIG. 9 is a perspective view showing the overall configuration of the device detection system of the present modification, and shows the wall surface 45 of the building in addition to the components shown in FIG. The position marker 5 of FIG. 3 is provided on the passage 42 installed in the center of the floor 41, but the position marker 5 of FIG. 9 is provided at a place other than the floor 41, specifically, the wall surface 45. ing. FIG. 9 shows a plurality of position markers 5 provided at high positions on the wall surface 45 extending in the y direction. The position marker 5 of this modification may be described on the wall surface 45 with a paint or the like, or a plate or sheet on which the position marker 5 is described may be provided on the wall surface 45 in a form of being attached to the wall surface. ..

The image pickup device 3 of this modification includes the image pickup devices 3a to 3c attached to the crane 6 at an angle facing the floor 41, and the image pickup device 3d attached to the crane 6 at an angle facing the wall surface 45. The image pickup devices 3a to 3d move in the building together with the crane 6. The device detection device 1 of this modification detects the device marker 4 from the captured images 7 captured by the image pickup devices 3a to 3c, and detects the position marker 5 from the captured images 7 captured by the image pickup device 3d.

Examples of the captured image 7 captured by the imaging device 3 include the captured image 7a of FIG. 4A, the captured image 7b of FIG. 5A, the captured image 7c of FIG. 5B, and the captured image 7c of FIG. 5B, as described above. As will be described later, the captured image 7d in FIG. 10A, the captured image 7e in FIG. 10B, and the like. These figures show the X direction (X coordinate) corresponding to the x direction in the building, the Y direction (Y coordinate) corresponding to the y direction in the building, and the Z direction (Z coordinate) corresponding to the z direction in the building. ) Is shown. The device detection system of this modification manages the positions of the device marker 4 and the position marker 5 by the X coordinate, the Y coordinate, and the Z coordinate.

Further details of this modification will be described below. The configuration of the device detection system shown in FIG. 1 and the configuration of the device detection device 1 shown in FIG. 2 are also applied to this modification.

FIG. 10 is a diagram showing an example of an captured image 7 output from the imaging device 3 of the first modification of the first embodiment.

FIG. 10A shows the captured image 7d captured by the imaging device 3d as an example of the captured image 7. The captured image 7d includes three position markers 5 (position markers 5d to 5f). FIG. 10A shows the Y direction (Y coordinate) corresponding to the y direction in the building and the Z direction (Z coordinate) corresponding to the z direction in the building. FIG. 10A further shows the Y coordinate Ya of the position marker 5f and the width Wa of the captured image 7d in the Y direction. The device detection system of this modification manages the positions of the device marker 4 and the position marker 5 by the X coordinate, the Y coordinate, and the Z coordinate.

The position markers 5d and 5e hold the section information in the building by the two-digit numbers "4-3" and "4-4", respectively. For example, the position marker 5e is composed of a board on which the number "4-4" is printed, and the number "4-4" is the section information indicating one section in the building. The position markers 5d and 5e may hold the section information represented by a number other than two digits, or may hold the section information represented by a character or shape other than the number. However, it is desirable that the position markers 5d and 5e hold the division information so that the order in which the divisions are arranged can be known in the order of the size of the numbers.

The position marker 5f is provided to indicate the boundary between the sections in the building, and is composed of, for example, a board on which a triangle is printed. It should be noted that this position marker 5f does not hold an identifier of a position in the building, unlike the other position markers 5 described above. In this modification, a plurality of first position markers 5 holding an identifier like the position markers 5d and 5e are provided in a row on the wall surface 45, and a plurality of second positions not holding an identifier like the position markers 5f. The marker 5 is provided between the first position markers 5. The second position marker 5 of this modification is composed of a plate on which triangles are printed, but these triangles may be replaced with another figure (for example, a circle or a straight line). Further, the second position marker 5 may hold an identifier in the same manner as the first position marker 5.

FIG. 10B shows the captured image 7e captured by the imaging device 3a as an example of the captured image 7. The captured image 7e is an image captured at the same time as the captured image 7d. The captured image 7e includes one device 43 (device 43e) arranged on the floor 41 and one device marker 4 (device marker 4e) attached to the upper part of the device 43. FIG. 10B further shows the Y coordinate Y6 at the upper left corner of the device marker 4e and the Y coordinates Y7 and Y8 of the boundary line between the sections in the building.

In this modification, when the marker detection unit 32 detects the position markers 5d to 5f from the captured image 7d, the identification unit 33 identifies the division information of the position markers 5d and 5e from the position markers 5d and 5e, and the collation unit 34 , The partition information identified from the position markers 5d and 5e is collated with the partition information in the database 23. Then, when the partition information identified from the position markers 5d and 5e exists in the database 23, the position estimation unit 35 determines that the position marker 5f is the first position marker that holds the position marker 5d (4-3). It is determined that the second position marker 5 is located between the position marker 5) and the position marker 5e (the first position marker 5 that holds the division information of 4-4). As a result, the position estimation unit 35 can specify the Y coordinate Ya of the position marker 5f based on the information of the position markers 5d and 5e.

Note that the position estimation unit 35 detects one of the position markers 5d and 5e (for example, the position marker 5e) instead of determining that the position marker 5f is detected when both the position markers 5d and 5e are detected. In some cases, it may be determined that the position marker 5f has been detected. In fact, it is considered that the position marker 5d in the captured image 7d of FIG. 10A is not detected by the marker detection unit 32 because only half of the position marker 5d is included in the captured image 7d. Therefore, in the following description, it is assumed that the position estimation unit 35 determines that when the position marker 5e is detected, the position marker 5f adjacent to the position marker 5e is detected in the −Y direction of the position marker 5e.

On the other hand, the storage unit 37 holds in advance a position coordinate table showing the correspondence between each second position marker 5 and the Y coordinate of the boundary line between the sections. The position coordinate table shows, for example, that the position marker 5f having the Y coordinate Ya and the Y coordinates Y7 and Y8 of the boundary line between the sections correspond to each other. Therefore, when the position marker 5f is detected from the captured image 7d, the position estimation unit 35 determines that a boundary line having Y coordinates Y7 and Y8 exists in the captured image 7e captured at the same time as the captured image 7d. judge. As a result, the position estimation unit 35 of the present modification can perform the same processing as the processing for the captured image 7c of FIG. 5B.

Specifically, the position estimation unit 35 compares the Y coordinate Y6 of the upper left corner portion of the device marker 4e with the Y coordinate Y7 of the boundary line between the sections and the Y coordinate Y8 of the boundary line between the sections. Then, when Y6 is larger than Y7, the partition information indicating the partition P6 is determined as the position information indicating the position of the device marker 4e (device 43e) (Y6> Y7). When Y6 is smaller than Y7, equal to Y7, and larger than Y8, the partition information indicating the partition P7 is determined as the position information indicating the position of the device marker 4e (Y8 <Y6 ≦ Y7). On the other hand, when Y6 is smaller than Y8 or equal to Y8, the partition information indicating the partition P8 is determined as the position information indicating the position of the device marker 4e (Y6 ≦ Y8).

The section P6 is a section on the + Y direction side of the boundary line having the Y coordinate Y7, and the section P7 is a section on the + Y direction side of the boundary line having the Y coordinate Y8 on the −Y direction side of the boundary line having the Y coordinate Y7. The compartment P8 is a compartment on the −Y direction side of the boundary line having the Y coordinate Y8. In FIG. 10B, since Y6 is smaller than Y7 and larger than Y8, the partition information indicating the partition P7 is determined as the position information indicating the position of the device marker 4e. The determined partition information is sent to the registration unit 36 together with the registration number of the device marker 4e.

FIG. 11 is a diagram showing an example of a position coordinate table of the first modification of the first embodiment.

The storage unit 37 of this modification holds in advance a position coordinate table showing the correspondence between the second position marker 5 such as the position marker 5f and the Y coordinate of the boundary line between the sections. The position coordinate table shows, for example, that the position marker 5f having the Y coordinate Ya and the Y coordinates Y7 and Y8 of the boundary line between the sections correspond to each other. It should be noted that in the position coordinate table of FIG. 11, the Y coordinate Ya of the position marker 5f is displayed in a unit system different from the Y coordinates Y7 and Y8 of the boundary line. The Y coordinate Ya of the position marker 5f in this modification indicates the center Y coordinate of the position marker 5f.

FIG. 12 is a flowchart for explaining a device detection method of the first modification of the first embodiment. FIG. 12 shows the processing performed on the position marker 5 by the device detection device 1 of this modification. In this description, reference is made to FIGS. 10 (a) and 10 (b).

First, the marker detection unit 32 detects the position marker 5e from the captured image 7d in FIG. 10A (step S31). Next, the identification unit 33 identifies the division information indicated by the position marker 5e (step S32). Next, after collating the partition information identified from the position marker 5e with the partition information in the database 23, the process proceeds to step S33.

Next, the marker detection unit 32 detects the position marker 5f from the captured image 7d in FIG. 10B (step S33). The process of step S33 is performed by image processing such as template matching process and pattern recognition process, for example.

Next, the position estimation unit 35 receives the information regarding the position markers 5e and 5f, and reads the position coordinate table from the storage unit 37 (step S34). As a result, the position estimation unit 35 can determine that the position marker 5f is the second position marker 5 adjacent to the position marker 5e (the first position marker 5 holding the partition information of 4-4). Further, the position estimation unit 35 can specify the Y coordinate Ya of the position marker 5f based on the information of the position marker 5e. The position estimation unit 35 can further determine that a boundary line having Y coordinates Y7 and Y8 exists in the captured image 7e captured at the same time as the captured image 7d.

Next, the position estimation unit 35 compares the Y coordinate Y6 of the upper left corner of the device marker 4e in the captured image 7e with the Y coordinate Y7 of the boundary line between the sections and the Y coordinate Y8 of the boundary line between the sections. To do. Then, when Y6 is larger than Y7 (YES in step S35), the section information indicating the section P6 is determined as the position information indicating the position of the device marker 4e (device 43e) and transmitted to the device information management device 2. (Step S41). When Y6 is smaller than Y7, equal to Y7, and larger than Y8 (step S35 NO, step S36 YES), the partition information indicating the partition P7 is determined and transmitted as the position information indicating the position of the device marker 4e. (Step S42). On the other hand, when Y6 is smaller than Y8 or equal to Y8 (step S36 NO), the partition information indicating the partition P8 is determined and transmitted as the position information indicating the position of the device marker 4e (step S43).

As described above, the device detection system of this modification includes the position marker 5 provided on the wall surface 45 instead of the floor 41. As a result, it is possible to prevent the position marker 5 from being stepped on by a person walking in the building (for example, an article transporter), and it is possible to prevent the position marker 5 from being stepped on by a person and deteriorating. ..

(2) Second Modified Example The second modified example of the first embodiment will be described with reference to FIG. The configuration of the device detection system shown in FIG. 1 and the configuration of the device detection device 1 shown in FIG. 2 are also applied to this modification.

In FIG. 3, the position marker 5 is used to estimate the position of each device marker 4. On the other hand, in this modification, in order to estimate the position of each device marker 4, a characteristic portion existing in the building is used. Examples of such parts are the corners of pillars in a building, the color of the floor 41 in a building, and the like. Detection of such a portion is performed by image processing such as template matching processing or pattern recognition processing, for example.

For example, when the corner of the pillar is used to estimate the position of each device marker 4, the mark detection unit 32 detects the corner of the pillar from the captured image 7. In this case, the Y coordinate of the corner of the pillar is stored in the storage unit 37 in advance. Then, when the device marker 4 and the corner of the pillar are detected from the captured image 7, the position estimation unit 35 determines the position information of the device marker 4 using the Y coordinate of the corner of the pillar. This position information can be determined by performing the same processing as the processing for the captured image 7b in FIG. 5A.

As described above, the device detection system of the present modification uses a characteristic portion in the building instead of the position marker 5 to estimate the position of each device marker 4. As a result, it is possible to avoid the trouble required for the installation and maintenance of the position marker 5.

In this modification, a plurality of characteristic parts existing in the building may be used in order to estimate the position of the device marker 4. An example of such a part is the corner of multiple pillars in a building. In this case, the difference in shape and color between the pillars may be used in the same manner as the difference in the division information between the position markers 5. As a result, it is possible to determine that the first pillar is arranged in the first section in the building and the second pillar is arranged in the second section in the building. In this case, the identification unit 33 identifies the pillars, and the collation unit 34 determines whether or not the identification of the pillars is correct.

(3) Third Modified Example A third modified example of the first embodiment will be described with reference to FIG. The configuration of the device detection system shown in FIG. 1 and the configuration of the device detection device 1 shown in FIG. 2 are also applied to this modification.

In FIG. 3, the image pickup device 3 that captures the position marker 5 is a visible camera, and the position information of the device marker 4 is determined using the position marker 5 imaged by the image pickup device 3. On the other hand, in this modification, the position information of the device marker 4 is determined by using another device attached to the crane 6 together with the image pickup device 3. Examples of such devices are laser sensors and stereo cameras.

When using a laser sensor, the laser sensor is attached to the crane 6 so as to face, for example, the + y direction or the −y direction. The laser sensor moves in the building in the ± y direction together with the image pickup device 3 and the crane 6. When the laser sensor is attached to the crane 6 so as to face the + y direction, the laser sensor measures the distance to the wall of the building located in the + y direction of the crane 6 to determine the position of the imaging device 3. presume. On the other hand, when the laser sensor is attached to the crane 6 so as to face the −y direction, the laser sensor measures the distance to the wall of the building located in the −y direction of the crane 6 to obtain an imaging device. Estimate the position of 3. The distance between the wall and the laser sensor can be measured, for example, by known methods.

In this case, when the device marker 4 is detected from the captured image 3, the position estimation unit 35 measures the position information of the device marker 4 by the laser sensor when the captured image 3 is captured. Determined based on distance. Specifically, the position of the image pickup device 3 is estimated based on the above distance measured by the laser sensor, and the position information of the device marker 4 is determined based on the estimated position of the image pickup device 3.

When using a stereo camera, the stereo camera is attached to the crane 6 so as to face, for example, in the + y direction or the −y direction. The stereo camera moves in the building in the ± y direction together with the image pickup device 3 and the crane 6. When the stereo camera is attached to the crane 6 so as to face the + y direction, the stereo camera measures the distance to the wall of the building located in the + y direction of the crane 6 to determine the position of the image pickup device 3. presume. On the other hand, when the stereo camera is attached to the crane 6 so as to face the −y direction, the stereo camera measures the distance to the wall of the building located in the −y direction of the crane 6 to obtain an imaging device. Estimate the position of 3. The distance between the wall and the stereo camera can be measured, for example, by a known method of calculating the distance in the depth direction by the principle of triangulation. The position information of the device marker 4 in this case can be determined in the same manner as in the case of the laser sensor.

As described above, the device detection system of the present modification uses a laser sensor or a stereo camera instead of the image pickup device 3 to estimate the position of each device marker 4. As a result, it is possible to avoid the trouble required for the installation and maintenance of the position marker 5. The laser sensor or stereo camera of this modification may measure the distance between the image pickup device 3 and a portion other than the wall in the building.

As described above, the device detection device 1 of the present embodiment acquires the captured image 7 from the image pickup device 3 moving in the building, and uses the device marker 4 detected from the captured image 7 to obtain the device 43 in the building. The position of the device 43 (device marker 4) is determined based on the information from the device that identifies and moves with the image pickup device 3 or the image pickup device 3. For example, the device detection device 1 of the present embodiment determines the position of the device 43 (device marker 4) based on the position marker 5 detected from the captured image 7, as described with reference to FIG. 3 and the like. ..

Therefore, according to the present embodiment, even if the image pickup device 3 is not installed at a distance close to the device 43, the device 43 installed in a wide range is detected by moving the image pickup device 3 so as to approach the device 43. It becomes possible to do. As described above, according to the present embodiment, it is possible to efficiently detect the device 43 installed in a wide range.

Although the image pickup device 3 is moved by the crane 6 in this embodiment, it may be moved by other means. For example, the image pickup device 3 may be attached to a forklift that moves in the building, or may be attached to a helmet of a worker who moves in the building. This also applies to the second embodiment and the third embodiment described later.

(Second Embodiment)
FIG. 13 is a block diagram showing the configuration of the device detection device 1 of the second embodiment. The device detection device 1 of the present embodiment includes a movement detection unit 38 in addition to the components shown in FIG. The present embodiment will be described with reference to FIG. The configuration of the device detection system shown in FIG. 1 is also applied to this embodiment.

In the case of the first embodiment, the imaging device 3 may continue imaging all day long, or may start imaging at regular time intervals and end imaging after a certain period of time has elapsed. In the latter case, the storage unit 37 stores a list of shooting start time and shooting end time. The imaging unit 31 acquires the captured image 7 from such an imaging device 3.

In the case of the present embodiment, the movement detection unit 38 detects whether or not the image pickup device 3 is moving based on the captured image 7 from the image pickup device 3. Then, the movement detection unit 38 controls the timing at which the image pickup unit 31 acquires the captured image 7 from the image pickup device 3 for the detection of the device marker 4 based on the detection result of whether or not the image pickup device 3 is moving. To do. For example, when it is detected that the image pickup apparatus 3 is not moving, the movement detection unit 38 transmits a signal requesting that the image pickup image 7 is not acquired from the image pickup apparatus 3 to the image pickup apparatus 31. On the other hand, when it is detected that the image pickup device 3 is moving, the movement detection unit 38 transmits a signal requesting that the image pickup image 7 is acquired from the image pickup device 3 to the image pickup unit 31. The imaging unit 31 determines whether or not to acquire the captured image 7 from the imaging device 3 in response to such a signal. As a result, the device detection device 1 of the present embodiment can perform automatic control such that the image pickup unit 31 acquires the captured image 7 for detection of the device marker 4 only when the image pickup device 3 is moving. ..

According to the present embodiment, the movement detection unit 38 detects the movement of the image pickup apparatus 3 by image processing, so that it is possible to automatically set the timing for starting the detection processing of the device marker 4 and the position marker 5. Become. According to the present embodiment, since such a detection process can be performed only when the image pickup apparatus 3 is moving, the number of captured images 7 to be detected can be reduced, and the captured images can be reduced. It is possible to reduce the storage amount of 7.

FIG. 14 is a flowchart for explaining the device detection method of the second embodiment. The flowchart of FIG. 14 includes step S8 before steps S1 to S7 shown in FIG.

In step S8, the movement detection unit 38 determines whether or not the image pickup device 3 has moved based on the captured image 7 acquired from the image pickup device 3. Specifically, the movement detection unit 38 calculates the amount of change in time series of a plurality of captured images 7. The amount of change in the time series can be calculated, for example, by comparing each captured image 7 with the captured image 7 of the preceding frame or the rear frame by background subtraction processing or inter-frame difference processing and acquiring a difference image. Further, the amount of change in time series may be calculated by, for example, calculating the movement of each pixel by optical flow processing. Alternatively, the movement detection unit 38 compares the captured image with the template image by a template matching process using the captured image 7 before the start of movement of the imaging device 3 as a template image to determine whether or not the imaging device 3 has moved. May be judged.

If it is determined that the image pickup apparatus 3 has not moved (step S8 NO), the process of step S8 is performed again later. On the other hand, when it is determined that the image pickup apparatus 3 has moved (YES in step S8), the process proceeds to step S1. In step S1, the imaging unit 31 acquires the captured image 7 for detecting the device marker 4 from the imaging device 3.

As described above, the device detection device 1 of the present embodiment includes a movement detection unit 38 that detects the movement of the image pickup device 3. Therefore, according to this embodiment, it is possible to reduce the number of captured images 7 to be detected by the device marker 4 and the like, and to reduce the storage amount of the captured images 7.

(Third Embodiment)
FIG. 15 is a block diagram showing the configuration of the device detection device 1 of the third embodiment. The device detection device 1 of the present embodiment includes an image connecting unit 39 in addition to the components shown in FIG. This embodiment will be described with reference to FIGS. 3 and 16. FIG. 16 is a diagram showing an example of a connected image of the third embodiment. The configuration of the device detection system shown in FIG. 1 is also applied to this embodiment.

The image connecting unit 39 connects a plurality of captured images 7 captured by one or more imaging devices 3, and represents the state of the entire installation location of the device 43 obtained by connecting these captured images 7. Output a concatenated image. Examples of such a connected image are the connected image 8a of FIG. 16A and the connected image 8b of FIG. 16B. The linked image 8a of FIG. 16A can be obtained, for example, by connecting a plurality of captured images 7 captured by the imaging devices 3a to 3c of FIG. The linked image 8b of FIG. 16B is obtained, for example, by connecting a plurality of captured images 7 captured by the imaging devices 3a to 3d of FIG. Hereinafter, when an arbitrary connected image using these connected images 8a and 8b as an example will be described, this will be referred to as “connected image 8”. According to the present embodiment, by outputting the connected image 8 that gives a bird's-eye view of the entire building, it is possible to display the position of each device 43 and the positional relationship between the devices 43 in an easy-to-understand manner.

The image connecting unit 39 receives a plurality of captured images 7 from the imaging unit 31, and connects these captured images 7 based on the information received from the position estimation unit 35. For example, the image connecting unit 39 receives the position information of the plurality of device markers 4 included in the captured images 7, and the positional relationship between the device markers 4 in the generated connected image 8 is indicated by the position information. These captured images 7 are connected so as to match the position of the device marker 4. Information other than the position information of the device marker 4 may be used to generate the connected image 8. Examples of such information are the Y coordinate of the boundary line between the sections, the section information of the position marker 5, and the like.

The image connecting unit 39 may connect the captured images 7 to each other by other methods. For example, the image connecting unit 39 may connect the captured images by superimposing similar regions of the captured images 7 by template matching processing. Further, the image connecting unit 39 calculates the feature amount in the captured image 7 such as the ORB feature amount, and superimposes the similar regions of the captured images 7 based on the feature amount to connect the captured images 7 to each other. May be good.

The connected image 8a of FIG. 16A is displayed in a form in which the X-axis is vertically oriented and the Y-axis is horizontally oriented, that is, the original orientation of each captured image 7 is rotated by 90 degrees. , The connected image 8a may be displayed without such rotation. When the connected image 8a is displayed on the display, the connected image 8a may be rotated and displayed, or may be enlarged or reduced. This also applies to the connected image 8b of FIG. 16 (b).

Here, the connected image 8b of FIG. 16B will be described.

FIG. 16B shows additional information 9 added to the connected image 8b. The additional information 9 is information about a certain device 43 in the connected image 8b, and is added in the vicinity of the position where the device 43 is arranged in the connected image 8b. The additional information 9 of FIG. 16B includes the registration number and the division information of the device 43. The image connecting unit 39 determines a position to add the additional information 9 in the connected image 8b based on the section information of the device 43. For example, the connected image 8b is output in a form in which the additional information 9 is added to the connected image 8b, and is displayed on the display in a form in which the additional information 9 is superimposed and displayed on the connected image 8b.

The connected image 8b of FIG. 16B further includes a plurality of position markers 5 provided on the wall surface 45 (see FIG. 9). These position markers 5 may be the position markers 5 actually included in the captured image 7, or may be an adduct added to the connected image 8b as in the additional information 9.

Further, the additional information 9 may be information other than the registration number of the device 43 and the division information. For example, the additional information 9 may be device information (information about the device 43) stored in the database 23 or device information stored in advance in the storage unit 37. The display color when displaying the additional information 9 may be any color. For example, the additional information 9 may be displayed in a specific color that can be separated from the background such as white, or may be displayed with some transparency so that the image hidden under the additional information 9 can be seen. ..

FIG. 17 is a flowchart for explaining the device detection method of the third embodiment. In the flowchart of FIG. 17, step S7 of the flowchart of FIG. 7 is replaced with steps S9 and S10.

In the flowchart of FIG. 17, when the estimation of the position information is completed in step S6, the image connecting unit 39 connects a plurality of captured images 3 based on the information received from the position estimating unit 35 to create the connected image 8. (Step S9).

Next, the registration unit 36 uses the partition information of the device marker 4 determined by the position estimation unit 35 together with the registration number of the device marker 4 and the connection image 8 created by the image connection unit 39 in the device information management device 2. (Step S7). As a result, the registration number and section information of the device marker 4 and the above-mentioned connected image 8 are registered in the database 23.

As described above, the device detection device 1 of the present embodiment includes an image connecting unit 39 that connects a plurality of captured images 7 captured by one or more imaging devices 3 and outputs a linked image 8. .. Therefore, according to the present embodiment, by outputting the connected image 8 that gives a bird's-eye view of a wide area in the building, it is possible to display the position of each device 43 and the positional relationship between the devices 43 in an easy-to-understand manner.

The captured images 3 connected by the image connecting unit 39 may be captured images 3 captured by the same imaging device 3 at different times, or may be captured images 3 captured by different imaging devices 3 at the same time. However, the captured images 3 may be captured at different times by different imaging devices 3.

Here, the program 14 in the device detection device 1 of FIG. 1 will be described.

This program 14 is a file in an installable format or an executable format, and is recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). May be provided. Further, the program 14 may be provided by storing it on a computer connected to a network such as the Internet and downloading it via the network. Further, the program 14 may be provided or distributed via a network such as the Internet. In the first to third embodiments, the program 14 may be provided in the device detection device 1 by any method. Further, this program 14 may be preliminarily incorporated in the ROM in the device detection device 1.

Although some embodiments have been described above, these embodiments are presented only as examples and are not intended to limit the scope of the invention. The novel devices and methods described herein can be implemented in a variety of other forms. In addition, various omissions, substitutions, and changes can be made to the forms of the apparatus and method described in the present specification without departing from the gist of the invention. The appended claims and their equivalent scope are intended to include such forms and variations contained within the scope and gist of the invention.

1: Device detection device, 2: Device information management device,
3, 3a, 3b, 3c, 3d: Imaging device,
4, 4a, 4b, 4c, 4d, 4e: device markers,
5, 5a, 5b, 5c, 5d, 5e, 5f: Position marker,
6: Crane, 7, 7a, 7b, 7c, 7d, 7e: captured image,
8, 8a, 8b: Concatenated image, 9: Additional information,
11: Input / output device, 12: Arithmetic logic unit, 13: Storage device, 14: Program,
21: Arithmetic logic unit, 22: Storage device, 23: Database,
31: Imaging unit, 32: Marker detection unit, 33: Identification unit,
34: Collation unit, 35: Position estimation unit, 36: Registration unit,
37: Storage unit, 38: Movement detection unit, 39: Image connection unit,
41: Floor, 42: Aisle, 43, 43a, 43b, 43c, 43d, 43e: Equipment,
44: Vehicle, 45: Wall

Claims (14)

An imaging unit that acquires an image captured by an imaging device that moves inside the building,
From the captured image, a detection unit that detects a device identification figure, which is a figure provided on or near the device for identifying the device in the building, and
The first identification information obtained from the device identification figure is collated with the second identification information stored in the storage unit as the identification information of the device, and the first identification information corresponding to the first identification information is collated. An extraction unit that extracts the identification information of 2 from the storage unit,
When the second identification information corresponding to the first identification information is extracted, the building is based on the information output from the image pickup apparatus or the information output from the apparatus moving with the image pickup apparatus. A determination unit that determines position information indicating the position of the device in the device,
A registration unit that associates and registers the identification information of the device with the position information determined by the determination unit, and
A device detector equipped with. The detection unit detects a position identification figure, which is a figure provided in the building for identifying a position in the building, from the captured image.
The extraction unit collates the third identification information obtained from the position identification figure with the fourth identification information stored in the storage unit as the identification information of the position in the building, and the first The fourth identification information corresponding to the identification information of 3 is extracted from the storage unit, and the fourth identification information is extracted from the storage unit.
The determination unit determines the position information based on the position identification figure from which the fourth identification information corresponding to the third identification information is extracted.
The device detection device according to claim 1. The device detection according to claim 2, wherein the determination unit determines the position information indicating the position of the device based on the position identification figure detected from the same captured image as the device identification figure of the device. apparatus. The device detection device according to claim 2 or 3, wherein the position identification figure is installed on the floor of the building. The device detection device according to claim 2 or 3, wherein the position identification figure is installed on the wall surface of the building. The detection unit detects a predetermined part in the building from the captured image, and then
The device detection device according to claim 1, wherein the determination unit determines the position information based on the predetermined portion detected from the captured image. The device detection device according to claim 1, wherein the determination unit determines the position information based on a distance to a predetermined location in the building measured by a device that moves with the image pickup device. The device detection device according to any one of claims 1 to 7, wherein the image pickup device is attached to a moving body that moves the image pickup device. The device detection device according to any one of claims 1 to 8, wherein the position information is information about a section in which the device is located in the building. A movement detection unit for detecting whether or not the image pickup device is moving based on the captured image is further provided.
The movement detection unit controls the timing at which the image pickup unit acquires the captured image from the image pickup device for detecting the device identification figure based on the detection result of whether or not the image pickup device is moving. , The device detection device according to any one of claims 1 to 9. An image connecting unit that concatenates a plurality of the captured images captured by the imaging device based on the position information determined by the determining unit and outputs a concatenated image obtained by concatenating the plurality of captured images. The device detection device according to any one of claims 1 to 10, further comprising. The device detection device according to claim 11, wherein the image connecting unit adds information on the device to the vicinity of the position of the device in the connected image and outputs the connected image. Acquire the captured image captured by the imaging device that moves in the building,
From the captured image, a device identification figure, which is a figure provided on or near the device for identifying the device in the building, is detected.
The first identification information obtained from the device identification figure is collated with the second identification information stored in the storage unit as the identification information of the device, and the first identification information corresponding to the first identification information is collated. The identification information of 2 is extracted from the storage unit, and
When the second identification information corresponding to the first identification information is extracted, the building is based on the information output from the image pickup apparatus or the information output from the apparatus moving with the image pickup apparatus. Determine the position information indicating the position of the device in the
The identification information of the device and the determined position information are associated and registered.
Equipment detection methods including that. In the detection, a position identification figure, which is a figure provided in the building for identifying the position in the building, is detected from the captured image.
In the extraction, the third identification information obtained from the position identification figure is collated with the fourth identification information stored in the storage unit as the identification information of the position in the building, and the third identification information is collated. The fourth identification information corresponding to the identification information of is extracted from the storage unit, and the fourth identification information is extracted from the storage unit.
In the determination, the position information is determined based on the position identification figure from which the fourth identification information corresponding to the third identification information is extracted.
The device detection method according to claim 13.