JP5343210B2 - Subject area calculation device, subject area calculation system, and subject area calculation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a subject area calculation device capable of calculating the area of an object domain of a subject, only by an image including a simple mark. <P>SOLUTION: The device includes: a reading-out means 12 for reading out a first image including the subject and the mark imaged by a first image capturing apparatus, and a second image including the subject and the mark imaged from an angle different from that of the first image capturing apparatus by a second image capturing apparatus; an image capturing apparatus coordinate estimation means 13 for estimating a value of a three-dimensional coordinate of a first position of the first image capturing apparatus, and a value of a three-dimensional coordinate of a second position of the second image capturing apparatus; a specific point coordinate estimation means 14 for estimating a value of each three-dimensional coordinate of a plurality of specific points included in the subject based on the first position and the second position; and an area calculation means 15 for calculating the area of the subject based on the value of each three-dimensional coordinate of the plurality of specific points. The mark has a two-dimensional shape including three or more pointers. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a subject area calculation device, a subject area calculation system, and a subject area calculation method that can calculate the area of a subject imaged by an imaging device based on image information.

  It is preferable that the areas of the walls of these buildings and structures are calculated in advance when the walls of the buildings and structures are painted or signboards or posters are installed. This is because if the area of the wall surface is unknown, it is unclear how much paint or material is needed and how much area of signboard or poster can be installed. If the area of the wall surface is known in advance, the efficiency of the construction of painting or installing a signboard or a poster is improved.

  In order to calculate the area of a region with such a wall surface, it is common to perform actual measurement based on dangerous work, or to perform survey using a reference point survey or a stereo camera.

  However, surveying by actual measurement is dangerous and has the inconvenience of requiring another surveying operation by actual measurement if the target region is changed.

  Work with a reference point survey or a stereo camera can only be performed on the spot where the building or building to be surveyed can be seen in addition to the increase in work time and work man-hours.

  In recent years, many businesses want to install signboards and posters in various places of buildings and buildings in various places. Rather than deciding the signboards and posters to be installed and entering the wall area survey, these companies want to review the types of signboards and posters to be installed based on the wall area. This is because the company seeks to improve the advertising effect by signboards and posters on the wall.

  In many cases, a business operator who installs such a signboard or poster wants to survey an arbitrary area on the wall surface after the installation outside the installation site. In particular, there may be a case where an area having an area where various signs and posters can be installed is arbitrarily searched for in the wall surface.

For this reason, the technique which measures the wall surface of a building or a building using the image imaged with the imaging device is proposed (for example, refer to patent documents 1).
JP 2007-147522 A

  However, Patent Document 1 only discloses a technique for adjusting the parallax of a subject included in images captured from a plurality of angles.

  For this reason, Patent Document 1 does not disclose a technique for calculating the area of a subject. Also, the orientation reference body of Patent Document 1 is difficult to apply to calculate the position and area of the subject. In particular, it is difficult to identify any point of the orientation reference body included in images taken from various angles with high accuracy, and it is necessary to calculate the area of the subject using the images taken afterwards. It is difficult to obtain data. This is because, depending on the angle and position of the photographed photograph, the way in which the orientation reference body is reflected varies greatly, and it is difficult to obtain the relationship between the orientation reference body and the subject.

  In view of the above problems, an object of the present invention is to provide a subject area calculation apparatus that calculates the area of a target region in a subject using only an image including a simple sign.

In view of the above problems, the subject area calculation device of the present invention includes a subject including a target region that is an area calculation target imaged by the first imaging device, and a marker that is in the range of the target region and is installed on the front surface. Reading means for reading out the first image including the subject and the second image including the subject imaged from the angle different from that of the first imaging device by the second imaging device and the sign placed in the target area and placed in front Imaging device coordinate estimation means for estimating the value of the three-dimensional coordinates of the first position of the first imaging device and the value of the three-dimensional coordinates of the second position of the second imaging device, and the first position and the second position Based on the specific point coordinate estimation means for estimating the value of each of the three-dimensional coordinates of the plurality of specific points included in the subject, and the area of the target region based on the values of the three-dimensional coordinates of the plurality of specific points. Area calculating means for calculating The provided, the label has a Rutotomoni known area having a two-dimensional shape with three or more pointers having a substantially spherical shape, the imaging device coordinate estimating means, the three pointers labels included in the first image By calculating the inclination of the plane constituted by each, the first direction with respect to the sign of the first imaging device is estimated, and the inclination of the plane constituted by each of the three pointers of the sign included in the second image is calculated. Thus, the second direction relative to the sign of the second imaging device is estimated, and the first distance between the first imaging device and the sign is determined based on a comparison between the area of the sign included in the first image and the known area. And calculating a second distance between the second imaging device and the sign based on a comparison between the area of the sign included in the second image and the known area, and calculating the second distance based on the first direction and the first distance. Estimate the value of the three-dimensional coordinates of the first position of one imaging device Based on the second direction and the second distance, the value of the three-dimensional coordinate of the second position of the second imaging device is estimated, and the specific point coordinate estimating means is configured to transfer the specific point from the first position to one of the specific points. The intersection of the line indicating the direction and the line indicating the direction from the second position to one point of the specific point is calculated, the intersection is estimated as one three-dimensional coordinate of the plurality of specific points, and the plurality of specific points The three-dimensional coordinates of the other points are estimated, and the area calculating means calculates the area of the target region surrounded by the three-dimensional coordinates of each of the plurality of specific points.

  The subject area calculation apparatus of the present invention can calculate the area of the target region in the subject based only on a plurality of images including the sign and the subject. In particular, since the area can be calculated based on the image, the area of the target region in the subject can be calculated in various places as long as there is only the image as well as the site where the subject is located. For this reason, the calculated area can be utilized for various business uses.

  Furthermore, since the recognition and position identification of the sign is easy and accurate even when the angle of imaging is various, the subject area calculation apparatus of the present invention can calculate the area of the target region in the subject with high accuracy.

  Furthermore, the subject area calculation apparatus of the present invention can be calculated including the error rate of the calculated area, and thus has high usability.

A subject area calculation device according to a first aspect of the present invention includes a subject including a target region that is a target of area calculation captured by the first imaging device, a marker that is within the target region, and is placed in front of the target region. And a second image including a subject imaged by the second imaging device from a different angle from the first imaging device, and a sign that is in the target area and is placed on the front surface. Means for estimating the three-dimensional coordinate value of the first position of the first image pickup device and the three-dimensional coordinate value of the second position of the second image pickup device, and the first position and the second position. Based on the position, specific point coordinate estimation means for estimating the value of each of the three-dimensional coordinates of the plurality of specific points included in the subject, and based on the value of each of the three-dimensional coordinates of the plurality of specific points, An area calculating means for calculating an area; The provided, the label has a Rutotomoni known area having a two-dimensional shape with three or more pointers having a substantially spherical shape, the imaging device coordinate estimating means, the three pointers labels included in the first image By calculating the inclination of the plane constituted by each, the first direction with respect to the sign of the first imaging device is estimated, and the inclination of the plane constituted by each of the three pointers of the sign included in the second image is calculated. Thus, the second direction relative to the sign of the second imaging device is estimated, and the first distance between the first imaging device and the sign is determined based on a comparison between the area of the sign included in the first image and the known area. And calculating a second distance between the second imaging device and the sign based on a comparison between the area of the sign included in the second image and the known area, and calculating the second distance based on the first direction and the first distance. Estimate the value of the three-dimensional coordinates of the first position of one imaging device Based on the second direction and the second distance, the value of the three-dimensional coordinate of the second position of the second imaging device is estimated, and the specific point coordinate estimating means is configured to transfer the specific point from the first position to one of the specific points. The intersection of the line indicating the direction and the line indicating the direction from the second position to one point of the specific point is calculated, the intersection is estimated as one three-dimensional coordinate of the plurality of specific points, and the plurality of specific points The three-dimensional coordinates of the other points are estimated, and the area calculating means calculates the area of the target region surrounded by the three-dimensional coordinates of each of the plurality of specific points.

  With this configuration, the area of the subject can be calculated with high accuracy using only the pre-captured image. Particularly, since the value of the three-dimensional coordinate of the subject can be calculated using a sign having three or more pointers, the calculation accuracy is improved.

  In the subject area calculation apparatus according to the second aspect of the present invention, in addition to the first aspect, each of the three or more pointers has a different chromaticity.

  This configuration makes it easy to distinguish each pointer in the image. As a result, the pointer selection accuracy increases, and the position estimation accuracy of the imaging device increases.

In the subject area calculation device according to the third aspect of the present invention, in addition to the first or second aspect, the phase is a triangle having apexes of three pointers having different chromaticities and having a substantially spherical shape. Have

  With this configuration, the distance from the imaging device to the sign can be easily calculated.

In the subject area calculation apparatus according to the seventh invention of the present invention, in addition to any of the first to fourth inventions, the specific point coordinate estimation means is configured to apply the specific point coordinate estimation means from the first position to one of a plurality of specific points. When the line indicating the direction (hereinafter referred to as “first straight line”) and the line indicating the direction from the second position to one of the plurality of specific points (hereinafter referred to as “second straight line”) do not intersect. The distance between the first straight line and the second straight line at a position where the first straight line and the second straight line are close to each other is calculated, the distance is calculated as a mismatch amount, and the area calculation means is based on the mismatch amount, The error rate of the area of the target area is calculated.

  With this configuration, it is possible to know the estimation accuracy of the coordinates of a specific point.

In the subject area calculation apparatus according to the eighth aspect of the present invention, in addition to the seventh aspect, the area calculation means includes a percentage value of the error relative to the calculated area and a level value indicating the error step by step based on the error rate. Then, the area of the target region is calculated by any one of the area ranges including the error to be calculated.

  With this configuration, the user can recognize the certainty of the calculated area.

In the subject area calculation apparatus according to the ninth aspect of the present invention, in addition to any of the first to eighth aspects, each of the plurality of specific points is included in the vertex of the target area.

  With this configuration, the target area is formed by specific points.

The subject area calculation apparatus according to the tenth aspect of the present invention further includes display means for displaying the area of the target region calculated by the area calculation means in addition to any of the first to ninth aspects.

  With this configuration, the user can easily recognize the calculation result.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
Embodiment 1 will be described.

(Overview of the entire system)
FIG. 1 is a conceptual diagram of a subject area calculation system according to Embodiment 1 of the present invention.

  The subject area calculation system 1 analyzes a captured image of the subject 2, the sign 3 that is close to the subject 2, the image pickup device 7 that captures the subject 2 and the sign 3, and the area of the subject 2. A subject area calculation device 8 for calculating

  The subject 2 is a part or the whole of a building or a building, particularly a wall surface of the building or the building. It is a part including a region where the area is to be calculated. For example, the example of the subject 2 may be a wall surface of a building or a condominium, or a signboard installation board of a building or a condominium.

  The sign 3 is a reference member that includes three pointers 4, 5, and 6, and calculates the area of the subject 2 from the three-dimensional coordinates of the imaging device 7 with reference to the sign 3.

  The imaging device 7 is an electronic device or an electronic device having an imaging function such as a camera, a digital camera, a digital video camera, a mobile terminal, or a mobile phone.

  As the subject area calculation device 8, a personal computer is shown as an example in FIG. 1, but not only a personal computer but also a wide range of dedicated devices and general-purpose devices that can perform processing for calculating the subject area. Used.

  In the subject area calculation system 1, first, the sign 3 is installed close to the subject 2, and the imaging device 7 captures the sign 3 and the subject 2 at the same time. At this time, the imaging device 7 receives the first image including the sign 3 and the subject 2 and the second image (or the third image or more) including the sign 3 and the subject 2 captured from an angle different from the first image. ).

  FIG. 2 shows how the first imaging device and the second imaging device image the subject 2 and the sign 3. FIG. 2 is a schematic diagram illustrating imaging of the first image and the second image in the first embodiment of the present invention. The first imaging device 7a captures the subject 2 and the sign 3 from a predetermined direction to obtain a first image. On the other hand, the second imaging device 7b captures the subject 2 and the marker 3 from a different angle from the first imaging device 7a to obtain a second image.

  The subject area calculation device 8 obtains a first image and a second image from the imaging device 7. The first image and the second image include the subject 2 and the sign 3 as described above. The subject area calculation device 8 includes a reading unit, an imaging device coordinate estimation unit, a specific point coordinate estimation unit, and an area calculation unit. The reading unit reads the first image and the second image. The imaging device coordinate estimation means includes a first position represented by a three-dimensional coordinate of an imaging device that captures the first image (referred to as a first imaging device) and an imaging device that captures a second image (referred to as a second imaging device). The second position represented by the three-dimensional coordinates is estimated. The first imaging device 7a and the second imaging device 7b are simply described as “first and second” for convenience of explanation, and the first imaging device 7a and the second imaging device 7b are the same. It may be a device or a different device. This point is the same throughout the present specification.

  When the first position and the second position are estimated, the specific point coordinate estimation means estimates the three-dimensional coordinates of each of the plurality of specific points included in the subject 2 based on the first position and the second position. . If the three-dimensional coordinates of a plurality of specific points included in the subject 2 are determined, the area can be calculated. For this reason, the area calculation means can calculate the area of the subject 2 based on the three-dimensional coordinates of the specific point. The area calculating means calculates the area of a predetermined region surrounded by the specific point in the subject 2.

  The subject area calculation device 8 can calculate the area of the predetermined region of the subject 2 using the first image and the second image obtained from the imaging device 7 in this way.

  The subject area calculation system includes the sign 3, the imaging device 7, and the subject area calculation device 8, so that the area of the predetermined region of the subject 2 can be calculated.

  When the subject 2 is a building or a wall surface of the building, the area of the predetermined area is calculated. For example, the size of a poster or signboard attached to the predetermined area can be grasped before installation. Alternatively, since the area of the predetermined region is calculated, the amount of paint for painting in the predetermined region can be grasped in advance. Furthermore, as long as there are a plurality of images including the subject 2 and the sign 3, the area of the subject 2 is calculated using the subject area calculation device 8 at a place other than the site where the subject 2 is imaged. System.

(Subject area calculation device)
Next, an outline of the subject area calculation device will be described.

  FIG. 3 is a block diagram of the subject area calculation apparatus according to Embodiment 1 of the present invention.

  The subject area calculation device 10 includes a reading unit 12 that reads an image stored in the storage unit 11, an imaging device coordinate estimation unit 13 that estimates a three-dimensional coordinate of the imaging device based on a sign included in the image, and an imaging device 3 Specific point coordinate estimation means 14 for estimating the three-dimensional coordinates of each of a plurality of specific points included in the subject from the dimensional coordinates, and area calculation means 15 for calculating the area of the subject based on the estimated three-dimensional coordinates of the specific points. With. The area value calculated by the area calculating unit 15 is displayed on the display unit 16.

  In FIG. 3, the storage unit 11 and the display unit 16 are displayed outside the subject area calculation device 10, but the subject area calculation device 10 may include the storage unit 11 and the display unit 16. And you may have it outside.

  The subject area calculation device 10 may be realized by a personal computer 8 as shown in FIG. 1 or may be realized by a dedicated device.

  The storage unit 11 stores images including subjects and signs taken from various positions. For example, a first image captured by a first imaging device installed at a first position and a second image captured by a second imaging device installed at a second position, which is a position different from the first position. Remember. Each of the first image and the second image includes a sign and a subject in the image. The storage unit 11 may store an image every time the imaging apparatus captures an image, or may store a plurality of images captured in advance.

  The reading unit 12 reads an image stored in the storage unit 11. At this time, reading may be performed one by one or a plurality of images may be read at a time. The reading unit 12 outputs the read image data to the imaging device coordinate estimation unit 13.

  The imaging device coordinate estimation unit 13 estimates the three-dimensional coordinates of the first position where the first imaging device is arranged, using the plurality of images obtained from the reading unit 12. Similarly, the three-dimensional coordinates of the second position where the second imaging device is arranged are estimated. If there is another image, the three-dimensional coordinates of the third position where the third imaging device is arranged (also the three-dimensional coordinates of the n-th position where the n-th imaging device is arranged) are estimated.

  The imaging device coordinate estimation means 13 estimates the direction between the imaging device and the sign based on the inclination of the sign included in the image. Next, the distance between the imaging device and the sign is estimated based on the area of the sign included in the image. The imaging device coordinate estimation means 13 estimates the three-dimensional coordinates of the position of the imaging device using this direction and distance. The label has a known size, shape, area, and the like.

  The coordinates of the imaging device having local coordinates that are two-dimensional coordinates are converted into global coordinates that are three-dimensional coordinates based on the sign by the imaging device coordinate estimation means 13. Note that the imaging device coordinate estimation unit 13 estimates the value of each three-dimensional coordinate of the imaging device at a plurality of positions. As a result, the value of the three-dimensional coordinate of each imaging device at a different position is estimated.

  The specific point coordinate estimation unit 14 estimates the three-dimensional coordinates of the specific point included in the subject based on the estimated three-dimensional coordinates of the imaging apparatus. The subject has a region for which the area is to be calculated, and the specific point coordinate estimation means 14 determines a plurality of points along the outer edge of this region as specific points. If the values of the global three-dimensional coordinates of the plurality of specific points are obtained, the area of the region surrounded by the specific points can be calculated.

  The value of the three-dimensional coordinates of the imaging device is estimated by the imaging device coordinate estimation means 13. The three-dimensional coordinates of the specific point are estimated based on the three-dimensional coordinates of the imaging device. That is, the direction of a specific point is estimated from a plurality of imaging devices, and the overlapping point in the direction of the specific point estimated from the plurality of imaging devices becomes the value of the three-dimensional coordinate of the specific point. Thus, the value of one three-dimensional coordinate of the specific point is calculated based on the positions of the plurality of imaging devices. The specific point coordinate estimation means 14 calculates the value of the three-dimensional coordinates of each of the plurality of specific points by performing an estimation operation on each of the plurality of specific points.

  The specific point coordinate estimation unit 14 outputs the estimated three-dimensional coordinate values of the plurality of specific points to the area calculation unit 15.

  The area calculating unit 15 calculates the area of the target region of the subject surrounded by the plurality of specific points, using the obtained three-dimensional coordinates of the plurality of specific points (each based on the same coordinate series). Here, it is assumed that the subject is reflected in the image together with the sign.

  If the subject is a building and the target region whose area is to be calculated is a part of the wall surface of the building, the specific point is a point on a line surrounding the part of the wall surface. By calculating based on the three-dimensional coordinates of the specific point, the area calculating unit 15 can calculate the area of the target region. This is because if the three-dimensional coordinates are determined, the area of the target region can be calculated by integral calculation based on the coordinate values.

  The area calculation unit 15 outputs the calculated area value to the display unit 16 as necessary. The display means 16 can display the area value in various forms such as text, video, and audio. The display means 16 may be shared with a monitor of a personal computer. In this case, the display means 16 also functions as a screen for confirming input of data and instructions.

  As described above, the subject area calculation apparatus 10 can calculate the area of the subject using a plurality of images from different positions including the sign and the subject. At this time, the subject area calculation device 10 calculates the global three-dimensional coordinates of the imaging device based on the sign (its shape, size, area, etc. are known) reflected along with the subject. The three-dimensional coordinates of the specific points that form the target area of the subject can be easily calculated. Using the three-dimensional coordinates of the specific point, the subject area calculation device 10 can easily calculate the subject area. In particular, since the sign and the subject appear in the captured image, the subject area calculation device 10 can calculate the area at any location as long as the storage unit 11 stores the image. In addition, if the subject is reflected along with the sign, the area of various target areas can be calculated by selecting the specific point, so the user can select the target area flexibly and calculate the area of the target area. it can.

  Next, the detail of each part is demonstrated.

  Note that the subject area calculation device 10 increases the accuracy of area calculation based on many images taken from various positions and angles with respect to an image in which a subject and a sign are captured. However, since the three-dimensional coordinates of the specific point are obtained from at least two images taken from different positions and angles, the area can be calculated from the two images.

  In the following, for convenience of explanation, the operation of each unit for calculating the area based on two images (first image and second image) will be described with reference to FIG. Note that the use of two images of the first image and the second image is merely an example, and it is needless to say that three or more images may be used. In addition, the first imaging device and the second imaging device may be given names due to differences in imaging positions by the same imaging device, or names given to different imaging devices. There may be. The terms “first, second” are not intended to limit the first imaging device and the second imaging device to separate devices.

(Memory part)
The memory | storage part 11 memorize | stores the image imaged with the imaging device.

  Here, the memory | storage part 11 memorize | stores the 1st image imaged by the 1st imaging device 7a and the 2nd image imaged by the 2nd imaging device 7b.

  As shown in FIG. 2, the first imaging device 7a and the second imaging device 7b are arranged at different positions (different angles with respect to the subject and the sign). The first imaging device 7a and the second imaging device 7b may be any device that can capture moving images and still images, such as a digital camera and a digital video camera.

  The first imaging device 7a and the second imaging device 7b capture an image so that the subject 2 and the marker 3 are in the same image. Further, the first imaging device 7a and the second imaging device 7b image the subject 2 and the marker 3 from different angles with the marker 3 as a reference. For this reason, the obtained first image and second image display the subject 2 and the sign 3 appearing at different angles.

  FIG. 4 shows examples of the first image and the second image. FIG. 4 is a photograph example of the first image and the second image in the first embodiment of the present invention. Each of the first image and the second image includes a block wall that is the subject 2 and a sign 3 having a triangular shape. In addition, a partial area of the subject 2 is surrounded by a line, and the area surrounded by the frame is a target area 20 that is an area calculation target.

  As shown in the first image and the second image in FIG. 4, the first image and the second image capture the subject 2 and the sign 3 from different angles.

  The storage unit 11 stores a first image and a second image as exemplified in FIG.

  The storage unit 11 includes various generally known storage elements and storage devices such as an HDD (hard disk drive), a magnetic memory, a semiconductor memory, a ROM, a RAM, a CD, a DVD, and a card memory. The storage unit 11 may be an independent device or device, or may be an element integrated with the subject area calculation device 10. For example, when the subject area calculation device 10 is realized by a personal computer or a general-purpose device, the storage unit 11 is realized by an HDD or a recording device included in the personal computer or general-purpose device.

  The storage unit 11 may be a portable recording device, and may be configured such that the reading unit 12 can read an image from the storage unit 11 by electrically connecting the storage unit 11 to the reading unit 12. In addition, the storage unit 11 may be connected to the subject area calculation device 10 via a network.

(Reading means)
The reading unit 12 reads an image stored in the storage unit 11. For example, the reading unit 11 includes a data reading device provided in an HDD or a DVD drive, software for driving the reading device, and the like. When the storage unit 11 is a semiconductor memory or an optical disk, the reading unit 12 designates an address including an image to be read and reads data corresponding to the address.

  The reading unit 12 only needs to read an image from the storage unit 11, and the realization unit and configuration thereof may be a known technique or a commonly used technique. When the storage unit 11 can also write data, the reading unit 12 may write data to the storage unit 11 as necessary. For example, the reading unit 12 may write the result calculated by the area calculating unit 15 in the storage unit 11. For this reason, the reading unit 12 is not limited to reading data.

  When the first image and the second image shown in FIG. 4 are stored in the storage unit 11, the reading unit 12 reads the first image and the second image. The reading unit 12 outputs the read data of the first image and the second image to the imaging device coordinate estimation unit 13.

  Note that the reading unit 12 may be realized by hardware, software, or a mixture of hardware and software.

(Imaging device coordinate estimation means)
The imaging device coordinate estimation means 13 estimates the value of the three-dimensional coordinate based on the global coordinates of the first position where the first imaging device 7a is arranged and the second position where the second imaging device 7b is arranged. The imaging device coordinate estimation means 13 estimates the value of the three-dimensional coordinates of the first position and the second position using the marker 3. Based on the values of the three-dimensional coordinates of the first position and the second position, the value of the three-dimensional coordinates of the specific point of the subject 2 can be estimated.

  Here, the imaging device coordinate estimation means 13 estimates the direction of the first imaging device 7a relative to the marker 3 as the first direction based on the inclination of the marker 3 shown in the first image. Next, the imaging device coordinate estimation means 13 estimates the distance between the first imaging device 7a and the marker 3 as the first distance based on the area of the marker 3 in the first image.

  Similarly, the imaging device coordinate estimation unit 13 estimates the direction of the second imaging device 7b with respect to the marker 3 as the second direction based on the inclination of the marker 3 in the second image. Next, the imaging device coordinate estimation means 13 estimates the distance between the second imaging device 7b and the marker 3 as the second distance based on the area of the marker 3 in the second image.

  Note that the imaging device coordinate estimation means 13 estimates the direction between the imaging device and the marker 3 based on a specific part (pointer) included in the marker 3 specifically. For this reason, the sign 3 needs to have a pointer. In particular, since it is necessary to detect the inclination from the pointer, the indicator 3 needs to have three or more pointers.

  The imaging device coordinate estimation means 13 estimates the three-dimensional coordinate value of the first position based on the first direction and the first distance, and the three-dimensional coordinate of the second position based on the second direction and the second distance. Estimate the value of. At this time, the imaging device coordinate estimation means 13 uses the inclination and area of the sign 3 shown in the first image and the second image. For this reason, it is preferable that the label | marker 3 has the structure which can calculate the inclination and area easily only in the state which moved to the image.

  For example, it is preferable that the sign 3 has a known shape and area, and has a shape in which the overall shape of the sign 3 and the area in the image can be easily calculated. In addition, it is preferable that the sign 3 has a shape that allows only the image to be easily discriminated from the inclination of the sign 3.

(About signs)
FIG. 5 is a front view of the sign according to Embodiment 1 of the present invention. As described above, FIG. 5 shows an example in which the sign 3 is easily recognized in the image. Of course, the present invention is not limited to the shape and configuration shown in FIG.

  The sign 3 has three or more pointers. In FIG. 5, the sign 3 has three pointers 21, 22, and 23. The sign 3 has a substantially triangular shape with the three pointers 21, 22, and 23 as vertices by a rod-shaped member that connects the three pointers 21, 22, and 23. At this time, the triangle forming the marker 3 has a predetermined area that is known. If the original area of the sign 3 is known and the area of the sign 3 in the image (that is, the area of the triangle) is calculated, the sign from the imaging device is calculated based on the scale ratio between the calculated area and the original area. A distance up to 3 can be calculated.

  Further, since the sign 3 has a triangle as shown in FIG. 5, the sign 3 has an area with the minimum number of vertices, so that the sign 3 is easily discriminated in the image and the area and size are discriminated. Cheap.

  Thus, since the sign 3 has a triangle having the three pointers 21, 22, and 23 as vertices, (1) the sign 3 in the image can be easily and reliably determined. (2) In the image of the sign 3 It is easy to calculate the area at (3). The advantage is that the distance from the imaging device to the marker 3 can be easily calculated by comparing the area of the marker 3 in the image with the area of the actual marker 3. .

  The imaging device coordinate estimation means 13 estimates the direction between the imaging device and the sign 3 from the inclination of the sign 3 in the image. For example, in the first image in FIG. 4, the sign 3 is tilted to the right when viewed from the front. On the other hand, in the second image in FIG. 4, the sign 3 is inclined to the left as viewed from the front. Whether the first image or the second image is tilted, it can be easily determined how the sign 3 is inclined. In order to estimate the direction of the imaging device and the marker 3, it is important for the imaging device coordinate estimation means 13 that the inclination of the marker 3 can be easily and reliably determined.

  Here, it is also preferable that the three pointers 21, 22, and 23 have different chromaticities. Since the chromaticities are different from each other, the discrimination of the marker 3 in the image is ensured. In particular, since the imaging device coordinate estimation means 13 estimates the directions of the imaging device and the marker 3 with reference to the pointers 21, 22, and 23, the pointers 21, 22, and 23 can be reliably distinguished from each other. preferable. Since the chromaticities of the pointers 21, 22, and 23 are different from each other, the pointers 21, 22, and 23 shown in the image can be individually distinguished.

  Each of the three pointers 21, 22, 23 preferably has a substantially spherical shape.

  The imaging device coordinate estimation unit 13 estimates the direction from the imaging device with reference to each of the pointers 21, 22, and 23 included in the sign 3. For this reason, the imaging device coordinate estimation means 13 needs to be able to reliably recognize the pointer in the image regardless of the manner in which the sign 3 is reflected in the image. At this time, since each of the pointers 21, 22, and 23 has a substantially spherical shape, the shape of the pointers 21, 22, and 23 does not change regardless of the angle at which the marker 3 is imaged (shown in the image). In shape). For this reason, the imaging device coordinate estimation means 13 can reliably recognize the pointer from the image.

  Actually, the direction and distance between the pointer and the imaging device are calculated by the user selecting the pointer in the first image or the second image displayed on the display means. At this time, if the shape of the pointer differs depending on the angle being imaged, accurate selection cannot be made. Since the pointer has a substantially spherical shape, the pointer can be accurately selected without depending on the angle at which the image is captured (without using a different image).

  As described above, the pointers 21, 22, and 23 have different chromaticities or have a substantially spherical shape, so that (1) the pointer in the image can be easily and reliably determined. (2) the image (3) Since the pointer can be accurately selected, there is a merit that the direction of the imaging device relative to the marker 3 can be reliably estimated.

  The imaging device coordinate estimation means 13 estimates the direction of the first imaging device 7a relative to the marker 3 from each of the pointers 21, 22, and 23 shown in the first image. Similarly, the imaging device coordinate estimation means 13 estimates the direction of the second imaging device 7b with respect to the marker 3 from each of the pointers 21, 22, and 23 shown in the second image.

  FIG. 6 shows a state in which the direction of the imaging device relative to the marker 3 is estimated. FIG. 6 is a schematic diagram of direction estimation of the imaging apparatus according to Embodiment 1 of the present invention.

  The sign 3 has three pointers 21, 22, and 23. The first imaging device 7a disposed at the first position captures the first image and is disposed at the second position. 7b images the second image. A sign 3 having three pointers 21, 22, 23 is shown in both the first image and the second image.

  The imaging device coordinate estimation means 13 selects the pointer 21, the pointer 22, and the pointer 23 in the first image, and the inclination of the plane composed of each of the pointers 21, 22, 23 (the triangular plane of the sign 3) is determined. calculate. By this inclination, the first direction with respect to the marker 3 of the first imaging device 7a is estimated. Note that the imaging device coordinate estimation unit 13 may estimate the first direction based on a table that clearly indicates the correlation between the shape of the marker 3 in the image and the inclination of the marker 3.

  Similarly, the imaging device coordinate estimation unit 13 selects the pointers 21, 22, and 23 in the second image, and the inclination of the plane formed by each of the pointers 21, 22, and 23 (the triangular plane of the sign 3). Is calculated. Based on this inclination, the second direction with respect to the marker 3 of the first imaging device 7b is estimated. The second direction may also be estimated based on a table that clearly shows the correlation between the shape of the sign 3 in the image and the inclination of the sign 3.

  As shown in FIG. 6, both the first direction and the second direction are estimated by vectors connecting the points of the pointers 21, 22, and 23 and the imaging device.

  Next, the imaging device coordinate estimation means 13 estimates the distance from the imaging device to the marker 3 from a comparison between the area of the marker 3 in the image and the area of the original marker 3.

  FIG. 7 shows the state of distance estimation. FIG. 7 is a schematic diagram illustrating estimation of the distance between the imaging device and the sign according to Embodiment 1 of the present invention.

  As is clear from FIG. 7, the imaging apparatus images the sign 3 while maintaining the linearity, and therefore the distance from the imaging apparatus to the sign 3 is the area of the sign 3 at the position of the imaging apparatus (a known unique area). And the area of the marker 3 calculated in the image.

  The imaging device coordinate estimation means 13 calculates the area of the marker 3 in the first image and compares it with the known area, and estimates the distance from the first imaging device 7a to the marker 3 as the first distance. Similarly, the imaging device coordinate estimation means 13 calculates the area of the marker 3 in the second image and compares it with the known area, and estimates the distance from the second imaging device 7b to the marker 3 as the second distance. .

  If the direction and distance of the first imaging device 7a with reference to the sign 3 are known, the two-dimensional coordinates in the image of the first imaging device 7a are converted into global three-dimensional coordinates with the reference 3 as a reference. That is, the value of the three-dimensional coordinates of the first position is obtained. Similarly, if the direction and distance of the second imaging device 7b with reference to the marker 3 are known, the two-dimensional coordinates in the image of the second imaging device 7b are converted into global three-dimensional coordinates with the marker 3 as a reference. The

  From this, the imaging apparatus coordinate estimation means 13 estimates the value of the three-dimensional coordinates of the first position from the first direction and the first distance, and the three-dimensional coordinates of the second direction, the second distance and the empty second position. Estimate the value of.

  Through the above processing procedure, the imaging device coordinate estimation means 13 estimates the values of the three-dimensional coordinates between the first position where the first imaging device 7a is arranged and the second position where the second imaging device 7b is arranged. To do.

(Specific point coordinate estimation means)
The imaging device coordinate estimation unit 13 outputs the values of the three-dimensional coordinates of the first position and the second position to the specific point coordinate estimation unit 14. The specific point coordinate estimation means 14 estimates the value of the three-dimensional coordinate of the specific point of the subject 2 based on the value of the three-dimensional coordinate of the first position and the second position.

  FIG. 8 is a photograph for explaining the estimation of specific point coordinates in the first embodiment of the present invention. The image shown in FIG. 8 includes the subject 2 imaged together with the sign 3, and the subject 2 includes a target region 20 that is a target for calculating an area surrounded by a frame.

  The target area 20 includes a plurality of specific points that are arbitrarily determined. If the three-dimensional coordinates of the specific point are known, the area of the target area 20 is calculated. In FIG. 8, the target area 20 is a rectangular area, and four vertices of the target area 20 are specified as four specific points 30, 31, 32, and 33. The specific point may be arbitrarily determined, but may be included at the apex of the target region for ease of area calculation. Alternatively, the specific point may be determined by combining an arbitrary point on the side of the outer frame of the target region and the vertex.

  Actually, the user selects an arbitrary point as a specific point on the image (image illustrated in FIG. 8) displayed on the display means.

  The specific point coordinate estimation unit 14 estimates the direction from the first position and the direction from the second position for each of the specific points 30, 31, 32, and 33. For example, the direction from the first position and the direction from the second position with respect to one specific point 30 are estimated. At this time, since each of the first position and the second position has a three-dimensional coordinate value, the direction to the specific point 30 is estimated based on the three-dimensional coordinate value. Furthermore, the specific point coordinate estimation means 14 obtains the value of the three-dimensional coordinates of the intersection of the direction from the first position and the direction from the second position. The value of the three-dimensional coordinate of the intersection is the value of the three-dimensional coordinate of the specific point 30 to be obtained. FIG. 9 shows the estimation state of the three-dimensional coordinates of the specific point.

  FIG. 9 is a schematic diagram for explaining the estimation of the three-dimensional coordinates of the specific point in the first embodiment of the present invention.

  As shown in FIG. 9, a line 40 indicating the direction from the first imaging device 7 a (first position) to the specific point 30 and a line indicating the direction from the second imaging device 7 b (second position) to the specific point 30. 41 is drawn. As is clear from FIG. 9, the line 40 and the line 41 intersect at a certain position. This position is the intersection 42. The line 40 is a line connecting the specific point 30 shown in the first image and the first position, and the line 41 is a line connecting the specific point 30 shown in the second image and the second position. The three-dimensional coordinates of the specific point 30 cannot be accurately obtained by only a single image depending on the captured angle and position. On the other hand, the common item (intersection point) of the direction to the specific point 30 in each image is obtained by a plurality of images having different imaging angles. This intersection point can be closer to the position of the specific point 30 than a single image alone.

  In this way, in each of the first image and the second image, the line connecting the specific point from the first position and the line connecting the specific point from the second position are calculated, and the intersection of the lines is calculated. Thus, the three-dimensional coordinates of the specific point (coordinates common to the three-dimensional coordinates of the first position and the second position) are obtained.

  The specific point coordinate estimation means 14 also obtains the three-dimensional coordinate values of the remaining specific points 31, 32, and 33.

  Here, the processing for obtaining the three-dimensional coordinates of the specific point based on the two images of the first image and the second image has been described, but based on three or more images (that is, arranged at the third position). The value of the three-dimensional coordinates of the specific point may be calculated (based on the third imaging device).

  The specific points may be some points on the outer frame surrounding the target area, or the side is selected and a number of specific points are selected on the assumption that the sides of the outer frame are a collection of specific points. May be. In any case, the greater the number of specific points, the higher the accuracy of area calculation of the target region in the area calculation means 15 described later.

  Further, the specific point coordinate estimation means 14 may estimate the intersection of the straight line from the first position and the straight line from the second position as the position of the specific point, but the direction from the first position to the specific point First, the value of the three-dimensional coordinate of the specific point based on the first position (referred to as the first specific coordinate) is estimated, and then the specification based on the second position is performed based on the direction and distance from the second position to the specific point. After estimating the value of the three-dimensional coordinate of the point (referred to as the second specific coordinate), the value of the three-dimensional coordinate of the specific point may be estimated based on the first specific coordinate and the second specific coordinate.

  At this time, the distance to the subject 2 including the specific point may be considered to be substantially the same as the distance from the first position or the second position to the marker 3. The direction is determined by the angle between the position selected as the specific point and the coordinates of the first position (second position).

  At this time, when the first specific coordinate and the second specific coordinate match, the value of the first specific coordinate and the second specific coordinate is estimated as the value of the three-dimensional coordinate of the specific point. When the first specific coordinates and the second specific coordinates do not match, as described in the second embodiment, the proximity position between the first specific coordinates and the second specific coordinates is determined as a position that approximates the specific point. It is done.

  As described above, the specific point coordinate estimation unit 14 estimates the three-dimensional coordinates of the specific point and outputs the estimation result to the area calculation unit 15.

  The specific point coordinate estimation means 14 may be realized by hardware, software, or a mixture of hardware and software.

(Area calculation means)
Next, the area calculation means 15 will be described.

  The area calculating unit 15 calculates the area of the target region 20 based on the value of the three-dimensional coordinates of the specific point indicating the target region 20 estimated by the specific point coordinate estimating unit 14.

  The plurality of specific points have a common three-dimensional coordinate value. The area of the target region 20 surrounded by a plurality of specific points can be calculated using three-dimensional coordinates. Since the length of each side forming the target region 20 and the angle formed by the sides are calculated based on the value of the three-dimensional coordinates, the area calculating unit 15 uses the length and the angle to calculate the area of the target region 20. It can be calculated.

  Here, the area calculation means 15 can use various known techniques and known techniques for calculating the area of the region based on the three-dimensional coordinates of the points forming the region. These may be referred to various known techniques and known techniques, and will not be described.

  The area calculation unit 15 outputs the calculated area value of the target region 20 to the display unit 16 as necessary.

  The display means 16 includes a liquid crystal screen, a CRT display, and the like, and displays the calculated area value as text or an image.

  In response to this indication, the user can know the area of the target area in consideration of painting in the building that is the subject, and can plan the painting and arrange the materials based on this area. .

  The display means 16 includes at least the subject 2, the target area 20 in the subject 2, the estimated three-dimensional coordinate values of the first position and the second position, the estimated three-dimensional coordinate value of the specific point, and at least the calculated area. Display a part. All these elements may be displayed simultaneously. The display means 16 can display all the various information provided for the convenience of the user, and only one of the information may be displayed depending on the selection of the user.

  The subject area calculation device 10 described above may be realized by a dedicated device in which a part or all of the functions are hardware elements (semiconductor integrated circuits and electronic circuits), or hardware for general-purpose devices such as personal computers. These elements may be added, or may be realized by incorporating software (computer program) having some or all of the functions into a general-purpose device.

  Actually, a subject area calculation device mounted on a computer or a dedicated device is used by a user to calculate the area of the target region.

  The user reads an image stored in the storage unit 11 using a computer or a key input device such as a dedicated device. At this time, the reading unit 12 operates to read the image stored in the storage unit 11. The read images are, for example, a first image and a second image as shown in FIG. The user drags each pointer of the sign 3 using the mouse or pointing device in the first image and the second image. By this dragging, the imaging device coordinate estimation means 13 is operated, and the value of the three-dimensional coordinates of the first position, which is the position where the first image is captured, and the three-dimensional position of the second position, which is the position where the second image is captured. A coordinate value is estimated.

  Next, as shown in FIG. 8, the user uses the mouse or pointing device to specify specific points 30, 31, 32, and 33 that specify the target area 20 in the first image or the second image. Drag. By this dragging, the specific point coordinate estimating means is operated, and the values of the three-dimensional coordinates of the specific points 30, 31, 32, 33 are estimated. Further, the area calculating means 15 is operated to calculate the area of the target region 20 formed by the specific points 30, 31, 32, 33. In both cases, the drag on the image with the mouse or pointing device is due to the determination of the coordinate value in the image.

  As described above, when the subject area calculation device is mounted on a computer or a dedicated device, the user designates a necessary point (a pointer of the marker 3 or a specific point of the target area 20) in the displayed image. The area of the target region 20 can be calculated simply by doing.

  As described above, the subject area calculation apparatus according to Embodiment 1 can calculate the area of the target region with high accuracy even in a place where the subject does not actually exist.

  In particular, in order to increase the accuracy of calculating the area of the target region, it is necessary that the estimation accuracy of the three-dimensional coordinates of the specific point is high. In order to increase the estimation accuracy of the three-dimensional coordinates of the specific point, the three-dimensional coordinates of the position of the imaging device when the image is captured must be estimated with high accuracy.

  The subject area calculation apparatus according to Embodiment 1 uses a sign having three or more pointers that are substantially spherical shapes that are easy to identify regardless of the imaging angle and that have different chromaticities in calculating the three-dimensional coordinates of the position of the imaging apparatus. . This pointer serves as a reference for the three-dimensional coordinates of the imaging apparatus. By using an image obtained by capturing a sign having such characteristics together with the subject, the subject area calculation device can estimate the three-dimensional coordinates of the imaging device and the specific point with high accuracy. The area of the target region can be calculated.

(Embodiment 2)
Next, a second embodiment will be described.

  In the second embodiment, a device for estimating a specific point coordinate will be described.

  The specific point coordinate estimation means estimates the three-dimensional coordinates of the specific point from the first position and the second position. At this time, the intersection of the straight line extending from the first toward the specific point and the straight line extending from the second position toward the specific point is estimated as the position of the specific point. At this time, not only the intersection from the two positions of the first position and the second position, but also based on the third position, the three-dimensional coordinates of the specific point can be estimated with high accuracy. That is, based on three or more captured images from different angles, the specific point coordinate estimation means can estimate the three-dimensional coordinates of the specific point with high accuracy.

  As described above, the area of the target region of the subject is calculated with high accuracy by estimating the three-dimensional coordinates of the specific point not only from the two imaging positions but also from three or more imaging positions.

  Here, it may occur that each straight line formed from each imaging position to a specific point does not intersect at one point but is twisted. The straight line from the first position to the specific point is determined by the direction and distance between the specific point selected in the image and the first position, and the straight line from the second position to the specific point is the specific point selected in the image. And the direction and distance from the second position. At this time, the specific point indicates the position selected by the user in the image that is a plane, and is not necessarily the same position as the original specific point that is uniquely determined in three dimensions. For this reason, the straight line formed from the first position to the specific point (specific point selected in the image) and the straight line formed from the second position to the specific point (specific point selected in the image) intersect. Not happening can happen.

  FIG. 10 shows such a twisted state. FIG. 10 is an explanatory diagram for estimating the three-dimensional coordinates of a specific point according to Embodiment 2 of the present invention. FIG. 10 shows a case where each of the three straight lines 50, 51, 52 formed from the three imaging positions to the specific point does not intersect.

  In the case shown in FIG. 10, the three-dimensional coordinates of the specific point are estimated by approximation. In addition, also when estimating the three-dimensional coordinate of a specific point from two imaging positions, the approximation demonstrated below can be used.

  This will be described with reference to FIGS. 3, 10, and 11. FIG. 11 is an explanatory diagram for explaining approximation of a specific point in the second embodiment of the present invention.

  The specific point coordinate estimation unit 14 determines the 3 of the specific points based on the first image, the second image, and the third image captured by the first imaging device 7a, the second imaging device 7b, and the third imaging device 7c, respectively. Estimate dimensional coordinates. First, the imaging device coordinate estimation means 13 uses the first position, which is the three-dimensional coordinate of the first imaging device 7a, the second position, which is the three-dimensional coordinate of the second imaging device 7b, and the three-dimensional coordinate of the third imaging device 7c. A certain third position is estimated. The imaging device coordinate estimation unit 13 outputs information on the first position, the second position, and the third position to the specific point coordinate estimation unit 14.

  The specific point coordinate estimation unit 14 forms a straight line 50 extending from the first position toward the specific point. Similarly, the specific point coordinate estimation unit 14 forms a straight line 51 extending from the second position toward the specific point, and a straight line 52 extending from the third position toward the specific point. As shown in FIG. 10, each of the straight line 50, the straight line 51, and the straight line 52 does not intersect all but has a twisted relationship. Thus, when the three straight lines 50, 51, 52 do not intersect, the three-dimensional coordinates of the specific point cannot be uniquely determined. In FIG. 10, the straight line 50 and the straight line 51 do not intersect, the straight line 51 and the straight line 52 do not intersect, and the straight line 52 and the straight line 50 do not intersect.

  Therefore, as shown in FIG. 11, the specific point coordinate estimation unit 14 approximates the three-dimensional coordinates of the specific point from the three straight lines 50, 51, 52.

  First, the specific point coordinate estimation unit 14 obtains a proximity point between a straight line 50 extending from the first position of the first imaging device 7a and a straight line 51 extending from the second position of the second imaging device 7b. At this time, on the line connecting the positions where the straight line 50 and the straight line 51 are close to each other, the specific point coordinate estimating means 14 obtains the close point by weighting. Here, the weighting of the straight line 50 from the first position and the straight line 51 from the second position is 1: 1. Therefore, an intermediate position between the positions close to the straight line 50 and the straight line 51 is set as a proximity point between the straight line 50 and the straight line 51. A proximity point between the straight line 50 and the straight line 51 is defined as a first proximity point.

  Next, the specific point coordinate estimation unit 14 calculates a proximity point between the first proximity point and the straight line 52. The specific point coordinate estimation unit 14 obtains a proximity point by weighting on a line connecting positions where the first proximity point and the straight line 52 are close to each other. A proximity point between the first proximity point and the straight line 52 is defined as a second proximity point. Here, the weight of the proximity point between the straight line 50 and the straight line 51 and the straight line 52 is 2: 1.

  This is because the first proximity point indicates a position where the two straight lines of the straight line 50 and the straight line 51 are close to each other, and therefore it is appropriate to set the weighting ratio of the straight line 52 that is one straight line to 2: 1.

  The specific point coordinate estimation means 14 determines the position where the ratio of the first specific point and the straight line 52 is 2: 1 on the line connecting the position where the straight line 52 is close to the first specific point and the first specific point. Calculated as 2 proximity points. This second proximity point is a point approximated as an intersection of the three straight lines 50, 51, 52. The specific point coordinate estimating unit 14 estimates the value of the three-dimensional coordinate of the second proximity point as the value of the three-dimensional coordinate of the specific point, and outputs it to the area calculating unit 15.

  When estimating the three-dimensional coordinates of the specific point based on the two images, if the straight line from the first position does not intersect with the straight line from the second position, the adjacent point between the straight line 50 and the straight line 51 is used. A specific point is approximated in the way of calculating a certain first proximity point.

  As described above, the three-dimensional coordinates of the specific point can be estimated with higher accuracy by obtaining the three-dimensional coordinates of the specific point based on three or more images captured at three or more imaging positions.

  In the second embodiment, the approximation by weighting has been described. However, the three-dimensional coordinates of a specific point may be estimated by various known approximations. For example, linear approximation, least squares, or bundle adjustment may be used.

  Further, the specific point coordinate estimation unit 14 determines the first specific coordinate which is a three-dimensional coordinate of the specific point determined by the direction and distance from the first position to the specific point, and the direction and distance from the second position to the specific point. You may approximate the three-dimensional coordinate of a specific point based on the 2nd specific coordinate (or 3rd specific coordinate is included) which is the three-dimensional coordinate of the defined specific point.

  For example, the first specific coordinates are (x1, y1, z1), the second specific coordinates are (x2, y2, z2), and the weighting ratio is the first specific coordinates: second specific coordinates = 1: When it is 1, the value of the three-dimensional coordinates of the specific point to be approximated is ((x1 + x2) / 2, (y1 + y2) / 2, (z1 + z2) / 2). Of course, when based on three or more imaging positions, the value of the three-dimensional coordinate of the specific point is approximated by weighting calculation with the value of the third specific coordinate or the fourth specific coordinate.

(Other aspects of specific point coordinate estimation)
In the specific point coordinate estimation, it is also preferable that a plurality of straight lines extending from the position of each of the plurality of imaging devices to the specific point are used, and a range surrounded by the plurality of straight lines is estimated as the coordinates of the specific point.

  FIG. 13 is a schematic diagram for explaining the estimation of specific point coordinates in the second embodiment of the present invention. FIG. 13 shows a range surrounded by a plurality of straight lines by extending a plurality of straight lines from the first imaging device 7a toward a specific point and extending a plurality of straight lines from the second imaging device 7b toward the specific point. , It is regarded as the location of the three-dimensional coordinates of the specific point.

  The first position of the first imaging device 7 a is determined by the imaging device coordinate estimation means 13. The user selects a position that seems to be a specific point in the first image. A straight line is formed from the first position to the position of the selected specific point with reference to the three-dimensional coordinates of the first position. At this time, by changing the position regarded as the specific point with a minute difference, a plurality of minutely different straight lines are formed from the first position toward the specific point.

  In FIG. 13, a specific point is selected at three positions 71a, 71b, 71c with the first position as a reference. Similarly, a specific point is selected at three positions 72a, 72b, 72c with the second position as a reference. From the first position, three straight lines are formed at each of the positions 71a, 71b, and 71c, and from the second position, three straight lines are formed at each of the positions 72a, 72b, and 72c. That is, a total of six straight lines are formed for one specific point. The six straight lines form an outer frame with each other. A region 70 is formed by the outer frame. The specific point coordinate estimation means 14 calculates the value of the three-dimensional coordinate of this area 70 and estimates this value as the value of the three-dimensional coordinate of the specific point.

  The original specific point is estimated to be located in this region 70. Since the region 70 formed based on many options (here, the selected positions 71a, 71, 71c, etc.) estimates the position of the specific point, the accuracy of estimating the specific point is improved.

  Furthermore, as shown in FIG. 14, the region 75 surrounded by all the straight lines including the straight line from the third position, which is the imaging position of the third imaging device 7c, to the selected position is the case of FIG. Compared to the region 70, the region is further limited. For this reason, it is estimated that the specific point is located in the region 75, and the estimation accuracy of the specific point is increased. FIG. 14 is a schematic diagram for explaining estimation of specific point coordinates according to the second embodiment of the present invention.

  As described above, the specific point coordinate estimation unit 14 calculates the coordinates of the specific point with high accuracy based on the plurality of straight lines formed from the positions of the plurality of imaging devices toward the position selected as the specific point. presume.

(Calculation of area error rate)
The subject area calculation device 10 also preferably calculates at least one of the calculated area error rate and the area error range based on the estimation accuracy of the specific point.

  When the straight line from the first position to the specific point and the straight line from the second position to the specific point do not intersect, the specific point coordinate estimation unit 14 calculates the amount of mismatch caused by the non-intersection. Alternatively, the amount of mismatch between the first specific coordinates and the second specific coordinates is calculated.

  When the straight line from the first position to the specific point and the straight line from the second position to the specific point do not intersect, a separation amount of each straight line is generated. The specific point coordinate estimation unit 14 calculates a proximity point at a position where the straight line from the first position and the straight line from the second position are close to each other. At this time, the distance between the two straight lines at the position where the proximity point is calculated is calculated. The large separation amount indicates that the calculation accuracy of the proximity point calculated as the approximate position of the specific point is low, and the small separation amount indicates that the calculation accuracy of the proximity point calculated as the approximate position of the specific point is high. Show. In other words, it indicates the estimation accuracy of a specific point. A high estimation accuracy of the three-dimensional coordinates of the specific point indicates that the calculation accuracy of the area calculated by the area calculation means 15 is high. Conversely, a low estimation accuracy of the three-dimensional coordinates of the specific point indicates that the calculation accuracy of the area calculated by the area calculation means 15 is low.

  The specific point coordinate estimation means 14 calculates this separation amount as a mismatch amount.

  Alternatively, the specific point coordinate estimation unit 14 may determine the first specific coordinate determined by the direction and distance from the first position to the specific point and the second specific coordinate (or the second specific coordinate determined by the direction and distance from the second position to the specific point). When the three-dimensional coordinates of the specific point are estimated based on (including the third specific coordinate and thereafter), the difference value between the value of the first specific coordinate and the value of the second specific coordinate is calculated as a mismatch amount.

  The area calculation means 15 calculates an error rate of the area of the target region to be calculated based on this mismatch amount. For example, the area calculation unit 15 calculates an area error rate by multiplying the mismatch amount by the calculated area. Alternatively, the error rate is calculated based on the relationship between the level value (level evaluation from level 1 to level 5 depending on the amount of mismatch) and the area value, not the absolute value. For example, if the discrepancy amount is level 1 when the area is within a predetermined range, the error rate is defined as 1%. If the discrepancy amount is level 2, the error rate is defined as 2%. To do.

  Alternatively, the calculated area may be calculated by clearly indicating an area error range, such as “a range from ** square meters to XX square meters”. This is because the convenience of the user is further increased by specifying the error range.

  As described above, the specific point coordinate estimation unit 14 calculates the mismatch amount, and the area calculation unit 15 calculates the error rate. The calculated error rate is displayed on the display unit 16 together with the calculated area. Displaying the error rate increases the convenience for the user.

  For example, as shown in FIG. 12, the target region 61 and the area and error rate 62 are displayed at a time on the display screen 60. FIG. 12 is a schematic diagram of a display unit according to Embodiment 2 of the present invention. By this display, the user can visually recognize the shape and form of the target area selected by the user, and can also visually recognize the calculated area and its error rate.

  The subject area calculation apparatus according to the second embodiment can improve the calculation accuracy of the subject area and can also calculate the error rate of the calculated area.

(Embodiment 3)
Next, a third embodiment will be described.

  In the third embodiment, a subject area calculation system that provides various conveniences to the user using the subject area calculation device described in the first and second embodiments will be described.

  FIG. 15 is a schematic diagram of a subject area calculation system according to Embodiment 3 of the present invention.

  The subject area calculation system 80 includes a subject area calculation device 81, an imaging device 82, a first display device 83, and a second display device 84. The subject area calculation device 81 has the functions and configurations described in the first and second embodiments. The imaging device 82 is a device that can capture an image, such as a digital camera or a digital video camera.

  The imaging device 82 includes a transmission unit that transmits the captured image to the subject area calculation device 81. The transmission unit transmits the captured image to the subject area calculation device 81 as digital data or analog data regardless of wireless or wired. The subject area calculation device 81 includes receiving means for receiving transmitted image data. The transmitting means and the receiving means may use, for example, infrared communication, digital communication, communication using a connection cable, or the like.

  The subject area calculation device 81 stores the transmitted image and calculates the area of the subject based on the user's operation.

  The first display device 83 displays information and results related to area calculation such as the calculated area and error rate. The first display device 83 includes a liquid crystal display, a CRT display, and the like.

  The second display device 84 displays information that is convenient for the user based on the calculated area.

  For example, the subject area calculation device 81 creates an answer to a matter requested by the user based on the calculated area of the target region. As an example, the user needs to apply paint to the target area, and the user instructs the subject area calculation device 81 to calculate the amount of paint required in the target area. Suppose that In this case, the subject area calculation device 81 calculates the amount of paint required to paint the target region from the amount of paint required per unit area and the calculated area value. The second display device 84 displays the calculated amount of paint.

  Alternatively, it is assumed that the user instructs the subject area calculation device 81 to calculate the size of a signboard that can be set in the target area. In this case, the subject area calculation device 81 compares the calculated area of the target area with the type of signboard size, and calculates the size of the signboard that can be set in the target area. For example, it is calculated that an L-size signboard can be installed. The second display device 84 displays the size of the signboard that can be set as “L size”.

  As described above, the subject area calculation device 81 calculates various items that can be calculated based on the area, and displays them on the second display device 84. Of course, the first display device 83 may display these items.

  When the required amount of paint and the size of the signboard are calculated, the calculation result may be output to a program for painting work or installation work flow. For example, when a subject area calculation system is combined with an apparatus for creating a signboard, a size instruction is added to the design prepared in advance from the subject area calculation system. As a result, an apparatus for creating a signboard can create a signboard of an optimal size for the installation location without requiring user input or instruction.

  In this manner, various operations can be automated by combining a device for performing painting work, a device for creating a signboard, and a subject area calculation system in series.

  For convenience of explanation, the first display device 83 and the second display device 84 have been described as separate elements. However, the first display apparatus 83 and the second display device need not be separate elements. 84 may be integrated.

(Embodiment 4)
Next, a fourth embodiment will be described.

  The subject area calculation device described in the first and second embodiments may be realized as a program operable on a computer.

  FIG. 16 is a block diagram of a subject area calculation apparatus according to the fourth embodiment of the present invention, and FIG. 17 is a flowchart showing subject area calculation processing according to the fourth embodiment of the present invention.

  The computer 100 is a processing device capable of executing a program having the function of the subject area calculation device described in the first and second embodiments. The computer 100 may be a general-purpose computer, a workstation, or a part or all of a processing device having a dedicated device.

  The computer 100 includes a storage unit 101 inside or outside. The storage unit 101 includes various devices that can store electronic data, such as a hard disk drive, a semiconductor memory, a magnetic memory, an optical disk, and a magnetic disk. The computer 100 also includes a CPU (Central Processing Unit) 102 that executes program reading and instruction processing, a ROM 103 that stores programs, a RAM (Random Access Memory) 104 that reads and writes data necessary for operations, and instructions according to the CPU 102. A communication I / F 106 and a display unit 107 for exchanging data with a calculation unit 105 that performs calculation, a display unit 107 that displays calculation results and images, and the like are provided. Note that the display unit 107 is also provided inside or outside the computer 100. Further, the elements included in the computer 100 may be provided inside or outside the computer 100.

  The CPU 102 reads a program stored in the ROM. The program has a function of calculating the area of the subject (subject area of the subject) included in the image in accordance with a user instruction. The CPU 102 reads out a plurality of images including the subject and the sign stored in the storage unit 101 in accordance with the instructions of the read program. The CPU 102 temporarily stores the read image in the RAM 104. Furthermore, the CPU 102 displays the read image on the display unit 107. By displaying an image on the display unit 107, the user can designate a pointer included in the sign, designate a specific point of the target region, and can instruct calculation of the area of the target region. The CPU 102 can display an image on the display unit 107 via the communication I / F 106 and can receive a user instruction through the image via the communication I / F 106.

  In accordance with the user's instruction, the CPU 102 also uses the calculation unit 105 to estimate the three-dimensional coordinates of the imaging apparatus, and then estimates the three-dimensional coordinates of the specific point. When the three-dimensional coordinates of the specific points that form the target area of the subject are estimated, the CPU 102 also calculates the area of the target area using the calculation unit 105.

  The CPU 102 outputs various information such as the calculated area and error rate to the display unit 107 via the communication I / F 106.

  As described above, the computer 100 executes the program stored in the ROM 103 and calculates the area of the subject from the stored image.

  Here, the program stored in the ROM 103 includes steps for calculating the area of the subject while also using instructions from the user. That is, the program reads a first image including a subject and a sign captured by the first imaging device, a second image including the subject and a sign captured by the second imaging device, and the first imaging device. Estimating a three-dimensional coordinate value between a first position that is an imaging position and a second position that is an imaging position of the second imaging device, and a plurality of identifications included in the subject based on the first position and the second position A step of estimating the value of the three-dimensional coordinate of the point, and a step of calculating the area of the subject based on the value of the three-dimensional coordinate of the plurality of specific points.

  The steps of the program will be described with reference to FIGS. In the flowchart of FIG. 17, the work performed by the user is clearly shown on the right side, and the function of the program itself and the operation of the computer 100 based on this function are clearly shown on the left side.

  In step ST1, the user starts the program. In addition, the memory | storage part 101 shall memorize | store the imaged image previously.

  In step ST2, the CPU 102 reads out the program stored in the ROM 103. Next, in step ST3, the CPU 102 reads out images (first image, second image,...) Stored in the storage unit 101. Next, in step ST <b> 4, the CPU 102 displays the read image on the display unit 107 via the communication I / F 106. Note that the CPU 102 may display a plurality of images on the display unit 107 while reading them one by one, or may read a plurality of images at a time and display them together on the display unit 107.

  When the display unit 107 displays the image, the user can give an instruction to the image. In step ST5, the user designates a pointer of a sign included in the image. For example, drag the pointer using a mouse or pointing device. Based on this designation, the CPU 102 estimates the value of the three-dimensional coordinates of the position of the imaging device (first imaging device, second imaging device,...) In step ST6.

  Further, in step ST7, the user designates a specific point that forms the target area of the subject included in the image. This is the same operation as specifying a pointer. In response to this designation, the CPU 102 estimates the value of the three-dimensional coordinates of the specific point in step ST8.

  Next, in step ST9, the CPU 102 calculates the area of the target region based on the value of the three-dimensional coordinates of the specific point. At this time, as described in Embodiment 2, the error rate of the calculated area may also be calculated.

  Finally, in step ST110, the CPU 102 displays the calculated area value, error rate, error range, imaging position, 3D coordinate value of the specific point, and the like on the display unit 107. The user confirms the display result and obtains necessary information.

  Note that the program may include various detailed steps described in the first and second embodiments in estimating the value of the three-dimensional coordinates of the specific point and calculating the area. Further, as described in the third embodiment, a separate step of using the calculated area may be included.

  As described above, some or all of the functions of the subject area calculation device are implemented by a computer-operable program, so that it can be executed on a general-purpose computer or used for a wide range of purposes. Benefits arise.

  As described above, the subject area calculation device, the subject area calculation system, and the subject area calculation method described in the first to fourth embodiments are examples for explaining the gist of the present invention, and modifications are made without departing from the gist of the present invention. And including modifications.

1 is a conceptual diagram of a subject area calculation system in Embodiment 1 of the present invention. It is a schematic diagram explaining the imaging of the 1st image and 2nd image in Embodiment 1 of this invention. 1 is a block diagram of a subject area calculation device in Embodiment 1 of the present invention. FIG. It is an example of a photograph of the 1st picture and the 2nd picture in Embodiment 1 of the present invention. It is a front view of the label | marker in Embodiment 1 of this invention. It is a schematic diagram of the direction estimation of the imaging device in Embodiment 1 of this invention. It is a schematic diagram explaining the estimation of the distance between the imaging device and the sign in Embodiment 1 of the present invention. It is a photograph explaining the estimation of the specific point coordinate in Embodiment 1 of this invention. It is a schematic diagram explaining estimation of the three-dimensional coordinate of the specific point in Embodiment 1 of this invention. It is explanatory drawing which estimates the three-dimensional coordinate of the specific point in Embodiment 2 of this invention. It is explanatory drawing explaining the approximation of the specific point in Embodiment 2 of this invention. It is a schematic diagram of the display part in Embodiment 2 of this invention. It is a schematic diagram explaining the estimation of the specific point coordinate in Embodiment 2 of this invention. It is a schematic diagram explaining the estimation of the specific point coordinate in Embodiment 2 of this invention. It is a schematic diagram of the subject area calculation system in Embodiment 3 of the present invention. It is a block diagram of the to-be-photographed area calculation apparatus in Embodiment 4 of this invention. 14 is a flowchart illustrating subject area calculation processing according to Embodiment 4 of the present invention.

DESCRIPTION OF SYMBOLS 1 Subject area calculation system 2 Subject 3 Marking 4, 5, 6 Pointer 7 Imaging device 8, 10 Subject area calculation device 11 Storage part 12 Reading means 13 Imaging device coordinate estimation means 14 Specific point coordinate estimation means 15 Area calculation means 16 Display means

Claims (14)

A first image including a subject including a target region that is an area calculation target captured by the first imaging device, a sign that is in the range of the target region and is set in front, and the second imaging device Reading means for reading out the subject imaged from an angle different from that of the one imaging device and a second image that is in the range of the target area and includes a sign placed on the front surface ;
Imaging device coordinate estimation means for estimating the value of the three-dimensional coordinate of the first position of the first imaging device and the value of the three-dimensional coordinate of the second position of the second imaging device;
Specific point coordinate estimating means for estimating values of three-dimensional coordinates of a plurality of specific points included in the subject based on the first position and the second position;
Area calculating means for calculating the area of the target region based on the value of the three-dimensional coordinates of each of the plurality of specific points;
Wherein the label has a Rutotomoni known area having a two-dimensional shape with three or more pointers having a substantially spherical shape,
The imaging device coordinate estimation unit calculates a first direction with respect to the marker of the first imaging device by calculating an inclination of a plane formed by each of the three pointers of the marker included in the first image. Estimating a second direction relative to the sign of the second imaging device by calculating an inclination of a plane formed by each of the three pointers of the sign included in the second image, and Based on a comparison between the area of the sign included in one image and the known area, a first distance between the first imaging device and the sign is calculated, and the area of the sign included in the second image Based on the comparison with the known area, a second distance between the second imaging device and the sign is calculated, and the first distance of the first imaging device is calculated based on the first direction and the first distance. Estimate the 3D coordinates of the position , Based on the second direction and the second distance, and estimate the value of the three-dimensional coordinates of the second position of the second imaging device,
The specific point coordinate estimation means includes a line indicating a direction from the first position to one of the specific points, and a line indicating a direction from the second position to one of the specific points. Calculating an intersection, estimating the intersection as one three-dimensional coordinate of the plurality of specific points, estimating a three-dimensional coordinate of another point of the plurality of specific points;
The subject area calculation device , wherein the area calculation means calculates an area of the target region surrounded by three-dimensional coordinates of each of the plurality of specific points .   The subject area calculation apparatus according to claim 1, wherein each of the three or more pointers has a different chromaticity.   The subject area calculation apparatus according to claim 1, wherein the marker has a triangle shape having three vertices having different chromaticities and having a substantially spherical shape as apexes. The reading means is the range of the subject and the target area captured by the third imaging device from angles different from those of the first imaging device and the second imaging device, and the sign placed on the front surface. A third image including
The imaging device coordinate estimation means estimates a value of a three-dimensional coordinate of a third position of the third imaging device;
The specific point coordinate estimation means includes a line indicating a direction from the first position to one of the plurality of specific points, and a line indicating a direction from the second position to the one point of the specific point. Calculating an intersection of a line indicating a direction from the third position to the one point of the plurality of specific points, and estimating the intersection as a three-dimensional coordinate of the one point of the plurality of specific points to subject area calculation device according to any one of claims 1 to 3 A line indicating a direction from the first position to one of the plurality of specific points, a line indicating a direction from the second position to one of the specific points, and the plurality of specific points from the third position If all of the lines indicating the direction of a point to one point do not coincide and intersect,
The specific point coordinate estimation means indicates a line indicating a direction from the first position to one of the plurality of specific points, and indicates a direction from the second position to the one of the plurality of specific points. On the line connecting the positions where the line is close, a position determined by a predetermined weight is calculated as the first proximity point,
Next, on the line connecting positions where the first proximity point and a line indicating the direction from the third position to the one point of the plurality of specific points are close to each other, a position determined by a predetermined weight is Calculated as 2 proximity points,
The subject area calculation apparatus according to claim 4, wherein the second proximity point is estimated as a three-dimensional coordinate of the one point of the plurality of specific points. The predetermined weight in the calculation of the first proximity point includes a line indicating a direction from the first position to one of the plurality of specific points, and the one of the plurality of specific points from the second position. On the line connecting the positions where the line indicating the direction to the point is close, it is 1: 1,
The predetermined weighting in the calculation of the second proximity point connects the first proximity point and a position where a line indicating a direction from the third position to the one point of the plurality of specific points is close to each other. The subject area calculation apparatus according to claim 5, wherein 2: 1 is set on the line . The specific point coordinate estimating means includes a line indicating a direction from the first position to one of the plurality of specific points (hereinafter referred to as “first straight line”), and the plurality of specific points from the second position. When the line indicating the direction to the one point (hereinafter referred to as “second straight line”) does not intersect, the first straight line and the second straight line at a position where the first straight line and the second straight line are close to each other Calculate the distance between two straight lines,
Calculating the separation amount as a mismatch amount;
The subject area calculation apparatus according to claim 1, wherein the area calculation unit calculates an error rate of the area of the target region based on the mismatch amount. The area calculation means calculates the area of the target region based on the error rate based on one of a percentage of the error to the area to be calculated, a level value indicating the error in stages, and an area range including the error to be calculated. The subject area calculation device according to claim 7. The subject area calculation apparatus according to claim 1, wherein each of the plurality of specific points is included in a vertex of the target region. Further comprising an object area calculating apparatus according to any one of claims 1 to 9 display means for displaying the area of the target region said area calculation means has calculated. The subject area calculation apparatus according to claim 10 , wherein the display unit can simultaneously display the subject, a target region in the subject, and an area of the target region. A subject area calculation apparatus according to any one of claims 1 to 11 ,
An imaging device for imaging the sign and the subject;
A subject area calculation system comprising: transmission means for transmitting an image captured by the imaging device to the subject area calculation device. A first image including a subject including a target region that is an area calculation target captured by the first imaging device, a sign that is in the range of the target region and is set in front, and the second imaging device A reading step of reading out the subject imaged from an angle different from that of the one imaging device and a second image that is in the range of the target area and includes a sign placed on the front surface ;
An imaging device coordinate estimation step for estimating a value of a three-dimensional coordinate of the first position of the first imaging device and a value of a three-dimensional coordinate of the second position of the second imaging device;
A specific point coordinate estimation step for estimating a value of a three-dimensional coordinate of each of a plurality of specific points included in the subject based on the first position and the second position;
An area calculating step for calculating the area of the target region based on the value of the three-dimensional coordinates of each of the plurality of specific points,
Wherein the label has a Rutotomoni known area having a two-dimensional shape with three or more pointers having a substantially spherical shape,
The imaging device coordinate estimation step calculates a first direction of the first imaging device with respect to the marker by calculating an inclination of a plane formed by each of the three pointers of the marker included in the first image. Estimating a second direction relative to the sign of the second imaging device by calculating an inclination of a plane formed by each of the three pointers of the sign included in the second image, and Based on a comparison between the area of the sign included in one image and the known area, a first distance between the first imaging device and the sign is calculated, and the area of the sign included in the second image Based on the comparison with the known area, a second distance between the second imaging device and the sign is calculated, and the first distance of the first imaging device is calculated based on the first direction and the first distance. The value of the 3D coordinates of the position Constant, and based on the second direction and the second distance, and estimate the value of the three-dimensional coordinates of the second position of the second imaging device,
The specific point coordinate estimation step includes: a line indicating a direction from the first position to one of the plurality of specific points; and a line indicating a direction from the second position to one of the specific points. Calculating an intersection, estimating the intersection as one three-dimensional coordinate of the plurality of specific points, estimating a three-dimensional coordinate of another point of the plurality of specific points;
The area calculation step is a subject area calculation method of calculating an area of the target region surrounded by three-dimensional coordinates of each of the plurality of specific points . A program operable on a computer,
A first image including a subject including a target region that is an area calculation target captured by the first imaging device, a sign that is in the range of the target region and is set in front, and the second imaging device A reading step of reading out the subject imaged from an angle different from that of the one imaging device and a second image that is in the range of the target area and includes a sign placed on the front surface ;
An imaging device coordinate estimation step for estimating a value of a three-dimensional coordinate of the first position of the first imaging device and a value of a three-dimensional coordinate of the second position of the second imaging device;
A specific point coordinate estimation step for estimating a value of a three-dimensional coordinate of each of a plurality of specific points included in the subject based on the first position and the second position;
An area calculating step of calculating an area of the target region based on a value of each three-dimensional coordinate of the plurality of specific points;
Wherein the label has a Rutotomoni known area having a two-dimensional shape with three or more pointers having a substantially spherical shape,
The imaging device coordinate estimation step calculates a first direction of the first imaging device with respect to the marker by calculating an inclination of a plane formed by each of the three pointers of the marker included in the first image. Estimating a second direction relative to the sign of the second imaging device by calculating an inclination of a plane formed by each of the three pointers of the sign included in the second image, and Based on a comparison between the area of the sign included in one image and the known area, a first distance between the first imaging device and the sign is calculated, and the area of the sign included in the second image Based on the comparison with the known area, a second distance between the second imaging device and the sign is calculated, and the first distance of the first imaging device is calculated based on the first direction and the first distance. The value of the 3D coordinates of the position Constant and, based on the second direction and the second distance, and estimate the value of the three-dimensional coordinates of the second position of the second imaging device,
The specific point coordinate estimation step includes: a line indicating a direction from the first position to one of the plurality of specific points; and a line indicating a direction from the second position to one of the specific points. Calculating an intersection, estimating the intersection as one three-dimensional coordinate of the plurality of specific points, estimating a three-dimensional coordinate of another point of the plurality of specific points;
The area calculating step calculates the area of the target region surrounded by the three-dimensional coordinates of each of the plurality of specific points .