JP7202337B2 - Information acquisition device and program - Google Patents

Description

The present invention relates to an information acquisition device that acquires a photographed construction image or the like.

Images taken by digital cameras are overwhelmingly superior in immediacy compared to silver-salt photographs, and are therefore widely used as evidence photographs in various business fields. For example, at construction sites and civil engineering sites, systems using image data captured by digital cameras as evidence photographs are spreading. Furthermore, at a construction site, shape measurement of members such as pillars and beams is also performed using captured image data.
Various measurement techniques using captured images have also been proposed. For example, a three-dimensional camera has been proposed that identifies the color distribution of an image with a color sensor and recognizes the height of an object to be measured (Patent Document 1).

JP-A-6-3122

At a construction site, an image of a site for which construction or work has been completed is taken in units of construction according to the progress of the construction, and the photographed images are managed. At a construction site, there are many cases where a portion that has already been constructed is mixed with a portion that is currently under construction, or construction of different types of portions is progressing at the same time. Therefore, it is not uncommon for the image to include parts other than the object to be photographed (for example, a member that has already been photographed).
In such a situation, if the shape of the member is measured based on the image data, there will be measurement errors such as measuring the part that is not the target of measurement, or measuring the total of the parts including the target and other parts. may occur. In addition, when different types of parts are mixed in a photographed image, even if it can be distinguished at the time of photographing, it may not be possible to recognize which part is the part to be photographed after photographing. Therefore, it is necessary to be able to identify the part to be measured or photographed in the photographed image.
An object of the present invention is to provide an information acquisition device that identifies a target part from the parts of a photographed image in view of the above problems.

In order to achieve the above object, in an information acquisition device for acquiring a photographed image of a member, a target portion determination unit that determines a portion corresponding to a specific member from among a plurality of members appearing in the photographed image as a target portion; a measurement unit that measures the target part from the captured image and measures the shape of the part; and a shooting distance calculation part that calculates the shooting distance corresponding to the part in the captured image. , the target part determination unit determines the shape of the part as a feature, determines to which type of the member the part belongs based on the feature, and determines whether the part is the same as the specific member from among the parts. determining the part belonging to the member including the specific member and having the same type as the specific member, and determining the part belonging to the member including the specific member and having the same type as the specific member according to the photographing distance; After grouping, the target part is determined from the parts based on the group.
Further, in an information acquisition device for acquiring a photographed image of a member, a target portion determination unit for determining a portion corresponding to a specific member among a plurality of members appearing in the photographed image as a target portion; a measurement unit that measures the shape of the target site by measuring the target site, and a shooting distance calculation unit that calculates a shooting distance corresponding to the site in the captured image; determines the shape of the part as a feature, determines which type of the member the part belongs to based on the feature, and determines from among the parts that the part is of the same type as the specific member and the specific member determining the part belonging to the member including the member of and further grouping the part belonging to the member that is the same type as the specific member and that includes the specific member according to the photographing distance, and photographing The part belonging to the group existing on the side close to the direction in which the camera for is present is determined as the target part.
Furthermore, in an information acquisition device for acquiring a photographed image of a member, a target portion determination unit that determines a portion corresponding to a specific member from among a plurality of members appearing in the photographed image as a target portion; and a measurement unit that measures the shape of a specific part by performing measurement on the part, the target part determination part determines the shape of the part as a feature, and based on the feature, the part is a type of member The part belonging to the member of the specific type is determined from among the parts, and further, the part that belongs to the member of the specific type is present on the side close to the direction in which the photographing camera exists. Among a plurality of planes configured and set by other parts, the plane most directly facing the camera is specified, and the part belonging to the specified plane is determined as the target part.

According to the present invention, it is possible to provide an information acquisition device that identifies a target part from the parts of a photographed image.

It is a figure which shows the example of construction imaging|photography. It is a figure which shows the example of construction imaging|photography. It is a block diagram of the whole information acquisition system. 3 is a detailed block diagram of a target part determination unit; FIG. It is a hardware block diagram of an information acquisition device. 1 is a flowchart 1 for explaining target part determination processing; FIG. 2 is a flowchart 2 for explaining target part determination processing; FIG. FIG. 3 is a flowchart 3 for explaining target part determination processing; FIG. It is an example of a setting screen for setting the processing mode of the information acquisition device. This is an example of highlighting. It is an example of a highlighted guide screen. It is an example of a highlighted guide screen. It is a monocular system and is a figure which shows the principle which measures a site|part shape from a picked-up image. It is a figure which shows the principle which measures a site|part shape from a picked-up image by a binocular system. It is an example of configuration information by a design drawing. It is a guide screen by the 2nd composition information. This is the result confirmation screen. This is the result confirmation screen. This is a subroutine for target part determination 1. FIG. FIG. 4 is an explanatory diagram of target part determination 1; FIG. 4 is an explanatory diagram of target part determination 1; This is a subroutine for target part determination 2. FIG. This is a specific example of target part determination 2. FIG. This is a subroutine for target part determination 3. FIG. This is a specific example of target part determination 3. FIG. This is a specific example of target part determination 3. FIG. This is a subroutine for target part determination 4. FIG. This is a specific example of target part determination 4. FIG. 9 is a flowchart of processing for displaying a shooting range guide; In this example, a shooting range guide is displayed in the first configuration information. In this example, a shooting range guide is displayed in the second configuration information. In this example, the shooting range guide display is displayed in the third configuration information. In this example, a panorama image is displayed in the photographing range guide display. It is a figure which shows the principle which measures the space|interval of a pillar in a straight facing state. It is a figure which shows the principle which measures the space|interval of a column in an oblique direction. It is a figure which shows the principle which measures the space|interval of the base of a floor.

Embodiments of the present invention will be described below with reference to the drawings. The information acquisition system 1 of the present invention will be described using an example of use in construction photography. Construction photography is photography for acquiring evidence photographs and the like of construction details at a construction site. FIG. 1A and FIG. 1B are diagrams showing how construction is photographed.

FIG. 1A shows a photographer carrying a photographing device (hereinafter referred to as an information acquisition device 10) preparing for photographing in a room in which a plurality of pillars have been constructed. Construction usually proceeds from the back side. The front side is the pillar PA to be photographed this time, and the back side is the pillar PB which has been photographed. In the following description, construction sites to be photographed, such as columns and beams, are also referred to as parts or members. In the photographed image, the photographed member is also called a photographed part or simply a part. Members corresponding to parts are also abbreviated as parts unless it is unclear.

FIG. 1B shows the photographer holding the information acquisition device 10 and facing the pillar PA, which is the object to be photographed, and photographing. At the construction site, photographs are taken in order from the point where the construction is completed. Therefore, as the construction progresses, it becomes difficult to photograph only the pillar PA, which is the object of photography this time. There are many. Therefore, the photographed image often includes both the pillar PA to be photographed and the already photographed pillar PB.

FIG. 2 is a configuration diagram of the entire information acquisition system 1. As shown in FIG. The information acquisition system 1 is broadly composed of an information acquisition device 10 and an external device 40 . The information acquisition device 10 is a device that acquires a captured image. The information acquisition device 10 may be a digital camera, an information processing device with a camera, or a system in which a camera and an information processing terminal (for example, a tablet type information processing terminal) are combined in a wired or wireless manner. The external device 40 is, for example, a server that is connected to the information acquisition device 10 via a communication line and stores the results of imaging by the information acquisition device 10 .

The information acquisition device 10 includes a camera section 100, a control section 120, a storage section 150, a display section 160, a touch panel 165, a communication section 170, and the like. Note that the camera unit 100 is also simply called a camera.

The camera unit 100 has an imaging unit 102 , an image processing unit 104 , an operation unit 106 , an operation determination unit 108 and an orientation sensor 110 . The photographing unit 102 has a lens unit 102a and an imaging device 102b (not shown), and converts a subject image into an image signal. An image processing unit 104 converts an image signal output from the imaging unit 102 into digital data and performs various image processing.

The operation unit 106 is an instruction member for adjusting the focal position and focal length of the lens unit 102a of the imaging unit 102. FIG. An operation determination unit 108 determines an operation on the operation unit 106 . An orientation sensor 110 is a sensor that detects the orientation and direction of the camera unit 100 . The attitude sensor 110 has, for example, an inclination sensor (acceleration sensor) and an electronic compass.

The control unit 120 centrally controls the entire information acquisition device 10 . The control unit 120 includes an information processing unit 122, a camera control unit 124, a configuration information acquisition unit 126, an imaged part determination unit 128, a measurement unit 130, an imaging distance calculation unit 132, a reference information creation unit 134, a target part determination unit 136, Emphasis display setting unit 138, shooting range setting unit 140, guide screen creation unit 142, and display control unit 1
44, etc.

The information processing section 122 manages various determination processes and information input from the external device 40 and the camera section 100 . The camera control unit 124 controls the camera unit 100 based on instructions input from the touch panel 165 to switch the operation mode of the camera unit 100 and execute shooting and the like.

The configuration information acquisition unit 126 acquires configuration information from the design drawing information 422 of the external device 40 or the image captured by the camera unit 100 . The configuration information is information indicating the configuration of a plurality of members to be photographed. The members to be photographed are construction sites such as pillars, walls, and beams. In the following, two types of configuration information, first configuration information and second configuration information, will be mainly described. The first configuration information is information indicating configurations of a plurality of members based on the captured image. The second configuration information is information indicating the configuration of a plurality of members based on the design drawing.

The configuration information acquisition unit 126 acquires, as the first configuration information, an image captured by the camera unit 100 and in which the entirety of the plurality of members is captured. Also, the configuration information acquisition unit 126 acquires the second configuration information from the design drawing of the external device 40 . Details of the configuration information will be described later.

The photographed part determination unit 128 determines a photographed member by sequentially photographing a plurality of members. At a construction site, there are many members with similar shapes, so by identifying members that have already been photographed during photographing, it is possible to reduce the waste of repeated photographing and prevent photographing omissions.

The measuring unit 130 measures the shape of the part from the captured image and outputs the measured value. When the member to be imaged is a pillar, the shape of the part is, for example, the width (diameter) and height of the pillar. When the member is a beam, it is the thickness of the beam.

As shown in FIG. 1B, if a portion of a target member (hereinafter also referred to as a target portion) and other portions coexist on the screen, it is impossible to set a measurement target. Therefore, the measurement unit 130 performs measurement based on the target part determined by the target part determination unit 136 . In addition, when a plurality of target parts are determined by the target part determining unit 136, the measuring unit 130 measures the determined plural parts.

The shooting distance calculation unit 132 calculates the distance from the camera unit 100 to a member (for example, a pillar) that is shot. The photographing distance calculation unit 132 may calculate the photographing distance at the position of the focusing lens of the photographing unit 102, for example. Then, by setting a focus detection point according to the position of each part in the photographed image and individually calculating the photographing distance corresponding to each part, the photographing distance to each part can be obtained. Further, when the imaging unit 102 is of a binocular type, the imaging distance calculation unit 132 may calculate the imaging distance by phase difference ranging using the left and right imaging elements 102b. Here, the phase-difference ranging means detecting the shift of the image position corresponding to the point of interest on the subject based on the degree of correlation between the left and right subject images, and calculating the distance of the point of interest based on the principle of triangulation. This is a known method for calculating, and the distance of each part in the photographed image can be obtained by one photographing.

The reference information creation unit 134 creates reference information that serves as a reference for construction photography, referring to pre-photographed images and general standard reference photography procedures (recorded in an external DB (Data Base) or storage unit 150). is created. For example, the reference information creation unit 134 provides an imaging procedure commonly used on site as a "standard imaging guide". In addition, the reference information creating unit 134 may provide a photographing procedure that has already been used in photographing another floor in the same building as a “similar photographing guide”.

The target part determination unit 136 determines a part corresponding to a specific member in the captured image as a target part. That is, the target part determination unit 136 determines the target part from the screen in which the target part and other parts are mixed. In this embodiment, the target part determination unit 136 determines a part (member) with the latest construction period as the target part.

Specifically, the target site determination unit 136 determines the surface layer site as the target site among the sites in the captured image. The surface layer part refers to a part corresponding to a member existing on the side nearer to the direction in which the camera unit 100 exists, among the parts of the photographed image. This is because it can be considered that the surface layer part corresponds to the member with the latest construction period. As mentioned above, at a construction site, construction usually progresses from the back side, so correspondingly, shooting is also carried out from the completed members on the back side. Therefore, in construction photography, the member that is closest to the photographer is usually the member with the latest construction period.

Also, the member with the latest construction period may be specified directly from the construction period information of the member. Note that the location of the latest construction period is not limited to one member, and may be a plurality of members. For example, this is the case where n pillars are constructed together. In this case, a group consisting of a plurality of parts corresponds to the target parts.

The target part determination unit 136 determines the target part by singly or in combination with the following conditions. A target part is a part to be identified. As will be described later, the target site is a measurement target or a highlight display target.

The target part determination unit 136 determines the target part from the imaging distance corresponding to the part of the captured image. In addition, the target site determination unit 136 may acquire the shape of the member to be imaged and determine the target site using pattern matching. Furthermore, the target part determination unit 136 may determine the target part by taking into account the group of photographing distances and the construction period of members.

In addition, the target part determination unit 136 may determine a plurality of target parts in one screen based on image features, imaging distance groups, and construction periods. If a plurality of target parts can be collectively determined and measured within one screen, the number of captured images can be reduced.

FIG. 3B is a detailed block diagram of the target part determination unit 136. As shown in FIG. The target part determination unit 136 has an image feature determination unit 136a, a parts determination unit 136b, a distance determination unit 136c, and a construction period determination unit 136d.

The image feature determination unit 136a determines features (shape, size, and color) of each imaged part. Specifically, the image feature determination unit 136a determines the shape and color of prisms, cylinders, beams, etc. as features. Based on the shape and size of each part determined by the image feature determining part 136a, the parts determination unit 136b determines what type (prism or cylinder) each part is.

The target part determination unit 136 determines the target part by comparing the specifications of each member acquired from the design drawing information 422 and the member DB 424 with the type of the part determined by the parts determination unit 136b. For example, if the member to be identified this time is a prism and the part with the screen is determined to be a cylinder, the part with the screen is determined not to be the target part.

The distance determination unit 136c classifies each part according to the shooting distance based on the shooting distance to each part of the shot image calculated by the shooting distance calculation unit 132 . The distance determination unit 136c classifies each part that has been photographed, for example, at intervals of 1 m photographing distance.

The construction period determination unit 136d acquires the construction period of each member from the design drawing information 422 and the member DB 424, and determines the construction period of each member. The target part determination unit 136 determines the member with the latest construction period based on the determined construction period of each member, and determines the part corresponding to the member with the latest construction period from the image features and the like.

The highlighting setting unit 138 sets highlighting for the part determined to be the target part by the target part determining unit 136 . A photographed image in which the target region is highlighted is displayed on a live view image, a rec view image, or a guide screen, which will be described later. When there are a plurality of parts determined to be target parts by the target part determination unit 136, the highlighting setting unit 138 sets highlighting to the plurality of determined parts.

The imaging range setting unit 140 sets the imaging range according to the required accuracy for the measured value of the shape of the member. If the photographing distance is lengthened, the photographing range is widened, and a plurality of members can be photographed collectively, but the accuracy of measurement deteriorates accordingly due to the resolution of the image sensor 102b. In other words, if it is desired to measure the shape of a member with high accuracy, it is necessary to take a close-up image.

The photographing range setting unit 140 calculates the photographing distance for satisfying the required measurement accuracy based on the characteristics of the camera unit 100 . The characteristics of the camera unit 100 include the MTF and distortion of the lens unit 102a and the resolution of the imaging device 102b. Then, the photographing range setting unit 140 sets the corresponding photographing range based on the calculated photographing distance and the characteristics of the camera unit 100 (angle of view of the lens unit 102a). The angle of view of the lens unit 102a is determined by the F-number, focal length, focus position, and the like.

The guide screen creation unit 142 creates a guide screen including the acquired configuration information and adding the situation information to the part indicated by the configuration information. The status information is the imaging situation based on the determination by the imaged part determination unit 128, and the imaging status is, for example, the progress or progress of imaging. The guide screen is displayed at any time during the progress of shooting.

Then, the guide screen creation unit 142 causes the guide screen to include, in addition to the configuration information to which the situation information is added, the photographed image of the part (live view image or rec view image) and the photographing condition information. The imaging condition information includes the measured part shape (dimensions), the imaging distance, the required accuracy of the part shape, the recommended imaging distance that ensures the required accuracy, and the like. Note that the guide screen is also called a shooting guide screen.

Further, the guide screen creation unit 142 creates a guide screen by adding the shooting range set by the shooting range setting unit 140 to the configuration information. The set shooting range is indicated as a shooting range guide on the screen showing the configuration information. Details of the shooting range guide will be described in the second embodiment.

The guide screen is based on the performance and function information of the camera unit 100, according to the environment at the time of photography and measurement, and the object of photography and measurement, and provides information that enables efficient, accurate, and appropriate photography. It is intended to inform the person. In addition to these information, measurement accuracy, target object, environment, etc., the guide screen creating unit 142 may display, notify, etc. the reference information created by the reference information creating unit 134 . With these, a guide screen is displayed that allows correct measurement of the part and efficient measurement of many measurements at once. The guide screen creation unit 142 creates a guide screen based on the performance of the camera unit 100, the accuracy of the measurement unit 130, and the actual arrangement and size of each part. The guide screen may be created in consideration of the above.

The display control unit 144 causes the display unit 160 to display a setting screen for setting a photographed image, a mode, or the like, or a guide screen.

Storage unit 150 is a non-volatile storage unit. The storage unit 150 is a non-volatile storage unit, and stores a captured image 152, configuration information 154, camera unit characteristic information 156, and the like. A captured image 152 is image data captured and recorded by the camera unit 100 . The captured image 152 may include information on the target part determined by the target part determination unit 136 . Also, an image highlighted by the highlighting setting unit 138 may be used as the captured image 152 . The configuration information 154 is information acquired by the configuration information acquisition unit 126 . The stored configuration information 154 may be either the first configuration information or the second configuration information, or both the first configuration information and the second configuration information.

The camera unit characteristic information 156 is characteristic information regarding the camera unit 100, particularly the lens unit 102a and the image sensor 102b. The characteristic information of the lens unit 102a includes brightness, resolution, distortion, etc., in addition to specifications. The characteristic information of the image sensor 102b includes the pixel size, the number of pixels, the pixel pitch, and the like.

A display unit 160 is, for example, an LCD, and displays a photographed image, a guide screen, and the like. A touch panel 165 is an operation unit integrally formed with the display unit 160, and receives various instructions from the photographer. The communication unit 170 is an interface that communicates various information with the external device 40 . The communication unit 170 communicates with the external device 40 via a network (not shown). The communication unit 170 can determine the connection destination, its request, etc. by reception, connection setting, etc., thereby allowing the control unit 120 to determine the required specifications of various measurements. Through this communication, the external device 40 may transmit the required specifications.

The external device 40 has a control section 400 , a communication section 410 and a storage section 420 . The external device 40 is a server that manages construction photography. The control unit 400 comprehensively controls the external device 40 . The communication unit 410 is an interface that performs information communication with an external device such as the information acquisition device 10 . The storage unit 420 includes design drawing information 422 and a member DB (Data Base) 424 . The part DB will be described later with reference to FIG. 7B. The storage unit 420 stores imaging results (images of imaging regions, reports described later, etc.) obtained by the information acquisition device 10 .

FIG. 3 is a hardware block diagram of the information acquisition device 10. As shown in FIG. The information acquisition device 10 has a CPU (Central Processing Unit) 120a, a DRAM (Dynamic Random Access Memory) 120b, a ROM (Read Only Memory) 120c, a flash memory 150a, a display section 160, a touch panel 165, and a communication section 170.

The CPU 120a reads and executes a control program stored in the ROM 120c, and controls the information acquisition device 10 by software processing. The DRAM 120b provides a working area for temporarily storing control programs and various data. The ROM 120c nonvolatilely stores the control program. The control unit 120 is composed of a CPU 120a, a DRAM 120b and a ROM 120c.

The flash memory 150a stores various data tables and captured images. The storage unit 150 is composed of a flash memory 150a. Since the display unit 160, the touch panel 165, and the communication unit 170 have been described above, description thereof will be omitted.

Image acquisition processing by the information acquisition device 10 will be described below. Image acquisition processing includes target region determination processing. 4A to 4C are flow charts 1 to 3 for explaining target part determination processing. Image acquisition processing is mainly executed by the control unit 120 of the information acquisition device 10 .

First, the control unit 120 determines whether the construction mode has been selected in the information acquisition device 10 (step S100). FIG. 5 is a screen displayed on the display unit 160, which is a setting screen for setting the processing mode of the information acquisition device 10. As shown in FIG.

A setting screen 301 in FIG. 5 is a mode setting screen, and the second construction mode from the top is selected. K1 is a return button. A setting screen 302 in FIG. 5 is a screen on which the construction mode is selected and specific processes included in the construction mode are displayed. The construction mode includes sub-modes of "construction photography", "image reproduction", "design drawing acquisition", "result storage" and "result confirmation".

“Construction photographing” is a process of photographing a construction site. "Image reproduction" is processing for reproducing a photographed image.

“Design drawing acquisition” is a process of obtaining the design drawing information 422 from the storage unit 420 of the external device 40 as the second configuration information in the configuration information 154 . “Result confirmation” is a process in which the photographer or administrator confirms the contents of the image based on the image of each part imaged by the information acquisition device 10 . The photographer or administrator confirms on a confirmation screen, which will be described later, that there are no omissions or defects in the photographed parts. "Result storage" is a process of transmitting the results of imaging by the information acquisition device 10 (photographed images, shape information of parts, information of determined target parts) to the external device 40 and storing them in the external device 40. be.

When the control unit 120 determines that the construction mode has not been selected in the information acquisition device 10 (NO in step S100), the other selected mode is executed (step S102).

When the control unit 120 determines that the construction mode has been selected in the information acquisition device 10 (YES in step S100), it determines whether or not "construction photography" has been selected in the construction mode setting screen 302 of FIG. 5 (step S104). ).

When the control unit 120 determines that "construction photography" has been selected (YES in step S104), the camera unit 100 starts acquiring an image and displaying the acquired image as a live view image (step S106).

The control unit 120 determines whether or not to use the configuration information 154 based on instructions from the photographer (step S108). When determining that the configuration information 154 is not used (NO in step S108), the control unit 120 displays the live view image and the measurement image (step S110).

The target part determination unit 136 determines the target part based on the live view image (step S112). Details of determination of the target portion will be described later with reference to FIGS. 12A to 15B. The measuring unit 130 measures the determined shape of the target site (step S114). The highlighting setting unit 138 sets to highlight the target part. The control unit 120 displays the highlighted captured image and the measurement result (step S116).

FIG. 6 is a display example of a live view image and a measurement image. The left side of the screen is the normal live view image, and the right side of the screen is the measurement screen. The measurement screen displays a live view image highlighted and to which the measured shape (dimension) is added. The four pillars PA on the near side are highlighted with respect to the five pillars PB on the back side as viewed from the camera section 100 . Also, the measurement results of the four front pillars PA are all displayed as 30 cm.

The control unit 120 determines whether or not a photographing instruction has been issued (step S118). When control unit 120 determines that an instruction to shoot has been issued (YES in step S118), shooting is performed (step S118). The measurement unit 130 measures the width of the column of the target site based on the captured image (step S120). The control unit 120 records the captured image and the measurement result in the storage unit 150 (step S122). After step S122, or if the control unit 120 determines that no photographing instruction has been issued (NO in step S118), the process proceeds to step S130.

When control unit 120 determines to use configuration information 154 (YES in step S108), control unit 120 determines whether there is configuration information 154 (step S130). In addition, when distinguishing the types of the configuration information 154 below, they are described as first configuration information 154a and second configuration information 154b.

The control unit 120 determines whether the storage unit 150 stores the first configuration information 154a or the second configuration information 154b. When determining that there is no configuration information (NO in step S130), the control unit 120 determines whether or not the type of configuration information to be acquired is the entire image (first configuration information 154a) according to the designation of the photographer (step S132). . When the control unit 120 determines that the whole image is designated as the type of configuration information to be acquired (YES in step S132), it instructs the photographer to shoot the whole image. Whole image photography is performed by the photographer. The control unit 120 records the entire image captured by the photographer in the storage unit 150 as the first configuration information 154a (step S134).

On the other hand, when control unit 120 determines that the type of configuration information 154 is not a whole image, that is, that the type of specified configuration information 154 is a design drawing (NO in step S132), control unit 120 sends an external image via communication unit 170. The design drawing information 422 is acquired from the storage unit 420 of the device 40 and recorded in the storage unit 150 as the second configuration information 154b (step S136). After step S134 or step S136, control unit 120 proceeds to step S150.

When the control unit 120 determines that there is the configuration information 154 (YES in step S130), the user selects the type of the configuration information 154 to be used (step S150). The guide screen creating unit 142 creates a guide screen from an image obtained by synthesizing the selected configuration information 154 and the live view image. The control unit 120 displays the guide screen including the configuration information 154 and the live view image on the display unit 160 (step S152). The configuration information image and the live view image are displayed separately.

The target part determination unit 136 determines the target part based on the configuration information or the live view image (step S154). Determination of the target part will be described with reference to FIGS. 12A to 15B as described above. The measuring unit 130 measures the determined shape of the target site (step S156). The highlighting setting unit 138 sets to highlight the target part. The control unit 120 displays the highlighted captured image and the measurement result (step S158).

7A and 7B are examples of highlighted images on the guide screen. FIG. 7A shows a screen in which the photographer is considering composition. In FIG. 7A, the left side of the screen is the live view image, and the right side of the screen is the configuration information. As the configuration information, a pre-captured overall image of the room is displayed. In the configuration information, the pillar PA existing in front of the camera unit 100 is determined as the imaging target part, and the pillar PA is highlighted with respect to the pillar PB on the far side. Note that although FIG. 7A shows an example in which the configuration information is highlighted in the image, it may be highlighted in the live view image.

Note that if only one of the first configuration information 154a and the second configuration information 154b is stored in the storage unit 150, step S150 is omitted, and the configuration information stored in the storage unit 150 is stored in step S152. 154 is displayed.

Then, the measurement unit 130 measures the target site based on the live view image (step S156). The guide screen creation unit 142 may add the measurement result (part shape) to the guide screen. The control unit 120 displays the enhanced configuration information (whole image) to which the measurement result is added (step S158, see FIG. 7A).

8A and 8B are diagrams schematically showing the principle of measuring the part shape (here, the width of the pillar P) from the photographed image. In both cases, the camera unit 100 in the photographing state is viewed from above. 8A shows an example of a monocular imaging unit 102, and FIG. 8B shows an example of a binocular imaging unit 102. FIG.

In FIG. 8A, the width W of the part (pillar P), the width X0 of the imaging element 102b, the angle of view φ of the lens unit 102a, the focal length F of the lens unit 102a, and the photographing distance D are assumed. The width X0, the angle of view φ, and the focal length F are included in the camera section characteristic information 156 . The shooting distance D is calculated by the shooting distance calculator 132 based on the focus position detected by the contrast AF, for example. As described above, when the photographing unit 102 is a monocular type, the width W of the pillar P is calculated by the following formula.
W=D×X/F Expression (1)
φ=2×arctan(X0/(2×F)) Expression (2)
Note that the portion F (focal length) in equations (1) and (2) is precisely the focal length F plus the image plane movement due to focusing (in the case of a fixed focus system, the fixed image surface distance), but was set to F for simplification of explanation. Hereinafter, similar simplifications will be used unless otherwise specified.

Here, X, which is the width of the part (pillar P) in the subject image on the imaging device, can be obtained by various known methods. It is also an example to obtain the number of pixels of the extracted main subject portion (pillar P portion) by applying the method.

In FIG. 8B, the lens unit 102aR and the image sensor 102bR are the right lens and image sensor as seen from the photographer. Also, the lens unit 102aL and the imaging device 102bL are the lens and imaging device on the left side as seen from the photographer. Further, the parallax between the left and right imaging elements 102b is B, and the relative image position is ΔX. Assume that the center of the pillar P is arranged on the optical axis of the lens portion 102aR. When the imaging unit 102 is a binocular type, the imaging distance D is calculated by the imaging distance calculation unit 132 using the parallax B and the relative image position ΔX.
D=F×B/ΔX Expression (3)
W is calculated by the formula (1) from D calculated by the formula (3).

Proceed to FIG. 4B. The control unit 120 determines whether or not there is an operation by the photographer (step S170). The operator's operation is, for example, a zoom operation of the operation unit 106 . When the control unit 120 determines that there is an operation by the photographer (YES in step S170), the camera control unit 124 performs control reflecting the operation result (step S172).

When the control unit 120 determines that there is no operation by the photographer (NO in step S170), it waits for a photographing instruction (step S174). When control unit 120 determines that a photographing instruction has been issued (YES in step S174), it executes photographing (step S176).

The measurement unit 130 measures the shape of the target site based on the rec view image (step S178). The measuring unit 130 measures the part determined as the target part in step S154. Control unit 120 displays a rec view image (step S180). FIG. 7B is a screen after the photographing instruction is given in FIG. 7A. The left side of the screen is the recording view image, and the right side is the configuration information image. In the recview image, the target site is highlighted (hatched) and the measured value of the target site is also displayed.

When control unit 120 determines that there is no photographing instruction (NO in step S174), it returns to step S104. Control unit 120 determines whether the OK button has been clicked (step S182). When control unit 120 determines that the OK button has not been clicked, it returns to step S104 (NO in step S182).

When the control unit 120 determines that the OK button has been clicked (YES in step S182), it records the imaging result in the storage unit 150 in association with the configuration information 154 (step S184). Also, the photographed image and the measurement result of the part shape are recorded in the storage unit 150 .

The control unit 120 displays a guide screen in which an electronic marker indicating the imaging situation is added to the configuration information (step S186). The electronic marker Q is an example of status information. In FIG. 7A, it is assumed that the pillar PA on the right end is clicked at the time of photographing. Then, on the guide screen of FIG. 7B, an electronic marker Q written as "completed" indicating completion of photographing is added to the rightmost pillar PA. The guide screen creation unit 142 adds the electronic marker Q based on the determination by the imaged part determination unit 128 . Control unit 120 returns to step S104.

Proceed to FIG. 4C. If control unit 120 determines that "construction photography" has not been selected (NO in step S104), it determines whether "image reproduction" has been selected (step S200).

When control unit 120 determines that "image reproduction" has been selected (YES in step S200), image reproduction is performed (step S202). The control unit 120 determines whether an instruction to change the reproduced image has been issued (step S204), and if it determines that an instruction to change the reproduced image has been issued (YES in step S204), changes the reproduced image in accordance with the instruction (step S206). ). When control unit 120 determines that no instruction to change the reproduced image has been issued (NO in step S204), control unit 120 returns to step S104.

Further, when determining that "image reproduction" is not selected (NO in step S200), control unit 120 determines whether "design drawing acquisition" is selected (step S210). When the controller 120 determines that "acquire blueprint" has been selected (YES in step S210), the controller 120 requests the blueprint information 422 from the external device 40 via the communication unit 170, and receives the blueprint transmitted from the external device 40. Information 422 is received (step S212). The process of step S212 is similar to the process of step S136 described above. The control unit 120 records the received engineering drawing information 422 in the storage unit 150 as the second configuration information 154b.

Next, highlighting by the second configuration information 154b based on the received engineering drawing information 422 will be briefly described.

FIG. 9 is an example of displaying the second configuration information 154b based on the design drawing. A setting screen 310 in FIG. 9 is an example of a setting screen when obtaining a design drawing. When “acquire design drawing” is selected on the construction mode setting screen 310 , the design drawing is obtained from the external device 40 . A setting screen 311 in FIG. 9 is a screen for specifically designating a portion to be displayed in the acquired design drawing. The photographer moves the cursor to specify the location of the configuration information to be displayed.

The setting screen 312 of FIG. 9 is a screen of the blueprint of the place (D room of the A company building 4F) specified on the setting screen 311 . Five pillars PA and six pillars PB are arranged in the room. The pillar PA is taken as an object to be photographed. When the adoption button on the setting screen 312 is clicked, the blueprint of the D room of the A company building 4F being displayed is adopted as the second configuration information 154b.

FIG. 10 is a guide screen based on the second configuration information 154b employed in the setting screen 312 of FIG. The left side of the screen is the live view image, and the right side of the screen is the second configuration information 154b. Note that the live view image has a composition that does not include the pillar on the right end. In the live view image, the pillar PA determined as the target part is highlighted. Moreover, the measured value of the measured pillar PA is also displayed.

Return to step S210 of FIG. 4C. When the control unit 120 determines that "acquire design drawing" is not selected (NO in step S210), it determines whether "confirm result" is selected (step S220). The "result confirmation" process will be described with reference to the setting screens of FIGS. 11A and 11B. FIG. 11A is the setting screen 320 in which "confirm result" is selected in the construction mode.

When control unit 120 determines that "confirm result" has been selected (YES in step S220), it displays a location designation (step S222). A setting screen 321 in FIG. 11A is an example of a screen for designating a location for confirming results. Here, it is assumed that the D room of the 4th floor of the A company building shown in FIG. 9 is designated as the location for confirming the result. The setting screen 322 in FIG. 11A is the imaged result of the D room of the designated A company building 4F.

On the setting screen 322, five pillars PA on the design drawing are attached with status information (OK mark) indicating their numbers and shooting status. The OK mark indicates that the pillar, which is the target part, has been photographed and that the measured value of the pillar (the width of the pillar) conforms to the standard.

The control unit 120 determines whether or not an instruction to create a report on the photographing results has been issued (step S224). The control unit 120 determines that an instruction to create a report has been issued when "Report OK" on the setting screen 322 of FIG. 11A is clicked.

When control unit 120 determines that report creation is not instructed (NO in step S224), it does not create a report. Then, for example, when the No. 5 pillar is clicked on the guide screen of the setting screen 322, the control unit 120 displays a detailed confirmation screen as shown in the setting screen 323 of FIG. 11B (step S228). The photographed image (evidence image) and the measured value of the No. 5 pillar stored in the storage unit 150 are displayed on the detail confirmation screen.

When control unit 120 determines that a report creation instruction has been issued (YES in step S224), control unit 120 displays the created report on a text basis (step S226). The setting screen 324 of FIG. 11B is an example of the text-based content of the created report.

If control unit 120 determines that "confirm result" has not been selected (NO in step S220), it determines whether "store result" has been selected (step S230). When the control unit 120 determines that "store results" has been selected (YES in step S230), the control unit 120 transmits the imaging results to the external device 40 (step S232). The photographed result includes the photographed image, the report created in step S226, and the like. The external device 40 stores the transmitted imaging results in the storage unit 420 .

When control unit 120 determines that "store result" is not selected (NO in step S230), it returns to step S104. Also after step S206, step S212, step S226, step S228, step S232, or the like, the process returns to step S104.

The target part determination shown in steps S112 and S154 will be described with reference to FIGS. 12A to 15B. As target part determination processing, representative methods will be described as target part determination 1 to target part determination 4. FIG. There are various types of parts to be photographed, such as columns, beams, reinforcing bars, and foundations, and the construction environment is not limited to ordinary buildings and houses, but also includes underground construction, tunnel construction, and the like. Therefore, in order to accurately determine the target part according to such environment and conditions, the target part determination unit 136 appropriately selects or combines the following four determination methods and executes determination processing.

FIG. 12A is a subroutine of target part determination 1. FIG. 12B and 12C are schematic diagrams for explaining the process of target part determination 1. FIG. Target part determination 1 is a process of determining a surface layer part (target part) based on the appearance of the photographed part and the photographing distance to the member corresponding to the part.

The target part determination unit 136 determines and classifies the image features such as the outline of each part and the imaging distance of each part for each part (P1 to Pn) (step S300). n is the number of sites present on the screen. In the screen of FIG. 12C, n=7. The image feature such as the contour of each part is determined by the image feature determination unit 136a. The imaging distance of each part is calculated by the imaging distance calculator 132 .

Let k be an integer and k=1 to n. The target part determination unit 136 is initialized with k=1 (step S302). The target part determination unit 136 classifies the image features such as the contour of each part and the photographing distance of each part based on the photographed image, and examines the overlapping relationship between the parts. Then, the target part determination unit 136 determines whether there is another part that overlaps the part Pk (step S306).

When the target part determining unit 136 determines that there is no other part overlapping the part Pk (NO in step S306), it determines that Pk is a superficial part (step S308). A superficial portion is a portion that is closer to the camera section 100 than the relevant portion and has no other portion. As mentioned above, at a construction site, construction usually progresses from the back, so the part corresponding to the member closest to the camera corresponds to the latest construction part, that is, the object to be measured or directly photographed. It is from.

A specific example will be described with reference to FIGS. 12 and 12C. FIG. 12B is a top view of the imaging state. FIG. 12B shows the positional relationship between the members No. 1 to No. 7 corresponding to the parts P1 to P7 of the captured image and the camera section 100. As shown in FIG. Assume that members No. 1 to No. 5 are vertical columns, and members No. 6 and No. 7 are horizontal beams. FIG. 12C is a photographed image of each part P1 to P7. Member No. 2 and member No. 4 closer to the camera unit 100 are members to be photographed and measured. Sites P2 and P4 corresponding to member No. 2 and member No. 4 are determined to be surface layer sites in step S308.

When the target part determination unit 136 determines that there is another part overlapping the part Pk (YES in step S306), the target part determination unit 136 determines that the part Pk is not a surface layer part (step S310). After step S308 or step S310, the target part determination unit 136 determines whether k=n (step S312), and if k=n (YES in step S312), ends this process. When the target part determination unit 136 determines that k=n is not true (NO in step S312), it sets k=k+1 (step S314) and returns to step S304.

FIG. 13A is a subroutine of target part determination 2. FIG. FIG. 13B is a diagram for explaining the process of target part determination 2. FIG. Target part determination 2 is a process of determining the photographing distance of a part, classifying the part according to the photographing distance, and judging the surface layer part according to the classification.

The target part determination unit 136 determines the photographing distance of each part (P1 to Pn) and classifies them according to the distance (step S330). The imaging distance of each part is calculated by the imaging distance calculator 132 .

FIG. 13B is a diagram for explaining the process of target part determination 2. FIG. FIG. 13B, like FIG. 12B, is a top view of the imaging state. Members No. 10 to 12 belong to the shooting distance range of 1 to 2 m. Members Nos. 20 to 23 belong to the shooting distance range of 2 to 3 m. Members No. 30 to 32 belong to the photographing distance range of 3 to 4 m. A member within a shooting distance range of 1 to 2 m near the camera unit 100 is a member to be photographed and measured. Note that the setting of the shooting distance range described above is an example. The shooting distance range may be set in advance by the photographer based on the conditions of the construction site.

The target part determination unit 136 is initialized with k=1 (step S332). The target part determination unit 136 determines whether the part Pk is included in the closest distance group (step S334). The closest distance group is the 1-2 m group in FIG. 13B.

When the target part determination unit 136 determines that the part Pk is included in the closest distance group (YES in step S334), the target part determination unit 136 determines that the part Pk is a surface part (step S336). When the target part determination unit 136 determines that Pk is not included in the closest distance group (NO in step S334), it determines that part Pk is not a surface part (step S338).

The target part determination unit 136 determines whether k=n (step S340), and when it determines that k=n (YES in step S340), ends this process. When the target part determination unit 136 determines that k=n is not true (NO in step S340), it sets k=k+1 (step S342) and returns to step S334.

When a plurality of target parts are arranged in a horizontal row, the imaging distance to the target part at the center of the screen becomes the shortest, and the imaging distance of each target part varies. According to the processing of the target part determination 2, the target part can be determined collectively because the determination is performed in groups of a predetermined range.

FIG. 14A is a subroutine of target part determination 3. FIG. 14B and 14C are diagrams for explaining the process of target part determination 3. FIG. Target part determination 3 acquires the construction period of each member at the current construction site from the design drawing (design drawing information 422) and the parts list (member DB 424), and determines the part corresponding to the member with the latest construction period as the part of the photographed image. Identify from In the target part determination 3, the specified part is determined as the target part.

The target part determination unit 136 determines the member with the latest construction period (step S360). Specifically, the target part determination unit 136 refers to the design drawing information 422 and the member DB 424 acquired from the external device 40 to determine the member with the latest construction period at the predetermined construction site. FIG. 14B is an example of a member list based on the member DB 424. FIG. Based on FIG. 14B, member Nos. 301 to 305 are determined to be the latest construction period. FIG. 14C is an example of corresponding blueprint information.

The target part determining unit 136 associates the member with the part based on the image features (the shape of the member, the construction position of the member, etc.) and the photographing distance determination (step S362). Specifically, the target part determination unit 136 refers to the design drawing information 422 and the part information from the part DB 424, and associates the part with the part by image feature and photographing distance determination. Image features and shooting distance determination are the same as those in step S300.

The target part determination unit 136 performs initialization with k=1 (step S364). The target part determination unit 136 determines whether the part Pk is included in the newest construction period group (step S366).

When the target part determination unit 136 determines that the part Pk is included in the newest construction period group (YES in step S366), it determines that it is a target part (step S368). When the target part determination unit 136 determines that Pk is not included in the newest construction period group (NO in step S366), it determines that it is not a target part (step S370).

The target part determination unit 136 determines whether k=n (step S372), and if it determines that k=n (YES in step S372), ends this process. When the target part determination unit 136 determines that k=n is not true (NO in step S372), it sets k=k+1 (step S374) and returns to step S366.

According to the process of target part determination 3, the target part is determined without being directly affected by the distance of the photographing. Therefore, even if there is another member at a position closer to the camera unit 100 than the target part, it is possible to prevent erroneous determination.

FIG. 15A is a subroutine of target part determination 4. FIG. FIG. 15B is a diagram for explaining the processing of target part determination 4. FIG. Target part determination 4 is a process that enables determination of a target part even when the image is taken in a state in which the subject is not facing the subject.

The principle of target part determination 4 will be briefly described. First, the target part determination unit 136 creates plane coordinates of the members, specifies the member closest to the camera unit 100 from the plane coordinates, and determines this member as one of the surface layer members. Next, the target part determination unit 136 sets a plurality of planes composed of the specified member and other members. The target part determination unit 136 identifies the plane that faces the camera unit 100 most directly from among the plurality of planes that have been set. Then, the target part determination unit 136 determines the part corresponding to the member belonging to the specified plane as the target part.

FIG. 15B, like FIG. 12B, is a top view of the imaging state. Assume that members (columns) Nos. 20 to 44 are arranged at the construction site. The members No. 20 to 23, the members No. 30 to 34, and the members No. 40 to 44 are all constructed at the same time. The members No. 20 to 23 near the camera unit 100 are the members with the latest construction period (target portions). The members No. 30 to 34 and the members No. 40 to 44 are older than the members No. 20 to 23. The camera section 100 is in an oblique state with respect to the pillar. This is because, depending on the state of the construction site, there are cases where the image can only be photographed from an oblique direction.

The target part determination unit 136 creates plane coordinates of the member corresponding to each part (step S380). The target part determination unit 136 creates the plane coordinates of the member based on the image features and the shooting distance. Furthermore, the target part determination unit 136 may also refer to the design drawing information 422 and the member DB 424 to create the plane coordinates of the member. The plane coordinates are, for example, coordinates based on XY orthogonal coordinates (see FIG. 15B) with the optical axis C of the camera unit 100 as a reference.

The target part determination unit 136 uses, for example, the bottom right of the pillar as the position reference of the member. The target part determination unit 136 identifies the member closest to the camera unit 100 (step S382). In the example of FIG. 15B, the No. 20 member is identified.

The target part determination unit 136 sets a plane including the identified No. 20 member (step S384). In FIG. 15B, for example, the plane H1 is set for members No. 20 to 23, the plane H2 is set for members No. 20 and 34, the plane H3 is set for members No. 20 and 44, and the plane H3 is set for members No. 20, 30, and 40. A plane H4 is set. Other planes are omitted.

The target part determination unit 136 identifies the plane most facing the camera unit 100 among the set planes. The target part determination unit 136 identifies a plane with the shallowest angle with respect to the X-axis orthogonal to the optical axis C. FIG. A plane with the shallowest angle is called a specific plane. In the example of FIG. 15B, plane H1 is identified. The target part determination unit 136 identifies parts corresponding to members No. 20 to No. 23 belonging to the specific plane H1 as target parts. Since the plane coordinate values of the members belonging to the specific plane H1 are calculated, the parts corresponding to the members belonging to the specific plane H1 are determined according to the image characteristics and the shooting distance as described above.

The target part determination unit 136 performs initialization with k=1 (step S388). The target part determination unit 136 determines whether the part Pk corresponds to a member placed on the specific plane (hereinafter referred to as a corresponding part) (step S390).

When the target part determination unit 136 determines that the part Pk is the corresponding part (YES in step S390), the target part determination unit 136 determines that the part Pk is the surface layer part (step S392). When the target part determination unit 136 determines that the part Pk is not the corresponding part (NO in step S390), it determines that the part Pk is not the surface layer part (step S394).

The target part determination unit 136 determines whether k=n (step S396), and if it determines that k=n (YES in step S396), ends this process. When the target part determination unit 136 determines that k=n is not true (NO in step S396), it sets k=k+1 (step S398) and returns to step S390.

According to the process of target part determination 4, the target part can be determined even when the camera unit 100 cannot face the member directly, so the restrictions on the imaging position and imaging direction are reduced, and the freedom of imaging is increased. increases.

According to the processes of target part determination 1 to 4 described above, even if the photographed image contains a mixture of new and old parts, the newly constructed part can be accurately determined. As a result, the shape of the site is accurately measured.

In addition, the body part to be imaged is highlighted and displayed in the live view image, the rec view image, or the image of the configuration information. For this reason, it is possible to prevent omissions of photographing, duplicate photographing, etc., without checking the designated photographing locations and photographing contents one by one while referring to design drawings and the like.

Note that an approval button for the determination result of the target site and a change button for instructing change of the target site may be provided on the screen. This is because an appropriate part may not be determined depending on the situation of the construction site. When an instruction to change the target part is given, the target part determination unit 136 switches the method of the target part determinations 1 to 4 to be used or changes the priority order of the methods of the target part determinations 1 to 4. Further, the target part determination unit 136 may allow the photographer to select the method of target part determination 1 to 4 to be used. Alternatively, the target part determination unit 136 may first cause the photographer to specify the target part on the screen, and select a target part determination method according to the specified target part.

Further, in the processes of target part determination 1 to 4 described above, determination of a surface layer part has been described as a specific example. However, the target sites for imaging and measurement are not limited to surface layer sites. A part other than the superficial layer part, for example, the second layer part, may be the part to be imaged. In that case, for example, by the target part determination 3, the target part can be specified based on the member with the latest construction period.

<Embodiment 2>
In the second embodiment, the photographing range is set according to the required accuracy of measurement, and the photographing range is added to the part of the configuration information of the guide screen and displayed. Since the configuration of the information acquisition system 1 and the hardware configuration of the information acquisition device 10 are the same as those shown in FIGS. 2A, 2B, and 3, description thereof will be omitted.

FIG. 16 is a flowchart for explaining guide screen display processing according to the second embodiment. The flowchart of FIG. 16 is obtained by adding a process of displaying a shooting range guide indicating a recommended shooting range to the flowchart of FIG. 4A. The recommended shooting range is a shooting range corresponding to a shooting distance that satisfies the required accuracy for the measured value of the member shape. The photographing range setting unit 140 described above sets the photographing range according to the required accuracy for the measurement value. Steps S400 to S406 are added processing.

In addition, below, it demonstrates with reference to FIG. 17 suitably. Note that the guide screen in FIG. 17 is an example in which the first configuration information 154a is displayed as the configuration information. The measurement unit 130 measures the distance T between the pillars based on the live view image (step S400). The guide screen creating unit 142 creates the guide screen 220 (FIG. 17) to which the information of the interval T (1 m) is added. The guide screen 220 is provided with a button for displaying reference information created by the reference information creation unit 134 . "Standard guide display" is a button for displaying a standard shooting guide. “Display similarity guide” is a button for displaying a similarity shooting guide.

19A to 19C are diagrams showing the principle of measuring the distance between pillars using captured images. FIG. 19A shows a case where the pillar P is photographed from the front. In FIG. 19A, the interval T between the columns P, the focal length F of the lens portion 102a, the image width Y1, and the photographing distance D are assumed. The focal length F is included in the camera section characteristic information 156 . The shooting distance D is calculated by the shooting distance calculation unit 132 through focus detection or the like. As described above, the interval T between the pillars P is calculated by the following formula.
T=D×Y1/F Expression (4)

FIG. 19B shows a case where the pillar P is photographed from an oblique direction. In FIG. 19B, the interval T between the columns P, the focal length F of the lens portion 102a, the image width Y1, and the photographing distances D1 and D2. The focal length F is included in the camera section characteristic information 156 . The photographing distances D1 and D2 are calculated by the photographing distance calculator 132 through focus detection or the like. The interval T between the pillars P is calculated based on the following formula.
ΔD=D2-D1 Expression (5)
T=W1/cos θ=W1/cos(arctan(ΔD/W1)) Expression (6)

In this manner, the interval T between the pillars P can be calculated even when the camera unit 100 is facing the pillar P in an oblique direction instead of facing the pillar P directly. By photographing from an oblique direction, it becomes easier to photograph the entire image, and the degree of freedom during photographing increases.

FIG. 19C shows the principle of measurement when photographing the base of the floor and the beams and girders of the ceiling. This is because the part to be imaged is not limited to a vertical member such as a pillar. For example, P is a floor member. When photographing a floor member, the photographer positions the camera unit 100 diagonally downward as shown in FIG. 19C. The tilt of camera unit 100 is detected by orientation sensor 110 . The interval T between the bases P is calculated based on the following formula.
ΔD=D2−D1 Expression (7)
T=W1/cos θ Expression (8)

Return to FIG. The control unit 120 instructs the photographer to input the required accuracy of measurement (step S402). If the required measurement accuracy has already been input, the control unit 120 does not perform this process. For example, assume that 1 mm is input as the required accuracy of the pillar P.

The control unit 120 determines the shooting distance and the angle of view (step S404). The shooting range setting unit 140 calculates a shooting distance that satisfies the required accuracy of measurement. The higher the required accuracy of measurement, the shorter the shooting distance is calculated. As described above, the imaging range setting unit 140 calculates the imaging distance for satisfying the required measurement accuracy based on the characteristics of the camera unit 100 (the MTF and distortion of the lens unit 102a, the resolution of the image sensor 102b, etc.). The characteristics of the camera section 100 are included in the camera section characteristic information 156 .

Based on the calculated shooting distance and the characteristics of the camera unit 100, the shooting range setting unit 140 sets a shooting range that satisfies the required measurement accuracy. The guide screen creation unit 142 adds a shooting range guide G to a predetermined position of the configuration information 154 based on the set shooting range. The control unit 120 displays a guide screen to which the shooting range guide G is added at a predetermined position of the configuration information 154 (step S406).

A guide screen 221 in FIG. 17 is an example of a guide screen to which a shooting range guide G is added. "Accuracy of 1 mm" on the guide screen 221 is the input required accuracy. The "recommended shooting distance of 1 m" is a distance calculated to satisfy the required accuracy. A shooting range guide G is displayed in the first configuration information 154a. The photographing range guide G indicates the range photographed at the recommended photographing distance of 1 m. The photographing range guide G proposes a composition in which two pillars are arranged on the left and right sides of the screen. Note that the screen on the left is a live view image. The required accuracy may be manually entered or pre-stored. Further, the required accuracy may be a numerical value that changes according to the determination of the object from the image, the situation, or the required output form.

On the guide screen 221, the range that includes the first and second pillars from the left is taken as the first shot, the range that includes the third and fourth pillars P is taken as the second shot, and the fifth shot is taken. and the sixth pillar P are recommended to be photographed for the third time. The guide screen 221 also displays that the number of shots is three. This is because the required accuracy is satisfied even if the six pillars are photographed one by one at a distance of 1 m or less, but the accuracy is satisfied and the efficiency is better if the two pillars are photographed together.

A guide screen 222 in FIG. 17 is a screen after the first photographing is completed. A photographing range guide G covering the first and second pillars is displayed, for example, by hatching Q2. Hatching Q2 corresponds to the electronic marker Q in FIG. That is, hatching Q2 is status information. This makes it clear that the first and second pillars P have been photographed.

In addition, it is better to create a guide screen considering whether it is possible to actually shoot, and obtain information such as the structure of the building and dangerous places from an external database, etc., and actually shoot according to the guide. It may be possible to determine whether or not it is possible. In other words, it is possible to calculate from the angle of view and the photographing distance which position should be photographed from the guided photographing range. In that case, as shown in FIG. 19B, it may be recommended to shoot from an oblique angle rather than from the front. Even when shooting obliquely, it is sufficient to guess whether the place is safe or not. In addition to the method of temporarily determining the shooting guide and then verifying the shooting position, the shooting area and accuracy can be determined as appropriate from safe and suitable location candidates (judging from maps, floor plans, building design data, etc.). There is also a way to decide the shooting location and go.

18A to 18C are examples showing other display modes of the shooting range guide G. FIG. FIG. 18A is an example of a guide screen including second configuration information 154b. The guide screen 230 of FIG. 18A is an example in which the second configuration information 154b is included, and the imaging range guide G is displayed by horizontal lines in the second configuration information 154b. An electronic marker Q similar to that in FIG. 7B is displayed in the photographing range guide G that has been photographed. Also, "Focusing distance OK" displayed on the guide screen 230 indicates that the current photographing distance is within the photographing distance that satisfies the required accuracy.

FIG. 18B is an example of a guide screen that includes configuration information 154 in a new form. The configuration information 154 in the new form is also referred to as third configuration information 154c in a state in which six pillars are expanded in a plane.

On the guide screen 231, three shooting range guides G are displayed so as to cover the corresponding pillars in the third configuration information 154c. The numbers (1, 2, 3) described in the shooting range guide G indicate the shooting order. In the shooting range guide G numbered 1, an electronic marker Q indicating that shooting has been completed is displayed as situation information.

FIG. 18C is an example of a guide screen 232 in which multiple shots are displayed in a panoramic image style. As the configuration information, the same third configuration information 154c as in FIG. 19B is displayed. On the guide screen 232, shooting is proposed in which the edges of the images are overlapped. Therefore, in the third configuration information 154c of the guide screen 232, the photographing range guide G is displayed such that the end portions of the photographed images overlap. Note that the guide screen 232 in FIG. 18C is in the state of the second shooting after the first shooting is completed.

The horizontally long image displayed on the lower side of the guide screen 232 is an image in which two images are pasted together. The left side is the first photographed image (rec view image). The image on the right is the second image, which is a live view image. The central portion V is the overlapping portion of the first image and the second image.

According to the photographing range guide G display described above, it is possible to prevent omissions of photographing and duplicate photographing without checking the photographing locations and photographing contents instructed for each construction site while referring to the design drawing. In addition, since the photographing range that satisfies the required accuracy of part measurement is specified, appropriate construction image photographing can be easily performed.

In addition to measuring the parts of buildings described above, a group of buildings in a specific area, each tree in a forest, crops in a field, and animals and plants in other areas may be considered as target parts. In addition, macroscopically, the present invention can be applied to three-dimensional measurement of parts, wiring patterns, semiconductors, and the like measured by microscope images, and the present application can be applied by considering each object as each part.

In other words, applications for appropriately, accurately, and efficiently measuring each site existing in a specific region and forming a group can be said to be covered by the present application. In addition, although the shape of each of these objects is not as simple as a building, the start and end points of the part to be measured can be easily measured not only by simple contour determination such as edge enhancement, but also by various image processing. A step may be interposed, and if there are many measurement examples that are frequently measured and used as models, these may be used as teacher images and teacher measurement examples, and judgment may be made using the results of machine learning by artificial intelligence. . Note that the information acquisition device 10 may be an information processing device including the control unit 120 , the storage unit 150 and the display unit 160 excluding the camera unit 100 .

It should be noted that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the spirit of the present invention at the implementation stage. Also, various inventions can be formed by appropriate combinations of the plurality of constituent elements disclosed in the above embodiments. For example, all the components shown in the embodiments may be combined as appropriate. Furthermore, components across different embodiments may be combined as appropriate. It goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

Based on the above description, representative aspects of the present invention other than those described in the claims are as follows.

(1) In an information acquisition device that acquires a photographed image of a member, a target region determination unit that determines a region corresponding to a specific member from the photographed image as a target region, and a target region from the photographed image. a measurement unit that measures the shape of the specific member by measuring the shape of the specific member; and determining a target portion corresponding to the specific member from among the portions of the photographed image.

(2) The information acquisition device according to (1), further comprising a control unit that emphasizes and displays the determined target region in the captured image.

(3) A photographing distance calculation unit that calculates a photographing distance corresponding to a part of the photographed image is provided, and the target part determination unit judges the target part from the photographing distance of the part of the photographed image. The information acquisition device according to (1).

(4) The target part determination unit divides the parts of the photographed image into groups according to the photographing distance of the part of the photographed image, and selects the part of the photographed image belonging to the group with the shortest photographing distance as the target part. The information acquisition device according to (3), characterized in that it is determined as a part.

(5) The information acquisition device according to (3), wherein the target site determination unit determines the target site from features of the shape of the site in the captured image.

(6) In an information acquisition device that acquires a photographed image of a member, a target part determination unit that determines a part corresponding to a specific member from the photographed image as a target part, and a target part from the photographed image. and a measurement unit that measures the shape of the specific member by performing measurement on the object, and the target part determination unit determines, in the photographed image in which a plurality of members with different construction periods are photographed, at the photographing distance Accordingly, determine the part corresponding to the latest construction period member as the target part,
An information acquisition device characterized by:

(7) The target part determination unit refers to the construction period information of the member, and determines the part corresponding to the member with the latest construction period among the photographed images of a plurality of members with different construction periods as the target part. The information acquisition device according to (6), characterized by:

(8) A photographing distance calculation unit for calculating a photographing distance corresponding to a part of the photographed image is further provided, and the photographed image is such that the member of the latest construction period is in front of the photographing camera than the member of the other construction period. In the image of the photographed construction work, the target part determination unit selects a part corresponding to the member with the latest construction period according to the photographing distance among the photographed images in which a plurality of members with different construction periods are photographed. as the target part, the information acquisition device according to (6).

(9) In a measurement method for measuring the shape of a member based on a photographed image, a determination step of determining a portion corresponding to a specific member from the photographed image as a target portion, and determining the target portion from the photographed image. and a measurement step of measuring the shape of the specific member by measuring the shape of the specific member. A measuring method, wherein a target portion corresponding to the specific member is determined from among the portions of the photographed image.

(10) In a program for causing a computer to execute a measurement method for measuring the shape of a member based on a photographed image of the member, a determination step of determining a portion corresponding to a specific member from the photographed image as a target portion; a measurement step of measuring the shape of the specific member by measuring the target part from the image, wherein the determination step exists on the side close to the direction in which the photographing camera exists. A program, wherein the member is the specific member, and a target portion corresponding to the specific member is determined from among the portions of the photographed image.

(11) In an information acquisition device that acquires a photographed image of a member, a target portion determination unit that determines, as a target portion, a portion corresponding to a specific member from among a plurality of members as configuration information acquired from the photographed image; and a measurement unit for measuring the shape of the specific member by measuring the target portion from the photographed image, wherein the target portion determination unit is arranged in the direction in which the photographing camera exists. An information acquiring apparatus, wherein a member existing on a near side is defined as the specific member, and a target portion corresponding to the specific member is determined from among the portions of the photographed image.

(12) The information acquisition device according to (11), further comprising a control unit that emphasizes and displays the determined target region in the captured image.

(13) A photographing distance calculation unit that calculates a photographing distance corresponding to a part of the photographed image is provided, and the target part determination unit judges the target part from the photographing distance of the part of the photographed image. The information acquisition device according to (11).

(14) In a measuring method for measuring the shape of a member based on a photographed image, a determination step of determining, as a target portion, a portion corresponding to a specific member from among a plurality of members as configuration information acquired from the photographed image; a measuring step of measuring the target part from the photographed image to measure the shape of the specific member, wherein the determining step is closer to the direction in which the photographing camera exists. A measuring method, wherein a member existing in the body is defined as the specific member, and a target portion corresponding to the specific member is determined from among the portions of the photographed image.

(15) In a program that causes a computer to execute a measurement method for measuring the shape of a member based on the photographed image of the member, a portion corresponding to a specific member is selected from among a plurality of members as configuration information acquired from the photographed image. and a measuring step of measuring the target part from the photographed image and measuring the shape of the specific member. A program for determining a target portion corresponding to the specific member from among portions of the photographed image, with a member existing on a side closer to a direction in which a camera exists as the specific member.

1 Information acquisition system 10 Information acquisition device 40 External device 100 Camera unit 120 Control unit 126 Configuration information acquisition unit 128 Imaged part determination unit 130 Measurement unit 132 Shooting distance calculation unit 134 Reference information creation unit 136 Target part determination unit 138 Emphasis display setting Unit 140 Shooting range setting unit 142 Guide screen creation unit 144 Display control unit 150 Storage unit 154 Configuration information 154a First configuration information 154b Second configuration information 156 Camera unit characteristic information 160 Display unit 165 Touch panel 170 Communication unit 400 Control unit 410 Communication unit 420 Storage unit 422 Design drawing information 424 Member DB

Claims (5)

In an information acquisition device that acquires a photographed image of a member,
a target part determination unit that determines a part corresponding to a specific member from among the plurality of members appearing in the photographed image as a target part;
a measurement unit that measures the target part from the captured image and measures the shape of the part;
a shooting distance calculation unit that calculates a shooting distance corresponding to the part of the shot image,
The target part determination unit determines the shape of the part as a feature, determines to which type of member the part belongs based on the feature, and selects the same type as the specific member from among the parts. Determining the part belonging to the member containing the specific member ,
Furthermore, the parts belonging to the members that are the same type as the specific member and that include the specific member are grouped according to the imaging distance, and the target part is determined from the parts based on the groups. do,
An information acquisition device characterized by: In an information acquisition device that acquires a photographed image of a member,
a target part determination unit that determines a part corresponding to a specific member from among the plurality of members appearing in the photographed image as a target part;
a measurement unit that measures the target part from the captured image and measures the shape of the part;
a shooting distance calculation unit that calculates a shooting distance corresponding to the part of the shot image,
The target part determination unit determines the shape of the part as a feature, determines which type of the member the part belongs to based on the feature, and selects the same type as the specific member from among the parts. Determining the part belonging to the member containing the specific member ,
Furthermore, the parts belonging to the member that is the same type as the specific member and that includes the specific member are grouped according to the photographing distance, and the group that exists on the side close to the direction in which the photographing camera exists. Determining the part belonging to as the target part,
An information acquisition device characterized by: A control unit that emphasizes and displays the determined target part in the captured image,
3. The information acquisition device according to claim 1 or 2 , characterized in that: In a program that causes a computer to execute a measurement method for measuring the shape of a member based on a photographed image of the member,
a determination step of determining a site corresponding to a specific member from among a plurality of members appearing in the photographed image as a target site;
a measurement step of measuring the target part from the captured image to measure the shape of the part;
a calculating step of calculating a photographing distance corresponding to the part of the photographed image;
In the determination step, the shape of the part is determined as a feature, and based on the feature, it is determined which type of the member the part belongs to, and which part is of the same type as the specific member from among the parts. to determine the portion belonging to the member containing the specific member ,
Furthermore, the parts belonging to the members that are the same type as the specific member and that include the specific member are grouped according to the photographing distance, and the target part is determined from the parts based on the groups. do,
A program characterized by In a program that causes a computer to execute a measurement method for measuring the shape of a member based on a photographed image of the member,
a determination step of determining a site corresponding to a specific member from among a plurality of members appearing in the photographed image as a target site;
a measurement step of measuring the target part from the captured image to measure the shape of the part;
a calculating step of calculating a photographing distance corresponding to the part of the photographed image;
In the determination step, the shape of the part is determined as a feature, and based on the feature, it is determined which type of the member the part belongs to, and which part is of the same type as the specific member from among the parts. to determine the portion belonging to the member containing the specific member ,
Furthermore, the parts belonging to the member that is the same type as the specific member and that includes the specific member are grouped according to the photographing distance, and the group that exists on the side closer to the direction in which the photographing camera exists. Determining the part belonging to as the target part,
A program characterized by

Images