JP6639832B2 - Imaging device, imaging method, imaging program - Google Patents

Description

  The present invention relates to an imaging apparatus that easily creates bird's-eye view data and floor plan data of a building from a captured image of the building and generates imaging guide information, and an imaging method and an imaging program thereof.

  2. Description of the Related Art Conventionally, an optical image formed by an imaging optical system is sequentially photoelectrically converted using a photoelectric conversion element or the like (hereinafter, referred to as an imaging element), and an obtained image signal is recorded on a storage medium as digital image data of a predetermined form. In addition, an imaging device configured to include an image display device or the like that displays a still image or a moving image based on the digital image data has been practically used and widely used.

  2. Description of the Related Art Conventional imaging devices are used for purposes such as sequentially recording a plurality of images (photographs) with a series of actions at various events, progress of construction and work, processes, and the like as imaging targets. There is. Specifically, there are, for example, records of various events such as weddings, progress records of construction works such as civil engineering structures and buildings, and records of manufacturing work processes.

  Conventionally, a two-dimensional plan view (for example, a sketch drawing) is created from image data acquired by using this type of imaging device, for example, two-dimensional image data obtained by capturing the entire image of an object, Various devices for generating three-dimensional data from data have been proposed and put into practical use, for example, in JP-A-2001-033246 and JP-A-2004-280196.

  For example, an apparatus disclosed in Japanese Patent Application Laid-Open No. 2001-033246 calculates an imaging position of a subject image based on the subject image including a polygon having a known shape and size on a plane. A two-dimensional plan view is created from the subject image with reference to the information on the imaging position.

  An apparatus disclosed in Japanese Patent Application Laid-Open No. 2004-280196 generates three-dimensional data on a computer for a subject having a space such as a room in which the image is taken, based on a two-dimensional image.

JP 2001-033246 A JP 2004-280196 A

  By the way, an image acquired by a conventional imaging device or the like plays an important role not only in recording memories of individual scenes but also in various scenes. However, the relationship between the image data acquired in each scene and the actual target may become unclear, and easy-to-understand assistance to the user is important.

  In addition, in applications where a conventional imaging apparatus is used, for example, in various applications (events) or recording applications such as construction and work, a large amount of image data is acquired for each individual event (event) or construction. Become. Then, the acquired large amount of image data is sorted and sorted, such as the flow, process, progress, etc. of each event, in a time series or per related matter, and arranged in a systematic manner. Will be. Conventionally, these sorting and classifying operations have relied on human power by a photographer or the like who handles the imaging apparatus. Also in this case, the relationship between the acquired image data and the actual object may become unknown, which may hinder the sorting and sorting of the image data at a later date.

  However, in the prior art disclosed in, for example, JP-A-2001-033246 and JP-A-2004-280196, sketch drawing data of a two-dimensional plan view and the like are obtained from two-dimensional image data of an entire image of an object. Is generated, and a series of subsequent imaging guide information is generated and displayed based on this. However, these are merely conversions of how information obtained from an image is displayed.

  The present invention has been made in view of the above points, and has as its object the purpose of generating imaging guide information for assisting an imaging operation that is easy to understand and does not fail, thereby improving user's workability. An object of the present invention is to provide an imaging device having a function that can be improved, an imaging method thereof, and an imaging program.

In order to achieve the above object, an imaging device according to one embodiment of the present invention includes an imaging unit that receives an optical image of a structure and generates image data, and displays an image based on the image data generated by the imaging unit. A display unit to perform, a contour determination unit that determines the contour of the structure, and an intersection of a plurality of contours of the contour of the structure to predict the shape of the structure, and an intersection of the detected contour. And, based on the predicted shape of the structure, the imaging recommendation area of the structure is specified , and in the imaging operation for the individual structures in the imaging recommendation area, the necessary imaging is imaged without omission. An area specifying unit that generates imaging guide information for guiding to obtain, and a display control unit that causes the display unit to display the imaging guide information, wherein the area specifying unit recommends imaging of the identified structure. Out of the area The blind spot area, which is a surface of a photographable structure that is a blind spot that is not photographed in the image data obtained by receiving the optical image of the structure, is a line where the contour generated by contacting the floor, ceiling or wall surface intersects the rear area includes you guess set as blind spot section.

An imaging method according to one embodiment of the present invention includes the steps of: receiving an optical image of a structure to generate image data; displaying an image based on the generated image data; and determining an outline of the structure A step, a detection / prediction step of detecting the intersection of a plurality of contours of the contour of the structure and predicting the shape of the structure, and an intersection of the contour detected in the detection / prediction and the predicted structure. The imaging recommendation area of the structure is specified based on the shape of the above , and among the imaging recommendation areas of the specified structure, the blind spot which is not captured in the image data obtained by receiving the optical image of the structure is obtained. the surface is a blind area of the recordable structures, guess by setting the rear area of the contour resulting in contact with the floor or ceiling or wall skilled comprises lines which intersect the blind section, the imaging recommended area In the imaging operation of the definitive individual structures an imaging target, an area specifying step of generating shooting guide information for guiding so as imaged without omission imaging in need, and displaying the photographed guide information, the Including.

An imaging program according to one embodiment of the present invention receives an optical image of a structure, generates image data, displays an image based on the generated image data, determines an outline of the structure, The intersection of a plurality of outlines of the outline is detected to predict the shape of the structure, and based on the intersection of the detected outline and the predicted shape of the structure, an imaging recommendation area of the structure is specified . Of the imaging recommendation areas of the specified structure, a blind spot area that is a surface of a photographable structure that is a blind spot that is not photographed in image data obtained by receiving an optical image of the structure, a floor or a ceiling or The back side of the plane including the line where the contour arising from contact with the wall surface intersects is set as a blind spot , and the necessary imaging is performed in the imaging operation with the individual structures in the imaging recommended area as the imaging target. Take without omission It generates shooting guide information for guiding so as to, to execute the imaging method to display the shooting guide information to the computer.

  According to the present invention, there is provided an imaging apparatus having a function capable of improving the workability of a user by generating imaging guide information for assisting an imaging operation that is easy to understand and without failure, and an imaging method and an imaging program for the imaging apparatus. To provide.

FIG. 1 is a block diagram illustrating a main configuration of an imaging device (camera) according to a first embodiment of the present invention. FIG. 1 is a conceptual diagram showing a situation when an imaging operation is performed using the imaging device (camera) of FIG. 1. Example of display of auxiliary data (floor view data) acquired under the imaging conditions in FIG. Display example of the display unit when performing the imaging operation after the acquisition of the auxiliary data (drawing data) in FIG. 3 Flowchart showing a camera control processing sequence of the imaging device (camera) according to the first embodiment of the present invention (imaging mode) Flowchart showing a camera control processing sequence of the imaging apparatus (camera) according to the first embodiment of the present invention (other than in the imaging mode) 5 is a flowchart showing a detailed sequence of a blind spot prediction process (step S111) in the camera control process sequence of FIG. FIG. 1 is a conceptual diagram showing a simplified file structure of image data acquired when an imaging operation is performed using the imaging device (camera) of FIG. 1. A conceptual diagram showing a situation when performing an imaging operation using the imaging device (camera) of the second embodiment of the present invention. Display example of auxiliary data (floor plan data) acquired under the imaging conditions in FIG. 9 A flowchart showing details of a blind spot prediction process (a process corresponding to the process of step S111 in FIG. 5) in the operation of the imaging device (camera) according to the second embodiment of the present invention.

  Hereinafter, the present invention will be described with reference to the illustrated embodiments. Each drawing used in the following description is schematically shown, and in order to show each component in a size recognizable in the drawing, the dimensional relationship and scale of each member are different for each component. In some cases. Therefore, the present invention is limited to the illustrated form only with respect to the number of components, the shape of the components, the ratio of the size of the components, the relative positional relationship between the components, and the like described in these drawings. Not something.

  In each embodiment of the present invention described below, for example, an optical image formed by an imaging optical system is converted into, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor). A photoelectric conversion element or the like (hereinafter, referred to as an image pickup element) such as a film semiconductor) type image sensor or the like (hereinafter, referred to as an imaging element), and an obtained image signal is converted into image data in a predetermined form (for example, a still image or a moving image). An image display device that records the digital image data on a storage medium and reproduces and displays a still image or a moving image based on the digital image data recorded on the storage medium, such as a liquid crystal display (LCD) or Organic Electro-Luminescence (OEL) displays, etc. This is an example of an imaging device provided and configured, for example, a digital camera or a video camera (hereinafter, simply referred to as a camera).

  The form of the camera as an imaging device to which the present invention can be applied does not need to be particularly limited, and can be widely applied to various types of cameras that are generally put into practical use. For example, a so-called lens-integrated camera in which a lens barrel provided with an imaging optical system is integrally formed with the apparatus main body, or a so-called lens-integrated camera in which the lens barrel is detachably mounted on the apparatus main body. In addition to existing forms of cameras such as interchangeable lens cameras, for example, an image pickup apparatus having a minimum function for realizing an image pickup function and an image recording function and a communication function with an external device (even with an integrated lens barrel). A communication control device having a communication function with an external device including the imaging device and a function capable of executing various signal processing and control processing, for example, a general distribution system. The present invention can be applied to an imaging system or the like constructed by combining a tablet computer or a smartphone type mobile phone.

  The imaging system performs mutual communication between the communication control device and the imaging device under the action of an application program introduced into the communication control device, controls the imaging device by the communication control device, and performs an imaging operation. Or transmit the image data obtained by the imaging device from the imaging device to the control device via a communication function, perform various signal processing in the communication control device, or perform these signal processing. It is configured so that image data and the like obtained as a result can be recorded.

  First, before describing the details of the present invention, a schematic configuration of an imaging device (camera) to which the present invention is applied will be described below mainly with reference to FIG.

[First Embodiment]
FIG. 1 is a block diagram showing a main configuration of the imaging apparatus (camera) according to the first embodiment of the present invention. In FIG. 1, the form of the camera 1 is shown without any particular limitation.

  The camera 1 that is the imaging device of the present embodiment receives an optical image of an object formed by the imaging optical system, photoelectrically converts the optical image to obtain image data, records the obtained image data, or An image reproducing / recording apparatus having a configuration capable of displaying an image.

  For this purpose, as shown in FIG. 1, the camera 1 includes an image processing control unit 11, an imaging unit 12, a communication unit 13, a recording unit 14, an operation determination unit 16, an attitude sensor 17a, a GPS 17b, It has an orientation sensor 17c, a display unit 18, a touch panel 19 and the like.

  The image processing control unit 11 is an electronic circuit unit including a central processing unit (CPU) and the like. The image processing control unit 11 records image signals, audio signals, and the like in addition to predetermined signal processing, for example, various digital image signal processing such as color signal generation processing and matrix conversion processing, for image signals from the imaging unit 12. An electronic circuit unit serving as an image signal processing unit that performs various signal processes such as performing an encoding process or reading out recorded image data and audio data from the recording unit 14 and performing a decoding process; An electronic circuit unit as a control unit for controlling the overall operation of the camera 1 is provided. As the software program executed in the image processing control unit 11, a program stored in advance in the recording unit 14 or a separately provided ROM (Read Only Memory) not shown is used.

  The image processing control unit 11 includes a display control unit 11a, a structure contour determination unit 11b, a guide display control unit 11c, and a blind spot prediction unit 11d. Each of these components (11a, 11b, 11c, 11d) and the like may be constituted by a hardware circuit specifically prepared in the camera 1, or a software program executed by the image processing control unit 11 And the form is not limited.

  The display control unit 11a receives the image signal generated for display by the image processing control unit 11 based on the image signal acquired by the imaging unit 12, and displays the image signal on the display unit 18 to display the image signal as an image. It is an electronic circuit unit to be controlled.

  The structure contour determination unit 11b is an electronic circuit unit that determines a contour portion of a specific imaging target based on an image signal acquired by the imaging unit 12.

  The guide display control unit 11c is an electronic circuit unit for performing guide display such as advice on use, warning, notification, and notification to the user (user) on the display screen of the display unit 18 at a predetermined timing.

  The blind spot prediction unit 11d is an electronic circuit unit that predicts, from the determination result by the structure outline determination unit 11b, a part or a range that is a blind spot from the camera 1 in the target structure to be imaged. Here, the blind spot is a part or range that cannot be seen from a certain position of the camera 1 in a target structure to be imaged.

  The blind spot prediction unit 11d includes a rectangular parallelepiped prediction unit 11e and an imaging plane determination unit 11f. Among them, the rectangular parallelepiped prediction unit 11e is an electronic circuit unit serving as a structure shape prediction unit that predicts the shape of the target structure to be imaged based on the determination result by the structure outline determination unit 11b. Here, the rectangular parallelepiped prediction unit 11e as the structure shape prediction unit predicts that the shape of the imaging target structure is a rectangular parallelepiped.

  In addition, the imaging plane determination unit 11f includes an area where imaging is not possible among the surfaces of the rectangular solid of the imaging target structure predicted by the rectangular parallelepiped prediction unit 11e, that is, an area where imaging is recommended (specifically, a blind spot part). , A blind spot range). The imaging recommendation area of the structure is a portion where the surface of the rectangular parallelepiped constituting the structure is exposed and is ready for imaging. Such an exposed surface is important because it is easy to monitor changes over time and becomes an important evaluation item later.

  The imaging unit 12 includes a lens barrel (not shown) including an imaging optical system such as an optical lens, a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor), and a complementary metal oxide film. This is a configuration unit configured by an electronic circuit unit and the like including a photoelectric conversion element (imaging element) such as a semiconductor) image sensor.

  The imaging unit 12 receives an optical image formed on an imaging surface by an imaging optical system (not shown) of the lens barrel unit and performs a photoelectric conversion process to convert an image signal of a still image or a moving image including a subject image. It is configured to acquire and output this image signal to the image processing control unit 11. The driving of the imaging unit 12 is controlled by the image processing control unit 11. That is, the image processing control unit 11 outputs a drive signal to the imaging unit 12, captures the image signal acquired by the imaging unit 12, performs various signal processing on the image signal, and performs predetermined signal processing. , For example, digital image data representing a still image or a moving image.

  The image data generated in this way is associated with angle-of-view information of the imaging optical system, various information at the time of imaging (subject distance information, exposure information, and the like), and the like. Various types of information (image data related information) associated with the image data are recorded in an image data file. It should be noted that the image data related information may be in a form recorded in another data file associated with the image data, in addition to the above-described form.

  The communication unit 13 performs data transmission with an external device (not shown) using a wireless communication function, receives a control signal from the external device (not shown), and transmits data to an external communication line (not shown) such as the Internet. ), And an electronic circuit unit provided for performing data communication with various external databases (not shown) and the like via this.

  The recording unit 14 is configured to include a recording medium (not shown) such as a semiconductor memory card and a drive circuit for the recording medium, and to store image data, audio data, and the like handled by the image processing control unit 11 in the recording medium. This is an electronic circuit unit for recording and for reading out the same image data, audio data and the like recorded on the recording medium by the image processing control unit 11. Note that the recording unit 14 may have a form corresponding to a storage medium (not shown) such as a semiconductor memory card detachably mounted on the camera 1, or a semiconductor fixed on an internal electronic board of the camera 1. A configuration in which a memory or the like is used as a storage medium may be used.

  The operation determination unit 16 includes various switches linked to various operation members (not shown) of the camera 1, such as a release switch, a power on / off switch, an operation direction selection instruction switch, a menu switch, an operation mode switch, and the like. The electronic circuit unit receives various operation instruction signals generated from the touch panel 19 or the like and determines the content of an instruction operated by a user.

  The operation determination unit 16 generates an instruction signal corresponding to an operation of various operation members or the like by a user (user) and outputs the instruction signal to the image processing control unit 11. In response to this, the image processing control unit 11 executes various control processes corresponding to the input instruction signal.

  The attitude sensor 17a is an electronic circuit unit including, for example, an acceleration sensor and the like. This posture sensor 17a is provided for example in the posture state of the camera 1 (whether the imaging screen is in a vertical position or in a horizontal position, or the state of the screen tilting with respect to a horizontal line or a vertical line, etc.), and (of the imaging optical system of the camera 1). It is provided in order to determine the state of elevation of the optical axis, etc., as well as the state of shake of the camera 1 itself.

  The GPS 17b is an electronic circuit for receiving a signal from a GPS satellite in a Global Positioning System (GPS) and acquiring various kinds of predetermined information such as position information, time information, and azimuth information of the camera 1. Department.

  The azimuth sensor 17c is an electronic circuit unit for detecting the direction of the camera 1 by detecting, for example, terrestrial magnetism, specifically, the direction in which the optical lens of the imaging optical system (the imaging surface of the imaging device) is facing. A so-called electronic compass or the like is applied.

  The display unit 18 displays an image based on the image data generated by the camera 1 or the image data recorded in the recording unit 14, and displays various setting screens (menu display and the like) in the camera 1. A display panel, for example, a display unit such as a liquid crystal display (LCD) or an organic electro-luminescence (organic EL) display, and a constituent unit including a drive circuit and the like for the display device. is there.

  The touch panel 19 is a display unit that includes an operation input panel and the like arranged on the surface of the display screen of the display unit 18. In response to this, the display control unit 11a of the image processing control unit 11 performs control for displaying an operation display or the like on the display screen of the display unit 18. Then, upon receiving the operation instruction signal generated from the touch panel 19, the image processing control section 11 executes various controls. That is, the touch panel 19 functions as a designation operation unit that can designate any desired position on the display screen.

  In the camera 1 of the present embodiment configured as described above, the image processing control unit 11, the recording unit 14, and the display unit 18 perform various processes on the image signal output from the imaging unit 12, and Alternatively, image processing is performed on a moving image. That is, the image processing control unit 11 sequentially performs predetermined signal processing on the image signal from the imaging unit 12 to generate an image signal, and sequentially transmits the image signal to the display unit 18 via the display control unit 11a. Output. In response to this, the display unit 18 sequentially displays the corresponding images, whereby real-time moving image display, that is, live view display is performed.

  Further, the image processing control unit 11 performs, for example, a compression process on the processed image signal received from the imaging unit 12 and outputs the processed image signal to the recording unit 14. In response to this, the recording unit 14 records on a recording medium (not shown). Further, the image processing control unit 11 reads out the image data recorded on the recording medium of the recording unit 14 and performs a decoding process, and then outputs the data to the display unit 18 via the display control unit 11a. In response, the display unit 18 displays the corresponding image.

  In addition, in the configuration of the camera 1, in addition to the above, there are many configuration units and the like not shown in FIG. However, these constituent units and the like have substantially the same configuration as an imaging device (camera) that has been conventionally practically used and widely used, and a description thereof will be omitted.

  Next, an operation when an imaging operation is performed using the camera 1 according to the present embodiment configured as described above will be described with reference to FIGS. FIG. 2 is a conceptual diagram illustrating a situation when an imaging operation is performed using the camera 1 of the present embodiment. FIG. 3 is a display example of auxiliary data (floor view data) acquired under the imaging situation of FIG. FIG. 4 is a display example of the display unit when performing the imaging operation after the acquisition of the auxiliary data (drawing diagram data) in FIG. 3.

  First, an outline of the operation of the camera of the present embodiment will be described with reference to FIGS. FIG. 2 shows an example of a situation when an imaging operation is performed using the camera 1 of the present embodiment. The example illustrated in FIG. 2 illustrates a situation in which an imaging operation for recording a construction process of the imaging target structure is performed inside a structure or a building (hereinafter, simply referred to as an imaging target structure) to be an imaging target. Is shown. Specifically, in FIG. 2, a user (user) 100 carrying the camera 1 of the present embodiment is in a predetermined room (hereinafter, referred to as an imaging target area) 200 on a predetermined floor of an imaging target structure. This shows a situation in which the camera 1 is turned to perform an imaging operation.

  In this situation, the user (user) 100 first captures the entire image of the imaging target region 200 in which the capturing operation is to be performed as a two-dimensional image. Thereby, the camera 1 acquires the entire image of the imaging target area 200 as one two-dimensional image data. It is desirable that the two-dimensional image displayed by the two-dimensional image data acquired at this time covers as wide a range as possible. For this purpose, for example, a wide-angle lens is used as the imaging optical system. Further, when executing the imaging operation of the entire image, a display indicating that the zoom position of the imaging optical system is set to the wide angle side may be performed. Further, at the time of the imaging operation at this time, it is desirable to maintain the posture of the camera 1 at a predetermined posture. That is, by maintaining the posture of the camera 1 in a predetermined posture, the vertical side of the rectangular imaging screen frame (not shown; an image as an imaging result) coincides with the vertical direction, and the horizontal side substantially coincides with the horizontal direction. It is desirable to be. Thus, for example, a level display or the like may be displayed on the display unit 18 using the detection results of the attitude sensor 17a, the orientation sensor 17c, and the like. The user captures an image while maintaining the camera 1 so as to maintain the horizontal and vertical directions of the posture of the camera 1 while watching the level display and the like on the display unit 18.

  In the example of the imaging target area 200 shown in FIG. 2, when standing at the entrance of the room, the pillars R1, R2, and R3 are arranged at predetermined intervals along one (right side) wall surface, and the other. An indoor space in which the pillars L1, L2, and L3 are arranged at predetermined intervals along the (left side as viewed) wall surface is assumed. In this case, in the assumed example, the pillars R1 and R2 are arranged such that one surface thereof is in contact with the right side wall 201. It is assumed that the pillar R3 is disposed so that two surfaces thereof are in contact with the right side wall 201 and the rear side wall 202. The columns L1, L2, L3 are arranged with a distance D1 between one surface and the left side wall 203. Further, it is assumed that the column L3 is arranged with a distance D2 between the other surface different from the one surface and the inner side wall 202.

  In the camera 1 of the present embodiment, as described above, when the image data of the entire image of the imaging target area 200 is acquired, various signal processes are performed in the image processing control unit 11 based on the image data. . For example, the display control unit 11a performs display image processing corresponding to the acquired image data, and the processed display image data is sent to the display unit 18 and displayed on the display screen of the display unit 18. Display the image corresponding to.

  At the same time, the structure outline determining unit 1b executes image processing based on the image data, determines an outline portion of each structure such as a pillar in the image represented by the image data, and For example, a process of estimating the arrangement of the structures and generating sketch data of a two-dimensional plane, which is auxiliary data for assisting imaging, is performed.

  Here, the sketch data as auxiliary data is various information on the shape, position, and the like of various structures (pillars and the like) included in the imaging region 200 based on the entire image of the imaging target region 200. The sketch data includes, for example, information (a so-called bird's-eye sketch) in a form in which the image-capturing target area 200 is viewed in a bird's-eye view, as well as various types of detailed information on a plurality of imaging objects scheduled to be captured. . Note that the sketch data generated by the camera 1 of the present embodiment is simple because it is generated from only at least one image data of the entire image of the imaging target area 200.

  Further, the determination result of the structure outline determination unit 1b is sent to the blind spot prediction unit 11d. The blind spot prediction unit 11d is based on image data obtained by capturing the entire image, and includes imaging guide information including information to be added to the sketch data, for example, position information on the sketch data regarding the blind spot portion, blind spot range, and the like of each structure. And so on. Here, the imaging guide information is information for assisting the imaging operation. For example, individual imaging operations after the above-described entire image imaging operation, that is, individual structures in the imaging target area 200 are set as imaging targets. In the imaging operation described above, the information is used to guide necessary imaging to be able to be imaged without omission. Specifically, the imaging guide information is, for example, information for displaying, on the display unit 18, a part or range that is a blind spot from the current imaging position.

  When the sketch data is generated in this way, the sketch data is displayed on the display screen of the display unit 18 in a form as shown in FIG. 3 and simultaneously stored in a predetermined storage area such as the recording unit 14. Is done.

  FIG. 3 shows a mode in which a rectangular sketch image 18a generated based on the sketch data is displayed in a substantially central area of the display screen of the display unit 18. At this time, in the margin of the display screen of the display unit 18, a display 18 d indicating that the image being displayed is “facsimile data” and a return icon indicating that the display image of the display unit 18 is switched to return to the original state. 18b and the like are displayed.

  In the example of the sketch data image 18a illustrated in FIG. 3, for example, a rectangular region is displayed as a bird's-eye view sketch of the imaging target region 200 (room), and a plurality of structures (six pillars R1, R2, in this example) are provided therein. R3, L1, L2, L3) and an arrow icon 18c indicating the position and direction of the camera 1. Further, a part of each structure displays a blind spot display DA that indicates a part and a range that is expected to be a blind spot from the current position of the camera 1 with a thick line or colored form. In the example shown in FIG. 3, the blind spot display DA indicates the outline of the corresponding part and range of each structure by a thick line. The blind spot display DA displayed in this manner is imaging guide information at the time of an imaging operation for each structure.

  Note that the reference numerals shown in FIG. 3 are provided for explaining the display, and these reference numerals are displayed when the sketch data is actually displayed on the display screen of the display unit 18. (The same applies to FIG. 4 described later).

  Also, in FIG. 3, the wall behind the photographing is also displayed as being assisted, but this may be displayed in anticipation of the photographing result in front, and the result of a camera that photographs the back side may be displayed. It may be used, or may be inferred from the result of shooting a moving image. In addition, for the sake of easy explanation and understanding, the notation “drawing plan” is used, but any information display that can be referred to by the user at the time of photographing may be used, and the actual photographing result as shown in FIG. (Highlighted one) may output guide information in augmented reality, or each structure or its outline may be displayed as a line drawing, and the image may be replaced with an illustration-like image. In this case, since it is important that a plurality of measurement objects including a visible surface of the pillar and the presence or absence of the blind spot are displayed, the expression may be referred to as an object list display diagram separately from the floor plan.

  After the generation of the sketch data (or the object list display diagram), when the imaging operation of the individual structures is continuously performed inside the imaging target region 200, the display unit 18 of the camera 1 includes the information shown in FIG. Is displayed as shown in FIG. In the example shown in FIG. 4, a live view (LIVE VIEW) display area 18e for sequentially displaying images based on image signals continuously acquired by the imaging unit 12, a sketch display area 18f for displaying the sketch data, This shows a form in which the return icon 18b is displayed together with one display screen of the display unit 18. As described above, the sketch map display area 18f may be an object list display map, and need not be a “sketch map” in a strict sense. In other words, any information that can be referred to by the user at the time of shooting may be used, and guide information may be output in augmented reality in the actual shooting result (or in a highlighted form), or each structure or its outline may be drawn as a line. The image may be displayed in the style of an illustration, by replacing the image with something like an illustration.

  The user (user) 100 performs an imaging operation of an individual structure (specifically, an individual pillar or the like) while moving in the space of the imaging target area 200. In the case of the imaging operation, Referring to the sketch data, an image of a blind spot portion or the like that could not be obtained in the first entire image capturing operation is captured again. Here, when an image of an individual structure is captured, a display (check mark) indicating that the image has been captured may be attached on the drawing.

  The operation of the camera 1 according to the present embodiment configured as described above will be described below with reference to FIGS. FIGS. 5 and 6 are flowcharts showing a camera control processing sequence of the imaging apparatus (camera) according to the first embodiment of the present invention. FIG. 5 is a flowchart showing a processing sequence in an imaging mode, which is a main part of the camera control processing sequence. FIG. 6 is a flowchart which is a part of the camera control processing sequence and schematically shows a processing sequence other than the imaging mode. The gist of the present embodiment is mainly in the imaging mode. Therefore, the operation in the other operation modes is simplified as shown in FIG. 6 assuming that the same processing as that of the conventional camera is performed.

  FIG. 7 is a flowchart showing a detailed sequence of the blind spot prediction process (step S111) in the camera control process sequence of FIG. FIG. 8 is a conceptual diagram showing a simplified file structure of image data acquired when an imaging operation is performed using the imaging device (camera) of the present embodiment.

  The camera control processing sequence shown in FIGS. 5 and 6 will be described below. This camera control processing sequence also assumes the same imaging situation as described with reference to FIGS. Therefore, in the following description, FIGS. 2 to 4 may be referred to.

  First, it is assumed that the power state of the camera 1 is in the ON state and the camera 1 is in the activated state. In this case, in step S101 in FIG. 5, the control unit of the image processing control unit 11 checks whether the operation mode of the camera 1 is set to an imaging mode in which an imaging operation can be performed. Here, when it is confirmed that the imaging mode is set, the process proceeds to the next step S102. If the imaging mode has not been set, the process proceeds to step S121 in FIG. 6 (see reference numeral A in FIGS. 5 and 6).

  In step S102, the image processing control unit 11 controls the imaging unit 12 and the like to sequentially acquire image data (control unit), performs appropriate image processing on the acquired image data as appropriate (image signal processing unit), and displays the image data. The live view display process is started by controlling the unit 18 (control unit). Thereafter, the process proceeds to step S103.

  In step S103, the control unit of the image processing control unit 11 checks whether sketch drawing data as auxiliary data already exists. Here, the auxiliary data (facsimile data) is stored in, for example, a temporary storage unit or the recording unit 14 built in the camera 1. In this case, when it is confirmed that the auxiliary data (the sketch data) already exists, the process proceeds to step S104. If it is confirmed at this point that no auxiliary data (drawing data) exists, the process proceeds to step S105.

  In step S104, the control unit of the image processing control unit 11 controls the display unit 18 via the display control unit 11a to execute auxiliary data (floor view data) display processing. Thereafter, the process proceeds to step S107.

  Here, although the detailed description of the auxiliary data (floor view data) display processing is omitted, the following processing is performed.

  The case where the auxiliary data (drawing data) is already stored in the camera 1 is, for example, as described above, generated based on the image data acquired by the camera 1 and recorded in the recording unit 14 or the like. In other cases, auxiliary data (facsimile data) that has been acquired at another opportunity or generated by another device has been copied and recorded in the recording unit 14 or the like in advance.

  In the auxiliary data (drawing data) display processing, the recorded auxiliary data (drawing data) is read from the recording unit 14 or the like, and is displayed using the display unit 18 in the above-described display form as shown in FIG. Perform processing.

  In this case, there may be a plurality of auxiliary data (facsimile data) recorded in the recording unit 14 or the like. In such a case, for example, first, a list of auxiliary data (facsimile data) recorded in the recording unit 14 is displayed, and then a user (user) is allowed to select a desired data item from the list. The same display processing (display form in FIG. 4 and the like) may be performed on the selected auxiliary data (floor view data).

  If the process proceeds to step S105 without confirming the presence of the auxiliary data (drawing diagram data) in the above-described process of step S103, the control unit of the image processing control unit 11 Monitor the signal to check for any user operation. Here, the user operation refers to an operation of various operation members and the like by a user (user). When this user operation is performed, an instruction signal corresponding to the operation is generated. Therefore, when the generation of the instruction signal is confirmed, it is determined that a user operation has been performed, and the process proceeds to step S106. If the generation of the instruction signal is not confirmed, it is determined that the user operation has not been performed, and the process proceeds to step S107.

  In step S106, the control unit of the image processing control unit 11 executes an operation process according to the user operation generated in the process of step S105 described above. Thereafter, the process proceeds to step S107.

  Here, the operation process according to the user operation includes different operation processes for various operation members. For example, when the operation mode switching member is operated and a switching signal to another operation mode is generated, a predetermined operation mode switching process is performed. When the power button is operated to generate an off signal, a predetermined power off process is performed. When a release signal is generated by an imaging operation for starting an imaging operation, that is, a pressing operation of a release button or the like, a predetermined release process is executed. In addition, there are operation processes corresponding to various operation members, but detailed description is omitted.

  In step S107, the control unit of the image processing control unit 11 checks whether the user operation in step S105 is an imaging operation for starting an imaging operation. If the operation is an imaging operation, the process proceeds to step S108. If it is not an imaging operation, the process returns to the above-described step S101, and the same process is repeated thereafter.

  In step S108, the control unit of the image processing control unit 11 checks whether or not an operation has been performed on the auxiliary data (facsimile data). In this state, for example, when a predetermined touch operation or the like is performed via the touch panel 19 on a desired area on the sketch based on the sketch data displayed on the display unit 18, a predetermined area corresponding to the touch position is changed. After the AF processing, the AE processing, or the like for the imaging target is performed, a touch release signal is generated, and the release processing may be performed. Therefore, in step S108, the control unit of the image processing control unit 11 monitors an output signal of the touch panel 19. When the touch release signal is confirmed, the process proceeds to step S114. If the touch release signal has not been confirmed, the process proceeds to step S109.

  In step S109, the control unit of the image processing control unit 11 controls the imaging unit 12, the display unit 18, the recording unit 14, and the like to execute a normal imaging process. Thereafter, the process proceeds to step S110.

  In step S110, the control unit of the image processing control unit 11 checks whether or not the above-described imaging process in step S109 is an imaging process of the entire image. Here, in the case of the whole image capturing process, the process proceeds to step S111. If it is not the whole image capturing process, the process proceeds to step S115.

  In step S111, the control unit of the image processing control unit 11 executes a predetermined blind spot prediction process. The details of the blind spot prediction processing will be described later with reference to the subsequence of FIG.

  Next, in step S112, the control unit of the image processing control unit 11 confirms whether to record data on the sketch displayed on the display unit 18 based on the generated sketch data. Here, when the user (user) wants to record the sketch data, for example, the user presses an OK button of the operation members. In the case of retrying without recording, for example, another operation member such as a cancel button for explicitly instructing a non-recording operation is operated. Alternatively, it is also possible to instruct to cancel by waiting for a predetermined time to elapse without performing any operation.

  For that purpose, the control unit monitors the operation determination unit 16. In this case, if the instruction signal of the OK button is confirmed, the process proceeds to step S113. When an instruction signal other than the OK button is confirmed, or when no operation is performed for a predetermined time, the process returns to the above-described step S101 and the subsequent processes are repeated.

  In step S113, the control unit of the image processing control unit 11 executes auxiliary data (drawing diagram data) creation processing. Note that the details of the auxiliary data (drawing diagram data) creation processing will be omitted. Thereafter, the process returns to the above-described step S101, and the subsequent processes are repeated.

  On the other hand, in the process of step S108, the touch release signal is confirmed, and the process proceeds to the process of step S114. In step S114, the control unit of the image processing control unit 11 performs an imaging process on each structure. After that, a process of displaying information indicating that the part has been imaged at a corresponding position on the sketch data is executed. Thereafter, the process proceeds to step S115.

  In step S115, the control unit of the image processing control unit 11 executes an image recording process of the imaging result obtained in the process of step S114. In the image recording process, a process of adding various types of related information to the acquired image data and associating the information is also executed at the same time. Here, the various types of related information include, in addition to general imaging information, information related to the auxiliary data (including sketch drawing data and image data of the entire image). As an example of the data structure of the image data generated in the image recording process, for example, a form as shown in FIG.

  As shown in FIG. 8, the image data file includes file number (No. xxx) information, image data itself, various types of image information (imaging information such as date and time information, position information, and imaging conditions), and image thumbnails. , A thumbnail of the entire image, and information such as a sketch thumbnail. Note that the date and time information is information on the date and time at the time of imaging acquired from an internal clock or the like of the control unit of the camera 1. The position information is, for example, information relating to the position of the camera 1 on the earth at the time of image capturing obtained from the GPS 17b, specifically, for example, latitude and longitude information. The imaging information such as the imaging conditions include the lens name of the imaging optical system, focal length, aperture value, white balance, ISO sensitivity, color space, operation mode, AE mode, AF mode, scene type, serial number, drive mode, It is various kinds of information including resolution, processing application, and the like, and corresponds to so-called Exif information.

  On the other hand, if the process proceeds to the process of step S121 in FIG. 6 without confirming the setting of the imaging mode in the process of step S101 (reference A in FIGS. 5 and 6), the image processing is performed in step S121 in FIG. The control unit of the control unit 11 checks whether the currently set operation mode is set to a reproduction mode in which a reproduction operation can be performed. Here, when it is confirmed that the reproduction mode is set, the process proceeds to the next step S122. If the reproduction mode has not been set, the process proceeds to step S124.

  In step S122, the image processing control unit 11 controls the recording unit 14, the display unit 18, and the like to execute a predetermined image reproduction process. The details of the image reproduction processing are the same as those performed in the conventional camera 1, and the description thereof is omitted. Thereafter, the process proceeds to the process in step S123.

  In step S123, the control unit of the image processing control unit 11 monitors the operation determination unit 16 and checks whether or not an operation for changing the playback image has been performed. Here, when a change operation is performed by a predetermined operation member to instruct a change of a reproduced image, a predetermined change instruction signal is generated. When this change instruction signal is confirmed, the flow proceeds to the processing in step S125. If the change instruction signal is not confirmed, the process returns to step S101 in FIG. 5, and the subsequent processes are repeated (return).

  In the process of step S123, the generation of the change instruction signal is confirmed, and the process proceeds to step S125. In this step S125, the control unit of the image processing control unit 11 executes a predetermined image change process. Thereafter, the process returns to the process of step S101 in FIG. 5, and the subsequent processes are repeated (return).

  In the process of step S121, it is confirmed that the reproduction mode is not set (for example, the image communication mode is set), and the process proceeds to step S124. The control unit of the unit 11 executes a predetermined image communication process. Thereafter, the process returns to the process of step S101 in FIG. 5, and the subsequent processes are repeated (return).

  Next, details of the blind spot prediction processing (the processing of step S111 in FIG. 5) will be described below with reference to FIG. The surface of the rectangular parallelepiped constituting the structure is exposed in one of the imaging recommendation regions of the structure, and is a portion where imaging is possible. Such an exposed surface is important because it is easy to monitor changes over time and becomes an important evaluation item later, but it may be a blind spot in one shot, and such a blind spot is judged and guided. It will be very beneficial for users.

  As described above, when the process shifts to the blind spot prediction processing sequence in FIG. 7, first, in step S201 in FIG. 7, the control unit of the image processing control unit 11 refers to the detection results of the attitude sensor 17a, the direction sensor 17c, and the like. Then, it is checked whether or not the posture of the camera 1 is held horizontally. Here, if it is confirmed that the posture of the camera 1 is held horizontally (the posture is OK), the process proceeds to step S203. If the posture of the camera 1 is not held horizontally (the posture is not OK), the process proceeds to step S202. Here, the emphasis on horizontality is given to the fact that the pillars that support it greatly affect the quality of the building, and the pillars stand vertically from the floor to support the building against gravity. Even if the image is not inclined and the horizontal or gravity direction is known, even if the image is oblique, the pillar or floor can be determined based on the image. When a guide is provided for a structure other than a pillar, an arch or a beam may be determined based on a wall or the like, but in such a case, it is not always necessary to keep the horizontal.

  In step S202, the control unit of the image processing control unit 11 controls the display unit 18 and the like by the display control unit 11a and the guide display control unit 11c, and displays an advice display indicating that the posture of the camera 1 should be held horizontally. The posture advice processing to be performed is executed. Thereafter, the process returns to the process of step S201, and the subsequent processes are repeated until the posture of the camera 1 is held horizontally (posture OK).

  In step S203, the control unit of the image processing control unit 11 performs image signal processing on the image data obtained by capturing the entire image by using the structure outline determination unit 11b and the like. In the image signal processing in this case, for example, a horizontal line (a line indicating a boundary between a floor and a ceiling; see reference numerals 204 and 205 in FIG. 2) is determined in the image. Check whether a vertical line exists at the right end. Here, the vertical lines at the left and right ends are assumed to be, for example, the vertical lines of the pillars located at the innermost position in the example shown in FIG. That is, by checking the vertical line at one of the left and right ends and the horizontal line (204, 205) between the floor and the ceiling, it is determined whether or not an imaging area appropriate for capturing the entire image is covered. Check. Specifically, for example, when the boundary horizontal line (204, 205) between the floor and the ceiling is included and only one of the vertical lines at the left and right ends is determined, it is determined that the entire image cannot be captured. You. In this case, since the angle of view is insufficient or the distance is short, a wider area can be imaged by setting the zoom setting to the wide side or lowering the user (user) having the camera 1. Therefore, a process for providing such advice is provided in the next process of step S204.

  That is, in step S204, the control unit of the image processing control unit 11 controls the display unit 18 and the like by the display control unit 11a and the guide display control unit 11c, and executes the advice process regarding the angle of view and the distance. After that, the process returns to step S203.

  In step S205, the control unit of the image processing control unit 11 uses the structure outline determination unit 11b and the like to confirm whether or not there is an area in which three vertical lines are present in the image data of the entire image in proximity to each other. Do. Here, the case where three vertical lines are close to each other is assumed to be, for example, a case where a rectangular parallelepiped pillar is viewed from an oblique direction as shown in FIG. In this case, when three vertical lines are close to each other, the process proceeds to the next step S207. If the three vertical lines do not exist close to each other, the process proceeds to step S206. Further, in this case, for example, when a rectangular parallelepiped pillar is viewed from the front, only two vertical lines are visible. In such a case, for example, if the position of the camera 1 is moved right and left, there is a possibility that three vertical lines appear to be close to each other. Therefore, in the processing of the next step S206, processing for giving advice to that effect is provided.

  In step S206, the control unit of the image processing control unit 11 controls the display unit 18 and the like by the display control unit 11a and the guide display control unit 11c, and executes the advice process regarding the imaging position in the left-right direction. After that, the process returns to step S205.

  In step S207, the control unit of the image processing control unit 11 receives the determination result of the structure outline determination unit 11b or the like in the process of step S205, and detects a rectangular solid in the image of the entire image. I do. The rectangular parallelepiped detection process is a process of detecting a column having a rectangular parallelepiped shape, and is executed by the rectangular parallelepiped prediction unit 11e included in the blind spot prediction unit 11d.

  Next, in step S208, the control unit of the image processing control unit 11 estimates the arrangement of each rectangular parallelepiped detected according to the position and distance of the whole image in the image, and makes a sketch of the whole image, that is, the imaging target area 200 ( A bird's-eye view sketch is generated based on the whole image of the room. Thereafter, the process proceeds to step S209. In this case, “floor sketch” is written, but any image that can be expressed so that the user does not forget to take a picture may be a live-action image or an image expressed as an illustration.

  In step S209, the control unit of the image processing control unit 11 determines an intersection where the three contours (three vertical lines) detected in the above-described processing are gathered at one point at an acute angle, for example, reference numerals L1A and L2A in FIG. , L3A, R1A, R2A, R3A and the like are checked. If there is an intersection where three contours are gathered at one point at an acute angle, the process proceeds to the next step S210. If there is no intersection point where the three contours are gathered at one point at an acute angle, the series of processing is terminated, and the process returns to the original sequence (return).

  In step S210, the control unit of the image processing control unit 11 pays attention to the rectangular parallelepiped (column portion) detected in the above-described processing, and determines the contours of three vertical lines among the contours forming the rectangular parallelepiped (see FIG. 2). Extension lines (OR3, OL3, OL4 in FIG. 2) extending two outlines (see OR2, OL2 in FIG. 2) except for the outlines (OR1, OL1 in FIG. 2) and a vertical outline (OL1, OR1 in FIG. 2) and It is checked whether or not any of the floor contours (205 in FIG. 2) intersect. Here, if the above conditions are confirmed, the process proceeds to step S211. In other cases, the process proceeds to step S212.

  In step S211, the control unit of the image processing control unit 11 determines that the floor contour 205 intersects with the extension line (reference sign OR3, OL3, OL4 in FIG. 2) of the two contours (see reference signs OR2, OL2 in FIG. 2). The back side of the surface (reference characters AL2, AR2 in FIG. 2) including the line (reference characters OL2B, OR2B in FIG. 2) is estimated and set as the blind spot. Thereafter, a series of processing is completed, and the process returns to the original sequence (return). The blind spot area does not need to be determined based on a contour line that abuts on a part other than the floor or the ceiling. If a plane including a line where a contour 205 that abuts on a wall surface intersects is used, an arch-shaped structure may be used. It is possible to guess and set a non-photographed side such as a beam or a beam (in this case, not necessarily a strictly rectangular parallelepiped) as a blind spot. Such applications are, of course, covered by the present invention.

  On the other hand, in step S212, the control unit of the image processing control unit 11 sets the floor surface of the extension (OR3, OL3, OL4 in FIG. 2) extending from the two contours (OR2, OL2 in FIG. 2). When there is a line (OR3, OL4 in FIG. 2) that intersects the contour 205, the surface of the column facing the wall surface (201, 203) including the intersecting floor contour 205 is estimated and set as a blind spot. Thereafter, a series of processing is completed, and the process returns to the original sequence (return). In addition, although blind spots are too narrow, there is no space to hold the camera and there are blind spots where shooting is not possible, but displaying this as a guide will be stressful to the user, so the outline of the image It may be determined from the distance or the like whether or not shooting is possible, and if shooting is not possible, a guide display may be devised so as not to be displayed. The lengths of the distances D1 and D2 shown in FIG. 2 can be inferred from the extension of the contour. These numerical values can be measured from the focal length at the time of shooting, the distance information at the time of focusing the object, and the like. Here, the method of determining the blind spot is specially described because the blind spot area is an area that has not been shot in the first shooting (FIG. 2 and the like) and, for example, has a shooting result (image) as illustrated in FIG. 2. This is because if a blind spot area exists, it cannot be guaranteed at all whether or not the entire structure has a correct construction or painting surface. In other words, if there is a construction error in the blind spot area, for example, it is not possible to confirm the error only with this entire image. May not be possible. For example, depending on the construction process, the wall may be blocked (hidden) due to interior construction, for example, the application of wallpaper to the wall, and it will be determined later whether the correct construction work (construction) has been performed. Cannot be verified in some cases, and in some cases, work such as redoing work (such as removing the interior) may occur. Therefore, waste occurs in cost and construction period. For such a reason, the imaging recommended area (blind spot area) of the structure includes a surface of a photographable structure that is a blind spot and is not photographed in the entire image data obtained by receiving the optical image of the structure. That is important. It is important to give advice as imaging assistance from the viewpoint that it is easily overlooked because of the blind spot area.

  As described above, according to the first embodiment, when image data of the entire image of the imaging target is acquired, sketch data as auxiliary data can be generated based on the image data of the entire image. It is possible to generate imaging guide information for guiding imaging by presenting an unimaged portion, an imaging direction, and the like, such as predicting and displaying a blind spot or the like that is not obtained from the image data of the entire image.

  When arranging a series of image data acquired in large quantities later, it was difficult to specify an imaging area or an imaging point only with the image data. By taking an image, a simple sketch can be created based on the image data of the entire image, and for the structure related to the sketch data, individual image data to be imaged thereafter are automatically converted to the above-described image data. Since it is associated with the sketch data, it can be easily specified, for example, at a later time when organizing images. Therefore, reliable and accurate image organization can be performed. Note that, here, the guide information is displayed after being displayed on the sketch, but the guide information may be provided to the user in an expression method in which the blind spot information is displayed as augmented reality in the actual shooting result.

[Second embodiment]
By the way, in recent years, a small unmanned aerial vehicle (a so-called drone or the like) capable of performing radio control or autonomous flight is mounted with an imaging device to fly and easily capture a moving image or a still image from the sky. Is available. By using such technology, even in an area where humans cannot easily enter, such as a site of a natural disaster or accident, it is possible to easily grasp the entire situation by taking an image from the sky. It is possible to check the individual partial situation by observing the image of the acquired whole image, and if necessary, to perform more detailed imaging of the partial situation, so that the detailed situation can be confirmed. ing.

  Even in such a case, it is convenient if simple sketch data can be generated by, for example, capturing image data of the entire image. The second embodiment of the present invention described below is an example in which such a situation is assumed.

  FIG. 9 is a conceptual diagram illustrating a situation when an imaging operation is performed using the imaging device (camera) according to the second embodiment of the present invention. FIG. 10 is a display example of auxiliary data (facsimile data) acquired under the imaging situation of FIG.

  In the example illustrated in FIG. 8, for example, the small unmanned aerial vehicle 21 equipped with the imaging device 1 is flown over an imaging target area including the building 300 and the like (specifically, for example, a disaster site or an accident site). 2 shows a situation in which the imaging device 1 is remotely operated to perform imaging.

  In this case, first, by remotely operating the imaging apparatus 1, an entire image of a predetermined range including the building 300 in the imaging target area is acquired as one image data. At this time, the attitude of the camera 1 can be maintained to a certain degree by the autonomous flight control of the small unmanned aerial vehicle 21 side. Note that even if the attitude control of the camera 1 cannot be completely performed, horizontal and vertical directions are defined in the acquired image data with reference to the detection results of the attitude sensor 17a, the azimuth sensor 17c, etc., based on the straight line in the image. You may make it. In the example shown in FIG. 9, the building 300 has a structure in which a plurality of substantially cubic shapes 301 and 302 are combined.

  When the image data of the entire image of the imaging target area is obtained in this manner, the image processing control unit 11 of the camera 1 executes various signal processes based on the image data, and, for example, executes the in-image Then, the outline portion of the building 300 is determined, a rectangular parallelepiped detection process is performed, and a process of generating sketch data as auxiliary data is executed. The sketch data in this case is, for example, data of a bird's-eye sketch drawing indicating a schematic arrangement of the building in the imaging target area. Further, the blind spot prediction unit 11d generates information to be added to the sketch data, that is, for example, imaging guide information relating to a blind spot portion, a blind spot range, and the like of each building. The sketch data generated in this manner is displayed on the display screen of the display unit 18 in the form shown in FIG. 10 and is stored in a predetermined storage area such as the recording unit 14 at the same time.

  FIG. 10 shows a mode in which a sketch image 18a generated based on the sketch data is displayed using the entire display area of the display screen of the display unit 18. At this time, at one corner of the display screen of the display unit 18, for example, a display 18 d indicating that the image being displayed is “drawing plan data”, a return icon 18 b, and a position of the camera 1 when the whole image is captured And an arrow icon 18c indicating the direction.

  In FIG. 10, a part or a part of the building 300 which becomes a blind spot from the position of the camera 1 when the entire image is captured is displayed by a thick line or a unit for coloring. , DA. This part DA is referred to as a blind spot display. The blind spot display DA displayed in this manner serves as imaging guide information when performing detailed imaging of the building 300.

  Also, in FIG. 10, each reference numeral is used to explain the display, and these reference numerals are displayed when the sketch data is actually displayed on the display screen of the display unit 18. This is not the same as in FIGS. 3 and 4 in the first embodiment described above. The other display systems and the details of the operation are substantially the same as those of the first embodiment.

  The main control processing sequence of the operation of the camera 1 of the present embodiment configured as described above is substantially the same as that of the above-described first embodiment. In the present embodiment, the processing sequence of the blind spot prediction processing is slightly different. Therefore, only the part of the operation of the camera according to the present embodiment that is different from that of the camera according to the above-described first embodiment will be described below. FIG. 11 is a flowchart illustrating details of a blind spot prediction process (a process corresponding to the process of step S111 in FIG. 5) in the operation of the imaging apparatus (camera) according to the second embodiment of the present invention.

  The processing in steps S201 and S202 is the same as in the first embodiment (see FIG. 7). In the present embodiment, the attitude advice processing in step S202 may be an advice display for the operator of the small unmanned aerial vehicle 21 or, if the small unmanned aerial vehicle 21 is under autonomous control, the small unmanned aerial vehicle 21 Advice control to the control unit of the flying object 21 may be performed.

  In step S203A, the control unit of the image processing control unit 11 executes image signal processing on image data obtained by capturing the entire image by using the structure outline determination unit 11b and the like. The image signal processing in this case determines, for example, a horizontal line (a line indicating a rooftop boundary; see FIG. 9) in the image, and determines whether a vertical line exists at the left and right ends of the determined horizontal line. Confirm whether or not. Here, the vertical lines at the left and right ends are assumed to be vertical contour lines at both left and right ends of the building 300 in the example shown in FIG. 9, for example. That is, by checking the vertical lines at the left and right ends and the horizontal line on the rooftop boundary, the rooftop area of the target building 300 is determined. Here, if the roof determination is not made, the process proceeds to step S204A. On the other hand, when a rooftop determination is made, the process proceeds to step S305.

  In step S204A, the control unit of the image processing control unit 11 controls the display unit 18 and the like by the display control unit 11a and the guide display control unit 11c, and executes an advice process regarding an angle of view, a distance, an altitude, and the like. After that, the process returns to step S203A.

  In step S305, the control unit of the image processing control unit 11 checks whether or not the roof outline is completed. Completion of the rooftop contour is a confirmation as to whether or not the rooftop portions of the plurality of rectangular parallelepiped shapes 301 and 302 have been detected in the building 300, for example. Here, if the roof outline is completed, the process proceeds to the next step S307. If the roof contour is not completed, the process returns to step S203A, and the subsequent processes are repeated.

  In step S307, the control unit of the image processing control unit 11 performs the rectangular parallelepiped detection process in the image of the whole image as a result of the determination by the structure outline determination unit 11b or the like in the process of step S203A described above. In the rectangular parallelepiped detection processing in this case, for example, processing that further takes into account color uniformity is executed.

  Subsequently, in step S308, the control unit of the image processing control unit 11 estimates the arrangement of each of the rectangular parallelepipeds 301 and 302 detected according to the position and distance of the whole image in the image, and makes a sketch of the whole image, that is, captures the image. A bird's-eye view sketch is generated based on the entire image of the target area. Thereafter, the process proceeds to step S309. Note that the processing in step S308 described above is the same as the processing in step S208 in FIG. 7 in the first embodiment.

  Subsequently, in step S309, the control unit of the image processing control unit 11 performs two contours of the plurality of contours constituting the roof contour detected in the above-described processing and one vertical line of the rectangular parallelepiped. It is confirmed whether or not the three contours (refer to the symbol OS in FIG. 9) are connected at one point at an obtuse angle (whether there is an intersection that intersects; refer to the symbol S1 in FIG. 9). Here, when the intersection (reference numeral S1) exists, the process proceeds to step S310. If the intersection does not exist, the series of processing ends, and the process returns to the original sequence (return).

  In step S310, the control unit of the image processing control unit 11 determines that two surfaces including the diagonal of the vertical line (see reference symbol S2 in FIG. 9) among the three contours (see reference symbol OS in FIG. 9) are blind spot surfaces. I do. Thereafter, a series of processing is completed, and the process returns to the original sequence (return). Other operations are the same as those of the first embodiment.

  As described above, even in the second embodiment, even in the above-described situation, that is, in a case where a wide-area imaging region using a small unmanned aerial vehicle such as a drone is targeted, the above-described second embodiment is used. Almost the same effects as those of the first embodiment can be obtained.

  Each processing sequence described in each of the above-described embodiments can allow a change in procedure as long as it does not violate its nature. Therefore, for example, the order of each processing step may be different from the above-described processing sequence each time the execution order of each processing step is changed, a plurality of processing steps are simultaneously executed, or a series of processing sequences are executed. It may be. That is, even if the operation flow in the claims, the description, and the drawings is described using “first”, “next”, and the like for convenience, it is essential that the operation be performed in this order. It does not mean. In addition, it is needless to say that the steps constituting these operation flows can be omitted as appropriate for portions that do not affect the essence of the invention.

  Of the techniques described here, many of the controls and functions mainly described in the flowcharts can be set by software programs in many cases, and the software programs described above are read and executed by a computer. Controls and functions can be realized. The software program may be used as a computer program product in advance in the product manufacturing process, such as the storage medium and the storage unit, specifically, for example, a portable medium such as a non-volatile memory such as a flexible disk, a CD-ROM, a hard disk, and a volatile medium. Electronic data stored or recorded in whole or in part in a storage medium such as a volatile memory. Apart from this, it can be distributed or provided at the time of product shipment or via a portable medium or a communication line. Even after the product is shipped, the user can operate the software program by downloading the software program and installing it on a computer via a communication network or the Internet, or installing the software program on a computer from a storage medium. Accordingly, the imaging apparatus according to the present embodiment can be easily realized.

  Note that the present invention is not limited to the above-described embodiment, and it is needless to say that various modifications and applications can be made without departing from the gist of the invention. Further, the embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some components are deleted from all the components shown in the one embodiment, if the problem to be solved by the invention can be solved and the effect of the invention can be obtained, this configuration is deleted. The configuration can be extracted as an invention. Further, components of different embodiments may be appropriately combined.

  The present invention is not limited to an imaging device specialized for an imaging function of a digital camera or the like, but is applicable to other types of electronic devices having an imaging and observation function, such as a movie camera, a mobile phone, a smartphone, an electronic organizer, and an electronic device. It can be widely applied to various electronic devices with voice recording / playback functions, such as dictionaries, portable information terminals, personal computers, tablet terminals, game devices, televisions, clocks, navigation devices using GPS (Global Positioning System), etc. it can. Industrial equipment or the like often inspects a structure similar to a building, but the guide display of the present application may be applied to such an inspection device.

1 ... Camera,
11 image processing control unit,
11a ... display control unit,
11b ... structure outline determination unit,
11c: guide display control unit,
11d: blind spot prediction unit,
11e: rectangular parallelepiped predictor,
11f... Imaging surface determination unit,
12 ... Imaging unit,
13 Communication unit,
14 ... Recording unit,
16 ... operation determination unit,
17a posture sensor,
17b ... GPS,
17c ... direction sensor,
18 Display unit,
19 ... Touch panel,
21 ... Small unmanned aerial vehicle,

Claims (5)

An imaging unit that receives the optical image of the structure and generates image data,
A display unit that displays an image based on the image data generated by the imaging unit;
A contour determining unit for determining a contour of the structure;
The intersection of a plurality of contours of the outline of the structure is detected to predict the shape of the structure, and the imaging recommended area of the structure is determined based on the intersection of the detected contour and the predicted shape of the structure. identify the imaging operation of the imaging target individual structures in the imaging recommended area, the area specifying unit which generates a shooting guide information for guiding so as imaged without omission imaging in need,
A display control unit for displaying the shooting guide information on the display unit,
Has,
The area identification unit is a blind spot, which is a surface of a photographable structure that is a blind spot that is not photographed in image data obtained by receiving an optical image of the structure, in the imaging recommendation area of the structure. region, imaging device profile resulting in contact with the floor or ceiling or wall those is characterized that you guess sets the back side of the plane containing the line of intersection as a blind spot section.   The imaging apparatus according to claim 1, wherein the outline determination unit predicts and determines that the structure is a rectangular parallelepiped. The photographing guide information imaging apparatus according to claim 1, characterized in that the said imaging recommended region specified by the region specifying unit. Receiving an optical image of the structure and generating image data;
Displaying an image based on the generated image data;
Determining the contour of the structure;
A detection / prediction step of detecting intersections of a plurality of contour lines of the contour of the structure to predict the shape of the structure,
The imaging recommendation area of the structure is specified based on the intersection of the contour detected in the detection / prediction step and the predicted shape of the structure. The back side of a plane including a line where a contour formed by contacting a floor, a ceiling, or a wall surface with a blind spot area, which is a face of a photographable structure that becomes a blind spot and is not photographed in image data obtained by receiving an image, intersects Is assumed as a blind spot and is set, and in an imaging operation in which an individual structure in the above-mentioned imaging recommended area is to be imaged, an area identification for generating imaging guide information that guides the required imaging so that it can be imaged without omissions Steps and
And the step of displaying the shooting guide information,
An imaging method comprising: Receiving an optical image of the structure to generate image data,
Displaying an image based on the generated image data,
Determining the contour of the structure,
Detecting the intersection of a plurality of contour lines of the contour of the structure to predict the shape of the structure,
An imaging recommended area of the structure is specified based on the intersection of the detected contour and the predicted shape of the structure, and the imaging recommended area of the specified structure is received by receiving an optical image of the structure. A blind spot area, which is a surface of a photographable structure that is not a blind spot in the captured image data and is a blind spot, is estimated as a blind spot on a rear side of a plane including a line where a contour generated by contacting a floor, a ceiling, or a wall surface intersects. In the imaging operation with the individual structures in the imaging recommended area as an imaging target, imaging guide information that guides the required imaging to be able to be imaged without omission is generated,
Imaging program for executing an imaging method for displaying a photographic guide information to the computer.

Images