JP2018029239A - Portable electronic device and program for taking/managing photo - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a portable electronic device capable of preventing the occurrence of unfinished imaging due to an artificial error in a fixed-point inspection and the like.SOLUTION: A portable electronic device includes: a touch screen; a cameta; image storage means; a photograph management DB which associates images with image management information including a photographing date/place and manages the same; image list display means which produces a photographic image list in which images of each photographing date at the same photographing place are vertically aligned in image sets of preliminary/post images, displays the list on the touch screen, and displays non-photographing indication image in stead of the post image if there is no post image; underlying inspection scenery screen display means which superimposes and displays an underlying image generated from the preliminary image on an electronic view finder and displays the same on the touch screen, if a non-photographing indication image in the photographic image list is selected; and post image taking-in means which stores the post image in the image storage means and associates the same with the image management information including the photographing date/place of the post image, if a shutter is selected.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a portable electronic device technology for taking and managing photographs, and in particular, a portable electronic device technology used for taking and managing photographs before and after the same spot in a fixed point temporal change survey or the like. About.

  As a portable electronic device used for taking pre- and post-photographs of the same spot, those described in Patent Documents 1-5 are known. In the portable electronic device described in Patent Documents 1-5, in the case where the same spot is shot a plurality of times, the previously taken image is displayed as an underlay image on the electronic viewfinder image so that the shooting angle is the previous time. It is configured so that it can be determined whether or not it is coincident with the shooting.

  In the electronic camera described in Patent Document 1, a captured image and information indicating that the captured image is in the fixed point shooting mode are stored in a storage unit (flash memory) in the camera in association with each other. The photographed image in the fixed point photographing mode is displayed on the display means (liquid crystal display device) to prompt the user to select it, and the selected photographed image is used as a composition image (underlay image), and the subject image (electronic viewfinder image) during standby for photographing. And the selected composition image are superimposed on the display means. In this case, as the composition image, an outline extraction image, a monochromatic image, a light color image, or the like of the original photographed image is used. Thereby, it is possible to shoot at the same shooting angle as the previous shooting by determining the shooting angle so that the subject image matches the composition image.

  Similarly, the imaging apparatus described in Patent Document 2 also superimposes an electronic viewfinder image and a template image selected from a previously captured image by performing a light color process when the “watermark shooting mode” is set. It is configured to display. The same applies to the imaging devices described in Patent Documents 3 and 4.

JP 2002-94854 A JP 2016-46676 A US Patent Publication No. 2005/0264668 US Patent Publication No. 2009/0015702 JP 2008-199411 A

Yoshifumi Nishida, Nobuo Nakajima, Seiko Hattori, “Study on High-Accuracy Absolute Direction Acquisition Method for Autonomous Navigation for Pedestrians”, IEICE 2010 Society Conference Proceedings, The Institute of Electronics, Information and Communication Engineers, 2010 August 31, 2012, B-20-5. Munetoshi Suganuma, Kenichiro Kurata, Shinya Toyama, Takeshi Horie, “Autonomous Location Detection Method Applying Human Walking Motion Recognition Technology”, IEICE Transactions, Fundamentals / Boundaries, The Institute of Electronics, Information and Communication Engineers, January 1, 2004, J87-A (1), pp.78-86. Masakatsu Oki, Takeshi Kurata, “Indoor / Outdoor Pedestrian Navigation with Built-in GPS / Self-contained Sensor System (General Session (3), Human Detection / Measurement / Recognition)”, IEICE Technical Report. PRMU, Pattern recognition and media understanding, The Institute of Electronics, Information and Communication Engineers, May 19, 2006, Vol.106, No.73, pp.7580. Tatsuya Furudate, Fuyumi Toyose, Miyoshi Horikawa, Mitsumasa Sugawara, “Proposal of Location Measurement Method Using Received Signal Strength”, Information Processing Society of Japan 76th Annual Conference, Information Processing Society of Japan, Vol.76, No.3, pp.3.207-3.208. Tomomi Sato, Satoshi Komiyama, Masahiko Shimoda, Liu Sabae, Kazumasa Yokota, “Examination of location estimation using Bluetooth radio field strength”, DEIM Forum 2011, B9-4, 2011, <URL: http: // db -event.jpn.org/deim2011/proceedings/pdf/b9-4.pdf>. Shigematsu Fumiya, Kawamura Yusuke, Shimamura Kazunori, “A Proposal of Location Information Integration Method Using RFID and GPS”, IEICE Technical Report (ISSN: 0913-5685), The Institute of Electronics, Information and Communication Engineers, vol. 109, no.382, pp.51-55. Masakatsu Oki and Takeshi Kurata, “Personal Positioning Method Based on Walking Motion Analysis Using Inertial Sensors and Wearable Cameras”, IEICE Technical Report. PRMU, Pattern Recognition / Media Understanding, The Institute of Electronics, Information and Communication Engineers , March 11, 2004, vol.103, no.737, pp.25-30. Hironobu Fujiyoshi, “The Institute of Electronics, Information and Communication Engineers, Knowledge Base, Knowledge Forest, 2 Groups, 2 Volumes, Chapter 4 Video Analysis”, [online], September 2010, The Institute of Electronics, Information and Communication Engineers, [August 2016 10 days search], Internet, <URL: http://www.ieice-hbkb.org/portal/doc_208.html>

  It is necessary to periodically photograph the same point over a long period of time and to investigate and record changes over time such as cracks in inspections of industrial damage and crack fixed points in dams and tunnels. At this time, since it is necessary to make the shooting angle the same when shooting the same spot, it is effective to use a technique for superimposing and displaying as an underlay image on the electronic viewfinder image as mentioned in the background art.

  On the other hand, in the construction inspection and crack fixed point survey, the number of points to be photographed is generally very large, and it is necessary to photograph many points without leaving them. However, in the prior art, thumbnails of captured images (previous images) that will be the base image are displayed in a list on the display, and the base image is selected from these images for shooting. Since it is necessary to manage separately whether or not the post-shooting of the point has been completed, there is a problem in that it is easy to leave behind a shot due to human error. In addition, there is a problem that the management work is very complicated and the work effort is large. For example, it is necessary to separately manage the photographed data stored in the film or the storage device in association with the investigation / shooting point.

  Therefore, the object of the present invention is to effectively prevent the occurrence of leftovers due to human error when taking pre- and post-photographs of the same point in inspections of construction damage and fixed point aging. Is to provide a portable electronic device for photography and management.

  Another object of the present invention is to provide a portable electronic device for taking and managing photographs, which is easy to record and manage photographing points when managing pre- and subsequent photographs of the same point. .

  Another object of the present invention is to provide a portable electronic device for photography and management that makes it easier to align the shooting angle than before when managing pre- and post-photographs at the same point. There is to do.

A first configuration of a portable electronic device according to the present invention is a portable electronic device for photography / management that includes a touch screen device, a camera device, and an internal clock,
Image storage means for storing an image photographed by the camera device as an image file;
A photo management database for managing images stored in the image storage means in association with image management information including shooting date / time information of the image and shooting location information;
Read each image management information registered in the photo management database,
Images taken at the same shooting location at each shooting date and time are arranged in the order of a prior image whose shooting date and time is the previous image and a subsequent image whose shooting date and time is the later image. Create a list and display the captured image list on the screen of the touch screen device,
In the photographed image list, when the posterior image is not stored in the image storage means, an image that displays an unphotographed instruction image indicating that it has not been photographed instead of the posterior image. List display means;
In the photographed image list, when the unphotographed instructional image in the photographed image list is selected by the touch screen device,
A finder superimposed image is generated by superimposing an underlying image generated from the previous image of the image set to which the shooting button image belongs on an electronic viewfinder image captured in real time from the camera device,
Underlay finder screen display means for displaying an underlay finder screen including the finder superimposed image and a shutter button image indicating a tap operation position for inputting an imaging execution instruction on the screen of the touch screen device;
When the shutter button image is selected by the touch screen device on the underlay finder screen, the electronic viewfinder image at the time of selection is stored in the image storage means as the posterior image, and from the internal clock Post-registration of the posterior image in the photo management database in association with image management information including shooting date / time information of the posterior image to be captured and shooting location information of the pre-image used as the underlying image of the finder superimposed image An image capturing means,
The image list display means, when the posterior image is newly registered in the photo management database by the posterior image capturing means, the touched image list including the newly registered posterior image as the touch list. It is characterized by being redisplayed on the screen of the screen device.

  According to this configuration, in the captured image list displayed by the image list display unit, when the posterior image has not been captured yet (when there is no posterior image in the image storage unit), the posterior image display area of the image set In the case where an unphotographed teaching image is displayed and the posterior image is still photographed, the posterior image is displayed in the posterior image display area of the image set. It is possible to immediately recognize an image group that has not yet been photographed. Therefore, it is possible to effectively prevent the occurrence of unrecorded images due to human error.

  Here, “photographing date / time information” of an image refers to information on the photographing date / time of the image. The “shooting location information” of an image refers to information regarding the shooting location of the image. As shooting location information, for example, latitude / longitude / altitude information captured using a GNSS (Global Navigation Satellite System) device, a code indicating a position written in a drawing of a separately prepared building, or the like is used. It is done. “Underlay image” refers to an image generated from a prior image and used as a reference for determining a shooting angle. For example, an image obtained by translucent the prior image, or a thin line obtained by performing edge extraction on the prior image. A digitized image or the like can be used. An “electronic viewfinder image” refers to an electronic viewfinder image. An “electronic viewfinder (EVF)” is an image taken on the screen of a display device (in this case, a display of a touch screen device) that is widely used in ordinary digital cameras, mobile phones, smartphones, and the like. A finder of a system that allows a video signal of a subject input to an element (camera device) to be displayed and confirmed in real time.

In a second configuration of the portable electronic device according to the present invention, in the first configuration, the pre-image in the captured image list is selected by the touch screen device in the captured image list. In this case, the pre-image is read from the image storage unit and displayed on the touch screen device, and a close-up button image indicating a tap operation position for inputting a close-up shooting execution instruction is displayed on the touch screen device. Display means;
When the close-up button image is selected by the touch screen device, an electronic viewfinder image captured in real time from the camera device and a tap operation position for inputting an imaging execution instruction to the touch screen device are displayed. Close-up viewfinder screen display means for displaying a close-up viewfinder screen including
When the shutter button image is selected by the touch screen device on the close-up viewfinder screen, the electronic viewfinder image at the time of selection is stored in the image storage unit as the pre-image and is taken from the internal clock The pre-image shooting date and time information, and the image management information including the shooting location information of the pre-image displayed on the touch screen device when the close-up button image was previously selected. Close-up image capturing means for registering an image in the photo management database,
When the pre-image is newly registered in the photo management database by the close-up image capturing unit, the image list display unit displays the photographed image list including the pre-registered new image as the touch screen. It is characterized by being redisplayed on the device screen.

  According to this configuration, when close-up shooting is performed at a shooting point where only whole shooting is not sufficient, it is easy to record and organize close-up images, and the efficiency of shooting and recording can be improved.

  According to a third configuration of the portable electronic device of the present invention, in the first or second configuration, the finder screen display unit with the base is a source of the base image on the finder screen with the base. The touch screen includes a shooting angle correction instruction mark that calculates a displacement vector of the electronic viewfinder image with respect to the pre-article image in real time, and indicates a direction in which the camera device should be moved in accordance with the calculated displacement vector. A photographing angle correction instruction means for displaying on the screen of the device is provided.

  In this way, by displaying the shooting angle correction instruction mark, the user can glance at which direction the portable electronic device should be moved to adjust the shooting angle, and can immediately recognize the position. Increases efficiency.

  Here, the calculation of the displacement vector can be performed by applying a known method such as a template matching (block matching) method, an active search method, a Mean Shift method, or a particle filter used in calculations such as subject tracking. (For example, refer nonpatent literature 8).

A portable electronic device according to a fourth aspect of the present invention is the portable electronic device according to any one of the first to third aspects, wherein the underlay finder screen display means is configured to display the underlay image on the underlay finder screen. Calculating in real time a displacement vector of the electronic viewfinder image with respect to the original prior image;
The posterior image capturing means refers to the displacement vector calculated by the underlay finder screen display means, and when the amount of displacement of the electronic viewfinder image with respect to the underlay image falls below a predetermined threshold, It is characterized in that it is provided with photographing angle match notifying means for notifying that the shooting angles are matched.

  Thereby, the user can glance at whether or not the shooting angles are well matched and can immediately visually recognize it, thereby improving the efficiency of the alignment work.

In a fifth configuration of the portable electronic device according to the present invention, in any one of the first to third configurations, the finder screen display unit with the underlay is configured to display the base image on the finder screen with the base. Calculating in real time a displacement vector of the electronic viewfinder image with respect to the original prior image;
The posterior image capturing means refers to the displacement vector calculated by the underlay finder screen display means, and when the amount of displacement of the electronic viewfinder image with respect to the underlay image falls below a predetermined threshold, The electronic viewfinder image at the time of the selection was stored in the image storage means as the posterior image, and was used as the underlying image of the finder superimposed image and the shooting date / time information of the posterior image captured from the internal clock The posterior image is registered in the photo management database in association with image management information including shooting location information of the previous image.

  Thus, the user can automatically shoot when the shooting angles are well matched, and the efficiency of the alignment shooting work is improved.

A sixth configuration of the portable electronic device according to the present invention includes a positioning module that performs positioning of the current position in any one of the first to fifth configurations,
A short-range wireless communication module for performing short-range wireless communication;
An orientation sensor that detects the current orientation;
The current absolute position is detected by the positioning module,
Wireless communication is performed with an RF tag installed at a position to be imaged by the short-range wireless communication module, and a distance to the RF tag is detected.
Detecting the visual axis direction of the photographing visual axis of the camera device by the orientation sensor,
Photographing position calculation means for calculating the absolute position of the RF tag based on the detected absolute position, the distance to the RF tag, and the visual axis direction,
The image list display means displays, on the screen of the touch screen device, a list addition button image indicating a tap operation position for inputting a list addition instruction for adding a new pre-image in addition to the photographed image list. And
When the list addition button image is selected by the touch screen device, a shutter button indicating a tap operation position for inputting an electronic viewfinder image captured in real time from the camera device and an imaging execution instruction to the touch screen device. Normal finder screen display means for displaying a normal finder screen including an image;
When the shutter button image is selected by the touch screen device on the normal viewfinder screen, the electronic viewfinder image at the time of selection is stored in the image storage unit as the pre-image and is taken from the internal clock The new image is registered in the photo management database in association with the image management information including the shooting date and time information of the shot image and the shooting location information including the absolute position of the RF tag calculated by the shooting position calculation means. An additional image capturing means,
When the previous image is newly registered in the photo management database by the new additional image capturing unit, the image list display unit is configured to touch the captured image list including the newly registered previous image with the touch. It is characterized by being redisplayed on the screen of the screen device.

  According to this configuration, since the position of the RF tag installed at the position of the photographing target is automatically acquired by the photographing position calculating means and is associated with the photographed image and registered in the photograph management database, the position of the photographing target Since it is no longer necessary to record (sketches) on the drawings prepared separately (for example, drawings of walls and ceilings of each surveyed property in the case of a construction damage survey), the efficiency of photographing and recording work is greatly improved. In addition, even if the surveyed property is very large and it is difficult to locate the location on the drawing, such as a fixed point progress survey of a crack on the wall of a dam, the location of the subject is recorded by recording the absolute location of the subject. Can be easily done.

  Here, as the “positioning module”, a GNSS module that measures the current position using the global positioning system, Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), or the like was used. An LPD module that performs positioning of the current position using a local positioning system (LPD) in a town, indoors, underground space, or the like can be used. “Direction sensor” includes an electronic compass (geomagnetic direction sensor) that detects the direction by detecting geomagnetism, and a geomagnetic direction sensor, a gyro sensor, and an acceleration sensor to correct turbulence in the underground space and buildings. Can be used (for example, see Non-Patent Documents 1 and 2). The “photographing visual axis” refers to an optical axis line connecting the center of the imaging surface of the camera device and the photographing center point (gaze point) of the photographing target. “Visual axis azimuth” refers to the azimuth (horizontal azimuth angle, or horizontal azimuth angle and vertical elevation angle) of the imaging visual axis.

According to a seventh configuration of the portable electronic device according to the present invention, in any one of the first to sixth configurations, the shutter button is used by the touch screen device on the underlay finder screen or the normal finder screen. When an image is selected, image editing means for displaying an image editing screen including a captured image that is the electronic viewfinder image at the time of selection on the touch screen device;
In the photographed image, a scale reference image serving as a scale reference registered in advance is searched, and when the scale reference image is searched, the photographing is performed based on the size of the scale reference image in the photographed image. A size conversion ratio calculating means for calculating a size conversion ratio that is a ratio of the size of the actual subject to the size on the image;
When the size conversion ratio is calculated by the size conversion ratio calculation means, when two points are specified by the touch screen device in the captured image on the image editing screen, the two points on the captured image And a size measuring unit that calculates the distance L2 between the two points in the subject by multiplying the distance L1 by the size conversion ratio and displays the distance L2 in the image editing screen. And

  According to this configuration, by using the scale reference image in the photographed image and specifying two points of the photographed image on the image editing screen by the touch screen device, the distance between the two corresponding points on the actual subject is automatically set. Can be measured. Thereby, it is possible to measure and record the distance on the image editing screen without actually performing measurement on the subject, and work efficiency in the investigation is improved.

  The portable electronic device photography / management program according to the present invention is read and executed by a portable electronic device including a touch screen device, a camera device, an internal clock, and a computer. The electronic device is made to function as the portable electronic device having any one of the first to fifth configurations.

  The portable electronic device photography / management program according to the present invention includes a touch screen device, a camera device, an internal clock, a positioning module, a short-range wireless communication module, an orientation sensor, and a computer. When the portable electronic device is read and executed, the portable electronic device functions as the portable electronic device having the sixth or seventh configuration.

  As described above, according to the portable electronic device of the present invention, when a posterior image has not yet been captured in the captured image list displayed by the image list display means (when there is no posterior image in the image storage means). In this case, an unphotographed teaching image is displayed in the posterior image display area of the image set, and if the posterior image has already been shot, the posterior image is displayed in the posterior image display area of the image set. It is possible to glance at the photographed image list and view an image group for which a subsequent photograph has not yet been photographed. Therefore, it is possible to effectively prevent the occurrence of unrecorded images due to human error.

It is an external view of the portable electronic device of Example 1 of this invention. (A) Front view, (b) Rear view. It is a figure showing the hardware constitutions of the portable electronic device of Example 1 of this invention. FIG. 2 is a block diagram illustrating a functional configuration of a portable electronic device according to a first embodiment configured by installing an application program in the portable electronic device of FIG. 1. FIG. 4 is an ER diagram (Entity Relationship Diagram) representing the configuration of the photo management database 32 of FIG. 3 (based on IDEF1X notation). It is a figure showing the transition of the screen display in the portable electronic device of Example 1. FIG. It is a figure showing the transition of the screen display in the portable electronic device of Example 1. FIG. It is a figure showing the transition of the screen display in the portable electronic device of Example 1. FIG. It is a figure showing the transition of the screen display in the portable electronic device of Example 1. FIG. It is a figure showing the transition of the screen display in the portable electronic device of Example 1. FIG. It is a figure explaining the registration process to the investigation management table at the time of close-up photography of a prior image, and the redisplay process of a picked-up image list screen. It is a figure explaining the registration process to the investigation management table at the time of close-up photography of a posterior image, and the redisplay process of a picked-up image list screen. It is a figure explaining the investigation photography additional photography process at the time of subsequent photography. It is a figure showing the structure of the investigation management table in the photograph management database 32 of Example 2. FIG. It is a figure which shows an example of a mode that a to-be-photographed object is image | photographed using the portable electronic device 1 of Example 2. FIG. (A) An example of photographing using a crack scale with an RF tag, (b) An example of photographing using a pointing stick with an RF tag. FIG. 12 is a diagram illustrating an electronic viewfinder screen at the time of subsequent image shooting in the third embodiment. (A) When the positional deviation amount is equal to or greater than a predetermined threshold value, (b) When the positional deviation amount falls below a predetermined threshold value. FIG. 10 is a diagram illustrating an example of specific data contents of an investigation management table of a photo management database 32 in the portable electronic device 1 according to a fourth embodiment. (A) A specific example of the contents of a survey management table, (b) Of the survey records stored in the survey management table, a diagram showing a relationship between a series of survey records for one fixed point. It is a figure showing the example of a display of the one photograph column 115a with respect to the fixed point investigation in the picked-up image list screen in the portable electronic device 1 of Example 4. FIG. FIG. 10 is a diagram illustrating a captured image list screen in the portable electronic device 1 according to the fourth embodiment. It is a block diagram showing the function structure of the portable electronic device of Example 5 comprised by installing an application program in the portable electronic device of FIG. It is the figure which showed the example which image | photographs the scale reference | standard thing P with the to-be-photographed object A. FIG. FIG. 10 is a diagram illustrating an image editing screen in a pointer editing mode according to a fifth embodiment. FIG. 10 is a diagram illustrating an image editing screen in a pointer editing mode according to a fifth embodiment. FIG. 10 is a diagram illustrating an image editing screen in a pointer editing mode according to a sixth embodiment.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

  In the first embodiment, an example in which the present invention is applied as a system for taking a photograph and managing a work damage investigation will be described.

(1) Hardware Configuration FIG. 1 is an external view of a portable electronic device according to a first embodiment of the present invention. In FIG. 1, a portable electronic device 1 according to the present embodiment is configured by a portable terminal with a built-in computer, such as a normal smartphone or tablet terminal, and includes a touch screen device 2 on the front side and a camera device 3 on the back side. It has.

  FIG. 2 is a diagram illustrating a hardware configuration of the portable electronic device according to the first embodiment of the present invention. In FIG. 2, a portable electronic device 1 includes a touch screen device 2, a camera device 3, a GNSS module 4, a mobile line communication module 5, a short-range wireless communication module 6, an acceleration sensor 7, an orientation sensor 8, an atmospheric pressure sensor 9, and a gyro. The sensor 10 includes an input / output bus 19, an I / O subsystem 20, a storage device 21, a storage controller 22, a peripheral interface 23, a processor 24, an internal clock 25, a memory 26, and a memory bus 27.

  The touch screen device 2 is a display that includes a touch panel and can input characters and select menus by touching the screen with a finger. The camera device 3 is a device that includes an image sensor and outputs an image captured by the image sensor. The GNSS module 4 currently uses a Global Navigation Satellite System (GNSS) such as the Global Positioning System (GPS) and the Quasi-Zenith Satellite System (QZSS). It is a device that measures position and outputs longitude, latitude, and altitude position information. The mobile line communication module 5 is a module that communicates with the outside via a normal mobile phone line. The short-range wireless communication module 6 is a module that performs wireless communication at a short distance of about several centimeters to several tens of meters. The acceleration sensor 7 is a sensor that detects an acceleration vector of the main body of the portable electronic device 1. The direction sensor 8 is a sensor that detects the line-of-sight direction of the camera device 3 by detecting geomagnetism. The atmospheric pressure sensor 9 is a sensor that detects the atmospheric pressure around the portable electronic device 1.

  The I / O subsystem 20 is an input / output between each device or module of the touch screen device 2, the camera device 3, the GNSS module 4, the mobile line communication module 5, and the short-range wireless communication module 6 and the peripheral interface 23. It is a controller which controls. Each device or module is controlled by a display controller 20a, a camera input controller 20b, a GNSS controller 20c, a mobile line communication controller 20d, and a short-range wireless communication controller 20e in the I / O subsystem 20, respectively. The storage device 21 is configured by a solid state drive (SSD) or a magnetic disk drive (HDD), and is a storage device that stores data such as image files, programs, and the like. The storage controller 22 is a controller that performs input / output control between the storage device 21 and the input / output bus 19. The peripheral interface 23 is a controller that performs input / output control between the I / O subsystem 20 and various sensors (acceleration sensor 7, orientation sensor 8, atmospheric pressure sensor 9, gyro sensor 10) and the input / output bus 19. The processor 24 executes a program to control the entire apparatus. The internal clock 25 outputs the current time. The memory 26 is an internal memory such as a DRAM, and the memory bus 27 is a bus for performing communication between the processor 24 and the memory 26.

(2) Functional Configuration The portable electronic device of this embodiment is configured on a software basis by loading and executing an application program on hardware incorporating a computer as shown in FIG. FIG. 3 is a block diagram illustrating a functional configuration of the portable electronic device of the first embodiment configured by installing an application program in the portable electronic device 1 of FIG. FIG. 3 shows only functional components related to the present invention, and other functional components are omitted. In the present embodiment, the application program is downloaded from an external server via the mobile line by the mobile line communication module 5 and installed in the mobile electronic device 1.

  3, the touch screen device 2, the camera device 3, the GNSS module 4, the short-range wireless communication module 6, the acceleration sensor 7, the direction sensor 8, the atmospheric pressure sensor 9, the gyro sensor 10, and the internal clock 25 are the same as those in FIG. Corresponds to the components with reference numerals. In addition to these components, the portable electronic device of the present embodiment further includes an image file storage unit 31, a photo management database 32, a shooting position calculation unit 33, an image list display unit 34, and a captured image display as functional configurations. Unit 35, underlay finder screen display unit 36, posterior image capture unit 37, new point record addition unit 38, normal finder screen display unit 39, new additional image capture unit 40, close-up viewfinder screen display unit 41, close-up image capture A loading unit 42, a white board display / editing unit 43, an image editing unit 44, a login unit 51, a survey list display unit 52, a new survey registration unit 53, a survey selection unit 54, and an owner registration unit 55 are provided. These are realized when the I / O subsystem 20, the storage device 21, the storage controller 22, the peripheral interface 23, and the processor 24 cooperate to execute an application program.

  The image file storage unit 31 stores an image taken by the camera device 3 as an image file. The photo management database 32 is a database that manages images stored in the image file storage unit 31 in association with image management information including shooting date / time information of the image and shooting location information.

  The photographing position calculation unit 33 detects the absolute position (latitude / longitude / altitude) of the portable electronic device 1. Here, the shooting position calculation unit 33 is the position information obtained by the GNSS module 4 (and the atmospheric pressure sensor 9), or the short-range wireless communication module 6 (Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark). The current absolute position (latitude / longitude / altitude) is detected based on position information obtained by a local positioning system (LPS) realized by using a local positioning system (LPS). As for LPS, various methods such as a method using Wi-Fi and a method using Bluetooth are currently disclosed (for example, refer to Non-Patent Document 3-5), and is not directly related to the present invention. Therefore, detailed description is omitted.

  The image list display unit 34 reads each piece of image management information registered in the photo management database 32 and displays a photographed image list on the screen of the touch screen device 2. Here, the captured image list is an image set in which images at the same shooting location are arranged side by side in the order of a previous image whose shooting date is the previous image and a subsequent image whose shooting date is the later image. This is a list in which the image sets are vertically arranged. The pre-image and post-image to be displayed are thumbnails (reduced images). Also, in the photographed image list, when the posterior image is not stored in the image file storage unit 31, an unphotographed instructional image (for example, a camera mark or the like) indicating that it has not been photographed is used instead of the posterior image. ) Is displayed. Further, the image list display unit 34 displays a list addition button image (a button image indicating a tap operation position for inputting a list addition instruction for adding a new pre-image) in addition to the photographed image list on the screen of the touch screen device 2. Display above.

  When the pre-image or the posterior image in the photographed image list is selected (tapped) by the touch screen device 2 in the photographed image list, the photographed image display unit 35 receives the pre-image from the image file storage unit 31. The image or posterior image is read and displayed on the touch screen device 2, and a close-up button image (a button image indicating a tap operation position for inputting a close-up shooting execution instruction) is displayed on the touch screen device 2.

  When the touchscreen device 2 selects an unphotographed instructional image in the photographed image list in the photographed image list, the underlay finder screen display unit 36 displays the finder superimposed image and the shutter button image (imaging execution instruction). And a finder screen with an underlay including a tap operation position for inputting) on the screen of the touch screen device 2. Here, the finder superimposed image is an image generated by superimposing and displaying an underlay image on an electronic viewfinder image captured in real time from the camera device 3, that is, an image of an electronic viewfinder in which the underlay image is superimposed and displayed. is there. The underlay image is generated by performing a lightening process, a monochromatic process, a contour extraction process, or the like on the prior image of the image set to which the selected shooting button image belongs.

When the shutter button image is selected by the touch screen device 2 on the above-described underlay finder screen, the posterior image capturing unit 37 captures the electronic viewfinder image at the time of selection as a posterior image, and stores it in the image file storage unit 31. save. In addition, the posterior image capturing unit 37 is associated with image management information including shooting date / time information of the posterior image captured from the internal clock 25 and shooting location information of the prior image used as the underlying image of the finder superimposed image, The subsequent image is registered in the photo management database 32.

  When the list addition button image is selected by the touch screen device 2 in the state in which the above-described photographed image list is displayed on the touch screen device 2, the new point record adding unit 38 is information on the photographing point to be added ( Hereinafter, a shooting point information input screen for inputting “additional shooting point information”) is displayed on the touch screen device 2. Further, the new location record adding unit 38 displays an imaging start button image indicating a tap operation position for inputting an imaging start instruction on the imaging location information input screen. When the additional shooting point information is input from the shooting point information input screen by the touch screen device 2 and the shooting start button image is selected (tapped), the new point record adding unit 38 manages the additional shooting point information in the photograph management. Register as a new record in the database 32.

  The normal viewfinder screen display unit 39 displays the shooting point information input screen on the touch screen device 2 and when the shooting start button image is selected (tapped) by the touch screen device 2, the normal viewfinder screen display unit 39 displays the touch screen device 2 on the touch screen device 2. A normal viewfinder screen including an electronic viewfinder image captured in real time from the camera device 3 and a shutter button image indicating a tap operation position for inputting an imaging execution instruction is displayed.

  When the shutter button image is selected (tapped) by the touch screen device 2 on the above-described normal finder screen, the newly added image capturing unit 40 uses the electronic viewfinder image at the time of selection as an advance image as an image file storage unit. Save to 31. Further, the newly added image capturing unit 40 registers the pre-image in the photo management database 32 in association with the image management information including the shooting date / time information and the shooting location information of the captured image captured from the internal clock 25.

  When the close-up finder screen display unit 41 selects (tap) the close-up button image displayed on the touch screen device 2 by the captured image display unit 35 by the touch screen device 2, the close-up view finder screen display unit 41 receives the touch screen device 2 from the camera device 3. A close-up viewfinder screen including an electronic viewfinder image captured in real time and a shutter button image indicating a tap operation position for inputting an imaging execution instruction is displayed.

  When the shutter button image is selected by the touch screen device 2 on the above-described close-up viewfinder screen, the close-up image capturing unit 42 stores the electronic viewfinder image at the time of selection in the image file storage unit 31 as a pre-image. Further, the close-up image capturing unit 42 captures the shooting date information of the pre-image captured from the internal clock 25 and the pre-image or the post-image displayed on the touch screen device 2 when the close-up button image is selected first. The prior image is registered in the photo management database 32 in association with the image management information including the shooting location information.

  The white board display / editing unit 43 performs editing processing of white board information stored as image management information in the photo management database 32. The image editing unit 44 edits an image taken by the newly added image capturing unit 40, the posterior image capturing unit 37, and the close-up image capturing unit 42 such as insertion of an instruction arrow or a line drawing figure. The login unit 51 performs login processing. The survey list display unit 52 displays a survey list screen including a registered survey item list. The new survey registration unit 53 performs processing for adding a new survey item to the survey item list on the survey list screen. The survey selection unit 54 performs a process of selecting a survey item from the survey item list on the survey list screen. The owner registration unit 55 performs registration processing of the owner information of the survey property.

(3) Database Structure FIG. 4 is an ER diagram (Entity Relationship Diagram) representing the configuration of the photo management database 32 of FIG. 3 (based on IDEF1X notation). The photo management database 32 stores an owner management table, a survey record book management table, and a survey management table. These tables are related to each other as shown in FIG. It is configured.

  The owner management table is a table for storing information regarding the owner of the property subject to inspection, and includes fields of “owner ID”, “name”, “address”, and “phone number”. The survey record book management table is a table for storing information related to the survey record book of each property damage investigation target property. “Survey record book ID”, “Survey record book name”, “Survey type”, “ It has fields of “owner ID” and “cover image file name”. The survey management table includes “survey ID”, “survey record book ID”, “survey number”, “date / time”, “building number”, “photo number”, “location name”, “survey name”, “photographed” “Property Name”, “Subject”, “Image File Name”, “Finish”, “Width”, “Length”, “Number of Damaged Locations”, “Photographing Type”, “Preliminary Survey ID”, “Overall Survey ID” ”Field. Information stored in each field of each table is as shown in the table in FIG.

  Here, “Survey name” in the survey management table indicates the type of survey conducted in the industrial damage survey, and “Internal survey”, “External survey”, “External survey”, “Inclined survey”, “Level survey”. Are selected from the following five. "Internal investigation" is investigation of floor creakings, tile gaps, joint cuts, cross cracks, gaps, painted wall cracks, dust cuts, ceiling leaks, etc., "external investigation" is foundation cracks and damage・ Investigation of cracks in outer walls, joint breaks, roof tile displacement, etc., “External Survey” is a survey of cracks in concrete between concrete, cracks in gaps and dogs, cracks in joints and cracks in gaps and fences, etc. "Survey of column inclination, sill inclination, floor inclination, etc." Level investigation "is a survey on level measurement to determine whether ground subsidence has occurred in the corners of the building or dogs. is there. It should be noted that the “survey name” can be appropriately added as needed.

(4) Operation of Portable Electronic Device The operation of the portable electronic device 1 of the present embodiment configured as described above will be described. 5 to 9 are diagrams illustrating transition of screen display in the portable electronic device of the first embodiment.

(Step 1) (Login)
When the application program installed in the portable electronic device 1 is activated, the login unit 51 first displays the login screen shown in FIG. 5A on the touch screen device 2.

  When the user selects the login button 101 with the touch screen device 2 on this login screen, the investigation list display unit 52 next displays the investigation record list 102 of FIG. Here, the “selection” operation in the touch screen device 2 refers to an operation of tapping a location to be selected on the screen, as is widely performed in general touch screen devices. The “survey record book” is a book that records a series of survey records for one project. For example, in the case of a construction damage investigation related to “XX Junior High School Second Gymnasium Construction Work”, a book that summarizes a series of investigation records for investigation of construction damage in the construction project is a survey record book for the construction project. “Survey record book list” refers to a list of survey record books.

(Step 2) (Select survey catalog)
As shown in FIG. 5B, the survey record book list 102 includes a survey record book list 102a for a case in which an industrial damage investigation is ongoing, and a survey record book list 102b for past work damage investigations. In addition, a new survey addition button image 103 for inputting an instruction to add a new survey record list is displayed at the upper right of the survey record list 102 on the touch screen device 2. Is done. In this screen, when the user selects the new survey addition button image 103 using the touch screen device 2, the process proceeds to the process for adding a new survey catalog to the survey catalog list in step 3. . On the other hand, when the user selects any item in each survey record list in the survey record list 102, FIG. 7 (a) (when no survey photo exists in the survey record list) or FIG. Move to the screen of 8 (d) (when a survey photo exists in the survey record book).

(Step 3) (Addition of survey record book)
When the new survey addition button image 103 is selected, the new survey registration unit 53 first displays the initial registration screen of FIG. 5C on the touch screen device 2 and searches the survey record book of the photo management database 32. Add a new record (new registration record) to the table. In this initial registration screen, a survey start instruction button image 104 is displayed at the bottom of the screen.

  When the user selects the survey start instruction button image 104 by the touch screen device 2 on the initial registration screen, the new survey registration unit 53 is placed on the touch screen device 2 in order to prompt the user to take a display image. The electronic viewfinder screen of FIG. 5D is displayed. On the electronic viewfinder screen, an electronic viewfinder image 105 captured in real time from the camera device 3 is displayed at the center, and a shutter button image 106 is displayed beside the electronic viewfinder image, in the same manner as a camera function in a normal smartphone. The user selects the shutter button image 106 toward the survey target property that is the target of the survey item to be newly registered, and takes a photograph of the appearance of the survey target property. This appearance photograph is used as a cover image of the survey record book of the survey case. When the shutter button image 106 is selected, the new survey registration unit 53 captures the electronic viewfinder image 105 at the time of selection, and the new survey registration unit 53 is shown on the touch screen device 2 as shown in FIG. A cover photo confirmation screen like this is displayed. At this time, the new survey registration unit 53 takes in the current absolute position information (latitude / longitude / elevation) output from the photographing position calculation unit 33 and the current date / time information output from the internal clock 25, and these information are also covered. Display on the photo confirmation screen.

  On the cover photo confirmation screen, a cover image 107 photographed at the center is displayed, and a re-photograph button image 108 and a cover confirmation button image 109 are displayed at the bottom. Also, the absolute position information at the time of shooting is displayed in the shooting position display area 110 on the lower left side of the cover image 107, and the date and time information at the time of shooting is displayed in the shooting time display area 111 on the lower right side of the cover image 107. When the user selects the re-shoot button image 108 with the touch screen device 2 on this cover photo confirmation screen, the user can return to the electronic viewfinder screen of FIG. 5D and re-take the cover image. On the other hand, when the user selects the cover confirmation button image 109 with the touch screen device 2 on the cover photo confirmation screen, the new survey registration unit 53 saves the cover image 107 in the image file storage unit 31 and sets the image file name as the image file name. Register in the “Cover Image File Name” field of the newly registered record in the survey record book table. At this time, the new investigation registration unit 53 adds the absolute position information of the shooting position and the date / time information at the time of shooting to the cover image 107 as a digital watermark or image additional information (for example, shooting in Exif (Exchangeable image file format) in JPEG). Embedded as date / time information and GNSS information) and stored in the image file storage unit 31. Then, the image list display unit 34 displays the photographed image list screen of FIG. 6B (FIG. 7A) on the touch screen device 2.

(Step 4) (Setting owner information)
In the initial state when a new survey catalog is created, there are no survey photos in the survey catalog. Accordingly, an empty photographed image list 115 is displayed in the lower half of the center of the photographed image list screen as shown in FIG. 6B (FIG. 7A). Although FIG. 6B and FIG. 7A are illustrated separately for convenience of explanation, they represent the same screen. On the upper edge of the photographed image list 115, five survey name tab button images 116 ("External", "Internal", "External", "Inclined", "Level") for selecting the type of survey are displayed. Button image) is displayed in a horizontal row. These are the five types of surveys, “Internal Survey”, “External Survey”, “External Survey”, “Inclined Survey”, and “Level Survey”, which correspond to the selection of each button. A photographed image list 115 of survey photographs belonging to the survey type is displayed, and the name of the selected survey type is displayed in the survey type display area 117 at the top of the photographed image list screen. In the cover image display area 118 in the upper half of the center portion of the photographed image list screen, the cover image that has been photographed first is displayed. On the lower part of the photographed image list screen, a current photograph button image 119 for inputting a photograph instruction for a current photograph and an external photograph button image 120 for inputting a photograph instruction for an external photograph are displayed. Here, the “current picture” is a picture for recording the current situation at the time of shooting. “External photo” refers to a photo for recording changes before and after the event.

  When the user sets owner information (owner of the property to be investigated) on the captured image list screen of FIG. 6B (FIG. 7A), the cover of the captured image list screen of the touch screen device 2 Tap the image display area 118. When the cover image display area 118 is tapped, the owner registration unit 55 displays the screen of FIG. 6C on the touch screen device 2. On the screen of FIG. 6C, an owner edit instruction button image 125, a cover photo confirmation button image 126, and a cancel button image 127 are displayed. Here, when the user selects the cover photo confirmation button image 126 with the touch screen device 2 on this screen, the screen returns to the image shown in FIG. When the cancel button image 127 is selected, the screen returns to the image shown in FIG. 6B (FIG. 7A). On the other hand, when the user selects the owner editing instruction button image 125 using the touch screen device 2, the owner registration unit 55 displays an owner editing screen as shown in FIG. 6D on the touch screen device 2. To do.

  In the owner edit screen of FIG. 6D, a list of owner information currently registered in the owner table of the photo management database 32 is displayed in the owner list display area 128 in the center of the screen. Also, an owner addition button image 129, a save button image 130, and a cancel button image 131 are displayed at the top of the owner edit screen. When the user selects the cancel button image 131 by the touch screen device 2 on the owner edit screen, the screen returns to the image shown in FIG. When the owner addition button image 129 is selected, the owner registration unit 55 displays the owner information input screen shown in FIG. 6E on the touch screen device 2. When the user inputs the owner's name, address, and telephone number using the touch screen device 2 and selects the return button image 132 from the owner information input screen, the owner registration unit 55 owns the added owner. The owner information is stored in the owner table of the photo management database 32, and the screen returns to the owner edit screen shown in FIG. At this time, owner information of the added owner is additionally displayed in the owner list display area 128.

  Further, when the user selects any owner of the owner list display area 128 by the touch screen device 2 and selects the save button image 130 in that state on the owner editing screen of FIG. The registration unit 55 saves the owner ID of the selected owner in the “owner ID” field (see FIG. 4) of the current survey item record in the survey record table of the photo management database 32, and FIG. Return to the photographed image list screen of (b). Thereby, the owner information is set.

(Step 5) (Additional photo of survey photo)
When the user performs additional photography of the survey photograph on the photographed image list screen of FIG. 6B (FIG. 7A), the current photograph photograph button image 119 on the photographed image list screen of the touch screen device 2 or the external A photograph button image 120 (these two are collectively referred to as “list addition button image”) is selected. Here, when the current state photography button image 119 is selected, the current state photography instruction state is entered, and when the outside photography button image 120 is selected, the state is an external photography instruction state. When the current state photography button image 119 or the external photography button image 120 is selected, the whiteboard display / editing unit 43 displays the photographing point information input screen of FIG. 7B on the touch screen device 2. The shooting point information input screen is a screen for inputting information related to a shooting point to be added. In the case of the present embodiment, specifically, information to be entered on the white board to be photographed together with the photographing target point in the conventional engineering damage investigation is input on the photographing point information input screen.

  In FIG. 7 (b), for convenience of explanation, the shooting point information input screen is shown in a vertically longer form than the other screens, but what is actually displayed on the touch screen device 2 is Only the screen size portion of the touch screen device 2 is displayed in the shooting point information input screen shown in FIG. Then, the display portion can be moved up and down by swiping or flicking the touch screen device 2 up and down (putting the finger on the screen and paying up and down). This operation is the same as the operation method currently used with the smart phone and tablet terminal currently widely used. The same applies to the description in FIG.

  On the shooting point information input screen, a shooting point information input area 141 is displayed at the center, and a shooting start button image 142 is displayed at the bottom thereof. In the shooting point information input area 141 of this shooting point information input screen, the user can add information about the shooting point to be added from the touch screen device 2 (building number, room name (location name), part name (object name), finishing). Material, damage type (object to be imaged), size (width, length), number of damaged parts) can be entered. When information regarding the shooting point is input to the shooting point information input area 141, the whiteboard display / editing unit 43 stores the input information as a new record in the investigation management table of the photo management database 32.

  When the user selects the shooting start button image 142 from the touch screen device 2 on the shooting point information input screen, the normal viewfinder screen display unit 39 displays the electronic image shown in FIG. A viewfinder screen is displayed, and in the case of the current photo shooting instruction state, the electronic viewfinder screen of FIG. 8C is displayed. Hereinafter, the operation in each state is as follows.

(A) External photograph shooting instruction state In the electronic viewfinder screen of FIG. 7C, the normal viewfinder screen display unit 39 displays an electronic viewfinder image 151 captured in real time from the camera device 3 at the center, and next to it. The shutter button image 106 is displayed, and the white board image 153 is superimposed and displayed at any one of the left, right, upper, and lower corners of the electronic viewfinder image 151. Here, in the white board image, information regarding the additional shooting point input on the shooting point information input screen is displayed as a white board. Here, when the user taps the white board image 153, the normal finder screen display unit 39 enlarges and displays the white board image on the entire screen of the touch screen device 2, as shown in FIG. When the user taps the close window button image 154 at the upper right of the white board image of FIG. 7D, the screen returns to the electronic viewfinder screen of FIG. Thereby, the user can take an external photograph while confirming the white board information.

  When the shutter button image 106 is selected by the user on the electronic viewfinder screen, the newly added image capturing unit 40 captures the electronic viewfinder image at the time of selection as a pre-image, and the shooting position calculation unit 33 outputs it. The current position information and the current date / time information output from the internal clock 25 are captured, and then the image editing unit 44 displays the image editing screen of FIG. 8A on the touch screen device 2. On the image editing screen, a pre-image captured in the central image display area 161 is displayed, and a white board image 153 is superimposed on the upper left corner. The white board image 153 is moved in the image display area 161 when the user swipes the white board image 153 on the touch screen device 2 (performs an operation of pushing the screen with a finger and sweeping in a certain direction). Can do. Further, on the right side of the image display area 161, an editing operation selection button image 162, a re-shooting button image 108, and a save button image 130 are displayed.

  The editing operation selection button image 162 is a button for switching the editing operation. When the user taps the editing operation selection button image 162, the image editing screen (in the pointer editing mode in FIG. 8A) is displayed. The image editing screen) and the image editing screen in FIG. 8B (the image editing screen in the pen editing mode) are switched by the toggle operation. On the image editing screen in the pointer editing mode in FIG. 8A, an arrow image (pointer) is displayed at a position where the user taps on the image display area 161. The arrow image can be moved in the image display area 161 by swiping the arrow image on the touch screen device 2. In addition, a plurality of arrow images can be displayed, and a new arrow image is added each time the user taps a position where there is no arrow image on the image display area 161. On the other hand, when the user taps an arrow image already displayed on the image display area 161, the arrow image can be deleted. Further, on the image editing screen in the pen editing mode in FIG. 8B, a locus line image is displayed on a locus swiped on the image display area 161 by the user. In the image editing screen of FIG. 8B, a pen point button image 163 and an eraser button image 164 are additionally displayed on the left side of the image display area 161. When adding a trajectory line image, the user selects the pen tip button image 163 and then swipes the place where the trajectory line image is to be added on the image display area 161. Thereby, a trajectory line image is displayed along the swipe trajectory. A plurality of trajectory line images can be added. On the other hand, when erasing a locus line image, the user can erase the tapped locus line image by selecting the eraser button image 164 and then tapping the locus line image to be erased. The above editing process and white board position movement process are executed by the image editing unit 44.

  In the image editing screens of FIGS. 8A and 8B, when the re-photo button image 108 is selected by the user, the screen returns to the electronic viewfinder screen of FIG. On the other hand, when the save button image 130 is selected, the newly added image capturing unit 40 displays the prior image and the white board image displayed in the image display area 161, the arrow image added by the editing operation, and the like (hereinafter “ The edited image ") is stored in the image file storage unit 31 as a multilayer image including a photographic image layer, a white board image layer, and an edited image layer. At that time, the shooting date / time information and shooting location information of the previous image are embedded as a digital watermark or image additional information in the previous image. Further, the file name of the prior image stored in the image file storage unit 31 is registered as a new record in the investigation management table of the photo management database 32 in association with the image management information including the shooting date / time information and the shooting location information. At this time, “photo number” is automatically numbered consecutively, “shooting type” is “external photo”, and “preliminary survey ID” and “overall survey ID” are invalid values (for example, NULL). Is set. Then, the image list display unit 34 displays a photographed image list screen as shown in FIG. At this time, the newly added prior image is displayed in the captured image list 115 on the captured image list screen. However, since there is no posterior image corresponding to the prior image at this time, the prior image in the captured image list 115 is displayed. In this column, the preliminary image and the unphotographed instructional image 181 are displayed side by side. In addition, an external photograph mark 182 (“preliminary” and “subsequent” marks) indicating that the photograph is an external photograph is displayed at the bottom of the preliminary image column (see FIG. 8G).

(B) Current photograph shooting instruction state In the electronic viewfinder screen of FIG. 8C, the normal viewfinder screen display unit 39 displays an electronic viewfinder image 151 captured in real time from the camera device 3 at the center, and next to it. A shutter button image 106 is displayed. In this case, the white board image 153 is not displayed. When the user selects the shutter button image 106, the newly added image capturing unit 40 captures the electronic viewfinder image at the time of selection as a pre-image, and also outputs the current position information and the internal clock output from the shooting position calculating unit 33. The current date / time information output by the CPU 25 is captured and stored in the image file storage unit 31. At that time, the shooting date / time information of the pre-image captured from the internal clock 25 and the shooting location information calculated by the shooting position calculation unit 33 are embedded as a digital watermark or image additional information in the pre-image. Further, the file name of the prior image stored in the image file storage unit 31 is registered as a new record in the investigation management table of the photo management database 32 in association with the image management information including the shooting date / time information and the shooting location information. At this time, “photo number” is automatically numbered consecutively, “shooting type” is “current photo”, and “preliminary survey ID” and “overall survey ID” are invalid values (for example, NULL). Is set. Then, a captured image list screen as shown in FIG. 8D is displayed. At this time, the newly added prior image is displayed in the captured image list 115 on the captured image list screen. However, since the posterior image is not necessary in the current state photograph, the column of the prior image in the captured image list 115 includes Only the prior image is displayed. In addition, a current state photo mark 183 (a “current state” mark) indicating that it is a current state photo is displayed at the bottom of the advance image column.

(Step 6) (Browsing each column of the photographed image list)
FIG. 8D shows the state of the photographed image list screen at the stage where the work damage investigation has progressed. The display processing of the photographed image list screen on the touch screen device 2 is executed by the image list display unit 34. In the initial stage of the industrial damage investigation, the photographed image list 115 is empty as shown in FIG. 6B (FIG. 7A), but in FIG. 8D, a plurality of photograph columns 115a are added to the photographed image list 115. Has been. As shown in the enlarged view of FIG. 8 (e), one photo column 115a includes a pre-image display area 191 in which a pre-image is displayed, a post-image display area 192 in which a post-image is displayed, a place name and a subject image. A place / photographed property name display area 193 where the property name is displayed, a shooting type display area 194 where the shooting type (“present status” / “pre / post”) is displayed, and a photo number display area 195 where the photo number is displayed , And a pull-down arrow mark display area 196 in which a pull-down arrow mark image is displayed. The pre-image display area 191 and the post-image display area 192 are provided in parallel in the left and right in the upper range of about 4/5 of the photo column 115a, and the place / photographed object name display area 193, the shooting type display area 194, and the pull-down menu. The arrow mark display areas 196 are provided side by side from left to right in the range of about 1/5 of the lower part of the photo column 115a. The photo number display area 195 is provided so as to overlap the upper right corner of the pre-image display area 191.

  When the “photographing type” is a current picture (“current situation”), the posterior image display area 192 is blank, and “current situation” is displayed in the field of the photographing type display area 194.

  When the “shooting type” is an external photograph (“pre-post / post-post”) and the post-image is in the image file storage unit 31, the post-image is displayed in the post-image display area 192, and the shooting type is displayed. In the column of the display area 194, the characters “previous” and “subsequent” are displayed in dark colors (see the photo column 115a of the photo number 1-3 in FIG. 8D and FIG. 8H). Here, when the image list display unit 34 determines whether or not the posterior image corresponding to the pre-image of each photo column 115a is in the image file storage unit 31, the image list display unit 34 Referring to the survey management table of the photo management database 32, first obtain the “survey ID” of the preliminary image, and then use the survey ID of the acquired preliminary image as a key to specify “preliminary survey ID” in each record of the survey management table. "" Is searched for a record that matches the survey ID of the prior image. If there is no matching record, it is determined that the posterior image is not in the image file storage unit 31 (the posterior image has not yet been photographed). If there is a matching record, the image file corresponding to the record is the posterior image. To be judged.

  When the shooting type is external photography (“pre / post”) and the post image has not been taken yet (when there is no post image in the image file storage unit 31), the post image display area 192. In FIG. 8D, an unphotographed teaching image 181 (an image in which a character string “post-shooting” is displayed under the camera mark) as shown in FIG. 8D is displayed. In the field of the photographing type display area 194, characters “prior” and “post fact” are displayed, “prior” is displayed in dark color, and “post fact” is displayed in light color (the photo number in FIG. 8D). 1-2, see the photo column 115a, FIG. 8 (g)). As a result, the user can glance at the photographed image list screen and visually recognize the photo field 115a in which the subsequent photograph has not yet been photographed.

  When the user taps the pull-down arrow mark display area 196 in the lower right corner of each photograph field 115a on the photographed image list screen, the image list display unit 34 displays the bottom of the image field as shown in FIG. In addition, information regarding the shooting point (white board information) is displayed as a pull-down list 184. The pull-down list 184 includes “room name (location name)”, “measurement location (name of the object to be photographed)”, “finishing material”, “damage type (object to be photographed)”, “size (width, length). "," Property number (building number) "column is displayed, and a close-up photography button image 184 is displayed in the lower right corner. The image list display unit 34 reads the information of each column corresponding to the photo column 115 a where the pull-down arrow mark display area 196 is tapped from the photo management database 32 and displays the information in each column of the pull-down list 184.

  In the state where the pull-down list 184 is displayed, when the close-up photo shooting button image 184 is selected by the user, a close-up shooting operation is executed. The close-up shooting operation will be described in step 9.

(Step 7) (Capturing the subsequent image)
In a state where the captured image list screen of FIG. 8D is displayed on the touch screen device 2 (or in a state where the pull-down list 184 of FIG. 8E is expanded), the user can postpone any of the photo fields 115a. When the unphotographed instructional image 181 displayed in the image display area 192 is tapped, first, the underlay finder screen display unit 36 displays a pre-image (hereinafter referred to as a pre-image) of the photo column 115a to which the tapped unphotographed instructional image 181 belongs. The survey record of “corresponding pre-image”) is read out from the survey management table of the photo management database 32, and then the whiteboard display / editing unit 43 displays the same shooting point information input screen as FIG. Encourage the user to add shooting point information. Here, in each input field of the whiteboard input screen to be displayed, the shooting point information of the corresponding prior image read out in advance is previously entered as a default value. Therefore, the user only needs to input the column that he wants to change. When the input of the shooting point information is finished, the user selects the shooting start button image 142 (see FIG. 7B) from the touch screen device 2 as it is. When the shooting start button image 142 is selected by the user, the underlay finder screen display unit 36 displays the electronic viewfinder screen of FIG. 9A on the touch screen device 2. At this time, the underlay finder screen display unit 36 reads the corresponding preliminary image from the image file storage unit 31, and adds the base image generated from the corresponding preliminary image to the electronic viewfinder image captured in real time from the camera device 3. A superimposed viewfinder image is generated. Then, the generated finder superimposed image is displayed in the viewfinder area 201 at the center of the electronic viewfinder screen. Here, the underlying image is generated by making the read corresponding preliminary image lighter and translucent. The user moves the portable electronic device 1 while observing the finder superimposed image, and determines the shooting angle so that the electronic viewfinder image that changes in real time exactly overlaps the underlying image. Can be matched.

  The underlay finder screen display unit 36 displays the shutter button image 106 on the side of the viewfinder area 201 on the electronic viewfinder screen, and a white plate image 153 is superimposed on the upper left corner of the viewfinder area 201. At this time, the underlay finder screen display unit 36 reads the record of the corresponding pre-image from the investigation management table of the photo management database 32, and captures the image regarding the additional shooting point previously input on the shooting point information input screen to the white board image 153. The point information is displayed as a white board (see FIG. 9A).

  When the user selects the shutter button image 106 from the touch screen device 2 in the state in which the electronic viewfinder screen of FIG. 9A is displayed, the posterior image capturing unit 37 selects the electronic image at the time of selection. The viewfinder image is captured as a posterior image, and the current absolute position information (latitude, longitude, altitude) output from the photographing position calculation unit 33 and the current date and time information output from the internal clock 25 are captured. Next, the image editing unit 44 displays the image editing screen of FIG. On the image editing screen, the posterior image captured in the central image display area 161 is displayed, and a white board image 153 is superimposed and displayed at any one of the left, right, upper and lower corners. The position of the white board image 153 in the image display area 161 can be moved by the user swiping the white board image 153 on the touch screen device 2. Further, on the right side of the image display area 161, an editing operation selection button image 162, a re-shooting button image 108, and a save button image 130 are displayed. Since the editing operation on the image editing screen has already been described in the above “Step 5”, the description thereof will be omitted.

  When the user selects the save button image 130 from the touch screen device 2 on the image editing screen, the posterior image capturing unit 37 adds the pre-image and the white board image displayed in the image display area 161 and the editing operation. The arrow image or the like (hereinafter referred to as “edited image”) is stored in the image file storage unit 31 as a multilayer image including a photographic image layer, a white board image layer, and an edited image layer. At that time, the shooting date / time information and shooting location information of the previous image are embedded as a digital watermark or image additional information in the previous image. Further, the file name of the prior image stored in the image file storage unit 31 is registered in the investigation management table of the photo management database 32 in association with the image management information including the shooting date / time information and the shooting location information. At this time, “Photo number” is set to the same number as the photo number of the corresponding pre-image, “Photographing type” is set to “External photo”, and “Pre-survey ID” is set to the survey ID of the corresponding pre-image. Then, the “overall survey ID” is set to an invalid value (for example, NULL). Thereby, the association with the record of the image of the pre-photo is made. Then, the screen returns to the photographed image list screen as shown in FIG.

(Step 8) (Operation for confirming photographed image)
In a state where the photographed image list screen in FIG. 8D is displayed on the touch screen device 2 (or in a state in which the pull-down list 184 in FIG. 8E is expanded), the user can pre-select any of the photo fields 115a. When the captured previous image displayed in the image display area 191 or the captured posterior image displayed in the posterior image display area 192 is tapped, the captured image display unit 35 is displayed on the touch screen device 2 as shown in FIG. A captured image confirmation screen as shown in (b) is displayed. The captured image confirmation screen displays the tapped prior image or subsequent image (hereinafter referred to as “selection confirmed image”) in the central image display area 161, and the image type name (“preliminary image” above the image display area 161. "/" Post image ") and a closing window image 154 are displayed, and a close-up photography button image 185, a re-shooting button image 108, and an image deletion button image 202 are displayed below the image display area 161. This photographed image confirmation screen allows the user to enlarge and confirm the photographed selection confirmation image.

  When the user selects the window closing button image 154 from the touch screen device 2 on the photographed image confirmation screen, the photographed image display unit 35 closes the photographed image confirmation screen and displays the photographed image list screen in FIG. Return. When the image deletion button image 202 is selected, the captured image display unit 35 deletes the image file and the image file name of the selected confirmation image from the image file storage unit 31 and the photo management database 32, and confirms the captured image confirmation. The screen is closed and the screen returns to the photographed image list screen of FIG. At this time, the deleted image is not displayed on the photographed image list screen, but a close-up photograph photographing button image 185 is displayed instead.

  When the user selects the re-capture button image 108 from the touch screen device 2 on the captured image confirmation screen, the screen returns to the electronic viewfinder screen of FIG. 7C, and the previous image or the subsequent image corresponding to the selected confirmation image is displayed again. Move to shooting operation. The description of the shooting operation and the image editing operation after FIG.

  When the user selects the close-up photography button image 185 from the touch screen device 2 on the photographed image confirmation screen, the process proceeds to the next close-up photography operation.

(Step 9) (Close-up shooting operation)
When the close-up photography button image 185 in FIG. 8E or FIG. 9B is selected, the following close-up photography operation is executed. First, the close-up viewfinder screen display unit 41 takes a record of a pre-image or a post-image (hereinafter referred to as “selection source image”) in the photo column 115a that is selected when the close-up photography button image 185 is selected. Read from the survey management table of the management database 32, obtain the survey ID of the record, and set this as the overall survey ID. Here, “overall survey ID” refers to the survey ID of the record of the entire image with respect to the close-up image record of the survey management table. Next, the close-up viewfinder screen display unit 41 displays the electronic viewfinder screen of FIG. 7C on the touch screen device 2, and takes and edits the close-up image by the same operation as described in step 4. .

  When the user selects the save button image 130 on the image editing screen in FIG. 8A or 8B, the close-up image capture unit 42 displays the close-up image displayed in the image display area 161. And the white board image and the arrow image added by the editing operation (hereinafter referred to as “edited image”) are stored in the image file storage unit 31 as a multilayer image including a photographic image layer, a white board image layer, and an edited image layer. At that time, the shooting date / time information and shooting location information of the close-up pre-image are embedded in the close-up image as a digital watermark or image additional information. Furthermore, the file name of the close-up image stored in the image file storage unit 31 is registered as a new record in the investigation management table of the photo management database 32 in association with the image management information including the shooting date / time information and shooting location information. At this time, “photo number” is automatically numbered consecutively, “shooting type” is set to “external photo”, and the overall survey ID set previously is set to “overall survey ID”. As a result, the entire picture is associated with the record of the image. At this time, the “prior survey ID” is set to an invalid value (for example, NULL). This is because when the selection source image is a prior image, there is no previous previous image, and when the selection source image is a posterior image, there is a survey record of the prior image corresponding to the close-up image. This is because it does not exist.

  Then, the image list display unit 34 redisplays the captured image list screen as shown in FIG. At this time, as shown in FIG. 8D, the image list display unit 34 sets the overall survey ID for the photo field 115a of the record (child record) in which the overall survey ID is set in the “overall survey ID” field. Is displayed so as to be continuous below the photo field 115a of the record (parent record), and in the photo field 115a of the child record, a close-up display mark 186 is displayed on the left side of the photo field 115a so that it can be seen that the photo is close-up. Is displayed. Thereby, the user who has looked at the photographed image list screen can easily visually recognize the distinction between the photo column 115a of the whole photo and the photo column 115a of the close-up photo and the relationship thereof.

(4.1) Supplementary explanation of registration processing in the investigation management table and close-up image list screen display processing during close-up photography Next, a close-up image investigation record by the close-up image capture unit 42 during close-up photography A supplementary explanation will be given of the registration processing in the survey management table and the re-display processing of the photographed image list screen by the image list display unit 34.

(A) At the time of photographing a pre-image For example, a case where the photographed image list 115 on the photographed image list screen is in the state of FIG. 10A1 and the investigation management table of the photograph management database 32 is in the state of FIG. Think. Note that symbols such as A1, A2 and the like displayed in the prior image display area 191 in the photographed image list 115 in FIG. 10 represent photo numbers.

  (A1) In this state, the user selects (tap) a pre-image of the photo number A1, and then selects (tap) the close-up photo button 185 on the screen of FIG. 9B to take a close-up photo. In this case, the close-up image capture unit 42 newly creates a survey record (survey record with survey ID = 3) for the close-up photo in the survey management table as shown in FIG. 10 (b2). In the “overall survey ID” of the survey record, the survey ID (1) of the survey record of the prior image of the photo number A1 that is the selection source image is stored. Thus, the survey record (survey ID = 3, photo number = A1-1) is associated with the survey record (survey ID = 1, photo number = A1) as a close-up image of the prior image. At this time, the close-up image capture unit 42 copies the value of the corresponding field in the survey record of the selection source image A1 to the other field of the survey record for the close-up photo. Then, when the survey management table is updated, the image list display unit 34, as shown in FIG. 10 (a2), the newly registered close-up image survey record (survey ID = 3, photo number = A1-). The photo column 115a of 1) is displayed continuously below the photo column 115a of the survey record (survey ID = 1, photo number = A1) of the selection source image, and a close-up display mark 186 is displayed on the left side of the photo column 115a. indicate.

  (A2) Next, the user selects (tap) the pre-image of the photo number A1 again or selects (tap) the pre-image of the photo number A1-1, and then the close-up photography button on the screen of FIG. 9B. When the image 185 is selected (tapped) and a close-up photo is taken, the close-up image capturing unit 42 stores a search record (investigation ID) for the close-up photo in the search management table, as shown in FIG. = 4 survey record) is newly created, and an invalid value (NULL) is set in the “overall survey ID” field of the survey record of the selection source image in the “overall survey ID” of the survey record ( That is, when the selection source image is the photo number A1, the survey ID (1) of the survey record of the selection source image is set to a valid value in the “overall survey ID” field of the survey record of the selection source image. Is the case are (i.e., if the selected original image Image Number A1-1) To store the survey ID (1) that is set in the "overall survey ID" field. Thus, the survey record (survey ID = 4, photo number = A1-2) is associated with the survey record (survey ID = 1, photo number = A1) as a close-up image of the prior image. At this time, the close-up image capture unit 42 copies the value of the corresponding field in the survey record of the selection source image A1 to the other field of the survey record for the close-up photo. Then, when the survey management table is updated, the image list display unit 34, as shown in FIG. 10 (a3), the newly registered close-up image survey record (survey ID = 4, photo number = A1-). 2) the photo column 115a is displayed continuously below the photo column 115a of the survey record (survey ID = 1, photo number = A1) of the selection source image, and a close-up display mark 186 is displayed on the left side of the photo column 115a. indicate.

  Similarly, the close-up image corresponding to the photograph number A1 can be sequentially added to the investigation management table and the photographed image list 115 in the photograph management database 32. The close-up image corresponding to the photo number B1 can be added sequentially in the same manner.

(B) At the time of shooting a subsequent image For example, a case where the shot image list 115 on the shot image list screen is in the state of FIG. 11 (a1) and the investigation management table of the photo management database 32 is in the state of FIG. 11 (b1). Think. Note that symbols such as A1, A1-1, A2, and B1 displayed in the pre-image display area 191 and the post-image display area 192 in the photographed image list 115 in FIG. 11 represent photo numbers.

  (B1) In this state, the user selects (tap) the posterior image of the photo number B1, and then selects (tap) the close-up photo button 185 on the screen of FIG. 9B to take a close-up photo. In this case, the close-up image capturing unit 42 newly creates a survey record (survey record with survey ID = 5) for the close-up photo in the survey management table as shown in FIG. 11 (b2). In the “overall survey ID” of the survey record, the survey ID (4) of the survey record of the posterior image of the photo number B1 which is the selection source image is stored. Further, since there is no prior image for the close-up photograph, the “pre-survey ID” of the survey record stores a non-existing code 0 that is a number indicating that “the previous image does not exist for the subsequent image”. Thereby, the survey record (survey ID = 5, photo number = B1-2) is associated with the survey record (survey ID = 4, photo number = B1) as a close-up image of the posterior image. At this time, the close-up image capture unit 42 copies the value of the corresponding field in the survey record of the selection source image B1 to the other field of the survey record for the close-up photo. Then, when the survey management table is updated, the image list display unit 34, as shown in FIG. 11 (a2), the newly registered close-up image survey record (survey ID = 5, photo number = B1- 2) the photo column 115a is displayed continuously below the photo column 115a of the survey record (survey ID = 4, photo number = B1) of the selection source image, and a close-up display mark 186 is displayed on the left side of the photo column 115a. indicate. Also, since there is no prior image for the newly added close-up image B1-2, the prior image display area 191 of the prior image for the close-up image B1-2 is empty.

  Here, the close-up image A1-1 is stored in association with the pre-image A1 for the posterior image B1 that is the selection source image, but the newly added close-up image B1-2 is a close-up of the existing pre-image. Note that it is not associated with the image A1-1 (see FIGS. 11A2 and 11B2). If the user takes a posterior image of the close-up image A1-1 of the pre-image, the unphotographed instruction displayed in the posterior image display area 192 next to the pre-image display area 191 of the close-up image A1-1. The image 181 may be selected (tapped).

  (B2) Next, the user selects (tap) the pre-image of the photo number B1 again or selects (tap) the post-image of the photo number B1-2, and then the close-up photography button on the screen of FIG. 9B. When the image 185 is selected (tapped) and a close-up photo is taken, the close-up image capturing unit 42 stores a search record (investigation ID) for the close-up photo in the search management table, as shown in FIG. = 6 survey record) is newly created, and an invalid value is set in the “overall survey ID” field of the survey record of the selection source image in the “overall survey ID” of the survey record (that is, selected) If the original image is photo number B1, the survey ID (4) of the survey record of the selected source image is set to a valid value in the “overall survey ID” field of the survey record of the selected source image. (I.e., the selected original image for photographic number B1-2) storing survey ID that is to be set to the "entire survey ID" field (4). Further, since there is no prior image for the close-up photograph, the “pre-survey ID” of the survey record stores a non-existing code 0 that is a number indicating that “the previous image does not exist for the subsequent image”. Thus, the survey record (survey ID = 6, photo number = B1-3) is associated with the survey record (survey ID = 4, photo number = B1) as a close-up image of the posterior image. At this time, the close-up image capture unit 42 copies the value of the corresponding field in the survey record of the selection source image B1 to the other field of the survey record for the close-up photo. Then, when the survey management table is updated, the image list display unit 34, as shown in FIG. 11 (a3), the newly registered close-up image survey record (survey ID = 6, photo number = B1- 3) The photo column 115a is displayed continuously below the photo column 115a of the survey record (survey ID = 4, photo number = B1) of the selection source image, and a close-up display mark 186 is displayed on the left side of the photo column 115a. indicate. Further, since there is no pre-image for the newly added close-up image B1-3, the pre-image display area 191 of the pre-image for the close-up image B1-3 is empty.

  Similarly, the close-up image for the photo number B1 can be sequentially added to the investigation management table and the photographed image list 115 in the photo management database 32.

(4.2) Supplementary explanation about survey photo additional shooting processing at the time of subsequent shooting Finally, a supplementary description of survey photo additional shooting processing at the time of shooting the subsequent image will be given. Note that the survey photo additional shooting process at the time of shooting the pre-image is as already described in step 5. In actual work damage investigation sites, there are cases where it is desired to add a new post image even if there is no pre image in the post survey. In such a case, the user may add a new posterior image by selecting (tapping) the current state photography button image 119 or the external photography button image 120 on the photographed image list screen of FIG. it can. Hereinafter, this process will be described.

  When the portable electronic device 1 automatically determines whether the current time is in the “previous image photographing state” or the “post-image photographing state”, the investigation management table in the photo management database 32 is referred to, For a record set in which the “record ID” field is the current survey record ID, a search is performed to determine whether there is a survey record in which a value other than an invalid value (NULL) is set in the “preliminary survey ID” field. If there is no survey record set with a value other than the invalid value (NULL), it is automatically determined that the current state is the “previous image shooting state”, and a survey record set with a value other than the invalid value (NULL) is set. If it exists, it is automatically determined that the current state is the “post-image shooting state”. For example, when the pre-images A1 and A2 and the post-image B1 are displayed in the photographed image list 115 as shown in FIG. 12 (a1) on the photographed image list screen in FIG. The portable electronic device 1 automatically determines that the current time is in the “post-image shooting state”. On the other hand, when the photographed image list 115 displays only the previous images A1 and A2 as shown in FIG. 12C and no posterior image is displayed, the portable electronic device 1 indicates that the current time is “prior”. It is automatically determined that the state is “image shooting state”.

  When the portable electronic device 1 is automatically determined to be in the “post-action image capturing state”, the user has selected (tapped) the external photography button image 120 from the captured image list screen of FIG. In this case, the external photo shooting instruction state is entered, and in the same manner as in step 5, the new point record adding unit 38, the normal finder screen display unit 39, and the new additional image capturing unit 40 are used to edit the whiteboard information, shoot the post image, and Image editing processing is performed. At this time, when registering as a new record in the survey management table of the photo management database 32, the newly added image capturing unit 40 displays “new survey record” in the “preliminary survey ID” field to indicate that the new record is a subsequent image. The non-existence code 0 which is a number indicating that “the previous image does not exist for the posterior image” is stored (refer to the investigation record (survey ID = 4, photo number = B3) in FIG. 12B2). Then, when the survey management table is updated, the image list display unit 34, as shown in FIG. 12 (a2), the newly registered post-image survey record (survey ID = 4, photo number = B3). The photographed image list 115 to which the photograph field 115a is added is displayed again on the photographed image list screen. Further, since there is no pre-image for the newly added post-image B3, the pre-image display area 191 of the pre-image for the close-up image B3 is empty, and “pre-” characters are light in the shooting type display area 194. “After” is displayed in dark color.

  When the portable electronic device 1 is automatically determined to be in the “post-action image shooting state”, the user has selected (tapped) the current photo shooting button image 119 from the shooting image list screen of FIG. The same applies to the case. In this case, a new survey record (survey ID = 4, photo number = B3) is added to the survey management table of the photo management database 32 as shown in FIG. As shown in (a3), the captured image list 115 to which the photograph field 115a of the newly registered post-image survey record (survey ID = 4, photo number = B3) is added is displayed again on the captured image list screen. . In this case, since there is no pre-image for the newly added post-image B3, the pre-image display area 191 of the pre-image for the close-up image B3 is empty, and the “current” character is dark in the shooting type display area 194. Is displayed.

  When the portable electronic device 1 is automatically determined to be in the “previous image shooting state” (for example, when the subsequent image has not been shot yet as shown in FIG. 12C). However, there are cases where it is desired to add a new survey record as a post-mortem image. In such a case, the portable electronic device 1 determines whether a new survey record should be added as a prior image or a new survey record as a subsequent image on the captured image list screen of FIG. Judgment is made based on whether the user has selected the current state photography button image 119 or the external photography button image 120 by a single tap or a double tap. When the selection is made by a single tap, the portable electronic device 1 maintains the “previous image shooting state” automatically determined. On the other hand, when selected by double tap, the portable electronic device 1 changes the state from “pre-image shooting state” to “post-image shooting state”, and then adds a new post-image as described above. Execute the process.

  In the present embodiment, an embodiment will be described in which an RF tag is used to automatically capture the position of an object to be imaged. The hardware configuration of the portable electronic device 1 of the present embodiment is the same as that of FIG. 1 and FIG. The functional configuration is the same as that of FIG. 3 of the first embodiment except for the position detection function of the photographing position calculation unit 33 and the table structure of the investigation management table of the photo management database 32. Therefore, description of the same parts as those in the first embodiment is omitted.

  FIG. 13 is a diagram illustrating the structure of the investigation management table in the photo management database 32 according to the second embodiment. Compared to FIG. 4 of the first embodiment, a “target position” field is added, and the rest is the same as the first embodiment. In this “object position” field, the object to be photographed (for example, “crack / breakage”, “edge break”, “peeling”, “clearance”, “deviation”, “dirt break”, “leakage trace” ") Is stored in absolute position information (latitude, longitude, altitude).

  In the present embodiment, the photographing position calculation unit 33 is realized by using the position information obtained by the GNSS module 4 (and the atmospheric pressure sensor 9) or the short-range wireless communication module 6 as in the case of the first embodiment. The current absolute position (latitude / longitude / altitude) is detected from position information obtained by LPS. Further, the direction (direction vector) of the portable electronic device 1 is detected by the direction sensor 8. Here, the azimuth sensor 8 basically detects the azimuth by measuring the terrestrial magnetism in three axial directions with a geomagnetic sensor. In this case, the correct azimuth can be detected by correcting with the acceleration sensor 7 or the gyro sensor 10 (see Non-Patent Document 7). Further, the shooting position calculation unit 33 of the present embodiment detects the distance to the RF tag installed at the position of the object to be imaged, and the visual axis direction of the shooting visual axis of the camera device 3 from the direction of the portable electronic device 1. Are detected, and the absolute position of the RF tag is calculated based on the detected absolute position, the distance to the RF tag, and the visual axis direction. In this case, an RF tag having a frequency band corresponding to the short-range wireless communication module 6 provided as a standard in the main body may be used, or a dedicated adapter for reading the RF tag is attached to the main body. The short-range wireless communication module 6 may be used. Hereinafter, detection of the absolute position of the RF tag installed at the position of the object to be imaged will be described.

  FIG. 14 is a diagram illustrating an example of a state in which the object to be photographed is photographed using the portable electronic device 1 according to the second embodiment. FIG. 14A shows an example of photographing using a crack scale with an RF tag, and FIG. 14B shows an example of photographing using a pointing stick with an RF tag. It is assumed that the object A is a crack generated in the wall surface S. In FIG. 14 (a), when shooting the object A, the user assigns the cracked object 210 to the object A and the crack scale 210. Here, the crack scale 210 is provided with a seal surface that can be repeatedly applied to the wall surface S, and the user can take a picture with the crack scale 210 attached to the wall surface S. The crack scale 210 includes an RF tag 211 in which ID information is embedded, and wireless communication can be performed with the short-range wireless communication module 6 of the portable electronic device 1. FIG. 14B is an example in which the pointing stick 212 is used in a place where it is difficult to attach the crack scale 210 to the object A (for example, a ceiling, a dirt). At the time of shooting, a user's assistant places a pointing stick 212 with a built-in RF tag 211 at the tip of the object A.

When shooting is performed in this state (the shutter button image 106 is selected in FIG. 7C, FIG. 8C, or FIG. 9A), the shooting position calculation unit 33 is the same as that described in the first embodiment. Similarly, the electronic viewfinder image, the shooting date / time information output from the internal clock 25, the direction (direction vector) of the portable electronic device 1 output from the direction sensor 8, and the GNSS module 4 (and the atmospheric pressure sensor 9) are obtained. The current absolute position (latitude / longitude / altitude) is acquired from the position information or position information obtained by LPS realized using the short-range wireless communication module 6. Then, the short-range wireless communication module 6 performs wireless communication with the RF tag 211 installed at the position of the object A, and detects the distance L to the RF tag 211. The distance detection technology to the RF tag 211 is currently widely used in tag readers in the technical field of RFID (radio frequency identifier) (see, for example, Non-Patent Document 6 and Patent Document 5). For example, the distance L between the near field communication module 6 and the RF tag 211 is detected from the intensity of the reflected wave reflected from the RF tag 211 with respect to the transmission wave emitted from the near field communication module 6 to the RF tag 211. It is possible to detect the distance L from the phase difference between the transmitted wave and the reflected wave. Since the camera device 3 is fixedly incorporated in the portable electronic device 1, the viewing axis direction vector u = (u _x , u _y , of the photographing viewing axis of the camera device 3 is determined from the orientation of the portable electronic device 1. u _z ) (‖u‖ = 1) can be calculated. Therefore, from the absolute coordinates P = (p _x , p _y , p _z ) of the portable electronic device 1 (the latitude, longitude, and altitude converted into absolute coordinates), the visual axis direction vector u, and the distance L, the RF tag 211 Can be easily obtained by Q = P + L · u. The photographing position calculation unit 33 outputs the absolute position information (the absolute coordinate Q converted into latitude, longitude, and altitude) of the RF tag 211 obtained in this way. The post-image capture unit 37, the newly added image capture unit 40, or the close-up image capture unit 42 saves the pre-image, the post-image, or the close-up image in the image file storage unit 31. The absolute position information and the shooting date and time information are embedded and stored in the image. At the same time, when a record related to the image is stored in the investigation management table of the photo management database 32, the absolute position information of the RF tag 211 is stored in the “target position” field.

  As described above, in the portable electronic device 1 according to the present embodiment, the absolute position information of the RF tag 211 installed at the position of the object A to be photographed is captured at the time of photographing, and is embedded in the photographed image and photographed. Management database 32 By storing the image in association with the image, it is no longer necessary to manage the location of the object A by writing it separately on a separately prepared drawing of the building, etc., and management work can be saved significantly. It becomes.

  In the present embodiment, a description will be given of an embodiment in which notification or automatic shooting is performed when the shooting angle coincides with the previous image at the time of shooting the posterior image. The hardware configuration of the portable electronic device 1 of the present embodiment is the same as that of FIG. 1 and FIG. Also, the functional configuration is the same as that of FIG. 3 of the first embodiment except for a part of the operations of the finder screen display unit 36 with the underlay and the post image capturing unit 37. Therefore, description of the same parts as those in the first embodiment is omitted.

  The underlay finder screen display unit 36 of the present embodiment displays a pre-image (hereinafter referred to as “corresponding pre-image”) that is the source of the underlay image displayed in the viewfinder area 201 on the electronic viewfinder screen of FIG. The displacement vector V (I1, I2 (t)) of the electronic viewfinder image I2 (t) (t represents time) for I1 is calculated in real time. Here, the displacement vector V (I1, I2 (t)) = (P (I1, I2 (t)), R (I1, I2 (t))) is a translation component P (I1, I2 (t)). ) And a rotational movement component R (I1, I2 (t)). Also, a known method is used to calculate the positional deviation vector V (I1, I2 (t)). For example, a method of calculating a positional deviation vector V (I1, I2 (t)) having the largest correlation coefficient by performing an image correlation operation between the corresponding prior image I1 and the electronic viewfinder image I2 (t) is used. it can. Further, a known method such as a template matching (block matching) method, an active search method, a mean shift method, or a particle filter that is used in calculation of subject tracking can be applied (see Non-Patent Document 8).

  The underlay finder screen display unit 36 moves the direction of the portable electronic device 1 according to the calculated displacement vector V (I1, I2 (t)) as shown in FIG. Shooting angle correction instruction marks 155a and 155b that indicate the power direction are displayed on the screen of the touch screen device 2. Here, the shooting angle correction instruction mark 155a represents the direction in which the camera device 3 should be translated, and the shooting angle correction instruction mark 155b represents the direction in which the camera device 3 should be rotated. In this way, by displaying the shooting angle correction instruction marks 155a and 155b, the user can see at a glance which direction the portable electronic device 1 should be moved to match the shooting angle, The efficiency of the alignment work is improved.

  On the other hand, the posterior image capturing unit 37 refers to the displacement vector V (I1, I2 (t)) calculated in real time by the finder screen display unit 36 with the underlying sheet, and the positional displacement amount of the electronic viewfinder image with respect to the underlying image. It is checked whether (norm of misregistration vector) ‖ V (I1, I2 (t)) 下 is below a predetermined threshold ε. Note that the threshold value ε is appropriately set in consideration of the actual allowable amount of blurring. When the value falls below the threshold ε, the posterior image capturing unit 37 displays a shooting angle match mark 156 informing that the shooting angle is matched in the screen of the touch screen device 2 as shown in FIG. indicate. Thereby, the user can glance at whether or not the shooting angles are well matched and can immediately visually recognize it, thereby improving the efficiency of the alignment work.

  In this embodiment, the posterior image capturing unit 37 detects the shooting angle coincidence mark 156 when the amount of displacement ‖V (I1, I2 (t)) ‖ of the electronic viewfinder image with respect to the underlying image is less than the threshold ε. However, instead of displaying the shooting angle match mark 156, an alarm (electronic sound) may be sounded to notify the user.

  Further, instead of notifying with the shooting angle coincidence mark 156 or the like, the shutter is automatically released, and the electronic viewfinder image I2 (at time t2 when the positional deviation amount 位置 V (I1, I2 (t)) ‖ falls below the threshold ε ( In addition to capturing t2) as a posterior image, the photographic date / time information of the posterior image captured from the internal clock 25 and the shooting location information of the corresponding pre-image are captured, and the process proceeds to the editing screen of FIG. You can also

  In this embodiment, an example will be described in which the portable electronic device of the present invention is applied to a fixed point survey for recording a change over time such as a crack in a dam or a tunnel. The difference from the case of the industrial damage survey is that in the case of the industrial damage investigation, it is only necessary to manage the two images before and after the same survey point. It is necessary to manage two or more images to be recorded.

  The portable electronic device 1 of the present embodiment is basically the same as the portable electronic device 1 of the second embodiment, except for a part of the function of the image list display unit 34. Here, differences from the second embodiment of the portable electronic device 1 of the present embodiment will be described.

  In the portable electronic device 1 according to the present embodiment, in addition to the current photography button image 119 and the external photography button image 120 on the photographed image list screen as shown in FIG. 6B (FIG. 7A). Then, another fixed point photography button image 121 is added (see FIG. 18). When the user performs additional photography of a survey photograph on the photographed image list screen of FIG. 18, a current photograph button image 119 or an external photograph button button 120 or a fixed point photograph button on the photographed image list screen of the touch screen device 2. The image 121 (these three are collectively referred to as “list addition button image”) is selected. Here, when the current state photography button image 119 is selected, the current state photography instruction state is entered, and when the external photography button image 120 is selected, the state is an external photography instruction state, and the outside photography button image is displayed. When 120 is selected, a fixed-point photography instruction state is entered. When the current state photography button image 119, the external photography button image 120, or the fixed point photography button image 121 is selected, the whiteboard display / editing unit 43 inputs the photographing point information of FIG. 7B on the touch screen device 2. Display the screen. As described above, in this embodiment, a mode of “fixed point photography” is newly added.

  The structure of the photo management database 32 in the portable electronic device 1 of this embodiment is the same as that shown in FIG. FIG. 16 is a diagram illustrating an example of specific data contents of the investigation management table of the photo management database 32 in the portable electronic device 1 according to the fourth embodiment. FIG. 16A shows a specific example of the contents of the survey management table, and FIG. 16B shows the relationship between a series of survey records for one fixed point among the survey records stored in the survey management table. It is. As shown in FIG. 16, in the fixed point survey (fixed point photography instruction state), the “shooting type” field of the survey management table is set to “fixed point”. In the “preliminary survey ID” field, the survey ID of the survey record of the survey performed immediately before the survey record is stored. For example, when a fixed point A is subjected to the first fixed point survey on March 1, 2015, the second fixed point on May 1, 2015, and the third fixed point survey on July 1, 2015, the fixed point A An invalid value (NULL) is set in the “preliminary survey ID” of the survey record of the first fixed point survey of (see the survey record of survey ID (1) in FIG. 16), and the second fixed point of fixed point A The survey ID of the first survey record is set in the “preliminary survey ID” of the survey record of the survey (refer to the survey record of survey ID (14) in FIG. 16), and the third fixed point survey of fixed point A The survey ID of the second survey record is set in the “preliminary survey ID” of the survey record (see the survey record of survey ID (32) in FIG. 16). In this way, the relationship between a series of survey records for one fixed point is performed.

  FIG. 17 is a diagram illustrating a display example of one photo column 115a for a fixed point survey in the photographed image list screen in the portable electronic device 1 according to the fourth embodiment. In this embodiment, as in the second embodiment, the photographed image list screen is displayed in the same manner as in FIG. 8D, and FIG. 17 shows one photo field 115a in the photographed image list 115 in the photographed image list screen. Represents. FIG. 17A shows a photo column 115a in a state where photographing of the posterior image in the current survey is not completed, and FIG. 17B shows a photo column 115a in a state where photographing of the posterior image in the current survey is completed.

  The image list display unit 34 first sets the survey date of the current survey to the current date and time output by the internal clock 25 when displaying the survey record whose “imaging type” is “fixed” in the captured image list 115. . Then, a record set of survey records having the same “photo number” is extracted from the survey management table, and if there is a survey record having the same “date and time” as the survey date in the extracted record set, a series of records of the record set is extracted. In the fixed point survey, it is determined that the shooting of the posterior image has been completed. If there is no survey record whose “date and time” is the same as the survey date in the extracted record set, it is determined that the posterior image has not been captured in a series of fixed point surveys of that record set.

  When it is determined that the shooting of the subsequent image is incomplete, the image list display unit 34 extracts the survey record with the latest “date and time” in the record set (hereinafter referred to as “tail survey record”). Then, the image file for the end survey record is read from the image file storage unit 31 and displayed in the pre-image display area 191 of the photo column 115a for the record set. In addition, the unphotographed instructional image 181 is displayed in the posterior image display area 192 of the photo column 115a for the record set (see FIG. 17A).

  On the other hand, when it is determined that the posterior image capturing is completed, the image list display unit 34 refers to a survey record in which “date and time” is the same as the survey date in the record set (hereinafter referred to as “tail survey record”). ) Is extracted from the image file storage unit 31 and displayed in the posterior image display area 192 of the photo column 115a for the record set. In addition, the image list display unit 34 has the latest survey record (hereinafter referred to as “second end”) in the record set whose “date and time” is earlier than the survey date (hereinafter collectively referred to as “preliminary image”). "Investigation record") is extracted, and the image file for the second end investigation record is read from the image file storage unit 31 and displayed in the pre-image display area 191 of the photo column 115a for the record set.

  The image list display unit 34 displays “fixed point” in the photo number display area 195 of the photo column 115a for a series of fixed point survey record sets, and the place / photographed property name display area. In 193, the place name and the name of the object to be copied are displayed, and the “date” of the image displayed in the pre-image display area 191 is displayed. Further, record move button images 197 and 198 are displayed on the left and right of the prior image display area 191.

  In FIG. 17A or FIG. 17B, when the record movement button image 197 is selected by the user, the image list display unit 34 is currently displayed in the previous image display area 191 in the previous image. The previous image of the shooting date and time before the existing image is displayed in the previous image display area 191, and the display of the place / photographed property name display area 193 is updated accordingly. When the record movement button image 198 is selected by the user, the image list display unit 34 displays the previous image of the shooting date and time one after the image currently displayed in the previous image display area 191 in the previous image. Is displayed in the prior image display area 191 and the display of the place / photographed article name display area 193 is updated accordingly. In this way, the user can switch and confirm the display of the prior image.

  Further, when the unphotographed instruction image 181 is selected by the user in FIG. 17A, the posterior image photographing operation is executed by the same procedure as that described in step 7 of the first embodiment. At this time, the base image is created using the image displayed in the prior image display area 191 when the unphotographed instruction image 181 is selected.

  As described above, the portable electronic device of the present invention can also be applied to a fixed point survey in which the same spot is repeatedly photographed and recorded a plurality of times.

  In the present embodiment, an embodiment of a portable electronic device having an automatic calculation and recording function of subject size information will be described. Since the configuration of the portable electronic device in the present embodiment is basically the same as that of the first embodiment, differences from the first embodiment will be mainly described here. In the present embodiment, an example in which the portable electronic device according to the present invention is applied to the industrial damage investigation will be described.

  FIG. 19 is a block diagram illustrating a functional configuration of the portable electronic device of the fifth embodiment configured by installing an application program in the portable electronic device of FIG. 19, the touch screen device 2, the camera device 3, the GNSS module 4, the short-range wireless communication module 6, the acceleration sensor 7, the direction sensor 8, the atmospheric pressure sensor 9, the gyro sensor 10, the internal clock 25, the image file storage unit 31, Photo management database 32, photographing position calculation unit 33, image list display unit 34, photographed image display unit 35, underlay finder screen display unit 36, post image capture unit 37, new point record addition unit 38, normal finder screen display The unit 39, the newly added image capturing unit 40, the close-up finder screen display unit 41, the close-up image capturing unit 42, the whiteboard display / editing unit 43, and the image editing unit 44 correspond to the components having the same reference numerals in FIG. ing. In the present embodiment, a size conversion ratio calculation unit 45 and a size measurement unit 46 are newly added.

  The size conversion ratio calculation unit 45 is registered in advance in a captured image captured as a posterior image, a prior image, or a current image by the posterior image capturing unit 37, the newly added image capturing unit 40, or the close-up image capturing unit 42. When a scale reference image serving as a reference for the scale is searched and the scale reference image is searched, the actual object A for the size on the photographed image is based on the size of the scale reference image in the photographed image. The size conversion ratio R, which is the ratio of the sizes of, is calculated.

  When the size conversion ratio is calculated by the size conversion ratio calculation unit 45, the size measurement unit 46 is a captured image on the image editing screen displayed on the touch screen device 2 by the image editing unit 44, and is touched by the user. When two points are designated using the device 2, a distance L1 between the two points on the captured image is calculated. Then, the distance L1 is multiplied by the size conversion ratio R to calculate the distance L2 between the two points in the actual object A and displayed in the image editing screen.

  In the portable electronic device of the present embodiment configured as described above, the operation centering on the automatic calculation and recording function of subject size information will be described below. In the present embodiment, when automatic calculation of subject size information is performed, it is necessary to photograph the scale reference object P, which serves as a scale reference, together with the object A when the object A is photographed. FIG. 20 is a diagram illustrating an example of photographing the scale reference object P together with the object A to be photographed. In the present embodiment, a crack scale 210 with a slide case as shown in FIG. The crack scale 210 includes a slide case 210a and a scale plate 210b. The scale plate 210b is formed of a colorless and transparent resin so as to be freely bent, is formed in a rectangular sheet shape, and scales are displayed along the long sides. The slide case 210a has a flat long box shape, a rectangular through window 210c is formed at the center, and an insertion hole through which the scale plate 210b is inserted is formed on the left and right side surfaces. The scale plate 210b is slidably attached to the slide case 210a through the insertion holes on the left and right side surfaces of the slide case 210a. Moreover, the adhesive part is provided in the back surface of the slide case 210a, and it can be peeled off with respect to the wall surface S where the to-be-photographed article A exists. Furthermore, detection marks 210d and 210d are displayed on the left and right of the through window 210c in front of the slide case 210a. The detection marks 210d and 210d are marks for identifying the scale reference image in the photographed image. For example, the detection marks 210d and 210d are characteristic features that can be easily distinguished from other parts such as a two-dimensional bar code and a characteristic figure. It is marked. At the time of shooting, as shown in FIG. 20A, a crack scale 210 is attached near the center of the object A to be cracked, and the scale of the scale plate 210b is the width of the object A to be photographed. This is photographed by the camera device 3 of the portable electronic device 1 as a state matched to the above. At this time, in order to reduce the error in the size conversion calculation on the photographed image due to the lens aberration or the like of the camera device 3, the crack scale 210 should be on the visual axis of the camera device 3 as much as possible and the view of the camera device 3 as much as possible. Photographing is performed so that the axis is perpendicular to the wall surface S.

  In this way, when the user selects the shutter button image 106 from the touch screen device 2 on the electronic viewfinder screen shown in FIG. 7C or 9A, the newly added image capturing unit 40 is selected. Alternatively, the posterior image capturing unit 37 or the close-up image capturing unit 42 captures the electronic viewfinder image at the time of selection as a pre-image, and outputs the current position information output from the photographing position calculation unit 33 and the internal clock 25. Capture the current date and time information. Then, the image editing unit 44 in this embodiment displays the image editing screen in FIG. 21A on the touch screen device 2. In the image editing screen, the pre-image captured in the central image display area 161 is displayed, and a white board image 153 is superimposed and displayed at any one of the left, right, upper and lower corners. The white board image 153 is displayed on the touch screen device by the user. The position in the image display area 161 can be moved by swiping the white board image 153 on the same as in the first embodiment. Also, on the right side of the image display area 161, an editing operation selection button image 162, a re-shooting button image 108, and a save button image 130 are displayed. When the user taps the editing operation selection button image 162, FIG. Similar to the first embodiment, the image editing screen (a) (image editing screen in the pointer editing mode) and the image editing screen (image editing screen in the pen editing mode) in FIG. is there. In this embodiment, in addition to this, the image editing unit 44 displays a pointer group selection button image 165 and a measurement mode selection button image 166 on the right side of the image display area 161. The pointer group selection button image 165 and the measurement mode selection button image 166 are displayed only on the image editing screen in the pointer editing mode.

  The pointer group selection button image 165 is a palette button image for selecting a group of arrow images (pointers). The group of arrow images is displayed in color on the image editing screen. The user can select a group (color) of arrow images used for editing by selecting a color from the palette displayed on the pointer group selection button image 165. The measurement mode selection button image 166 is a toggle button for switching between the distance measurement mode and the non-measurement mode. Here, the “distance measurement mode” refers to an operation mode in which automatic calculation and recording of subject size information is performed by the size conversion ratio calculation unit 45 and the size measurement unit 46, and the “non-measurement mode” refers to subject size information. An operation mode in which automatic calculation and recording are not executed. When the user taps the measurement mode selection button image 166 on the touch screen device 2, the distance measurement mode and the non-measurement mode can be switched alternately. In the distance measurement mode, the color of the measurement mode selection button image 166 is changed to a selected color (bright and dark), and in the non-measurement mode, the color of the measurement mode selection button image 166 is changed to a non-selected color (dark and light). Is done.

  Since the operation of the image editing unit 44 in the non-measurement mode is the same as that of the first embodiment, the description thereof is omitted. In this embodiment, the color of the arrow image displayed in the image display area 161 of the image editing screen is the color selected by the pointer group selection button image 165, and a plurality of colors are displayed in the same image display area 161. It is possible to add an arrow image.

  An editing operation when the distance measurement mode is selected by the user will be described below. When the distance measurement mode is selected, the measurement mode selection button image 166 becomes a selected color (bright color). Then, the size conversion ratio calculation unit 45 searches for a scale reference image 167 serving as a reference for a scale registered in advance in the captured image displayed in the image display area 161. In the present embodiment, the scale reference image 167 is an image of the slide case 210 a of the crack scale 210. Here, as described above, the detection marks 210d and 210d are displayed on the left and right sides of the through window 210c in front of the slide case 210a. Therefore, the size conversion ratio calculation unit 45 can detect the detection marks 210d and 210d in the captured image. The image of the slide case 210a can be easily identified and extracted by searching for the image.

  When the scale reference image 167 cannot be extracted, the size conversion ratio calculation unit 45 displays an error message (for example, a pop-up box “Scale reference image cannot be extracted!”) On the image editing screen. Return to the non-measurement mode. On the other hand, when the scale reference image 167 can be extracted, the size conversion ratio calculation unit 45 calculates the actual size of the object A with respect to the size on the photographed image based on the size of the scale reference image 167 in the photographed image. The size conversion ratio R, which is the ratio, is calculated. For example, the length of the scale reference image 167 in the longitudinal direction on the captured image is w1 pixels. Also, assume that the actual length of the slide case 210a in the longitudinal direction is w2 (see FIG. 20A). At this time, the size conversion ratio R is R = w2 / w1.

Next, the user selects a group of arrow images (pointers) with the pointer group selection button image 165 and designates a position where the arrow image is to be placed on the captured image in the image display area 161. FIG. 21B shows an example of a state where the positions of five arrow images are designated by the user. In FIG. 21B, it is assumed that the positions of the arrow images are designated in the order of circled numbers 1 to 5. When the position of the second and subsequent arrow images Y _i (i = 2, 3,...) Is designated, the size measurement unit 46 displays the image on the captured image from the position of the arrow image Y _i−1 designated before that. The distance L1 (Y _i−1 , Y _i ) is calculated by multiplying the distance L1 (Y _i−1 , Y _i ) by the size conversion ratio R and the distance L2 between the two points in the object A Y _i−1 , Y _i ) = R · L 1 (Y _i−1 , Y _i ) is calculated. Then, the calculated value of the distance L2 (Y _i−1 , Y _i ) is displayed at an intermediate position between the arrow images Y _i−1 and Y _i on the captured image in the image display area 161. At the same time, the size measurement unit 46 writes the calculated distance L2 (Y _i−1 , Y _i ) in the “length” field (see FIG. 4) of the investigation record for the captured image in the photo management database 32. Store as column data. The stored character string data in the “length” field is simultaneously reflected in the white board image 153. For example, in the example of FIG. 21B, (L2 (Y ₁ , Y ₂ ), L2 (Y ₂ , Y ₃ ), L2 (Y ₃ , Y ₄ ), L2 (Y ₄ , Y ₅ )) = ( 51.1, 39.1, 28.9, 27.9). In this case, the character string “51.1 + 9.1 + 28.9 + 27.9” is stored in the “length” field of the survey record, and this character string is the length of the white board image 153. This is also reflected in the column (S).

Next, when the crack in the photographed image is branched in the middle, if the user wants to measure the distance from another point after discontinuing the distance measurement, the user selects the pointer group selection button image. After selecting an arrow image of another pointer group by 165, the position where the arrow image is to be placed is designated on the captured image in the image display area 161. FIG. 22 shows an example of a state in which the position of two arrow images is further specified by the user after switching the pointer group from the state of FIG. In this example, it is assumed that the positions of the arrow images are designated in the order of the encircled numbers 6 and 7. In this case, when the position of the second and subsequent arrow images Y _i (i = 7,...) Is designated after changing the pointer group, the size measuring unit 46 designates the arrow image Y _i−1 designated before that. The distance L1 (Y _i−1 , Y _i ) on the photographed image from the position of the image is calculated, and the distance L1 (Y _i−1 , Y _i ) is multiplied by the size conversion ratio R so that The distance L2 (Y _i−1 , Y _i ) = R · L1 (Y _i−1 , Y _i ) between the two points is calculated. Then, the calculated value of the distance L2 (Y _i−1 , Y _i ) is displayed at an intermediate position between the arrow images Y _i−1 and Y _i on the captured image in the image display area 161. At the same time, the size measuring unit 46 stores the calculated distance L2 (Y _i−1 , Y _i ) as character string data in the “length” field of the investigation record for the captured image in the photo management database 32. The stored character string data in the “length” field is simultaneously reflected in the white board image 153. For example, in the example of FIG. 22, (L2 (Y ₁ , Y ₂ ), L2 (Y ₂ , Y ₃ ), L2 (Y ₃ , Y ₄ ), L2 (Y ₄ , Y ₅ ); L2 (Y ₆ , Y ₇ )) = (51.1, 39.1, 28.9, 27.9; 19.4) (where “;” represents the pointer group delimiter). In this case, the “length” field of the survey record contains a string “51.1 + 9.1 + 28.9 + 27.9, 19.4” is stored, and this character string is also reflected in the length (L) column of the white board image 153.

  When the image editing is completed, the user selects the save button image 130 on the image editing screen. Then, similar to the first embodiment, processing such as storage of a photographed image is performed by the posterior image capturing unit 37, the newly added image capturing unit 40, or the close-up image capturing unit 42.

As described above, in this embodiment, by using the scale reference image 167 in the photographed image, the two points of the photographed image on the image editing screen are designated by the touch screen device 2, so that the actual object A The corresponding distance L2 (Y _i , Y _i −1) between the two points can be automatically measured and recorded. Accordingly, it is possible to measure and record the distance on the image editing screen without actually measuring the object A with a ruler or a tape measure, and the work efficiency in the survey is improved.

  In the fifth embodiment, the embodiment related to the automatic calculation and recording function of the length (L) in the subject size information has been described, but the width (W) can be measured and recorded on the captured image. In the sixth embodiment, an embodiment of a portable electronic device having a width measurement and recording function will be described. The configuration of the portable electronic device of this embodiment is the same as that of the fifth embodiment, and only the function of the size measuring unit 46 is different.

  In this case, only the slide case 210a excluding the scale plate 210b is used as the scale reference object P in the crack scale 210 of FIG. Then, at the time of shooting the object A, the slide case 210a, which is the scale reference object P serving as a scale reference, is imaged together with the object A.

  In this way, when the user selects the shutter button image 106 from the touch screen device 2 on the electronic viewfinder screen shown in FIG. 7C or 9A, the newly added image capturing unit 40 is selected. Alternatively, the posterior image capturing unit 37 or the close-up image capturing unit 42 captures the electronic viewfinder image at the time of selection as a pre-image, and outputs the current position information output from the photographing position calculation unit 33 and the internal clock 25. Capture the current date and time information. Then, the image editing unit 44 in this embodiment displays the image editing screen in FIG. 23A on the touch screen device 2. The image editing screen of FIG. 23A is basically the same as that of FIG. 21A of the fifth embodiment, but the difference from the case of the fifth embodiment is that instead of the measurement mode selection button image 166. That is, the distance measurement mode selection button image 166a is provided, and the width measurement mode selection button image 166b is newly provided. Note that the distance measurement mode selection button image 166a in FIG. 23 (a) is only the display name of the measurement mode selection button image 166 in FIG. Since this is the same as the measurement mode selection button image 166 of FIG.

  The width measurement mode selection button image 166b is a toggle button for switching between the width measurement mode and the non-measurement mode. Here, the “width measurement mode” refers to an operation mode in which the size measurement unit 46 executes acquisition and recording of size information in the width direction of the object A. When the user taps the width measurement mode selection button image 166b on the touch screen device 2, the width measurement mode and the non-measurement mode can be switched alternately. When the user taps the width measurement mode selection button image 166b in the “distance measurement mode” state, the “distance measurement mode” is canceled and becomes the “width measurement mode”, and the user in the “width measurement mode” state. When the distance measurement mode selection button image 166a is tapped, the “width measurement mode” is canceled and the “distance measurement mode” is set. In the width measurement mode, the color of the width measurement mode selection button image 166b is changed to a selected color (bright and dark), and in the non-measurement mode, the color of the width measurement mode selection button image 166b is a non-selected color (dark and light). Changed to

  In addition, when the user performs a pinch-in operation (operation to move two fingers on the touch screen device 2 so as to pinch (narrow)) in the image display area 161 in the image editing screen, The image editing unit 44 displays the captured image displayed on the image display area 161 in a reduced size, and also performs a pinch-out operation from the touch screen device 2 (put two fingers on the touch screen device 2 and When the image editing unit 44 performs an operation of moving the image to expand the image, the image editing unit 44 displays the captured image displayed on the image display area 161 in an enlarged manner. In the editing screen of FIG. 23B, the vicinity of the scale reference image 167 is enlarged and displayed by a pinch-out operation from the editing screen of FIG.

  The editing operation when the width measurement mode is selected by the user will be described below. When the width measurement mode is selected, as shown in FIG. 23B, the width measurement mode selection button image 166b becomes a selected color (bright and dark color). At the same time, a scale image 168 for width measurement is displayed at the center of the through window 210c of the image of the slide case 210a which is the scale reference object P. This width measurement scale image 168 corresponds to a scale attached to the scale plate 210b of the crack scale 210. The size of the scale image for width measurement 168 on the image editing screen is calculated from the relative size ratio with the scale reference image 167. In the image editing screen in FIG. 23B, seven width measurement scale images 168 are displayed, and the center width measurement scale image 168 is regarded as the currently selected scale. When the user performs a swipe operation or flip operation from the touch screen device 2 on the width measurement scale image 168, the same movement as sliding the scale plate 210b of the crack scale 210 in accordance with the movement of the finger. Thus, the width of the scale displayed by scrolling each of the width measurement scale images 168 changes (see the scale enlarged view on the left side of FIG. 23B). In this way, by selecting the width measurement scale image 168 suitable for the crack width in the captured image, the width (W) can be measured and recorded on the captured image. The size measuring unit 46 stores the width W of the selected scale image for width measurement 168 as character string data in the “width” field (see FIG. 4) of the investigation record for the photographed image in the photo management database 32. The stored character string data in the “width” field is also reflected in the white board image 153 at the same time. For example, in the example of FIG. 23B, W = 0.95 mm is selected. In this case, the character string “0.95” is stored in the “width” field of the survey record, and this character string is a whiteboard image. This is also reflected in the width (W) field of 153.

  As described above, in this embodiment, the width measurement scale image 168 displayed together with the scale reference image 167 in the photographed image is used to adjust the size of the photographed image on the image editing screen with the width of the object A. And measuring the width W, the width W on the actual object A can be measured and recorded. Accordingly, it is possible to measure and record the width W on the image editing screen without performing actual measurement with the crack scale 210 on the object A, and work efficiency in the investigation is improved.

DESCRIPTION OF SYMBOLS 1 Portable electronic device 2 Touch screen device 3 Camera device 4 GNSS module 5 Mobile line communication module 6 Short-range wireless communication module 7 Acceleration sensor 8 Direction sensor 9 Pressure sensor 10 Gyro sensor 19 System bus 20 I / O subsystem 20a Display Controller 20b Camera input controller 20c GNSS controller 20d Mobile line communication controller 20e Short-range wireless communication controller 21 Storage device 22 Storage controller 23 Peripheral interface 24 Processor 25 Internal clock 26 Memory 27 Memory bus 31 Image file storage unit 32 Photo management database 33 Shooting position calculation unit 34 Image list display unit 35 Captured image display unit 36 Underlay finder screen display unit 37 Subsequent Image capturing unit 38 New point record adding unit 39 Normal viewfinder screen display unit 40 New additional image capturing unit 41 Close-up viewfinder screen display unit 42 Close-up image capturing unit 43 Whiteboard display / editing unit 44 Image editing unit 45 Size conversion ratio calculation Unit 46 Size measurement unit 51 Login unit 52 Survey list display unit 53 New survey registration unit 54 Survey selection unit 55 Owner registration unit

Claims (9)

A portable electronic device for photography and management comprising a touch screen device, a camera device, and an internal clock,
Image storage means for storing an image photographed by the camera device as an image file;
A photo management database for managing images stored in the image storage means in association with image management information including shooting date / time information of the image and shooting location information;
Read each image management information registered in the photo management database,
Images taken at the same shooting location at each shooting date and time are arranged in the order of a prior image whose shooting date and time is the previous image and a subsequent image whose shooting date and time is the later image. Create a list and display the captured image list on the screen of the touch screen device,
In the photographed image list, when the posterior image is not stored in the image storage means, an image that displays an unphotographed instruction image indicating that it has not been photographed instead of the posterior image. List display means;
In the photographed image list, when the unphotographed instructional image in the photographed image list is selected by the touch screen device,
An electronic viewfinder image captured in real time from the camera device is superimposed on the base image generated from the prior image of the image set to which the unphotographed instructional image belongs to generate a finder superimposed image,
Underlay finder screen display means for displaying an underlay finder screen including the finder superimposed image and a shutter button image indicating a tap operation position for inputting an imaging execution instruction on the screen of the touch screen device;
When the shutter button image is selected by the touch screen device on the underlay finder screen, the electronic viewfinder image at the time of selection is stored in the image storage means as the posterior image, and from the internal clock Post-registration of the posterior image in the photo management database in association with image management information including shooting date / time information of the posterior image to be captured and shooting location information of the pre-image used as the underlying image of the finder superimposed image An image capturing means,
The image list display means, when the posterior image is newly registered in the photo management database by the posterior image capturing means, the touched image list including the newly registered posterior image as the touch list. A portable electronic device characterized by being redisplayed on a screen of a screen device. In the captured image list, when the pre-image in the captured image list is selected by the touch screen device, the pre-image is read from the image storage unit and displayed on the touch screen device, and close-up shooting A photographed image display means for displaying a close-up button image indicating a tap operation position for inputting an execution instruction on the touch screen device;
When the close-up button image is selected by the touch screen device, an electronic viewfinder image captured in real time from the camera device and a tap operation position for inputting an imaging execution instruction to the touch screen device are displayed. Close-up viewfinder screen display means for displaying a close-up viewfinder screen including
When the shutter button image is selected by the touch screen device on the close-up viewfinder screen, the electronic viewfinder image at the time of selection is stored in the image storage unit as the pre-image and is taken from the internal clock The pre-image shooting date and time information, and the image management information including the shooting location information of the pre-image displayed on the touch screen device when the close-up button image was previously selected. Close-up image capturing means for registering an image in the photo management database,
When the pre-image is newly registered in the photo management database by the close-up image capturing unit, the image list display unit displays the photographed image list including the pre-registered new image as the touch screen. 2. The portable electronic device according to claim 1, wherein the portable electronic device is displayed again on the screen of the device.   The underlay finder screen display means calculates, in real time, a position displacement vector of the electronic viewfinder image with respect to the prior image that is the source of the underlay image on the underlay finder screen, and calculates the calculated position displacement vector. 3. An imaging angle correction instruction means for displaying an imaging angle correction instruction mark for instructing a direction in which the direction of the camera device should be moved is displayed on the screen of the touch screen device. The portable electronic device as described. The underlay finder screen display means calculates, in real time, a position displacement vector of the electronic viewfinder image with respect to the prior image that is the source of the underlay image on the underlay finder screen,
The posterior image capturing means refers to the displacement vector calculated by the underlay finder screen display means, and when the amount of displacement of the electronic viewfinder image with respect to the underlay image falls below a predetermined threshold, The portable electronic device according to any one of claims 1 to 3, further comprising shooting angle match notifying means for notifying that the shooting angles are matched. The underlay finder screen display means calculates, in real time, a position displacement vector of the electronic viewfinder image with respect to the prior image that is the source of the underlay image on the underlay finder screen,
The posterior image capturing means refers to the displacement vector calculated by the underlay finder screen display means, and when the amount of displacement of the electronic viewfinder image with respect to the underlay image falls below a predetermined threshold, The electronic viewfinder image at the time of the selection was stored in the image storage means as the posterior image, and was used as the underlying image of the finder superimposed image and the shooting date / time information of the posterior image captured from the internal clock The portable electronic device according to any one of claims 1 to 3, wherein the posterior image is registered in the photo management database in association with image management information including shooting location information of the prior image. A positioning module that measures the current position;
A short-range wireless communication module for performing short-range wireless communication;
An orientation sensor that detects the current orientation;
The current absolute position is detected by the positioning module,
Wireless communication is performed with an RF tag installed at a position to be imaged by the short-range wireless communication module, and a distance to the RF tag is detected.
Detecting the visual axis direction of the photographing visual axis of the camera device by the orientation sensor,
Photographing position calculation means for calculating the absolute position of the RF tag based on the detected absolute position, the distance to the RF tag, and the visual axis direction,
The image list display means displays, on the screen of the touch screen device, a list addition button image indicating a tap operation position for inputting a list addition instruction for adding a new pre-image in addition to the photographed image list. And
When the list addition button image is selected by the touch screen device, a shutter button indicating a tap operation position for inputting an electronic viewfinder image captured in real time from the camera device and an imaging execution instruction to the touch screen device. Normal finder screen display means for displaying a normal finder screen including an image;
When the shutter button image is selected by the touch screen device on the normal viewfinder screen, the electronic viewfinder image at the time of selection is stored in the image storage unit as the pre-image and is taken from the internal clock The new image is registered in the photo management database in association with the image management information including the shooting date and time information of the shot image and the shooting location information including the absolute position of the RF tag calculated by the shooting position calculation means. An additional image capturing means,
When the previous image is newly registered in the photo management database by the new additional image capturing unit, the image list display unit is configured to touch the captured image list including the newly registered previous image with the touch. 6. The portable electronic device according to claim 1, wherein the portable electronic device is displayed again on a screen of the screen device. When the shutter button image is selected by the touch screen device on the underlay finder screen or the normal finder screen, the image includes a photographed image that is the electronic viewfinder image at the time of selection on the touch screen device. Image editing means for displaying an editing screen;
In the photographed image, a scale reference image serving as a scale reference registered in advance is searched, and when the scale reference image is searched, the photographing is performed based on the size of the scale reference image in the photographed image. A size conversion ratio calculating means for calculating a size conversion ratio that is a ratio of the size of the actual subject to the size on the image;
When the size conversion ratio is calculated by the size conversion ratio calculation means, when two points are specified by the touch screen device in the captured image on the image editing screen, the two points on the captured image And a size measuring unit that calculates the distance L2 between the two points in the subject by multiplying the distance L1 by the size conversion ratio and displays the distance L2 in the image editing screen. A portable electronic device according to any one of claims 1 to 6.   The portable electronic device according to any one of claims 1 to 5, wherein the portable electronic device is read and executed by a portable electronic device including a touch screen device, a camera device, an internal clock, and a computer. A program that functions as an electronic device.   The portable electronic device is loaded and executed by a portable electronic device including a touch screen device, a camera device, an internal clock, a positioning module, a short-range wireless communication module, an orientation sensor, and a computer. A program for causing a device to function as the portable electronic device according to claim 6 or 7.