JP2016072972A - Application program, smart device, information processing apparatus, information processing system and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve detection accuracy about alteration of a construction report representing detailed contents of a construction.SOLUTION: An application program functions a smart device including a storage as: an imaging part for capturing a photographic image; an input part for inputting data to a predetermined item of a virtual display panel image on which the item can be entered; an image editing part for creating a photographic with a virtual display panel by superposing and disposing the virtual display panel image in which the data are inputted to the item by the input part, on the photographic image that is captured by the imaging part; a first collation information generation part for generating collation information for the photographic image with the virtual display panel on the basis of at least the photographic image in the photographic image with the virtual display panel; and a collation information storage part for creating a photographic image with the collation information and the virtual display panel by storing the collation information and the data inputted to the item as metadata for the photographic image with the virtual display panel.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an application program, a smart device, an information processing apparatus, an information processing system, and an information processing method.

  Conventionally, one of the management work required at construction sites such as construction work and civil engineering work is a work of photographing a site photograph (evidence photo). In particular, in the case of public works ordered from the national government (or local government), it is required to submit a construction report. In the construction report, in order to report the details of the construction work, A photo is used (attached). Also, in the case of private-ordered construction, a construction report is usually created in the same way as in public construction, so that on-site photography is being carried out at most construction sites.

  Specifically, an operator (construction person) brings a digital camera to a construction site when taking a photo of the site, but first, it is necessary to prepare a blackboard to be photographed together with the shooting site. On the blackboard, for example, “construction name”, “construction location”, “constructor”, etc. are entered with chalk or a pen so that the contents of the photographing site (photographing target) to be photographed can be understood. Also, sometimes survey data measured on the spot such as “measurement point” and “actual measurement dimension” are entered. When the blackboard is ready, one of the workers prepares for the shooting with the blackboard in his / her back against the shooting site (photographing target). Another worker uses a digital camera to shoot the shooting site including the blackboard.

  Here, in most cases, the construction is carried out through multiple construction processes such as foundation work, main body work, etc., but after the main work, the foundation work part will be invisible, so even at the foundation work stage the site photo Need to shoot. Also, in each process in the foundation work and main body work, it is necessary to take on-site photographs according to the stage. For example, in the case of a relatively large building, the number of on-site photographs taken may reach several thousand. is there.

  Site photographs taken at the construction site in this way are stored in a digital camera and then taken back to the office or office to be edited for a construction report. For example, after taking a site photograph into a PC (Personal Computer), an operator checks the information of the site photograph while looking at the blackboard that appears in the site photograph, and arranges the site photograph. Also, sometimes, the construction report is created by attaching the photos of the site while inputting the information on the blackboard in the construction report.

  As a technique related to a construction report, a construction album creation support system that makes it easy to create a construction album (construction report) for reporting construction results such as civil engineering work to an orderer is known (for example, patents). Reference 1).

  In particular, in the case of public works, a construction report is submitted to the country (or local public entity), and it is inspected whether the construction report has been tampered with by the country. Specifically, it is inspected whether the contents of the on-site photo and the blackboard have been tampered with.

  Here, since the contents described on the site photograph and the blackboard represent the details of the construction work, it is required that the accuracy of detecting tampering is high.

  An object of the present invention is to improve the detection accuracy of alteration of a construction report representing details of construction contents in view of the above points.

An application program of an embodiment of the disclosure is
A smart device equipped with a storage device
A shooting section for taking photographic images;
An input unit for inputting data to the item of the virtual display panel image in which a predetermined item can be entered;
An image editing unit for creating a photographic image with a virtual display board in which a virtual display board image in which data is input to the item by the input unit is superimposed on the photographic image taken by the photographing unit;
A first collation information generating unit that generates collation information for the photographic image with a virtual display board based on at least the photographic image among the photographic images with the virtual display board;
The collation information and the data input to the items are made to function as a collation information storing unit that creates collation information and a photo image with a virtual display board, which are stored as metadata for the photographic image with a virtual display board.

  According to the embodiment of the disclosure, it is possible to improve the detection accuracy of alteration of a construction report representing details of construction contents.

The whole block diagram concerning this embodiment is shown. It is a hardware block diagram which shows the main structures of the smart device 1 which concerns on this embodiment. An example of the functional block diagram of the smart device 1 which concerns on this embodiment is shown. It is a hardware block diagram which shows the main structures of PC6 which concerns on this embodiment. An example of a functional block diagram of PC6 concerning this embodiment is shown. An example of the TOP screen A according to the present embodiment is shown. An operation screen transition example according to the present embodiment is shown. An example of screen B1 concerning this embodiment is shown. An example of screen B2-1 according to the present embodiment is shown. The example of screen B3 concerning this embodiment is shown. An example of screen B2-2 according to the present embodiment is shown. A modification of screen B3 according to the present embodiment is shown. An example of screen B4 concerning this embodiment is shown. The arrangement | positioning position and size change operation example of the virtual blackboard which concern on this embodiment are shown. The process which creates collation information and a field photograph with a virtual blackboard is shown. An example of an Exif data format is shown. The screen D1 example (the 1) which concerns on this embodiment is shown. An example of screen E1 concerning this embodiment is shown. The example of the selection list of the initial value of each setting parameter concerning this embodiment is shown. It is a figure explaining the preservation | save method of the favorite virtual blackboard and the photograph with a virtual blackboard which concerns on this embodiment. 6 shows an example of favorite storage management information according to the present embodiment. The favorite virtual blackboard image example which concerns on this embodiment is shown. The management information example for photo preservation | save with a virtual blackboard which concerns on this embodiment is shown. The screen D1 example (the 2) which concerns on this embodiment is shown. The example (the 3) of screen D1 which concerns on this embodiment is shown. The example (the 4) of screen D1 which concerns on this embodiment is shown. The output JPEG image which concerns on this embodiment is shown. The example of EXIF data which concerns on a prior art example is shown. An example of EXIF data of an output JPEG image according to the present embodiment is shown. The flowchart of the process which outputs the collation information which concerns on this embodiment, and a JPEG image with a virtual blackboard is shown. 4 shows a flowchart of outputting a JPEG image according to the present embodiment. The process which determines whether the photograph image with a virtual blackboard is not falsified is shown. The flowchart of the process which determines whether the photograph image with a virtual blackboard concerning this embodiment is not falsified is shown. The 1st modification of collation information and a photographic image with a virtual blackboard is shown. The 2nd modification of collation information and a photographic image with a virtual blackboard is shown. The 3rd modification of collation information and a photographic image with a virtual blackboard is shown.

  Next, the form for implementing this invention is demonstrated, referring drawings based on the following Examples. Examples described below are merely examples, and embodiments to which the present invention is applied are not limited to the following examples. In all the drawings for explaining the embodiments, the same reference numerals are used for those having the same function, and repeated explanation is omitted.

  In recent years, portable information terminals such as mobile phones and PDAs have become widespread, and by operating the portable information terminals, e-mail and the Internet can be easily performed. In addition, recently, a so-called smart device, which is an evolution of such a conventional portable information terminal, has become widespread, and its share is expanding at a tremendous speed. This smart device is represented by a smart phone, a tablet terminal, etc., for example.

  A typical feature of smart devices is that, in addition to improving the processing performance, terminal operations can be performed only by touch operations on the screen. In conventional portable information terminals, terminal operation is performed by hardware key operations provided, but some smart devices do not have hardware keys at all, touch operations from the screen, software All terminal operations can be performed with just a key operation. Paying attention to such operability characteristics of smart devices, we will perform on-site photo shooting using smart devices at the construction site.

[System configuration]
(overall structure)
FIG. 1 is an overall configuration diagram according to this embodiment. The smart device 1 according to the present embodiment is an apparatus used for taking a site photograph at a construction site. Specifically, the smart device 1 can be realized by a smartphone, a tablet terminal, or the like. At the construction site, the worker uses the camera attached to the smart device 1 to shoot the shooting site (shooting target) that is the subject. Although it is necessary that a blackboard on which various types of information are entered is shown in the photographed on-site photo, in the present embodiment, the blackboard as described above (the actual blackboard) is not used, and the photograph is taken with the camera of the smart device 1. The application inserts a virtual electronic blackboard (hereinafter referred to as a “virtual blackboard”) into the photograph of the photographed shooting site, thereby constructing the site photo.

  Note that a blackboard generally refers to a board coated with black or green paint for writing information such as letters and drawings. In this embodiment, a blackboard is not only a black or green board, but also a white board (white board). Board)). That is, the display board for displaying information, such as a character and a figure, may be sufficient. The virtual blackboard can be rephrased as a virtual display board.

  The surveying instrument / measuring instrument 2 and the construction document 3 provide the smart device 1 with information on the spot of the construction site. When entering survey data measured on the spot such as “Spot” and “Measured dimensions” on the virtual blackboard, the operator surveys and measures the survey data with the survey instrument / measurement instrument 2, and the survey instrument / measurement instrument 2 transmits survey data to the smart device 1 through data communication such as BlueTooth (registered trademark). Also, when general construction information such as “construction name”, “construction location”, “constructor”, etc. is entered on the virtual blackboard, the operator shoots the construction material 3 with the camera of the smart device 1 and constructs from the same image. Information is input as text, or construction information is input through various types of code information such as a barcode or QR code (registered trademark). These construction information is reflected in the virtual blackboard.

  The server 4 is a management server that is installed in an office, office, data center, or the like and is used for construction information management. For example, construction information is input to the smart device 1 via data communication such as a wireless LAN (such as WiFi (registered trademark)) or a telephone network / IP network line (such as 3G), or vice versa. 4 uploads the on-site photo taken via the data communication. If the construction site is located in the back of the mountain and the surrounding communication conditions are not good, for example, the smart device 1 cannot communicate with the server 4, so the construction information is input in advance prior to entering the construction site. The site photos stored in the smart device 1 will be uploaded later.

  The PC 5 is a terminal that creates a construction report. In creating the construction report, the worker obtains construction information and already uploaded site photos from the server 4. Alternatively, a field photograph is obtained from a digital camera via a USB or a recording medium. The PC 5 can transmit a construction report to a delivery destination such as the PC 6 through data communication such as a wireless LAN (such as WiFi) or a telephone / IP network line (such as 3G).

  The PC 6 is a delivery destination terminal for inspecting a construction report in the country or the like, and receives the construction report transmitted by the PC 5. The PC 6 inspects whether the on-site photo attached to the construction report has been tampered with.

  The cloud storage 7 is a storage device that is installed on a network such as the Internet and stores (accumulates) on-site photographs. For example, the smart device 1 uploads the photographed on-site photograph to the cloud storage 7 through data communication such as a wireless LAN (such as WiFi) or a telephone network / IP network line (such as 3G). Since the storage area for storing the on-site photos taken by the smart device 1 has a limited capacity, the cloud storage 7 can be used as an alternative storage area.

  The field photos stored in the cloud storage 7 can be used in various forms thereafter. For example, when the PC 5 creates a construction report, construction information and already uploaded site photos are acquired from the cloud storage 7. Further, the construction report created by the PC 5 is uploaded to the cloud storage 7. In addition, if linked with a print service such as cloud printing, it is possible to arbitrarily select and print on-site photos stored in the cloud storage 7. In the past, once the site photos were organized and printed by the PC 5, there is an advantage that printing can be performed directly from the smart device 1.

  The PC 8 is a terminal that acquires a construction report from the cloud storage 7. The PC 8 can transmit a construction report to a delivery destination such as the PC 6 via data communication such as a wireless LAN (such as WiFi) and a telephone / IP network line (such as 3G). When transmitting the construction report to a delivery destination such as the PC 6, the PC 8 can confirm whether or not the on-site photo attached to the construction report has been tampered with. Thus, the construction report can be transmitted to a delivery destination such as the PC 6 via another terminal different from the terminal that created the construction report. Here, the construction report can be transmitted to a delivery destination such as the PC 6 via a plurality of terminals.

(Smart device hardware)
A hardware configuration of the smart device 1 according to the present embodiment will be described. FIG. 2 is a hardware configuration diagram illustrating a main configuration of the smart device 1 according to the present embodiment. The smart device 1 includes a CPU (Central Processing Unit) 11, a storage device 12, a display device 13, an input device 14, a communication device 15, an external interface device 16, a photographing device 17, and a GPS 18 as main components. .

  The CPU 11 is composed of a microprocessor and its peripheral circuits, and is a circuit that controls the entire apparatus. Processing based on an OS (Operating System) for smart devices and various application programs is executed.

  The storage device 12 is a memory that stores an OS for smart devices, various applications, and other various data. The storage device 12 includes a RAM (Random Access Memory) used as a work area (work area) when the CPU 11 executes various programs and performs various controls, and a non-volatile ROM (Read that stores predetermined control programs) Only Memory) and HDD (Hard Disk Drive) that stores relatively large amounts of data. In the present embodiment, it is assumed that an application program for smart devices (for example, “construction site blackboard application”) is installed in advance here.

  The display device 13 is a display that displays various data on a display screen. The input device 14 is a device for a user to perform various input operations. The input device 14 of the smart device 1 according to the present embodiment includes a touch panel switch provided so as to be superimposed on the display screen of the display device 13 in addition to the hardware key. In this case, the software key is realized by the touch panel switch control by the OS input system program. That is, all terminal operations can be performed only by touch operations and software key operations from the screen.

  The communication device 15 is a device that performs communication with other devices. Supports communication according to various network forms including wired and wireless networks. The smart device according to the present embodiment specifically supports, for example, a wireless LAN (such as WiFi), a telephone / IP network line (such as 3G), Bluetooth (registered trademark), and infrared communication.

  The external interface device 16 is an interface for connecting an external recording medium and inputting / outputting data in the recording medium in the terminal. For example, information from the outside can be acquired or output by setting a memory card such as a USB memory or an SD card.

  The photographing device 17 is a digital camera provided in the smart device 1. An imaging lens is provided, and the captured image (photo image) captured is taken into the smart device 1 as image data (photo data). When various types of code information such as a barcode or QR code (registered trademark) of the construction material 3 are imaged from the photographing device 17, it is possible to read text data in the code via a predetermined application.

  The GPS 18 is a GPS receiver that receives current position information on the earth from a Global Positioning System. That is, the GPS 18 acquires current position information (position coordinates) from a GPS satellite.

(Smart device functions)
Next, the main functional configuration of the smart device 1 according to the present embodiment will be described. FIG. 3 shows an example of a functional block diagram of the smart device 1 according to the present embodiment. As shown in the figure, the smart device 1 includes a storage unit 101, a touch panel 102, a photographing unit 103, a communication unit 104, an input unit 105, an image editing unit 106, and a collation information calculation unit (first collation information generation unit) 107. A collation information storage unit 108, an image reproduction unit 109, and an image output unit 110.

  The storage unit 101 is realized by the storage device 12, for example, and stores in advance a virtual blackboard image provided with predetermined entry items (unfilled). In addition, when a photo image with a virtual blackboard (for example, a site photo with a virtual blackboard) in which a virtual blackboard image is arranged on a photo image (for example, a site photo) is created, the storage unit 101 stores this. At this time, as a storage method, the storage unit 101 captures a photographed photo image, a virtual blackboard image stored in advance as a template, position information of the virtual blackboard image on the photo image, and a virtual blackboard image on the photo image. The size information and the data input to the predetermined entry of the virtual blackboard image are stored in association with each other.

  In addition, a photo image with a virtual blackboard and a photo image with the virtual blackboard are automatically generated when the photo image with the virtual blackboard is saved as a JPEG file, for example, based on metadata in an Exif (Exchangeable Image File Format) format. Collation information used to inspect whether a photographic image and metadata have been tampered with is calculated, and a photographic image with a virtual blackboard (for example, collation information and an on-site photo with a virtual blackboard) in which the collation information is stored Once created, the storage unit 101 saves it.

  The touch panel 102 is a touch panel display including an operation unit 102 a realized by the input device 14 and a display unit 102 b realized by the display device 13, for example. As described above, since the smart device 1 according to the present embodiment includes the touch panel display, it is possible to perform all terminal operations only by touch operations and software key operations from the screen.

  The photographing unit 103 is a digital camera that is realized by, for example, the photographing device 17 and is provided in the smart device 1. The captured image (photo image) taken is taken into the smart device 1 as image data (photo data). When various types of code information such as a barcode or QR code (registered trademark) of the construction material 3 are imaged from the imaging unit 103, it is also possible to read text data in the code via a predetermined application.

  The communication unit 104 is realized by, for example, the communication device 15 and performs communication with other devices. Specifically, the smart device according to the present embodiment supports, for example, a wireless LAN (such as WiFi), a telephone network / IP network line (such as 3G), Bluetooth (registered trademark), infrared communication, and the like. Communication is performed with various devices such as the device 2, the server 4, and the PC 5.

  The input unit 105 inputs data related to the construction site to a predetermined entry item (unfilled) of the virtual blackboard image stored in the storage unit 101. As a result, predetermined data relating to the construction can be input to the virtual blackboard image that has not been filled in. In addition, although the data input here follow the entry provided in the virtual blackboard image, specifically, for example, survey data (“spot”, “measured dimension”) surveyed by the surveying instrument / measurement instrument 2 Etc.) and general construction information ("construction name", "construction location", "constructor", etc.) managed by the server 4 or various code information. Note that data can be input via a software key from the touch panel 102 by the user or the communication unit 104.

  The image editing unit 106 superimposes and arranges the virtual blackboard image in which the data is input by the input unit 105 on the photo image taken by the photographing unit 103, so that the photo image with the virtual blackboard (for example, the site with the virtual blackboard) Photo). The created photographic image with virtual blackboard is stored in the storage unit 101. Further, the image editing unit 106 converts the photographic image with a virtual blackboard into a JPEG file, and inputs it to the collation information calculation unit 107 and the collation information storage unit 108.

  The collation information calculation unit 107 generates collation information for a photographic image with a virtual blackboard based on at least the photographic image among the photographic images with a virtual blackboard in the JPEG file format input from the image editing unit 106. For example, the collation information calculation unit 107 acquires Exif type metadata that is automatically created when creating a JPEG file from a photo image with a virtual blackboard in the JPEG file format input from the image editing unit 106. . Then, the collation information calculation unit 107 calculates the collation information from the photo image with virtual blackboard and the Exif type metadata, and inputs the collation information to the collation information storage unit 108.

  The collation information storage unit 108 inputs the collation information input from the collation information calculation unit 107 and the input unit 105 entered on the virtual blackboard to the photographic image with the virtual blackboard in the JPEG file format input from the image editing unit 106. Stored data (virtual blackboard information) is stored. Thereby, the collation information storage unit 108 creates collation information and a photographic image with a virtual blackboard (for example, collation information and a field photo with a virtual blackboard). The created collation information and the photographic image with the virtual blackboard are stored in the storage unit 101.

  The image reproduction unit 109 reproduces the photographic image with a virtual blackboard stored in the storage unit 101 and displays it on the display device. When reproducing the photographic image with the virtual blackboard, the image reproduction unit 109 acquires a virtual blackboard image associated with the photographic image from the storage unit 101, and data corresponding to a predetermined entry item of the virtual blackboard image. Is reproduced by superimposing the virtual blackboard image of the size specified by the corresponding size information on the photographic image based on the corresponding position information.

  The image output unit 110 outputs the photographic image with a virtual blackboard as JPEG format image data for the reproduced photographic image with the virtual blackboard stored in the storage unit 101. Specifically, a virtual blackboard with data input is superimposed on a photographic image to generate and output a JPEG file of a photographic image with a virtual blackboard in which the virtual blackboard and the photographic image are integrated. Further, the image output unit 110 outputs collation information and a photographic image with a virtual blackboard stored in the storage unit 101.

  These functions are actually realized in the computer by the smart device OS and the smart device application program (for example, the “construction site blackboard application”) executed by the CPU 11 of the smart device 1.

(PC hardware)
A hardware configuration of the PC 6 according to the present embodiment will be described. FIG. 4 is a hardware configuration diagram showing the main configuration of the PC 6 according to the present embodiment. The PC 6 includes a CPU 61, a storage device 62, a display device 63, an input device 64, a communication device 65, and an external interface device 66 as main components.

  The CPU 61 is composed of a microprocessor and its peripheral circuits, and is a circuit that controls the entire apparatus. Processing based on the OS for PC and various application programs is executed.

  The storage device 62 is a memory for storing an OS for PC, various applications, and other various data. The storage device 62 includes a RAM that is used as a work area (work area) when the CPU 61 executes various programs and performs various controls, a nonvolatile ROM that stores predetermined control programs, and a relatively large amount of data. Including an HDD that stores. In the present embodiment, it is assumed that a PC application program (for example, “construction site blackboard application”) is installed in advance here.

  The display device 63 is a display that displays various data on a display screen. The input device 64 is a device for the user to perform various input operations. The input device 64 of the PC 6 according to the present embodiment includes a touch panel switch provided so as to be superimposed on the display screen of the display device 63 in addition to the hardware key. In this case, the software key is realized by the touch panel switch control by the OS input system program. That is, all terminal operations can be performed only by touch operations and software key operations from the screen.

  The communication device 65 is a device that communicates with other devices. Supports communication according to various network forms including wired and wireless networks. Specifically, the PC 6 according to the present embodiment supports, for example, a wireless LAN (such as WiFi), a telephone network / IP network line (such as 3G), Bluetooth (registered trademark), and infrared communication.

  The external interface device 66 is an interface for connecting an external recording medium and inputting / outputting data between the PC 6 and the recording medium. For example, information from the outside can be acquired or output by setting a memory card such as a USB memory or an SD card.

(PC function)
Next, the main functional configuration of the PC 6 according to the present embodiment will be described. FIG. 5 shows an example of a functional block diagram of the PC 6 according to the present embodiment. As shown in the figure, the PC 6 includes a storage unit 601, a touch panel 602, a communication unit 603, an image acquisition unit 604, and a tampering determination unit 605.

  The storage unit 601 is realized by, for example, the storage device 62, and stores collation information and a photographic image with a virtual blackboard output by the smart device 1. The collation information and the photo image with the virtual blackboard are based on, for example, Exif format metadata automatically created when the photo image with the virtual blackboard and the photo image with the virtual blackboard are saved as a JPEG file. The virtual blackboard-attached photo image and collation information used to inspect whether the metadata has been tampered with are calculated and the collation information is stored.

  The touch panel 602 is a touch panel display including an operation unit 602a realized by the input device 64 and a display unit 602b realized by the display device 63, for example. As described above, the PC 6 according to the present embodiment includes the touch panel type display in addition to the hardware keys, so that any terminal operation can be performed only by touch operation and software key operation from the screen. Yes.

  The communication unit 603 is realized by the communication device 65, for example, and performs communication with other devices. Specifically, the PC 6 according to the present embodiment supports, for example, a wireless LAN (such as WiFi), a telephone network / IP network line (such as 3G), Bluetooth (registered trademark), infrared communication, and the like, and includes the server 4, the PC 5, Communication is performed with various devices such as the PC 8.

  The image acquisition unit 604 is realized by, for example, the CPU 61, acquires the collation information and the photo image with virtual blackboard transmitted from the PC 5, PC 8, etc. received by the communication unit 603, and stores the storage unit 601, or the storage unit 601, and the tampering determination. To the part 605. The user can specify whether to input to the storage unit 601 or to the storage unit 601 and the tampering determination unit 605 by operating the operation unit 602a. For example, when it is desired to determine whether or not tampering has occurred immediately after receiving the collation information and the photographic image with the virtual blackboard, the storage unit 601 and the tampering determination unit 605 are designated to be input.

  The tampering determination unit 605 is realized by, for example, the CPU 61, determines whether the photographic image with virtual blackboard has been tampered with based on the collation information input by the image acquisition unit 604 and the photographic image with virtual blackboard, and the result is It is displayed on the display unit 602b. Therefore, the falsification determination unit 605 is provided with a collation information removal unit (collation information acquisition unit, collation information generation image creation unit) 605a, a collation information calculation unit illustrated in FIG. 5 according to a command from the CPU 61 illustrated in FIG. (Second verification information generation unit) 605b and verification information verification unit 605c are realized.

  The collation information removal unit 605a is a collation information acquisition unit that obtains, for example, the collation information and the first collation information stored in the photo image with the virtual blackboard and the virtual blackboard information. The collation information removing unit 605a is an image for generating second collation information, for example, an image obtained by removing at least the first collation information from the first collation information and the virtual blackboard-attached photo image. It is a collation information generation image creation unit that creates a collation information generation image. In this embodiment and each modification described below, an image for generating collation information (for example, second collation information) for checking tampering is a “collation information generation image”.

  For example, the collation information removing unit 605a removes the first collation information and the virtual blackboard information from the collation information and the photo image with the virtual blackboard. The collation information removal unit 605a sends the first collation information, the collation information from which the virtual blackboard information is removed, and the photo image with the virtual blackboard, that is, the photo image with the virtual blackboard (the collation information generation image) to the collation information calculation unit 605b. In addition to the input, the acquired first verification information is input to the verification information verification unit 605c. This photo image with virtual blackboard includes, for example, metadata in Exif format that is automatically created when the photo image with virtual blackboard is saved as a JPEG file.

  Based on the first collation information input from the collation information removal unit 605a, the collation information from which the virtual blackboard information has been removed, and the photo image with virtual blackboard (collation information generation image), The verification information is generated by calculation or the like. In addition, the collation information calculation unit 605b inputs the generated second collation information to the collation information collation unit 605c.

  The collation information collation unit 605c collates by determining whether or not the second collation information input from the collation information calculation unit 605b matches the first collation information input from the collation information removal unit 605a. . When the matching information matches, the matching information matching unit 605c displays on the display unit 602b that the photographic image with virtual blackboard has not been tampered with. When the matching information does not match, the photographic image with virtual blackboard has been tampered with. Is displayed on the display unit 602b.

  These functions are actually realized in the computer by the PC OS and the PC application program (for example, “construction site blackboard application”) executed by the CPU 61 of the PC 6.

[Operation screen transition example of smart device 1]
FIG. 6 shows an example of the TOP screen A according to this embodiment. The TOP screen A is a screen that is first displayed on the display screen when the user first starts an application program for smart devices (for example, “construction site blackboard application”). As shown in the figure, the TOP screen A is provided with operation buttons such as “favorite”, “blackboard selection”, “photographing”, “photo storage”, “various settings”, and the like.

FIG. 7 shows an operation screen transition example according to this embodiment. When the user first activates the smart device application program and operates each operation button on the TOP screen A, the screen changes as follows.
“Favorites”: To screen C1-2 “Select blackboard”: To screen C1-1 “Photographing”: To screen B1 “Photo storage”: To screen D1 “Various settings”: To screen E1 This is a creation operation. In the B operation, the verification information, the photographic image with the virtual blackboard, and the photographic image with the virtual blackboard are finally created through the steps of shooting the site photo, selecting the virtual blackboard, inputting the management items, and arranging the virtual blackboard.

  The C operation is an operation for creating an on-site photograph in the same manner as the B operation, although the operation order is different from the B operation. In the C operation, after first selecting a virtual blackboard, inputting management items, taking a photo of the site, and placing a virtual blackboard, finally, collation information, a photo image with a virtual blackboard, and a photo image with a virtual blackboard Is created.

  The D operation is an operation for referring to a photograph with a virtual blackboard. Since the collation information and the photo image with the virtual blackboard created by the B operation or the C operation and the photo with the virtual blackboard are stored, the collation information and the photo image with the virtual blackboard already created by the D operation and the photo with the virtual blackboard Or selecting an arbitrary photo and outputting it to the server 4, the PC 5, or a recording medium.

  The E operation is an operation for performing various settings. In the E operation, for example, the initial color, initial position, initial size, etc. of the virtual blackboard can be set.

  Hereinafter, each operation will be described in detail with reference to FIG.

(B operation example)
First, since the B operation will be described here, it is assumed that the operation button “shoot” is operated (for example, touched) on the TOP screen A, and at this time, the screen transits to the screen B1.

  FIG. 8 shows an example of a screen B1 according to the present embodiment. As shown in the figure, on the screen B1, the camera application built in the smart device 1 starts up and shifts to the shooting mode. On the screen of the smart device 1, a construction site (in the figure, an example of a cylindrical bench in a building site) that is captured through the photographing unit 103 is depicted, and the photograph is taken by the user performing a photographing operation.

  FIG. 9 shows an example of a screen B2-1 according to the present embodiment. When a picture is taken on screen B1, the screen transitions to screen B2-1. On the screen B2-1, as shown in the figure, a plurality of patterns of virtual blackboard templates (models) are preinstalled and provided, and the user selects them by touching the virtual blackboard to be used. To do.

  FIG. 10 shows an example of a screen B3 according to the present embodiment. (A) is the virtual blackboard selected by the user. Further, as shown in the figure, the virtual blackboard is provided with predetermined entry items (unfilled), and management information relating to the photographed images is input by the input unit 105 respectively. Note that the position information (position coordinates) of the current location automatically acquired from the GPS 18 has already been input to “position” among the entries on the virtual blackboard. It is also possible to automatically input a specific address here based on the position information.

  Then, as shown in (b), when the “construction name” blank is touched in order to input “construction name” among the unfilled items in the virtual blackboard, the software key will start up. Enter a specific construction name using the software key. Here, for example, “XX building renovation” is entered. In the same manner, the user inputs a specific work type using the software key. Here, for example, “architecture (architecture)” is input.

  In the virtual blackboard, in the free space below “Position”, you can attach, for example, a construction drawing called a bean figure, or enter any other construction information related to construction. is there.

  When the above input is completed, as shown in (c), the virtual blackboard is in a state where all entry items have been input. When all the inputs are completed, the user touches the operation button “apply blackboard” on the screen (c).

  If the operation button “blackboard registration” is touched in the screen (c), the virtual blackboard in which all the entry items have been entered is separately saved as a favorite virtual blackboard (screens B2-2, C1). -2).

  FIG. 11 shows an example of a screen B2-2 according to the present embodiment. Here, a virtual blackboard registered as “blackboard registration” is displayed as a favorite virtual blackboard. Since the one in which the entry item has been input is stored for the virtual blackboard that has not been filled in, the efficiency of the input work can be improved by reusing the virtual blackboard. For example, when a plurality of site photographs are taken within the same construction site, the input of the same information can be omitted by reusing this virtual blackboard.

  Reference is again made to FIG. For example, instead of the screen B2-1, the virtual blackboard 1 is selected from favorites through the screen B2-2. In this case, in the next screen B3, the favorite virtual blackboard in which all the entry items have already been entered is called up and displayed (for example, see FIG. 10C), and the user needs to input anything here. In the screen (c), the operation button “apply blackboard” may be touched.

  FIG. 12 shows a modification of the screen B3 according to the present embodiment. (A) is the virtual blackboard selected by the user. As in FIG. 10A, position information (position coordinates) of the current location automatically acquired from the GPS 18 is already input to “position” among the entries on the virtual blackboard. In this modification, the “construction name” is already entered. In the smart device 1 according to the present embodiment, as described above, survey data (“measurement point”, “measured dimension”, etc.) surveyed by the surveying instrument / measurement instrument 2 via the communication unit 104, the server 4 or various codes. It is possible to input general construction information ("construction name", "construction location", "constructor", etc.) that is also managed by information. Accordingly, among the entries on the virtual blackboard, the “construction name” is automatically input by the input unit 105 based on construction information input from the outside.

  Subsequently, as shown in (b), when the user touches an empty column of “work type” in order to input “work type” among the unfilled items in the virtual blackboard, a selection list is displayed. In the case of “type of work”, a standard input value is input, and therefore a plurality of pieces of information on possible input values are held in advance in the selection list. Therefore, the user can easily input an input value by selecting one “work type” from the selection list. If the “construction type” of 1 is determined and entered from the construction information, the construction information entered from the outside is also entered in the “construction type” as in the above “construction name” without going through the selection list. The input value is automatically input based on the above.

  As described above, when the input is completed in the present modification, as shown in (c), the virtual blackboard is in a state where all the entry items have been input. When all the inputs are completed, the user touches the operation button “apply blackboard” on the screen (c).

  FIG. 13 shows an example of a screen B4 according to this embodiment. On the screen B4, the image editing unit 106 arranges a virtual blackboard on which the entry items have been input, superimposed on the photographed photo. Here, it can be seen that the virtual blackboard in which the entry items are input in FIG. 10C or FIG. 12C is arranged on the photograph taken in B1 of FIG. This photographic image is a photographic image (site photo) with a virtual blackboard.

  Here, the user can change the arrangement position and size of the virtual blackboard by an operation from the touch panel. The initial position of the virtual blackboard and the initial value of the size follow the initial settings. However, depending on the photograph taken, if the initial value remains the same, the image of the construction site may not be visible due to the presence of the virtual blackboard. Therefore, after confirming the screen B4, the user changes the placement position and size of the virtual blackboard if the initial placement position and initial size of the virtual blackboard are inconvenient.

  FIG. 14 shows an example of an arrangement position and size change operation of the virtual blackboard according to the present embodiment. (A) shows change operation of the arrangement position of a virtual blackboard. As shown in the figure, the user can easily change the position of the virtual blackboard by operating the touch panel 102 to trace the virtual blackboard with a finger. Further, (b) shows an operation for changing the size of the virtual blackboard. As shown in the figure, the user can easily change the size of the virtual blackboard by operating the touch panel 102 to reduce (or enlarge) the virtual blackboard with a finger. Needless to say, the placement position and size of the virtual blackboard can be changed simultaneously.

  As described above, when the user finishes confirming the photo image with the virtual blackboard, the user saves the photo image by touching the operation button “Save” in the screen of FIG. The photograph image with virtual blackboard stored here is stored in a “photo storage” (screen D1), which will be described later, on the application.

  Further, the collation information is calculated from the photo image with the virtual blackboard and the metadata in the Exif format, and a photo image with the virtual blackboard (for example, the collation information and the on-site photo with the virtual blackboard) is created and stored. 101 is stored.

  FIG. 15 shows a process of creating collation information and an on-site photo with a virtual blackboard. For example, when an on-site photo image is saved as a JPEG file, Exif format metadata is created (1). That is, the on-site photographic image includes a photographic image and metadata. Then, by placing a virtual blackboard on the photo image of the site, a site photo with a virtual blackboard is created (2), and collation information is calculated based on the site photo with the virtual blackboard and metadata (3). For example, based on a predetermined function such as a hash function that is set in advance, an on-site photo with a virtual blackboard and a value (collation information) representative of metadata are calculated. The collation information is stored in a predetermined area of the metadata together with the virtual blackboard information.

  FIG. 16 shows an example of the Exif data format. Exif data is accompanied by SOI, APP1, APP5, DQT, DHT, SOF, SOS, Compression Data, EOI, and the like. The information incidental to the Exif data includes a secure private area unique to the manufacturer, such as APP5, and a definition statement is preset for storing information in this private area. In accordance with this definition statement, verification information and virtual blackboard information are stored in the Exif data.

(C operation example)
The C operation is an operation for creating an on-site photograph in the same manner as the B operation, although the operation order is different from the B operation. In the C operation, a virtual blackboard-attached photo image is finally created through the steps of selecting a virtual blackboard, inputting management items, shooting a field photo, and arranging the virtual blackboard. This will be briefly described below.

  Please refer to FIG. In the case of the C operation, on the TOP screen A, the operation button “select blackboard” or “favorite” is operated (for example, touched).

  When “Select Blackboard” is operated, the screen changes to the screen C1-1. The screen C1-1 is provided with a plurality of patterns of virtual blackboard templates (models) pre-installed in advance, as in FIG. 9 (see FIG. 9). The user touches and selects the virtual blackboard to be used.

  In the screen C2, similar to FIGS. 10 and 12, the selected virtual blackboard is provided with a predetermined entry item (unfilled), and the user uses a software key for the unfilled item in the virtual blackboard. Then, a specific input value is input (see FIG. 10). Alternatively, as described above, in the smart device 1 according to the present embodiment, as described above, the surveying data measured by the surveying instrument / measurement instrument 2, the server 4 or various code information is also managed via the communication unit 104. In this case, the input values are automatically entered in the entries on the virtual blackboard based on the construction information input from the outside. (See FIG. 10).

  On the other hand, when the operation button “favorite” is operated on the TOP screen A, the screen transitions to the screen C1-2. Similar to FIG. 11, a virtual blackboard registered as “blackboard registration” is displayed as a favorite virtual blackboard (see FIG. 11). If a virtual blackboard with entries already entered is stored (registered), one virtual blackboard can be selected from favorites. In this case, on the next screen C2, the virtual blackboard in which all the entry items have been input is called and displayed, so that the user does not need to perform any input work here.

  Next, the screen transitions to a screen C3. On the screen C3, as in FIG. 8, the camera application built in the smart device 1 starts up and shifts to the shooting mode. Since the construction site (in the figure, an example of a cylindrical bench in the building site) captured through the photographing unit 103 is projected on the screen of the smart device 1, the photograph is taken by the user performing a photographing operation (FIG. 8). reference).

  Next, the screen transitions to a screen C4. On the screen C4, a virtual blackboard on which the entry items have been input is arranged on the photographed image (see FIG. 13). Here, the user can change the arrangement position and size of the virtual blackboard by an operation from the touch panel.

  As described above, after confirming the photograph image with the virtual blackboard, the user touches the operation button “Save” on the screen to save the photograph image. In addition, the photograph with a virtual blackboard saved here is preserve | saved on a "photo storage" (screen D1) on an application. Further, the collation information is calculated from the photo image with the virtual blackboard and the metadata in the Exif format, and a photo image with the virtual blackboard (for example, the collation information and the on-site photo with the virtual blackboard) is created and stored. 101 is stored.

(D operation example)
The D operation is an operation for referring to a photograph with a virtual blackboard. Since the photo image with a virtual blackboard created by the B operation or the C operation is stored, the user refers to the photo with the virtual blackboard already created from here or selects an arbitrary photo, and then the server 4 and the PC 5 Or output to a recording medium or the like. Further, the user can refer to the collation information already created and the photo with the virtual blackboard, or select an arbitrary photo and output it to the server 4, the PC 5, or the recording medium.

  FIG. 17 shows an example (part 1) of the screen D1 according to the present embodiment. When the operation button “photo storage” is operated on the TOP screen A, the screen changes to this screen. As shown in the figure, in the “photo storage” on the screen D1, the photos with virtual blackboards created so far by the B operation or the C operation are displayed in a list. In addition, the collation information created by the B operation or the C operation and the photos with the virtual blackboard can be displayed in a list. Each photo is displayed in a reduced size in a thumbnail format. For example, when the user touches a desired photo, the photo is enlarged and displayed on the screen. When the operation button “slide show” is touched, the photos in the “photo storage” are sequentially displayed in a sliding manner.

  In the “photo storage”, one or a plurality of arbitrary photos can be selected and output to the server 4, the PC 5, or a recording medium. The user finally fetches the collation information and the photo image with virtual blackboard in the “photo storage” into the PC 5 and creates a construction report using the collation information and the photo image with virtual blackboard.

(Example of E operation)
The E operation is an operation for performing various settings. In the E operation, initial values of the setting parameters regarding the virtual blackboard can be set.

  FIG. 18 shows an example of a screen E1 according to the present embodiment. On the screen E1, as shown in the figure, for example, initial values of “blackboard color”, “blackboard position”, “blackboard size”, and “blackboard shape” can be set. When the user changes the setting of the initial value, the corresponding selection list is displayed when the corresponding setting value column is touched.

  FIG. 19 shows an example of a selection list of initial values of the setting parameters according to the present embodiment. (A) "Blackboard color" selection list, (b) "Blackboard position" selection list, (c) "Blackboard shape" selection list, (d) "Blackboard size" selection It is a selection list example of a list. The user selects and sets a desired (arbitrary) initial value from the selection list.

  Refer to FIG. 13 again. On the screen B4, a virtual blackboard on which the entry items have been entered is superimposed on the photographed photo, and the color, position, size, and shape of the virtual blackboard arranged as a default (standard) here are as follows: The initial value set in the screen E1 is followed. In this way, the user sets the desired initial value in advance on the screen E1 for each setting parameter related to the virtual blackboard, so that the virtual blackboard that is most convenient to use is created in the initial state when creating the on-site photo with the virtual blackboard. Can be arranged as

  Of course, the initial value set on the screen E1 is merely for setting the state of the virtual blackboard arranged as the initial state. As described above, the user can change the placement position and size of the virtual blackboard by operating from the touch panel. Can be changed. In other words, the initial position of the virtual blackboard and the initial value of the size follow the initial settings, but if the initial position of the virtual blackboard and the initial size are inconvenient, the position and size of the virtual blackboard can be arbitrarily changed.

[How to save your favorite virtual blackboard and photos with virtual blackboard]
FIG. 20 is a view for explaining a method of saving a favorite virtual blackboard and a photograph with a virtual blackboard according to the present embodiment. As already described, the virtual blackboard used in the present embodiment is preinstalled and provided as a template (model) of a virtual blackboard having a plurality of patterns. Therefore, the storage unit 101 has predetermined entries ( A virtual blackboard image provided with (unfilled) is stored in advance. When a photographic image is taken by the photographing unit 103, the photographed photographic image is stored in the storage unit 101 as image data.

(My favorite virtual blackboard)
First, a method for saving a favorite virtual blackboard will be described. As described above, in the smart device 1 according to the present embodiment, a virtual blackboard in which entry items have been input can be separately stored as a favorite virtual blackboard. Therefore, on the application, it is possible to omit the input of the same information from the next time by calling and reusing the virtual blackboard saved from the favorites (see FIG. 11).

  FIG. 21 shows an example of favorite storage management information according to the present embodiment. The favorite storage management information is information for storing a virtual blackboard in which entry items have been input and reconstructing (reproducing) the image as an image. As described above, when a virtual blackboard into which entry items have been input is separately stored as a favorite virtual blackboard, “favorite storage management information” for storing and managing the favorite virtual blackboard is generated. In the case of FIG. 21, three virtual blackboards indicate favorite storage management information registered as favorites.

  For example, “favorite No.”: Template 01-1 holds input data for “Template 01.jpg” which is a virtual blackboard image of a predetermined entry item (unfilled) stored in the storage unit 101. In other words, “Favorite No.”: Template01-1 has “construction subject”: station square renovation work, “construction place”: XX station square, “constructor”: Tanaka Construction with respect to the virtual blackboard image template “Template01.jpg”. It is a virtual blackboard image that is reproduced by using (stock) as an input value in an entry.

  FIG. 22 shows a favorite virtual blackboard image example according to the present embodiment. The favorite virtual blackboard image (list) shown in the figure corresponds to the storage management information of FIG. 21 in which three virtual blackboards are registered as favorites. For example, in the case of the above-mentioned “favorite No”: Template01-1, the virtual blackboard image template “Template01.jpg” is acquired from the storage unit 101, and the “construction subject”: station square in the virtual blackboard image Renovation work, “construction place”: XX station square, “constructor”: Tanaka Construction Co., Ltd. is input and the virtual blackboard image reproduced.

  As described above, in the smart device 1 according to the present embodiment, when the virtual blackboard in which the entry item has been input is separately stored as a favorite virtual blackboard, the virtual blackboard “image” in which the entry item has already been input is not stored, It can be saved in text format information called favorites storage management information. That is, while the virtual blackboard image template and the input data are stored separately, they are stored in association with each other. As described above, it can be said that the smart device 1 is not necessarily large by reproducing the favorite virtual blackboard image using the virtual blackboard image template and the favorite storage management information stored in advance in the storage unit 101. It is possible to improve the editability of the favorite virtual blackboard while saving the storage capacity.

(Photo image with virtual blackboard)
Next, a method for saving a photo with a virtual blackboard will be described. As described above, on the application, the photo with the virtual blackboard is stored in the “photo storage”. When the operation button “photo storage” is operated on the D operation, that is, the TOP screen A, a transition is made to “photo storage” (see FIG. 17), and the user refers to the already created photo image with a virtual blackboard. Is possible.

  On the other hand, in the smart device 1, an actual photograph with a virtual blackboard is stored in the storage unit 101. As one of the storage methods, a display image in which a virtual blackboard image is arranged on a photographic image is used. One example is a method of saving an on-site photo image with a virtual blackboard as a new image. As another method, the photographed photo image, the virtual blackboard image stored as a template in advance, the position information of the virtual blackboard image on the photo image, the size information of the virtual blackboard image on the photo image, and the virtual blackboard There is a method of associating and storing data input to predetermined entry items of images.

  When a photograph image is photographed by the photographing unit 103, the photographed photograph image is stored in the storage unit 101 as image data. Then, when the virtual blackboard image is arranged on the photographed photograph image and saved in the “photo storage” (see FIG. 17), “Photo Blackboard with Photo Blackboard Management” is stored and managed. Information "is generated.

  FIG. 23 shows an example of management information for storing photos with a virtual blackboard according to the present embodiment. The “virtual blackboard-added photo storage management information” is information that can be used when a photo with a virtual blackboard is stored and reconstructed (reproduced) as an image. As shown in (0), “Photo Image No”, which is a unique identifier for storage management of photo images, “Photo Image” (file name) for specifying a basic photo image, and a blackboard to be used A “blackboard image” (file name) for specifying an image template and an “initialization parameter for blackboard image”, which is each initial value related to the blackboard image, are managed in association with each other. Further, as shown in (1), (2), (3), etc., “Photo Image No” and input data (input value) for each blackboard image are managed in association with each other.

  Here, for example, referring to (0), when the smart device 1 (image reproduction unit 109) reconstructs (reproduces) a photographic image with a virtual blackboard stored as “photographic image No”: 001 in the first row. Take the example. First, “photographic image”: 001. Get jpg file data. The blackboard image template to be used is “blackboard image”: Template02. jpg, so that Template02. Get jpg file data.

  Next, “blackboard image” which is a blackboard image template to be used: Template02. Refer to (2) to obtain each input value input to a predetermined entry on jpg. Corresponding to “Photo Image No”: 001 is “Construction Name”: XX Building Renovation, “Work Type”: Complete Architecture (Architecture), “Position”: (36.140), Free Space: Since there is nothing, each of these input values is acquired.

  It is assumed that the “blackboard image” is Template01. If it is jpg, refer to (1) in order to obtain each input value input to a predetermined entry on Template01.jpg, and if the “blackboard image” is Template03.jpg. If it is jpg, “(3) may be referred to in order to obtain each input value input to the predetermined entry on Template03.jpg. Since the predetermined entry differs depending on the“ blackboard image ”, Management information is sorted for each blackboard image template.

  Referring again to (0), “Photo Image No.”: “001 Blackboard Image”: Template02. Each initial value for jpg is “blackboard color”: black, “blackboard position”: upper left, “blackboard shape”: horizontal, “blackboard size”: W250 × H200, so obtain these initial values. To do.

  And "blackboard image": Template02. The blackboard image is edited so that “blackboard color”: black, “blackboard shape”: horizontal, and “blackboard size”: W250 × H200 with respect to jpg. Next, “blackboard image”: Template02. For the given entry on jpg, enter the input value of “construction name”: XX building renovation, “work type”: complete building (architecture), “position”: (36.140), free space: none By doing so, a blackboard image in which entry items have been entered is created. “Photo image” which is the last photo image taken: 001. By placing this blackboard image on “blackboard position”: upper left so as to be superimposed on jpg, a photographic image with a virtual blackboard is reconstructed and displayed (see FIG. 13). Of the above-described methods for saving a photo image with a virtual blackboard, in the former method, the photo image with a virtual blackboard to be reproduced is already stored, so there is no management information for storing a photo with a virtual blackboard. It is possible to reproduce photographic images with a virtual blackboard.

  In the smart device 1 according to the present embodiment, when the created photo with virtual blackboard is saved, the photo with virtual blackboard “image” is saved in the storage unit 101 as it is. The smart device 1 stores, in the storage unit 101, a photographed photograph image, a virtual blackboard image template, “photograph management management information with a virtual blackboard” (position information and size information of the virtual blackboard image on the photograph image) , Data input to predetermined entry items of the virtual blackboard image) may be associated with each other and stored. In this case, the smart device 1 uses the virtual blackboard image template, the photographed photo image, and the virtual blackboard-added photo storage management information stored in advance in the storage unit 101 to generate a virtual blackboard-added photo image. Can be reproduced.

(About editing collation information and photo images with virtual blackboard)
Here, for the collation information and the method for saving the virtual blackboard-attached photographic image, data is input to all predetermined entries on the virtual blackboard, the virtual blackboard is placed on the photographic image at a predetermined position, size, etc., and a data storage request is made Is instructed, collation information is calculated and stored as collation information and a photo “image” with a virtual blackboard. That is, by combining the virtual blackboard and the photographic image together, it is difficult to post-edit, that is, tamper with the collation information, the collation information with the virtual blackboard information and the photographic image with the virtual blackboard.

(About editing photo images with virtual blackboard)
In the smart device 1 according to the present embodiment, when the created photo with virtual blackboard is saved, the photo with virtual blackboard “image” is saved in the storage unit 101 as it is. Until the editing and saving are completed, the photographed photographic image, the virtual blackboard image template, and “virtual blackboard-attached photo storage management information” (location information of the virtual blackboard image on the photographic image, The size information and the data input to the predetermined entry of the virtual blackboard image are associated with each other and temporarily stored. When the above-mentioned photo with the virtual blackboard is stored, the other temporarily stored The data of is deleted.

  Note that, in the smart device 1 according to the present embodiment, among the “blackboard image setting parameters” included in the “management information for storing virtual photo with blackboard” described above, at the time of editing before the photo with virtual blackboard is stored. The information may be changed by rewriting only the information on “blackboard position” or “blackboard size”. For example, referring to (0), in the photo with virtual blackboard stored as “Photo Image No”: 003 in the third row, “position of blackboard” (position coordinates (10.10) on photo image) and “ The “blackboard size” (W120 × H100) is changed from the initial value (see, for example, FIGS. 19B and 19D).

  However, the storage method of storing the photographed photograph image, the virtual blackboard image template, and the “virtual blackboard-added photo storage management information” in association with each other is “construction site blackboard application”. It is desirable to restrict so that it is possible only with the smart device 1 or the PC that has the device.

  In this embodiment, in order to submit a photographic image with a virtual blackboard as a construction report to a construction orderer or the like, collation information in which the virtual blackboard is integrated on the photographic image and a photographic image with a virtual blackboard are generated. .

  The “construction site blackboard application” of the smart device 1 according to the present embodiment can finally output a JPEG image in which such a virtual blackboard is integrally synthesized on a photographic image. In addition, in order to be able to check whether the JPEG image has been tampered with at the construction ordering party, etc., the collation information and the photo with the virtual blackboard at the “save” timing (B4 in FIG. 7, FIG. 13). An image can be generated and output.

  Further, as described above, the collation information and the on-site photo with virtual blackboard stored in the smart device 1 are the server 4, the PC 5, or the recording medium after selecting one or more arbitrary photos in the “photo storage”. However, in this case, if the external receiving side supports the same storage management format stored in the smart device 1, it can be output as it is.

[Output method of collation information and photos with virtual blackboard]
Next, the output method of collation information and a photograph with a virtual blackboard will be described. In the “construction site blackboard application” of the smart device 1 according to the present embodiment, collation information and a photo image with a virtual blackboard are stored. In the “construction site blackboard application”, in consideration of editability, as in the latter storage method described above, the photographed photograph image in the storage unit 101, the virtual blackboard image stored in advance as a template, and “ "Photo management management information with virtual blackboard" (position information and size information of the virtual blackboard image on the photo image and data input to the predetermined entry of the virtual blackboard image) can be associated with each other and stored. It is.

  However, if the editability is high (when editing becomes easier), the risk of falsification of the photographic image and the virtual blackboard image increases accordingly. Therefore, for example, collation information and a photographic image with a virtual blackboard are transmitted as a construction report to the construction orderer. As a result, the construction orderer can easily determine whether or not tampering has occurred based on the collation information and the collation information attached to the photo image with the virtual blackboard.

(D operation example)
In outputting the collation information and the photographic image with the virtual blackboard, first, the user selects the target collation information and the photographic image with the virtual blackboard. Specifically, D operation for referring to a photograph with a virtual blackboard is performed (FIG. 7). Since the created collation information and the photo image with the virtual blackboard are stored in the “photo storage”, the user can select any photo from the collation information and the photo image with the virtual blackboard already created from here. Output.

  FIG. 24 shows an example (part 2) of the screen D1 according to the present embodiment. When the operation button “photo storage” is operated on the TOP screen A, the screen changes to this screen (same as FIG. 17). As shown in the figure, in the “photo storage” of the collation information on the screen D1 and the photo image with virtual blackboard, the collation information and the photo with virtual blackboard created so far by the B operation or C operation are displayed in a list. Each photo is displayed in a reduced size in a thumbnail format. For example, when the user touches a desired photo, the photo is enlarged and displayed on the screen. Here, it is assumed that the rightmost figure in the figure is touched.

  FIG. 25 shows an example (part 3) of the screen D1 according to the present embodiment. On the screen D1, the selected collation information and the photo image with the virtual blackboard are displayed. Subsequently, in order to output the collation information and the virtual blackboard-attached photographic image as image data in JPEG format, the user touches the operation button “image output” on the screen to output the photographic image.

  The JPEG image output here is stored in an arbitrary memory of the smart device 1. The storage destination can be output not only to the smart device 1 built-in memory and recording medium (for example, SD card) but also to the server 4, the PC 5, the cloud storage 7, and the like.

  FIG. 26 shows an example (part 4) of the screen D1 according to the present embodiment. As shown in the figure, when the selected collation information and the photo image with virtual blackboard are output as a JPEG image, a display to that effect is displayed.

  FIG. 27 shows an output JPEG image according to the present embodiment, that is, collation information and a photographic image with a virtual blackboard. As shown in the figure, this JPEG image is the one in which the selected collation information and the photographic image with a virtual blackboard are synthesized as a single image on the photographic image and output as image data of a JPEG file. . Next, Exif data of the output JPEG image according to the present embodiment will be described.

(Exif data)
First, general Exif data (also referred to as Exif metadata, Exif information) will be described. Generally, when a photograph is taken using a digital camera and the photographed photograph image is saved as a JPEG file, for example, metadata in Exif format is automatically created. This metadata is called Exif data.

  Specifically, the digital camera embeds information such as the camera model and shooting conditions in the image as Exif data. This Exif data can be referred to together with a photographic image by an image viewer or Exif editing software. When printing a photographic image with a printer or the like, the photographic image can be automatically optimized based on information at the time of shooting included in the Exif data, and then printed out.

  FIG. 28 shows an example of EXIF data according to the conventional example. The EXIF data is data that is automatically created by a general digital camera (software included in a general digital camera) when a photographed photographic image is stored as a JPEG file. Specifically, it includes various types of information such as the camera model and shooting conditions as shown in FIG.

  On the other hand, FIG. 29 shows an example of EXIF data of the output JPEG image according to the present embodiment. The EXIF data shown in the figure is data that is automatically created by the smart device 1 according to the present embodiment (“construction site blackboard application” of the smart device 1) when a photographic image with a virtual blackboard is output as a JPEG image. It is. Specifically, in addition to the camera-specific data such as the camera model and shooting conditions as shown in the figure, the virtual blackboard information and collation information input in the predetermined entry items of the virtual blackboard image are included.

  Exif data includes a storage area (standard area) for storing information (Exif standard data) such as main information and sub-information such as camera models and shooting conditions standardized and defined in the standard, as well as manufacturers, etc. Provides a storage area (private area) where users can define parameters and store information. Each manufacturer can add “unique manufacturer information” defined in the private area. Therefore, when a photo image with a virtual blackboard is output as a JPEG image by the smart device 1 (“construction site blackboard app” of the smart device 1) according to the present embodiment, the “manufacturer unique information” of the Exif data includes: Data (virtual blackboard information) and collation information input to a predetermined entry in the virtual blackboard are stored (added).

(JPEG image output processing)
FIG. 30 shows a flowchart of processing for outputting collation information and a virtual blackboard-attached JPEG image according to this embodiment. A description will be given with reference to FIG.

  S2: The image editing unit 106 performs JPEG image generation processing on the photographed photograph image. Specifically, the image editing unit 106 superimposes the virtual blackboard on the photographic image according to the blackboard image setting parameters (the size and position of the virtual blackboard).

  S4: The verification information calculation unit 107 calculates verification information based on the photo image with virtual blackboard and the metadata. Specifically, collation information is generated when an operation button on the screen is touched after the photographic image with a virtual blackboard is displayed.

  S6: The collation information storage unit 108 stores virtual blackboard information and collation information in a private area of Exif data according to a predetermined definition sentence. Here, as the virtual blackboard information, the “construction name” and “shooting date” of the input data are used. However, the items to be used are not limited to this example. Absent.

  S8: The image output unit 110 determines whether or not there is an output request for the photo image with virtual blackboard (hereinafter referred to as “verification information and photo image with virtual blackboard”) in which the collation information and the virtual blackboard information are stored. . Specifically, when the operation button “image output” on the screen is touched with respect to the collation information being displayed and the photographic image with virtual blackboard, it can be determined that there is an output request (FIG. 25).

  S10: If the image output unit 110 determines that there is an output request, it outputs collation information and a photographic image with a virtual blackboard (FIG. 29).

[Processing example of PC6]
FIG. 32 shows processing for determining whether or not the collation information and the virtual blackboard-attached photographic image in the PC 6 have been tampered with.

  The collation information and the photo image with virtual blackboard transmitted from the smart device 1 are received by the communication unit 603 and input to the image acquisition unit 604. When the operation for acquiring the collation information and the photographic image with the virtual blackboard is performed, the image acquisition unit 604 inputs the collation information and the photographic image with the virtual blackboard to the storage unit 601, and the collation information and the photographic image with the virtual blackboard are input. When the operation for inspecting whether or not tampering has been performed is performed, the collation information and the photographic image with the virtual blackboard are input to the storage unit 601 and the tampering determination unit 605.

  When the collation information and the photo image with virtual blackboard are input to the falsification determination unit 605, the collation information removal unit 605a converts the verification information and the private image area of the photo image with virtual blackboard (1) into a predetermined definition sentence. Accordingly, the virtual blackboard information and the collation information (hereinafter referred to as “first collation information”) are acquired and removed. The collation information removing unit 605a sends the virtual blackboard information, the collation information from which the collation information is removed, and the photo image with the virtual blackboard (hereinafter referred to as “photo image with the virtual blackboard with the collation information removed”) to the collation information calculation unit 605b. In addition to the input, the first verification information is input to the verification information verification unit 605c.

  The collation information calculation unit 605b is based on a predetermined function such as a preset hash function based on the photo image with the virtual blackboard with the collation information removed (image for collation information generation) input from the collation information removal unit 605a. The collation information (hereinafter referred to as “second collation information”) is generated by calculation or the like and input to the collation information collating unit 605c (2). The predetermined function is not limited to the hash function described above. For example, when an arbitrary function can be set among a plurality of preset functions, the same function with the smart device 1 is used. Will be set.

  The collation information collating unit 605c performs collation by determining whether or not the first collation information input from the collation information removing unit 605a matches the second collation information input from the collation information calculating unit 605b. (3).

  When the first collation information and the second collation information match, the collation information collation unit 605c determines that the on-site photo with virtual blackboard has not been falsified, and indicates that the display unit 602b has not falsified. indicate. On the other hand, when the first verification information and the second verification information do not match, the verification information verification unit 605c determines that the site photo with virtual blackboard has been tampered with, and the verification information and site photo with virtual blackboard Is moved to the error folder.

(Falsification judgment processing)
FIG. 33 shows a flowchart of processing for determining whether or not the collation information and the photo image with virtual blackboard in the PC 6 according to the present embodiment have been tampered with.

  S12: The collation information and the virtual blackboard-attached photographic image transmitted from the smart device 1 are received by the communication unit 603 and input to the image acquisition unit 604. The image acquisition unit 604 stores the collation information and the photographic image with a virtual blackboard in the storage unit 601.

  S14: The manipulation check tool of the PC application program (for example, “construction site blackboard application”) is activated when the user operates the operation unit 602a.

  S16: The display unit 602b displays collation information stored in the storage unit 601 and a photo image thumbnail with a virtual blackboard.

  S18: By operating the operation unit 602a, the user can select a desired one from the collation information displayed as a thumbnail by the display unit 602b and the photo image with the virtual blackboard. The tampering determination unit 605 checks whether the collation information and the photo image with virtual blackboard selected by the user operating the operation unit 602a have been tampered with.

  S20: The tampering determination unit 605 determines a check result. The falsification determination unit 605 can determine whether or not the falsification has been made based on whether or not the first collation information and the second collation information match.

  S22: If the collation information and the photo image with virtual blackboard are falsified, the falsification determination unit 605 moves the collation information and the photo image with virtual blackboard to the error folder.

  S24: After the collation information and the photo image with virtual blackboard are moved to the error folder in step S22, or when the collation information and the photo image with virtual blackboard are not falsified, the check result is output.

  In the Exif data, the collation information and the photo image with virtual blackboard according to the present embodiment represent not only data unique to the digital camera such as the camera model and shooting conditions but also the on-site photo with virtual blackboard and metadata. Contains value (matching information). For this reason, the collation information is calculated from the collation information and the virtual blackboard-attached photographic image obtained by removing the collation information at the delivery destination terminal, and the collation information and the collation information and the removed collation information are matched. It can be determined whether or not the photographic image with the virtual blackboard has been tampered with. Since the verification information is stored in the private area of the Exif data, it is not obtained from a third party.

(Modification of photo information with collation information and virtual blackboard)
Here, the collation information and the photographic image with virtual blackboard used in the above-described falsification determination process are not limited to the example of FIG. 15 described above. Below, the modification of the collation information applied to this embodiment and the photograph image with a virtual blackboard is demonstrated using figures. In the following description, each modification is different in the content of the photographic image with a virtual blackboard in which the collation information stored in the storage unit 101 is stored. When the confirmation is completed, the operation button “Save” in the screen of FIG. 13 is touched to save the photograph image. Further, the collation information is calculated from the photo image with the virtual blackboard and the metadata in the Exif format, and a photo image with the virtual blackboard (for example, the collation information and the on-site photo with the virtual blackboard) is created and stored. 101 is stored.

(First modification)
FIG. 34 is a diagram for explaining a first modification of the collation information and the photographic image with a virtual blackboard. In the processing shown in FIG. 34, the construction site is photographed by the camera (imaging unit 103) of the smart device 1 upon receipt of a photographing operation by the user (1), and the photograph image is acquired (2). In the imaging control (2), Exif format metadata is created by the application of the smart device 1.

  In addition, the smart device 1 (2) configures the on-site photograph by superimposing (inserting) the above-described virtual blackboard on the acquired photograph image (3). Next, in response to a user instruction, when there is a request to save an image, the collation information calculation unit 107 performs collation information (for example, hashing) from the photographic image with a virtual blackboard in the JPEG file format input from the image editing unit 106. Value) is generated by calculation or the like (4). In the example of FIG. 15 described above, the collation information is generated from the photo image with virtual blackboard and its Exif type metadata. However, in the first modification, the photo with virtual blackboard obtained in (3). Collation information is generated using an image. In the first modification, when generating the collation information, the entire photographic image with a virtual blackboard may be used, or a part of a preset area may be used.

  The collation information storage unit 108 stores the collation information generated by the collation information calculation unit 107 in a private area of the metadata. In the first modification, when each piece of metadata (Exif type) is falsified, it cannot be determined from the collation information. In particular, since the definition of Exif standard data is open to the public, tampering is easy. Therefore, in the first modified example, the collation information storage unit 108 may store information (Exif standard data that should not be tampered with) preset from standard metadata (Exif standard Exif standard data) in the private area. Good. The preset information includes, for example, “shooting date” data input as Exif standard data by a camera function, but is not limited thereto. By storing these pieces of information in the private area, even if the metadata is falsified, the necessary information can be obtained from the private area. Moreover, since the private area is a secure area unique to the manufacturer, information cannot be obtained or altered by a third party.

  In the first modification, the collation information storage unit 108 may store information such as a camera memo in a private area. A camera memo is additional data associated with an image that can be arbitrarily set by an application program that uses an imaging device or an imaging function. In addition to the above-described virtual blackboard information such as “construction name”, the collation information storage unit 108 can store information such as “manager” as a camera memo. The “manager” may be a user name registered in advance in the smart device 1 as information related to the construction, and the construction site blackboard application of the smart device 1 has a login function. It may be the user name of the user who has logged into the app. Thereby, collation information and a photograph image with a virtual blackboard (for example, collation information and a field photograph with a virtual blackboard) are created. The created collation information and photographic image with virtual blackboard are stored in the storage unit 101 (5).

  In the first modification described above, the collation information is generated using part or all of the photo image with the virtual blackboard. However, for example, collation information may be generated using part or all of the metadata. Good.

  Thereby, for example, in the process of determining whether or not the collation information and the virtual blackboard-attached photo image on the PC 6 described above have been tampered with, the first modification performs the same process as the process shown in FIG. The collation information to be compared is different. In the first modification, the falsification determination unit 605 calculates collation information based on a predetermined function such as a preset hash function based on the photographic image with virtual blackboard (collation information generation image). Further, the tampering determination unit 605 obtains collation information from the private area of the metadata (Exif data), and compares the two pieces of collation information to perform a collation check. Further, if the two collation information matches with the collation result, the falsification determination unit 605 determines that the on-site photo with virtual blackboard has not been falsified and displays that the falsification has not been performed on the display unit 602b. If the two pieces of collation information do not match, the falsification determination unit 605 determines that the on-site photo with the virtual blackboard has been falsified, and moves the collation information and the on-site photo with the virtual blackboard to the error folder. Further, in the first modification, camera memos and Exif standard data that have not been tampered with can be acquired from the private area. The Exif standard data stored in the private area is compared with the Exif standard stored in the Exif standard area (an area in which main information and sub information are stored) to determine whether the Exif standard area has been tampered with. May be.

(Second modification)
FIG. 35 is a diagram for explaining a second modification of the collation information and the photographic image with a virtual blackboard. In addition, about (1)-(3) of the process of FIG. 35, since the process same as the 1st modification mentioned above is performed, description here is abbreviate | omitted.

  In the second modification, when generating the collation information, the collation information calculation unit 107 calculates the collation information based on the on-site photo with virtual blackboard and metadata as in FIG. 15 described above (4). However, in the second modification, the above-described camera memo is stored in a private area of metadata (Exif data), and collation information is calculated including the metadata storing the camera memo. By storing information such as a camera memo in the private area, collation information including information stored in the private area can be generated, and collation information can be generated more securely.

  The generated collation information is stored in the private area of the metadata, and the created collation information and the photo image with the virtual blackboard are stored in the storage unit 101 (5).

  In the second modification, in the process of determining whether or not the above-described collation information and the virtual blackboard-attached photo image on the PC 6 have been tampered with, the collation information removing unit 605a described above removes the collation information, and Since the same processing as that in FIG. 32 is performed except that the photo image with the virtual blackboard with the verification information removed without removing the camera memo is input to the verification information calculation unit 605b, the description is omitted here.

(Third Modification)
FIG. 36 is a diagram for explaining a third modification of the collation information and the photographic image with a virtual blackboard. In addition, about (1)-(2) of the process of FIG. 36, since the process same as the 1st modification mentioned above is performed, description here is abbreviate | omitted.

  For example, in the case of detecting tampering of a construction report or the like showing details of construction contents, it may be sufficient to prevent falsification of contents entered on a virtual blackboard and photographic images of construction sites and the like. Therefore, in the third modification, the collation information calculation unit 107 calculates collation information using a photo image without a virtual blackboard as a collation information generation image (3). The generated collation information is stored in a private area of metadata. In the third modification, the collation information storage unit 108 stores part or all of the virtual blackboard information of the contents written on the virtual blackboard in the private area as a camera memo. The camera memo may include coordinate information indicating the position of the virtual blackboard on the construction photo. Further, the coordinate information may be stored directly in the private area without being included in the camera memo. The generated collation information and photo image with virtual blackboard are stored in the storage unit 101 (4). In the third modification, the photo image with virtual blackboard is the latter of the above-described storage methods, that is, the photographed photo image, the virtual blackboard image template, and the management information for photo storage with virtual blackboard. And is output to the PC 6.

  Thereby, for example, in the process for determining whether or not the collation information and the virtual blackboard-attached photo image on the PC 6 described above have been tampered with, the third modification performs the same process as the process shown in FIG. The collation information to be compared is a different photographic image. In the third modification, the tampering determination unit 605 calculates the collation information based on a predetermined function such as a preset hash function based on the photographic image. In addition, the falsification determination unit 605 obtains collation information from the private area of the metadata (Exif data), and compares the two pieces of collation information (the calculated collation information and the obtained collation information) to perform a collation check.

  Further, the tampering determination unit 605 may acquire character information from the image of the virtual blackboard by character recognition processing such as OCR (Optical Character Recognition) and collate it with information described in the camera memo. Further, the tampering determination unit 605 acquires the position (coordinates) of the virtual blackboard by matching using a template corresponding to a preset size of the virtual blackboard, and acquires the acquired position information and the position information included in the camera memo May be compared.

  The tampering determination unit 605 determines that the on-site photo with virtual blackboard has not been tampered with and displays on the display unit 602b if the two pieces of information to be compared match based on the collation result. Further, if the two pieces of information to be compared do not match, the tampering determination unit 605 determines that the site photo with virtual blackboard has been tampered with, and moves the verification information and the site photo with virtual blackboard to the error folder.

  As a result, in the third modification, it is determined whether the photographic image has been tampered with based on the collation information generated from the photographic image (collation information generation image), and the virtual blackboard is based on the virtual blackboard information (camera memo) stored in the private area. It is possible to determine whether or not the content entered in the item is falsified. Further, in the third modification, the on-site photo with the virtual blackboard is not integrally synthesized, so that the on-site photo with the virtual blackboard displayed as described above can be edited. Naturally, editing that leads to tampering should not be possible, but for example, the position of the virtual blackboard can be shifted, so by enabling such editing, a photographic image of the area where the virtual blackboard is arranged It becomes possible to confirm the contents of. Thereby, for example, a concealing act of hiding a portion of the photographic image that is not desired to be seen on the virtual blackboard can be prevented.

  Note that it is possible to determine whether the virtual blackboard in the received photographic image with the virtual blackboard is arranged in an area that matches the coordinate information, using the coordinate information stored in the private area. This determines whether the position of the virtual blackboard is shifted or editing such as reduction / enlargement is performed until the PC 6 receives the photo image with the virtual blackboard stored in the smart device 1. be able to.

  In the above-described modification, one collation information is generated from part or all of the on-site photo with a virtual blackboard and a photographic image. However, in the present embodiment, the present invention is not limited to this. May be generated. In this case, for example, collation information for the photographic image is generated, and collation information is also generated from the virtual blackboard and stored in the private area. In the falsification determination process, falsification is performed using a plurality of pieces of collation information stored in the private area. Judgment can be made. As the verification information increases, more detailed tampering determination can be performed.

  The PC 6 is an example of an information processing apparatus. JPEG is an example of a predetermined image file.

[Summary]
As described above, according to the present embodiment, when a site photograph is taken at the construction site, the efficiency of the work is improved, and the photograph image with the virtual blackboard in which the virtual blackboard is arranged on the photographed site photograph is altered. It is possible to provide a system capable of detecting this.

  In the above-described embodiment, the present invention has been described using an on-site photo shooting operation at a construction site as an example. However, the scope of the present invention is not limited to this. For example, as one aspect of the present invention, additional information (corresponding to the virtual blackboard information in the above-described embodiment) input by the user at the timing of taking a photo and added to the photo, and the taken photo This is applicable when it is desired to prevent the image data from being tampered with after taking a picture. Also, for example, in the business of electronically certifying report information (information on work status, work progress, site situation, etc.) with evidence that is input by a user and declared for a photographic image. It can be applied as a method of remaining evidence. An example of electronic evidence-remaining is to leave evidence by taking images to prove the state after the completion of a predetermined work (cleaning) in cleaning operations such as toilets, butts, and garbage collection. However, the present invention is not limited to this.

  Although the present invention has been described based on each embodiment, the present invention is not limited to the requirements shown here, such as combinations with other elements listed in the above embodiments. With respect to these points, the present invention can be changed within a range that does not detract from the gist of the present invention, and can be appropriately determined according to the application form. Moreover, what applied the component, expression, or arbitrary combinations of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, etc. is also effective as an aspect of this invention.

  Although the present invention has been described with reference to specific embodiments and modifications, each embodiment and modification is merely an example, and those skilled in the art will recognize various modifications, modifications, alternatives, substitutions, and the like. Will understand. For convenience of explanation, an apparatus according to an embodiment of the present invention has been described using a functional block diagram, but such an apparatus may be implemented in hardware, software, or a combination thereof. The present invention is not limited to the above embodiments, and various modifications, modifications, alternatives, substitutions, and the like are included without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Smart device 2 Surveying instrument and measuring instrument 3 Construction material 4 Server 5 PC
6 MFP / LP
11 CPU
12 Storage Device 13 Display Device 14 Input Device 15 Communication Device 16 External Interface Device 17 Imaging Device 18 GPS
61 CPU
62 storage device 63 display device 64 input device 65 communication device 66 external interface device 101 storage unit 102 touch panel 103 imaging unit 104 communication unit 105 input unit 106 image editing unit 107 collation information calculation unit (first collation information generation unit)
108 collation information storage unit 109 image reproduction unit 110 image output unit 601 storage unit 602 touch panel 603 communication unit 604 image acquisition unit 605 falsification determination unit 605a collation information removal unit (collation information acquisition unit, collation information generation image creation unit)
605b Collation information calculation unit (second collation information generation unit)
605c Verification information verification unit

JP 2000-287164 A

Claims (12)

A smart device equipped with a storage device
A shooting section for taking photographic images;
An input unit for inputting data to the item of the virtual display panel image in which a predetermined item can be entered;
An image editing unit for creating a photographic image with a virtual display board in which a virtual display board image in which data is input to the item by the input unit is superimposed on the photographic image taken by the photographing unit;
A first collation information generating unit that generates collation information for the photographic image with a virtual display board based on at least the photographic image among the photographic images with the virtual display board;
Application for causing the collation information and the data input to the items to function as a collation information storage unit for creating collation information and a photo image with a virtual display board, which are stored as metadata for the photo image with a virtual display board program. The image editing unit
As one image obtained by combining the virtual display board image on the photographic image, creating the photographic image with the virtual display board according to a predetermined image format,
The first collation information generation unit
The application program according to claim 1, wherein the collation information is generated based on the photographic image with a virtual display board. The image editing unit
Create a photographic image with the virtual display board containing metadata,
The first collation information generation unit
The application program according to claim 1, wherein the verification information is generated based on the photographic image with a virtual display board including the metadata. The first collation information generation unit
The application program according to claim 3, wherein the collation information is generated based on the photographic image with a virtual display board including the metadata in which data input to the item is stored. The metadata is
A standard area for storing information standardized and defined in the standard, and a private area for storing uniquely defined information,
The collation information storage unit
The application according to any one of claims 1 to 4, wherein the collation information and the photographic image with a virtual display board are created by storing the collation information and data input to the item in the private area in the metadata. program.   The application program according to claim 1, wherein the photographic image with a virtual display board is a JPEG image format, and the metadata is Exif data. An information processing apparatus that receives the collation information and a photographic image with a virtual display board created by a smart device that functions according to the application program according to any one of claims 1 to 6,
A collation information obtaining unit for obtaining the collation information from the received collation information and the photographic image with a virtual display panel, and data input to the item;
Collation information generation image creation for creating a collation information generation image that is an image in which at least the collation information is removed from the received collation information and photo image with virtual display panel, and is an image for generating collation information And
A second verification information generation unit that generates verification information based on the verification information generation image;
An application program that functions as a collation information collation unit that collates collation information acquired by the collation information acquisition unit and collation information generated by the second collation information generation unit and outputs a collation result. A smart device comprising a storage device,
A shooting section for taking photographic images;
An input unit for inputting data to the item of the virtual display panel image in which a predetermined item can be entered;
An image editing unit for creating a photographic image with a virtual display board in which a virtual display board image in which data is input to the item by the input unit is superimposed on the photographic image taken by the photographing unit;
A first collation information generating unit that generates collation information for the photographic image with a virtual display board based on at least the photographic image among the photographic images with the virtual display board;
A smart device comprising: the collation information; collation information in which data input to the item is stored as metadata for the photo image with virtual display board; and a collation information storage unit that creates a photo image with virtual display board. An information processing apparatus for receiving the collation information and the photo image with a virtual display board created by the smart device according to claim 8,
A collation information obtaining unit for obtaining the collation information from the received collation information and the photographic image with a virtual display panel, and data input to the item;
Collation information generation image creation for creating a collation information generation image that is an image in which at least the collation information is removed from the received collation information and photo image with virtual display panel, and is an image for generating collation information And
A second verification information generation unit that generates verification information based on the verification information generation image;
An information processing apparatus comprising: a collation information collation unit that collates collation information acquired by the collation information acquisition unit and collation information generated by the second collation information generation unit and outputs a collation result. An information processing system comprising a smart device comprising a storage device and an information processing device for receiving a photographic image created by the smart device,
The smart device is
A shooting section for taking photographic images;
An input unit for inputting data to the item of the virtual display panel image in which a predetermined item can be entered;
An image editing unit for creating a photographic image with a virtual display board in which a virtual display board image in which data is input to the item by the input unit is superimposed on the photographic image taken by the photographing unit;
A first collation information generating unit that generates collation information for the photographic image with a virtual display board based on at least the photographic image among the photographic images with the virtual display board;
A collation information storing unit that creates the collation information and the collation information stored as metadata for the photo image with the virtual display board and the data input to the item, and a photo image with the virtual display board;
The information processing apparatus includes:
A collation information obtaining unit for obtaining the collation information from the received collation information and the photographic image with a virtual display panel, and data input to the item;
Collation information generation image creation for creating a collation information generation image that is an image in which at least the collation information is removed from the received collation information and photo image with virtual display panel, and is an image for generating collation information And
A second verification information generation unit that generates verification information based on the verification information generation image;
An information processing system comprising: a collation information collation unit that collates collation information acquired by the collation information acquisition unit with collation information generated by the second collation information generation unit and outputs a collation result. An information processing method in a smart device including a storage device,
Take a photo image,
Enter data into the item on the virtual display panel image where you can fill in the predetermined item,
Create a photographic image with a virtual display board in which a virtual display board image in which data is input to the item is superimposed on the photographic image,
Based on at least the photographic image of the photographic image with a virtual display board, to generate first verification information for the photographic image with a virtual display board,
An information processing method for creating collation information and a photographic image with a virtual display board in which the first collation information and data input to the item are stored as metadata for the photographic image with a virtual display board. An information processing method in an information processing apparatus for receiving the collation information and the photo image with a virtual display board created by the smart device according to claim 11,
Obtaining the first collation information from the received collation information and the photographic image with a virtual display panel, and data input to the item,
A collation information generation image that is an image in which at least the collation information is removed from the received collation information and the photographic image with a virtual display panel, and is an image for generating collation information,
Generating second collation information based on the collation information generation image;
An information processing method for collating the first collation information with the second collation information and outputting a collation result.

Images