JP6705988B2 - Information processing system, control method thereof, and program - Google Patents

Description

Information processing system capable of allowing a user to easily store an image of an object to be read in association with a type selected from a plurality of types indicating the object to be read by an arbitrary storage method, and a control method thereof , And the program.

In recent years, it is common practice to scan a paper medium with an image scanner or the like, digitize it, and manage it.

For example, at bank counters, it was said that the identity verification document brought by the customer was copied and saved on paper media, but now it is possible to electronically save it by scanning the identity verification document. Many are done.

For example, Patent Document 1 below discloses a mechanism for scanning and saving a document placed on a placement surface by imaging the document from above.

Further, Patent Document 2 below discloses a mechanism for projecting an operable range on a projection surface onto a user's hand.

JP, 2013-258612, A JP, 2014-203176, A

MEANS TO SOLVE THE PROBLEM This invention relates to the image which image | photographed the to-be- readed object is linked|related with the classification selected by the user from the some classification|category which shows a to-be-readed object, and a user can memorize|store easily by arbitrary storage methods. The purpose is to provide.

To achieve the above object, an information processing system of the present invention, reading an object An information processing system imaging unit controls whether to store the images taken, a plurality of types showing the object to be read A reception unit that accepts a selection of a type to be captured by the image capturing unit from a user, and an area that overlaps with the captured first image when the object to be read is captured by the image capturing unit. When the second image in which the reading object is photographed by the photographing unit is stored before the first image, the first image is stored in association with the type of which selection is accepted by the accepting unit. When the second image is overwritten on the second image and the second image captured in a region that does not overlap with the region of the first image is stored, the first image is stored in the second image. Storage control means for controlling to store the first image and the second image in association with the type of which selection is accepted by the acceptance means without overwriting.
The control method for an information processing system of the present invention is a control method of an information processing system imaging unit read object controls whether to store the image taken, the information processing system, the object to be read a receiving step of receiving the user selection of the type to be photographed by the photographing unit from among a plurality of types shown, the information processing system, when the object to be read by the imaging unit was taken was the shooting If the second image in which the reading target is photographed by the photographing unit is stored in the area overlapping the first image before the first image, the selection is accepted in the acceptance step. If the first image is overwritten on the second image in association with the type, and the second image captured in a region that does not overlap the region of the first image is stored, A storage control step of controlling to store the first image and the second image in association with the type of which selection is accepted in the accepting step without overwriting the first image on the second image ; It is characterized by including .
Further, a program of the information processing system of the present invention is a program for causing a computer to function as the above information processing system.

ADVANTAGE OF THE INVENTION According to this invention, the image which image | photographed the read-out target object is linked|related with the type selected from the several type which shows a read-out target object, and a user can memorize|store easily by arbitrary storage methods. The effect of being able to provide.

It is a figure showing an example of a system configuration of an information processing system in an embodiment of the present invention. 3 is a diagram showing an example of an outline of an external structure and an internal structure of the projection imaging device 102. FIG. It is a figure which shows an example at the time of performing a product description using the projection image projected using the projection imaging device 102. FIG. 1 is a diagram showing an example of a hardware configuration of an information processing device 100 and a projection imaging device 102. 3 is a diagram showing an example of a functional configuration of the information processing device 100. FIG. It is a figure which shows an example of the flowchart which shows the flow of a series of processes in embodiment of this invention. It is a figure which shows an example of the flowchart which shows the flow of a detailed process of the reading target object reading process of FIG. FIG. 8 is a diagram showing an example of a flowchart showing a detailed processing flow of the image storage processing of FIG. 7. FIG. 9 is a diagram showing an example of a screen configuration of a read object list screen 900. It is a figure which shows an example of a screen structure of the reading target reading screen 1000 at the time of photographing a reading target. It is a figure which shows an example of the reading target reading screen 1000 in case a user moves a reading target after imaging. It is a figure which shows an example of the reading target read instruction|indication screen 1200 at the time of imaging a reading target and putting a reading target in the position which is projecting the image obtained by photography, and imaging. FIG. 13 is a diagram showing an example of a read target object reading instruction screen 1300 when a read target object is imaged and the read object is placed at a position other than the position where the image obtained by the image capture is projected. It is a figure which shows an example of the table structure of the image table 1400. It is a figure which shows an example of a table structure of the reading target position table 1500. It is a figure which shows an example of a screen structure of a customer ID input screen 1600. It is a figure which shows an example of a table structure of a customer ID table 1700. It is a figure which shows an example of a screen structure of a menu screen 1800. It is a figure which shows an example of the flowchart which shows the detailed flow of a reading target reading process in the 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing an example of a system configuration of an information processing system according to an embodiment of the present invention. The information processing system includes an information processing device 100 and a projection imaging device 102. The information processing apparatus 100 is communicatively connected to the projection imaging apparatus 102, and further communicatively connected to the display 410 and the input device 409. In FIG. 1, these devices are connected by wire, but may be wireless. Note that the configurations of various terminals connected to the system of FIG. 1 are examples, and there are various configuration examples depending on the use and purpose.

The information processing device 100 is a general-purpose device equipped with an operating system. The information processing apparatus 100 instructs various devices connected to the information processing apparatus 100 to execute various operations.

The projection imaging device 102 is a device including a digital camera 421 (imaging unit), a gesture sensor 422, and a projector 423 (projection unit). Projection is performed by the projector 423 according to an instruction from the information processing apparatus 100, the projected projection surface such as a desk is imaged by the digital camera 421, and the movement of the user's hand and the position of the object are recognized on the projection surface. ..

An example of the outline of the external structure and the internal structure of the projection imaging apparatus 102 will be described with reference to FIG.

FIG. 2 is a front view and a right side view of the projection imaging device 102. The hidden lines of the internal structure are represented by broken lines. The projection imaging apparatus 102 includes a storage unit 203 that stores a digital camera 421 (imaging unit), a gesture sensor 422, and a projector 423 (projection unit). The storage unit 203 includes a bottom surface 201, a left side surface 202a, and a right side surface 202b. It is sandwiched and fixed from the left and right.

Further, the upper surface 204 is installed above the left side surface 202a and the right side surface 202b, and a reflecting mirror 207 for reflecting the light emitted from the projector 423 is provided below the upper surface 204. That is, the projector 423 is installed so that it can project toward the reflecting mirror 207 through the first opening 205. Similarly, since the digital camera 421 also captures an image of the projection surface reflected by the reflecting mirror 207, the digital camera 421 is installed toward the reflecting mirror 207 through the second opening 206.

The gesture sensor 422 is, for example, a depth sensor that uses infrared rays. This calculates the distance from the gesture sensor 422 to these objects in order to specify the position and movement of the user's hand, paper document (hereinafter referred to as document), etc. on the projection surface projected by the projector 423. The gesture sensor 422 calculates a distance to an object by projecting a dot pattern of near infrared rays on the real space and acquiring the shape and interval of the dots. Alternatively, the distance may be calculated by projecting infrared light, reading the infrared light reflected by the camera included in the gesture sensor 422, and calculating the time when the infrared light travels back and forth. Any method may be used for calculating the distance of the gesture sensor 422 as long as the position and movement of the object in the physical space can be acquired, and a sensor other than the gesture sensor 422 may be used. Further, the gesture sensor 422 is also installed toward the reflecting mirror 207 through the second opening 206, similarly to the digital camera 421. Infrared rays are emitted toward the reflecting mirror 207.

As described above, the projection imaging device 102 largely includes three devices, which are connected to the information processing device 100 so that they can communicate with each other. In the present embodiment, the information processing device 100 and the projection imaging device 102 have different casings, but they may have the same casing. That is, the projection imaging device 102 may further include the information processing device 100.

Next, the positional relationship between the projection imaging apparatus 102 and the user in this embodiment will be described with reference to FIG. The projection imaging device 102 is installed on the desk 303. In this embodiment, there are two users across the desk. A teller 301 who explains the product at a bank window and a customer 302 who receives the product description.

When the teller 301 operates the input device 409 to instruct the information processing apparatus 100 to display a screen, the information processing apparatus 100 transmits the screen projection instruction to the projector 423. Upon receiving this, the projector 423 projects a left-right inverted screen toward the reflecting mirror 207. This is because the image projected on the projection surface 304 is horizontally inverted by being reflected by the reflecting mirror 207. In this way, the designated screen is projected on the projection surface 304. The process of inverting the screen to the left or right may be performed by the information processing device 100 or the projector 423. The projection surface 304 is preferably a white surface without a pattern so that the projected light is easy to see.

When the light emitted from the projector 423 is reflected by the reflecting mirror 207 in the direction indicated by the dashed arrow in FIG. 3, the light is projected on the projection surface 304. The teller 301 explains the product to the customer 302 while looking at the screen represented by this light. In the present embodiment, the description will be made on the premise that the teller 301 and the customer 302 are positioned with the desk in between, and are operated while looking at the projected screens.

FIG. 4 is a diagram illustrating an example of a hardware configuration of the information processing device 100 and the projection imaging device 102.

The CPU 401 centrally controls each device and controller connected to the system bus 404.

Further, the ROM 402 or the external memory 411 (corresponding to a storage unit) executes a control program of the CPU 401 such as a BIOS (Basic Input/Output System), an operating system program (hereinafter referred to as OS), and the information processing apparatus 100. Various programs and the like necessary for realizing the function are stored. The RAM 403 functions as a main memory, a work area, etc. of the CPU 401.

The CPU 401 loads various programs necessary for executing the processing into the RAM 403 and executes the programs to realize various operations.

Further, the general-purpose input/output controller 405 controls input/output with the input device 409 such as a keyboard and a mouse, and further with the digital camera 421 and the gesture sensor 422 of the projection imaging apparatus 102. These are connected by a bus standard such as USB (Universal Serial Bus).

The video controller 406 controls display on a display device or a projection device such as the display 410 or the projector 423 of the projection imaging device 102. The display 410 is, for example, a CRT or a liquid crystal display. Further, the video controller 406 instructs the display 410 and the projector 423 of the projection imaging apparatus 102 to display and project the same screen or different screens. These can be so-called mirroring and multi-monitor.

The memory controller 407 is a card type memory connected to a hard disk or a flexible disk or a PCMCIA card slot for storing a boot program, browser software, various applications, font data, user files, edit files, various data, etc. through an adapter. Access to the external memory 411.

The communication I/F controller 408 connects and communicates with an external device via a network and executes communication control processing on the network. For example, internet communication using TCP/IP is possible.

Note that the CPU 401 enables the display on the display 410 and the projection by the projector 423 of the projection imaging apparatus 102 by executing the outline font rasterization process in the display information area in the RAM 403, for example. Further, the CPU 401 enables a user instruction with a mouse cursor or the like (not shown) on the display 410. Further, the CPU 401 enables the user instruction by touching the screen projected on the projection surface 304 with the user's hand. That is, depending on the coordinate position of the user's hand in the physical space detected by the gesture sensor 422, it is detected whether a button or the like on the screen is pressed, and the CPU 401 is caused to execute the operation.

Various programs and the like used by the information processing apparatus 100 of the present invention to execute various processes described later are recorded in the external memory 411 (corresponding to a storage unit), and are loaded into the RAM 403 as necessary. It is executed by the CPU 401. Further, definition files and various information tables used by the program according to the present invention are stored in the external memory 411.

Next, the module configuration of the information processing apparatus 100 will be described with reference to FIG. Note that the module configuration of the device in FIG. 5 is an example, and there are various configuration examples according to the use and purpose.

The information processing device 100 includes, as functional units, a storage unit 501, a display control unit 502, a projection control unit 503, an imaging control unit 504, a sensor control unit 505, a coordinate conversion unit 506, an operation detection unit 507, A position determination unit 508 and an image management unit 509 are provided.

The storage unit 501 is a functional unit for storing information on various tables and information configuring various screens described below. Add/update/delete information as necessary.

The display control unit 502 is a functional unit for controlling display on the display 410 communicably connected to the information processing apparatus 100. When the display control unit 502 gives a display instruction for a screen or the like, the display instruction is transmitted to the display 410 via the video controller 406. Then, when the display 410 receives this instruction, the display 410 displays a screen or the like according to the instruction.

The projection control unit 503 is a functional unit that controls the projection of the projector 423 that is communicably connected to the information processing apparatus 100. When the projection control unit 503 gives a projection instruction for a screen or the like, the projection instruction is transmitted to the projector 423 via the video controller 406 similarly to the display control unit 502. When the projector 423 receives this instruction, the projector 423 projects a screen or the like on the real space according to the instruction.

The imaging control unit 504 is a functional unit for controlling imaging of the digital camera 421 communicatively connected to the information processing apparatus 100. When the image capturing control unit 504 issues an image capturing instruction of the projection surface 304, the digital camera 421 is instructed to image capturing via the general-purpose input/output controller 405. Upon receiving this instruction, the digital camera 421 captures an image and transmits the image data generated by the capture to the information processing device 100. The imaging control unit 504 receives this, horizontally inverts the image data, and then the storage unit 501 stores it in the external memory 411 or the like. As described above, since the digital camera 421 captures the image reflected on the reflecting mirror 207, the left and right are inverted. In order to correct this, the imaging control unit 504 laterally reverses the image data before storing the image data in the storage unit 501.

The sensor control unit 505 is a functional unit for receiving information on the real space (presence or absence of an object, position, etc.) on the projection plane 304, which is transmitted from the gesture sensor 422 communicatively connected to the information processing apparatus 100. .. The information in the physical space detected by the gesture sensor 422 is transmitted from the gesture sensor 422 to the information processing apparatus 100 one by one. The sensor control unit 505 receives this and passes it to the coordinate conversion unit 506 described later.

The coordinate conversion unit 506 is a functional unit for converting the coordinate value indicating the position of the object on the projection plane 304, which is included in the information on the physical space received by the sensor control unit 505. The coordinate conversion unit 506 converts this coordinate value into a coordinate value on the screen projected on the projection surface 304. That is, which coordinate value on the screen managed by the information processing device 100 corresponds to the coordinate value detected by the gesture sensor 422 is specified using the coordinate conversion unit 506. Since the method of coordinate conversion uses a conventional technique, detailed description thereof will be omitted.

The operation detection unit 507 is a functional unit for detecting an operation from the user. It detects an instruction from the user transmitted through the input device 409 connected to the information processing apparatus 100, or detects an instruction issued by the user on the screen projected on the projection surface 304. To detect the latter instruction, the coordinate conversion unit 506 converts the coordinate value of the object detected on the projection surface 304 to detect which button on the screen is pressed, or the rectangle of the object on the projection surface. The object to be read is detected by detecting the.

The position determination unit 508 compares the position of the read object on the projection surface detected by the operation detection unit 507 with the shooting position 1404 stored corresponding to the image stored in the external memory 411, and the positions match. It is a functional unit that determines whether or not to do.

The image management unit 509 is a functional unit that manages images obtained by shooting with the digital camera 421. Depending on the position of the already read object to be read and the position of the object to be read, the image already taken is overwritten or a new image is saved.

This is the end of the description of the module configuration of the information processing apparatus 100.

Next, a flow of a series of processing according to the embodiment of the present invention will be described using the flowchart shown in FIG.

In step S601, the CPU 401 of the information processing apparatus 100 activates the projection application stored in the external memory 411 or the like according to an instruction from the user. The projection application is an application for executing each processing of steps S602 to S608 described later. By operating the projection application, an image is obtained by projecting a button or the like for reading the reading target object on the projection surface 304 or capturing an image of the reading target object placed on the projection surface 304. The image can be projected on the projection surface 304. In the embodiment of the present invention, the description of the process in which the teller 301 explains to the customer 302 will be omitted.

In step S602, CPU 401 of information processing device 100 causes display 410 to display customer ID input screen 1600. For example, on the customer ID input screen 1600 shown in FIG. 16, a customer ID input field 1601 and a next button 1602 are arranged. The customer ID input field 1601 is a field for inputting a customer ID which is an ID for identifying the customer 302 to which the teller 301 corresponds. The next button 1602 is a button for moving to the next process by accepting the depression.

The customer ID 1701 is stored in the customer ID table 1700 shown in FIG. The customer ID table 1700 stores a customer ID 1701 and a customer name 1702 in association with each other, and the customer ID 1701 is an ID for uniquely identifying the customer. Customer name 1702 is the name of the customer.

In step S603, the CPU 401 of the information processing device 100 receives the input of the customer ID 1701 from the input device 409.

In step S604, the CPU 401 of the information processing apparatus 100 accepts the press of the next button 1602.

In step S605, the CPU 401 of the information processing device 100 displays the menu screen 1800 on the display 410. On the menu screen 1800 shown in FIG. 18, a product description button 1801, a read target object read button 1802, and an end button 1803 are arranged. By accepting selection by the input device 409, processing corresponding to each is executed. In the embodiment of the present invention, the description of the product is omitted. After the product explanation to the customer 302 is finished, the teller 301 reads the read object necessary for the customer 302 to conclude a contract, which is the read object reading processing. Details will be described later.

In step S606, the CPU 401 of the information processing apparatus 100 determines whether or not the read target object read button 1802 on the menu screen has been pressed. If it is determined that the read object read button 1802 is pressed, the process proceeds to step S607. If it is determined that the read object read button 1802 is not pressed, the process proceeds to step S608.

In step S<b>607, the CPU 401 of the information processing apparatus 100 performs a reading target object reading process of acquiring an image of the reading target object by capturing an image of the reading target object placed on the projection surface 304 with the digital camera 421. Details will be described with reference to the flowchart shown in FIG.

In step S608, the CPU 401 of the information processing apparatus 100 determines whether the end button on the menu screen has been pressed. If it is determined that the end button has been pressed, the process proceeds to step S609, and if it is determined that the end button has not been pressed, the process returns to step S606.

In step S609, the CPU 401 of the information processing device 100 ends the projection application activated in step S601.

This is the end of the description of the flow of the series of processes according to the embodiment of the present invention.

Next, a detailed description of the reading target object reading process of the flowchart shown in FIG. 6 will be given using the flowchart shown in FIG. 7.

In step S<b>701, the CPU 401 of the information processing apparatus 100 causes the projector 423 to project the read object list screen 900 on the projection surface 304 and displays it on the projection surface 304. The reading object list screen 900 is a list of reading objects necessary for, for example, making a mortgage. An image of the object to be read is captured in the information processing apparatus 100 as image data. As shown in FIG. 9, both the teller 301 side and the customer 302 side display an area (hereinafter referred to as a read object type column) of the read object type (identity confirmation document etc.) that the customer 302 needs to submit, When a touch operation on any of the read object type columns is detected, a read object reading screen 1000 shown in FIG. 10 is projected and displayed. The reading target is read on the reading target reading screen 1000, that is, the digital camera 421 captures an image, and the information processing apparatus 100 acquires the image data generated by the imaging. When the reading is successfully completed, the screen returns to the reading object list screen 900, and the read image data is displayed in the frame of the reading object for which the touch operation is detected. For example, as shown in FIG. 9, when a touch operation on the driver's license column 903 is detected, a read target object reading screen 1000 for reading the driver's license is displayed. Then, the customer 302 photographs the driver's license placed on the projection surface 304. When the shooting is completed, the screen returns to the read object list screen 900, and thumbnail images of the images captured in the driver's license areas on both the teller 301 side and the customer 302 side are displayed. Image data 1403 corresponding to the read object type 1402 set for the area is acquired and a thumbnail image is displayed.

In step S<b>702, the CPU 401 of the information processing apparatus 100 accepts selection for any of the read target object types in the read target object type column as described above. The read object type 1402 is associated with the read object type, and the read object type 1402 that has been selected is temporarily stored in the RAM 403. The read object type 1402 is an ID for uniquely identifying the read object type.

In step S703, the CPU 401 of the information processing apparatus 100 projects the reading target object reading screen 1000 on the projection surface 304. On the reading target object reading screen 1000, a reading target object reading button 1001, a cancel button 1002, and a completion button 1003 are arranged. When the reading target object 1005 is placed on the reading target object reading screen 1000, a rectangle of the reading target object 1005 is specified, and the photographing position 1404 is projected according to the rectangle. This allows the user to know that the information processing apparatus 100 recognizes the reading target 1005. Then, when the user presses the reading target object reading button 1001, the projection surface 304 is photographed. When the cancel button 1002 is pressed, the image related to the read target object type 1402, the selection of which is accepted in step S702, is deleted. When the completion button 1003 is pressed, the image capturing process ends, and the image obtained by the image capturing is displayed in the driver's license column 903.

In step S704, the CPU 401 of the information processing apparatus 100 determines whether or not the reading target object is placed on the reading target object reading screen 1000. In the present embodiment, the gesture sensor 422 determines whether a rectangular object is placed. The determination as to whether or not the object to be read is placed is an example, and may be determined by image processing of an image acquired from the digital camera 421. If the information processing apparatus 100 determines that the reading target object is not placed, the process proceeds to step S715. If it is determined that the read object is placed, the process proceeds to step S705.

In step S705, the CPU 401 of the information processing apparatus 100 specifies the position and size of the read object that is determined to be placed on the projection surface 304 in step S704 (corresponding to an acquisition unit). More specifically, based on the information sent from the gesture sensor 422, the coordinates of two points on the projection surface 304 that define the size and position of the reading object are specified. The coordinates of the two points are stored in the read object position table 1500 shown in FIG. The reading object position table 1500 is a table for managing the reading object position 1501. The reading object position 1501 stores the coordinates of two points on the projection plane 304 that define the position and size of the reading object.

In step S706, the CPU 401 of the information processing apparatus 100 displays the frame 1004 surrounding the rectangle indicated by the coordinates of the two points specified in step S705. By performing this display, the user can know that the information processing device recognizes the reading target.

In step S707, the CPU 401 of the information processing device 100 determines whether or not the read target object read button 1001 displayed in step S706 has been pressed. If it is determined that the read object read button 1001 is pressed, the process proceeds to step S708. If it is determined that the read object read button 1001 is not pressed, the process returns to step S704.

In step S708, the CPU 401 of the information processing device 100 controls to hide the frame 1004 displayed in step S706. The frame 1004 is hidden here because it does not affect the white balance of the image when the digital camera 421 captures the image.

In step S709, the CPU 401 of the information processing apparatus 100 issues a shooting instruction to shoot the projection surface 304 with the digital camera 421 (corresponding to a shooting control unit).

In step S710, the CPU 401 of the information processing device 100 projects an animation, which indicates that shooting is being performed, onto the projection surface 304. By projecting the animation, it is possible to make the user recognize that the image is being captured.

In step S711, the CPU 401 of the information processing apparatus 100 performs an image saving process that is a process of saving an image captured by the digital camera 421.

In step S712, the CPU 401 of the information processing device 100 specifies the position and size of the reading target object on the projection surface 304. The specific method is the same as that in step S705, and thus the description is omitted. The coordinates of the two points are stored in the read object position table 1500 shown in FIG.

In step S713, the CPU 401 of the information processing device 100 determines whether or not the reading target has moved from the position when the reading target was photographed. More specifically, it is determined whether or not the rectangle indicated by the imaging position 1404 corresponding to the read object type 1402 identified in step S702 and the rectangle indicated by the read object position 1501 acquired in step S712 overlap. If it is determined that the rectangle indicated by the photographing position 1404 and the rectangle indicated by the read target object position 1501 acquired in step S712 overlap, it is determined that the read target object has not moved, and the process returns to step S712. If it is determined that the rectangle indicated by the shooting position 1404 and the rectangle indicated by the read target position 1501 acquired in step S712 do not overlap, it is determined that the read target has moved, and the process proceeds to step S714. The determination of overlap does not require that the two rectangles be exactly the same.

In step S714, the CPU 401 of the information processing apparatus 100 acquires the image stored in step S711 from the image data 1403 and projects the image onto the area indicated by the shooting position 1404 (corresponding to a projection control unit). For example, the reading target object reading screen 1000 shown in FIG. 11 will be described. When the reading target 1005 is moved from the shooting position, the image 1101 obtained by shooting the reading target 1005 is projected and displayed at the shooting position in the actual size. Then, after such an image 1101 is projected, the cancel button 1002 and the completion button 1003 can be pressed as in S715.

In this way, by projecting the imaged read object when the read object moves from the position at which the image was read, the area to be projected on the projection surface 304 is secured separately from the area where the image was taken. Without doing so, it is possible to project the read object and allow the user to confirm it. Furthermore, the visibility of the projection is improved by projecting the image of the reading target after the reading target moves. Further, the user cannot be sure to confirm the stored image by preventing the completion button 1003 from being pressed only after the reading object is moved and the stored image is projected.

In step S715, the CPU 401 of the information processing apparatus 100 determines which of the buttons arranged on the reading object reading screen 1000 has been pressed. If it is determined that the cancel button 1002 is pressed, the process proceeds to step S716. If it is determined that the completion button 1003 has been pressed, the process proceeds to step S717. If no button is pressed, the process returns to step S704, and if the read object is removed thereafter, it is determined No in step S704, and the process returns to step S715.

In step S716, the CPU 401 of the information processing apparatus 100 deletes the image corresponding to the read target object type 1402 of the read target object type selected in step S702.

In step S717, the CPU 401 of the information processing device 100 determines whether or not an end button (not shown) displayed on the display 410 has been pressed. If it is determined that the end button has been pressed, the process is terminated, and if it is determined that the end button has not been pressed, the process returns to step S701.

This is the end of the detailed description of the reading target object reading process in the flowchart shown in FIG.

Next, a detailed description of the image storage processing of S711 of the flowchart shown in FIG. 7 will be given using the flowchart shown in FIG.

In step S801, the CPU 401 of the information processing device 100 acquires an image from the digital camera 421.

In step S802, the CPU 401 of the information processing device 100 stores the image acquired in step S801 in the external memory 411.

In step S803, the CPU 401 of the information processing apparatus 100 cuts out the image of the area corresponding to the position of the reading object from the image stored in step S802. The image acquired in step S801 is an image showing the entire projection surface 304, and is not an image showing only the read object. Therefore, by performing such processing, an image showing only the read object is generated. In order to acquire the image of only the read object, the shooting range and shooting magnification of the digital camera 421 may be adjusted at the time of shooting by the digital camera 421 to acquire an image of only the read object.

In step S804, the CPU 401 of the information processing device 100 saves the image generated by clipping in step S803 in the RAM 403.

In step S805, the CPU 401 of the information processing apparatus 100 refers to the image table 1400, and determines whether or not there are images with the same shooting position 1404 among the images with the same read target object type 1402 selected in step S702. Yes (corresponds to the determination means). That is, it is determined whether or not there are images in which the positions (areas) of the read object on the projection surface overlap. The overlapping does not need to completely overlap the imaging positions 1404, and even if a part of them overlap, it is included in the overlapping in the present invention. If it is determined that there is an image in which the position (area) of the read object on the projection surface overlaps, the process proceeds to step S806, and an image in which the position (area) of the read object on the projection surface overlaps. If it is determined that there is none, the process proceeds to step S807.

A specific example will be described with reference to FIGS. 12 and 13.

In FIG. 12, the customer 302 places the reading object 1005, which is the same reading object as the image 1101 and is the actual object, again so as to overlap the position of the image 1101 already projected in step S714. When placed, the information processing apparatus 100 identifies the position and size of the reading target object 1005 and projects the frame 1004. When the teller 301 presses the reading target reading button 1001, the projected image 1101 and the frame 1004 are hidden, and the projection surface 304 is photographed. The image obtained by shooting is overwritten on the projected image 1101. In this way, by placing the image of the read object at a position that overlaps with the position of the projected image 1101 (position where the image of the read object has already been taken) and taking an image, it is possible to easily overwrite the image. it can.

In FIG. 13, the customer 302 places the read object 1301 at a position different from the position of the image 1101 already projected in step S714. When placed, the information processing apparatus 100 identifies the position and size of the reading target object 1005 and projects the frame 1004. When the teller 301 presses the reading target reading button 1001, the projected image 1101 and the frame 1004 are hidden, and the projection surface 304 is photographed. An image obtained by photographing the read target 1301 is stored as the same read target type as the projected image 1101. When storing the image, as shown in FIG. 14, a plurality of pieces of image data can be associated and stored as the read object type selected in step S702 (in FIG. 14, the read object type 1402 is stored). Multiple pieces of image data are registered as a "driver's license"). For example, if the same type of object to be read, such as the front and back of a license, and multiple images need to be stored in association with each type of object to be read, the type of object to be read is taken every time the image is taken. It is possible to easily associate and save them without accepting the designation of.

In step S806, the CPU 401 of the information processing apparatus 100 overwrites the image indicated by the image data 1403 corresponding to the image index 1401 of the image determined to overlap in step S805 (corresponding to a storage control unit). The overwrite of the present embodiment means that the image with the same file name is saved, but the image indicated by the image data 1403 is deleted (corresponding to the delete instruction), and the image data 1403 is newly saved. You may overwrite by rewriting to the file path.

In step S807, the CPU 401 of the information processing apparatus 100 stores and saves the image saved in the RAM 403 in step S804 in an appropriate file path in the external memory 411 (saved as a new image, which corresponds to a storage control unit).

In step S808, the CPU 401 of the information processing apparatus 100 newly adds a record to the image table 1400 shown in FIG. The image table 1400 stores an image index 1401, a read object type 1402, image data 1403, a shooting position 1404, and a customer ID 1701 in association with each other. The image index 1401 is a number for uniquely identifying and stores a value obtained by adding 1 to the bottom record. The read object type 1402 stores the read object type selected in step S702. The image data 1403 stores the file path in which the image is stored in step S807. The shooting position 1404 stores the value of the reading target position 1501. The image table 1400 is an example and is not limited to this.

This is the end of the detailed description of the image storage process shown in FIG. 7.

By carrying out such processing, when the capturing target is placed on the area where the capturing target is projected and the image is taken, the projected image is overwritten and the capturing target is projected. When an object to be captured is placed in an area other than the area and is photographed, it is possible to store a new image in association with the object type 1402 to be read.

Therefore, when capturing a plurality of objects to be read, the user can capture the objects to be read one after another by placing the objects to be read in an area different from the projected image and capturing the images. Yes, you can shoot while checking the capture result. When a problem occurs in the reading result, the user may overwrite the image by placing the reading target corresponding to the image on the image (projection) in which the problem occurs and giving a shooting instruction. Images can be saved. In this way, it is possible to provide a user interface that can easily capture multiple pages of the read object one after another, and if there is a problem with the captured result, simply re-capture and save it. Has a great effect.

This is the end of the description of the first embodiment of the present invention.

Next, the description of the second embodiment of the present invention will be started.

In the first embodiment, in the determination step in step S713 of FIG. 7, the rectangle indicated by the photographing position 1404 corresponding to the read target object type 1402 identified in step S702 and the read target object position 1501 acquired in step S712 are indicated. When determining whether or not the rectangle overlaps, it is determined whether or not the read object has moved by determining whether or not the two rectangles completely overlap. In the second embodiment, in the determination step corresponding to this step S713, the overlap does not have to be a complete match, it is determined whether or not at least a part of them overlap, and it is determined that there is no overlap. In this case, it is determined that the read object has been removed from the rectangle indicated by the photographing position 1404 (that is, the read object has moved).

The details will be described below. Note that the hardware configuration, functional configuration, flowchart, table, and the like other than FIG. 19 corresponding to FIG. 7 of the first embodiment are the same as those of the first embodiment, so description will be omitted. Only FIG. 19 which is different in processing from the first embodiment will be described.

Steps S1901 to S1912 are the same as the steps S701 to S712 of FIG.

In step S1913, the CPU 401 of the information processing device 100 determines whether or not the reading target has moved from within the coordinate area when the reading target was photographed. More specifically, the rectangle (eg: (X1, Y1) (X2, Y2)) indicated by the shooting position 1404 corresponding to the read target object type 1402 identified in step S1902 and the read target object position acquired in step S1912. It is determined whether or not the rectangle indicated by 1501 (eg, (X7, Y7) (X8, Y8)) partially overlaps.

Specifically, using the following conditional expressions, it is determined that (X2<X7||X1>X8||Y2<Y7||Y1>Y8) does not overlap, and in other cases, It can be determined that they overlap. It should be noted that not only determination is made by overlapping of rectangles, but a determination region is provided around the rectangle indicated by the photographing position 1404, and it is determined whether or not the determination region and the rectangle indicated by the read object position 1501 overlap. Good.

When it is determined that the rectangle indicated by the photographing position 1404 and the rectangle indicated by the reading target position 1501 acquired in step S1912 partially overlap, it is determined that the reading target has not moved, and the process returns to step S1912. If it is determined that the rectangle indicated by the photographing position 1404 and the rectangle indicated by the reading target position 1501 acquired in step S1912 do not partially overlap, it is determined that the reading target has been moved, and the process proceeds to step S1914. Proceed.

Steps S1914 to S1917 are similar to the processing of steps S714 to S717 shown in FIG.

As described above, according to the second embodiment, it is determined whether or not the read object has moved to the outside of the coordinate area when the image was captured, and the image saved when the read object has moved to outside the coordinate area is displayed. By projecting, there is an effect that it is possible to realize the intuitive operation feeling of the user, that is, when the object to be read is removed from the imaged position, the imaged image data is projected.

As described above, according to the embodiments of the present invention, it is possible to project the photographed document so that the user can appropriately check the document.

The present invention can be embodied as, for example, a system, an apparatus, a method, a program, a storage medium, or the like. Specifically, the present invention may be applied to a system including a plurality of devices. It may be applied to a device consisting of one device.

It should be noted that the present invention includes a software program that realizes the functions of the above-described embodiments, which directly or remotely supplies the system or device with the software program. The present invention also includes a case in which the computer of the system or the device reads and executes the supplied program code.

Therefore, the program code itself installed in the computer to implement the functional processing of the present invention by the computer also implements the present invention. That is, the present invention includes the computer program itself for realizing the functional processing of the present invention.

In that case, the program may take any form such as an object code, a program executed by an interpreter, or script data supplied to an OS as long as it has the function of the program.

Examples of the recording medium for supplying the program include a flexible disk, a hard disk, an optical disk, a magneto-optical disk, MO, a CD-ROM, a CD-R, a CD-RW, and the like. There are also magnetic tapes, non-volatile memory cards, ROMs, DVDs (DVD-ROMs, DVD-Rs), and the like.

In addition, as a method of supplying the program, a browser of a client computer is used to connect to a home page on the Internet. The computer program itself of the present invention or a compressed file containing an automatic installation function can be supplied from the home page by downloading it to a recording medium such as a hard disk.

It can also be realized by dividing the program code forming the program of the present invention into a plurality of files and downloading each file from different homepages. That is, a WWW server that allows a plurality of users to download a program file for implementing the functional processing of the present invention on a computer is also included in the present invention.

Further, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and the key information for decrypting the encryption is downloaded from the home page via the Internet to users who have satisfied predetermined conditions. Let Then, it is also possible to execute the encrypted program by using the downloaded key information and install the program in a computer for implementation.

Moreover, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, the OS or the like running on the computer performs a part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments can also be realized by the processing.

Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted in the computer or a function expansion unit connected to the computer. After that, based on the instructions of the program, a CPU or the like included in the function expansion board or the function expansion unit performs a part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

Note that the above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be limitedly interpreted by these. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

100 Information Processing Device 102 Projection Imaging Device 401 CPU
402 RAM
403 ROM
404 System bus 405 General-purpose input/output controller 406 Video controller 407 Memory controller 408 Communication I/F controller 409 Input device 410 Display 411 External memory 421 Digital camera 422 Gesture sensor 423 Projector

Claims (5)

An information processing system for controlling whether to store an image obtained by capturing an image of an object to be read ,
Accepting means for accepting a selection of a type to be photographed by the photographing unit from a plurality of types indicating an object to be read by a user,
When the reading target is photographed by the photographing unit, the reading target is photographed by the photographing unit before the first image in an area overlapping with the photographed first image. When the second image is stored, the first image is overwritten on the second image in association with the type of which selection is accepted by the accepting unit, and overlaps with the area of the first image. When the second image captured in the area not stored is stored, the first image is not overwritten on the second image, and the first image is associated with the type for which selection is accepted by the accepting unit. Storage control means for controlling to store one image and the second image ;
An information processing system comprising: The information processing system according to claim 1, wherein the storage control unit controls to store an image photographed by the photographing unit for each type for which selection has been accepted by the acceptance unit . The information processing system includes a projection unit, and further includes a projection control unit that controls the projection unit to project the image captured by the imaging unit,
The information processing system according to claim 1 or 2, wherein the projection control unit controls the image stored by the storage control unit to be projected onto the projection surface. A control method of an information processing system for controlling whether to store an image obtained by capturing an image of an object to be read ,
A reception step in which the information processing system receives a selection of a type to be photographed by the photographing unit from a plurality of types indicating an object to be read by a user,
When the object to be read is imaged by the image capturing unit, the information processing system reads the image by the image capturing unit prior to the first image in an area overlapping the imaged first image. When the second image of the object is stored, the first image is overwritten on the second image in association with the type of which the selection is accepted in the accepting step, and the first image is also stored. If the second image captured in a region that does not overlap the region of the image is stored, the selection is accepted by the accepting step without overwriting the first image with the second image . A storage control step of controlling to store the first image and the second image in association with a type ;
A method for controlling an information processing system, comprising: A program for causing a computer to function as the information processing system according to any one of claims 1 to 3 .

Images